Case Studies

Identify new and better products people want to drive Dell sales and foster customer loyalty.

Share and vote on ideas about what you’d like Dell to sell.

Dell’s Ideastorm is an open forum where participants can contribute and vote on ideas. While the crowd is free to create ideas and rank them, Dell alone selects winning ideas based on a secret criteria, and the crowd only sees what happens to a few ideas implemented.

Open to the public, anyone can join and contribute to Dell’s Ideastorm. There’s no purchase necessary.

The reward is the chance to influence Dell products available – hopefully getting you the computer you want! There is no cash reward for ideas provided – even ideas chosen for use – Dell’s legal terms make all agree that anything you post on Ideastorm Dell has royalty free rights to use and implement it, without compensation.

Dell added links to Ideastorm on the community part of their website, currently attracting about 15,000 unique visitors per month. They also made a promotional video, the current version having close to ten thousand views which we think is worth watching, if just for the music...

Dell has full-time dedicated moderators oversee Ideastorm conducting housekeeping like fusing like ideas into one, and raise the profile of ideas it favours, and kills others Dell dislikes. An expert Dell panel behind the scenes – like Simon and his crew on American Idol – review ideas and select winners for use. Dell really takes first level feedback from its community, but does not foster a further relationship with them.

Dell has received close to 13,000 ideas and 80,000 comments. It has implemented, or partially implemented, more than 389 ideas from the crowd. Among the most notorious and profitable-Dell customers can now have their laptops shipped with Linux or Ubuntu installed.

Great things like Dell’s Linux models have come from Ideastorm and the company has proven it listens to and applies crowd input. Dell’s chosen crowdsourcing model is popular (Starbucks uses it too) and has value but it could further enlist crowd help with selecting best-selling products.

In a typical crowdsourcing path, there are five activities: 1. Submit ideas 2. Discuss & Comment 3. Vote 4. Shortlist ideas 5. Winner pick Dell invites the crowd in the first three activities, which leaves the company doing a lot of costly, time-intensive heavy-lifting to filter to quality ideas. The crowd could do that filtering if Dell was willing to share its criteria used to select winning ideas. Let’s say ideas are assessed on mass appeal, time to develop, opportunity for market differentiation and fit with the brand – a 5 star score on each criteria could be required. Even if Dell wishes to keep the filtering to itself for matters of competitive confidentiality, it could still better explain why an idea has won to help Ideastorm participants contribute even better ideas. A better display of crowd-powered ideas and a timeline of their launch to market could also foster enthusiasm and help enlist more contributors.

Lastly, while it’s common to take ownership of submitted ideas, Dell could improve how it rewards contributors by providing a bonus gift (and it could be Dell’s onto this already!). Whether it’s a personalized version of the product created sent to you for free with a BIG THANKS card, or a Dell for life discount or at least a discount off next purchase – rewards to the crowd will ultimately work to drive Dell’s success.

Achieve the most interesting, most accurate editorial on the war in Iraq.

Edit, contribute to and rewrite The L.A. Times’ first draft of War and Consequences to make it great – and a worthy destination for readers.

A collaborative crowdsourcing exercise, anyone could contribute to and edit the story wiki.

Open to the public, anyone could contribute.

There was no cash paid out but contributors earned the reputation credit of being a direct part of recording history and impacting one of the world’s first high-profile newspaper crowdsourcing efforts.

The call for contribution was broadcast to The L.A. Times online readership – now around 5 million monthly. The newspaper wrote the first draft of the editorial and promoted it to its online readers at latimes.com and in a print story. As The LA Times Wikitorial took place in 2005, a full year before the first official crowdsourced journalism project, Assignment Zero, it was a provocative undertaking and garnered publicity from heavy weights like Jimmy Wales, the founder of Wikipedia, was an early contributor, fostering much discussion.

There was a high-trust, self-moderate approach taken to the Wikitorial, with anyone able to make edits or contributions that would be published instantly for the world to see. Submissions were not reviewed prior to publish any new contribution could over-write a prior entry. The LA Times only management role in the “public beta” Wikitorial was to assign a guardian to watch the wiki and revert to a previous version if anything untoward was posted. The goal was simple and short-term – produce one editorial in a new way, so LA Times did not aim to form relationships with contributors through communication or networking with contributors.

The Wikitorial site was live for only three days during which the first draft editorial grew from less than 1100 words to 2700 words, It was pulled down after a vandal started posting pornographic material on the main page of the wiki which overwhelmed the LA Times. With so little time live, the goal of achieving the most read, most respected Iraq war article was stymied.

The LA Times was forging into unknown territory with its Wikitorial idea. We salute their initiative, but here are some recommendations for anyone following in their path.

The LA Times had a vision of creating a self-moderating community with really no editorial oversight needed. The newspaper had great foresight to understand the power of user-generated content and online community– but it overlooked the need for a dedicated LA Times Wikitorial editor to manage the editorial evolution pro-actively. They needed more than a guardian with the power to “revert” editions, but someone to watch contributor behaviour and in the case of the porn hacker – curtail an individual’s behaviour without halting the editorial’s progress. Perhaps they imagined that like on Wikipedia a few expert and dedicated crowd members would naturally become wiki champions to police content for accuracy and appropriateness. This is wise recognition of the power of crowds, however it takes time and upfront editorial intervention to make that happen – out of the question for the first days of its first Wikitorial.

While we’ll never know the editorial content that would have been achieved had the Wikitorial stayed live, experience in other crowdsourcing instances suggests how the LA Times launch of the wikitorial effort stymied its success. Never declare the outcome of a crowdsourcing before it’s begun. When The LA Times announced that the Wikitorial might be a failure, or it might be a new way to write editorials, but seemed to emphasize that “this was only a trial beta” they curtailed participation and constrained creativity. If people are told ahead of time their efforts might be for something that may be doomed they lose interest in getting involved and contributing. What would have worked instead was a series of articles – the first announcing the concept and inviting the world in, the next talking about participation and progress milestones, and the final publishing the crowd-created ultimate editorial.

Secure Netflix market leadership as the guys who help match customers to movies they’ll like by bettering the company’s movie recommendation technology.

Develop a system to predict movies people will like that’s at least 10 per cent more accurate than Netflix Cinematch and win $1 million. But there’s a hitch, participants must share their method, how they did it and why it worked not just with Netflix but the world.

The Netflix Prize was a crowdsourcing competition that pitted participants head-to-head to come up with the best system. Netflix provided people with a lot of anonymous rating data, and a Cinematch baseline of prediction accuracy to beat. As soon as a contestant achieved a 10 percent accuracy improvement and shared their system, that started a 30-day window for others to beat it. The sharing and showdown fostered collaborations between multiple participants to build “super-teams” to try and knock the leaders out of top spot.

Everyone was invited to participate, there was no cost, nor did you have to be a Netflix member. According to the website, there were 51,051 contestants on 41,305 teams and those teams submitted 44.014 valid submissions.

The winner reward was one million dollars. Every participant contributing a system was broadcast on a leaderboard for bragging rights and to fuel competition.

Netflix got the word out originally through the website – easy to do when you are ranked at one of the Top 200 sites in the world with an average of around 20 million unique visitors per month. The contest was widely followed by technologists, scientists, researchers and the press because the predictive system being developed could be applied to much more than movie picks.

The Netflix prize was set up as a low-touch community of people working on one clear goal. As Netflix gave participants everything they needed to complete the goal, they only implemented a simple forums page for contestants to discuss any questions, comments or ideas they have about the prize, including a call for papers.

Netflix broke through - landing a system that seemed an impossible technical leap and entrenching them as a service leader. As Netflix Chief Product Officer says, “Accurately predicting the movies Netflix members will love is a key component of our service.” And as a testament of how valuable prediction is to their business – they have already launched the Netflix Prize #2.

We only have one upgrade, which Netflix has already recognized. It took three years for participants to solve the Netflix challenge and for someone to win the prize. That’s a long haul without incentives along the way. To provoke participation and avoid crowd fatigue before the challenge is complete, we’d recommend trickling some payout to key contenders along the way. Netflix Prize #2 is doing just that, pacing prize money with grand prizes at six months and eighteen months.

Find the MP’s expenses that warrant further investigation by The Guardian in over 458,832 pages of public records

Join us in investigating your MP’s expenses.

Crowd production with no contest framework

An open call to the public

Volunteers weren’t paid in cash, but in reputation prowess. The Guardian posted the top contributing volunteers and aimed to make the experience pleasurable as a reward. Beyond just flagging expenses in the documents, volunteers were invited to tag items as Not Interesting, Interesting, Interesting but Known and Investigate This! The exercise was also highly publicized adding to the fame and glory allure.

The Guardian online and in print was used to announce and promote the campaign (yah, it helps to be a media player sometimes!).

The Guardian opted to let the community self manage with only an email us if you have questions link.

Within 90 minutes of the launch, 1700 users had audited MP’s expenses, getting through more documents than The Guardian could have managed in months.

The challenge with this sort of crowdsourced production is sustaining crowd interest. Three weeks in, 43% of the documents have been reviewed. The jury’s still out on whether the crowd will finish investigation all of the documents.

We would have recommended batching the documents into perhaps 50,000 page chunks, hosting week by week contests, then a final showdown to motivated winners along the way. Periodic success milestones in a crowdsourcing campaign help to fuel the story, foster crowd recruitment and motivate contributors to stick with it.

A prize for the top three overall contributors would have fuelled participation too. Although cash shortage is what motivated the Guardian’s call to the crowd in the first place, one of their major advertisers would have likely gladly offered a prize for the promotions hit.

We’ll stay tuned and report on the final outcome.

Intel wanted to learn early about "next to win" companies to strike up conversation with them and get Intel chips to power their products. Intel was also keen on buying in - its investment division, Intel Capital, has in past years made more investments than large venture capital firms.

What do you think is the next cool software?

A digg-style message board of innovative software products. Members discussed product ideas, voted up their favorites, and products that received the most votes ranked a prominent spot on the site which Intel execs watched for investing/partnering leads.

The CoolSW initiative involved Intel’s more than 80,000 employees.

Employees received no cash for participating or predicting top products. Gaining profile as a smart predictor or wise commenter were the major incentives (what employee wouldn’t want that on their resume?), along with satisfying their curiosity about software trends ahead.

Word went out to Intel’s employees to participate via message boards, blogs, forums, and Intel’s intranet.

CoolSW was moderated by multiple folks within Intel’s Software and Solutions Group responsible for tracking independent software companies and helping them grow with Intel’s help.

Intel’s internal site had 500,000 visitors, and 1.4 million page views on 80,000 contributions.

One company that came to Intel’s attention through the effort was email provider Zimbra, recently acquired by Yahoo. Intel met with Zimbra, and consulted with its management team so that Zimbra could work well on Intel’s chip platform.

Beyond Intel inside and investment deals struck so far, the CoolSW initiative has been a funnel into Intel’s partner program which starts Intel’s insider relationship with up and comers. While the money made on new product sales or investments from this effort wasn’t made public, it cost $40,000 to get CoolSW of the ground internally and for that small sum the payoff is certain!

Intel itself described CoolSW as a "kind of message board with voting" where more votes ranked a product idea at the top of the site. The problem with this digg-style forum is that they are ultimately a shallow popularity contest susceptible to bias, and easy for groups to vote up an idea and "game the system." The consequence for Intel is that the top ranked cool software may not the most worthy of its investment or partnership.

If Intel wanted true predictive results from its crowd, it would need to use crowdsourcing technology that strips out bias, and mines both explicit behavior (what people say and vote) and implicit behavior (what people read and really do). With an unbiased ranking of product ideas, Intel could save itself from manually mining the message boards for ideas and truly rely on the crowd’s rank of product winners as economically most-likely-to-succeed.

We salute Intel’s ongoing dialogue with its communities. Up next for the company is to tap its crowds for true market prediction and product innovation.

Goldcorp under pressure to find next big strike on its property as the company was besieged by labor strikes, lingering debts, high cost of production all of which led them to temporarily cease mining operations. Despite test drilling suggesting as much as thirty times more deposits were to be found, employees were stumped on where to look and moving slowly to find out. Shareholders were tense and growing more so.

Tell us where we’re going to find the next six million ounces of gold.

Using a contest model, the “Goldcorp Challenge” launched on the Goldcorp public website in March 2000 with big cash prizes up for grabs. Abandoning the long-standing, incredibly secretive convention of the mining industry, all of the geophysical data Goldcorp had gathered since 1948 on its 55,000-acre property was offered up online to anyone who might predict where the next gold discovery would be.

The mass public was invited into the “Goldcorp Challenge.” Submissions from all over the globe came in – predictably from geologists – but also from graduate students, consultants, mathematicians, and military officers.

Big cash was used as incentive by Goldcorp - $575,000 in prize money available overall, to participants with the best methods and estimates.

Press released, promoted on Goldcorp website, word out to major geological associations.

Remembering this was way back in 2000, the interaction with participants wasn’t as rich as today’s social networking. From all accounts though, Goldcorp was highly responsive to enquiries using an old fashioned ask and get answered approach.

Contestants identified 110 targets on the Red Lake property, 50 percent of which had not been previously identified by Goldcorp. Over 80 percent of the new targets yielded substantial quantities of new gold.

Since the challenge was initiated, 8 million ounces of gold have been found. Goldcorp CEO, Rob McEwen estimates the crowdsourcing effort saved the company two to three years of exploration time.

It’s hard to envision what could have made the “Goldcorp Challenge” any more successful. We’re sure there are communities that would have liked to be partners that weren’t asked, and yes, there are likely gold nuggets unturned, but in its day, it was done perfectly.

Take the risk out of selling t-shirts to a notoriously fickle youth crowd by having them say what they’d buy, producing just those, and selling them all.

Vote on t-shirt designs, or submit your own to compete for prestige and a prize.

Contest model with really active voting + crowd production of shirt designs.

The mass public – youth crowd majority

Rewards both for participation and t-shirt design victors. Design winners now earn $2500 cash, $500 per reprint, and $500 gift certificate on top. Beyond the vibrant reputation economy that fuels Threadless (being in the community – especially active in the community is worth cred), there’s payoff for beig part of it. Submit a photo of yourself in a Threadless shirt and earn $1.50 store credit, refer a friend who buys a shirt earns you $3.

Grassroots buzz through design community and highly socially networked youth demographic. Any shirt designer motivated to spread the word to vote their design up for a shot at prize money and the reward of being produced.

Keeping up with youth demographic demands community prowess. Threadless is non-stop interactive with its crowd in voting conversations, by blog, forum, Twitter and of course in social networks Facebook, MySpace and Flickr.

Threadless has sold out of every t-shirt its produced. Generated $17 million in revenues in 2006. Sells an average of 90,000 t-shirts a month. Threadless spends $5 to produce a shirt that sells for between $12 and $25. Prize payouts amount to $1million a year and Threadless keeps all the intellectual property of t-shirt design submissions. They don’t miss anyone’s birthday ; )

While Threadless has nailed the model for the youth market, we suggest it reconsiders some aspects of its model for its Kids & Baby line extension. Threadless needs to watch closely if it’s current community includes enough hip moms, dads, aunts, uncles and other baby buddies to make a market for the kiddie clothes. Threadless may find it needs a different promotion strategy (or one at all) to reach time-starved mom’s who may not be the same social networking mavens as its youth crowd.

Achieve a market-grabbing TV commercial for Heinz Ketchup and gather some new customers along the way.

Make the next great Heinz Ketchup commercial.

Contest model - Heinz Top This TV Challenge

The mass public – anyone with a creative bent, video recording gear and a YouTube propensity.

The grand prize winner earns $57,000 and a spot on national TV. Four runners up win $5,700 and their commercials also air.

For an elite brand like Heinz, there's no escaping the need for marketing excellence. From the website to promote the ad contest to the advertising and PR to invite the world to participate, it took heavy lifting and deep pockets to draw in advertising hobbiests, pros and fence-sitters.

Heinz folk actively conversed with contributors and kept an eye on YouTube, Flickr and other hubs where Heinz ad competitors were hanging out.

Thousands of commercials were created – all now property of Heinz. Though many are non-usable as finished ads, among the cream of the crop lies creative inspiration for future Heinz campaigns. How many bottles of Ketchup have the ads sold? No one can say. ROI on TV ads remains black art.

But beyond the ads produced, the contest is a humanist be loyal-to-Heinz reminder to the crowd which helps solidify the market-leading brand.

We salute the courage it takes to invite a crowd to evolve such a marquee brand as Heinz Ketchup. So our hats off to Heinz. Perhaps as they move ahead, they could get a little bolder with their brand collaboration and trust the crowd a little more.

Great crowdsourcing starts with the right call to the crowd. In the case of Heinz it might go beyond “make a great Heinz commercial” to giving guidance on vitals of its ‘brand” to convey. Does it wish to be the stickiest to french fries, the most fun to lick off a lovers’ cheek? Cast out some emotions its looking to evoke and the results might be more inspired and usable.

To really benefit from open innovation Heinz also needs to adopt a complete crowdsourcing model - not just in an open call for entries, but in a call to the crowd to evaluate and at least short-list finalists that its experts can then evaluate to declare the winner. To date, Heinz has used an expert panel and outside promotions firm to review entries. This protects Heinz from publicly airing ad ideas that may be offensive or even unlawful (copyright infringement), but enlisting the crowd to vote and filter submissions would make the contest more interesting, fuel competition and entries. Yes, it’s more risky but the controversy may up the notoriety of the Heinz Ketchup brand and would certainly reduce the campaign cost.

Mega consumer goods with 23 brands worth over $1billion in sales and 170 year of history needed to keep performing at the rate it has historically. The company sought game-changing new product ideas which weren't coming fast enough from staff alone.

Solving diverse problems that consumers have in product design or developing new product ideas. Calls for solutions vary from seeking methods and ingredients that reduce caloric density of snack foods without affecting taste, to device ideas to moisten dry toilet tissue for better cleansing.

P&G hosts contests within research and development communities like InnoCentive and the NESTA network, and invites the public to submit innovation on its Connect + Develop website.

More than a single contest, P&G launched its Connect + Develop initiative to continuously collaborate with bright minds outside its organization.

The broad public with many scientists, researchers, and product designers among them. The InnoCentive community, for example, is a network of 140,000 scientists.

Solvers of Procter & Gamble problems within InnoCentive earn from $10,000 to $100,000 per solution. Within the Connect + Develop website, the party line is that there is no standard amount as each deal is evaluated on its own merits through discussions with P&G.

P&G crowdsources primarily by partnering with communities that exist for open innovation – and with existing infrastructure to pose problems to a crowd. This partner approach gives P&G immediate reach to engaged participants with little or no promotion. The company press releases some innovation initiatives, and keeps followers apprised in its Connect + Develop community, but participation is very high with very little buzz.

P&G moderates its own Connect + Develop community keeping it clear of misdirected requests for modifications to its existing products, or to provide feedback about a consumer experience with one of our products.

More than 30 percent of the problems posted on InnoCentive have been solved. Breakthrough, big-revenue products including the Swiffer have come from P&G’s open innovation initiatives. P&G stock price continues to climb upward and the company’s net profit tripled to $10 billion in 2007.

It is likely that every opportunity we can think of has already been discussed and may be in the works at P&G but here are some ideas. Considering the observation by Geoffrey Koch of Intel that "…innovation is driven by everyday end-users. … Today, more and more innovation comes from the myriad hobbyists and enthusiasts at the tech-savvy edges of the computing ecosystem," we would encourage P&G to add partnerships with non-science and research focused communities to its crowdsourcing mix. Why not reach out to communities of house cleaners, dieters, university design labs or architects and interior decorators to tap direction on product innovation? It’s not just open collaboration online that fuels successful crowdsourcing but an openness to non-traditional contributors.