Skip to main content
Hello Visitor!     Log In
Share |

In Search of Failure’s Silver Lining

ARTICLE | | BY Bengt-Arne Vedin


Bengt-Arne Vedin

Get Full Text in PDF

Technology offers a number of examples of serendipity, of random discovery, of experiments going off the rails - though we highlight mostly the ones where the offshoots were spectacular.

Aspartame was discovered - or involuntarily invented - in 1965 by James M Schlatter, a chemist working to produce an anti-ulcer drug. Schlatter just happened to experience a sweet taste when, wetting his finger to turn a page in his notebook he licked the finger contaminated with aspartame. Importantly, he had the presence of mind to trace the sweetness back to a simple molecule of two amino acids.

Teflon, discovered by Roy J Plunkett at DuPont Company's laboratory in 1938, was another accidental invention - unlike most other polymer products. But as Plunkett often told later, his mind was prepared by education and training to recognize novelty.

Plunkett's assignment was researching new chlorofluorocarbon refrigerants (we know them as Freons) which were seen as great advances over earlier refrigerants like sulfur dioxide and ammonia. Plunkett had produced some 100 pounds of tetrafluoroethylene gas (TFE) and stored it in small cylinders at dry-ice temperatures in preparation for chlorination. When he and his assistant made a cylinder ready for use, no gas came out, yet the cylinder weighed the same as before. They opened it and found a white powder, which Plunkett was clever enough to characterize for properties other than refrigeration potential. He found the substance to be heat resistant and chemically inert, and to display very low surface friction so that most other substances would not adhere to it. Plunkett realized that, against the predictions of polymer science of the day, TFE had polymerized to produce this substance - later named Teflon - with potentially useful characteristics. Other experts investigated the substance further.

At first Teflon seemed too expensive to produce to ever find a market. Its first use was found in the gaseous diffusion process of the Manhattan Project, resisting corrosion by for example fluorine (it may also be used for containing a number of very aggressive acids). Teflon covered pots and pans were invented years later.

Teflon is inert to virtually all chemicals and considered the most slippery material existing. These properties have made it one of the most valuable and versatile technologies ever invented, contributing to significant advancements in areas such as aerospace, communications, electronics, industrial processes, and architecture, and also as a soil and stain repellant for fabrics and textile products.

Kanthal is the trademark for a family of iron-chromium-aluminum alloys used in a wide range of resistance and high-temperature applications. The first such alloy was developed haphazardly by Hans von Kantzow in Hallstahammar in Sweden. The accidental material could withstand high temperatures while offering intermediate electric resistance. First von Kantzow had no time to explore the novelty further; only several years later did he develop a business around its use for various types of heating elements (in space applications, saunas and other ovens, bread toasters, hair dryers, soldering irons, and what not).

Saeid Esmaeilzadeh, at the Stockholm University, was doing what he regarded as a routine experiment in a hot oven. That oven suddenly crashed and its cooling water poured out over the material he had placed in a crucible. This equated with very fast cooling: an obviously failed experiment, Esmaeilzadeh concluded. Only it was no failure, because instead of the intended regular crystals, out of the oven came glassy lumps of a new material never before produced, a super-material, extremely hard and with, among other qualities, a refraction index almost impossibly high. Currently it is the hardness of the material that is exploited for bearings as well as in armor.

At 3M, known as keen on innovation, seasoned chemist (with more than 20 patents to his name) Spencer Silver developed a glue that turned out not to hold. He suggested several applications but none took off; he also told of his invention at several internal seminars, one attended by Art Fry, choral singer in his spare time. Well, you know the rest: Fry took Silver's paper with bad glue to serve as bookmarks in his hymnal, and the Post-It® was born. In effect, Spencer Silver had often told of his 'failure' so that others could learn from it, build on it.

Esmaeilzadeh's oven accident, Silver's failed glue: who would be searching for failure? Failures just happen, though we always hope to avoid them. A vile phenomenon, a vile word, no?

I would suggest searching for a more appropriate word for failure, one not tainted with negative connotations. Because failure is the high road to innovation, because failure is the inevitable investment in the future, because failure is - the road to success. Remember Edison retorting when pitied for having failed at 10,000 attempts (or some such number) to get the right filament in his light bulb: "no, no 10,000 failures but learning, identifying 10,000 ways that won't work".

Concepts such as learning, tests, trials, and experimentation offer alternate, more positive angles. As kids we experiment constantly, with language, with acquiring skills such as biking and swimming, maturing, going through a process of socialization. Learning to ride a bike is worth some bloody knees; corrected grammatical errors are just that, not failures. Yes, we talk about learning and maturing, not about failing. To the contrary, we are succeeding!

You may not publish a report on a failed scientific experiment, yet in science such experiments constitute part and parcel of its very process; Richard Feynman once argued that a publication on a failure might be as valuable, or even more so, than one on success - but fat chance that it would be published (or be regarded as a merit). One established way of testing and verifying theories is by going to the extremes, to try to find out what the limits of an established law or connection or interdependence are, where it fails. Or just measuring. Michelson-Morley failed at establishing the speed of light in different directions versus the supposed ever-present ether of their era's physical world. If a theory tested to its extremes fails, we know where it no longer holds. Such failure might signal the paradigm shifts heralded by Kuhn, or the emergence of new thought collectives around new thought styles after Fleck.

In some famous instances, such failure has generated fundamental breakthroughs. Henri Poincaré attempted to solve the three-body-problem thus winning a prize established by the King of Sweden and Norway. And indeed he won the Prize - by showing that there were no solutions to that problem. Thus, as a kind of by-product, he laid the foundation for chaos mathematics, which would start to flourish in earnest only sixty years later. Likewise, when Hilbert had established his famous list of mathematical problems, his second was the demand for proof that arithmetic was consistent, free of any internal contradictions. Gödel's work on that problem turned out to be a failure in the sense that he proved, very elegantly, that the statement is, in effect, false. It was a productive failure, like Einstein's obstinate refutation of the 'spooky interaction at a distance' that followed from quantum mechanics, and a spooky interaction that constitutes the basis for ideas about quantum computers with high degrees of security and enormous computing power.

Talking mathematics, there is a branch of topology that is dedicated to failure of sorts, René Thom's catastrophe math. There is a limited number of fundamental catastrophes, and in essence, here we learn that the trajectory, the order in which a development unfolds, is decisive for whether there will be a smooth journey, or a discontinuity, a dramatic drop (or hike) along the way, that is, a catastrophe. Another approach to mathematics, systems dynamics, also underlines the importance of the dynamics, that is the delays and the accumulation of what is handled internally in a system, making the behavior of the system profoundly counterintuitive. Then there are the paradoxes or incompatibilities between human action and the idea of rational behavior, conflicts mapped not least by Tversky and Kahneman.

The title "In search of…" intends to evoke the title of the business bestseller "In search of excellence". I heard its (co-)author Tom Peters explain what secret they had found behind the fact that some oil drilling companies were by far more successful than others - very simple: "they drill more holes"! The more failures the larger the number of successes. Losers become winners.

This drilling of more holes applies at several levels. Sir James Dyson is best known for inventing the Dual Cyclone bagless vacuum cleaner, an idea saving him from bankruptcy and the first building block in what would become a billion-dollar business. In an article in Wired magazine, he pleaded for failure:

"An inventor's path is chorused with groans, riddled with fist-banging and punctuated by head scratches. Stumbling upon the next great invention in an "ah-ha!" moment is a myth. It is only by learning from mistakes that progress is made. It's time to redefine the meaning of the word "failure" (emphasis added). On the road to invention, failures are just problems that have yet to be solved.

It started, Dyson continues, with a vacuum. When his bagged vacuum lost suction, he came up with the solution - cyclone technology. But having an idea was just the beginning. With a few rudimentary materials he mocked up a first prototype. Crude, but it worked (sort of). From cardboard and duct tape to ABS polycarbonate, it took 5,127 prototypes and 15 years to get it right. And, even then there was more work to be done.

By the time Dyson made his 15th prototype, his third child was born. By 2,627, he and his wife were really counting their pennies. By 3,727, his wife was giving art lessons for some extra cash. These were tough times, but each failure brought him closer to solving the problem. It wasn't the final prototype that made the struggle worth it. The process bore the fruit. He just kept at it.

When it comes to failure, Dyson claims to be trumped by Edison who famously said, "I have not failed. I've just found 10,000 ways that won't work." Those 10,000 detours resulted in the Dictaphone, mimeograph, stock ticker, storage battery, carbon transmitter, and the joint invention of the light bulb. In the end, 10,000 flops fade into insignificance alongside Edison's 1093 patents.

On the next, more aggregate level, corporate growth correlates with preferences for experimentation. In a Babson Executive Education analysis of 635 global companies, of those within the bracket of "least growth" (1-10%), just under a third preferred experimentation to other methodologies such as statistical analysis to identify revenue and operational improvement opportunities. Almost half (46%) of the companies with 11-20% growth preferred experiments, while well over half (56%) of the companies with more than 20% growth did opt for that approach.

No wonder more and more executives are championing experimentation over (or as precursor to) time-honored analytic approaches like strategic planning and market research. Many also struggle to democratize experimentation.

The rationale for such 'democratization' involving large swathes of organization is clear, since R&D labs have numerous natural constraints that can limit the true potential of experimentation. R&D personnel are often detached from the day-to-day running of the business and hence not always the best to experiment on topical problems. In addition, such labs are often physically separate from the operational centers of the business, so transmitting ideas from the lab to where they are needed constitutes a classic problem. And the number of R&D personnel is usually small compared to the number of all personnel. No matter how brilliant your R&D lab personnel, if you don't democratize experimentation you lose the knowledge and skill of 85-90% of your organization's employees who do not work in the lab. A radical way of outsourcing risk taking and thus benefitting from failure is through open innovation, that is, relying upon a multitude of external sources of ideas and experimentation.

On an even more aggregate level, and in principle, failure is in a sense a consequence of success: no innovations, no business models live for ever - there is always Schumpeter's gale of creative destruction. The best we can hope for, no, the very best we must put our trust in is the existence of provisional truths, provisional, ephemeral successes to be enjoyed as long as they last. It is not that technology moves on though it does, but all of society and markets change as the result of an intricate interplay between new ways of organizing, new conventions, tastes, demographics. Novelty and disruption are the order of the day, not the economists' fake equilibrium. All winners eventually become losers, so please, just move on. Cultures and systems that do not overly punish failure will generate more people willing to undertake risky experiments. As economist Nathan Rosenberg has noted, one of the hallmarks of capitalism is its tolerance of failure in a way the Soviet command economy did not display. Alongside this there is a diversity in organizational forms allowing for experimenting.

There is even statistical evidence that economies with more churn in the corporate sector display faster economic growth. The relationship actually seems causal: turnover today correlates with fast economic growth tomorrow. The real benefit of this creative destruction is not the appearance of "rising stars" but the disappearance of old, inefficient companies. Failure is not only common and unpredictable, it's healthy. On a corporate level, this corresponds to 3M's (and others') rule that x per cent of any business unit's revenue must come from products younger than y years, creating a need for new products as well as for pruning the existing product line.

Like the scientists Feynman reportedly worried about who saw no merit in failure, though that might have been worth reporting. Innovation - particularly radical innovation - does not happen enough and we should soften the failure consequences for individual researchers so that more risky experiments would take place. India's Tata Group is an enormously diversified company from tea over cars to steel. As the company seeks to grow beyond its roots in India and compete globally, innovation is becoming central. To that end, Chairman Ratan Tata has instituted a surprising competition: A prize for the best failed idea! To spark innovation and keep the company from avoiding risks, the prize is intended to communicate how important trying and failing can be. "Failure is a Gold Mine!" proclaims the Chairman who is near-retirement.

So: fail often, fail quickly, fail inexpensively. Tell, disseminate failed results overtly, like Spencer Silver did. Do not cover them up or try to forget them. Attempt to learn all the lessons embedded. Reward failure like Tata, or reward "risks that were worth taking even though they did not pan out". Look upon failed results as weighty trade secrets. Interestingly, when the Harvard Business Review dedicated an entire issue (April, 2011) to failure, numerous articles ended in a discussion of ethics.

Design for failure: design strategies for generating complementary failures that together make for success. Think about metrics, about failure inventories and audits, always with a view to profit from the learning experiences. Map out a failure strategy.

Find and design precise ways to fail often and with low costs. Re-use materials and what otherwise would be waste. Rely upon scenarios and stories, and not least upon sketching. Make rough, speedy models, mockups, and prototypes using whatever proxies are easily and cheaply available. What more? How to map the best set of failures to set out to make, to have the aggregate results contribute positively in the end? What about experiencing failure as part and parcel of an individual's development, of an organization's establishment of a joint culture and sense of mission and cohesion?

Perhaps we need new words, a typology and a taxonomy, also in the sense that we need to classify and in a way monitor and measure failure, as Tata's prize may indicate. Ericsson's AXE system is the world's all-time best-selling telephone switching system. It resulted out of not just one but two different failed attempts to design a digital system to substitute the preceding electromechanical systems. The investment, the costs sunk into those failed systems made it alluring to carry on to improve and adjust those, to salvage them.

Resisting this temptation was difficult but the argument for starting from scratch eventually won because the failures were so resounding. In a way, it was a question of 'failure quality'. It would seem better to fail gracefully, but not necessarily here.

If Ericsson's management had fallen for that temptation and invested more efforts into their failed but somewhat functioning AXE predecessor, for which there were customers and installations to be serviced, they might have experienced a surviving failure. This class of failure often represents those sunk costs that one hopes to retrieve, possibly political promises and investments in PR: the Concorde supersonic airplane has been seen as an example. There are also failure fallacies. One is the mythical man-month. Typically, in a project (particularly for developing computer software), staffing is decided through the calculating of man-months necessary for the task. When it turns out that there is a deficit, that is, the project is running late, the default remedy is most often seen as getting more people, so more man-months, engaged. Problem: this is a recipe for further delay because various communications demands increase man-month consumption, front-loaded in the sense that the first thing that happens is that much more time is lost initially, introducing the newcomers, getting them up to speed.

How could success be failure? Two artists who came to the same conclusion were Lucien Freud and Harold Cohen. Freud felt that he had become his own copier, creating one more Freud, and he was successful at it, selling well, enthusing the critics. Much the same for Cohen, which led him to experiment with computer art and become a pioneer in that field, while Freud changed his style drastically, offending some of his supporters and no longer meeting customer demand. They had defined it a failure of becoming one's own epigone, a failure begetting radical renewal and reinvention.

Having concluded "les Demoiselles d'Avignon", Picasso (who did not name the painting himself) turned it to the wall, avoided displaying or showing it. Established critic and Picasso expert John Berger terms it a failure; intelligence expert John Gardner hails it as a defining work for the 20th century.

These examples point to the difficulty of defining the very term failure. In some instances, the market may tell, but perhaps only in retrospect. In the 19th century, composer Johann Sebastian Bach was not regarded as a great composer, in contrast with later. There is an explanation in a model of creativity that sees this as depending, first, on the particular work, Bach's, Cohen's, Freud's, or Picasso's, then on the specific domain, music or painting, and then also the field, those experts and critics and peers who decide what is good music and what may be published in a physics journal. A new paradigm or a new style of thought comes associated with a new collective: the field. So failure depends on these three dimensions, but may also signal or initiate some sort of seismic shift.

Jazz and improvisational jazz is a long way from Bach. Improvisation is in a way about making failure productive. There are many famous solos or improvised cadences that have been triggered by someone in the band striking a false note, inspiring someone gifted enough to spin it out, to salvage the moment, by generating an entrancing novel melody.

About the Author(s)

Bengt-Arne Vedin

WAAS Fellow; Professor Emeritus, Royal Institute of Technology
Email :