The Dynamics of Innovation

min read

 

Reprinted by permission of the author and publishers, from The Internet and the University: Forum 2002, ed. Maureen Devlin, Richard Larson, and Joel Meyerson (Boulder, Colo.: EDUCAUSE and the Forum for the Future of Higher Education, 2003), pp. 81–102. James Utterback is David J. McGrath Jr. Professor of Management and Innovation and Chair of the Management of Technology Program at the Massachusetts Institute of Technology Sloan School of Management.

The role of technological evolution and innovation in shaping the destinies of industries and firms is often underestimated. Technological change is a key factor as both a creative force in the growth of enterprises and as a destructive force making those same enterprises vulnerable to competition. Analysis of how innovations enter and transform enterprises reveals several strategies for mastering innovation as a creative force for renewal rather than viewing it as a threat.

America's Ice Industry: A Case Study

The American ice industry thrived in nineteenth century New England. The "Ice King," Frederic Tudor of Boston, sent his first shipment of ice from the port of Charlestown, Massachusetts, to Martinique in the West Indies in 1806. Fifty years later, his company was shipping thousands of tons of ice per year to several U.S. ports, the Caribbean islands, Rio de Janeiro, Bombay, Hong Kong, and sites in between. Tudor benefited from collaboration with Nathaniel Jarvis Wyeth, who invented an ice plow and more than 50 other tools for harvesting natural ice from local ponds and storing it. These innovations are said to have cut the price of delivered ice by one-third.

Tudor's growing success attracted others to the business, so that by the late 1870s there were no fewer than 14 firms in the Boston area alone cutting almost 700,000 tons of ice each year. Maine and New Hampshire also had thriving ice companies. Innovations continued to make the product more uniform and lower the cost of production. The ice business became an important part of the U.S. economy and continued to expand as households became regular consumers of harvested ice. City dwellers had begun purchasing ice boxes in growing numbers after 1850, and these soon became a modern necessity.

Yet—though no one recognized it at the time—the early 1880s were to be the zenith of the harvested ice industry. The market for refrigeration was to continue expanding with the growing nation, but a radical innovation based on a totally different technology had already invaded the periphery of the industry.

Machine-made ice began to bite off chunks of the harvested ice industry during the 1860s. These inroads started slowly, as many early attempts to produce ice by mechanical or chemical means proved more costly than harvesting natural ice. Summer prices in the north typically ranged between $6 to $8 per ton of harvested ice; in the south, prices were much higher—often as much as $125 per ton. As a result, Southern markets offered the greatest receptivity to innovations affecting the supply and price of ice.

As experiments with vapor compression machines and various refrigerants improved ice-making technology in leaps and bounds, costs fell. In 1868, New Orleans got its first ice-making plant, which began manufacturing and selling ice for around $35 per ton—substantially less than the price of natural ice. Some 20 years later, by 1889, there were well over 200 ice plants, mostly in the South. New England ice was finding itself effectively driven out of the Southern markets.

Even as new technology and innovations were transforming the industry, the ice merchants of the North pushed ahead with their own improvements and still greater production. They developed and refined a production, storage, and distribution system for harvested ice that was remarkably efficient, so much so that the 1886 harvest was the biggest ever—25 million tons. The Northern ice merchants' response to the challenges to their dominant position, and their failure to recognize that the changes in their industry were indeed revolutionary, illustrates a widespread pattern: Powerful competitors often not only resist innovative threats, but also resist efforts to understand them, preferring instead to further deepen their commitment to their older products. Failing firms tend to be remarkably creative in defending their entrenched technologies, which often reach unimagined heights of elegance in design and technical performance only when their demise is clearly predictable.

The ice industry case also illustrates how the demise of a technology can be obscured by a growing market. While the demand for ice and refrigeration continued to expand, ice harvesters steadily lost markets to plant-made ice. The coup de gracecame after World War I when old-fashioned ice boxes began to give way to electric refrigerators. By the mid-1940s, the natural ice industry was gone for good. It had served, though, as an important step toward greatly expanding affordable refrigeration and the availability of fresh food, and likewise significantly improving the overall diet of the general population.

The Creative Power of Technology

The invasion process of a technological innovation, as described in the ice industry case, tends to follow a predictable pattern. Generally, in any product market there are periods of continuity, when the rate of innovation is incremental and major changes are infrequent, and periods of discontinuity, when major product or process changes occur. Radical changes create new businesses and transform or destroy existing ones.

At the time an invading technology first appears, the established technology generally offers better performance or cost than does the challenger, which is still unperfected. The new technology may be viewed as crude, leading to the belief that it will find only limited application. One problem with early machine-made ice, for example, was its cloudiness, making it seem less clean or pure than harvested ice. A more recent example is early word processors: Apple's first personal computer was difficult to master and produced only uppercase letters. Few were willing to give up their IBM Selectrics for it. The performance superiority of the established technology might prevail for quite some time, but if the new technology has real merit, it typically enters a period of rapid improvement—just as the established technology enters a stage of slow, incremental improvements. Eventually the newcomer improves its performance, matches the established technology, and rockets past it.

Our natural concern with the demise of old products when faced with the new tends to obscure the truly creative and expansive power of innovation. Such a confrontation is often viewed as a pure competition between two technologies. However, Carl Pistorius and I have pointed out that more subtle forces are at work. Seeing the process as competition can result in self-imposed and artificial limits to possible strategic options. We suggested a number of other modes of interaction, where in some cases the old may dominate and others the new will do so. Regardless of whether the older option retains most of the market (as in shaving with blades versus electric) or vice versa (as in watching movies on DVDs versus in a theater), both may become stronger and more useful as a result of the interaction. In special cases such as DVDs and movies or natural and harvested ice, the whole market and uses of the product may grow dramatically as a result of broader availability and functionality. We have termed this mode symbiosis. Taking into account the fact that the various modes of interaction have different characteristics, and that the interactions between various technologies can shift from one mode to another over time, we suggested that different strategies also apply. I speculate below that a symbiotic relationship may be the most likely scenario for the Internet, multimedia methods, and education. Far from being a disruption, these new tools may well help make education much more effective and widely available.1Discontinuities that expand the market are seemingly less threatening to established firms than are those that simply create substitute products. Table 1 presents a summary of discontinuities described at length in Mastering the Dynamics of Innovation.2

Ice and Refrigeration

No doubt a few local holdouts in places like Maine and northern Minnesota continue to harvest ice for commercial sale, but the other firms are gone. So too are most of the firms that displaced the "ice kings" with mechanically manufactured ice. They in turn fell prey to the innovation of mechanical and later electrical means of refrigeration. The broad market for ice produced by nature or by machines disappeared almost forty years ago. The agents of change were not the leaders of the established technology.

Typewriters

Discontinuities occurred between the advent of the manual typewriter, electric typewriter, dedicated word processors, and personal computers. Of the large manual typewriter firms of the early twentieth century, none were successful in jumping onto the bandwagon of electric typewriters. It was IBM, an outsider, that developed both the product and its market. Likewise, the move to word processors and then personal computers caused equal dislocation—virtually none of the original typewriter companies, excluding IBM, made the leap. Indeed, virtually none survive in their past forms.

Lighting

The change from lighting with candles to the modern system of electrical illumination occurred (in most of the western world) within a period of 150 years. Oil lamps displaced candles and were in turn displaced by gas in most urban areas. Electric lamps of the Edison design displaced gas, and fluorescent lamps have displaced these in many instances. Each wave of change has brought a different champion to the fore: with the exception of the non-leadership position of Edison's successor firm (GE) in fluorescents, no firms have successfully bridged the discontinuities. These in turn may be challenged by light-emitting diodes. Throughout all these changes lighting has become more widely available.

Plate Glass Making

The generations of discontinuous and incremental change in this industry have virtually eliminated all but a handful of highly capitalized, high-volume producers. There would be even fewer firms in this industry today had the Pilkington Company decided to protect its patents on the float glass process and simply run over its competition. Instead, it chose to license that breakthrough technology to other glass firms, allowing them to survive. With almost all the glass in the developed world being made under this process, those that did not switch to it were largely eliminated.

Imaging

The transitions from daguerreotype to modern film photography, and the emerging technology of electronic imaging, were punctuated by many discontinuities: tintype to wet and then dry-coated glass plates, and then sheet celluloid and roll film. Each transition was inspired by a different party, rarely the market leader, and each transition led to a period in which old producers were replaced by new ones. Again, photography has become much more widely used and functional with each change.

Framework for Analysis

The following three questions present a framework to discriminate between those cases where traditional firms prevail and those where new firms enter from beyond the circle of known competitors. Analysis based on these key questions results in a comprehensive and accurate picture of the whole:

  1. Does the discontinuity pertain to a product or a process?
  2. Is the discontinuity simply a substitution, or does it create a broadened market?
  3. Is the discontinuity competence-enhancing or competence-destroying for the established firms in the industry?

Linsu Kim and I hypothesized that each of the three factors is important, and that they will operate more powerfully jointly than separately.3

Does the Discontinuity Pertain to a Product or a Process?

Discontinuous changes in processes are usually introduced by established firms, often marginal ones seeking to expand market share, or by dominant firms under severe cost, supply, or regulatory pressure. Discontinuous changes in processes primarily emphasize real or potential cost reduction, improved product quality, and wider availability, and require movement toward more highly integrated and continuous production processes.

Discontinuous innovations in products almost always come from outside the industry; discontinuous innovations in processes come from inside the industry about as frequently as from the outside. Of all the cases Kim and I analyzed, fully three-quarters of the discontinuous innovations coming from inside the established industry fell into the process category.

I would argue that in many respects the changes anticipated in education are more centered in the realm of tools, or processes, and less in the realm of content, or products. If we stretch the lessons learned from innovation in the industrial world to the world of education, then we might expect as many path-breaking innovations to come from within the community of higher education as from new outside sources.

Is the Discontinuity a Substitute? Or Market Widener?

A discontinuous change may drastically increase the aggregate demand for the products of an industry. Examples abound: The invasion of machine-made ice tripled the demand for harvested ice from 5 million tons per year to 15 million tons per year. The replacement of the vacuum tube by the transistor, and later the integrated circuit, has increased the sales of the electronics industry from several billions of dollars to hundreds of billions. The replacement of piston aircraft engines by turbojets has both dramatically reduced the costs and increased the seat miles flown by commercial aircraft. The advent of the electronic calculator has made such equipment commonplace rather than something rarely encountered. The advent of Eastman's Kodak camera and roll film system transformed photography from a small professional market to the large and now familiar amateur market. Replacement of carbon filament incandescent lamps by those based on metal filaments multiplied the demand for incandescent lamps from 20 million to hundreds of millions per year in the United States alone. Each revolution in glass making led to a corresponding sharp increase in aggregate demand for flat glass, and the advent of on-site production of oxygen led to more than a doubling in the demand for oxygen.

Innovations that broaden a market create room for new firms to start. Innovation-inspired substitutions may cause established firms to hang on all the more tenaciously, making it extremely difficult for an outsider to gain a foothold along with the cash flow needed to expand and become a player in the industry. Discontinuous innovations that expand markets will almost always come from outside the industry. Discontinuous innovations that substitute for established products and processes often come from inside the industry.

In summary, some discontinuities broaden a market, allowing new firms to enter and survive. History indicates that in these cases established firms are more likely to fail than succeed, and new firms have roughly the same chances of failure or success as established firms. Some discontinuities do not broaden a market or create a new niche. In these cases, fewer and larger firms are most often the survivors: established firms are likely to enter successfully, but new firms experience tough going. Some discontinuities create a wholly new market niche, encouraging many new entrants. Here, established firms are unlikely to enter successfully, and new firms have better survival odds.

Is the Discontinuity Competence-Enhancing?

Tushman and Anderson characterized technological discontinuities as either competence-enhancing or competence-destroying.4 A competence-destroying discontinuity renders obsolete the expertise required to master the technology that it replaces. For example, the skills of mechanical watch manufacturers or vacuum-tube producers were rendered irrelevant by quartz watches and transistors, respectively. Similarly, the skills of the glass-making artisan were made obsolete by the Lubbers machine, which allowed unskilled operators to make glass cylinders.

Tushman and Anderson wrote,

A competence-enhancing discontinuity builds on know-how embodied in the technology that it replaces. For example, the turbofan advance in jet engines built on prior jet competence, and the series of breakthrough advancements in mechanical watch escapements built on prior mechanical competence. Similarly, the Edison cement kiln allowed cement makers to employ their existing rotary kiln knowledge to make much greater quantities of cement. Later retrofitting of process controls to cement kilns again allowed manufacturers to build on accumulated know-how while dramatically accelerating production through minute control of the process.5

Lessons Learned from Industry Studies

Discontinuous innovations that would be most disruptive are those in products that expand established markets and destroy established core competences. Such innovations virtually always come from outside the industry. Discontinuous innovations that would be least disruptive are those in processes that substitute for established methods and enhance established core competences (in technology). These innovations virtually always come from inside the industry. Intermediate cases—for example, ones that expand established markets or destroy established core competences—will normally come from outside the industry.

Earlier work on technological discontinuities has concluded that if not practically all established firms fail to master radical innovation, then at least it is a highly random and unpredictable process. Applying the three key questions outlined in the framework above, which discriminates between situations in which new entrants were advantaged and those in which established firms hold the cards, reveals a different story. While it is true that a large fraction of radical innovations are indeed introduced and taken up by competitors new to an industry, in about one-quarter of the cases studied, existing competitors either introduced or were able to quickly imitate radical innovations and survive as major players in their markets. Thus, established firms need not always fail in this arena. More importantly, they might be able to determine the conditions most favorable to their success and act accordingly.

Clearly technology is not the key in and of itself. Market conditions are an equally powerful influence. And while technology and markets are important, their importance must be understood in conjunction with the human factors determining organizational competence or core capabilities. When core capabilities are aligned, management can attempt to make a discontinuous change internally. When they are not, an alliance or outside venture is called for; otherwise, great effort must be invested to create appropriate human resources and cultural change before attempting the innovation.

Ironically, the oft-repeated advice to be market-driven in pursuing innovation, to seek out lead users, and to delight one's customers through continuous improvement of products might be either powerful mantras for success or roads to failure, depending on circumstances. These are good ways for a new entrant to identify and specify a valuable direction for change, and they offer good lessons to follow in promoting evolutionary change in well-understood product lines. When applied to a discontinuity, however, they may lead a strong firm into a dangerous trap. Similarly, ideas such as lean manufacturing (with regard to product and manufacturing competences) and mass customization (with regard to marketing and distribution competences) may be thought of as ways to build core competence and achieve success in differentiating well-known products. These concepts, too, might lead to a dead end when radical change is in the wind.

The destructiveness of a change such as machine-made ice is rather surprising. Product, market, and distribution linkages were left entirely intact. Only the ice harvesters' manufacturing competences were severed—but this still seemed like a total revolution to the industry. By the same reasoning, the electric refrigerator should have been, and was, much more competence-destroying, laying waste to product, marketing, and distribution competences as well as manufacturing. Now customers could make their own ice for drinks on demand! The key question is not just whether an innovation is competence-enhancing or -destroying, but for whom? Goodyear and Firestone saw radial tires as cutting into the market for bias-ply tires, while Michelin saw them as expanding the market both in size and geography. Kodak might see electronic imaging as cutting into its market for chemical imaging and eroding high profit margins, while Canon and Sony may see the same innovation as expansive and raising relatively lower margins.

Imagine if you will that the Internet and multimedia methods represent a new process for teaching, that our traditional competence in developing content is the primary skill required for adopting these new tools, and that the population of students taught does not change. In that case, though it stretches my analogy, one would expect little change in the institutional landscape. The artisans using traditional tools will simply adopt the new tools. Then imagine that teaching with the new tools is seen as providing a wholly new experience for a different group of students and that skills needed to develop content are totally novel. In that instance, one would expect an entirely new professional, learning, and institutional scene. I expect the real outcome will be more varied, with many intermediate cases, and that my model based on data from industry may apply only partially to the case of higher education.

There are some interesting straws in the wind. Most academic journals, for example, seem now to be provided and used primarily in electronic form—but this has only strengthened traditional publishers, who have an invaluable asset in their archives and in the archival nature of their product. Similarly, movie theater revenues have tripled in the past decade from 2.5 billion to nearly 8 billion dollars, while video rentals and sales have nearly quintupled from 4.4 billion to 21 billion dollars. One suspects that both these are symbiotic relationships.

Lessons Applied to Higher Education

Although I do not expect colleges and universities to meet the same fate as the natural ice industry, American higher education today faces similarly radical innovations in the development and delivery of its core product—knowledge. Given the invasion of technology and the rate of innovative improvements in the application of multimedia methods to teaching and learning, it is likely that colleges and universities will be transformed in this century. Thus, a review of the lessons learned from research on how leading enterprises in tangible products tend to react to the challenges posed by innovations may be useful in the higher education setting.

Machine-made ice vastly expanded the use of refrigeration and its value in preserving and distributing food. Similarly, electric refrigeration not only further expanded the use of refrigeration, it also made possible widespread air conditioning and other applications. The Internet and multimedia methods might well have similar effects on our profession. While it is difficult to judge who might win and who might lose as a result of such innovations, it is clear that judging the new primarily as a substitute for the established is a common error. Often innovations capture the richest emerging opportunities and, for a time, may be more symbiotic than threatening. Every ton of the new machine-made ice, for example, at first led to the sale of three additional tons of harvested ice by making refrigeration more reliable and widespread. It took nearly half a century for machine-made ice to become a mortal threat to natural ice in its most established markets.

Generally, established leaders in their fields face two major hurdles in their contest with invading innovations. First, leaders need to develop an awareness of their own vulnerability—a slow and difficult process for any firm or organization that has experienced substantial success. Nonetheless, recognition of an external threat is the first requirement for effective action.

Second is to make the necessary organizational adjustments. The organizational problem for most established firms is that they and their technology are often stuck in a relatively static stage of development, while the challenger and its innovations are still in a dynamic and fluid stage. Moreover, organizations are structured in the image of their technologies and processes. The challenger brings a new and perfectible product with better performance (or performance potential), organizational flexibility, and entrepreneurial spirit; the challenger is unencumbered by human and physical assets geared to highly specific production. The established firm, on the other hand, is more bureaucratic, enjoys economies of scale (but in the wrong product), has tremendous investments in inflexible systems, and is most likely led by career managers.

It's easy to understand how established firms can ignore radical innovation when it first appears. For one thing, in the early stages it is far from clear that the radical innovation will have much impact. Established firms also carry the burden of large investments in people, equipment, plants, materials, and knowledge, all of which are closely linked to the established technology. It takes a rare kind of leadership to shift resources away from these areas where one currently enjoys success to an area that is new and unproven.

Findings in fields as diverse as the history of technology, corporate strategy, and the dynamics of innovation have reached a common and disturbing conclusion: An unhappy byproduct of success in one generation of technology is a narrowing of focus and vulnerability to competitors championing the next technological generation. Modern managers must develop a broad vision of the future and nourish organizational capabilities that will carry them forward successfully. This might be the ultimate managerial challenge.

Research on the most successful firms and organizations over the past century reveals a timeless characteristic: respect for the value of human resources and skills and their continuing development. The most important strategy of all appears to lie in top managements' appreciation of the people who build and sustain their enterprises, and in respect for their ability to learn and adapt to changing and challenging circumstances.

Further, campus leaders would do well to develop organizational plans that encourage monitoring the external environment, self-examination, and constant renewal. The foremost leaders will also develop contingency plans to address the unexpected circumstances that are almost certain to arise. In the best of cases, those unexpected changes will open the way to greater opportunities than at first imagined. Indeed, technological advances have spurred many changes that hold much promise for improving higher education.

Envisioning how the Internet and related new media technologies can help advance higher education is an exciting exercise. Multimedia educational tools such as The Valley of the Shadow, developed by Edward Ayers and his colleagues at the University of Virginia's Institute for Advanced Technology in the Humanities, illustrate the potential of technology to enrich teaching and learning. The Valley of the Shadow catalogs online and in great detail the daily lives of the people who lived in quite similar towns at opposite ends of the Shenandoah Valley, placing them north and south of the Mason Dixon Line during the Civil War. This interactive tool not only enables students to deeply engage historical material in a totally new way—to learn by doing, through guided experience and mentoring—but also enables them to make scholarly contributions as they work with the material and break new ground.

New multimedia-based methods of working with students and content simply have no precedent. Among their many advantages, these tools help build networks that bring in diverse information and encourage experiments, keeping perspectives fresh and generating new knowledge. Perhaps most important, they mitigate the narrowing of focus that can be the downfall of organizations at the top of their fields.

The virtual classics consortium of the Associated Colleges of the South presents another level of the use of technology to enhance teaching and learning. There, 15 colleges have collaborated by using technology to pool the resources of their individual classics departments. They created a powerful example of the benefits to be gained by working together. The opportunities for classics students at any one of the member institutions are greatly enhanced by access to resources across the consortium; faculty too gain by collaborating on courses and thereby expanding their perspectives.

I believe we are on the threshold of breaking down divisions across disciplines as well as across institutions. Technology enables scholarship and experimentation that breaks out of the boundaries that separate us, vastly expanding the possibilities for discovery at the intersection of disciplines.

I do not think that one of the inevitable results of technological innovation for higher education will be the scaling up of colleges and universities to massive proportions, able to serve many more students at once as a result of the economies of scale made possible by technology. Indeed, one potential result of the multimedia tools and efforts I have described is to allow intellectual endeavors to be pursued on a small but efficient scale. In many ways, technology works against the idea of scale through the creative and collaborative means it enables.

The direction of change and progress is often unclear except in hindsight. Yet when or how change will occur is not as important as recognizing that it undoubtedly will happen. In the final analysis, only that understanding will allow institutions to successfully make the transition to a new future. For higher education to play a meaningful role in building a new and better future for the world's citizens, education must be made more widely available, more reasonably priced, and more tailored to learners. Innovations in technology and new learning media hold tremendous promise for achieving those goals.

Notes

1. Carl W. J. Pistorius and James M. Utterback, "Multi-Mode Interaction Among Technologies," Research Policy, Vol. 26, No. 1, March 1997, pp. 67–84.

2. James M. Utterback, Mastering the Dynamics of Innovation (Boston, Mass.: Harvard University Business School Press, 1994).

3. James M. Utterback and Linsu Kim, "Invasion of a Stable Business by Radical Innovation," in The Management of Productivity and Technology in Manufacturing, Paul Kleindorfer, ed. (New York: Plenum Press, 1986), pp. 113–151.

4. Philip L. Anderson and Michael L. Tushman, "Technological Discontinuities and Dominant Designs: a Cyclical Model of Technological Change," Administrative Science Quarterly, Vol. 35, No. 4, December 1990, pp. 604–633.

5. Ibid., p. 609.

Further Reading

James M. Utterback and William J. Abernathy, "A Dynamic Model of Product and Process Innovation," Omega, Vol. 3, No. 6, 1975, pp. 639–656.

Clayton M. Christensen, Fernando F. Suárez, and James M. Utterback, "Strategies for Survival in Fast-Changing Industries," Management Science, Vol. 44, No. 12, December 1998, pp. S207–S220.

William J. Abernathy and James M. Utterback, "Patterns of Innovation in Industry," Technology Review, Vol. 80, No. 7, June/July 1978, pp. 40–47.