chael schell knows a thing or two about measuring high performance. A baseball statistician par excellence, he’s put together a convincing argument for declaring the “all-time best sluggers” in the history of the game. To people outside baseball, it might seem curious that the task would be a hard one, much less that Schell’s argument would require 400-plus pages to defend (his book was published by Princeton University Press earlier this year). What could be so complicated about counting hits? But Schell knows different. What he’s claiming to have accomplished is so ambitious—the variables so legion, the data so asterisk laden—that he calls it the holy grail of baseball statistics. Plenty of people with other methods will try to show he’s wrong.
And so it goes, too, with declaring the greatest players in business. Except that it’s even harder. Consider, first, that we business spectators don’t even have the benefit of an agreed-upon scoreboard. Are the winners the ones with the highest market caps, the ones with the greatest sales growth, or simply the ones that remain standing at the end of the game? (And when’s the end of the game?) Then, too, there’s the impossibility of holding anything constant in terms of context. Are you better if you boomed in bust years or if you really boomed in boom years? Hardest of all is the follow-on task most business stat masters set for themselves: discovering not only who’s the greatest but why.
The challenge of measuring companies’ relative performance across industries and eras, declaring the top performers, and ﬁnding the common drivers of their success is so daunting that it might seem a fool’s errand to attempt. In fact, no one did for the first thousand or so years of business history. A scan of Harvard Business Review’s contents over 83 years suggests that the quest didn’t even occur to anyone until around 1980, when Tom Peters and Bob Waterman got down to work researching and writing In Search of Excellence. That probably explains why the book became such a publishing sensation. As management consultants, Peters and Waterman sat at the intersection of scholarship and practice, and their work cast down a gauntlet in each direction: They challenged managers by claiming that varying managerial actions and attitudes could account for the difference between winners and losers. At the same time, they challenged researchers by claiming that the problem of isolating the drivers of high performance was tractable. Did they get the answers right? Probably not. Famously, a number of their “excellent” companies have ceased to be so, which may or may not mean those businesses had the right stuff at the time. In the end, the impact of In Search of Excellence is less that it solved a problem than that it put the problem on the table.
To be sure, many management researchers continue to believe it’s a quix-otic quest, but several first-rate scholars have been unable to resist pursuing it. (See the chart “Doctors Differ: Ten Research Teams Discover the Keys to High Performance.”) And that seems to be a good thing, because with each new effort, methodological issues are reconsidered, new data become available, and the findings become more defensible and robust. To understand the pro-gress we’ve made toward a theory of high performance, it’s useful to review the research design questions that have had to be addressed in each outing.
What’s the Unit of Analysis?
The difficulties of studying high performance begin with determining where it chiefly resides. Is it in the individual, the team, the business unit, or the corporation? Plenty of theorists have focused on the individual level, churning out the managerial equivalents of self-help books for Welch wannabes and the time-management challenged. Jim Loehr and Tony Schwartz, with their Corporate Athlete Training System, make perhaps the most explicit promise of high performance, but all offer keys to getting to the top of your personal game. At the team level, likewise, there are analysts like J. Richard Hackman of Harvard University and Susan Lucia Annunzio of the Hudson Highland Center for High Performance (and in this issue of the magazine, Gallup’s John Fleming, Curt Coffman, and James Harter) who study high-performing work groups and the conditions that give rise to them. For many, the business unit is the right thing to focus on, as the Strategic Planning Institute’s famous PIMS (Profit Impact of Market Strategy) studies have. One reason to pick a business unit (or a business, for that matter) is that it has a bottom line—a generally agreed-upon measure of results that acts as a kind of weighting mechanism for the myriad factors that constitute performance. Perhaps for this reason, CEOs themselves are especially prone to focusing on this level, some by promoting internal rivalry among units as to who adds the most value to the business. Of course, at each level—individual, team, profit center—the variables multiply, and the problem of parsing high performance becomes more complex. And this culminates at the highly complicated level of the corporation; the efforts spotlighted in the “Doctors Differ” chart are all aimed at this level.
Who Gets Called a Winner?
The essence of the scientific method to which all these broad-based studies aspire is finding the fittest entities across the business landscape and subjecting them to close analysis. But figuring out who stands tallest is far from straightforward; it depends upon which yardstick you use. For the most part, there is agreement that success shows up in cash and that cash comes to businesses in various forms. So, for instance, professors John Kotter and James Heskett, looking for the links between strength of organizational culture and economic success, define that success in terms of annual growth in net income, average returns on invested capital, and appreciation in stock prices. Chris Zook, a partner at Bain & Co. who heads the firm’s Global Strategy Practice, goes in for a similar mix, including companies that have grown both revenues and profits and produced total shareholder returns in excess of the cost of capital. In both cases, importantly, they apply their screens more rigorously than Peters and Waterman, who, despite naming six different financial metrics, seem to have applied those measures unevenly—even conveniently. Almost 20 years after In Search of Excellence, Peters wrote in Fast Company, “For what it’s worth, okay, I confess: We faked the data.” According to his account, the book’s earliest readers asked about research methodology. “The big question was, How did you end up viewing these companies as ‘excellent’ companies?” His answer reads like something out of the Journal of Irreproducible Results:
How did we come up with them? We went around to McKinsey’s partners and to a bunch of other smart people who were deeply involved and seriously engaged in the world of business and asked, Who’s cool? Who’s doing cool work? Where is there great stuff going on? And which companies genuinely get it? That very direct approach generated a list of 62 companies, which led to interviews with the people at those companies. Then, because McKinsey is McKinsey, we felt that we had to come up with some quantitative measures of performance. Those measures dropped the list from 62 to 43 companies. General Electric, for example, was on the list of 62 companies but didn’t make the cut to 43—which shows you how “stupid” raw insight is and how “smart” tough-minded metrics can be.
Were there companies that, in retrospect, didn’t belong on the list of 43? I only have one word to say: Atari.
It’s hard to defend a research population that emerges from a coolness screen, but some researchers since have shared the same basic sense that the selection criteria can’t be purely financial. Just as the full measure of a man can’t be taken by his accountant, they believe there’s more to a great company than money. Although researcher Jim Collins relied on cumulative investor returns relative to the general stock market to draw the winner’s circle in Good to Great, he and Jerry Porras used a different method in their earlier work together on Built to Last. In that study, they looked at “companies [that] had risen to iconic stature and held it for five, ten, or 15 decades.” To discern which firms had achieved this stature, they surveyed “a carefully selected sample of CEOs from large and small companies.” If this sounds suspiciously like Peters and Waterman’s method, give Collins and Porras more credit. They were asking about “premier institutions—the crown jewels—in their industries, widely admired by their peers and having a long track record of making a significant impact on the world around them.” It’s fair to assume their CEO focus group didn’t ignore financial results in their nominations. The authors hoped to bypass tortured equations and incomplete data sources by tapping into the internalized, balanced scoring system that a seasoned corporate leader carries in his or her head. (The same argument underlies the compilation of Fortune’s annual Most Admired list.)
Figuring out who stands tallest is far from straightforward; it depends upon which yardstick you use.
But Collins and Porras’s real breakthrough was the idea of analyzing matched pairs of companies. In every case, they put their iconic company up against an also-ran that, at some point, held equal stature in the same industry. They studied how the two diverged from that point on and then looked for patterns across all the winners. It’s a decent way to deal with the problem that nothing can be held constant in the real world, and it corrects for the unequal returns enjoyed by emerging and mature industries. It provides a straightforward answer to the follow-on question about any company declared to be high achieving: Compared to what?
Accenture’s Paul Nunes, who is currently embroiled in that firm’s broad-based study of high-performance factors, says this may be the most important question in the research design. “Context is everything,” he says. “You can call anyone a winner depending on how you draw the set around them.” Bill Joyce, Nitin Nohria, and Bruce Roberson’s way of drawing that set may be the most useful yet. In their research for What Really Works,they composed “quads,” or groups of four competitors within an industry. They looked at the companies’ total shareholder returns relative to their peers’ over a ten-year period and named within each group a “winner” that consistently outperformed its rivals during the study period; a “loser” that consistently underperformed; a “climber” that improved its performance; and a “tumbler” that started off well but deteriorated over time. Which brings up the last point about the difficulty of declaring winners: They must win over some well-defined time horizon. As Nunes puts it: “Is the best athlete the one with the best career, the best season, or the onetime performance that set the world record?” Professor Arun Kumar Jain, in the research that led to his bookCorporate Excellence, looked at a four-year period (while freely admitting that some of his highfliers went into nosedives later). Contrast that approach with the emphasis Collins and Porras put on companies that endure for five, ten, or 15 decades. If there is a consensus forming on the right time frame to study, it seems to be around a decade. A ten-year standard would require a company to perform well over the tenure of two CEOs, on average, in North America. But Accenture applies its screen—which analyzes cumulative average growth rates—over four different time frames just to make sure that great companies in newer and cyclical industries aren’t dismissed.
What Constitutes a Pattern?
Take another look at the chart provided, at the final row, in which each research team’s keys to success are summarized. Consultant Jon Katzenbach writes of the importance of five paths: mission, values, and pride; process and metrics; entrepreneurial spirit; individual achievement; and recognition and celebration. Researchers Richard Foster and Sarah Kaplan tell us to transform our companies periodically rather than rely on steady, incremental improvement. Professors Karl Weick and Kathleen Sutcliffe tell managers they must inculcate a collective state of mindfulness in their companies. If all this seems vague—in some cases even banal—then that speaks to the challenge of finding the common ground shared by diverse practices across diverse industries. If the most successful retailers are creating loyalty programs, and the most successful product manufacturers are bringing buyers into the innovation process, then it may be reasonable to put a wrapper like “close to the customer” around these and other companies’ practices—and it may be impossible to put any finer point on it.
But what if all or some of the losers on the retail scene are also building loyalty programs? What if every manufacturer has jumped on the cocreation bandwagon? It’s an important question of research design whether to include or omit factors that are common to winners but not points of differentiation from losers. Collins and Porras, for example, found charismatic, effective leaders at the helms of their enduring companies—but also at the helms of many businesses that went by the wayside. Because the authors were out to find the distinguishing variables between winners and losers, great leadership didn’t make the list. (Still, as with all such “hygiene factors,” it seems unlikely a company could go far without it.)
An even bigger problem is getting past correlations in the data to be able to argue causality. If a researcher finds that highly successful companies tend to have formal knowledge management initiatives, for example, does that mean that explicit management of knowledge is a key to success? Or does it mean that knowledge management is the kind of organizational boondoggle that only a company flush with cash indulges in? Making the argument for causality in one direction or the other requires not only a sufficient data set but also a rational model for how the observed phenomena relate to known outcomes.
Kotter and Heskett struggled with this problem explicitly in their work to understand the impact of organizational culture. In acknowledging the questions raised about causality, they cite the perspective that “strong cultures cause strong performance, yet the reverse is known to occur, too—strong performance can help to create strong cultures. Could the latter explain most or all of any relationship found between culture strength and performance?” Likewise, Joyce, Nohria, and Roberson took special pains to get beyond correlation. By comparing not just winners and losers but also climbers and tumblers, both of whose performance changes over time, they developed a more nuanced sense of what was making the difference in outcomes.
Are the Answers Universal?
If the first requirement of a theory of high performance is that it have explanatory power—in other words, the patterns of practice identified truly do account for the superior outcomes—then the second requirement is that it have predictive and even prescriptive power. This move from explanation to advice is what’s known in academic circles as the shift from descriptive to normative theory. And this is where Harvard Business School’s Clay Christensen says that efforts to identify the best practices of successful companies have tended to fall on their faces.
In a working paper called “The Cycles of Theory Building in Management Research,” Christensen and Paul Carlile of Boston University write : “Management fads often are created when a researcher studies a few successful companies, finds that they share certain characteristics, concludes that he has seen enough, and then skips the categorization step entirely by writing a book asserting that if all managers would imbue their companies with the characteristics of these successful companies, they would be similarly successful.”
The categorization step he refers to is that critical stage in theory building where researchers create or adopt a classification system to help make sense of their observations. The right categories make clear under what conditions an action will reliably lead to an outcome. So, for instance, a researcher might realize that, while a certain factor characterized most of the successful companies under review, it was not present in any of the smaller ones—or that it was relevant only to particular industries or only to start-ups. It could be that some business practices are sensitive to national culture. Recently, professors Mansour Javidan and Robert House completed a ten-year study called Project GLOBE (Global Leadership and Organizational Behavior Effectiveness), which sought to nail down cultural variances around the world that would render different management practices more or less effective. The findings suggested, for instance, that while 360-degree feedback may improve management in American settings, it might be wholly ineffective in Thailand.
Is High Performance Timely or Timeless?
Another way to categorize what works in business, of course, is by that time-honored (and time-honoring) phrase, That was then; this is now. Christensen talks about the “killer question” he got from an engineer in the disk drive industry about his innovator’s dilemma theory. The man asked: “It clearly applies to the history of the disk drive industry. But does it apply to its future as well?”
It certainly seems fair to speculate that different things may work in different times. This was the rallying cry, after all, of the dot-com founders of the 1990s. The new economy, they claimed, operated by fundamentally unique rules. Time marches on for industries, too. As author Geoffrey Moore’s work makes clear, different kinds of investments pay off in different stages of an industry life cycle. At the company level, the different priorities of start-ups and established firms are often discussed. But do the researchers behind the studies discussed here admit that their prescriptions may have only a certain shelf life?
Arun Kumar Jain may be the most ready to concede this, especially since his corporate excellence framework is designed to be of a dynamic nature. (His findings pinpoint what produces good outcomes given current infrastructural constraints, while also acknowledging that these change over time.) For the most part, however, researchers have tended to ignore the question. The assumption seems to be that if it works for the companies that are most successful today, it will work for the foreseeable future. Stay tuned for updates.
A Breakthrough on the Horizon?
Writing for Harvard Business Review in July 2003, Joyce, Nohria, and Roberson described the two seemingly simple questions they had set out to answer in their Evergreen Project: “Why do some companies consistently outperform their competitors? And which of the hundreds of well-known business tools and techniques can help a company be great?” In fact, as we’ve seen, the questions aren’t simple at all.
But the research is getting better. People are working with richer data sets and more robust theories. Consider the distance we’ve come from Peters and Waterman’s “cool” research population. Today, a firm like Accenture—presumably just as interested as McKinsey was in flattering clients—puts its high performers through a rigorous and balanced screen before examining them under the microscope. Consider how Collins and Porras shifted the emphasis from the full set of things that winners do well to the distinguishing variables between leaders and laggards. Consider the volume of data collected and mined by the army of researchers working under Joyce, Nohria, and Roberson’s direction.
We have reached a critical point in the evolution of a theory of high performance—the point where management researchers have begun to build effectively on one another’s work.
Consider all these advances together, and it seems as though we have reached a critical point in the evolution of a theory of high performance—the point where management researchers have begun to build effectively on one another’s work. The quest to find the master keys to company success, which was spurred by the audacity of two consultants in 1982, has in some sense become a joint endeavor. It’s fair to say that, as an ongoing effort, it still falls short of excellence. But it’s making progress.