How to Stay Ahead When the Rules Change

Image

Illustration by Rune Fisker.

Change is hard. Even mighty firms, with all the necessary resources, perish because they never manage to respond effectively to a sudden shift in their environment — like, say, the rise of the web. (Remember Blockbuster?) Or web 2.0. (Remember Yahoo?)

But why is that? Why do companies so often fail to adapt? It’s a vital question in today’s fast-moving world. Surprisingly, says Julien Clement, assistant professor of organizational behavior at Stanford Graduate School of Business, we still don’t really know the answer.

Image

Julien Clement is an assistant professor of organizational behavior at Stanford GSB and the Business School Trust Faculty Scholar for 2019–2020. | Elena Zhukova

There’s no shortage of ways to fail, of course. Companies can respond with a flawed strategy. They can be hamstrung by vested interests or internal politics. Or they can simply fail to execute for lack of unfamiliar new skills. But at a more basic level, firms need to recognize the need for change. And as anyone who’s ever had a job knows, organizations tend to operate with a kind of blind momentum, often paddling furiously in the same direction as they sink into insolvency.

Clement found that rigidity especially perplexing. With all the smart, experienced people who make up any established company — not just in the C-suite, but also those on the front lines — why are they slow to change course? Why, he wondered, did firms so often respond to big challenges in small, inadequate ways?

Researchers have long been stymied in solving this puzzle. There are detailed case studies of companies that have failed to navigate shifting currents, but these are unavoidably subjective and anecdotal. It’s very difficult to study how companies adapt systematically, Clement says, because you need both a large sample of firms and a reliable way of quantifying their choices.

A few years ago, he had an idea. “I was working on a theoretical paper using agent-based modeling,” he recalls. “On a computer, we would simulate a world where the environment changes every once in a while and companies try to adapt to it. But this was all theoretical — we didn’t plan to use any real data.” Around that time, he sometimes relaxed before going to bed by watching video game tournaments on Twitch. One night it struck him: “I thought, wow, this is something I could look at in that context.”

To understand how businesses respond to change, he realized, video games — specifically, the professional multiplayer games known as eSports, which have exploded in popularity in the past few years — would provide an almost ideal natural laboratory. Wait, what? Video games?

Barbarians at the Gate

“I know, it sounds crazy,” Clement laughs. But all the elements of business competition are there: eSports teams are high-stakes, profit-seeking enterprises that can earn millions of dollars a year and that use strategy and tight coordination in an arena where sudden and unforeseen changes occur in the form of game updates. But unlike traditional organizations, in eSports every action by every player leaves a digital footprint. By tracing every mouse click, Clement could track teams’ strategies over time and see exactly how they react to change.

“It offered deep visibility into what’s happening, not just in the aggregate but at the micro level of individual behavior,” he says. “We’ve never had that kind of granularity in organizational research before.”

Clement decided to focus on a game called Defense of the Ancients 2. In it, two teams, each with five players, battle to destroy a mystical glowing structure called the Ancient in the other team’s fortress. Prior to the match, each player selects a certain “hero” with defined powers of varying sorts — much like functional experts in a firm, but with names like Bristleback, Enchantress, and Lifestealer.

Different heroes have different powers — from gravity-defying leaps to temporary invisibility — which allow each player to carry out a specific role in the team’s strategy. One might be responsible for hunting “creeps” to gather resources, while another provides cover. A team might use decoys to position itself for an assault, or play a guerrilla game, disrupting the enemy’s efforts to gather resources. (As in any enterprise, supply chains are vital. Gold is earned and traded for swords and bludgeons, healing potions, and myriad other useful assets.)

As part of his fieldwork, Clement attended DOTA tournaments and interviewed the pros. There was no doubt about their motivation: eSports matches are viewed by huge audiences online, and teams can earn millions of dollars for winning a single tournament. For the players, it’s a high-stakes, full-time job, and they are just as deeply committed to success as the members of any start-up with stock options would be.

Clement admits, somewhat sheepishly, that he also began to play the game himself. “I had to, you know, so I would understand it better. It’s all for the glory of science.”

The Illusion of Separateness

Image

The most prominent eSports matches are viewed by huge audiences, both live and online, and teams can earn millions of dollars for winning a single tournament. | Hu Chengwei / Stringrer

His theory was that something in the very structure of organizations — of any sort — shapes how the individuals inside them see the world. And that worldview, in turn, can make them slow to adapt.

When humans come together to pursue a goal, their first instinct is to divide the labor by function: a department for production, another for sales, and so on. The purpose, though we rarely think of it this way, is to minimize the burden of coordinating this complex social machine, Clement says.

Thus, highly interrelated tasks are given to a single group of people who work closely together: They have their own wing in the building, their own Slack channels. Organizations are basically networks of such clusters.

This division of labor makes a lot of sense. It allows us to focus on our own part of the puzzle and develop specialized skills. And, crucially, it keeps us from having to think about too much at once. “Cognitively, we form mental maps of our work environment,” he says, “and this structure lets us filter out a lot of the system-level complexity.”

The trouble comes when the world changes. Shocks that seem to impact a single cluster can ripple throughout an entire company — but it’s less likely that anyone will notice them. “When something changes,” Clement says, “we view it through the lens of our own habitual concerns.”

So while firms may respond at a local level, they often fail to consider the need for global changes in how the organization works. In other words, if this theory is correct, the very thing that makes a firm effective in stable times — a sharp division of labor — can make it ineffective in times of disruption.

Changing the Game

In Defense of the Ancients, that disruption comes in the form of revisions to the game’s code. The game’s publisher, Valve Corporation, made changes to the game every few months during the period Clement studied, from 2014 to 2017.

Some of these concerned the rules of play and directly altered the viability of certain strategies. Others tweaked the powers of specific heroes, often with an eye to restoring balance after players found synergies that tilted the scales. These latter updates can be seen as cluster-level changes that only affect one player of a team, but they also indirectly modify the optimal team-level strategy calculus.

For example, one update might boost a hero’s “slow” ability, enabling it to constrain an opponent for 5 seconds instead of 3 seconds. That makes it more attractive to use this hero, but it also raises the value of strategies that generally rely on constraining the movement of enemy fighters.

This was the engine that set Clement’s experiment in motion: When updates came out, would teams revise their game plan, or would they continue with the same plan and just swap in stronger heroes? Systemic change or business as usual with different personnel?

Sample size wasn’t going to be an issue here: DOTA boasts one of the largest worldwide pro circuits, and Clement was able to acquire a data set covering more than 14,000 matches on the platform. “For 275 teams over this three-year period, we got every mouse click by every player in every match,” he says.

Big data? Oh, yeah. After winnowing it down, he had a log of 300 million “meaningful actions,” which he sorted into a million action sequences. The analysis involved 850 million pairwise comparisons between these sequences. “Parts of it took weeks to run on a high-end desktop when I was a PhD student,” Clement recalls. “Even now, using cloud computing at Stanford, it’s an ordeal.”

But out of that he was able to gather an astonishing amount of information. The role of each player could be inferred by tracking their position on the game map over time. He could distinguish different strategies from their click patterns. And with an algorithm from bioinformatics (used to analyze DNA sequences), he could quantify and measure the size of the difference between any pair of strategies. That meant he could tell whether a team was trying something truly new or merely fiddling on the margins of its usual game plan.

Evidence of Component Focus

The results were unmistakable: When hero abilities changed, most teams responded only by switching heroes — sidelining, say, the Queen of Pain and subbing in a Dragon Knight. Only after a lag of several weeks did some begin to revise their strategies. In fact, immediately after a change, teams were less innovative than in stable periods.

Clement explains: “When the environment is constant, teams still evolve the way they operate at a certain rate. They’ll try new ideas and experiment, hunting for an edge. But when the world becomes more uncertain, they seem to refocus on doing what they already know. They reimplement their usual strategy, just with different pieces.”

Remarkably, these small, nimble teams were exhibiting the same inertia that big corporations exhibit when faced with change. But here, it couldn’t be attributed to bureaucracy or turf wars or fear of change. And Clement knew it wasn’t a failure to execute, because he could see whether they tried to adapt. Rather, it was a failure to recognize the full, systemic implications of external change.

By studying these small, stripped-down, highly motivated teams, Clement had found systematic real-world evidence that firms fail to adapt when they focus on specific tasks at the expense of system-wide behavior. This insight, he believes, should be fundamentally applicable to most organizations.

The Risk of Centralized Leadership

Clement also wanted to see if some teams are better at avoiding this problem than others. He was especially interested in the role of leaders. In times of tumult, could strong leaders guide their organization’s adaptation?

Much like real companies, eSports teams have leaders (one player per team designated as “captain”) whose role is to help the team find the right game plan. Clement was able to analyze the leaders’ roles from their in-game communications. He didn’t have a record of their comments, but by tracing in-game signals, called “pings,” he could get an idea of how much leaders involved other members in team decisions.

In some teams, information flowed in a hub-and-spoke pattern from the leader to the other players. In others, the flow was more even and distributed, as players seemed to be mutually adjusting to each other rather than just listening to the captain.

Surprisingly, top-down directions didn’t help very much, Clement says. Teams with flatter structures, where everybody seemed to be talking and contributing, were more likely to recognize the need to look for a new strategy rather than just swapping in new heroes. Only when the updates had an obvious and direct system-wide effect, such as far-reaching changes in the rules of the game, did teams with dominant leaders adapt faster.

“These two structures seem to facilitate change in different ways,” Clement says. ”A central leader may speed things up when it’s fairly clear how the organization should respond to change. But when the problem is to figure out how to respond, a less hierarchical structure might be more effective.”

Clement thinks future work should study how these results apply to traditional companies. Most companies, after all, have people whose whole job it is to think about strategy, scan the horizon, and run the show. The leaders of eSports teams, however, are player-captains — they also operate a hero in the game, so you might think they’d be more prone to forgetting about the big picture. But it does suggest that leaders in traditional organizations, when faced with change, should consider tapping more into their employees’ knowledge instead of searching for the best course of action on their own.

Lean In to Change

Clement’s study raises another question: Do organizations that adapt their strategies end up being more successful after a change?

Analyzing performance implications can be hard in traditional companies, where the right measure of success is often debatable. But in DOTA matches, there’s no ambiguity about which team ends up winning each game and earning a million-dollar payday. This means Clement could easily measure the effects of adaptation on performance.

Interestingly, he found that when teams introduced a new strategy after an update, they were more likely to lose that particular game. That actually makes sense, Clement says: “Experimenting with new ideas is a trial-and-error affair, it rarely pays off right away.” But when he took a longer view, he found that those innovating teams eventually outperformed the others and upped their winnings.

There was a fascinating wrinkle: Following hero updates, teams that worked out the ripple effects and altered their game plans ultimately did better. But when the updates had a direct system-wide effect, such as overhauls in the general rules of play, he found no discernible benefit to adaptation.

That might seem odd at first: Does adaptation really pay off least when change is the most significant? But the key, Clement says, is not whether a team’s strategy fits the environment. It’s whether it fits the environment better than its opponent’s strategy does. So when the effects of a change are evident, everyone adapts and no one gains an edge. When they’re not evident, being the first to figure them out can deliver a competitive advantage.

Seen in that light, companies also might view change as an opportunity, especially when the consequences of the change aren’t immediately obvious. When things are moving fast, it’s natural to narrow your focus and concentrate on keeping the ship on course. But it might be smart, this study suggests, to take a step back and think about where you’re headed and how the entire organization works.

“Sometimes the implications of change are a lot broader than they seem,” Clement says. “There might be components we don’t see as interdependent that actually are. Or maybe they weren’t before, but they are now. There might be new synergies to discover. The more you can broaden your perspective, the more likely you are to survive and thrive in times of change.”