The Brouhaha Over Rankings

One is a popularity contest.
Another is a number cruncher's dream.
As business school rankings proliferate, readers wonder:
What do they actually measure?
And whom do they serve?

By JENNIFER REESE

Illustration by Tim Bower

ONE AFTERNOON NEXT MONTH, as anyone involved in management education can tell you, copies of Business Week's biennial "best business schools" issue will make their long-anticipated appearance in university mailrooms across the country. A few anxious types already will have found a copy at an all-night newsstand. But everyone, from admissions directors and placement officers to professors, deans, and students, will pore over this issue. A few days later, alumni/ae will start to call and complain that their alma mater has dropped three spots; others will decide to donate money. And somewhere, a 26-year-old will apply to a school just because it has done so well. "Everyone waits for it," says Marty Geisel, dean of the Owen Graduate School of Management at Vanderbilt University. "It's like the Academy Awards."
       He's only half joking.. Love 'em or hate 'em, a decade after Business Week rolled out its first list of the best business schools, no one involved in management education can deny the impact of the splashy media rankings.
       "Rankings influence the number of applicants a school sees," says one business school dean. "Which is why people accord them so much respect and anguish and fear."
       "I checked everybody out in the rankings. Everyone who applies checks everyone out in the rankings," says Jason Rottenberg, a first-year student at Harvard. "Anyone who tells you they don't is lying."
       Business Week no longer owns the game. U.S. News & World Report began its own popular annual ranking of business schools in 1990. Success ranks the best programs for entrepreneurs. Working Woman is compiling a survey of the best schools for women. Forbes has been talking about starting its own list.
       The rankings bring obvious benefits to the magazines that compile them: Ranking issues fly out the door faster than Beanie Babies. The U.S. News graduate schools survey is among its 10 best-selling issues each year. Business Week's rankings sell thousands of extra copies at the newsstand.
       What the rankings do for business schools is complicated and something administrators feel ambivalent about. "Rankings are important for business schools because graduates don't get a license," says Robert Morse, deputy director of data research at U.S. News. "Doctors have to take exams; lawyers have to pass the bar. But with business schools, to some degree the validation for the credentials comes from the prestige of the school. Not that those words have ever been said by a dean."
       They've been said by students. Like Perry Solomon, a 1998 graduate of MIT's Sloan School of Management: "Your school's reputation acts as a calling card for you when you're out in the world. Obviously, if you go to a school that you think is terrific but isn't highly ranked, you've got more explaining to do about how good it is. And, by extension, how good you are."
       For this and other reasons, improving, or at the very least maintaining, their school's position on these various lists has become an increasingly important job for administrators. It is not, for the most part, a job they enjoy. While many administrators concede that rankings have helped make their schools more responsive to the needs of both students and the business community, they also think there are problems underlying the philosophy and the methodology of academic rankings. They believe the various lists now exert an undue influence on students and that they hardly offer a complete or accurate picture of an institution. Nor do they think that all the changes rankings have sparked have been good. "Things have never been quite the same," says Richard Kurovsky, former director of university relations at Stanford and now dean of marketing and communications at the Rotman School of Management at the University of Toronto. "And it's for better and it's for worse."
       It would be hard to find a more powerful spokesman for the positive impact of rankings than John A. Byrne, now a senior writer at Business Week. In the 1980s Byrne was struck by the fact that there were 50 schools blithely claiming to be in the "top 20" and 20 schools claiming to be in the "top 10." And MBAs, Byrne says, had become famous for their arrogance. They were known for being smart with numbers and terrible with people. Meanwhile, the schools that trained them seemed to him to be out of touch. "Business schools had become research institutions, which unfairly penalized students and unfairly benefited professors who were doing extremely esoteric research that had no meaning in the real world. That's OK in a different kind of school but not in a professional school that is serving the practitioner," says Byrne. "The sense was that academic institutions never had customers because even to think of a student or employer as a customer was to pander to them. It was considered anti-intellectual to think of yourself as a business. Even in a business school."
       Byrne devised a straight-up customer satisfaction survey, which he mailed out to companies recruiting MBAs. He asked them about the quality of the people who came out of various institutions. He also sent surveys to students, asking them to rate their experiences at those different schools. The results of the first survey were surprising: The school that ranked highest wasn't Stanford or Harvard or Wharton. It was Northwestern's previously unheralded Kellogg School.

Illustration by Tim Bower

       "That was incredibly provocative. Schools were arrogant enough to believe they were above scrutiny, especially by the mindless media," says Byrne. "It turned out that schools that thought they were doing a good job weren't always doing as good a job as they thought. Other institutions that were moving quietly along came out high."
       What was Kellogg doing right? For one thing, Byrne says, it interviewed all applicants, throwing personality into the mix of qualifications for admission. This showed up later with high marks from employers, who liked well-rounded Kellogg MBAs, according to Byrne, better than the nerdy geniuses from "better" schools. Kellogg also had developed a cooperative culture on campus that students seemed to appreciate. Soon, Byrne says, other schools began studying Kellogg to find out how it made its "customers" so happy. "Not that the ranking was solely responsible, but it really helped improve a lot of schools," says Byrne.
       Leaving aside the question of whether or not this is true, the idea implicit in the Business Week rankings that academia, like a short-order restaurant, is serving customers isn't something that everyone has gotten used to. Can institutions of higher learning be assessed like toasters in Consumer Reports? Should they be?
       "Business schools are increasingly viewed as a commodity, like soap or automobiles," says Charles Hickman, director of projects and services with the International Association for Management Education, the primary accrediting organization for American business schools. "A lot of people don't view that as progress."
       "It's not clear how much voice the consumers should have," adds George Daly, dean of New York University's Leonard N. Stern School of Business. "It's possible business schools might make changes in the curriculum that would appeal to students but not be in their best interest."
       But here's another question. Even if you desperately want to hear what they have to say, can students ranking their own school be as detached as shoppers rating a detergent? There are widespread rumors of schools coaching students on how to fill out the surveys; there are whispers that students at less prestigious institutions send back suspiciously upbeat surveys. After all, it's better to graduate from a top-ranked school than a mediocre one. And if exaggerating your school's merits helps, why not?
       "We're dealing with people who have spent $150,000 in forgone salary and tuition. They're very discriminating and very discerning," replies Byrne. "Basically, they tell it like it is. Of course there are some cheerleaders. There are people who have an axe to grind. The beauty of the survey is the cheerleaders get evened out by the axe-grinders."
       U.S. News, which publishes the other influential business school survey, has taken a different approach to ranking schools, one in which opinion is only part of the formula. The magazine asks recruiters and business school administrators (though not students) what they think of various institutions. Then it collects a bundle of hard data from each school, everything from starting salaries to GMAT scores, and throws it all together to come up with rankings.
       Complicated? This doesn't seem to be the biggest problem for most people. Probably the most common criticism of the U.S. News survey is that the data it collects from the schools isn't audited. "Stanford doesn't lie," says Vanderbilt's Geisel. "And Vanderbilt doesn't lie. But everyone else does."
       "People are lying through their teeth in reporting the data," agrees NYU's Daly. "Jack Welch may be the most respected CEO in America, but no one would believe unaudited numbers coming from him. Why do they believe them coming from business schools?"
       This is a good question, and one that U.S. News doesn't really answer. "We hear: 'We don't cheat, but everyone else does.' OK, prove it. If you know it's happening, find me the numbers," says U.S. News senior writer Mary Lord. "I guess I feel that these institutions are big players and have a lot to lose. If they got caught it would look terrible."
       But most administrators say there are ways of cooking the books that would appear relatively innocent if they ever came to light. For instance, some schools might exclude trailing spouses from their tabulations of students with jobs three months after graduation. Others might quietly eliminate foreign students when figuring starting salaries. "With the best of intentions, schools interpret questions different ways," says Hickman, "though there's a fairly widely held feeling that not all schools have the best of intentions. There's a lot at stake and this is self-reported."
       There are other doubts undermining the U.S. News survey. For instance, starting salaries of graduates are a factor, and the higher the better. This clearly favors schools that place students in consulting jobs in cities like Manhattan, where the cost of living is high--and so are paychecks. "We know if you place graduates in the Midwest, salaries will be 10 percent to 20 percent less than on the coasts," says Joe Alutto, dean of the Max M. Fisher College of Business at Ohio State.
       Does this mean that a Midwestern school is therefore inferior? Of course not. Small programs, like Stanford's, face another kind of bias. "Our perennial problem is we never have enough students to meet the needs of recruiters," says Karen Nierenberg, associate dean for external relations. "They never leave as happy as they'd like." Stanford always ranks low on recruiter surveys.
       So what do these rankings really mean to schools? How much does a disappointing ranking actually hurt? "You get gray hair, go bald, and don't sleep," says Andy Policano, dean of the University of Wisconsin-Madison School of Business, which plummeted 26 spots over the last two years in U.S. News. And in his school's case, he says, what looks like a severe deterioration of educational quality reflects nothing but a slight change in how the magazine interpreted employment data.
       Among the middle- and lower-ranked schools, the pressure to do well seems to be the most intense. A sudden drop can be extremely unsettling, though neither applications nor donations seem to be dramatically affected. But a heroic jump into the top 20--or better yet, the top 10--can rally alumni/ae to make major gifts and attract brighter applicants."If you're in the second- or third-tier group, making a substantial movement affects giving," says Steve Wheelwright, MBA '69, PHD '70, who runs Harvard's MBA program. "These schools do everything they can to get the ratings published and prevalent in the public's mind. That's how they get money," he says.
       Schools that consistently rank in the top five or ten seem to be less concerned. "All the fluctuations in the rankings haven't affected the percentage of those we accept who actually come," says Wheelwright. .

Illustration by Tim Bower

       When Stanford ranked seventh in Business Week's 1996 survey, did it affect the number of applications? "They didn't falter for a moment," says Nierenberg. This isn't to say Stanford paid no attention. "If you get hammered in the rankings it's not a lot of fun," says GSB Dean Michael Spence. "It turns out there was some unhappiness among recent graduates, which had to do mostly with certain parts of the core curriculum. We went to work trying to fix it."
       Which is one of the positive effects of rankings: Schools know when there's a problem--and so does the rest of the world. These days when students complain about an inefficient placement office, for instance, business schools are quicker to fix the problem. "I got personally criticized for being distant, aloof, and inaccessible," says Vanderbilt's Geisel. "I've made some changes."
       But with all the hoopla surrounding the rankings, to what extent do they actually influence decisions? Do prospective students take them seriously? Jason Rottenberg applied to a handful of schools--including Wharton, which Business Week named the Best B-School in America in its last rankings issue. But it took him three seconds to decide to say no to Wharton. "As soon as I got into Harvard, it was between Harvard and Stanford," says Rottenberg. "If you get into both of them, you have a decision to make. If you get into one but not the other, you have no decision to make." And this, by the way, has nothing to do with the fact that Harvard and Stanford were tied for first in the U.S. News rankings. That, says Rottenberg, is merely incidental.
       Says a first-year student at Berkeley's Haas School: "The bottom line is that Stanford and Harvard will always be number one and number two, no matter what the rankings say." Of course, he gave the rankings a thorough perusal when he started applying to schools. And the fact that Santa Clara University, which had a program he found appealing, didn't even show up factored in his decision not to apply. But, he says, "I don't think anyone who could get into a top school would be foolish enough to go only by a ranking."