Write or Wrong, Communication Counts
by Michele Zak
I'm worried that the new essay tests will become the glass ceiling to foreign-born students," read a letter to the editor of the San Jose Mercury News last April. Responding to an article about the new analytical writing assessment of the Graduate Management Admission Test (GMAT), Noriko Yamada conceded that "English proficiency is important." But, she warned, "It should not be the method to screen out applicants who have talents in other fields."
As the first major change to the GMAT in 30 years, the new essay requirement that goes into effect next month has generated a wide range of responses and stimulated discussion about the role of communication in management education. It has also unearthed profound misunderstandings about what the issues are. Key among these misunderstandings is the one harbored by the author of the letter to the Mercury News, that writing competency is measured strictly by a yardstick of grammatical correctness. If a writer, that view maintains, avoids dangling her participles and splitting her infinitives, she passes muster for graduate or professional study.
What, in fact, is GMAT planning to assess in this new effort? Everyone agrees with Ms. Yamada that English proficiency is important, but exactly what proficiency is beyond grammar and sentence structure, how it is demonstrated, what its relationship is to a career in management, and how much proficiency is enough are questions less likely to meet ready agreement.
Clearly, the Graduate Management Admission Council (GMAC) and its member institutions had specific objectives and criteria in mind as they developed this new assessment tool for applicants to MBA programs. What are these objectives and criteria? How were they spawned, how will they be implemented, and what will be the effect on MBA-granting institutions and on the business world? And, perhaps the most provocative question: why now?
Response to a mandate
Two years ago, seven business schools told GMAC they wanted to know more about applicants' abilities to "convey complex ideas in writing." Their request, they said, was a result of prodding by their graduates' employers. GMAC then polled 584 business schools, including member and nonmember institutions. The response was unequivocal. Of those schools surveyed, 88 percent indicated they wanted a writing assessment component in the GMAT for use in the selection process, as a diagnostic instrument to determine skills development needed in their degree programs, or both. This was a very different outcome from that of a similar survey over a decade ago, when plans to incorporate writing assessment into the GMAT were torpedoed. It suggests a major change in attitudes toward the importance of communication in management education.
Some of that change must certainly be a response to expressions of discontent with the communication skills of MBAs that business schools have been hearing from the corporate world. Similar emphasis on the need for high-level communication competency in their careers has been widely reported in alumni surveys mounted by business schools as they consider MBA curriculum changes and evaluate executive programs that must earn their keep in increasingly competitive marketplaces.
And if official confirmation were needed, it came in 1988 in the widely read report on the status of management education by Lyman W. Porter and Lawrence E. McKibbin, Management Education and Development: Drift or Thrust into the 21st Century? Porter and McKibbin report that "poor communication and interpersonal skills" is one of the four major criticisms currently leveled at MBA graduates by the corporate world and by deans and faculty. In their surveys, communication is defined broadly as "being able to get meaning across and to be persuasive."
Finally, changes in the business environment and, indeed, in ways of understanding the world put unprecedented demands on managers to use language effectively. If MBA programs are to produce managers who can flourish in the new environment, candidates must have, or potentially possess, the quality of mind that can synthesize, imagine, and be able to construct a complex argument. And of course they must have, or be able to develop, the ability to convey ideas, decisions, thoughts, and information clearly, effectively, and persuasively.
Devising the appropriate test
According to GMAC, the objective of the new analytical writing assessment (AWA) is to assess "the ability to think critically and to communicate complex ideas." Thanks to productive research and scholarship in written communication over the last decade or two, a good deal is known about the process by which those abilities develop and the indicators in written work of ability to think critically. An effective test will elicit demonstration of an applicant's capacity for abstract thought by requiring him or her to synthesize data or ideas and to construct evidence for an argument. It will measure critical thinking capacity by testing the applicant's ability to use evidence to establish conclusions and to relate conclusions to conceptual frameworks. The complexity of the argument, both in terms of its abstractions and the concrete detail marshalled to support the abstractions, will indicate much about the applicant's critical thinking skills and his or her ability to communicate that thinking to a targeted audience.
The AWA assesses these skills by requiring the test taker to complete two writing tasks that, as the test booklet explains, "explore the complexities of an issue or an opinion, ... take a position informed by an understanding of these complexities, and ... formulate an appropriate and constructive critique of a conclusion to a specific line of thinking." The writing tasks, the booklet assures us, are based on topics of general interest and do not presuppose any specific knowledge of business or other content area.
But how will the tests be scored? For many years, both the costs of reading large numbers of student essays and the difficulty of achieving consistency in scoring stood in the way of incorporating essays into large-scale tests like the GMAT. Certainly, a level of uneasiness has always existed about the subjective nature of scoring essay examinations. And often with good reason.
The problem of consistency in scoring, along with the issue of cost, appears to have been solved by the development of "holistic scoring." In holistic scoring, expert readers assign a single score to essays for the overall quality of the writing. Reading takes place in controlled groups, all members of which have agreed to a set of criteria for judging essays written in response to a carefully constructed assignment. GMAC reports that the reliability of this scoring system is quite high: "Any two readers of a test, working independently on a six-point scale, will differ by no more than one point approximately 95 percent of the time." On the few occasions when a difference of more than one point occurs, a third expert reader reads and scores the essay. Expert readers who have thoroughly discussed the assignments and have committed to a set of guidelines or criteria for judgment sustain remarkable unanimity in scoring essays.
Some very obvious uses of scores and essays suggest themselves immediately. Most obviously, although GMAC has expressed a hope that establishing authenticity of application essays will not be a primary use, the AWA will be invaluable in evaluating the command of English of international applicants and of any applicant whose first language is not English. Either as a selection criterion or as a way of determining whether remedial language or communication work is necessary after acceptance, this addition to the GMAT should benefit business schools.
Beyond that practical use, business schools, including the GSB, need to think about how we will use this assessment. A few years from now, GMAC will be able to survey institutions and report to us how the assessment is being used in sister institutions, in admissions, and in communication pedagogy. At about that time, they will also be able to offer us correlations of written assessment scores and the traditional multiple-choice verbal sections of the GMAT. Until then, however, most business schools will be exploring uncharted territory.
The Porter-McKibbin report tells us that the business world is generally happy with the preparation of business graduates in particular business subject matter and for analytical tasks in general, but less so with personal skills, "including both communications É and leadership that is capable of influencing others." We know from our alumni surveys and from interaction with the corporate world that this concern for communication, including interpersonal skills and ability to influence, is widely shared beyond the School boundaries. To some degree, GSB students share it as well, although they often define communication quite narrowly as a set of stand-up presentation skills that they clamor to be taught, and resist suggestions that they need work in writing/thinking. Like the author of the letter to the Mercury News editor, many tend to see writing as sentence- or paragraph-level English proficiency. And, indeed, Porter and McKibbin tell us that students' self-perceptions indicate a palpable lack of insight into how they are seen by the corporate world in their practice of communication.
Come October, a new window will open for MBA applicants, as well as business school faculty and administrators, for evaluation of competency in critical thinking and the concomitant ability to get meaning across and to persuade. It remains for us to use the new light we gain to best advantage for MBA students and the business world they will join.
About the author: Michele Zak
As a member of the GMAC Advisory Committee on Writing Assessment, Michele Zak helped to develop and test the new analytical writing assessment. Director of the Business School's Management Communication Program, Zak earned her PhD at Ohio State University.
This is an official Stanford Graduate School of Business webpage
Copyright © 1996 Stanford University - Graduate School of Business