Priming the Pump for Better Classroom Performance

May, 2003

By Kathleen O'Toole

When economist Paul Romer arrived at the Graduate School of Business in 1996, he assumed he would be teaching some of the smartest, most disciplined adults in the nation and there would be no need for pop quizzes. "They were old enough to make their own choices. If they paid their tuition and went to the golf course instead of my class on macroeconomics, that was fine with me," he recalls thinking. It wasn't long, however, before the MBA students were grumbling. Romer might be the esteemed father of new growth theory, but having the option to listen to him wasn't enough. They wanted him to push harder so they would learn more.

Out of this experience, a company that just might revolutionize higher education instruction was born. Aplia Inc. began selling Internet-based software to college teachers of economics last fall, and preliminary results look promising. According to one professor, Kristen Monaco of California State University, Long Beach: "Now that my students are using Aplia, they're coming to class more prepared." Adds Professor Melanie Marks of Longwood College in Farmville, Va.: "For the past several years, my testing style has been exactly the same and my average on the first test was always around 74. … The only change I have made this semester is that I am using Aplia for practice problems and quizzes. My students just averaged an 81 on my first test."

An avid classroom learner himself, Romer thought of the GSB students who wanted him to push them harder as lacking self-discipline until he had a chance to walk—or rather run—a mile in their shoes when he joined an athletic team to please his wife.

To his surprise, he started exercising much more and did things that he never thought he would be able to do, such as complete a triathlon. Suddenly it dawned on him: "I think my students are immature because they are asking somebody to make them put forth regular effort, yet here I am working with a coach and a team because I couldn't make myself exercise enough on my own."

On a two-year leave of absence from the Business School, Romer found investors and launched Aplia. For $28 a student, the company offers Internet-based instructional materials designed to push students to exert more effort outside of the classroom without imposing on professors the drudgery of collecting and grading homework.
Instructors at 100 institutions used the Aplia material to teach Principles of Microeconomics last fall to approximately 10,000 students. In January, the company announced it was adding macroeconomics and intermediate microeconomics to its product line. Stanford Business caught up with Romer recently at his corporate office in a warehouse district of San Carlos.

Q: You are the son of Roy Romer, the superintendent of the Los Angeles schools and the education-oriented former governor of Colorado. Did your interest in education come from him or vice versa?
A: I learn about what it's like out the trenches from talking to him. He learns from me about the connection between education and long-run economic growth. So our discussions are like a tennis match. After a while you forget who served.

My interest also comes from thinking about how our economy benefits immeasurably from investments that were made in the 19th century in creating the modern university system. For example, the Morrell Act created the land grant universities, a whole new system of higher education. The educational system that resulted spurred our economy to faster growth in the 20th century. New growth theory suggests the potential for us to do that again, to even further speed up the rate of growth. In so doing, we can help not just people in the United States but ultimately everybody in the world. Every problem we face, whether it is insufficient water or pollution or global warming, is going to be easier to deal with if we have more economic resources and better-trained people.

So I have this impulse to see how we can make the world better. One way for me to do that is to try to influence the policy process a little bit; e.g., to get the federal government to provide more support for the training of scientists and engineers. But another way is to try to change how the educational publishing industry works, change how it interacts with professors and raise the quality of the educational experience for kids in schools. By myself I'm not going to come up with something like the land grant universities, but as you agitate for the big things, you also try to build the little things and hope that somehow along the way you make a difference.

Q: Technology has been useful in many areas of economic life. How about in education?
A: There is this old saying that when you've got a hammer, everything looks like a nail. I think people took the technology tool and said, "We can computerize accounting, banking, education"—they started with the tool and went looking for something to hit with it. So far, in education, this hasn't helped. I think you need to start with an understanding of what the problem is; then you can figure out the ways various resources can help solve it.

The fundamental problem in higher education is how to give students enough structure to make sure they do enough work outside the classroom. That is something that technology can help with because a key part of any solution is information collection. You need to set up an incentive system that has little pats on the back and little wagging fingers. To implement this, you've got to measure what students have done, then pass that information to the professor and back to the student. This is the kind of thing that information technology is good at. Others have argued: "Let's turn the computer into a professor who will answer the students' questions," or "Let's use multimedia to bring these concepts alive." Those were impulses driven by "Wow, we have this new tool; what can we do with it?"—not by a clear statement of the problem and a well-designed solution.

Now there is a big push on distance education—we'll just teach the courses over the Web. I am very skeptical, because I think proponents underestimate the importance of this motivational problem. That human presence, the fact that there is a person who really cares, is a real important part of getting students to stick with the difficult and often unpleasant task of learning.

A lot of us who are in education got there because we love to learn, and we tend to think that everybody loves to learn, but the reality is that learning is painful for lots of people. It's kind of like running laps during training for a marathon. It's rewarding when you finally run the race and do well, but it can be pretty boring and unpleasant during training. We need to structure things so that people who know that it will ultimately be good for them can stick with it. The biggest folly in computer education is saying, "Oh, we'll use computers to make learning fun! We'll turn it into video games; lots of kids will learn their math skills while shooting down rockets." I think this is completely unrealistic. The reality is that lots of learning is not fun.

Q: You have software for economics courses that allows students to participate in a market for textbooks. Isn't that fun?
A: That is fun, but if we just let them do the fun part, they would say, "I love trading; that was great." But then you say, "Describe how that helps you understand what just happened in the market for oil last week." You have them engaged, but you've got to have some of that hard, tedious work in advance to get them ready to trade, and then you have to make them struggle with some hard concepts and solve some problems afterward to make sure they take the lesson out of the experience.

Q: Tell us how your Web materials help a professor do that.
A: The Aplia system offers problem sets and other homework that can be machine-graded, and it supplies results back to the students and professor before the next class begins. The professor doesn't have to waste as much class time going over material that students were supposed to read before class because now she can measure whether they actually read it. We are trying to remove the drudgery for the professor so he or she can have a more active, open, free-flowing classroom that in the process gets us out of that deadening lecture mode.

Q: You have talked about changing the way academic publishing is done. How do you see Aplia doing that?
A: Right now the business model for academic publishing is to incur very big fixed costs to generate content. The issue in this kind of business is where do you put in a big markup over cost so you can recoup your upfront costs. Book publishers put it on paper and ink, and they give away a lot of things on Web sites. What that means is that the quality of material on websites is very low. Nobody is actually building a business around providing high-quality software for teaching.

We are trying to provide a different business model. We prepare high-quality content, and we are trying to charge for access and use of online materials. We think this creates new incentives. For example, book publishers end up competing with the used book market. They have to bring out a new edition every three years just to try to make the old edition obsolete. In our business model you don't have that same incentive to create artificial turnover. If we have content that works on our website, there is no reason for us to try to change it. We can also publish and sell smaller pieces of content, so that instead of signing up a few big authors to write a whole textbook, we can work with individual contributors to write single problem sets, experiments, analyses of what's in the newspapers, and so on. We can rely on a much bigger pool of contributors and facilitate more of a professor-to-professor exchange about what works.

Q: In one news article, a community college student complained that Aplia's Web content was an add-on cost to the textbook. Do you see that changing?
A: People are unhappy about how much textbooks cost, and professors worry about asking students to pay a little bit more for our product. They have seen a lot of low quality content on the Web. Once they are familiar with what we offer, it seems well worth the cost.

Economists talk about the cost of time. We say it has gone up because there are a lot more things to do with a given hour in a day. College students have a cost of at least $8 or $9 an hour because they can make that working at Starbucks. Even if they don't pay any fees, they are spending time worth $24 or $27 per week just going to three hours of lectures. Based on that, $28 for an entire term to use our materials is trivial if it makes time spent in the course a success.

Q: Will you expand to different products in economics or to other subjects?
A: Initially we'll stay in economics because staying with what you know is the first imperative. Ultimately, I think the problems in economics education are pervasive in many other parts of education. The kind of machine grading we do of things involving graphs, diagrams, equations, and math translates to many other scientific disciplines. It would be much harder to translate to something like teaching language or writing.

Q: Why did you choose a for-profit business model?
A: I thought about trying to emulate something like the open-source movement where lots of professors contribute and everybody who is working on material for their classes would share. But that doesn't generate enough revenue to cover the fixed costs to do things such as offering an easy, useful interface for students. For the most part open-source software is usable by geeks, but it isn't very user friendly for others. It is missing some of those basic production values in terms of design and ease of use that, unfortunately, are expensive. There is lots of material out there on the Internet that is free, but it never has been put together in a package that has been important enough to change how people teach. That's what we're trying to do.

Q: If this type of program is widely adopted, will it lead to evaluation of higher education?
A: Our goal is to facilitate the really exciting, challenging, hard-to-quantify learning experience that can take place in the classroom by making sure that students are prepared before they go into class. To really measure how well universities are doing and see the benefits of that kind of exciting classroom experience, we need much more sophisticated assessment tools to measure what students have learned. Our Aplia problem sets are very good at motivating student effort outside the classroom—to do some reading, think through things, figure out how to solve problems—but our problems are not rich enough to judge the depth of what a student should learn in a first-rate economics course. You would have to have things like an essay or an oral exam where a student could really grapple with a nebulous problem. Ultimately, I would like to see educators develop better assessment tools so we could see how well we are doing. The same principles that apply to students apply to us as teachers. If we can measure how well we are doing, then we can provide feedback and create incentives for teachers to do a better job.

A distant dream for me would be to offer online education to hundreds of millions of kids in China and India, and to combine this with the kind of high-quality assessment that I just described. The quality of life in those countries is going to depend a lot on how much the informed elites understand about economics, and this will depend on how well universities teach them economics. Right now, we are flying blind. We don't know how well universities are doing in this basic task.

For now, we not only have to work on pushing the bounds of what our software can do in terms of more complicated questions and responses that we can process, but we are also out there trying to explain how it works to professors. There are people in economics who think the experiments we and others are designing will change the teaching of economics, but it takes some effort to explain how this works to someone who is not familiar with it.

Q: Give us an example.
A: The most basic experiment is one that was developed by Vernon Smith, who received the Nobel Prize for this work. You give students roles as either buyers or sellers; tell them, for example, "You just took a chemistry course; you have a used textbook to sell." You tell each seller what it costs him or her to deliver a textbook to somebody else, and you tell each buyer what the book is worth to him or her if he or she buys one. You turn the students loose to buy and sell on the Internet, and they see what happens. When they get to experience a phenomenon like a shortage or the change in a price caused by a new tax, they learn it in a way that they don't if they just read about it.

Preparing Students via the Web

Meeting in Seoul? Be There in Five

Bridging Valleys with Technology and Heart

Up for the Challenge

New Courses on Financial Reporting

Don Quixote's Lessons for Leadership