This paper explores the economics of investing in gradual process improvement. We seek to understand better the economics of the well known Just-in-Time and Total Quality Control philosophies, both of which promote gradual technological improvement as a means of achieving increased productivity and manufacturing competitiveness. In addition, there is empirical evidence that the cumulative effect of minor technical changes on productivity is often at least as important as the effect of major technological innovations. We formulate a general model of dynamic process improvement as a Markov decision process, analyze it, and apply it to the problem of setup reduction and process quality improvement. Instead of a deterministic model with a one-time investment opportunity for a large predictable technological advance, this paper builds a stochastic model with opportunities for many smaller investments over time, and in which potential process improvements may be unpredictable.* *See PDF for complete abstract text.