The analysis of structured countable stage decision processes, initiated in Porteus [10], is continued. The standard models of positive and negative dynamic programming are given in this context, thus extending these results to criteria other than the usual expected sum of rewards, such as expected utility criteria , certain stochastic games, risk sensitive Markov decision processes, and maximin criteria. For positive problems, (what are called) unimprovable strategies are optimal and the optimal value sequence is the least solution of the optimality equations exceeding an obvious lower bound. For negative problems, conserving strategies are optimal and if one strategy is a one-step improvement on another, then it nets a greater value. (This rules out cycling in the strategy iteration procedure.) Also, transfinite methods are used to prove that the optimal value sequence is the greatest solution of the optimality equations less than an obvious upper bound. We indicate how all these results can be extended to analogues of the essentially positive, essentially negative, and convergent cases.
-
Faculty
- Academic Areas
- Awards & Honors
- Seminars
-
Conferences
- Accounting Summer Camp
- California Econometrics Conference
- California Quantitative Marketing PhD Conference
- California School Conference
- China India Insights Conference
- Homo economicus, Evolving
-
Initiative on Business and Environmental Sustainability
- Political Economics (2023–24)
- Scaling Geologic Storage of CO2 (2023–24)
- A Resilient Pacific: Building Connections, Envisioning Solutions
- Adaptation and Innovation
- Changing Climate
- Civil Society
- Climate Impact Summit
- Climate Science
- Corporate Carbon Disclosures
- Earth’s Seafloor
- Environmental Justice
- Finance
- Marketing
- Operations and Information Technology
- Organizations
- Sustainability Reporting and Control
- Taking the Pulse of the Planet
- Urban Infrastructure
- Watershed Restoration
- Junior Faculty Workshop on Financial Regulation and Banking
- Ken Singleton Celebration
- Marketing Camp
- Quantitative Marketing PhD Alumni Conference
- Rising Scholars Conference
- Theory and Inference in Accounting Research
- Voices
- Publications
- Books
- Working Papers
- Case Studies
- Postdoctoral Scholars
-
Research Labs & Initiatives
- Cities, Housing & Society Lab
- Corporate Governance Research Initiative
- Corporations and Society Initiative
- Golub Capital Social Impact Lab
- Policy and Innovation Initiative
- Rapid Decarbonization Initiative
- Stanford Latino Entrepreneurship Initiative
- Value Chain Innovation Initiative
- Venture Capital Initiative
- Behavioral Lab
- Data, Analytics & Research Computing