An Experiment in Computing the Expected Discounted Return in a Finite Markov Chain

An experiment with a certain transformation, leading to a new iterative method, was carried out with the intent of speeding the computation of the infinite horizon expected discounted return in a finite Markov chain. At first the method seemed preferable to Gauss-Seidel iteration on an example but after reordering the states (equations), we reversed the conclusion. Not only is reordering important, but it appears to be easy to do it well. The idea is to try to put most of the probability transition matrix in its lower triangular part.