A cornerstone of applied probability, Markov chains can be used to help model how plants grow, chemicals react, and atoms diffuse-and applications are increasingly being found in such areas as engineering, computer science, economics, and education. To apply the techniques to real problems, however, it is necessary to understand how Markov chains can be solved numerically. In this book, the first to offer a systematic and detailed treatment of the numerical solution of Markov chains, William Stewart provides scientists on many levels with the power finally to put these techniques to use in the real world. His efforts make for essential reading in a rapidly growing field. Here Stewart explores all aspects of numerically computing solutions of Markov chains. He provides extensive background to both discrete-time and continuous-time Markov chains and examines many different numerical computing methods-direct, single- and multi-vector iterative, and projection methods.
More specifically, he considers recursive methods often used when the structure of the Markov chain is upper Hessenberg, iterative aggregation/disaggregation methods that are particularly appropriate when it is NCD (near