These notes were written based on a number of courses I taught over the years in the U.S.,
Greece and the U.K. They form the core material for an undergraduate course on Markov chains in discrete time. There are, of course, dozens of good books on the topic. The only new thing here is that I give emphasis to probabilistic methods as soon as possible.
Also, I introduce stationarity before even talking about state classification. I tried to make everything as rigorous as possible while maintaining each step as accessible as possible. The notes should be readable by someone who has taken a course in introductory (non-measuretheoretic) probability.
The first part is about Markov chains and some applications. The second one is specifically for simple random walks. Of course, one can argue that random walk calculations should be done before the student is exposed to the Markov chain theory. I have tried both and prefer the current ordering. At the end, I have a little mathematical appendix.
There notes are still incomplete. I plan to add a few more sections:
– On algorithms and simulation
– On criteria for positive recurrence
– On doing stuff with matrices
– On finance applications
– On a few more delicate computations for simple random walks
– Reshape the appendix
– Add more examples
These are things to come...
A few starred sections should be considered “advanced” and can be omitted at first reading.
I tried to put all terms in blue small capitals whenever they are first encountered. Also,
“important” formulae are placed inside a coloured