## Essentials of Stochastic Processes

### 1 Markov Chains

• 1.1 Definitions and Examples
• 1.2 Multistep Transition Probabilities
• 1.3 Classification of States
• 1.4 Stationary Distributions
• 1.5 Limit Behavior
• 1.6 Special Examples
• 1.7 Proofs of the Main Theorems*
• 1.8 Exit Distributions
• 1.9 Exit Times
• 1.10 Infinite State Spaces*
• 1.11 Chapter Summary
• 1.12 Exercises

### 2 Poisson Processes

• 2.1 Exponential Distribution
• 2.2 Defining the Poisson Process
• 2.3 Compound Poisson Processes
• 2.4 Transformations
• 2.5 Chapter Summary
• 2.6 Exercises

### 3 Renewal Processes

• 3.1 Laws of Large Numbers
• 3.2 Applications to Queueing Theory
• 3.3 Age and Residual Life*
• 3.4 Chapter Summary
• 3.5 Exercises

### 4 Continuous Time Markov Chains

• 4.1 Definitions and Examples
• 4.2 Computing the Transition Probability
• 4.3 Limiting Behavior
• 4.4 Exit Distributions and Hitting Times
• 4.5 Markovian Queues
• 4.6 Queueing Networks*
• 4.7 Chapter Summary
• 4.8 Exercises

### 5 Martingales

• 5.1 Conditional Expectation
• 5.2 Examples, Basic Properties
• 5.3 Gambling Strategies, Stopping Times
• 5.4 Applications
• 5.5 Convergence
• 5.6 Exercises

### 6 Mathematical Finance

• 6.1 Two Simple Examples
• 6.2 Binomial Model
• 6.3 Concrete Examples
• 6.4 Capital Asset Pricing Model
• 6.5 American Options
• 6.6 Black-Scholes formula
• 6.7 Calls and Puts
• 6.8 Exercises

### A Review of Probability

• A.1 Probabilities, Independence
• A.2 Random Variables, Distributions
• A.3 Expected Value, Moments