# Markov Chain Models

A discrete state-space Markov process, or *Markov chain*, is represented by a directed graph and
described by a right-stochastic transition matrix *P*. The distribution of states at time
*t* + 1 is the distribution of
states at time *t* multiplied by
*P*. The structure of *P* determines the evolutionary trajectory of
the chain, including asymptotics.

For an overview of the Markov chain analysis tools, see Markov Chain Modeling.

## Functions

## Topics

**Discrete-Time Markov Chains**Markov chains are discrete-state Markov processes described by a right-stochastic transition matrix and represented by a directed graph.

**Markov Chain Modeling**The

`dtmc`

class provides basic tools for modeling and analysis of discrete-time Markov chains. The class supports chains with a finite number of states that evolve in discrete time with a time-homogeneous transition structure.**Create and Modify Markov Chain Model Objects**Create a Markov chain model object from a state transition matrix of probabilities or observed counts, and create a random Markov chain with a specified structure.

**Visualize Markov Chain Structure and Evolution**Visualize the structure and evolution of a Markov chain model by using

`dtmc`

plotting functions.**Work with State Transitions**This example shows how to work with transition data from an empirical array of state counts, and create a discrete-time Markov chain (

`dtmc`

) model characterizing state transitions.**Determine Asymptotic Behavior of Markov Chain**Compute the stationary distribution of a Markov chain, estimate its mixing time, and determine whether the chain is ergodic and reducible.

**Compare Markov Chain Mixing Times**Compare the estimated mixing times of several Markov chains with different structures.

**Identify Classes in Markov Chain**Programmatically and visually identify classes in a Markov chain.

**Simulate Random Walks Through Markov Chain**Generate and visualize random walks through a Markov chain.

**Compute State Distribution of Markov Chain at Each Time Step**Compute and visualize state redistributions, which show the evolution of the deterministic state distributions over time from an initial distribution.