A Markov chain is a type of mathematical model that represents a system moving between different states, where the next state depends only on the current state, not on how you got there. This memoryless property is called the Markov property.
cumulative_distribution = cumsum(this_step_distribution);
This code converts the list of transition probabilities into a cumulative probability distribution. This is to perform random sampling to determine which state to move to next.
For example, let’s say there are 2 states (as in the provided code), and the current state is 2:
this_step_distribution = transition_probabilities(2, :) = [0.8, 0.2];
cumulative_distribution = [0.8, 1.0];
If a random number “r” is between 0.0 and 0.8, we choose state 1.
If “r” is between 0.8 and 1.0, we choose state 2.
This makes it possible to randomly select the next state according to the given probabilities.
Now, to answer the second question:
r = rand();
chain(i) = find(cumulative_distribution > r, 1);
The above snippet generates a random number between 0 and 1 and finds which cumulative interval the random number falls into (this determines the next state).
This process is called inverse transform sampling, and it ensures that the next state is chosen based on the given probabilities.
I hope it addresses what you wanted to ask!
I am also attaching the documentation links of functions used for reference:
