How to make Markov Chain Model with given sequence of data?

Markov Chain model considers only 1-step transition probabilities i.e. probability distribution of next state depends only on current state and not on previous state. I am using equation given below to make Markov Chain Model.

I have given sequence of data, i.e. Given observations of the system state are 10,10,2,10,10,5,9,2 so it has four different states i.e. 10,2,5,9. Now I have to make Markov Chain model using this data with the help above equations. Transition probability matrix will look like below