Note that sigma(TF) is an extension of bode(TF) for MIMO systems. So sigma(TF) is same as the gain plot of bode(TF) for SISO systems. To know more about sigma() and it's usage, refer to the following documentation:
But I would like to clarify your doubt on why sigma(H), for the case of an unstable system, gives finite values.
Let us consider an unstable system with the following transfer function:
H = tf(,[1,-5]);
Now plot in MATLAB bode(H); (whose magnitude plot is same as sigma(H)).
As you would see, although this is an unstable system, the magnitudes in the magnitude plot are finite. This is because the output magnitude is:
1/ ((j*w)-5)) (where j = sqrt(-1) and w is frequency).
This is equal to 1/sqrt(25+w^2)
So, you see, the denominator is never zero for any value of the frequency 'w'. It's always finite for all values of 'w'. Hence, the system is unstable doesn't mean the gains should be infinity. Rather, it means that the output of the system explodes over time.
Using the analogy, you can figure out why the sigma(H) gave finite values for your system.
I hope you will find this explanation helpful.