Calculating the entropy of a discrete memoryless source?
4 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
How could I calculate the entropy of a discrete memoryless source S={a,b,c} in unit of bits? What would be the matlab script for calculating this?
0 Commenti
Risposte (3)
Vinod Sudheesh
il 27 Mag 2015
Hello Daniel,
You could write the script for the same as follows
1) Create a function, say "esingle" that computes the "entropy" of a symbol in the source alphabet. The definition of the function "esingle" could be as follows
function Y=esingle(p)
if(p==0)
Y=0;
return;
else
Y=p*log2(1/p);
end
end
2) Now invoke this function "esingle" using the syntax below to calculate the "entropy" of the source alphabet "a".
a=[0 1 1];
E=sum(arrayfun(@esingle,a/sum(a)));
Hope this helps!
Thanks Vinod
Walter Roberson
il 27 Mag 2015
A source S={a,b,c} is a source with three alphabetic symbols of no particular numeric value. If we know the generating equations for S then we analyze it analytically to determine the entropy. Otherwise the best we can do is estimate the entropy from a stream of the generated symbols. If we have assigned definite and distinct numeric values to "a", "b", and "c" and if "s" is a finite stream of those exact values, then
counts = histc(s, sort([a,b,c]));
probs = counts ./ length(s);
nzprobs = probs(probs>0);
entropy = sum(nzprobs .* log2(1./nzprobs));
Note: this code will work even if the assigned values are characters and s is a string.
0 Commenti
rakesh ranjan
il 17 Nov 2020
function Y=esingle(p)
if(p==0)
Y=0;
return;
else
Y=p*log2(1/p);
end
end
0 Commenti
Vedere anche
Categorie
Scopri di più su QAM in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!