How to do one hot encoding of unusual letters in matlab?
2 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
Sanjana Sankar
il 19 Lug 2019
Commentato: Sanjana Sankar
il 31 Lug 2019
I have a table of german characters. I would like to one hot encode them so that i can input it to a neural network. How should i go about?
The problem is some of the characters are not accepted by matlab as characters. For example, 'ä' 'ö' 'ü' 'ß'
Regardless, I would like to know how to one hot encode any character from a TABLE in matlab.
Thanks in advance!
3 Commenti
Risposta accettata
Walter Roberson
il 19 Lug 2019
Modificato: Walter Roberson
il 19 Lug 2019
https://machinelearningmastery.com/why-one-hot-encode-data-in-machine-learning/ describes One-Hot Encoding (a term I was not aware of)
In MATLAB, see https://www.mathworks.com/help/deeplearning/ref/vec2ind.html
You might want to first construct a list of permitted characters, and map the input into an offset in that list. That will potentially save you from wasting bits on characters such as œ that you are not using.
5 Commenti
Walter Roberson
il 30 Lug 2019
permitted = ['A' : 'Z', 'a' : 'z', 'ä', 'ö', 'ü', 'ß', ' ', '.' ] ;
[found, idx] = ismember(YourText, permitted);
assert(all(found), 'unpermitted character detected')
OneHot = ind2vec(idx) ;
Più risposte (1)
Guillaume
il 30 Lug 2019
file containing the characters that I want to one hot encode. Do you know how I can go about now
%phoneme_set: A cell array of phonemes to one-hot encode
assert(numel(phoneme_set) > 64, 'Cannot one-hot encode more than 64 phonemes with a 64-bit integer')
phoneme_set(:, 2) = num2cell(2 .^ uint64(0:size(phoneme_set, 1)-1))
3 Commenti
Guillaume
il 30 Lug 2019
number binary pattern (64 bits)
2^0 0000000000000000000000000000000000000000000000000000000000000001
2^1 0000000000000000000000000000000000000000000000000000000000000010
2^2 0000000000000000000000000000000000000000000000000000000000000100
...
2^63 1000000000000000000000000000000000000000000000000000000000000000
This is what one-hot encoding is. As I commented, this is useful for FPGAs and similar which operate at the bit level. On the generic processor of a computer, it's a complete waste of space but my answer does what you asked.
You could encode the one-hot encoded numbers that I generate as a vector of 0 and 1 (double) for even more waste of space:
phoneme_set(:, 3) = num2cell(fliplr(eye(size(phoneme_set, 1), 64)), 2)
Note that the binary pattern of the 0s in that encoding is 0000000000000000000000000000000000000000000000000000000000000000b and of the 1s is 0011111111110000000000000000000000000000000000000000000000000000b.
However, I suspect that what we have here is a XY problem. You have some unspecified problem doing something and you think you can solve it by using one-hot encoding (without really understanding what it means) so ask about one-hot encoding instead of your actual problem.
Vedere anche
Categorie
Scopri di più su Image Data Workflows in Help Center e File Exchange
Prodotti
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!