allwords

Parse a sentence or any string into distinct "words"
1,7K download
Aggiornato 7 apr 2010

Visualizza la licenza

Sentence parsing can be done one word at a time using strtok. However, sometimes it is useful to (efficiently) extract all words into a cell array in one function call. The function allwords.m does exactly this.

Spaces, white space (tabs), carriage returns, and punctuation characters are all valid separator characters by default. In this example, I had a period at the end, as well as multiple spaces between some words.

str = 'The quick brown fox jumped over the lazy dog.';
words = allwords(str)
words =
'The' 'quick' 'brown' 'fox' 'jumped' 'over' 'the' 'lazy' 'dog'

This utility can also work on any integer vector. The default separators for numeric vectors are [-inf inf NaN], but you can assign any separators you desire. Here, parse a string of integers, with only NaN elements as the separator.

str = [1 2 4 2 inf 3 3 5 nan 4 6 5];
words = allwords(str,nan);
words{1}
ans =
1 2 4 2 Inf 3 3 5

words{2}
ans =
4 6 5

Finally, allwords is efficient. For example, on a random numeric string of length 1e6, allwords parses it into over 90000 distinct "words" in less than 0.5 seconds.

str = round(rand(1,1000000)*10);
tic
words = allwords(str,[0 10]);
toc
Elapsed time is 0.455194 seconds.

There were over 90000 different words that were extracted

numel(words)
ans =
90310

The longest word had length 104.

max(cellfun(@numel,words))
ans =
104

Cita come

John D'Errico (2024). allwords (https://www.mathworks.com/matlabcentral/fileexchange/27184-allwords), MATLAB Central File Exchange. Recuperato .

Compatibilità della release di MATLAB
Creato con R2010a
Compatibile con qualsiasi release
Compatibilità della piattaforma
Windows macOS Linux
Categorie
Scopri di più su String Parsing in Help Center e MATLAB Answers
Riconoscimenti

Ispirato da: wordcount

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!
Versione Pubblicato Note della release
1.1.0.0

Speed enhancement for character strings, plus I added a reference to the wordcount function.