Reading text file word by word

62 views (last 30 days)
Paolo Binetti
Paolo Binetti on 13 Oct 2018
Edited: Paolo Binetti on 13 Oct 2018
Input is the attached text file, with one long word, a newline, then several equal-length short words separated by white spaces. I would like to read the first word in a variable, then the other ones in another variable one by one, rather than all into a single string or into a single huge cell array. I tried to do this with fscanf in several ways, but failed, and even got the impression that fscanf is not complying with https://fr.mathworks.com/help/matlab/ref/fscanf.html no clue about what I am doing wrong.
fileID = fopen('dataset_300_8.txt');
long_word = fscanf(fileID, '%[$ACGT]'); % is there another way to stop reading at newline?
short_word = ' ';
while ~isempty(short_word)
short_word = fscanf(fileID, '%s'); % does not work: shouldn't %s stop as it encounters a white space?
% short_word = fscanf(fileID, '%10s'); % this also does not work
% short_word processing code here
end
fclose(fileID);
  4 Comments
Paolo Binetti
Paolo Binetti on 13 Oct 2018
Thank you but reading all the content of the file in one shot is what I want to avoid. The "file" variable from Madhan's code has 127134 Bytes. What I want is a variable long_word of 2002 Bytes plus a variable short_word of 20 Bytes, into which I want to read each of the short words in the text file one by one. I read one, I use it for computations, then I don't need it anymore, so I read the next one, and so forth. I want to get the job done with 2022 Bytes, rather than 127134 Bytes. My file is just a small sample, but for bigger files memory would be an issue.

Sign in to comment.

Accepted Answer

jonas
jonas on 13 Oct 2018
Edited: jonas on 13 Oct 2018
Try this minor change
short_word = fscanf(fileID, '%s+');
Edit: After further testing, any character after the %s gives the same results as it causes the fscanf to stop reading (due to mismatch). Another iteration begins where the previous attempt failed, so at the next word.
  6 Comments

Sign in to comment.

More Answers (1)

Image Analyst
Image Analyst on 13 Oct 2018
"I would like to read the first word in a variable, then the other ones in another variable one by one, rather than all into a single string or into a single huge cell array." <--- this is a really bad idea. I'm sure Stephen will soon give you the reasons why.
Better solution is to use fileread() followed by strsplit() to make the single cell array.
str = fileread('dataset_300_8.txt'); % Read entire file.
ca = strsplit(str, ' '); % Put each word into a cell
  1 Comment
Paolo Binetti
Paolo Binetti on 13 Oct 2018
Really wonder why. OK, what if the combined size of the small words is 100 GB: guess you can't throw all of it in a single variable right? The file is on the hard disk, which can handle big chunks of data, but once you read it into a variable it goes into RAM, which cannot, right? So the idea was to read a piece of data at a time, process it, then read the next one, process it, etc.

Sign in to comment.

Products


Release

R2018b

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by