Import data does not properly

Dear all,

I moved from the R2024 to R2026 version and now when I import data it shows a very curious data representations for e.g. two column

197 0000 0000->0

196 8000 8000->1

196 6000 6000->0

196 4000 4000->0

rather than

197,0000 0,494778

196,8000 1,07849

196,6000 0,965887

196,4000 0,839238

I did 'Set type' to Number, but no real effect.

How can I manage it?

Thanks

Best

Marco

[Marco Sette]

Does not work properly..

[Steven Lord]

Please attach a small segment of a file formatted the same way the file that didn't import as you expected is formatted. Also please describe in more detail how exactly you imported the file (posting the commands or showing pictures of the import dialog if you imported it that way.)

Without those details, it's likely going to be extremely difficult or impossible to offer any concrete advice.

[dpb]

It looks as though the file is formatted for the decimal point character to be a comma, not the default US period. Set the "Decimal Separator" in the import tool settings.

The more robust way is to use readmatrix that allows for control with the named parameter or by using an import options struct.

[Andreas Goser]

The user contacted Technical Support and asked the engineer to also close the loop with the community here what helped.

[Marco Sette]

Hi, it is not a comma as separator.

So I will continue with the Technical Support?

[dpb]

@Andreas Goser indicated earlier you had so would seem they should provide a solution, yes.

But, it might be useful for the Community if you were to follow up with @Steven Lord's suggestion to attach a representative file that fails and the code/method used to try to read it.

If the file doesn't use the comma for the decimal separator, something else is happening but only seeing the file structure itself would help.

Have you tried

data=readmatrix('yourfullyqualifiedfilename');

to see if it can read it successfully?

[Image Analyst]

Yes, if you have paid for technical support, calling them and talking to a live person will almost always be the fastest way to a solution. And since you're not attaching your data or code (despite being asked), we'll just be throwing out guesses here. And I cannot even begin to guess how you think 0000->0 should be turned into "0,494778". Same for the other strings in the second column - no idea where those desired numbers come from.

[Marco Sette]

Hi,

thanks for the suggestion. I did not find a place to attach the original file but its a simple text file:

TITLE KPi25mMpH6.5

DATA TYPE

ORIGIN JASCO

OWNER sette

DATE 26/05/12

TIME 11:55:01

SPECTROMETER/DATA SYSTEM

LOCALE 1040

RESOLUTION

DELTAX -0,2

XUNITS NANOMETERS

YUNITS CD [mdeg]

Y2UNITS HT [V]

FIRSTX 260,0000

LASTX 190,0000

NPOINTS 351

FIRSTY 0,62689

MAXY 1,47344

MINY 0,04477

XYDATA

260,0000 0,626894 246,062

259,8000 0,564078 246,116

259,6000 0,492014 246,225

259,4000 0,57834 246,321

259,2000 0,543424 246,424

259,0000 0,587415 246,533

....... and so on

and the output is

.260 0000--->0 626894--->246 62

259 8000--->0 564078--->246 116

259 6000--->0 492014--->246 225

259 4000--->0 57834--->246 321

259 2000--->0 543424--->246 424

Best

Marco

[Andreas Goser]

I have not considered this leads to this chain of comments :-D Let me try to clarify:

• It is totally fine users leverage multiple channels
• I am always a fan however of closing the loop with the community so it is known what the outcome of a support request is. At times it needs to be short as we cannot disclose details of a customer application. It is also a bit of coincidence a MathWorks staffer notices MATLAB Answers questions also being asked to support teams.

For this issue:

• The theories and hints in this thread are actually very promising! That was partly a reason I informed the support engineer about this post
• The user provided more technical information with support, so I suppose we will resolve it and the community does not have to help. But I surley do not want to discourage good intention.

Answer 1

[Marco Sette]

readmatrix also gives some problems

>> data=readmatrix('/Users/marcosette/Desktop/120526PAO1/buffer.txt');

>> data

data =

260 NaN NaN 62

259 NaN NaN 116

259 NaN NaN 225

259 NaN NaN 321

259 NaN NaN 424

259 NaN NaN 533

258 NaN NaN 695

258 NaN NaN 853

258 NaN NaN 975

258 NaN NaN 123

258 NaN NaN 246

257 NaN NaN 418

257 NaN NaN 516

257 NaN NaN 584

257 NaN NaN 733

257 NaN NaN 858

256 NaN NaN 963

256 NaN NaN 13

256 NaN NaN 291

256 NaN NaN 41

256 NaN NaN 517

255 NaN NaN 686

255 NaN NaN 865

255 NaN NaN 14

255 NaN NaN 147

255 NaN NaN 303

254 NaN NaN 443

254 NaN NaN 627

254 NaN NaN 782

254 NaN NaN 921

254 NaN NaN 8

253 NaN NaN 281

Answer 2

[Marco Sette]

Solved!

Set

Column delimiter = TAB

Delimiter options = comma

Thanks :)

Marco

Answer 3

[dpb]

There's a paperclip icon with which you can attach a file. I pasted into a local file and uploaded a copy here...

The "simple" text file is far more than just numeric data, however. And, as I suspected, it IS written with the decimal separator being the comma, not a period; that's why the line

260,0000	0,626894	246,062

was interpreted as

260, 0,    0, 626894,	246, 062

because there are three sets of two numbers in the line as it will be interpreted when a comma is a delimiter, not a decimal point.

L=readlines('marco.txt');          % read full file as string array
iXY=find(startsWith(L,'XYDATA'));  % find the XY Data section start
data=readmatrix('marco.txt','NumHeaderLines',iXY,'DecimalSeparator',',')

data = 6×3

260.0000 0.6269 246.0620 259.8000 0.5641 246.1160 259.6000 0.4920 246.2250 259.4000 0.5783 246.3210 259.2000 0.5434 246.4240 259.0000 0.5874 246.5330

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

clear l iXY                     % get rid of uneeded variables

An alternate way w/o the extra copy in memory as text would be

fid=fopen('marco.txt','r');     % open, get file handle
iXY=0;                          % initialize line counter
while ~feof(fid)                % loop until run our of data
  iXY=iXY+1;                    % increment counter
  l=fgetl(fid);                 % read a line
  if startsWith(l,'XYDATA'); break; end  % found the XY Data section start, quit
end
fid=fclose(fid);                % close the file
data=readmatrix('marco.txt','NumHeaderLines',iXY,'DecimalSeparator',',')

data = 6×3

260.0000 0.6269 246.0620 259.8000 0.5641 246.1160 259.6000 0.4920 246.2250 259.4000 0.5783 246.3210 259.2000 0.5434 246.4240 259.0000 0.5874 246.5330

<mw-icon class=""></mw-icon>

<mw-icon class=""></mw-icon>

clear l fid iXY                 % get rid of uneeded variables

[Marco Sette]

Thanks a lot.

This is very clear for me that I'm not expert in Matlab. This is also a good starting point to learn basic programming with your beautiful software.

A final question: how can split the final data matrix in order to have 3 separate variables, like x, y, z, for the 3 columns?

Thanks

Marco

[dpb]

@Marco, thanks for the response; as for not being expert, we all had to start from somewhere first...for the Q? about three variables instead of an array, there aren't as many tools in the bag; the venerable one textread has been relegated to the "not recommended" category in favor of textscan although it, other than low-level i/o via fscanf is the only way to return individually named variables since textscan returns a cell array rather than multiple variables. The issues there are that neither of those options has been extended to handle the 'DecimalSeparator' parameter so one has to convert the comma first.

One can, of course, simply assign names to the columns although that is laborious. Creating a struct of the array with field names to match the desired is another generic coding practice. Or, use a table that can have variables with their names.

folder='';                          % use your file location
fqn=fullfile(folder,'marco.txt');   % build fully qualified name
fid=fopen(fqn,'r');                 % open, get file handle
iXY=0;                              % initialize line counter
while ~feof(fid)                    % loop until run our of data
  iXY=iXY+1;                        % increment counter
  l=fgetl(fid);                     % read a line
  if startsWith(l,'XYDATA'); break; end  % found the XY Data section start, quit
end
fid=fclose(fid);                % close the file
data=readtable(fqn,'NumHeaderLines',iXY,'DecimalSeparator',',');
data.Properties.VariableNames={'X','Y','Z'}

data = 6×3 table
      X         Y         Z   
    _____    _______    ______

      260    0.62689    246.06
    259.8    0.56408    246.12
    259.6    0.49201    246.22
    259.4    0.57834    246.32
    259.2    0.54342    246.42
      259    0.58742    246.53

clear l fid iXY                 % get rid of uneeded variables

also illustrating good practice on builing file names. Another very useful paradigm in that regards is to use

d=dir('*.txt');                 % some appropriate wild card expression here
for i=1:numel(d)
  fqn=fullfile(d(i).folder,d(i).name);
  ...
end