Parse csv with complex numbers written by Python numpy
7 visualizzazioni (ultimi 30 giorni)
Mostra commenti meno recenti
I am not actually sure which Python features use this format (I think both numpy and pandas use it), or perhaps this is a part of a larger standard, but basically I have csvs with text that looks like
(-0.0053973628685668375-0.004476730131734169j),(0.005108157082444198-0.005597795916657765j),,,,,,,-298.0,-298.0,37293,-0.7617709422297042,0.7202575393833991,(0.001506298444580933-0.0035885955125266656j)
and I want to parse into a numeric array.
The real-valued scalar entries are easy (well I can do a str2double and it's not super fast but it's acceptable). The blanks are also not too bad because after a simple textscan with a comma delimeter I can find emptys and set to a desired value. But what the heck do I do with these rediculous complex number strings?
There's loopy solutions with regexp or finding the real and imag components but they are too slow when dealing with hundreds of thousands of entries. I could also do things like find entries containing a "j" and process them separately, but is there something better?
2 Commenti
Stephen23
il 16 Lug 2024
Please upload a sample data file by clicking the paperclip button.
Note that scipy has functions for saving and loading MAT files, so that might be a worthwhile alternative.
dpb
il 17 Lug 2024
The embarrassing problem with C i/o is it doesn't know anything about complex variables. Why Mathworks hasn't extended MATLAB to deal with them in 40 years is also beyond comprehension. For a scientific language purporting to be user friendly, this is and has always been a glaring oversight.
Ages ago I built a mex function that passed input off to a Fortran mex file with a format string and let the Fortran compiler i/o library that does know complex data types do the parsing. Unfortunately, I lost access to it when left last employer and they had already wiped the machine by the time I thought to ask about it and never got around to rebuilding it.
But
s='(-0.0053973628685668375-0.004476730131734169j),(0.005108157082444198-0.005597795916657765j)';
fmtC='(%f%fj)';
fmt=[fmtC ',' fmtC];
v=reshape(sscanf(s,fmt),[],2).'
v=complex(v(:,1),v(:,2))
For the full file, one has to go through the very painful with C process of building the format string to match the input record; tedious but doable if the file structure is regular.
Even the new(ish) readmatrix and friends with the import options won't let you set a variable to a complex type. Disgraceful imo. Back when I was still consulting I used to grouse regularly but joy has never ensued....which is why I wrote the mex routine.
Risposta accettata
Stephen23
il 16 Lug 2024
Modificato: Stephen23
il 16 Lug 2024
Forget about fiddling about with strings. Define the parentheses as delimiters and import as numeric:
format short G
M = readmatrix('test.txt', 'Delimiter',{'(',',',')'}, 'ConsecutiveDelimitersRule','join',...
'LeadingDelimitersRule','ignore', 'TrailingDelimitersRule','ignore')
Più risposte (0)
Vedere anche
Categorie
Scopri di più su String Parsing in Help Center e File Exchange
Community Treasure Hunt
Find the treasures in MATLAB Central and discover how the community can help you!
Start Hunting!