Merging multiple data files to a singel big file/ usage for Ai applications

Hello together,

I have never used matlab much, which is why I am currently stuck.

I am currently working on my Thesis and have to analyse some Data from am brandnew automotive Test Bench at my University.

I have attached a picture of the data, which the test bench generates. Oure current setup is that we test 2 asynchronous electrical machines against each other. This generates right now 4 different files. 1 for the first machine, 1 for the second machine, 1 for the frequency inverter an dthe last one for the environmental station data. The Data has a sampling frequency of 10 Hz and is already generating big data files with a lot of useless parameters (we are testing everything out, unfortunately there is no real documentation on the machine parameters). The low sampling rate of 10 Hz allows us to measure various points in one recording. This gives us at the moment 4 files which have all the measured points in it.

In the future we ar going to measure at a higher sampling frequency like 100 Hz or 1 KHz. this leads to the problem, that we have to measure every Point separetly (the recording files get to big and the system shuts down the measurement). Therfore we are going to get probabbly 1 recording file for each machine/ frequency inverter/ environmental station per measuring point. This means a lot of files which are also saved in one folder. We have not tried this one out yet and have to see how the System reacts or if there is another solution.

At the End the Data is supposed to be used in AI applications to predict the effficiency and temperature of electrical machines. Therfore we have to preprocess the Data correctly and i think this can be done in one big file better than in hundreds of small files.

My idea:

• to merge all the 4 different files of the machines/ frequency inverter/ environmental station first and than attaching alle the files from the measuring points in correct order to get a big file

Problem:

• I always used the import tool of Matlab to import my data
• unfortunately the import tool does not recognize the time stamp (yellow marked area in the Screenshot_example_data_file_pre_processed_import_tool) idont know what the Problem is
• How am i going to attache all the files of the measuring point sif i dont have the time stamp inthe Data?

I hope that i was abel to explain my problem good enough that most people are going to understand it.

I am looking forward to your answers and ideas.

Thank You.

[dpb]

Need to attach at least a small section of each file type (better would be to have a couple or more so have something to patch together), not images. Nothing can be done with the image other than look at it.

Then, there doesn't seem to be four file types in the directory; what are they and is the sampling synchronous so the times actually match identically?

Which order do you want the flile catenated in horizontally, or does it matter?

Is the end result to have the data in MATLAB or just the file for some other tool to use? And what does it need for timestamp format? In MATLAB, turning it into a timetable would probably be the way to go except for the very large size is going to run up against memory limits (see recent other thread here) very quickly so it simply won't be possible to build a single MATLAB data structure/array (nor will any other tool be able to handle it all in memory at once).

Need a clear definition of what the output file format is to be...

Answer 1

[Jeremy Hughes]

I can only answer part of this question. If you try to post as separate questions, you may get better answers.

The file appears to be semicolon delimited, and MATLAB is figuring that out, but that separates the date and the time into different variables. The second variable doesn't contain a date, only a timestamp. You will have better luck importing those as duration instead and adding them

Once you import the data, you can add the time to the dates with this line of code:

T.Date = T.Date + T.Time

I don't have any other thoughts on the rest of the questions.