How to produce a scalogram using a massive amount of data

Hey,

I'm attempting to generate a scalogram plot using cwt from a minimum of 432,000,000 data points to 2,000,000,000 or so. Is this possible, or is this data simply too large. I'm working with a sampled rate of 20kHz, and data is obtained over the course of hours, a minimum of 6 hours being necesarry for graphing.

I was thinking maybe finding the magnitude and frequency averages for every second of data (reducing 1.7 billion to ~84 thousand), and using that data to produce a scalogram, however I'm not exactly sure how to do that use the cwt function (or if that approach is recommend).

I'm unsure of how to approach this problem.

[Max Murphy]

Do you have the Signal Processing Toolbox or just Wavelet toolbox?

[Max Murphy]

Also, based on your specific application I would suggest considering whether the full bandwidth data is meaningful for your purposes. If not, consider using

data_dec = decimate(data,factor);

I commonly work with data sampled between 20kHz to 30kHz, and for spectral analysis it is often only meaningful to look at a tiny sliver of the possible spectrum (e.g. in my case, from 0.1 to ~200 Hz). Using that example, it means I can easily use decimate to reduce my number of samples by a factor of 20 or 30, which could also help in your case if it's applicable.

You can get an approximation for what you can decimate to by taking the highest frequency source you think will be present in your data and then doubling it. In my case I typically pad that fairly generously; so although I could get away with 400 Hz if I think that my maximum frequency source is 200 Hz, I will stay at 1 kHz just to be on the safe side.

Again, don't know what your application may be, but if you can do it decimate is your friend. It also has a built-in anti-aliasing chebyshev lowpass filter so you don't even have to worry about doing that part.

You may also look into the Wavelet Analyzer application, using the Continuous Wavelet 1-D (Using FFT) option and applying it to a much shorter (maybe a few minutes?) selection of your data to get a feel for what some good wavelet parameters might be.

[Nadir Altinbas]

quantum computing shall be used