How do I transform big java objects to Matlab

I'm facing following situation:

- I've retrieved a sizable amount of data (say tens of mb) from a MongoDb, and I've used the MongoDb Java API to load this data to Matlab. This results in a big Java object with a lot of nested fields.

- I would like to transform this Java object to a Matlab datamodel. This requires casting and vectorization, as the original Java object is not vectorized and contains Java objects.

- If I iterate over all fields (including nested fields), I can use the .get('fieldname') method to retrieve one particular value. However, it is not uncommon that I have to do this 100,000 times, and performance is not workable for this. The underlying performance bottleneck is likely related to following thread: http://www.mathworks.com/matlabcentral/newsreader/view_thread/246361.

The solution that I'm considering right now:

- write a java method that is able to do the complete transformation to a Matlab object. This java method can construct MWArray, MWArrayStruct, MWArrayCell, etc. by making use of the Matlab JA builder. This would handle the vectorization and casting on java side, and would simply pass back a Matlab recognized object.

- However, the situation feels overly complex due to its dependencies: JA Builder requires the Matlab compiler and the subsequent compiled java class requires installation of the MCR, even though I want to call my transformation method from within Matlab.

Questions:

- Does this seem the right way of approaching this problem? Any other ideas that might work?

Answer 1

[Roderick]

I had tried the code from that link before, it wasn't particularly good in terms of performance for my use case.

However, I've found a solution which seems to work quite well. It skips over the JABuilder logic in above explanation. As such:

- it uses the MongoDb Java API to load data to Matlab in terms of a Java object - it transforms the Java object into big vectors - the big vectors are passed to Matlab and assigned to variables

[Hedi]

thank you for your answer but the problem still persists (I just followed the instructions on the link)

Answer 2

[Max]

https://github.com/christianpanton/matlab-json