Changing contents of Cell Array mex files

4 visualizzazioni (ultimi 30 giorni)
I have a mex file where I have a cell array that contains some structure arrays. I want to grow those structure arrays as needed while processing a large dataset, and then finally return the cell array to matlab.
This has been painfully difficult. After many many matlab crashes, I finally have it working. I think the root cause of my trouble has something to do with how cell arrays manage memory. I see this mentioned in the documentation for mxGetCell:
..this might explain why when I tried to re-size my structure arrays (obtained from the mxGetCell call) using the method described in this link, things crashed horribly. However, if one looks at the online documentation for mxSetCell, you see this rather contradictory advice:
so which is it? What I have working is a rather brute-force solution. When I want to grow the structure array, I allocate it in the new size with a call to mxCreateStructArray, and then I manually copy every element and field with calls to mxGetFieldByNumber and mxSetFieldByNumber. I then call mxSetCell and I do not call mxDestroyArray on the original structure array, contrary to the advice in the mxSetCell documentation.
I guess this works, but it's inefficient and I'd rather use the more-efficient method for growing structure arrays utilizing reAlloc....but it seems like this might be incompatible with how cell arrays manage memory. In any case, it would be nice to understand what is going on so I can avoid problems in the future, and also some clarification on the inconsistencies in the documentation.
  2 Commenti
James Tursa
James Tursa il 26 Giu 2020
Modificato: James Tursa il 26 Giu 2020
Are you passing a cell array into the mex routine, adding on to it in the mex routine, and then passing that expanded result back to the caller? And you want to know how to do that correctly and efficiently?
Or are you creating the cell array from scratch inside the mex routine and expanding it inside the mex routine?
Can we see your actual code so that we can comment on it? If you explain what you are trying to do, we can show you the correct official way to do it. There is a fast unofficial way to do it as well, which may or may not apply in your case depending on what you are trying to do.
Todd Rearick
Todd Rearick il 26 Giu 2020
I'm creating a cell array inside the mex function. As I process data I first add struct arrays to the cell elements...and as I process more data I wind up having to grow the structure arrays so I allocate new ones and replace them inside the cell array.
I can post code, but it will take me some time to strip out stuff I can't post. I can likely create some simple example that demonstrates my problem more succinctly.

Accedi per commentare.

Risposta accettata

James Tursa
James Tursa il 26 Giu 2020
Modificato: James Tursa il 26 Giu 2020
When you mxDestroyArray a cell array or struct array, it does a deep destroy. Meaning all of the cell array or struct array elements are deep destroyed first, then the cell array or struct array itself is destroyed. That is why you should not call mxDestroyArray on the result of a mxGetCell or mxGetField call without cleaning things up, because if you do then you have invalidated the memory that is contained in the cell array or struct array and when the cell array or struct array eventually gets destroyed it will try to free invalid memory and bomb.
Regarding the documentation:
Do not call mxDestroyArray on an mxArray returned by the mxGetCell function
This refers to cell arrays that come from one of the prhs[ ] input variables, where doing so can screw up the workspace if it is a shared data copy of another variable. This sentence does not apply to cell arrays that you create and populate inside the mex routine where you know it is not a shared data copy of another variable, as long as you NULL out the corresponding spot in the cell array. Same thing is true for struct arrays btw.
To free existing memory, call mxDestroyArray on the pointer returned by mxGetCell before you call mxSetCell
This refers to cell array content that you originally created inside the mex routine, i.e. not part of a prhs[ ] variable. As long as you created it inside the mex routine, you can destroy it safely as well. E.g., start with this:
mxArray *mycell, *myvariable, *var;
mycell = mxCreateCellMatrix(1,1); /* 1x1 cell array */
myvariable = mxCreateDoubleScalar(5.0); /* Temporary, on garbage collection list */
mxSetCell( mycell, 0, myvariable ); /* myvariable is now Sub-Element of mycell */
The pointer contained in myvariable is put directly into mycell. Not a copy of the variable, but the actual pointer itself. And the type of the variable is changed from Temporary (on the garbage collection list) to Sub-Element (NOT on the garbage collection list). It's disposition is now entirely dependent on the disposition of the cell array it is part of.
What you cannot do is this:
var = mxGetCell( mycell, 0 ); /* this part is ok */
mxDestroyArray( var ); /* you have just made mycell invalid */
mxDestroyArray( mycell ); /* this will bomb MATLAB */
That last line will bomb MATLAB because mycell still contains the pointer of the variable you previously destroyed, so when MATLAB tries to subsequently destroy that element it will access invalid memory and bomb. That is, destroying var is actually OK since you originally created var in the mex routine ... but not cleaning up mycell properly will lead to a crash.
The correct way to extract and destroy a variable you originally created in a mex routine is:
var = mxGetCell( mycell, 0 ); /* this part is ok */
mxSetCell( mycell, 0, NULL ); /* NULL out the pointer that we just extracted */
/* See NOTE below */
mxDestroyArray( var ); /* OK since we originally created this inside the mex routine */
mxDestroyArray( mycell ); /* this will work OK */
NOTE: The var you extracted from mxGetCell( ) is actually not a Temporary variable anymore, it is a Sub-Element since it came from a cell array. Meaning, it is NOT on the garbage collection list and will NOT get automatically destroyed when the mex routine exits. This is true even though you originally created it inside the mex routine (as soon as you called mxSetCell( ) with this as input the type changed and it was removed from the garbage collection list). At this point you must do one of two things to avoid a memory leak. Either mxDestroyArray( var ) downstream in your code, or attach var to a cell or struct array. There are no API functions to put an mxArray back on the garbage collection list once it has been removed.
  12 Commenti
James Tursa
James Tursa il 30 Giu 2020
Modificato: James Tursa il 30 Giu 2020
This seems to work without crashing. It seems you simply have bad loop indexes and weren't filling any extra data. This is because sz points at the actual dimensions of the struct array, not a copy of the dimensions. So calling mxSetM( ) will immediately change the values that sz points to. See my NOTES.
#include "mex.h"
void
mexFunction(int nlhs,mxArray *plhs[],int nrhs,const mxArray
*prhs[])
{
mxArray *cell_array = NULL;
const char * fieldnames[] = {"field1", "field2","field3","field4"};
int n_fields = sizeof(fieldnames)/sizeof(*fieldnames);
// if (nlhs > 0) // NOTE: Don't need this
{
// create cell array to return
mwSize pi_dims[2];
pi_dims[0]=1;
pi_dims[1]=1;
cell_array = mxCreateCellArray(2,pi_dims);
int n_rows = 10000;
// create structure matrix with n_rows
mxArray *data_struct = mxCreateStructMatrix(n_rows, 1, n_fields, fieldnames);
// put something in it
for (int i=0;i<n_rows;i++)
{
for (int fn=0;fn<n_fields;fn++)
mxSetField(data_struct,i,fieldnames[fn],mxCreateDoubleScalar(i));
}
// put it in the cell array
mxSetCell(cell_array,0,data_struct);
// now we grow it w/ realloc
// get the structure from the cell array
mxArray *old_data_struct;
old_data_struct = mxGetCell(cell_array,0);
const mwSize *sz = mxGetDimensions(old_data_struct);
// double the size
mwSize new_rows = sz[0]+n_rows;
// reallocate the struct array to be larger
mwSize new_size = new_rows * n_fields * sizeof(mxArray *);
mxArray **re_allocated_data_struct_memory = (mxArray **)mxRealloc(mxGetData(old_data_struct), new_size); // NOTE: Changed to (mxArray **)
mxSetData(old_data_struct, re_allocated_data_struct_memory);
mxSetM(old_data_struct, new_rows); // NOTE: This changes the value of sz[0] to new_rows !!!
// put something in the expanded part
// for (int i=sz[0];i<new_rows;i++) // NOTE: This loop DOES NOTHING
for (int i=n_rows;i<new_rows;i++) // NOTE: Changed sz[0] to n_rows
{
for (int fn=0;fn<n_fields;fn++)
mxSetField(old_data_struct,i,fieldnames[fn],mxCreateDoubleScalar(i));
}
plhs[0] = cell_array;
}
}
Todd Rearick
Todd Rearick il 30 Giu 2020
THATS what's wrong! Yes..it was something stupid.
Thank you!

Accedi per commentare.

Più risposte (0)

Categorie

Scopri di più su MATLAB Algorithm Acceleration in Help Center e File Exchange

Prodotti


Release

R2018a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by