Can't merge tables with error of weird array size limitation, help?

2 visualizzazioni (ultimi 30 giorni)
I got two tables, one with training data and one with created features that I want to implement.
Table A has the size 307511x241 and is like so:
>> head(app_train, 3)
ans =
3×241 table
SK_ID_CURR NAME_CONTRACT_TYPE CODE_GENDER_M CODE_GENDER_F FLAG_OWN_CAR etc.
__________ __________________ _____________ _____________ ____________
100002 0 1 0 0 etc.
100003 0 0 1 0 etc.
100004 1 1 0 1 etc.
Table B has the size 307511x36 and is like so:
>> head(polyFeaturesTrain, 3)
ans =
3×36 table
Polynomial constant EXT_SOURCE_1 EXT_SOURCE_1^2 EXT_SOURCE_1^3 etc.
___________________ __________________ ___________________ __________________
1 0.0830369673913226 0.00689513795354757 0.0005725513454074 etc.
1 0.311267311381222 0.0968873391344949 0.0301578615592749 etc.
1 0.505997930505754 0.256033905676106 0.129552626411415 etc.
5 variables in Table B are the same as in Table A (EXT_SOURCE_1, EXT_SOURCE_2, EXT_SOURCE_3, DAYS_BIRTH and TARGET). The values for TARGET are the same in both tables, while for the other variables there are some missing values in Table A but not in Table B.
I want to merge Table B into Table A (rows stay the same, only add the variables) and for identical variables want to keep the values from Table B. I tried innerjoin, outerjoin, join and some different Name-Value Pair Arguments for those functions, nothing works. Also trying TableC = [TableA, TableB] doesn't work because of the identical variables.
When using the function I often get this error message:
Error using tabular/outerjoin (line 228)
Requested 1x80527655221 (600.0GB) array exceeds maximum array size preference. Creation of arrays greater than this limit may take a long time and cause MATLAB to become
unresponsive.
The table I want would be 307511x272, how can it be 600GB? I don't understand what Matlab is trying to tell me.
If someone knows what I am doing wrong, please help me out, I spent the last few hours making no progress because of this.

Risposta accettata

Stephen23
Stephen23 il 15 Mag 2021
"I want to merge Table B into Table A (rows stay the same, only add the variables) and for identical variables want to keep the values from Table B"
x = randi(9,7,1);
y = randi(9,7,1);
z = randi(9,7,1);
A = table(x,y)
A = 7×2 table
x y _ _ 4 6 1 7 1 9 8 1 1 3 2 5 8 7
y = round(y/2);
B = table(y,z)
B = 7×2 table
y z _ _ 3 4 4 5 5 4 1 4 2 2 3 4 4 3
V = setdiff(A.Properties.VariableNames,B.Properties.VariableNames);
C = [A(:,V),B]
C = 7×3 table
x y z _ _ _ 4 3 4 1 4 5 1 5 4 8 1 4 1 2 2 2 3 4 8 4 3

Più risposte (0)

Categorie

Scopri di più su Tables in Help Center e File Exchange

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by