Regression Learner: Importing batches from a database as input data

1 visualizzazione (ultimi 30 giorni)
My problem is that I have a large sqlite database and I would prefer not to import all the rows in memory at once and instead feed the data in batches.
I was experimenting with the fetch() method, but the Regression Learner only accepts tables from workspace and again, I would prefer if I dont have to import the full database into a table in workspace.
Is there a way to achieve this in Matlab?

Risposte (1)

Ishaan Mehta
Ishaan Mehta il 27 Giu 2022
Hi Dominik
As I understand, your problem can be solved by importing the table's rows in batches, instead of fetching the entire table at once. This can be done using the LIMIT clause in SQLite.
LIMIT clause accepts two parameters, the number of rows to fetch and the offset i.e. number of rows to skip from the top of your table.
Here is a code snippet to fetch data in batches using a for loop:
for i = 1:5 % set upper limit according to number of rows in your table.
rowsToFetch = 10; % fetch 5 rows at once
offset = (i-1) * rowsToFetch;
sqlquery = sprintf('SELECT * from tablename LIMIT %d,%d', offset, rowsToFetch);
partialResults = fetch(conn, sqlquery)
end
Hope it helps
Ishaan Mehta

Prodotti


Release

R2022a

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Translated by