Explore - Scientific Workflow Management Tool

Lightweight workflow management tool to automatically retrieve previous intermediate results and track provenance in scientific workflows.
49 download
Aggiornato 3 set 2019

Explore is a scientific workflow management tool. You can visualize the data provenance graph and when your code is changed and re-executed, only the necessary functions are re-executed to provide an updated version of your data. This can save a lot of time, especially when experiments are constantly refined. You can easily declare an experiment as a directed acyclic graph (DAG) where the nodes are functions and the edges represent variables that are produced and consumed by functions.

During the first execution of the graph, variables are persisted to the disk which implies a longer graph execution time (due to variable loading and saving duration). However, for future executions, if the node main function and all the sub-functions called during the node execution remain unchanged (a) and the input variables of the node also remain unchanged (b), then the results are retrieved from the disk and not re-computed.

For (a), the major time consumer is the code analysis to find out the sub-functions involved in the execution of a node. Therefore, this "node sub-function dependency" information is also persisted based on the content of the sub-function files. Then, if the sub-function files did not change, it is assumed the node's sub-function dependencies did not change.
For (b), either the variable content is hashed (i) or the variable mat-file date is taken (ii) as signature to check that the variables did not change. Option (i) is advantageous as re-computed nodes might still output the same values. In that case, unchanged child nodes will not be re-computed. However, option (ii) might be faster for bigger variables, where hashing would take too much time.

Thank you to:
- Jan (2019). DataHash (https://www.mathworks.com/matlabcentral/fileexchange/31272-datahash), MATLAB Central File Exchange. Retrieved April 15, 2019.
- Aslak Grinsted (2019). cachedcall (https://de.mathworks.com/matlabcentral/fileexchange/49949-cachedcall), MATLAB Central File Exchange. Retrieved April 15, 2019.
- Aslak Grinsted (2019). rgb.m (https://de.mathworks.com/matlabcentral/fileexchange/1805-rgb-m), MATLAB Central File Exchange. Retrieved April 15, 2019.
- All upstream dependencies

For tutorials, detailed information and bug reports, please check the Github (https://www.github.com/jahsue78/explore) or the corresponding conference paper [1].

[1] Ah Sue, Jonathan, et al. "Optimizing Exploratory Workflows for Embedded Platform Trace Analysis and Its Application to Mobile Devices." International Conference on Human-Computer Interaction. Springer, Cham, 2019.

Cita come

Jonathan A (2024). Explore - Scientific Workflow Management Tool (https://github.com/vacoa/explore), GitHub. Recuperato .

Compatibilità della release di MATLAB
Creato con R2017a
Compatibile con qualsiasi release
Compatibilità della piattaforma
Windows macOS Linux
Categorie
Scopri di più su Graphics Object Identification in Help Center e MATLAB Answers
Riconoscimenti

Ispirato da: cachedcall, DataHash, rgb.m

Community Treasure Hunt

Find the treasures in MATLAB Central and discover how the community can help you!

Start Hunting!

Le versioni che utilizzano il ramo predefinito di GitHub non possono essere scaricate

Versione Pubblicato Note della release
1.0.11

Add link to the paper

1.0.10

Update description and add new basic tutorial.

1.0.9

Change title and summary

1.0.8

Update description

1.0.7

Update description and acknowledgments

1.0.6

New description

1.0.5

Modify readme and add keywords.

1.0.4

Update description

1.0.3

Add Github reference.

1.0.2

Change keywords

1.0.1

Update image

1.0.0

Per visualizzare o segnalare problemi su questo componente aggiuntivo di GitHub, visita GitHub Repository.
Per visualizzare o segnalare problemi su questo componente aggiuntivo di GitHub, visita GitHub Repository.