La traduzione di questa pagina non è aggiornata. Fai clic qui per vedere l'ultima versione in inglese.

MapReduce

Tecnica di programmazione per l'analisi di insiemi di dati che non trovano spazio in memoria

mapreduce è una tecnica di programmazione adatta all'analisi di grandi insiemi di dati che altrimenti non potrebbero essere contenuti nella memoria del computer. Utilizzare un datastore per elaborare i dati in piccole parti: la tecnica è composta da una fase Map, che formatta i dati o esegue un calcolo preliminare, e da una fase Reduce, che aggrega tutti i risultati della fase Map. Per maggiori informazioni, vedere Getting Started with MapReduce.

Per informazioni sull'uso di altri prodotti con mapreduce, vedere Speed Up and Deploy MapReduce Using Other Products.

Funzioni

espandi tutto

Generale

`mapreduce`	Programming technique for analyzing data sets that do not fit in memory
`datastore`	Create datastore for large collections of data

Metodi degli oggetti ausiliari

`add`	Add single key-value pair to KeyValueStore
`addmulti`	Add multiple key-value pairs to KeyValueStore
`hasnext`	Determine if ValueIterator has one or more values available
`getnext`	Get next value from ValueIterator

Ambiente di esecuzione

`mapreducer`	Define execution environment for mapreduce or tall arrays
`gcmr`	Get current mapreducer configuration

Oggetti

`KeyValueStore`	Store key-value pairs for use with mapreduce
`ValueIterator`	An iterator over intermediate values for use with mapreduce

Argomenti

Getting Started with MapReduce
Learn about the MapReduce programming technique and run an example calculation.
Write a Map Function
Create a map function for use in a mapreduce algorithm.
Write a Reduce Function
Create a reduce function for use in a mapreduce algorithm.
Speed Up and Deploy MapReduce Using Other Products
Capabilities of other products to speed up and share mapreduce algorithms.

Risoluzione dei problemi

Debug MapReduce Algorithms

This example shows how to debug mapreduce algorithms in MATLAB^®. Debugging enables you to follow the movement of data between the different phases of mapreduce execution and inspect the state of all intermediate variables.

Esempi in primo piano

Analyze Big Data in MATLAB Using MapReduce

Use the mapreduce function to process a large amount of file-based data. The MapReduce algorithm is a mainstay of many modern "big data" applications. This example operates on a single computer, but the code can scale up to use Hadoop®.

Apri live script

Find Maximum Value with MapReduce

Find the maximum value of a single variable in a data set using mapreduce. It demonstrates the simplest use of mapreduce since there is only one key and minimal computation.

Apri live script

Compute Mean Value with MapReduce

Compute the mean of a single variable in a data set using mapreduce. It demonstrates a simple use of mapreduce with one key, minimal computation, and an intermediate state (accumulating intermediate sum and count).

Apri live script

Create Histograms Using MapReduce

Visualize patterns in a large data set without having to load all of the observations into memory simultaneously. It demonstrates how to compute lower volume summaries of the data that are sufficient to generate a graphic.

Apri live script

Compute Mean by Group Using MapReduce

Compute the mean by group in a data set using mapreduce. It demonstrates how to do computations on subgroups of data.

Apri live script

Simple Data Subsetting Using MapReduce

Extract a subset of a large data set.

Apri live script

Using MapReduce to Compute Covariance and Related Quantities

Compute the mean and covariance for several variables in a large data set using mapreduce. It then uses the covariance to perform several follow-up calculations that do not require another iteration over the entire data set.

Apri live script

Compute Summary Statistics by Group Using MapReduce

Compute summary statistics organized by group using mapreduce. It demonstrates the use of an anonymous function to pass an extra grouping parameter to a parameterized map function. This parameterization allows you to quickly recalculate the statistics using a different grouping variable.

Apri live script

Using MapReduce to Fit a Logistic Regression Model

Use mapreduce to carry out simple logistic regression using a single predictor. It demonstrates chaining multiple mapreduce calls to carry out an iterative algorithm. Since each iteration requires a separate pass through the data, an anonymous function passes information from one iteration to the next to supply information directly to the mapper.

Apri live script

Tall Skinny QR (TSQR) Matrix Factorization Using MapReduce

Compute a tall skinny QR (TSQR) factorization using mapreduce. It demonstrates how to chain mapreduce calls to perform multiple iterations of factorizations, and uses the info argument of the map function to compute numeric keys.

Apri live script

Compute Maximum Average HSV of Images with MapReduce

Use ImageDatastore and mapreduce to find images with maximum hue, saturation and brightness values in an image collection.

Apri live script