groupBy

Class: matlab.compiler.mlspark.RDD
Namespace: matlab.compiler.mlspark

Return an RDD of grouped items

Syntax

result = groupBy(obj,func,numPartitions)


Description

result = groupBy(obj,func,numPartitions) groups the elements of obj according a user-specified criteria denoted by func. numPartitions specifies the number of partitions to create in the resulting RDD.

Input Arguments

obj — Input RDD

RDD object

An input RDD, specified as a RDD object.

func — Function to group by

function handle

Function performing grouping, specified as a function handle.

Data Types: function_handle

numPartitions — Number of partitions to create

scalar value

Number of partitions to create, specified as a scalar value.

Data Types: double

Output Arguments

result — RDD containing grouped elements

RDD object

A pipelined RDD containing grouped elements of the input RDD, returned as a RDD object.

Examples

RDD of Grouped Items

Groups the elements of an RDD according a user-specified criteria.

%% Connect to Spark
sparkProp = containers.Map({'spark.executor.cores'}, {'1'});
conf = matlab.compiler.mlspark.SparkConf('AppName','myApp', ...
                        'Master','local[1]','SparkProperties',sparkProp);
sc = matlab.compiler.mlspark.SparkContext(conf);

%% groupBy
inRDD = sc.parallelize({1,2,3,4,5});
outRDD = inRDD.groupBy(@(x)(mod(x,2))).collect(); % {{0,{2,4}},{1,{1,3,5}

Version History

Introduced in R2016b