How to print the elements of a particular partition, say 5th, alone?
val distData = sc.parallelize(1 to 50, 10)
Using Spark/Scala:
val data = 1 to 50 val distData = sc.parallelize(data,10) distData.mapPartitionsWithIndex( (index: Int, it: Iterator[Int]) =>it.toList.map(x => if (index ==5) {println(x)}).iterator).collect
produces:
26 27 28 29 30
1.4m articles
1.4m replys
5 comments
57.0k users