Spark hangs after i perform the following operations
ArrayList<byte[]> bytesList = new ArrayList<byte[]>();
/*
add 40k entries to bytesList
*/
JavaRDD<byte[]> rdd = sparkContext.parallelize(bytesList);
System.out.println("Count=" + rdd.count());
If i add just one entry it works.
It works if i modify,
JavaRDD<byte[]> rdd = sparkContext.parallelize(bytesList)
to
JavaRDD<byte[]> rdd = sparkContext.parallelize(bytesList, 20);
There is nothing in the logs that can help understand the reason.
What could be reason for this ?
Regards,
Amit Kumar Karmakar
