Re: spark uses too much memory maybe (binaryFiles() with more than 1 million files in HDFS), groupBy or reduceByKey()

Konstantinos Kougios Thu, 11 Jun 2015 06:18:57 -0700


after 2h of running, now I got a 10GB long[], 1.3mil instances of long[]


So probably information about the files again.


---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Re: spark uses too much memory maybe (binaryFiles() with more than 1 million files in HDFS), groupBy or reduceByKey()

Reply via email to