I got 14GB of parquet data and when trying to apply order by using spark sql and save the first 1M rows but keeps failing with "Connection reset by peer: socket write error" on the executors.
I've allocated about 10g to both driver and the executors along with setting the maxResultSize to 10g but still fails with the same error. I'm using Spark 1.5.1. Are there any other alternative ways to handle this? Thanks!
