Davies Liu created SPARK-11767: ---------------------------------- Summary: Easy to OOM when cache large column Key: SPARK-11767 URL: https://issues.apache.org/jira/browse/SPARK-11767 Project: Spark Issue Type: Improvement Reporter: Davies Liu
The default batch size (10000) does not work well the large column (with serialized size about 100k), it's easy to OOM when unrolling the rows. We should limit the serialized size of batch. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org