Github user koertkuipers commented on the pull request:
https://github.com/apache/spark/pull/735#issuecomment-42896061
hey matei,
i think they always had this feature in kryo, at least in 2.x.
created jira here:
https://issues.apache.org/jira/browse/SPARK-1811
only reason to be careful with this feature is that there is some overhead
on buffer resizing (each time a new array is created with twice the size and
the data is copied), which is why i was reluctant to make the old setting as a
max.
how about defaults 2MB for initial and 16MB for max?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
