Github user andrewor14 commented on the pull request:
https://github.com/apache/incubator-spark/pull/612#issuecomment-35672119
Thanks again for finding this bug. A number of users have reported it
before and we had not been able to provide a good answer, but this patch pretty
much explains it.
I have left some comments regarding the corresponding test. In particular,
the one you have now does not induce spilling, and so does not trigger the
exception that you ran into. I left an example of how to make this happen.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. To do so, please top-post your response.
If your project does not have this feature enabled and wishes so, or if the
feature is enabled but not working, please contact infrastructure at
infrastruct...@apache.org or file a JIRA ticket with INFRA.
---
