[GitHub] spark issue #20691: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-02-28 Thread inpefess
Github user inpefess commented on the issue: https://github.com/apache/spark/pull/20691 Well, actually I just wanted to simply merge an older seemingly straightforward PR https://github.com/apache/spark/pull/15670 :) And @holdenk warned me that "it _should_ just be fixing the

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-02-28 Thread inpefess
Github user inpefess commented on the issue: https://github.com/apache/spark/pull/15670 @holdenk done: https://github.com/apache/spark/pull/20691 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #20691: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-02-28 Thread inpefess
Github user inpefess commented on the issue: https://github.com/apache/spark/pull/20691 @holdenk review needed --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail

[GitHub] spark pull request #20691: [SPARK-18161] [Python] Allow pickle to serialize ...

2018-02-28 Thread inpefess
GitHub user inpefess opened a pull request: https://github.com/apache/spark/pull/20691 [SPARK-18161] [Python] Allow pickle to serialize >4 GB objects when possible (Python 3.4+) ## What changes were proposed in this pull request? Since pickle protocol 4 (available

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2018-02-28 Thread inpefess
Github user inpefess commented on the issue: https://github.com/apache/spark/pull/15670 I'm still interested in merging this PR. I made a fork from the PR author's repo, pulled current Spark master branch and rebased the PR author's branch onto it. Now I have a branch

[GitHub] spark issue #15670: [SPARK-18161] [Python] Allow pickle to serialize >4 GB o...

2017-11-22 Thread inpefess
Github user inpefess commented on the issue: https://github.com/apache/spark/pull/15670 I'm also interested in pushing this PR forward. Is there anything else that is needed to be done to merge it except rebasing it and resolving conf