Github user coderfi commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-140974800
FYI, this works for us @ [NinthDecimal](http://www.ninthdecimal.com).
Thanks for the fix, it was a stumper!
Python 2.7.5
Spark-1.5 Branch @
[4c4a9ba28d9052f
Github user asfgit closed the pull request at:
https://github.com/apache/spark/pull/8707
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enab
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-140261303
I tried to find the root cause, but it seems hard to work in all Python
versions (you can see them in the older commit), finally switch to current
approach. merging this
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-140257880
Empirically, this seems to work, so unless you think that we should
investigate the root cause any further I'm fine with giving this an LGTM and
merging to 1.5. Feel f
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139666998
@JoshRosen BTW, this patch introduce a special case for namedtuple, it
should be safe to merge into branch-1.5.
---
If your project is set up for it, you can reply to t
Github user davies commented on a diff in the pull request:
https://github.com/apache/spark/pull/8707#discussion_r39320041
--- Diff: python/pyspark/tests.py ---
@@ -218,6 +218,11 @@ def test_namedtuple(self):
p2 = loads(dumps(p1, 2))
self.assertEqual(p1, p2
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139666459
Before 1.5, the old way work in CPython, but not PyPy (we don't have a unit
test for it).
---
If your project is set up for it, you can reply to this email and have you
Github user davies commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139631359
The HACK in serializers.py is used for cPickler, not cloudpickle.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139618711
Actually, one point of confusion: it looks like
`python/pyspark/serializers.py` had some methods for serializing namedtuple
classes with its `_restore` and `_hack_name
Github user JoshRosen commented on a diff in the pull request:
https://github.com/apache/spark/pull/8707#discussion_r39297676
--- Diff: python/pyspark/tests.py ---
@@ -218,6 +218,11 @@ def test_namedtuple(self):
p2 = loads(dumps(p1, 2))
self.assertEqual(p1,
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139612469
Do you have any intuition for why this worked prior to 1.5 without the
changes implemented here?
---
If your project is set up for it, you can reply to this email and
Github user JoshRosen commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139612356
Over at https://issues.apache.org/jira/browse/SPARK-10544, someone
commented to mention that other types of built-in types do not seem to be
pickleable in 1.5. For in
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139418770
Merged build finished. Test PASSed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139418771
Test PASSed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139418710
[Test build #42304 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42304/console)
for PR 8707 at commit
[`9326697`](https://github.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139415557
[Test build #42304 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42304/consoleFull)
for PR 8707 at commit
[`9326697`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139414834
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139414810
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139407232
[Test build #1739 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1739/console)
for PR 8707 at commit
[`a2f9f36`](https://github
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139402688
[Test build #1739 has
started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/1739/consoleFull)
for PR 8707 at commit
[`a2f9f36`](https://git
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139393142
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139393045
[Test build #42298 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42298/console)
for PR 8707 at commit
[`a2f9f36`](https://github.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139393145
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139386779
[Test build #42298 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42298/consoleFull)
for PR 8707 at commit
[`a2f9f36`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139386317
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139386347
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139381403
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139381398
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139381359
[Test build #42295 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42295/console)
for PR 8707 at commit
[`1d766aa`](https://github.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139378278
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139378275
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139378212
[Test build #42293 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42293/console)
for PR 8707 at commit
[`6b9095b`](https://github.
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139376155
[Test build #42295 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42295/consoleFull)
for PR 8707 at commit
[`1d766aa`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139375491
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139375520
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139374624
[Test build #42293 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42293/consoleFull)
for PR 8707 at commit
[`6b9095b`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139374384
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139374429
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139367578
Test FAILed.
Refer to this link for build results (access rights to CI server needed):
https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139367530
[Test build #42289 has
finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42289/console)
for PR 8707 at commit
[`d7ef6ce`](https://github.
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139367574
Merged build finished. Test FAILed.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project
Github user SparkQA commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139363730
[Test build #42289 has
started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/42289/consoleFull)
for PR 8707 at commit
[`d7ef6ce`](https://gith
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139361755
Merged build started.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/8707#issuecomment-139361699
Merged build triggered.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not h
GitHub user davies opened a pull request:
https://github.com/apache/spark/pull/8707
[SPARK-10542] [PYSPARK] fix serialize namedtuple
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/davies/spark fix_namedtuple
Alternatively you c
45 matches
Mail list logo
