Github user megatron-me-uk commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103463636
OK, seems I have to create an account etc. I will put it on my to-do list.
Thanks for the help!
---
If your project is set up for it, you can reply to this email
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103462714
I think it makes sense; really I was getting at making a JIRA to track this.
---
If your project is set up for it, you can reply to this email and have your
reply appear
Github user megatron-me-uk commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103462427
I believe that this change will bring pyspark more in line with the
operation of the scala implementation.
See:
https://github.com/apache/spark/blob/4ce2782a6
Github user megatron-me-uk commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103447402
Ah, I hadn't seen that! Will take a look.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your
Github user srowen commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103445969
@megatron-me-uk have a look at
https://cwiki.apache.org/confluence/display/SPARK/Contributing+to+Spark first
please -- you missed a few steps here. The change may well be
Github user AmplabJenkins commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103441440
Can one of the admins verify this patch?
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your pr
Github user megatron-me-uk commented on the pull request:
https://github.com/apache/spark/pull/6262#issuecomment-103441329
A simple test of this:
```python
a = sc.parallelize([1, 2, 3])
b = a.pipe('cc') # a clearly incorrect pipe command
b.collect()
```
GitHub user megatron-me-uk opened a pull request:
https://github.com/apache/spark/pull/6262
Raise Exception on non-zero exit from pipe commands
This will allow problems with piped commands to be detected.
This will also allow tasks to be retried where errors are rare (such as
ne
