[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-02-08 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20378 @HyukjinKwon @holdenk @ueshin @viirya @icexelloss @felixcheung @BryanCutler and @MrBago - This was one of the possible approach that I was running by you. I have proposed another approach at

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-02-08 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20378 @HyukjinKwon Identifying docstyle failures does not help much as it is not straightforward to exclude in this version. --- -

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-02-06 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20378 @HyukjinKwon I will check it over weekend.thanks --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 If you happen to be busy on working this one, I can take this when I have some time. That's fine. Either way works. --- -

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-02-06 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 @rekhajoshm, would you mind if I ask to take a quick look for ones causing actual problems and identify them? It might be kind of a grunt job tho. ---

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-31 Thread icexelloss
Github user icexelloss commented on the issue: https://github.com/apache/spark/pull/20378 I share the same concern of backporting. If we decide to do large amounts of format changes. Should we consider backporting the format changes in one batch so future backporting can be easier?

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 pydocstyle seems claiming PEP 257 - https://www.python.org/dev/peps/pep-0257. One option given https://github.com/apache/spark/pull/20378#issuecomment-361494109 and

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-30 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/20378 I looked at the example output to see see what the errors were. Specifically looking at `broadcast.__init__` ``` def __init__(self, sc=None, value=None, pickle_registry=None,

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-30 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 > One question I have is, do the current violations cause significant document error? I think this is a good point. Maybe, we could enable ones fixing actual significant problems, at

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-30 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20378 One question I have is, do the current violations cause significant document error? Overall this is a good idea. However, is it worth enforcedly applying this if we consider the effort of

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-29 Thread ueshin
Github user ueshin commented on the issue: https://github.com/apache/spark/pull/20378 I like this idea, too, but seems like there are too many violating files so we can't enable this for now. I'm wondering how we can encourage contributors to follow the style, hopefully

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-29 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20378 Thanks @HyukjinKwon for your update. @HyukjinKwon @holdenk @ueshin @viirya @icexelloss @felixcheung @BryanCutler and @MrBago - While you are thinking on it, below is my analysis.

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 Hey @holdenk, @ueshin, @viirya, @icexelloss, @felixcheung, @BryanCutler and @MrBago. What do you guys think about checking docstring and the list above? I think this could prevent nitpicking

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-29 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 So, seems we got: ``` First line should end with a period. 293 Multiline docstring should end with 1 blank line. 279 Blank line missing after one-line summary. 265 Return

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-28 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20378 Please give me few days .. Let me try to take a close look for this. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-28 Thread rekhajoshm
Github user rekhajoshm commented on the issue: https://github.com/apache/spark/pull/20378 Thanks @HyukjinKwon for review. This checks only for python files, as $PATHS_TO_CHECK is passed, and it takes only python files(find . -name "*.py").Also the single file version on

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86563/ Test FAILed. ---

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20378 **[Test build #86563 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86563/testReport)** for PR 20378 at commit

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20378 **[Test build #86563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86563/testReport)** for PR 20378 at commit

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20378: [SPARK-11222][Build][Python] Python document style check...

2018-01-23 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20378 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/170/