[GitHub] spark pull request #20502: [SPARK-23330][WebUI] Spark UI SQL executions page...

2018-02-03 Thread jiangxb1987
GitHub user jiangxb1987 opened a pull request: https://github.com/apache/spark/pull/20502 [SPARK-23330][WebUI] Spark UI SQL executions page throws NPE ## What changes were proposed in this pull request? Spark SQL executions page throws the following error and the page

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87041/ Test PASSed. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87042/ Test PASSed. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87042/testReport)** for PR 20499 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87041/testReport)** for PR 20498 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87042/testReport)** for PR 20499 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/571/

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165834947 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1587,6 +1600,51 @@ def replace(self, to_replace, value=None, subset=None): |null|

[GitHub] spark issue #20500: [SPARK-14023][MLlib] Make exceptions consistent regardin...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20500 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20500: [SPARK-14023][MLlib] Make exceptions consistent regardin...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87039/ Test PASSed. ---

[GitHub] spark issue #20500: [SPARK-14023][MLlib] Make exceptions consistent regardin...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20500 **[Test build #87039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87039/testReport)** for PR 20500 at commit

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/20501 I'm/We're aware of this and to keep diff small and manageable for backporting fixes, it is preferable to avoid broad changes like this. We can revisit when new releases are stable down,

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20501 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87040/ Test PASSed. ---

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20501 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20501 **[Test build #87040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87040/testReport)** for PR 20501 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/570/

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87041/testReport)** for PR 20498 at commit

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20501 **[Test build #87040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87040/testReport)** for PR 20501 at commit

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20501 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/569/

[GitHub] spark issue #20501: [SPARK-22430][Docs] Unknown tag warnings when building R...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20501 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20501: [SPARK-22430][Docs] Unknown tag warnings when bui...

2018-02-03 Thread rekhajoshm
GitHub user rekhajoshm opened a pull request: https://github.com/apache/spark/pull/20501 [SPARK-22430][Docs] Unknown tag warnings when building R docs with Roxygen 6.0.1 ## What changes were proposed in this pull request? Removed @export tag to get rid of unknown tag warnings

[GitHub] spark issue #20164: [SPARK-22971][ML] OneVsRestModel should use temporary Ra...

2018-02-03 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20164 @srowen yeah, all models exist this issue. Although, a little difference, for other models, it is very straightforward for user to call `setRawPrediction` to avoid overwrite the same name

[GitHub] spark issue #20500: [SPARK-14023][MLlib] Make exceptions consistent regardin...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20500 **[Test build #87039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87039/testReport)** for PR 20500 at commit

[GitHub] spark issue #20500: [SPARK-14023][MLlib] Make exceptions consistent regardin...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20500 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/568/

[GitHub] spark issue #20500: [SPARK-14023][MLlib] Make exceptions consistent regardin...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20500 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20500: [SPARK-14023][MLlib] Make exceptions consistent r...

2018-02-03 Thread rekhajoshm
GitHub user rekhajoshm opened a pull request: https://github.com/apache/spark/pull/20500 [SPARK-14023][MLlib] Make exceptions consistent regarding fields and columns ## What changes were proposed in this pull request? Make exceptions consistent regarding fields and columns

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-02-03 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/19788 Thanks @wangyum for this nice data support! Now, we can see obvious time reduce from this feature. --- - To unsubscribe,

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165828519 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1587,6 +1600,51 @@ def replace(self, to_replace, value=None, subset=None): |null| null|null|

[GitHub] spark pull request #20499: [SPARK-23328][PYTHON] Disallow default value None...

2018-02-03 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165828448 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1587,6 +1600,51 @@ def replace(self, to_replace, value=None, subset=None): |null| null|null|

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20499 I think this is what originally proposed in the JIRA: >It is possible to use dict as to_replace, but we cannot skip or use None as the value value (although it is ignored). This requires

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2018-02-03 Thread akopich
Github user akopich commented on the issue: https://github.com/apache/spark/pull/19565 @hhbyyh who shall we ping? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20498 Yes sure,I'll rebase my PR after this is merged. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87036/ Test PASSed. ---

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20359 **[Test build #87036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87036/testReport)** for PR 20359 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87038/ Test PASSed. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON] Disallow default value None in na....

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87038/testReport)** for PR 20499 at commit

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87037/ Test PASSed. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87037/testReport)** for PR 20499 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20498 If possible, I prefer to doing it in this PR. We should merge this test-only PR to 2.3 and master for verifying the behavior of the changes made in 2.3. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/567/

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19565: [SPARK-22111][MLLIB] OnlineLDAOptimizer should filter ou...

2018-02-03 Thread hhbyyh
Github user hhbyyh commented on the issue: https://github.com/apache/spark/pull/19565 It's probably better to wait for the opinion from a committer. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20498 @gatorsmile yes, I see what you mean, we should keep it to check that changing `spark.sql.decimalOperations.allowPrecisionLoss` doesn't affect this case, you are right. I will comment on my PR and

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87038/testReport)** for PR 20499 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20498 @mgaido91 If we remove it, we still need test cases for verifying the effects of `spark.sql.decimalOperations.allowPrecisionLoss`. ---

[GitHub] spark issue #20499: [SPARK-23328][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20499 The linked JIRA targets 2.3.0 and it was an alternative of reverting https://github.com/apache/spark/pull/20496#issuecomment-362843558 .. Let me rebase it here anyway .. ---

[GitHub] spark pull request #20496: Revert "[SPARK-21658][SQL][PYSPARK] Add default N...

2018-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20496 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20498 @gatorsmile because this is an example of overflow, ie. what is covered in the new PR: in the new PR I added many tests for this case, so I felt this unnecessary. ---

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20499 Thanks! Also cc @ueshin @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20499 > We need a separate JIRA and target it to 2.3 Sure. --- - To unsubscribe, e-mail:

[GitHub] spark issue #20496: Revert "[SPARK-21658][SQL][PYSPARK] Add default None for...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20496 LGTM Thanks for your quick PR! Merged to master/2.3 --- - To unsubscribe, e-mail:

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20499 cc @rxin, @gatorsmile, @holdenk, @zero323 and @viirya, this is an alternative of reverting its alias matching, and a fix to address

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20499 **[Test build #87037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87037/testReport)** for PR 20499 at commit

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20499 Hm .. could't we backport this to 2.3 as well? Or, do you suggest to make the current change to branch-2.3 only? --- - To

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20499 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/566/

[GitHub] spark pull request #20495: [SPARK-23327] [SQL] Update the description and te...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20495#discussion_r165821762 --- Diff: python/pyspark/sql/functions.py --- @@ -1705,10 +1705,12 @@ def unhex(col): @ignore_unicode_prefix @since(1.5) def length(col):

[GitHub] spark issue #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value N...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20499 @HyukjinKwon We need a separate PR and target it to 2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark pull request #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default ...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20499#discussion_r165821715 --- Diff: python/pyspark/sql/tests.py --- @@ -2245,11 +2245,6 @@ def test_replace(self): .replace(False, True).first())

[GitHub] spark pull request #20497: [MINOR][DOC] Use raw triple double quotes around ...

2018-02-03 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20497 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20498 @mgaido91 We just remove this test case? I think we can still keep it, right? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #20499: [SPARK-19454][PYTHON][FOLLOWUP] Disallow default ...

2018-02-03 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20499 [SPARK-19454][PYTHON][FOLLOWUP] Disallow default value None when 'to_replace' is not a dictionary ## What changes were proposed in this pull request? This PR proposes to disallow

[GitHub] spark issue #20497: [MINOR][DOC] Use raw triple double quotes around docstri...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20497 Thanks! Merged to master/2.3 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/20498 @wangyum yes, you're right. I don't know how I missed it, I am very sorry for my mistake. Anyway, this is fixed also in this PR #20350, which is the "second part" of making Spark SQL-2011

[GitHub] spark pull request #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeti...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20498#discussion_r165821454 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalArithmeticOperations.sql --- @@ -49,7 +49,7 @@ select 1e35 / 0.1;

[GitHub] spark pull request #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeti...

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20498#discussion_r165820795 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalArithmeticOperations.sql --- @@ -49,7 +49,7 @@ select 1e35 / 0.1;

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87035/ Test PASSed. ---

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87035/testReport)** for PR 20498 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87034/ Test PASSed. ---

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix decimalArithmeticOperat...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87034/testReport)** for PR 20498 at commit

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-03 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165820198 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,42 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark pull request #20493: [SPARK-23326][WEBUI]schedulerDelay should return ...

2018-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20493#discussion_r165819565 --- Diff: core/src/test/scala/org/apache/spark/status/AppStatusUtilsSuite.scala --- @@ -0,0 +1,89 @@ +/* + * Licensed to the Apache Software

[GitHub] spark pull request #20495: [SPARK-23327] [SQL] Update the description of thr...

2018-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20495#discussion_r165819296 --- Diff: python/pyspark/sql/functions.py --- @@ -1705,10 +1705,12 @@ def unhex(col): @ignore_unicode_prefix @since(1.5) def

[GitHub] spark pull request #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20498#discussion_r165819002 --- Diff: sql/core/src/test/resources/sql-tests/inputs/typeCoercion/native/decimalArithmeticOperations.sql --- @@ -49,7 +49,7 @@ select 1e35 / 0.1;

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20359 **[Test build #87036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87036/testReport)** for PR 20359 at commit

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20359 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/565/

[GitHub] spark issue #20359: [SPARK-23186][SQL] Initialize DriverManager first before...

2018-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20359 Retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20484: [SPARK-23313][DOC] Add a migration guide for ORC

2018-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20484#discussion_r165818791 --- Diff: docs/sql-programming-guide.md --- @@ -1776,6 +1776,42 @@ working with timestamps in `pandas_udf`s to get the best performance, see

[GitHub] spark issue #20479: [SPARK-23305][SQL][TEST] Test `spark.sql.files.ignoreMis...

2018-02-03 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/20479 Thank you, @HyukjinKwon and @gatorsmile . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #20456: [SPARK-22624][PYSPARK] Expose range partitioning ...

2018-02-03 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20456#discussion_r165818765 --- Diff: python/pyspark/sql/dataframe.py --- @@ -667,6 +667,92 @@ def repartition(self, numPartitions, *cols): else:

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87035/testReport)** for PR 20498 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/564/

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87034/testReport)** for PR 20498 at commit

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20498 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/20498 retest please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20498 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/87029/ Test FAILed. ---

[GitHub] spark issue #20498: [SPARK-22036][SQL][FOLLOWUP] Fix imperfect test

2018-02-03 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20498 **[Test build #87029 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/87029/testReport)** for PR 20498 at commit

[GitHub] spark pull request #20057: [SPARK-22880][SQL] Add cascadeTruncate option to ...

2018-02-03 Thread Stephan202
Github user Stephan202 commented on a diff in the pull request: https://github.com/apache/spark/pull/20057#discussion_r165814532 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/jdbc/DB2Dialect.scala --- @@ -49,4 +49,16 @@ private object DB2Dialect extends JdbcDialect {

  1   2   >