[GitHub] spark pull request #22962: [SPARK-25921][PySpark] Fix BarrierTaskContext whi...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22962#discussion_r232448083 --- Diff: python/pyspark/taskcontext.py --- @@ -144,10 +144,19 @@ def __init__(self): """Construct a BarrierTaskContext, use get instead"""

[GitHub] spark pull request #22955: [SPARK-25949][SQL] Add test for PullOutPythonUDFI...

2018-11-10 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22955#discussion_r232447827 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PullOutPythonUDFInJoinConditionSuite.scala --- @@ -50,20 +50,11 @@ class Pul

[GitHub] spark pull request #22955: [SPARK-25949][SQL] Add test for PullOutPythonUDFI...

2018-11-10 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22955#discussion_r232447937 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/PullOutPythonUDFInJoinConditionSuite.scala --- @@ -0,0 +1,171 @@ +/*

[GitHub] spark pull request #22962: [SPARK-25921][PySpark] Fix BarrierTaskContext whi...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22962#discussion_r232447967 --- Diff: python/pyspark/taskcontext.py --- @@ -144,10 +144,19 @@ def __init__(self): """Construct a BarrierTaskContext, use get instead"""

[GitHub] spark issue #22962: [SPARK-25921][PySpark] Fix BarrierTaskContext while pyth...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22962 @xuanyuanking, mind explaining how and why it happens rather then what happens in PR description? --- - To unsubscribe, e-ma

[GitHub] spark issue #22955: [SPARK-25949][SQL] Add test for PullOutPythonUDFInJoinCo...

2018-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22955 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22955: [SPARK-25949][SQL] Add test for PullOutPythonUDFInJoinCo...

2018-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22955 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4910/

[GitHub] spark issue #22955: [SPARK-25949][SQL] Add test for PullOutPythonUDFInJoinCo...

2018-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22955 **[Test build #98676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98676/testReport)** for PR 22955 at commit [`38b1555`](https://github.com/apache/spark/commit/38

[GitHub] spark pull request #22962: [SPARK-25921][PySpark] Fix BarrierTaskContext whi...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22962#discussion_r232447862 --- Diff: python/pyspark/tests.py --- @@ -614,6 +614,18 @@ def context_barrier(x): times = rdd.barrier().mapPartitions(f).map(context_barri

[GitHub] spark issue #22955: [SPARK-25949][SQL] Add test for PullOutPythonUDFInJoinCo...

2018-11-10 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22955 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22986: [SPARK-25959][ML] GBTClassifier picks wrong impur...

2018-11-10 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22986#discussion_r232447655 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala --- @@ -538,7 +538,7 @@ private[ml] object GBTClassifierParams { Array("lo

[GitHub] spark issue #21161: [SPARK-21645]left outer join synchronize the condition f...

2018-11-10 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/21161 I think it has fixed by [SPARK-21479](https://issues.apache.org/jira/browse/SPARK-21479). --- - To unsubscribe, e-mail: reviews-

[GitHub] spark issue #22932: [SPARK-25102][SQL] Write Spark version to ORC/Parquet fi...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22932 double checked. A late LGTM too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e

[GitHub] spark issue #22994: [BUILD] refactor dev/lint-python in to something readabl...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22994 I agree with this change. The current script is a total mess - I will try to help take a look when the tests pass. BTW, it would be awesome if PR description contains what this PR tries to fix l

[GitHub] spark pull request #22986: [SPARK-25959][ML] GBTClassifier picks wrong impur...

2018-11-10 Thread mgaido91
Github user mgaido91 commented on a diff in the pull request: https://github.com/apache/spark/pull/22986#discussion_r232447366 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala --- @@ -538,7 +538,7 @@ private[ml] object GBTClassifierParams { Array("lo

[GitHub] spark issue #22963: [SPARK-25962][BUILD][PYTHON] Specify minimum versions fo...

2018-11-10 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22963 I also agree with @srowen's (https://github.com/apache/spark/pull/22963#issuecomment-437133365) --- - To unsubscribe, e-mail

[GitHub] spark issue #22893: [SPARK-25868][MLlib] One part of Spark MLlib Kmean Logic...

2018-11-10 Thread mgaido91
Github user mgaido91 commented on the issue: https://github.com/apache/spark/pull/22893 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark pull request #22938: [SPARK-25935][SQL] Prevent null rows from JSON pa...

2018-11-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22938#discussion_r232446781 --- Diff: docs/sql-migration-guide-upgrade.md --- @@ -15,6 +15,8 @@ displayTitle: Spark SQL Upgrading Guide - Since Spark 3.0, the `from_json` f

[GitHub] spark pull request #22938: [SPARK-25935][SQL] Prevent null rows from JSON pa...

2018-11-10 Thread MaxGekk
Github user MaxGekk commented on a diff in the pull request: https://github.com/apache/spark/pull/22938#discussion_r232446534 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala --- @@ -550,15 +550,33 @@ case class JsonToStructs(

[GitHub] spark pull request #22986: [SPARK-25959][ML] GBTClassifier picks wrong impur...

2018-11-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22986#discussion_r232446160 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala --- @@ -538,7 +538,7 @@ private[ml] object GBTClassifierParams { Array("logi

[GitHub] spark pull request #22986: [SPARK-25959][ML] GBTClassifier picks wrong impur...

2018-11-10 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22986#discussion_r232445980 --- Diff: mllib/src/main/scala/org/apache/spark/ml/tree/treeParams.scala --- @@ -538,7 +538,7 @@ private[ml] object GBTClassifierParams { Array("logi

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22899 **[Test build #98675 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98675/testReport)** for PR 22899 at commit [`3a32007`](https://github.com/apache/spark/commit/3a

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22899 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4909/

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22899 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread dilipbiswal
Github user dilipbiswal commented on the issue: https://github.com/apache/spark/pull/22899 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22899 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/98674/ Test FAILed. ---

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22899 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22899: [SPARK-25573] Combine resolveExpression and resolve in t...

2018-11-10 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22899 **[Test build #98674 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98674/testReport)** for PR 22899 at commit [`3a32007`](https://github.com/apache/spark/commit/3

<    1   2   3