[GitHub] spark issue #20391: [SPARK-23208][SQL] Fix code generation for complex creat...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20391 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86623/ Test PASSed. ---

[GitHub] spark issue #20391: [SPARK-23208][SQL] Fix code generation for complex creat...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20391 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20391: [SPARK-23208][SQL] Fix code generation for complex creat...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20391 **[Test build #86623 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86623/testReport)** for PR 20391 at commit

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163769201 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/WholeStageCodegenSuite.scala --- @@ -228,4 +229,35 @@ class WholeStageCodegenSuite extends

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19285 **[Test build #86630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86630/testReport)** for PR 19285 at commit

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-24 Thread ConeyLiu
Github user ConeyLiu commented on a diff in the pull request: https://github.com/apache/spark/pull/19285#discussion_r163768689 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -232,78 +236,93 @@ private[spark] class MemoryStore(

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19285 **[Test build #86629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86629/testReport)** for PR 19285 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/18931 LGTM, pending jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20224 LGTM, pending jenkins --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163767104 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -629,6 +629,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19285 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86619/ Test PASSed. ---

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19285 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19285 **[Test build #86619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86619/testReport)** for PR 19285 at commit

[GitHub] spark pull request #20292: [SPARK-23129][CORE] Make deserializeStream of Dis...

2018-01-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20292 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20292: [SPARK-23129][CORE] Make deserializeStream of DiskMapIte...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20292 thanks, merging to master/2.3! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #20372: Improved block merging logic for partitions

2018-01-24 Thread ash211
Github user ash211 commented on the issue: https://github.com/apache/spark/pull/20372 Tagging folks who have touched this code recently: @vgankidi @ericl @davies This seems to provide a more compact packing in every scenario, which should improve execution times. One risk

[GitHub] spark issue #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20390 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20367: [SPARK-23166][ML] Add maxDF Parameter to CountVectorizer

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20367 **[Test build #4077 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4077/testReport)** for PR 20367 at commit

[GitHub] spark issue #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20390 **[Test build #86626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86626/testReport)** for PR 20390 at commit

[GitHub] spark issue #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86626/ Test PASSed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86622/ Test PASSed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86622 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86622/testReport)** for PR 18931 at commit

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86621/ Test PASSed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86621 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86621/testReport)** for PR 18931 at commit

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86628/testReport)** for PR 13599 at commit

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19285 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86620/ Test FAILed. ---

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86628/ Test FAILed. ---

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/19285 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18906: [SPARK-21692][PYSPARK][SQL] Add nullability support to P...

2018-01-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/18906 BTW, what do we get if it returns `None` but it's non-nullable, NEP? --- - To unsubscribe, e-mail:

[GitHub] spark issue #19285: [SPARK-22068][CORE]Reduce the duplicate code between put...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/19285 **[Test build #86620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86620/testReport)** for PR 19285 at commit

[GitHub] spark issue #20292: [SPARK-23129][CORE] Make deserializeStream of DiskMapIte...

2018-01-24 Thread jiangxb1987
Github user jiangxb1987 commented on the issue: https://github.com/apache/spark/pull/20292 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13599 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/223/

[GitHub] spark issue #20292: [SPARK-23129][CORE] Make deserializeStream of DiskMapIte...

2018-01-24 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/20292 LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

2018-01-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r163759284 --- Diff: python/pyspark/sql/functions.py --- @@ -2105,6 +2105,14 @@ def udf(f=None, returnType=StringType()): >>> import random >>>

[GitHub] spark pull request #18906: [SPARK-21692][PYSPARK][SQL] Add nullability suppo...

2018-01-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/18906#discussion_r163757006 --- Diff: python/pyspark/sql/functions.py --- @@ -2264,6 +2272,16 @@ def pandas_udf(f=None, returnType=None, functionType=None): ...

[GitHub] spark issue #13599: [SPARK-13587] [PYSPARK] Support virtualenv in pyspark

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13599 **[Test build #86628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86628/testReport)** for PR 13599 at commit

[GitHub] spark pull request #20384: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-24 Thread gatorsmile
Github user gatorsmile closed the pull request at: https://github.com/apache/spark/pull/20384 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20292: [SPARK-23129][CORE] Make deserializeStream of DiskMapIte...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20292 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86618/ Test PASSed. ---

[GitHub] spark issue #20292: [SPARK-23129][CORE] Make deserializeStream of DiskMapIte...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20292 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20292: [SPARK-23129][CORE] Make deserializeStream of DiskMapIte...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20292 **[Test build #86618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86618/testReport)** for PR 20292 at commit

[GitHub] spark pull request #19862: [SPARK-22671][SQL] Make SortMergeJoin shuffle rea...

2018-01-24 Thread gczsjdy
Github user gczsjdy closed the pull request at: https://github.com/apache/spark/pull/19862 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #19862: [SPARK-22671][SQL] Make SortMergeJoin shuffle read less ...

2018-01-24 Thread gczsjdy
Github user gczsjdy commented on the issue: https://github.com/apache/spark/pull/19862 @cloud-fan Ok, thanks for your time, I will close this. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20390 **[Test build #86626 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86626/testReport)** for PR 20390 at commit

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20224 **[Test build #86627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86627/testReport)** for PR 20224 at commit

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20224 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/222/

[GitHub] spark issue #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20390 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20224 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20390 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/221/

[GitHub] spark pull request #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread huaxingao
Github user huaxingao commented on a diff in the pull request: https://github.com/apache/spark/pull/20390#discussion_r163759921 --- Diff: python/pyspark/sql/tests.py --- @@ -2855,6 +2855,10 @@ def test_create_dataframe_from_old_pandas(self): with

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-24 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163757952 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -325,6 +326,28 @@ object WholeStageCodegenExec {

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-24 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163757687 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -629,6 +629,13 @@ object SQLConf { .booleanConf

[GitHub] spark issue #20367: [SPARK-23166][ML] Add maxDF Parameter to CountVectorizer

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20367 **[Test build #4077 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4077/testReport)** for PR 20367 at commit

[GitHub] spark pull request #20389: [SPARK-23205][ML] Update ImageSchema.readImages t...

2018-01-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20389#discussion_r163756853 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -169,8 +169,7 @@ object ImageSchema { var offset = 0

[GitHub] spark pull request #20389: [SPARK-23205][ML] Update ImageSchema.readImages t...

2018-01-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20389#discussion_r163756581 --- Diff: mllib/src/main/scala/org/apache/spark/ml/image/ImageSchema.scala --- @@ -169,8 +169,7 @@ object ImageSchema { var offset = 0

[GitHub] spark pull request #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20390#discussion_r163755876 --- Diff: python/pyspark/sql/tests.py --- @@ -2855,6 +2855,10 @@ def test_create_dataframe_from_old_pandas(self): with

[GitHub] spark pull request #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20390#discussion_r163755064 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1881,6 +1881,28 @@ def toDF(self, *cols): jdf = self._jdf.toDF(self._jseq(cols))

[GitHub] spark pull request #20390: [SPARK-23081][PYTHON]Add colRegex API to PySpark

2018-01-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20390#discussion_r163756176 --- Diff: python/pyspark/sql/dataframe.py --- @@ -1881,6 +1881,28 @@ def toDF(self, *cols): jdf = self._jdf.toDF(self._jseq(cols))

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20375 **[Test build #86625 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86625/testReport)** for PR 20375 at commit

[GitHub] spark issue #20388: [SPARK-23020][core] Fix race in SparkAppHandle cleanup, ...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20388 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86616/ Test PASSed. ---

[GitHub] spark issue #20388: [SPARK-23020][core] Fix race in SparkAppHandle cleanup, ...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20388 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20382: [SPARK-23097][SQL][SS] Migrate text socket source...

2018-01-24 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20382#discussion_r163755819 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/TextSocketStreamSourceV2.scala --- @@ -0,0 +1,247 @@ +/* + *

[GitHub] spark issue #20388: [SPARK-23020][core] Fix race in SparkAppHandle cleanup, ...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20388 **[Test build #86616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86616/testReport)** for PR 20388 at commit

[GitHub] spark pull request #20382: [SPARK-23097][SQL][SS] Migrate text socket source...

2018-01-24 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/20382#discussion_r163755636 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/TextSocketStreamSourceV2.scala --- @@ -0,0 +1,247 @@ +/* +

[GitHub] spark issue #20375: [SPARK-23199][SQL]improved Removes repetition from group...

2018-01-24 Thread heary-cao
Github user heary-cao commented on the issue: https://github.com/apache/spark/pull/20375 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/20377 The original test covers the original scenario. I think the one pointed by @vanzin is another issue. However, I do not have time to try it. ---

[GitHub] spark pull request #20384: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-24 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20384#discussion_r163755332 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala --- @@ -110,15 +110,39 @@ class BroadcastJoinSuite

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20226: [SPARK-23034][SQL] Override `nodeName` for all *ScanExec...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20226 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/220/

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86624/testReport)** for PR 18931 at commit

[GitHub] spark pull request #20382: [SPARK-23097][SQL][SS] Migrate text socket source...

2018-01-24 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/20382#discussion_r163753088 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/sources/TextSocketStreamSourceV2.scala --- @@ -0,0 +1,247 @@ +/* + *

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/219/

[GitHub] spark pull request #20392: Update ApplicationMaster.scala

2018-01-24 Thread Sangrho
Github user Sangrho closed the pull request at: https://github.com/apache/spark/pull/20392 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #18931: [SPARK-21717][SQL] Decouple consume functions of ...

2018-01-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/18931#discussion_r163752999 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -156,13 +162,96 @@ trait CodegenSupport extends

[GitHub] spark issue #20392: Update ApplicationMaster.scala

2018-01-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/20392 Close this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20369: [SPARK-23196] Unify continuous and microbatch V2 ...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20369#discussion_r163751348 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/StreamWriteSupport.java --- @@ -48,7 +48,7 @@ * @param options the

[GitHub] spark pull request #20369: [SPARK-23196] Unify continuous and microbatch V2 ...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20369#discussion_r163751286 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/streaming/DataStreamWriter.scala --- @@ -281,11 +281,9 @@ final class DataStreamWriter[T]

[GitHub] spark pull request #20369: [SPARK-23196] Unify continuous and microbatch V2 ...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20369#discussion_r163751198 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/streaming/MicroBatchWriteSupport.java --- @@ -1,60 +0,0 @@ -/* - * Licensed to

[GitHub] spark pull request #20369: [SPARK-23196] Unify continuous and microbatch V2 ...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20369#discussion_r163751107 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/WriteToDataSourceV2.scala --- @@ -62,7 +62,7 @@ case class

[GitHub] spark issue #20377: [SPARK-17088] [FOLLOW-UP] Fix 'sharesHadoopClasses' opti...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20377 I'm a little confused, I think [this test](https://github.com/apache/spark/pull/20169/files#diff-0456ca985f0d885d5b72654e10be77ccR204) should help us detect the wrong fix, but this PR passed all

[GitHub] spark pull request #20384: [SPARK-23195] [SQL] Keep the Hint of Cached Data

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20384#discussion_r163750722 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala --- @@ -110,15 +110,39 @@ class BroadcastJoinSuite

[GitHub] spark pull request #20359: [SPARK-23186][SQL] Initialize DriverManager first...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20359#discussion_r163750551 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/DriverRegistry.scala --- @@ -32,6 +32,9 @@ import

[GitHub] spark pull request #20359: [SPARK-23186][SQL] Initialize DriverManager first...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20359#discussion_r163750452 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/DriverRegistry.scala --- @@ -32,6 +32,9 @@ import

[GitHub] spark pull request #20359: [SPARK-23186][SQL] Initialize DriverManager first...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20359#discussion_r163750346 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/DriverRegistry.scala --- @@ -32,6 +32,9 @@ import

[GitHub] spark issue #20392: Update ApplicationMaster.scala

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20392 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/86615/ Test PASSed. ---

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18931 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20367: [SPARK-23166][ML] Add maxDF Parameter to CountVectorizer

2018-01-24 Thread ymazari
Github user ymazari commented on the issue: https://github.com/apache/spark/pull/20367 @srowen It seems that this PR needs Admin approval. Could you please help getting it to the next step? Thanks. --- - To

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19285#discussion_r163750079 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -702,6 +641,87 @@ private[spark] class MemoryStore( }

[GitHub] spark issue #20392: Update ApplicationMaster.scala

2018-01-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20392 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19285#discussion_r163749987 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -702,6 +641,87 @@ private[spark] class MemoryStore( }

[GitHub] spark issue #18931: [SPARK-21717][SQL] Decouple consume functions of physica...

2018-01-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18931 **[Test build #86615 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/86615/testReport)** for PR 18931 at commit

[GitHub] spark pull request #20392: Update ApplicationMaster.scala

2018-01-24 Thread Sangrho
GitHub user Sangrho opened a pull request: https://github.com/apache/spark/pull/20392 Update ApplicationMaster.scala I have one question. I think when maxNumExecutorFailures is calculated, MAX_EXECUTOR_FAILURES is already defined by specific by spark document (as numExecutors *

[GitHub] spark pull request #19285: [SPARK-22068][CORE]Reduce the duplicate code betw...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/19285#discussion_r163749065 --- Diff: core/src/main/scala/org/apache/spark/storage/memory/MemoryStore.scala --- @@ -232,78 +236,93 @@ private[spark] class MemoryStore(

[GitHub] spark issue #20224: [SPARK-23032][SQL] Add a per-query codegenStageId to Who...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/20224 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #20224: [SPARK-23032][SQL] Add a per-query codegenStageId...

2018-01-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20224#discussion_r163748166 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/WholeStageCodegenExec.scala --- @@ -325,6 +326,28 @@ object WholeStageCodegenExec {

  1   2   3   4   5   6   >