[GitHub] spark issue #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange when cac...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20831 **[Test build #88256 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88256/testReport)** for PR 20831 at commit [`e1f28e2`](https://github.com/apache/spark/commit/e1

[GitHub] spark issue #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange when cac...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20831 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange when cac...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20831 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1524/ Tes

[GitHub] spark pull request #20831: [SPARK-23614][SQL] Fix incorrect reuse exchange w...

2018-03-14 Thread viirya
GitHub user viirya opened a pull request: https://github.com/apache/spark/pull/20831 [SPARK-23614][SQL] Fix incorrect reuse exchange when caching is used ## What changes were proposed in this pull request? We should provide customized canonicalize plan for `InMemoryRelation`

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20689 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88253/ Test PASSed. ---

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20689 **[Test build #88253 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88253/testReport)** for PR 20689 at commit [`992e2c1`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88251/ Test PASSed. ---

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20817 **[Test build #88251 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88251/testReport)** for PR 20817 at commit [`b2062c7`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20825: add impurity stats in tree leaf node debug string

2018-03-14 Thread davies
Github user davies commented on the issue: https://github.com/apache/spark/pull/20825 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache

[GitHub] spark pull request #20803: [SPARK-23653][SQL] Show sql statement in spark SQ...

2018-03-14 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/20803#discussion_r174677799 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -635,6 +637,7 @@ class SparkSession private( * @since 2.0.0

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88254/ Test PASSed. ---

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20830 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20830 **[Test build #88254 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88254/testReport)** for PR 20830 at commit [`89cf69b`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1523/ Tes

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20539: [SPARK-22700][ML] Bucketizer.transform incorrectl...

2018-03-14 Thread zhengruifeng
Github user zhengruifeng closed the pull request at: https://github.com/apache/spark/pull/20539 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.o

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20827 **[Test build #88255 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88255/testReport)** for PR 20827 at commit [`68650ff`](https://github.com/apache/spark/commit/68

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/20827 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h..

[GitHub] spark issue #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-03-14 Thread goungoun
Github user goungoun commented on the issue: https://github.com/apache/spark/pull/20800 @rxin, checking empty is likely to be a common process in every ETL batch job. I think it is the right place to provide that functionality. When a basic function is missing already supposed to be p

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20830 BTW, I double checked it produces the stack trace fine by manually changing some tests locally. --- - To unsubscribe, e-mail

[GitHub] spark pull request #20829: [SPARK-23690][ML] Add handleinvalid to VectorAsse...

2018-03-14 Thread WeichenXu123
Github user WeichenXu123 commented on a diff in the pull request: https://github.com/apache/spark/pull/20829#discussion_r174675028 --- Diff: mllib/src/main/scala/org/apache/spark/ml/feature/StringIndexer.scala --- @@ -234,7 +234,7 @@ class StringIndexerModel ( val metadata

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20830 **[Test build #88254 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88254/testReport)** for PR 20830 at commit [`89cf69b`](https://github.com/apache/spark/commit/89

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20830 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20830 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1522/ Tes

[GitHub] spark issue #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20830 @ueshin and @BryanCutler, could you take a look when you are available? --- - To unsubscribe, e-mail: reviews-unsubscr...@spa

[GitHub] spark pull request #20830: [SPARK-23691][PYTHON] Use sql_conf util in PySpar...

2018-03-14 Thread HyukjinKwon
GitHub user HyukjinKwon opened a pull request: https://github.com/apache/spark/pull/20830 [SPARK-23691][PYTHON] Use sql_conf util in PySpark tests where possible ## What changes were proposed in this pull request? https://github.com/apache/spark/commit/d6632d185e147fcbe6724

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-03-14 Thread goungoun
Github user goungoun commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r174673621 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLocal:

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88245/ Test PASSed. ---

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20705 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20705: [SPARK-23553][TESTS] Tests should not assume the default...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20705 **[Test build #88245 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88245/testReport)** for PR 20705 at commit [`2975aff`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20689 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.a

[GitHub] spark issue #18329: [SPARK-19909][SS] Disabling the usage of a temporary dir...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18329 **[Test build #88252 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88252/testReport)** for PR 18329 at commit [`4a56ecf`](https://github.com/apache/spark/commit/4

[GitHub] spark issue #18329: [SPARK-19909][SS] Disabling the usage of a temporary dir...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18329 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88252/ Test FAILed. ---

[GitHub] spark issue #18329: [SPARK-19909][SS] Disabling the usage of a temporary dir...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/18329 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20689 **[Test build #88253 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88253/testReport)** for PR 20689 at commit [`992e2c1`](https://github.com/apache/spark/commit/99

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20689 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1521/ Tes

[GitHub] spark issue #20689: [SPARK-23533][SS] Add support for changing ContinuousDat...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20689 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20539: [SPARK-22700][ML] Bucketizer.transform incorrectly drops...

2018-03-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20539 So why this PR still open ? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-ma

[GitHub] spark issue #20829: [SPARK-23690][ML] Add handleinvalid to VectorAssembler

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20829 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88249/ Test FAILed. ---

[GitHub] spark issue #20829: [SPARK-23690][ML] Add handleinvalid to VectorAssembler

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20829 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20829: [SPARK-23690][ML] Add handleinvalid to VectorAssembler

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20829 **[Test build #88249 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88249/testReport)** for PR 20829 at commit [`c0c0e3d`](https://github.com/apache/spark/commit/c

[GitHub] spark pull request #20689: [SPARK-23533][SS] Add support for changing Contin...

2018-03-14 Thread xuanyuanking
Github user xuanyuanking commented on a diff in the pull request: https://github.com/apache/spark/pull/20689#discussion_r174667490 --- Diff: external/kafka-0-10-sql/src/main/scala/org/apache/spark/sql/kafka010/KafkaContinuousReader.scala --- @@ -164,7 +164,15 @@ case class KafkaCo

[GitHub] spark issue #18329: [SPARK-19909][SS] Disabling the usage of a temporary dir...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/18329 **[Test build #88252 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88252/testReport)** for PR 18329 at commit [`4a56ecf`](https://github.com/apache/spark/commit/4a

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1520/ Tes

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20817 **[Test build #88251 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88251/testReport)** for PR 20817 at commit [`b2062c7`](https://github.com/apache/spark/commit/b2

[GitHub] spark pull request #20790: [SPARK-23642][DOCS] AccumulatorV2 subclass isZero...

2018-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20790 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/1519/ Tes

[GitHub] spark issue #20817: [SPARK-23599][SQL] Add a UUID generator from Pseudo-Rand...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20817 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20817: [SPARK-23599][SQL] Add a UUID generator from Pseu...

2018-03-14 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/20817#discussion_r174665726 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/RandomUUIDGenerator.scala --- @@ -0,0 +1,38 @@ +/* + * Licensed to the Apac

[GitHub] spark issue #20790: [SPARK-23642][DOCS] AccumulatorV2 subclass isZero scalad...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20790 Merged to master and branch-2.3. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20800#discussion_r174665402 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala --- @@ -511,6 +511,14 @@ class Dataset[T] private[sql]( */ def isLoca

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88246/ Test FAILed. ---

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88250/ Test FAILed. ---

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20827 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20828 **[Test build #88250 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88250/testReport)** for PR 20828 at commit [`c3b96d4`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #20827: [SPARK-23666][SQL] Do not display exprIds of Alias in us...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20827 **[Test build #88246 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88246/testReport)** for PR 20827 at commit [`68650ff`](https://github.com/apache/spark/commit/6

[GitHub] spark issue #20800: [SPARK-23627][SQL] Provide isEmpty in Dataset

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20800 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@s

[GitHub] spark issue #20806: [SPARK-23661][SQL] Implement treeAggregate on Dataset AP...

2018-03-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20806 But I haven't benchmark. Maybe it do not worth to do codegen for treeAggregate. --- - To unsubscribe, e-mail: reviews-unsub

[GitHub] spark issue #20806: [SPARK-23661][SQL] Implement treeAggregate on Dataset AP...

2018-03-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20806 @viirya Yes. `treeAggregate` should only apply to global aggregate. But in this PR the API have to use `seqOp`/`combOp`. What I expect is that the dataframe version treeAggregate can expl

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20828 **[Test build #88250 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88250/testReport)** for PR 20828 at commit [`c3b96d4`](https://github.com/apache/spark/commit/c3

[GitHub] spark pull request #20687: [SPARK-23500][SQL] Fix complex type simplificatio...

2018-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20687#discussion_r174664343 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/ComplexTypes.scala --- @@ -22,54 +22,34 @@ import org.apache.spark.sql

[GitHub] spark issue #20829: [SPARK-23690] [ML] Add handleinvalid to VectorAssembler

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20829 **[Test build #88249 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88249/testReport)** for PR 20829 at commit [`c0c0e3d`](https://github.com/apache/spark/commit/c0

[GitHub] spark issue #20829: [SPARK-23690] [ML] Add handleinvalid to VectorAssembler

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20829 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20829: [SPARK-23690] [ML] Add handleinvalid to VectorAss...

2018-03-14 Thread yogeshg
GitHub user yogeshg opened a pull request: https://github.com/apache/spark/pull/20829 [SPARK-23690] [ML] Add handleinvalid to VectorAssembler ## What changes were proposed in this pull request? Introduce `handleInvalid` parameter in `VectorAssembler` that can take in `"keep

[GitHub] spark pull request #20687: [SPARK-23500][SQL] Fix complex type simplificatio...

2018-03-14 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/20687#discussion_r174663562 --- Diff: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/optimizer/complexTypesSuite.scala --- @@ -331,4 +330,37 @@ class ComplexTypesSuit

[GitHub] spark issue #20806: [SPARK-23661][SQL] Implement treeAggregate on Dataset AP...

2018-03-14 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/20806 @WeichenXu123 I feel `groupBy` is more SQL-like aggregation by which we can specify a key to grouping by. At least `rdd.treeAggregate` does not support key-specified aggregation. For typed g

[GitHub] spark issue #20824: With SPARK-20236, FileCommitProtocol.instantiate() looks...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20824 Or .. should it be `SPARK-23683`? I just saw you opened a JIRA. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apach

[GitHub] spark pull request #20826: [Spark-2489][SQL] Unsupported parquet datatype op...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20826#discussion_r174662613 --- Diff: sql/core/src/test/gen-java/org/apache/spark/sql/execution/datasources/parquet/test/avro/AvroArrayOfArray.java --- @@ -1,39 +1,45 @@ /**

[GitHub] spark pull request #20803: [SPARK-23653][SQL] Show sql statement in spark SQ...

2018-03-14 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20803#discussion_r174662445 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/SparkSession.scala --- @@ -635,6 +637,7 @@ class SparkSession private( * @since 2.0.0

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88242/ Test PASSed. ---

[GitHub] spark pull request #20824: With SPARK-20236, FileCommitProtocol.instantiate(...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/20824#discussion_r174661943 --- Diff: core/src/test/scala/org/apache/spark/internal/io/FileCommitProtocolInstantiationSuite.scala --- @@ -0,0 +1,146 @@ +/* + * Licensed to

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20816 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark pull request #20803: [SPARK-23653][SQL] Show sql statement in spark SQ...

2018-03-14 Thread LantaoJin
Github user LantaoJin commented on a diff in the pull request: https://github.com/apache/spark/pull/20803#discussion_r174662131 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/SQLExecution.scala --- @@ -34,6 +34,16 @@ object SQLExecution { private val

[GitHub] spark issue #20816: [SPARK-21479][SQL] Outer join filter pushdown in null su...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20816 **[Test build #88242 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88242/testReport)** for PR 20816 at commit [`b10879f`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #20824: With SPARK-20236, FileCommitProtocol.instantiate() looks...

2018-03-14 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/20824 Hey @steveloughran, can you fix the title to like `[SPARK-20236][SQL][FOLLOW-UP] ... ` BTW? --- - To unsubscribe, e-mail: re

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88248/ Test FAILed. ---

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20828 **[Test build #88248 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88248/testReport)** for PR 20828 at commit [`dc585b8`](https://github.com/apache/spark/commit/d

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20828 **[Test build #88248 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88248/testReport)** for PR 20828 at commit [`dc585b8`](https://github.com/apache/spark/commit/dc

[GitHub] spark issue #20824: With SPARK-20236, FileCommitProtocol.instantiate() looks...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20824 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20824: With SPARK-20236, FileCommitProtocol.instantiate() looks...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20824 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88241/ Test FAILed. ---

[GitHub] spark issue #20824: With SPARK-20236, FileCommitProtocol.instantiate() looks...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20824 **[Test build #88241 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88241/testReport)** for PR 20824 at commit [`a18ed58`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20828 **[Test build #88247 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88247/testReport)** for PR 20828 at commit [`25f236f`](https://github.com/apache/spark/commit/2

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88247/ Test FAILed. ---

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread jose-torres
Github user jose-torres commented on the issue: https://github.com/apache/spark/pull/20828 @tdas @zsxwing --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20828 **[Test build #88247 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88247/testReport)** for PR 20828 at commit [`25f236f`](https://github.com/apache/spark/commit/25

[GitHub] spark issue #19108: [SPARK-21898][ML] Feature parity for KolmogorovSmirnovTe...

2018-03-14 Thread MrBago
Github user MrBago commented on the issue: https://github.com/apache/spark/pull/19108 Thanks for the changes Weichen, this lgtm. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20828: [SPARK-23687][SS] Add a memory source for continuous pro...

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20828 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #20828: [SPARK-23687][SS] Add a memory source for continu...

2018-03-14 Thread jose-torres
GitHub user jose-torres opened a pull request: https://github.com/apache/spark/pull/20828 [SPARK-23687][SS] Add a memory source for continuous processing. ## What changes were proposed in this pull request? Add a memory source for continuous processing. Note that on

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20750 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/88243/ Test FAILed. ---

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-14 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20750 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #20750: [SPARK-23581][SQL] Add interpreted unsafe projection

2018-03-14 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20750 **[Test build #88243 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/88243/testReport)** for PR 20750 at commit [`7e96f4b`](https://github.com/apache/spark/commit/7

[GitHub] spark pull request #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml...

2018-03-14 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/20686 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #20806: [SPARK-23661][SQL] Implement treeAggregate on Dataset AP...

2018-03-14 Thread WeichenXu123
Github user WeichenXu123 commented on the issue: https://github.com/apache/spark/pull/20806 The API seems not dataframe style. What I expect is something like: ``` dataset.groupBy().setAggregateLevel(2).agg(Map("age" -> "max", "salary" -> "avg")) ``` ---

[GitHub] spark issue #20686: [SPARK-22915][MLlib] Streaming tests for spark.ml.featur...

2018-03-14 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/20686 Merging to branch-2.3 too --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: r

  1   2   3   4   >