[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21276 @gatorsmile @hvanhovell could you trigger tests? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21279: [SPARK-24219][k8s] Improve the docker building script to...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21279 **[Test build #90398 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90398/testReport)** for PR 21279 at commit

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21276 How about adding a `sql` method in `Aggregator` and printing it? --- - To unsubscribe, e-mail:

[GitHub] spark pull request #21279: [SPARK-24219][k8s] Improve the docker building sc...

2018-05-08 Thread jerryshao
GitHub user jerryshao opened a pull request: https://github.com/apache/spark/pull/21279 [SPARK-24219][k8s] Improve the docker building script to avoid copying everything under examples to docker image ## What changes were proposed in this pull request? Current docker build

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread zheh12
Github user zheh12 commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186936032 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21182: [SPARK-24068] Propagating DataFrameReader's options to T...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21182 shall we backport it to 2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21144: [SPARK-24043][SQL] Interpreted Predicate should initiali...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21144 shall we backport it to 2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r186934756 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala --- @@ -1,339 +0,0 @@ -/* --- End

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/21278 Thats a fair question -- I initially created a script was to handle Windows calls but I think we can do some of the split stuff inside R. Let me try that out. Regarding Java 9, do you

[GitHub] spark issue #21064: [SPARK-23976][Core] Detect length overflow in UTF8String...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21064 shall we backport it to 2.3? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/21278 also I think test fails on Java 9 - is there a way to exclude - doc sounds like it's a minimal version --- - To

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21278 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3063/

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21278 **[Test build #90397 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90397/testReport)** for PR 21278 at commit

[GitHub] spark issue #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
Github user shivaram commented on the issue: https://github.com/apache/spark/pull/21278 The need for both the Requirements field and the runtime check is documented at https://cran.r-project.org/doc/manuals/r-release/R-exts.html#Writing-portable-packages (Search for `Make sure that

[GitHub] spark pull request #21278: [SPARKR] Require Java 8 for SparkR

2018-05-08 Thread shivaram
GitHub user shivaram opened a pull request: https://github.com/apache/spark/pull/21278 [SPARKR] Require Java 8 for SparkR This change updates the SystemRequirements and also includes a runtime check if the JVM is being launched by R. The runtime check is done by querying `java

[GitHub] spark pull request #20973: [SPARK-20114][ML] spark.ml parity for sequential ...

2018-05-08 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/20973#discussion_r186931806 --- Diff: mllib/src/main/scala/org/apache/spark/ml/fpm/PrefixSpan.scala --- @@ -0,0 +1,96 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21251 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21251 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3062/

[GitHub] spark issue #21251: [SPARK-10878][core] Fix race condition when multiple cli...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21251 **[Test build #90396 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90396/testReport)** for PR 21251 at commit

[GitHub] spark pull request #21165: [Spark-20087][CORE] Attach accumulators / metrics...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21165#discussion_r186930874 --- Diff: core/src/main/scala/org/apache/spark/TaskEndReason.scala --- @@ -212,9 +212,15 @@ case object TaskResultLost extends TaskFailedReason { *

[GitHub] spark pull request #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21028#discussion_r186929981 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/collectionOperations.scala --- @@ -18,15 +18,50 @@ package

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21235 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21186: [SPARK-22279][SQL] Enable `convertMetastoreOrc` by defau...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21186 can you resolve the conflicts? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10....

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21070 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21070: [SPARK-23972][BUILD][SQL] Update Parquet to 1.10.0.

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21070 thanks, merging to master! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21204: [SPARK-24132][ML] Instrumentation improvement for...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21204 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to InputPar...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21145 any other suggestions about naming? we are going to rename `DataReaderFactory` to `InputPartition`. cc @rxin @marmbrus @jose-torres ---

[GitHub] spark pull request #21145: [SPARK-24073][SQL]: Rename DataReaderFactory to I...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21145#discussion_r186927701 --- Diff: sql/core/src/main/java/org/apache/spark/sql/sources/v2/reader/DataSourceReader.java --- @@ -76,5 +76,5 @@ * If this method fails (by

[GitHub] spark issue #21204: [SPARK-24132][ML] Instrumentation improvement for classi...

2018-05-08 Thread mengxr
Github user mengxr commented on the issue: https://github.com/apache/spark/pull/21204 LGTM. Merged into master. Thanks! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186927477 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/21235 I'll merge into master once the test passes. Thanks. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/21235 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21235 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21235 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3061/

[GitHub] spark pull request #21218: [SPARK-24155][ML] Instrumentation improvements fo...

2018-05-08 Thread mengxr
Github user mengxr commented on a diff in the pull request: https://github.com/apache/spark/pull/21218#discussion_r186926118 --- Diff: mllib/src/main/scala/org/apache/spark/ml/clustering/BisectingKMeans.scala --- @@ -278,6 +279,7 @@ class BisectingKMeans @Since("2.0.0") (

[GitHub] spark issue #21235: [SPARK-24181][SQL] Better error message for writing sort...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21235 **[Test build #90395 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90395/testReport)** for PR 21235 at commit

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/21231 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186925349 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child:

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90394 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90394/testReport)** for PR 21153 at commit

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90394/ Test PASSed. ---

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3060/

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90394 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90394/testReport)** for PR 21153 at commit

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21231 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90391/ Test PASSed. ---

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21231 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21231: [SPARK-24119][SQL]Add interpreted execution to SortPrefi...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21231 **[Test build #90391 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90391/testReport)** for PR 21231 at commit

[GitHub] spark issue #21277: [ML]Power Iteration Clustering is not displaying cluster...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3059/

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90393 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90393/testReport)** for PR 21153 at commit

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90393/ Test FAILed. ---

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21153 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21277: [ML]Power Iteration Clustering is not displaying cluster...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21277 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #21277: [ML]Power Iteration Clustering is not displaying ...

2018-05-08 Thread shahidki31
GitHub user shahidki31 opened a pull request: https://github.com/apache/spark/pull/21277 [ML]Power Iteration Clustering is not displaying cluster indices corresponding to some nodes. ## What changes were proposed in this pull request? 1) Currently PIC in ML displays cluster

[GitHub] spark issue #21153: [SPARK-24058][ML][PySpark] Default Params in ML should b...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21153 **[Test build #90393 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90393/testReport)** for PR 21153 at commit

[GitHub] spark pull request #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work ...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/21267#discussion_r186920316 --- Diff: python/pyspark/context.py --- @@ -211,9 +211,23 @@ def _do_init(self, master, appName, sparkHome, pyFiles, environment, batchSize,

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 Hm .. @jerryshao, seems it's a bit difficult to do so. The simplest way should be just to copy files into the directories in `SparkFiles.getRootDirectory`; however, `SparkEnv` is inaccessible

[GitHub] spark pull request #21243: [SPARK-24182][yarn] Improve error message when cl...

2018-05-08 Thread jerryshao
Github user jerryshao commented on a diff in the pull request: https://github.com/apache/spark/pull/21243#discussion_r186918972 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ApplicationMaster.scala --- @@ -346,7 +346,7 @@ private[spark] class

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r186918634 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala --- @@ -1,339 +0,0 @@ -/*

[GitHub] spark issue #21165: [Spark-20087][CORE] Attach accumulators / metrics to 'Ta...

2018-05-08 Thread advancedxy
Github user advancedxy commented on the issue: https://github.com/apache/spark/pull/21165 ping @cloud-fan --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration doc for ...

2018-05-08 Thread dongjoon-hyun
Github user dongjoon-hyun commented on the issue: https://github.com/apache/spark/pull/21269 Thank you, @HyukjinKwon and @cloud-fan . --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21266 **[Test build #90392 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90392/testReport)** for PR 21266 at commit

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90392/ Test FAILed. ---

[GitHub] spark issue #18447: [SPARK-21232][SQL][SparkR][PYSPARK] New built-in SQL fun...

2018-05-08 Thread mmolimar
Github user mmolimar commented on the issue: https://github.com/apache/spark/pull/18447 @felixcheung I think it should be fine now. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21273 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90388/ Test PASSed. ---

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21273 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21273 **[Test build #90388 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90388/testReport)** for PR 21273 at commit

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21266 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution/3058/

[GitHub] spark pull request #21257: [SPARK-24194] [SQL]HadoopFsRelation cannot overwr...

2018-05-08 Thread zheh12
Github user zheh12 commented on a diff in the pull request: https://github.com/apache/spark/pull/21257#discussion_r186915569 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala --- @@ -207,9 +207,25 @@ case class

[GitHub] spark issue #21266: [SPARK-24206][SQL] Improve DataSource read benchmark cod...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21266 **[Test build #90392 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90392/testReport)** for PR 21266 at commit

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread maropu
Github user maropu commented on the issue: https://github.com/apache/spark/pull/21273 We need to list what're differences (new features and bugs) between v2.5.9 and v2.6.3 for checking compatibility and others? ---

[GitHub] spark issue #21270: [SPARK-24213][ML]Power Iteration Clustering in SparkML t...

2018-05-08 Thread shahidki31
Github user shahidki31 commented on the issue: https://github.com/apache/spark/pull/21270 Thank you @jkbradly. Actually one more issue is there. Currently we are skipping some of the nodes which are not there in the ID column, but there in the neighboring column. Spark MLLib is

[GitHub] spark pull request #21266: [SPARK-24206][SQL] Improve DataSource read benchm...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21266#discussion_r186914136 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetReadBenchmark.scala --- @@ -1,339 +0,0 @@ -/* --- End

[GitHub] spark pull request #21231: [SPARK-24119][SQL]Add interpreted execution to So...

2018-05-08 Thread maropu
Github user maropu commented on a diff in the pull request: https://github.com/apache/spark/pull/21231#discussion_r186913964 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/SortOrder.scala --- @@ -147,7 +148,40 @@ case class SortPrefix(child:

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 Will try to put this into SparkSubmit. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21213: [SPARK-24120] Show `Jobs` page when `jobId` is missing

2018-05-08 Thread jongyoul
Github user jongyoul commented on the issue: https://github.com/apache/spark/pull/21213 Thanks for reviewing this PR. Concerning your comments: 1. It makes sense. I'll prolong the time to wait for the current page. 1. Personally, I wondered if redirecting page would be

[GitHub] spark issue #21275: [SPARK-24214][SS]Fix toJSON for StreamingRelationV2/Stre...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21275 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21275: [SPARK-24214][SS]Fix toJSON for StreamingRelationV2/Stre...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21275 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90390/ Test PASSed. ---

[GitHub] spark issue #21275: [SPARK-24214][SS]Fix toJSON for StreamingRelationV2/Stre...

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21275 **[Test build #90390 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90390/testReport)** for PR 21275 at commit

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 (I have tried to explain why it's specific to PySpark shell with Yarn client mode in PR description) --- - To unsubscribe,

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21267 Yea, this is specific to yarn client PySpark shell. In case of yarn client and cluster with submit, they are specially handled via #6360 but I think PySpark shell in yarn client mode was missed

[GitHub] spark issue #21267: [SPARK-21945][YARN][PYTHON] Make --py-files work with Py...

2018-05-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21267 Does it only happen in yarn client PySpark shell? I would suggest to fix this in the SparkSubmit side, to treat this as a special case and set the proper config. ---

[GitHub] spark issue #21268: [SPARK-24209][SHS] Automatic retrieve proxyBase from Kno...

2018-05-08 Thread jerryshao
Github user jerryshao commented on the issue: https://github.com/apache/spark/pull/21268 The changes here seems affect so many places, I'm wondering if there's any other way to minimize the changes? --- - To

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90386/ Test PASSed. ---

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21028 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21028: [SPARK-23922][SQL] Add arrays_overlap function

2018-05-08 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21028 **[Test build #90386 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/90386/testReport)** for PR 21028 at commit

[GitHub] spark pull request #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration d...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21269 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21273: [WIP][SPARK-17916][SQL] Fix empty string being parsed as...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21273 Sounds good @MaxGekk BTW mind adding `Closes #20068` in the PR description? --- - To unsubscribe, e-mail:

[GitHub] spark issue #21269: [SPARK-23355][SQL][DOC][FOLLOWUP] Add migration doc for ...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21269 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #21182: [SPARK-24068] Propagating DataFrameReader's optio...

2018-05-08 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21182 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21182: [SPARK-24068] Propagating DataFrameReader's options to T...

2018-05-08 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/21182 Merged to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21276 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression uses g...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21276 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21242 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/90383/ Test FAILed. ---

[GitHub] spark issue #21242: [SPARK-23657][SQL] Document and expose the internal data...

2018-05-08 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21242 Build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands,

[GitHub] spark pull request #21276: [SPARK-24216][SQL] Spark TypedAggregateExpression...

2018-05-08 Thread fangshil
GitHub user fangshil opened a pull request: https://github.com/apache/spark/pull/21276 [SPARK-24216][SQL] Spark TypedAggregateExpression uses getSimpleName that is not safe in scala ## What changes were proposed in this pull request? When we create a aggregator object

  1   2   3   4   5   >