[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95051/ Test FAILed. ---

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16478 **[Test build #95051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95051/testReport)** for PR 16478 at commit

[GitHub] spark issue #22139: [SPARK-25149][GraphX] Update Parallel Personalized Page ...

2018-08-21 Thread jkbradley
Github user jkbradley commented on the issue: https://github.com/apache/spark/pull/22139 LGTM I'll merge this with master Thanks @MrBago ! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/16478 **[Test build #95051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95051/testReport)** for PR 16478 at commit

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/16478 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #16478: [SPARK-7768][SQL] Revise user defined types (UDT)

2018-08-21 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/16478 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark pull request #22121: [WIP][SPARK-25133][SQL][Doc]Avro data source guid...

2018-08-21 Thread tgravescs
Github user tgravescs commented on a diff in the pull request: https://github.com/apache/spark/pull/22121#discussion_r211773604 --- Diff: docs/avro-data-source-guide.md --- @@ -0,0 +1,260 @@ +--- +layout: global +title: Apache Avro Data Source Guide +--- +

[GitHub] spark issue #22133: [SPARK-25129][SQL]Make the mapping of com.databricks.spa...

2018-08-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/22133 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20637 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95044/ Test PASSed. ---

[GitHub] spark issue #21756: [SPARK-24764] [CORE] Add ServiceLoader implementation fo...

2018-08-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21756 I don't understand the purpose of this change. Can you explain exactly what part of `SparkHadoopUtil` do you need to customize, and why that change can't just be made to the shared code?

[GitHub] spark issue #20637: [SPARK-23466][SQL] Remove redundant null checks in gener...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20637 **[Test build #95044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95044/testReport)** for PR 20637 at commit

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21976 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21976 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95034/ Test FAILed. ---

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22085 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95040/ Test PASSed. ---

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21976 **[Test build #95034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95034/testReport)** for PR 21976 at commit

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22085 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22085 **[Test build #95040 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95040/testReport)** for PR 22085 at commit

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22085 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95037/ Test PASSed. ---

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22085 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22085: [SPARK-25095][PySpark] Python support for BarrierTaskCon...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22085 **[Test build #95037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95037/testReport)** for PR 22085 at commit

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22135 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22135 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95038/ Test PASSed. ---

[GitHub] spark issue #22135: [SPARK-25093][SQL] Avoid recompiling regexp for comments...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22135 **[Test build #95038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95038/testReport)** for PR 22135 at commit

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21320 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21320 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95041/ Test PASSed. ---

[GitHub] spark issue #21320: [SPARK-4502][SQL] Parquet nested column pruning - founda...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21320 **[Test build #95041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95041/testReport)** for PR 21320 at commit

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/22009 @cloud-fan, I think that the scan config builder needs to accept the options and that SaveMode needs to be removed before we should merge this PR. I'm fine with following up with the WriteConfig

[GitHub] spark issue #22170: [MINOR] Add .crc files to .gitignore

2018-08-21 Thread HeartSaVioR
Github user HeartSaVioR commented on the issue: https://github.com/apache/spark/pull/22170 @HyukjinKwon Thanks for the reviewing and merging! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r211763717 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/BaseYarnClusterSuite.scala --- @@ -161,6 +162,11 @@ abstract class

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-21 Thread rdblue
Github user rdblue commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r211763465 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -60,14 +61,20 @@ private[spark] object PythonEvalType { */

[GitHub] spark issue #17342: [SPARK-12868][SQL] Allow adding jars from hdfs

2018-08-21 Thread steveloughran
Github user steveloughran commented on the issue: https://github.com/apache/spark/pull/17342 Well, no obvious answer there I'm afraid, except "don't put HDFS JARs on the classpath"; if you serve them up via HTTP all should work ---

[GitHub] spark issue #17342: [SPARK-12868][SQL] Allow adding jars from hdfs

2018-08-21 Thread rajeshcode
Github user rajeshcode commented on the issue: https://github.com/apache/spark/pull/17342 Sorry about above stack. The actual error is as below. Thats why relates to SPARK-21697 Removing commons-logging jar fixes it. copying back common-logging jar never complains again that

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r211748871 --- Diff: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/BaseYarnClusterSuite.scala --- @@ -161,6 +162,11 @@ abstract class

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r211750476 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -60,14 +61,20 @@ private[spark] object PythonEvalType { */

[GitHub] spark pull request #21977: SPARK-25004: Add spark.executor.pyspark.memory li...

2018-08-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/21977#discussion_r211752765 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/Client.scala --- @@ -91,6 +91,13 @@ private[spark] class Client(

[GitHub] spark issue #22140: [SPARK-25072][PySpark] Forbid extra value for custom Row

2018-08-21 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/22140 Does it make any sense to have less values than fields? Maybe we should check that they are equal, wdyt @HyukjinKwon ? ---

[GitHub] spark pull request #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread jose-torres
Github user jose-torres commented on a diff in the pull request: https://github.com/apache/spark/pull/22009#discussion_r211739221 --- Diff: external/kafka-0-10-sql/src/test/scala/org/apache/spark/sql/kafka010/KafkaContinuousTest.scala --- @@ -47,7 +47,9 @@ trait

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-21 Thread rdblue
Github user rdblue commented on the issue: https://github.com/apache/spark/pull/21977 @BryanCutler, thanks for taking a look at this. Despite the problems you hit when the limit was set too low, I think we do want to use that limit. It was the most reliable one from our

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21977 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95030/ Test PASSed. ---

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/20611 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21977 **[Test build #95050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95050/testReport)** for PR 21977 at commit

[GitHub] spark issue #20611: [SPARK-23425][SQL]Support wildcard in HDFS path for load...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20611 **[Test build #95030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95030/testReport)** for PR 20611 at commit

[GitHub] spark pull request #20761: [SPARK-20327][CORE][YARN] Add CLI support for YAR...

2018-08-21 Thread vanzin
Github user vanzin commented on a diff in the pull request: https://github.com/apache/spark/pull/20761#discussion_r211733592 --- Diff: resource-managers/yarn/src/main/scala/org/apache/spark/deploy/yarn/ResourceTypeValidator.scala --- @@ -0,0 +1,185 @@ +/* + * Licensed to

[GitHub] spark pull request #22104: [SPARK-24721][SQL] Exclude Python UDFs filters in...

2018-08-21 Thread icexelloss
Github user icexelloss commented on a diff in the pull request: https://github.com/apache/spark/pull/22104#discussion_r211733007 --- Diff: sql/core/src/main/scala/org/apache/spark/sql/execution/python/EvalPythonExec.scala --- @@ -117,15 +117,18 @@ abstract class

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211731921 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -17,24 +17,10 @@

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211731624 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread rednaxelafx
Github user rednaxelafx commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211731392 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211730772 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -17,24 +17,10 @@

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211730445 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Projection.scala --- @@ -180,7 +180,10 @@ object UnsafeProjection

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211729573 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@

[GitHub] spark pull request #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exc...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22154#discussion_r211729173 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/CodeGeneratorWithInterpretedFallback.scala --- @@ -63,7 +49,10 @@

[GitHub] spark pull request #22174: [SPARK-22779][SQL] Fallback config defaults shoul...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/22174#discussion_r211727767 --- Diff: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala --- @@ -1954,14 +1954,7 @@ class SQLConf extends Serializable with

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95023/ Test FAILed. ---

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22112 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22112: [SPARK-23243][Core] Fix RDD.repartition() data correctne...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22112 **[Test build #95023 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95023/testReport)** for PR 22112 at commit

[GitHub] spark issue #22174: [SPARK-22779][SQL] Fallback config defaults should behav...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22174 **[Test build #4284 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4284/testReport)** for PR 22174 at commit

[GitHub] spark issue #21976: [SPARK-24909][core] Always unregister pending partition ...

2018-08-21 Thread tgravescs
Github user tgravescs commented on the issue: https://github.com/apache/spark/pull/21976 Please hold off on merging this am investigating a weirdness I want to make sure isn't caused by this. --- - To unsubscribe,

[GitHub] spark issue #22160: Revert "[SPARK-24418][BUILD] Upgrade Scala to 2.11.12 an...

2018-08-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22160 Yeah that's right, we have had two parallel trees for different Scala versions just for reasons like this. There is less forked code now than with 2.10 vs 2.11. If you think we can merge all the

[GitHub] spark issue #21756: [SPARK-24764] [CORE] Add ServiceLoader implementation fo...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21756 **[Test build #95049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95049/testReport)** for PR 21756 at commit

[GitHub] spark issue #21977: SPARK-25004: Add spark.executor.pyspark.memory limit.

2018-08-21 Thread BryanCutler
Github user BryanCutler commented on the issue: https://github.com/apache/spark/pull/21977 @holdenk pyarrow uses a C++ based memory pool, so I'm not sure how that exactly works with rlimit but I ran some tests and looks like an error is thrown when the limit is set. **with

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95033/ Test FAILed. ---

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22009 **[Test build #95033 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95033/testReport)** for PR 22009 at commit

[GitHub] spark issue #22174: [SPARK-22779][SQL] Fallback config defaults should behav...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22174 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22174: [SPARK-22779] Fallback config defaults should behave lik...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22174 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22174: [SPARK-22779] Fallback config defaults should behave lik...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22174 ok to test --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22174: [SPARK-22779] Fallback config defaults should behave lik...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22174 Can one of the admins verify this patch? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95032/ Test FAILed. ---

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22009 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark pull request #22174: [SPARK-22779] Fallback config defaults should beh...

2018-08-21 Thread GregOwen
GitHub user GregOwen opened a pull request: https://github.com/apache/spark/pull/22174 [SPARK-22779] Fallback config defaults should behave like scalar defaults ## What changes were proposed in this pull request? The value returned by a call to `spark.conf.get(key)` may be

[GitHub] spark issue #22009: [SPARK-24882][SQL] improve data source v2 API

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22009 **[Test build #95032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95032/testReport)** for PR 22009 at commit

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21859 **[Test build #95048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95048/testReport)** for PR 21859 at commit

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22154 **[Test build #95047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95047/testReport)** for PR 22154 at commit

[GitHub] spark issue #21859: [SPARK-24900][SQL]Speed up sort when the dataset is smal...

2018-08-21 Thread kiszk
Github user kiszk commented on the issue: https://github.com/apache/spark/pull/21859 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22154 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22154 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22154 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/21912 **[Test build #95046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95046/testReport)** for PR 21912 at commit

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21912 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21912: [SPARK-24962][SQL] Refactor CodeGenerator.createUnsafeAr...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/21912 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] spark pull request #20725: [SPARK-23555][PYTHON] Add BinaryType support for ...

2018-08-21 Thread gatorsmile
Github user gatorsmile commented on a diff in the pull request: https://github.com/apache/spark/pull/20725#discussion_r211715497 --- Diff: python/pyspark/sql/types.py --- @@ -1597,6 +1598,12 @@ def to_arrow_type(dt): arrow_type = pa.decimal128(dt.precision, dt.scale)

[GitHub] spark issue #20838: [SPARK-23698] Resolve undefined names in Python 3

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/20838 **[Test build #95045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95045/testReport)** for PR 20838 at commit

[GitHub] spark pull request #21451: [SPARK-24296][CORE] Replicate large blocks as a s...

2018-08-21 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/21451 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #21451: [SPARK-24296][CORE] Replicate large blocks as a stream.

2018-08-21 Thread vanzin
Github user vanzin commented on the issue: https://github.com/apache/spark/pull/21451 LGTM. Merging to master. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail:

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/13440 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95018/ Test PASSed. ---

[GitHub] spark issue #22160: Revert "[SPARK-24418][BUILD] Upgrade Scala to 2.11.12 an...

2018-08-21 Thread dbtsai
Github user dbtsai commented on the issue: https://github.com/apache/spark/pull/22160 @srowen Just found Scala 2.12 has a separated REPL implementation which suffered from message order printing issue too. Let me change this PR so the Scala 2.12 version is still on 2.12.6

[GitHub] spark issue #13440: [SPARK-15699] [ML] Implement a Chi-Squared test statisti...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/13440 **[Test build #95018 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95018/testReport)** for PR 13440 at commit

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22154 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22154 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95015/ Test FAILed. ---

[GitHub] spark issue #22154: [SPARK-23711][SPARK-25140][SQL] Catch correct exceptions...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22154 **[Test build #95015 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95015/testReport)** for PR 22154 at commit

[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22136 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/95043/ Test PASSed. ---

[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...

2018-08-21 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22136 **[Test build #95043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/95043/testReport)** for PR 22136 at commit

[GitHub] spark issue #22136: [SPARK-25124][ML]VectorSizeHint setSize and getSize don'...

2018-08-21 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22136 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #21749: [SPARK-24785] [SHELL] Making sure REPL prints Spark UI i...

2018-08-21 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/21749 I am fairly neutral on whether to hack to support multiple Scala 2.11 versions, and 2.11.12, or just leave it at 2.11.8. I think this hack is pretty low risk because I can't see any more 2.11.x

<    1   2   3   4   5   6   7   8   >