[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22809 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97999/ Test PASSed. ---

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22809 **[Test build #97999 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97999/testReport)** for PR 22809 at commit [`6abf844`](https://github.com/apache/spark/commit/6

[GitHub] spark pull request #22455: [SPARK-24572][SPARKR] "eager execution" for R she...

2018-10-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22455 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22794: [SPARK-24570][SQL] Implement Spark own GetTablesOperatio...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22794 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4464/

[GitHub] spark issue #22794: [SPARK-24570][SQL] Implement Spark own GetTablesOperatio...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22794 **[Test build #98002 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98002/testReport)** for PR 22794 at commit [`80a8d21`](https://github.com/apache/spark/commit/80

[GitHub] spark issue #22794: [SPARK-24570][SQL] Implement Spark own GetTablesOperatio...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22794 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22455: [SPARK-24572][SPARKR] "eager execution" for R shell, IDE

2018-10-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22455 LGTM. merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: re

[GitHub] spark pull request #22810: [SPARK-24516][K8S] Change Python default to Pytho...

2018-10-24 Thread asfgit
Github user asfgit closed the pull request at: https://github.com/apache/spark/pull/22810 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark issue #22810: [SPARK-24516][K8S] Change Python default to Python3

2018-10-24 Thread felixcheung
Github user felixcheung commented on the issue: https://github.com/apache/spark/pull/22810 merged to master --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark pull request #22815: [SPARK-25821][SQL] Remove SQLContext methods depr...

2018-10-24 Thread felixcheung
Github user felixcheung commented on a diff in the pull request: https://github.com/apache/spark/pull/22815#discussion_r228045893 --- Diff: R/pkg/R/SQLContext.R --- @@ -343,7 +343,6 @@ setMethod("toDF", signature(x = "RDD"), #' path <- "path/to/file.json" #' df <- read.jso

[GitHub] spark issue #22621: [SPARK-25602][SQL] SparkPlan.getByteArrayRdd should not ...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22621 Let's say, this can be behaivour changes too since metrics are now changed. Should we update migration guide for safety? --- ---

[GitHub] spark issue #22690: [SPARK-19287][CORE][STREAMING] JavaPairRDD flatMapValues...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22690 cc @cloud-fan and @gatorsmile Should we update migration guide as well? --- - To unsubscribe, e-mail: reviews-unsub

[GitHub] spark issue #22503: [SPARK-25493][SQL] Use auto-detection for CRLF in CSV da...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22503 @justinuang, this might affect existing users application. Although this matches the behaviour to non-miltiline mode, can we explicitly mention it in migration guide? cc @cloud-fan and

[GitHub] spark issue #22747: [SPARK-25760][SQL] Set AddJarCommand return empty

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22747 This looks also external changes to existing application users. Shall we update migration guide? --- - To unsubscribe, e-mai

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22773 Yup, will encourage to update the migration guide in that way. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22773 Yes. The goal of migration guide is for helping end users upgrade their Spark. --- - To unsubscribe, e-mail: reviews-unsubscr

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97996/ Test PASSed. ---

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22754 **[Test build #97996 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97996/testReport)** for PR 22754 at commit [`b2ca621`](https://github.com/apache/spark/commit/b

[GitHub] spark issue #22512: [SPARK-25498][SQL] InterpretedMutableProjection should h...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22512 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97998/ Test FAILed. ---

[GitHub] spark issue #22512: [SPARK-25498][SQL] InterpretedMutableProjection should h...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22512 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22512: [SPARK-25498][SQL] InterpretedMutableProjection should h...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22512 **[Test build #97998 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97998/testReport)** for PR 22512 at commit [`aec5a53`](https://github.com/apache/spark/commit/a

[GitHub] spark issue #22728: [SPARK-25736][SQL][TEST] add tests to verify the behavio...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22728 (From https://github.com/apache/spark/pull/22773#issuecomment-432917994) @gatorsmile and @cloud-fan, let's say this will break `DESCRIBE FUNCTION EXTENDED`. Should we update migration guide as w

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22815 BTW, should we update migration guide too? --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread daviddingly
Github user daviddingly commented on the issue: https://github.com/apache/spark/pull/22803 > Thank you for your first contribution, @daviddingly > Could you [create issues ticket](https://issues.apache.org/jira/projects/SPARK) first and change the pr title to `[SPARK-x][BUILD]

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22773 Sure, so for clarification, we will document everything that affects to external users application, right? --- - To unsubscr

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22514 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97995/ Test PASSed. ---

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22514 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22773 All the things that could break the existing user applications should be documented in the migration guide. This will simplify the system upgrade of our end users. --- ---

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22514 **[Test build #97995 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97995/testReport)** for PR 22514 at commit [`5780a5e`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22775 **[Test build #98001 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98001/testReport)** for PR 22775 at commit [`dbf9cab`](https://github.com/apache/spark/commit/db

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22775 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22775 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4463/

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22775 @cloud-fan, looks we are going to start another RC. Would you mind if I ask to take a quick look before the new RC? --- - To

[GitHub] spark issue #22775: [SPARK-24709][SQL][FOLLOW-UP] Make schema_of_json's inpu...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22775 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: review

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22773 My impression so far was that we note things at migration notes when they are improvements (not bugs), and non-trivial and related to backward compatibility. Shall we clarify what to do

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22821 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22821 cc @dongjoon-hyun @gatorsmile @rxin --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22821 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4462/

[GitHub] spark issue #22821: [SPARK-25832][SQL] remove newly added map related functi...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22821 **[Test build #98000 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/98000/testReport)** for PR 22821 at commit [`7e919e3`](https://github.com/apache/spark/commit/7e

[GitHub] spark pull request #22821: [SPARK-25832][] remove newly added map related fu...

2018-10-24 Thread cloud-fan
GitHub user cloud-fan opened a pull request: https://github.com/apache/spark/pull/22821 [SPARK-25832][] remove newly added map related functions from FunctionRegistry ## What changes were proposed in this pull request? (Please fill in changes proposed in this fix)

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22773 I think the new names are better and expected, though it's safer to mention it in the migration guide in case some users care about it. --- --

[GitHub] spark issue #22798: [SPARK-25803][K8S] Fix docker-image-tool.sh -n option

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22798 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97993/ Test PASSed. ---

[GitHub] spark issue #22798: [SPARK-25803][K8S] Fix docker-image-tool.sh -n option

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22798 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22798: [SPARK-25803][K8S] Fix docker-image-tool.sh -n option

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22798 **[Test build #97993 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97993/testReport)** for PR 22798 at commit [`a07d105`](https://github.com/apache/spark/commit/a

[GitHub] spark pull request #22820: [SPARK-25828][K8S][BUILD] Bumping Kubernetes-Clie...

2018-10-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22820#discussion_r228030940 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/features/MountVolumesFeatureStep.scala --- @@ -56,8 +56,10 @@ pri

[GitHub] spark pull request #22820: [SPARK-25828][K8S][BUILD] Bumping Kubernetes-Clie...

2018-10-24 Thread liyinan926
Github user liyinan926 commented on a diff in the pull request: https://github.com/apache/spark/pull/22820#discussion_r228030819 --- Diff: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/KubernetesUtils.scala --- @@ -157,7 +157,10 @@ private[spark] obj

[GitHub] spark issue #22810: [SPARK-24516][K8S] Change Python default to Python3

2018-10-24 Thread holdenk
Github user holdenk commented on the issue: https://github.com/apache/spark/pull/22810 Let me look on Friday during weekly review time. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additiona

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22773 BTW, it's closer to bug rather then improvement tho. `from_json` should have default name `from_json` rather then `jsontostructs` - end users would have no idea why it's called `jsontostructs`.

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22773 That's the exact issue I raised before and we ended up with not keeping the compatibility in column names. @cloud-fan and @hvanhovell. --- -

[GitHub] spark issue #22773: [SPARK-25785][SQL] Add prettyNames for from_json, to_jso...

2018-10-24 Thread gatorsmile
Github user gatorsmile commented on the issue: https://github.com/apache/spark/pull/22773 This is an external change. This needs a migration guide update. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apac

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22809 **[Test build #97999 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97999/testReport)** for PR 22809 at commit [`6abf844`](https://github.com/apache/spark/commit/6a

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22809 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22809 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4461/

[GitHub] spark issue #22749: [SPARK-25746][SQL] Refactoring ExpressionEncoder to get ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22749 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22749: [SPARK-25746][SQL] Refactoring ExpressionEncoder to get ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22749 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97991/ Test PASSed. ---

[GitHub] spark issue #22749: [SPARK-25746][SQL] Refactoring ExpressionEncoder to get ...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22749 **[Test build #97991 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97991/testReport)** for PR 22749 at commit [`c00d5e4`](https://github.com/apache/spark/commit/c

[GitHub] spark issue #22819: [BUILD] Close stale PRs

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22819 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97990/ Test PASSed. ---

[GitHub] spark issue #22819: [BUILD] Close stale PRs

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22819 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22803 **[Test build #4396 has started](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4396/testReport)** for PR 22803 at commit [`8eaf81e`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22819: [BUILD] Close stale PRs

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22819 **[Test build #97990 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97990/testReport)** for PR 22819 at commit [`595ebe5`](https://github.com/apache/spark/commit/5

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread srowen
Github user srowen commented on the issue: https://github.com/apache/spark/pull/22803 Whoa, never seen a JVM crash like that! ``` [CodeBlob (0x7f6fc018add0)] Framesize: 2 Runtime Stub (0x7f6fc018add0): handle_exception_from_callee Runtime1 stub Could no

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-24 Thread dilipbiswal
Github user dilipbiswal commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228020366 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -727,4 +728,67 @@ class DataFrameAggregateSuite extends Quer

[GitHub] spark issue #22512: [SPARK-25498][SQL] InterpretedMutableProjection should h...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22512 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22512: [SPARK-25498][SQL] InterpretedMutableProjection should h...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22512 **[Test build #97998 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97998/testReport)** for PR 22512 at commit [`aec5a53`](https://github.com/apache/spark/commit/ae

[GitHub] spark issue #22512: [SPARK-25498][SQL] InterpretedMutableProjection should h...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22512 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4460/

[GitHub] spark issue #22813: [SPARK-25818][CORE] WorkDirCleanup should only remove th...

2018-10-24 Thread ouyangxiaochen
Github user ouyangxiaochen commented on the issue: https://github.com/apache/spark/pull/22813 Yes, it happened in our real environment. The scenario as follows: Some disk corruption in the production cluster which is normal. SPARK_LOCAL_DIRS = /data1/bigdata/spark/tmp, dis

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22803 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22803 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97997/ Test FAILed. ---

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22803 **[Test build #97997 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97997/testReport)** for PR 22803 at commit [`8eaf81e`](https://github.com/apache/spark/commit/8

[GitHub] spark issue #22755: [SPARK-25755][SQL][Test] Supplementation of non-CodeGen ...

2018-10-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22755 +1 --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apach

[GitHub] spark issue #22795: [SPARK-25798][PYTHON] Internally document type conversio...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22795 Thanks @viirya!! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-10-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r228013784 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -45,6 +46,11 @@ case class CreateHiveTa

[GitHub] spark issue #19788: [SPARK-9853][Core] Optimize shuffle fetch of contiguous ...

2018-10-24 Thread yucai
Github user yucai commented on the issue: https://github.com/apache/spark/pull/19788 @cloud-fan , exception screenshot. Let me know if you want any change. ![image](https://user-images.githubusercontent.com/2989575/47471258-1793ce00-d83c-11e8-90bf-107865fc9032.png) !

[GitHub] spark issue #22809: [SPARK-19851][SQL] Add support for EVERY and ANY (SOME) ...

2018-10-24 Thread cloud-fan
Github user cloud-fan commented on the issue: https://github.com/apache/spark/pull/22809 LGTM --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apa

[GitHub] spark pull request #22809: [SPARK-19851][SQL] Add support for EVERY and ANY ...

2018-10-24 Thread cloud-fan
Github user cloud-fan commented on a diff in the pull request: https://github.com/apache/spark/pull/22809#discussion_r228013503 --- Diff: sql/core/src/test/scala/org/apache/spark/sql/DataFrameAggregateSuite.scala --- @@ -727,4 +728,67 @@ class DataFrameAggregateSuite extends QueryT

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22803 **[Test build #97997 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97997/testReport)** for PR 22803 at commit [`8eaf81e`](https://github.com/apache/spark/commit/8e

[GitHub] spark issue #22795: [SPARK-25798][PYTHON] Internally document type conversio...

2018-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22795 Sorry for late. This LGTM. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread wangyum
Github user wangyum commented on the issue: https://github.com/apache/spark/pull/22803 retest this please --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #22795: [SPARK-25798][PYTHON] Internally document type co...

2018-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22795#discussion_r228012874 --- Diff: python/pyspark/sql/functions.py --- @@ -3023,6 +3023,42 @@ def pandas_udf(f=None, returnType=None, functionType=None): conversion on r

[GitHub] spark issue #22723: [SPARK-25729][CORE]It is better to replace `minPartition...

2018-10-24 Thread 10110346
Github user 10110346 commented on the issue: https://github.com/apache/spark/pull/22723 What you say is reasonable. But from the perspective of resource utilization, I think it is better to replace `minPartitions` with `defaultParallelism`, we can see `BinaryFileRDD` and `createNo

[GitHub] spark issue #22803: [SPARK-25808][BUILD] Upgrade jsr305 version from 1.3.9 t...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22803 **[Test build #4395 has finished](https://amplab.cs.berkeley.edu/jenkins/job/NewSparkPullRequestBuilder/4395/testReport)** for PR 22803 at commit [`8eaf81e`](https://github.com/apache/spark/commit/

[GitHub] spark pull request #22816: [SPARK-25822][PySpark]Fix a race condition when r...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22816#discussion_r228012237 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -114,7 +114,7 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark pull request #22816: [SPARK-25822][PySpark]Fix a race condition when r...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on a diff in the pull request: https://github.com/apache/spark/pull/22816#discussion_r228012035 --- Diff: core/src/main/scala/org/apache/spark/api/python/PythonRunner.scala --- @@ -114,7 +114,7 @@ private[spark] abstract class BasePythonRunner[IN,

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22754 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22754 **[Test build #97996 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97996/testReport)** for PR 22754 at commit [`b2ca621`](https://github.com/apache/spark/commit/b2

[GitHub] spark issue #22754: [SPARK-25776][CORE]The disk write buffer size must be gr...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22754 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4459/

[GitHub] spark issue #22237: [SPARK-25243][SQL] Use FailureSafeParser in from_json

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22237 Thanks all!! --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...

[GitHub] spark issue #22795: [SPARK-25798][PYTHON] Internally document type conversio...

2018-10-24 Thread HyukjinKwon
Github user HyukjinKwon commented on the issue: https://github.com/apache/spark/pull/22795 Thanks, @BryanCutler. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: rev

[GitHub] spark pull request #22514: [SPARK-25271][SQL] Hive ctas commands should use ...

2018-10-24 Thread viirya
Github user viirya commented on a diff in the pull request: https://github.com/apache/spark/pull/22514#discussion_r228009289 --- Diff: sql/hive/src/main/scala/org/apache/spark/sql/hive/execution/CreateHiveTableAsSelectCommand.scala --- @@ -45,6 +46,11 @@ case class CreateHiveTable

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22514 Merged build finished. Test PASSed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22815 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread viirya
Github user viirya commented on the issue: https://github.com/apache/spark/pull/22514 retest this please. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h.

[GitHub] spark pull request #22754: [SPARK-25776][CORE]The disk write buffer size mus...

2018-10-24 Thread 10110346
Github user 10110346 commented on a diff in the pull request: https://github.com/apache/spark/pull/22754#discussion_r228009755 --- Diff: core/src/main/scala/org/apache/spark/internal/config/package.scala --- @@ -495,8 +495,8 @@ package object config { ConfigBuilder("spark.

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22815 **[Test build #97988 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97988/testReport)** for PR 22815 at commit [`98ef77e`](https://github.com/apache/spark/commit/9

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread SparkQA
Github user SparkQA commented on the issue: https://github.com/apache/spark/pull/22514 **[Test build #97995 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/97995/testReport)** for PR 22514 at commit [`5780a5e`](https://github.com/apache/spark/commit/57

[GitHub] spark issue #22514: [SPARK-25271][SQL] Hive ctas commands should use data so...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22514 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/testing-k8s-prb-make-spark-distribution-unified/4458/

[GitHub] spark issue #22815: [SPARK-25821][SQL] Remove SQLContext methods deprecated ...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22815 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/97988/ Test FAILed. ---

[GitHub] spark issue #22820: [SPARK-25828][K8S][BUILD] Bumping Kubernetes-Client vers...

2018-10-24 Thread AmplabJenkins
Github user AmplabJenkins commented on the issue: https://github.com/apache/spark/pull/22820 Merged build finished. Test FAILed. --- - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional comma

  1   2   3   4   >