[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470663720 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470663712 Merged build finished. Test FAILed. This is an automat

[GitHub] [spark] SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470663455 **[Test build #103148 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103148/testReport)**

[GitHub] [spark] AmplabJenkins removed a comment on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 URL: https://github.com/apache/spark/pull/24002#issuecomment-470406127 Can one of the admins verify this patch? Thi

[GitHub] [spark] shaneknapp commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2

2019-03-07 Thread GitBox
shaneknapp commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 URL: https://github.com/apache/spark/pull/24002#issuecomment-470663130 ok to test This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470657594 Test FAILed. Refer to this link for build results (access rights to CI server n

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470657584 Merged build finished. Test FAILed. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470569758 Can one of the admins verify this patch?

[GitHub] [spark] SparkQA removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470567943 **[Test build #103145 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPull

[GitHub] [spark] AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470657584 Merged build finished. Test FAILed. -

[GitHub] [spark] AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470657594 Test FAILed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] SparkQA commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet

2019-03-07 Thread GitBox
SparkQA commented on issue #24010: [SPARK-26439][CORE][WIP] Introduce WorkerOffer reservation mechanism for Barrier TaskSet URL: https://github.com/apache/spark/pull/24010#issuecomment-470657186 **[Test build #103145 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequest

[GitHub] [spark] dilipbiswal commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

2019-03-07 Thread GitBox
dilipbiswal commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-470652269 @dongjoon-hyun One question.. In my understanding, option 2 as per @maropu and @cloud-fan is to not chang

[GitHub] [spark] dilipbiswal edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

2019-03-07 Thread GitBox
dilipbiswal edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-470652269 @dongjoon-hyun One question.. In my understanding, option 2 as per @maropu and @cloud-fan is to no

[GitHub] [spark] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470650586 Test FAILed. Refer to this link for build results (access rights to

[GitHub] [spark] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470650576 Merged build finished. Test FAILed. --

[GitHub] [spark] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470650586 Test FAILed. Refer to this link for build results (access rights to CI serve

[GitHub] [spark] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470650576 Merged build finished. Test FAILed. --

[GitHub] [spark] SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470650501 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/je

[GitHub] [spark] SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470649941 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/

[GitHub] [spark] Jeffwan commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2

2019-03-07 Thread GitBox
Jeffwan commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 URL: https://github.com/apache/spark/pull/24002#issuecomment-470648694 I also successfully finish integration test on v1.10.13 ! I will also try v1.11.7 v1.12.6 and v1.13.3 ``` Run com

[GitHub] [spark] AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470647864 Merged build finished. Test PASSed. -

[GitHub] [spark] SparkQA removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470555115 **[Test build #103143 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/1

[GitHub] [spark] AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470647878 Test PASSed. Refer to this link for build results (access rights to CI server needed): http

[GitHub] [spark] AmplabJenkins commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470647864 Merged build finished. Test PASSed. This

[GitHub] [spark] AmplabJenkins commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470647878 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://ampl

[GitHub] [spark] SparkQA commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
SparkQA commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470647170 **[Test build #103143 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103143/t

[GitHub] [spark] shaneknapp commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2

2019-03-07 Thread GitBox
shaneknapp commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 URL: https://github.com/apache/spark/pull/24002#issuecomment-470646832 testing against the 4.1.2 client passed. after i complete the upgrade on the ubuntu workers, i'll approve and trigger t

[GitHub] [spark] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470645316 Test FAILed. Refer to this link for build results (access rights to

[GitHub] [spark] AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470645313 Merged build finished. Test FAILed. --

[GitHub] [spark] SparkQA removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470644539 **[Test build #103151 has started](https://amplab.cs.berkeley.edu/jenkins/jo

[GitHub] [spark] SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470645300 **[Test build #103151 has finished](https://amplab.cs.berkeley.edu/jenkins/job/Spark

[GitHub] [spark] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470645316 Test FAILed. Refer to this link for build results (access rights to CI serve

[GitHub] [spark] AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470645313 Merged build finished. Test FAILed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support.

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support. URL: https://github.com/apache/spark/pull/23929#issuecomment-470644208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
SparkQA commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470644539 **[Test build #103151 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkP

[GitHub] [spark] AmplabJenkins removed a comment on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support.

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support. URL: https://github.com/apache/spark/pull/23929#issuecomment-470644204 Merged build finished. Test PASSed. This is a

[GitHub] [spark] AmplabJenkins commented on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support.

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support. URL: https://github.com/apache/spark/pull/23929#issuecomment-470644204 Merged build finished. Test PASSed. This is an automa

[GitHub] [spark] AmplabJenkins commented on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support.

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support. URL: https://github.com/apache/spark/pull/23929#issuecomment-470644208 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkele

[GitHub] [spark] holdenk commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown

2019-03-07 Thread GitBox
holdenk commented on issue #19045: [WIP][SPARK-20628][CORE][K8S] Keep track of nodes (/ spot instances) which are going to be shutdown URL: https://github.com/apache/spark/pull/19045#issuecomment-470643637 Jenkins retest this please -

[GitHub] [spark] SparkQA removed a comment on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support.

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support. URL: https://github.com/apache/spark/pull/23929#issuecomment-470540080 **[Test build #103139 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103139/testRep

[GitHub] [spark] SparkQA commented on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support.

2019-03-07 Thread GitBox
SparkQA commented on issue #23929: [SPARK-27022][DSTREAMS] Add kafka delegation token support. URL: https://github.com/apache/spark/pull/23929#issuecomment-470643508 **[Test build #103139 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103139/testReport)**

[GitHub] [spark] squito commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
squito commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470641526 lgtm. merging to 2.3, thanks @Ngone51 ! -

[GitHub] [spark] arunmahadevan commented on a change in pull request #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped

2019-03-07 Thread GitBox
arunmahadevan commented on a change in pull request #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#discussion_r263506634 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala

[GitHub] [spark] AmplabJenkins commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470637149 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470637149 Merged build finished. Test PASSed. --

[GitHub] [spark] AmplabJenkins removed a comment on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470637153 Test PASSed. Refer to this link for build results (acces

[GitHub] [spark] AmplabJenkins commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470637153 Test PASSed. Refer to this link for build results (access rights

[GitHub] [spark] AmplabJenkins removed a comment on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470636389 Test PASSed. Refer to this link for build results (access rights to CI server needed):

[GitHub] [spark] arunmahadevan commented on a change in pull request #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped

2019-03-07 Thread GitBox
arunmahadevan commented on a change in pull request #23912: [SPARK-21029][SS] StreamingQuery should be stopped when the SparkSession is stopped URL: https://github.com/apache/spark/pull/23912#discussion_r263506634 ## File path: core/src/main/scala/org/apache/spark/SparkContext.scala

[GitHub] [spark] AmplabJenkins removed a comment on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470636379 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA removed a comment on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470546035 **[Test build #103142 has started](https://amplab.cs.berkeley.ed

[GitHub] [spark] SparkQA commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
SparkQA commented on issue #24007: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24007#issuecomment-470636539 **[Test build #103142 has finished](https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] AmplabJenkins commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470636389 Test PASSed. Refer to this link for build results (access rights to CI server needed): http

[GitHub] [spark] AmplabJenkins commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470636379 Merged build finished. Test PASSed. --

[GitHub] [spark] SparkQA commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
SparkQA commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470634507 **[Test build #103150 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/10

[GitHub] [spark] dongjoon-hyun commented on issue #23985: [SQL][MINOR] List SparkSql reserved keywords in alphabet order

2019-03-07 Thread GitBox
dongjoon-hyun commented on issue #23985: [SQL][MINOR] List SparkSql reserved keywords in alphabet order URL: https://github.com/apache/spark/pull/23985#issuecomment-470634421 Is it a good time to transform into PostgrSQL style? For me, +1 for that kind of transition for the future.

[GitHub] [spark] attilapiros commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
attilapiros commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470634146 jenkins retest this please This is

[GitHub] [spark] shaneknapp commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2

2019-03-07 Thread GitBox
shaneknapp commented on issue #24002: [SPARK-26742][K8S] Update Kubernetes-Client version to 4.1.2 URL: https://github.com/apache/spark/pull/24002#issuecomment-470633794 yep, gonna run these changes manually against the k8s integration tests now. main conversation taking place here: htt

[GitHub] [spark] AmplabJenkins removed a comment on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24006#issuecomment-470632177 Test FAILed. Refer to this link for build results (acces

[GitHub] [spark] SparkQA removed a comment on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24006#issuecomment-470543015 **[Test build #103141 has started](https://amplab.cs.berkeley.ed

[GitHub] [spark] AmplabJenkins removed a comment on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24006#issuecomment-470632163 Build finished. Test FAILed. -

[GitHub] [spark] AmplabJenkins commented on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24006#issuecomment-470632177 Test FAILed. Refer to this link for build results (access rights

[GitHub] [spark] AmplabJenkins commented on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24006#issuecomment-470632163 Build finished. Test FAILed. -

[GitHub] [spark] SparkQA commented on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions

2019-03-07 Thread GitBox
SparkQA commented on issue #24006: [SPARK-23433][SPARK-25250] [CORE][BRANCH-2.3] Later created TaskSet should learn about the finished partitions URL: https://github.com/apache/spark/pull/24006#issuecomment-470631812 **[Test build #103141 has finished](https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] attilapiros commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor

2019-03-07 Thread GitBox
attilapiros commented on issue #23560: [SPARK-26632][Spark Core] Separate Thread Configurations of Driver and Executor URL: https://github.com/apache/spark/pull/23560#issuecomment-470631237 Regarding the `old time: Overall Time without Separation (s)` what was the number of threads for ser

[GitHub] [spark] srowen commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
srowen commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470627439 Oh OK, nice trick. The issue here is this is a trivial change and we've already spent a lot of review time and p

[GitHub] [spark] MaxGekk edited a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
MaxGekk edited a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470625725 @srowen I did that by creating new branch from master and merging range of commit from another PR like `

[GitHub] [spark] MaxGekk commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
MaxGekk commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470625725 @srowen I did that by creating new branch from master and merging range of commit from another PR like `git che

[GitHub] [spark] fitermay edited a comment on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
fitermay edited a comment on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#issuecomment-470621102 @srowen @attilapiros After digging into this further I found out what exactly is happening and why sorting he

[GitHub] [spark] fitermay edited a comment on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
fitermay edited a comment on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#issuecomment-470621102 @srowen @attilapiros After digging into this further I found out what exactly is happening and why sorting he

[GitHub] [spark] fitermay commented on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
fitermay commented on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#issuecomment-470621102 @srowen @attilapiros After digging into this further I found out what exactly is happening and why sorting here caus

[GitHub] [spark] squito commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
squito commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470620310 looks like it might be a real test failure This is an automated message

[GitHub] [spark] srowen commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
srowen commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470620208 Sure, but how do I do that? This is an automate

[GitHub] [spark] srowen commented on a change in pull request #23946: [SPARK-26860][PySpark] [SparkR] Fix for RangeBetween and RowsBetween docs to be in sync with spark documentation

2019-03-07 Thread GitBox
srowen commented on a change in pull request #23946: [SPARK-26860][PySpark] [SparkR] Fix for RangeBetween and RowsBetween docs to be in sync with spark documentation URL: https://github.com/apache/spark/pull/23946#discussion_r263488825 ## File path: python/pyspark/sql/window.py ##

[GitHub] [spark] dongjoon-hyun edited a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
dongjoon-hyun edited a comment on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470618244 Hi, @srowen . Could you preserve the original authorship (@WindCanDie) of https://github.com/apach

[GitHub] [spark] SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
SparkQA commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470618319 **[Test build #103149 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103149/testReport)** f

[GitHub] [spark] dongjoon-hyun commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception

2019-03-07 Thread GitBox
dongjoon-hyun commented on issue #24008: [SPARK-24783][SQL] spark.sql.shuffle.partitions=0 should throw exception URL: https://github.com/apache/spark/pull/24008#issuecomment-470618244 Hi, @srowen . Could you preserve the original authorship (@WindCanDie) of https://github.com/apache/spark

[GitHub] [spark] AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470617538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470617538 Test PASSed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkele

[GitHub] [spark] AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470617533 Merged build finished. Test PASSed. This is an

[GitHub] [spark] AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #24005: [WIP][SPARK-27085][SQL] Migrate CSV to File Data Source V2 URL: https://github.com/apache/spark/pull/24005#issuecomment-470617533 Merged build finished. Test PASSed. This is an automat

[GitHub] [spark] dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

2019-03-07 Thread GitBox
dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-470615225 Then, I agree. +1 for the second option, too. Also, @dilipbiswal , please add migration doc i

[GitHub] [spark] dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

2019-03-07 Thread GitBox
dongjoon-hyun edited a comment on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-470615225 Then, I agree. +1 for the second option, too. Also, @dilipbiswal , please add migration doc i

[GitHub] [spark] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470614850 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.e

[GitHub] [spark] dongjoon-hyun commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface

2019-03-07 Thread GitBox
dongjoon-hyun commented on issue #23982: [SQL][MINOR] Reconcile the join types between data frame and sql interface URL: https://github.com/apache/spark/pull/23982#issuecomment-470615225 Then, I agree. +1 for the second option, too. -

[GitHub] [spark] AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470614842 Merged build finished. Test FAILed. This is an automated

[GitHub] [spark] SparkQA removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470522177 **[Test build #103137 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103137/testReport)** for

[GitHub] [spark] AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470614850 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenki

[GitHub] [spark] AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470614842 Merged build finished. Test FAILed. This is an automated message

[GitHub] [spark] SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB

2019-03-07 Thread GitBox
SparkQA commented on issue #23393: [SPARK-26288][CORE]add initRegisteredExecutorsDB URL: https://github.com/apache/spark/pull/23393#issuecomment-470614445 **[Test build #103137 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103137/testReport)** for PR 233

[GitHub] [spark] srowen commented on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
srowen commented on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#issuecomment-470613691 There's a little detail in the JIRA that's pretty suggestive that this is the bottleneck; if there's a stack trace or more

[GitHub] [spark] AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470609724 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/j

[GitHub] [spark] attilapiros commented on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer

2019-03-07 Thread GitBox
attilapiros commented on issue #23986: [SPARK-27070] Fix performance bug in DefaultPartitionCoalescer URL: https://github.com/apache/spark/pull/23986#issuecomment-470610465 @fitermay It would be very nice to see some some rough numbers regarding the improvement caused by this PR. Could you

[GitHub] [spark] AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470609712 Merged build finished. Test FAILed. This is an automated mes

[GitHub] [spark] SparkQA removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
SparkQA removed a comment on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470561947 **[Test build #103144 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103144/testReport)** for PR

[GitHub] [spark] AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470609724 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//

[GitHub] [spark] AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
AmplabJenkins commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470609712 Merged build finished. Test FAILed. This is an automated message fro

[GitHub] [spark] SparkQA commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery

2019-03-07 Thread GitBox
SparkQA commented on issue #23783: [SPARK-26854][SQL] Support ANY/SOME subquery URL: https://github.com/apache/spark/pull/23783#issuecomment-470609520 **[Test build #103144 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/103144/testReport)** for PR 23783 at

[GitHub] [spark] attilapiros commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData

2019-03-07 Thread GitBox
attilapiros commented on issue #24011: [SPARK-27071][CORE] Expose additional metrics in status.api.v1.StageData URL: https://github.com/apache/spark/pull/24011#issuecomment-470607412 Unfortunately this change also increases the event log size meanwhile there are problems caused by the too

<    3   4   5   6   7   8   9   10   11   12   >