[GitHub] [spark] Ngone51 commented on pull request #33118: [WIP][SPARK-35917][SHUFFLE][CORE]Disable push-based shuffle feature to prevent it from being used

Ngone51 commented on pull request #33118: URL: https://github.com/apache/spark/pull/33118#issuecomment-871147518 I see. Make sense to me. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spec

[GitHub] [spark] SparkQA removed a comment on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA removed a comment on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871031005 **[Test build #140414 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140414/testReport)** for PR 32883 at commit [`fd44620`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

SparkQA commented on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-871147339 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44944/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33136: [SPARK-35932][SQL] Support extracting hour/minute/second from timestamp without time zone

SparkQA commented on pull request #33136: URL: https://github.com/apache/spark/pull/33136#issuecomment-871146874 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44941/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871146611 **[Test build #140414 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140414/testReport)** for PR 32883 at commit [`fd44620`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #33150: [SPARK-35948][INFRA] Simplify release scripts by removing Spark 2.4/Java7 parts

dongjoon-hyun commented on a change in pull request #33150: URL: https://github.com/apache/spark/pull/33150#discussion_r661181810 ## File path: dev/create-release/release-util.sh ## @@ -226,11 +226,5 @@ function init_maven_sbt { MVN="build/mvn -B" MVN_EXTRA_OPTS= SBT_O

[GitHub] [spark] SparkQA commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871145839 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44942/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33137: [SPARK-35935][SQL] Prevent failure of `MSCK REPAIR TABLE` on table refreshing

SparkQA commented on pull request #33137: URL: https://github.com/apache/spark/pull/33137#issuecomment-871145684 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44943/ -- This is an automated message from the A

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #33150: [SPARK-35948][INFRA] Simplify release scripts by removing Spark 2.4/Java7 parts

dongjoon-hyun edited a comment on pull request #33150: URL: https://github.com/apache/spark/pull/33150#issuecomment-871143724 cc @gengliangwang , @srowen , @cloud-fan , @HyukjinKwon , @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please l

[GitHub] [spark] ulysses-you closed pull request #33015: [SPARK-35853][SQL] Remark the shuffle origin to ENSURE_REQUIREMENTS as far as possible

ulysses-you closed pull request #33015: URL: https://github.com/apache/spark/pull/33015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] SparkQA commented on pull request #33150: [SPARK-35948][INFRA] Simplify release scripts by removing Spark 2.4/Java7 parts

SparkQA commented on pull request #33150: URL: https://github.com/apache/spark/pull/33150#issuecomment-871143765 **[Test build #140432 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140432/testReport)** for PR 33150 at commit [`538c86a`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #33150: [SPARK-35948][INFRA] Simplify release scripts by removing Spark 2.4/Java7 parts

dongjoon-hyun commented on pull request #33150: URL: https://github.com/apache/spark/pull/33150#issuecomment-871143724 cc @gengliangwang , @srowen , @cloud-fan , @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [spark] AmplabJenkins commented on pull request #33151: fixes bug for sparkContext stopped on client mode

AmplabJenkins commented on pull request #33151: URL: https://github.com/apache/spark/pull/33151#issuecomment-871143586 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661178449 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] mridulm commented on a change in pull request #33034: WIP: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

mridulm commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r661178449 ## File path: common/network-common/src/main/java/org/apache/spark/network/client/TransportClient.java ## @@ -222,7 +223,7 @@ public void sendMergedBlock

[GitHub] [spark] sunpe opened a new pull request #33151: fixes bug for sparkContext stopped on client mode

sunpe opened a new pull request #33151: URL: https://github.com/apache/spark/pull/33151 ### What changes were proposed in this pull request? Fixes a bug that the spark context was stopped while application is started on client mode. ### Why are the changes needed?

[GitHub] [spark] dongjoon-hyun opened a new pull request #33150: [SPARK-35948][INFRA] Simplify release scripts by removing Spark 2.4/Java7 parts

dongjoon-hyun opened a new pull request #33150: URL: https://github.com/apache/spark/pull/33150 ### What changes were proposed in this pull request? This PR aims to clean up Spark 2.4 and Java7 code path from the release scripts. ### Why are the changes needed? To simpl

[GitHub] [spark] MaxGekk closed pull request #33137: [SPARK-35935][SQL] Prevent failure of `MSCK REPAIR TABLE` on table refreshing

MaxGekk closed pull request #33137: URL: https://github.com/apache/spark/pull/33137 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] MaxGekk commented on pull request #33137: [SPARK-35935][SQL] Prevent failure of `MSCK REPAIR TABLE` on table refreshing

MaxGekk commented on pull request #33137: URL: https://github.com/apache/spark/pull/33137#issuecomment-871139935 GA passed. Merging to master. Thank you, @cloud-fan for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

AmplabJenkins commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871138114 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44945/ -- T

[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

SparkQA commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871138101 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44945/ -- This

[GitHub] [spark] SparkQA commented on pull request #33149: [SPARK-35947][INFRA] Increase JVM stack size in release-build.sh

SparkQA commented on pull request #33149: URL: https://github.com/apache/spark/pull/33149#issuecomment-871136536 **[Test build #140431 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140431/testReport)** for PR 33149 at commit [`f1fa0e2`](https://github.com

[GitHub] [spark] mridulm commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

mridulm commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871136009 Based on my limited understanding of standalone, this looks good to me. Would like others to comment as well @Ngone51 :-) -- This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

AmplabJenkins removed a comment on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871134686 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44938/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

AmplabJenkins removed a comment on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871134690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44939/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

AmplabJenkins removed a comment on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871134692 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

AmplabJenkins removed a comment on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871134689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140411/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871134687 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] mridulm commented on pull request #33118: [WIP][SPARK-35917][SHUFFLE][CORE]Disable push-based shuffle feature to prevent it from being used

mridulm commented on pull request #33118: URL: https://github.com/apache/spark/pull/33118#issuecomment-871135005 @Ngone51 The problem is that fixing the two correctness pending issues requires protocol changes - which means serde issues if 3.3 (where this is fixed) tries to work with 3.2 (

[GitHub] [spark] AmplabJenkins commented on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

AmplabJenkins commented on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871134686 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44938/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

AmplabJenkins commented on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871134690 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44939/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

AmplabJenkins commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871134696 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

AmplabJenkins commented on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871134689 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140411/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871134691 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] dongjoon-hyun commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

dongjoon-hyun commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871133359 Thanks. Ya, let's try with the latest one. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] MaxGekk closed pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

MaxGekk closed pull request #33143: URL: https://github.com/apache/spark/pull/33143 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] dongjoon-hyun commented on pull request #33149: [SPARK-35947][INFRA] Increase JVM stack size in release-build.sh

dongjoon-hyun commented on pull request #33149: URL: https://github.com/apache/spark/pull/33149#issuecomment-871131578 How do you think about this, @gengliangwang and @HyukjinKwon ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [spark] MaxGekk commented on pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

MaxGekk commented on pull request #33143: URL: https://github.com/apache/spark/pull/33143#issuecomment-871131271 Thanks all for reviews. Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] SparkQA commented on pull request #33136: [SPARK-35932][SQL] Support extracting hour/minute/second from timestamp without time zone

SparkQA commented on pull request #33136: URL: https://github.com/apache/spark/pull/33136#issuecomment-871131157 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44941/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33137: [SPARK-35935][SQL] Prevent failure of `MSCK REPAIR TABLE` on table refreshing

SparkQA commented on pull request #33137: URL: https://github.com/apache/spark/pull/33137#issuecomment-871130009 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44943/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

SparkQA commented on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-871129881 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44944/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun opened a new pull request #33149: [SPARK-35947][INFRA] Increase JVM stack size in release-build.sh

dongjoon-hyun opened a new pull request #33149: URL: https://github.com/apache/spark/pull/33149 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### H

[GitHub] [spark] SparkQA commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871128957 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44942/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA removed a comment on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871070332 **[Test build #140422 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140422/testReport)** for PR 32850 at commit [`6d8e915`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871127360 **[Test build #140422 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140422/testReport)** for PR 32850 at commit [`6d8e915`](https://github.co

[GitHub] [spark] wangyum commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

wangyum commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871125876 Last question. How do other common modules use these errors? For example: spark-unsafe, spark-network-common. -- This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871035782 **[Test build #140415 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140415/testReport)** for PR 33093 at commit [`673008d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871125623 **[Test build #140415 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140415/testReport)** for PR 33093 at commit [`673008d`](https://github.co

[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

viirya commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871122684 I'm not against to this point. I can change to latest guava and see what CI tells. -- This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] SparkQA commented on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

SparkQA commented on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871120244 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44938/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

SparkQA commented on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871119876 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44939/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871119589 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44937/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

SparkQA removed a comment on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871011016 **[Test build #140411 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140411/testReport)** for PR 32552 at commit [`29dd475`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871030899 **[Test build #140413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140413/testReport)** for PR 33093 at commit [`f221358`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

SparkQA commented on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871118314 **[Test build #140411 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140411/testReport)** for PR 32552 at commit [`29dd475`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871117789 **[Test build #140413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140413/testReport)** for PR 33093 at commit [`f221358`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #33130: [SPARK-35928][BUILD] Upgrade ASM to 9.1

dongjoon-hyun commented on pull request #33130: URL: https://github.com/apache/spark/pull/33130#issuecomment-871116923 Here is some update. Although the Jenkins status is super noisy due to the timeout, we got the green light for the following at least. - Maven with Hadoop 2.7/Java11: h

[GitHub] [spark] gengliangwang closed pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

gengliangwang closed pull request #33138: URL: https://github.com/apache/spark/pull/33138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] gengliangwang commented on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

gengliangwang commented on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871116358 Merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

[GitHub] [spark] cfmcgrady commented on a change in pull request #33146: [WIP][SPARK-35912][SQL] Fix cast struct contains null value to string

cfmcgrady commented on a change in pull request #33146: URL: https://github.com/apache/spark/pull/33146#discussion_r661147901 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -1098,29 +1100,37 @@ abstract class CastBase exten

[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

SparkQA commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871110977 **[Test build #140430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140430/testReport)** for PR 29326 at commit [`4e6da9c`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

AmplabJenkins removed a comment on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-870942164 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140363/ -

[GitHub] [spark] SparkQA commented on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

SparkQA commented on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-871110491 **[Test build #140429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140429/testReport)** for PR 33105 at commit [`ae4531b`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33137: [SPARK-35935][SQL] Prevent failure of `MSCK REPAIR TABLE` on table refreshing

SparkQA commented on pull request #33137: URL: https://github.com/apache/spark/pull/33137#issuecomment-871110470 **[Test build #140428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140428/testReport)** for PR 33137 at commit [`85954ae`](https://github.com

[GitHub] [spark] dongjoon-hyun commented on pull request #33133: [SPARK-35930][BUILD] Upgrade kinesis-client to 1.14.4

dongjoon-hyun commented on pull request #33133: URL: https://github.com/apache/spark/pull/33133#issuecomment-871110316 Since SPARK-34549 is reverted, I'll close this PR, @sarutak . Thank you for checking. After Guava issue is resolved, feel free to reopen this. -- This is an autom

[GitHub] [spark] dongjoon-hyun closed pull request #33133: [SPARK-35930][BUILD] Upgrade kinesis-client to 1.14.4

dongjoon-hyun closed pull request #33133: URL: https://github.com/apache/spark/pull/33133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins removed a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871109578 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44940/

[GitHub] [spark] AmplabJenkins commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871109578 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44940/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

AmplabJenkins removed a comment on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871108990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins removed a comment on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871071885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

AmplabJenkins removed a comment on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871108995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins removed a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871108992 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140425/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

AmplabJenkins removed a comment on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871108991 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

AmplabJenkins commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871108990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871108994 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44931/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871108992 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140425/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

AmplabJenkins commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871108997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

AmplabJenkins commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871109000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] SparkQA commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

SparkQA commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871107428 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44936/ -- This

[GitHub] [spark] SparkQA commented on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

SparkQA commented on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871106259 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44938/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

SparkQA commented on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871106119 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44939/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871105704 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44937/ -- This is an automated message from the Apache

[GitHub] [spark] wangyum commented on a change in pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

wangyum commented on a change in pull request #33105: URL: https://github.com/apache/spark/pull/33105#discussion_r661139555 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -24,7 +24,7 @@ import org.apache.spark.sql.catal

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

HyukjinKwon commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661139243 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions /

[GitHub] [spark] viirya commented on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

viirya commented on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-871104576 > Can you briefly introduce your idea? Sorting by height is stable and fast now. Basically, the steps are: 1. Propagate the `SubExprEliminationState` map for all sub

[GitHub] [spark] HyukjinKwon edited a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

HyukjinKwon edited a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871104226 Thank you @dongjoon-hyun !! > Since this is only at utill.py, we are still able to turn off the pined mode by PYSPARK_PIN_THREAD=false via java_gateway and Py4J

[GitHub] [spark] HyukjinKwon commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

HyukjinKwon commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871104226 Thank you @dongjoon-hyun !! > Since this is only at utill.py, we are still able to turn off the pined mode by PYSPARK_PIN_THREAD=false via java_gateway and Py4JServer.

[GitHub] [spark] dongjoon-hyun commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

dongjoon-hyun commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871104019 Thank you, @HyukjinKwon and @WeichenXu123 . Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] dongjoon-hyun closed pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

dongjoon-hyun closed pull request #33147: URL: https://github.com/apache/spark/pull/33147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] viirya closed pull request #32980: [SPARK-35829][SQL] Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

viirya closed pull request #32980: URL: https://github.com/apache/spark/pull/32980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubsc

[GitHub] [spark] viirya commented on pull request #32980: [SPARK-35829][SQL] Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

viirya commented on pull request #32980: URL: https://github.com/apache/spark/pull/32980#issuecomment-871102030 Thanks for review! Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

viirya commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871101530 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

[GitHub] [spark] SparkQA commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

SparkQA commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871100812 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44932/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871100625 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44933/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

SparkQA removed a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871088117 **[Test build #140425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140425/testReport)** for PR 33147 at commit [`27199ac`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

SparkQA commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871099454 **[Test build #140425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140425/testReport)** for PR 33147 at commit [`27199ac`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA removed a comment on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871051176 **[Test build #140418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140418/testReport)** for PR 33028 at commit [`54ed99e`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA removed a comment on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871051277 **[Test build #140419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140419/testReport)** for PR 32850 at commit [`d73bb83`](https://gi

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871098737 **[Test build #140419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140419/testReport)** for PR 32850 at commit [`d73bb83`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871098634 **[Test build #140418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140418/testReport)** for PR 33028 at commit [`54ed99e`](https://github.co

  1   2   3   4   5   6   7   8   9   10   >