[GitHub] [spark] SparkQA removed a comment on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

SparkQA removed a comment on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871011016 **[Test build #140411 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140411/testReport)** for PR 32552 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871030899 **[Test build #140413 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140413/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

SparkQA commented on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871118314 **[Test build #140411 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140411/testReport)** for PR 32552 at commit

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871117789 **[Test build #140413 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140413/testReport)** for PR 33093 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #33130: [SPARK-35928][BUILD] Upgrade ASM to 9.1

dongjoon-hyun commented on pull request #33130: URL: https://github.com/apache/spark/pull/33130#issuecomment-871116923 Here is some update. Although the Jenkins status is super noisy due to the timeout, we got the green light for the following at least. - Maven with Hadoop 2.7/Java11:

[GitHub] [spark] gengliangwang closed pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

gengliangwang closed pull request #33138: URL: https://github.com/apache/spark/pull/33138 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] gengliangwang commented on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

gengliangwang commented on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871116358 Merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cfmcgrady commented on a change in pull request #33146: [WIP][SPARK-35912][SQL] Fix cast struct contains null value to string

cfmcgrady commented on a change in pull request #33146: URL: https://github.com/apache/spark/pull/33146#discussion_r661147901 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -1098,29 +1100,37 @@ abstract class CastBase

[GitHub] [spark] SparkQA commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

SparkQA commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871110977 **[Test build #140430 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140430/testReport)** for PR 29326 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

AmplabJenkins removed a comment on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-870942164 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140363/

[GitHub] [spark] SparkQA commented on pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

SparkQA commented on pull request #33105: URL: https://github.com/apache/spark/pull/33105#issuecomment-871110491 **[Test build #140429 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140429/testReport)** for PR 33105 at commit

[GitHub] [spark] SparkQA commented on pull request #33137: [SPARK-35935][SQL] Prevent failure of `MSCK REPAIR TABLE` on table refreshing

SparkQA commented on pull request #33137: URL: https://github.com/apache/spark/pull/33137#issuecomment-871110470 **[Test build #140428 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140428/testReport)** for PR 33137 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #33133: [SPARK-35930][BUILD] Upgrade kinesis-client to 1.14.4

dongjoon-hyun commented on pull request #33133: URL: https://github.com/apache/spark/pull/33133#issuecomment-871110316 Since SPARK-34549 is reverted, I'll close this PR, @sarutak . Thank you for checking. After Guava issue is resolved, feel free to reopen this. -- This is an

[GitHub] [spark] dongjoon-hyun closed pull request #33133: [SPARK-35930][BUILD] Upgrade kinesis-client to 1.14.4

dongjoon-hyun closed pull request #33133: URL: https://github.com/apache/spark/pull/33133 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins removed a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871109578 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44940/

[GitHub] [spark] AmplabJenkins commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871109578 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44940/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

AmplabJenkins removed a comment on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871108990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins removed a comment on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871071885 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

AmplabJenkins removed a comment on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871108995 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins removed a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871108992 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140425/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

AmplabJenkins removed a comment on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871108991 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

AmplabJenkins commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871108990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871108994 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44931/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

AmplabJenkins commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871108992 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140425/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

AmplabJenkins commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871108997 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

AmplabJenkins commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871109000 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

SparkQA commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871107428 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44936/ --

[GitHub] [spark] SparkQA commented on pull request #33138: [SPARK-35937][SQL] Extracting date field from timestamp should work in ANSI mode

SparkQA commented on pull request #33138: URL: https://github.com/apache/spark/pull/33138#issuecomment-871106259 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44938/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

SparkQA commented on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871106119 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44939/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871105704 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44937/ -- This is an automated message from the Apache

[GitHub] [spark] wangyum commented on a change in pull request #33105: [SPARK-35908][SQL] Remove repartition if the child maximum number of rows less than or equal to 1

wangyum commented on a change in pull request #33105: URL: https://github.com/apache/spark/pull/33105#discussion_r661139555 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -24,7 +24,7 @@ import

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

HyukjinKwon commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661139243 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] viirya commented on pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

viirya commented on pull request #33142: URL: https://github.com/apache/spark/pull/33142#issuecomment-871104576 > Can you briefly introduce your idea? Sorting by height is stable and fast now. Basically, the steps are: 1. Propagate the `SubExprEliminationState` map for all

[GitHub] [spark] HyukjinKwon edited a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

HyukjinKwon edited a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871104226 Thank you @dongjoon-hyun !! > Since this is only at utill.py, we are still able to turn off the pined mode by PYSPARK_PIN_THREAD=false via java_gateway and

[GitHub] [spark] HyukjinKwon commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

HyukjinKwon commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871104226 Thank you @dongjoon-hyun !! > Since this is only at utill.py, we are still able to turn off the pined mode by PYSPARK_PIN_THREAD=false via java_gateway and

[GitHub] [spark] dongjoon-hyun commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

dongjoon-hyun commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871104019 Thank you, @HyukjinKwon and @WeichenXu123 . Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] dongjoon-hyun closed pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

dongjoon-hyun closed pull request #33147: URL: https://github.com/apache/spark/pull/33147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] viirya closed pull request #32980: [SPARK-35829][SQL] Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

viirya closed pull request #32980: URL: https://github.com/apache/spark/pull/32980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] viirya commented on pull request #32980: [SPARK-35829][SQL] Clean up evaluates subexpressions and add more flexibility to evaluate particular subexpressoin

viirya commented on pull request #32980: URL: https://github.com/apache/spark/pull/32980#issuecomment-871102030 Thanks for review! Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

viirya commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-871101530 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

SparkQA commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871100812 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44932/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871100625 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44933/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

SparkQA removed a comment on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871088117 **[Test build #140425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140425/testReport)** for PR 33147 at commit

[GitHub] [spark] SparkQA commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

SparkQA commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871099454 **[Test build #140425 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140425/testReport)** for PR 33147 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA removed a comment on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871051176 **[Test build #140418 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140418/testReport)** for PR 33028 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA removed a comment on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871051277 **[Test build #140419 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140419/testReport)** for PR 32850 at commit

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871098737 **[Test build #140419 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140419/testReport)** for PR 32850 at commit

[GitHub] [spark] SparkQA commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871098634 **[Test build #140418 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140418/testReport)** for PR 33028 at commit

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871098074 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44934/ -- This is an automated message from the

[GitHub] [spark] MaxGekk commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

MaxGekk commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661131121 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] SparkQA removed a comment on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA removed a comment on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871091753 **[Test build #140427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140427/testReport)** for PR 32883 at commit

[GitHub] [spark] MaxGekk commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

MaxGekk commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661131121 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871095284 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44931/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871093680 **[Test build #140427 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140427/testReport)** for PR 32883 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

AmplabJenkins commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871093708 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140427/ -- This

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

HyukjinKwon commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661128457 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] SparkQA commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871091753 **[Test build #140427 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140427/testReport)** for PR 32883 at commit

[GitHub] [spark] yaooqinn commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

yaooqinn commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661125753 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] sarutak commented on pull request #33145: Revert "[SPARK-34549][BUILD] Upgrade aws kinesis to 1.14.0 and java sdk 1.11.844"

sarutak commented on pull request #33145: URL: https://github.com/apache/spark/pull/33145#issuecomment-871090622 Let's try to upgrade again once Guava is successfully upgraded. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] HeartSaVioR closed pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

HeartSaVioR closed pull request #33091: URL: https://github.com/apache/spark/pull/33091 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HeartSaVioR commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

HeartSaVioR commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871089628 GA passed. Thanks! Merging to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] cxzl25 commented on a change in pull request #33114: [SPARK-35913][SQL] Create hive permanent function with owner name

cxzl25 commented on a change in pull request #33114: URL: https://github.com/apache/spark/pull/33114#discussion_r661123735 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -925,19 +925,19 @@ private[hive] class HiveClientImpl(

[GitHub] [spark] SparkQA commented on pull request #33136: [SPARK-35932][SQL] Support extracting hour/minute/second from timestamp without time zone

SparkQA commented on pull request #33136: URL: https://github.com/apache/spark/pull/33136#issuecomment-871088160 **[Test build #140426 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140426/testReport)** for PR 33136 at commit

[GitHub] [spark] SparkQA commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server in InheritableThread API

SparkQA commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871088117 **[Test build #140425 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140425/testReport)** for PR 33147 at commit

[GitHub] [spark] SparkQA commented on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

SparkQA commented on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871088106 **[Test build #140424 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140424/testReport)** for PR 33148 at commit

[GitHub] [spark] SparkQA commented on pull request #33091: [SPARK-35896][SS] Include more granular metrics for stateful operators in StreamingQueryProgress

SparkQA commented on pull request #33091: URL: https://github.com/apache/spark/pull/33091#issuecomment-871088025 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44932/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

AmplabJenkins removed a comment on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871087817 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44929/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871087814 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44930/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

AmplabJenkins removed a comment on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871087818 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44926/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-871087815 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33145: Revert "[SPARK-34549][BUILD] Upgrade aws kinesis to 1.14.0 and java sdk 1.11.844"

AmplabJenkins removed a comment on pull request #33145: URL: https://github.com/apache/spark/pull/33145#issuecomment-871087813 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140412/

[GitHub] [spark] AmplabJenkins commented on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

AmplabJenkins commented on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871087818 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44926/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-871087816 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871087814 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44930/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

AmplabJenkins commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871087817 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/44929/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33145: Revert "[SPARK-34549][BUILD] Upgrade aws kinesis to 1.14.0 and java sdk 1.11.844"

AmplabJenkins commented on pull request #33145: URL: https://github.com/apache/spark/pull/33145#issuecomment-871087813 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140412/ -- This

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-871087620 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44935/ --

[GitHub] [spark] SparkQA commented on pull request #33028: [SPARK-35714][FOLLOW-UP][CORE] Use a shared stopping flag for WorkerWatcher to avoid the duplicate System.exit

SparkQA commented on pull request #33028: URL: https://github.com/apache/spark/pull/33028#issuecomment-871087512 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44933/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on pull request #33131: [SPARK-35920][FOLLOWUP][BUILD] Fix Kryo Shaded dependency

dongjoon-hyun commented on pull request #33131: URL: https://github.com/apache/spark/pull/33131#issuecomment-871086897 Thank you for the confirmation, @gaoyajun02 ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] SparkQA commented on pull request #32850: [SPARK-34920][CORE][SQL] Add error classes with SQLSTATE

SparkQA commented on pull request #32850: URL: https://github.com/apache/spark/pull/32850#issuecomment-871085966 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44934/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33145: Revert "[SPARK-34549][BUILD] Upgrade aws kinesis to 1.14.0 and java sdk 1.11.844"

SparkQA removed a comment on pull request #33145: URL: https://github.com/apache/spark/pull/33145#issuecomment-871030849 **[Test build #140412 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140412/testReport)** for PR 33145 at commit

[GitHub] [spark] SparkQA commented on pull request #33145: Revert "[SPARK-34549][BUILD] Upgrade aws kinesis to 1.14.0 and java sdk 1.11.844"

SparkQA commented on pull request #33145: URL: https://github.com/apache/spark/pull/33145#issuecomment-871085470 **[Test build #140412 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140412/testReport)** for PR 33145 at commit

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-871084337 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44931/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32883: [SPARK-35725][SQL] Support optimize skewed partitions in RebalancePartitions

SparkQA commented on pull request #32883: URL: https://github.com/apache/spark/pull/32883#issuecomment-871084038 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44929/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS][WIP] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-871083639 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44930/ -- This is an automated message from the

[GitHub] [spark] c21 commented on a change in pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

c21 commented on a change in pull request #33148: URL: https://github.com/apache/spark/pull/33148#discussion_r661118652 ## File path: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ## @@ -41,7 +42,11 @@ import org.apache.spark.util.Utils *(or

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to `FileCommitProtocol`

HyukjinKwon commented on a change in pull request #33148: URL: https://github.com/apache/spark/pull/33148#discussion_r661117977 ## File path: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ## @@ -41,7 +42,11 @@ import org.apache.spark.util.Utils *

[GitHub] [spark] c21 commented on a change in pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to newly added methods of `FileCommitProtocol`

c21 commented on a change in pull request #33148: URL: https://github.com/apache/spark/pull/33148#discussion_r661117657 ## File path: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ## @@ -107,10 +111,9 @@ abstract class FileCommitProtocol extends

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

HyukjinKwon commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661117394 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to newly added methods of `FileCommitProtocol`

HyukjinKwon commented on a change in pull request #33148: URL: https://github.com/apache/spark/pull/33148#discussion_r661117144 ## File path: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ## @@ -107,10 +111,9 @@ abstract class FileCommitProtocol

[GitHub] [spark] c21 commented on pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to newly added methods of `FileCommitProtocol`

c21 commented on pull request #33148: URL: https://github.com/apache/spark/pull/33148#issuecomment-871081348 cc @HyukjinKwon and @cloud-fan please take a look when you have time, thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] c21 opened a new pull request #33148: [SPARK-33298][CORE][FOLLOWUP] Add Unstable annotation to newly added methods of `FileCommitProtocol`

c21 opened a new pull request #33148: URL: https://github.com/apache/spark/pull/33148 ### What changes were proposed in this pull request? This is the followup from https://github.com/apache/spark/pull/33012#discussion_r659440833, where we want to add `@Unstable` to newly

[GitHub] [spark] yaooqinn commented on a change in pull request #33114: [SPARK-35913][SQL] Create hive permanent function with owner name

yaooqinn commented on a change in pull request #33114: URL: https://github.com/apache/spark/pull/33114#discussion_r661116166 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -925,19 +925,19 @@ private[hive] class

[GitHub] [spark] cloud-fan commented on a change in pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

cloud-fan commented on a change in pull request #33143: URL: https://github.com/apache/spark/pull/33143#discussion_r661116046 ## File path: sql/core/src/main/scala/org/apache/spark/sql/functions.scala ## @@ -2928,31 +2928,6 @@ object functions { // DateTime functions

[GitHub] [spark] MrPowers commented on pull request #33143: Revert "[SPARK-33995][SQL] Expose make_interval as a Scala function"

MrPowers commented on pull request #33143: URL: https://github.com/apache/spark/pull/33143#issuecomment-871080235  Thanks for creating something better @MaxGekk and thanks for the ping @dongjoon-hyun. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] HyukjinKwon commented on pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server if InheritableThread API

HyukjinKwon commented on pull request #33147: URL: https://github.com/apache/spark/pull/33147#issuecomment-871077382 @WeichenXu123 can you take a quick look please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] c21 commented on a change in pull request #33012: [SPARK-33298][CORE] Introduce new API to FileCommitProtocol allow flexible file naming

c21 commented on a change in pull request #33012: URL: https://github.com/apache/spark/pull/33012#discussion_r661112797 ## File path: core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ## @@ -92,6 +92,35 @@ abstract class FileCommitProtocol extends

[GitHub] [spark] HyukjinKwon opened a new pull request #33147: [SPARK-35946][PYTHON] Respect Py4J server if InheritableThread API

HyukjinKwon opened a new pull request #33147: URL: https://github.com/apache/spark/pull/33147 ### What changes were proposed in this pull request? Currently ,we sets the environment variable `PYSPARK_PIN_THREAD` at the client side of `InhertiableThread` API for Py4J

[GitHub] [spark] SparkQA removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-871052721 **[Test build #140420 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140420/testReport)** for PR 32365 at commit

[GitHub] [spark] SparkQA commented on pull request #32552: [SPARK-34819][SQL] MapType supports comparable semantics

SparkQA commented on pull request #32552: URL: https://github.com/apache/spark/pull/32552#issuecomment-871076683 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/44926/ -- This is an automated message from the

  1   2   3   4   5   6   7   8   9   10   >