[GitHub] [spark] AngersZhuuuu commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AngersZh commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-888032835 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] Ngone51 commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

Ngone51 commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888031775 Sure. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cloud-fan closed pull request #33542: [SPARK-34399][DOCS][FOLLOWUP] Add docs for the new metrics of task/job commit time

cloud-fan closed pull request #33542: URL: https://github.com/apache/spark/pull/33542 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33542: [SPARK-34399][DOCS][FOLLOWUP] Add docs for the new metrics of task/job commit time

cloud-fan commented on pull request #33542: URL: https://github.com/apache/spark/pull/33542#issuecomment-888031436 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] cloud-fan closed pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

cloud-fan closed pull request #33531: URL: https://github.com/apache/spark/pull/33531 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33531: [SPARK-36312][SQL] ParquetWriterSupport.setSchema should check inner field

cloud-fan commented on pull request #33531: URL: https://github.com/apache/spark/pull/33531#issuecomment-888030550 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] cloud-fan closed pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to AQEShuffleRead in AQE

cloud-fan closed pull request #32776: URL: https://github.com/apache/spark/pull/32776 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to AQEShuffleRead in AQE

cloud-fan commented on pull request #32776: URL: https://github.com/apache/spark/pull/32776#issuecomment-888029450 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-888026840 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46263/ -- This is an automated message from the

[GitHub] [spark] cloud-fan closed pull request #33498: [SPARK-36275][SQL] ResolveAggregateFunctions should works with nested fields

cloud-fan closed pull request #33498: URL: https://github.com/apache/spark/pull/33498 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33498: [SPARK-36275][SQL] ResolveAggregateFunctions should works with nested fields

cloud-fan commented on pull request #33498: URL: https://github.com/apache/spark/pull/33498#issuecomment-888023831 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #33551: [SPARK-36323][SQL] Support ANSI interval literals for TimeWindow

SparkQA commented on pull request #33551: URL: https://github.com/apache/spark/pull/33551#issuecomment-888023713 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46264/ -- This is an automated message from the Apache

[GitHub] [spark] ulysses-you commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

ulysses-you commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677985437 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEUtils.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache

[GitHub] [spark] ulysses-you commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

ulysses-you commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677984868 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala ## @@ -257,13 +255,12 @@ object

[GitHub] [spark] SparkQA commented on pull request #33551: [SPARK-36323][SQL] Support ANSI interval literals for TimeWindow

SparkQA commented on pull request #33551: URL: https://github.com/apache/spark/pull/33551#issuecomment-888020241 **[Test build #141751 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141751/testReport)** for PR 33551 at commit

[GitHub] [spark] ulysses-you commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

ulysses-you commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888019993 cc @dongjoon-hyun @holdenk @vanzin @attilapiros Do you have time to take a look ? thanks -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

AmplabJenkins removed a comment on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-888019902 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46261/

[GitHub] [spark] AmplabJenkins commented on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

AmplabJenkins commented on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-888019902 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46261/ --

[GitHub] [spark] SparkQA commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-888014588 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46263/ -- This is an automated message from the Apache

[GitHub] [spark] mridulm edited a comment on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

mridulm edited a comment on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888013544 @Ngone51 Can you also change the @since for the new classes/enum/tec introduced in #32401 to 3.2.0 as well in this PR please ? This PR is ready to be merged, so let us

[GitHub] [spark] mridulm commented on pull request #33451: [SPARK-36206][CORE] Support shuffle data corruption diagnosis via shuffle checksum

mridulm commented on pull request #33451: URL: https://github.com/apache/spark/pull/33451#issuecomment-888013544 @Ngone51 Can you also change the @since for the new classes/enum/tec introduced in #32401 to 3.2.0 please ? This PR is ready to be merged, so let us make this a feature of

[GitHub] [spark] SparkQA commented on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

SparkQA commented on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-888010813 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46261/ -- This is an automated message from the

[GitHub] [spark] cloud-fan closed pull request #33527: [SPARK-36028][SQL][3.2] Allow Project to host outer references in scalar subqueries

cloud-fan closed pull request #33527: URL: https://github.com/apache/spark/pull/33527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33527: [SPARK-36028][SQL][3.2] Allow Project to host outer references in scalar subqueries

cloud-fan commented on pull request #33527: URL: https://github.com/apache/spark/pull/33527#issuecomment-888009030 thanks, merging to 3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan closed pull request #33526: [SPARK-34952][SQL][FOLLOW-UP] DSv2 aggregate push down follow-up

cloud-fan closed pull request #33526: URL: https://github.com/apache/spark/pull/33526 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #33526: [SPARK-34952][SQL][FOLLOW-UP] DSv2 aggregate push down follow-up

cloud-fan commented on pull request #33526: URL: https://github.com/apache/spark/pull/33526#issuecomment-888008467 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] sarutak commented on pull request #33543: [SPARK-36318][SQL][DOCS] Update docs about mapping of ANSI interval types to Java/Scala/SQL types

sarutak commented on pull request #33543: URL: https://github.com/apache/spark/pull/33543#issuecomment-888005293 Merged to `master` and `branch-3.2`. Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] cloud-fan commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

cloud-fan commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677968476 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala ## @@ -257,13 +255,12 @@ object

[GitHub] [spark] cloud-fan commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

cloud-fan commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677968327 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -113,29 +130,17 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

AmplabJenkins removed a comment on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888004563 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46260/

[GitHub] [spark] sarutak closed pull request #33543: [SPARK-36318][SQL][DOCS] Update docs about mapping of ANSI interval types to Java/Scala/SQL types

sarutak closed pull request #33543: URL: https://github.com/apache/spark/pull/33543 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] AmplabJenkins commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

AmplabJenkins commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888004563 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46260/ --

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-888004554 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46260/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

cloud-fan commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677967739 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEUtils.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache

[GitHub] [spark] sarutak opened a new pull request #33551: [SPARK-36323][SQL] Support ANSI interval literals for TimeWindow

sarutak opened a new pull request #33551: URL: https://github.com/apache/spark/pull/33551 ### What changes were proposed in this pull request? This PR proposes to support ANSI interval literals for `TimeWindow`. ### Why are the changes needed? Watermark also supports

[GitHub] [spark] cloud-fan commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

cloud-fan commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677967315 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEUtils.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AmplabJenkins removed a comment on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-88709 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141746/

[GitHub] [spark] AmplabJenkins commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AmplabJenkins commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-88709 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141746/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

AmplabJenkins removed a comment on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887999493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141743/

[GitHub] [spark] AmplabJenkins commented on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

AmplabJenkins commented on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887999493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141743/ -- This

[GitHub] [spark] SparkQA commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887999463 **[Test build #141750 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141750/testReport)** for PR 33034 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

SparkQA removed a comment on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887960010 **[Test build #141746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141746/testReport)** for PR 33457 at commit

[GitHub] [spark] SparkQA commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

SparkQA commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887999189 **[Test build #141746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141746/testReport)** for PR 33457 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

SparkQA removed a comment on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887919638 **[Test build #141743 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141743/testReport)** for PR 33455 at commit

[GitHub] [spark] SparkQA commented on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

SparkQA commented on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887998978 **[Test build #141743 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141743/testReport)** for PR 33455 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

AmplabJenkins removed a comment on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-887998570 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141747/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AmplabJenkins removed a comment on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887998575 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46259/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

AmplabJenkins removed a comment on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887998573 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141739/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

AmplabJenkins removed a comment on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887985666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

AmplabJenkins removed a comment on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-887998571 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141748/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33549: [SPARK-36320][PYTHON] Fix Series/Index.copy() to drop extra columns

AmplabJenkins removed a comment on pull request #33549: URL: https://github.com/apache/spark/pull/33549#issuecomment-887998576 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46258/

[GitHub] [spark] AmplabJenkins commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

AmplabJenkins commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-887998570 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141747/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

AmplabJenkins commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887998575 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46259/ --

[GitHub] [spark] SparkQA commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887998530 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46262/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

AmplabJenkins commented on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-887998571 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141748/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33549: [SPARK-36320][PYTHON] Fix Series/Index.copy() to drop extra columns

AmplabJenkins commented on pull request #33549: URL: https://github.com/apache/spark/pull/33549#issuecomment-887998576 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46258/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

AmplabJenkins commented on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887998573 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141739/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

AmplabJenkins commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887998542 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46262/ --

[GitHub] [spark] SparkQA commented on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

SparkQA commented on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-887998032 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46261/ -- This is an automated message from the Apache

[GitHub] [spark] cutiechi commented on pull request #33257: [SPARK-36039][K8S] Fix executor pod hadoop conf mount

cutiechi commented on pull request #33257: URL: https://github.com/apache/spark/pull/33257#issuecomment-887994632 @dongjoon-hyun @ScrapCodes Please help me review again, thx -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-887993444 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46260/ -- This is an automated message from the Apache

[GitHub] [spark] viirya commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

viirya commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677944894 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CoalesceShufflePartitions.scala ## @@ -27,12 +28,20 @@ import

[GitHub] [spark] Peng-Lei commented on a change in pull request #33538: [SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei commented on a change in pull request #33538: URL: https://github.com/apache/spark/pull/33538#discussion_r677954153 ## File path: core/src/main/resources/error/error-classes.json ## @@ -63,5 +63,67 @@ "WRITING_JOB_ABORTED" : { "message" : [ "Writing job

[GitHub] [spark] Peng-Lei commented on a change in pull request #33538: [SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei commented on a change in pull request #33538: URL: https://github.com/apache/spark/pull/33538#discussion_r677954099 ## File path: core/src/main/resources/error/error-classes.json ## @@ -63,5 +63,67 @@ "WRITING_JOB_ABORTED" : { "message" : [ "Writing job

[GitHub] [spark] Peng-Lei commented on a change in pull request #33538: [SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei commented on a change in pull request #33538: URL: https://github.com/apache/spark/pull/33538#discussion_r677954042 ## File path: core/src/main/resources/error/error-classes.json ## @@ -63,5 +63,67 @@ "WRITING_JOB_ABORTED" : { "message" : [ "Writing job

[GitHub] [spark] Peng-Lei commented on a change in pull request #33538: [SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei commented on a change in pull request #33538: URL: https://github.com/apache/spark/pull/33538#discussion_r677953940 ## File path: core/src/main/resources/error/error-classes.json ## @@ -63,5 +63,67 @@ "WRITING_JOB_ABORTED" : { "message" : [ "Writing job

[GitHub] [spark] SparkQA removed a comment on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

SparkQA removed a comment on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-887981165 **[Test build #141748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141748/testReport)** for PR 33466 at commit

[GitHub] [spark] Peng-Lei commented on a change in pull request #33538: [SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei commented on a change in pull request #33538: URL: https://github.com/apache/spark/pull/33538#discussion_r677953868 ## File path: core/src/main/resources/error/error-classes.json ## @@ -63,5 +63,67 @@ "WRITING_JOB_ABORTED" : { "message" : [ "Writing job

[GitHub] [spark] Peng-Lei commented on a change in pull request #33538: [SPARK-36107][SQL] Refactor first set of 20 query execution errors to use error classes

Peng-Lei commented on a change in pull request #33538: URL: https://github.com/apache/spark/pull/33538#discussion_r677953640 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/errors/QueryExecutionErrors.scala ## @@ -67,89 +68,116 @@ import

[GitHub] [spark] ulysses-you commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

ulysses-you commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677949386 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/OptimizeSkewedJoin.scala ## @@ -257,13 +255,12 @@ object

[GitHub] [spark] SparkQA commented on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

SparkQA commented on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-887989210 **[Test build #141748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141748/testReport)** for PR 33466 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

SparkQA removed a comment on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887898780 **[Test build #141739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141739/testReport)** for PR 33455 at commit

[GitHub] [spark] SparkQA commented on pull request #33455: [SPARK-36236][SS] Additional metrics for RocksDB based state store implementation

SparkQA commented on pull request #33455: URL: https://github.com/apache/spark/pull/33455#issuecomment-887987348 **[Test build #141739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141739/testReport)** for PR 33455 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

SparkQA removed a comment on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-887981085 **[Test build #141747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141747/testReport)** for PR 33550 at commit

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-887986020 **[Test build #141747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141747/testReport)** for PR 33550 at commit

[GitHub] [spark] SparkQA commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

SparkQA commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887985994 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46259/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA removed a comment on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887981289 **[Test build #141749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141749/testReport)** for PR 33034 at commit

[GitHub] [spark] SparkQA commented on pull request #33549: [SPARK-36320][PYTHON] Fix Series/Index.copy() to drop extra columns

SparkQA commented on pull request #33549: URL: https://github.com/apache/spark/pull/33549#issuecomment-887985729 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46258/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

AmplabJenkins commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887985666 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141749/ -- This

[GitHub] [spark] SparkQA commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887985630 **[Test build #141749 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141749/testReport)** for PR 33034 at commit

[GitHub] [spark] ulysses-you commented on a change in pull request #33541: [SPARK-36315][SQL] Only skip AQEShuffleReadRule in the final stage if it breaks the distribution requirement

ulysses-you commented on a change in pull request #33541: URL: https://github.com/apache/spark/pull/33541#discussion_r677946498 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AQEUtils.scala ## @@ -0,0 +1,62 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA commented on pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

SparkQA commented on pull request #33034: URL: https://github.com/apache/spark/pull/33034#issuecomment-887981289 **[Test build #141749 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141749/testReport)** for PR 33034 at commit

[GitHub] [spark] SparkQA commented on pull request #33466: [SPARK-36143][PYTHON] Adjust `astype` of fractional Series with missing values to follow pandas

SparkQA commented on pull request #33466: URL: https://github.com/apache/spark/pull/33466#issuecomment-887981165 **[Test build #141748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141748/testReport)** for PR 33466 at commit

[GitHub] [spark] SparkQA commented on pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

SparkQA commented on pull request #33550: URL: https://github.com/apache/spark/pull/33550#issuecomment-887981085 **[Test build #141747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141747/testReport)** for PR 33550 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33526: [SPARK-34952][SQL][FOLLOW-UP] DSv2 aggregate push down follow-up

AmplabJenkins removed a comment on pull request #33526: URL: https://github.com/apache/spark/pull/33526#issuecomment-887980014 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141735/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33544: [SPARK-34927][INFRA] Support TPCDSQueryBenchmark in Benchmarks

AmplabJenkins removed a comment on pull request #33544: URL: https://github.com/apache/spark/pull/33544#issuecomment-887980011 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141742/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33548: [SPARK-36314][SS] Update Sessionization examples to use native support of session window

AmplabJenkins removed a comment on pull request #33548: URL: https://github.com/apache/spark/pull/33548#issuecomment-887980012 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46257/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33549: [SPARK-36320][PYTHON] Fix Series/Index.copy() to drop extra columns

AmplabJenkins removed a comment on pull request #33549: URL: https://github.com/apache/spark/pull/33549#issuecomment-887980013 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141745/

[GitHub] [spark] AmplabJenkins commented on pull request #33549: [SPARK-36320][PYTHON] Fix Series/Index.copy() to drop extra columns

AmplabJenkins commented on pull request #33549: URL: https://github.com/apache/spark/pull/33549#issuecomment-887980013 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141745/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33526: [SPARK-34952][SQL][FOLLOW-UP] DSv2 aggregate push down follow-up

AmplabJenkins commented on pull request #33526: URL: https://github.com/apache/spark/pull/33526#issuecomment-887980014 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141735/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33544: [SPARK-34927][INFRA] Support TPCDSQueryBenchmark in Benchmarks

AmplabJenkins commented on pull request #33544: URL: https://github.com/apache/spark/pull/33544#issuecomment-887980011 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] ulysses-you opened a new pull request #33550: [SPARK-36321][K8S] Do not fail application in kubernetes if name is too long

ulysses-you opened a new pull request #33550: URL: https://github.com/apache/spark/pull/33550 ### What changes were proposed in this pull request? Use short string as executor pod name prefix if app name is long. ### Why are the changes needed? If we have a long

[GitHub] [spark] AmplabJenkins commented on pull request #33548: [SPARK-36314][SS] Update Sessionization examples to use native support of session window

AmplabJenkins commented on pull request #33548: URL: https://github.com/apache/spark/pull/33548#issuecomment-887980012 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46257/ --

[GitHub] [spark] HeartSaVioR commented on pull request #33548: [SPARK-36314][SS] Update Sessionization examples to use native support of session window

HeartSaVioR commented on pull request #33548: URL: https://github.com/apache/spark/pull/33548#issuecomment-887976306 Thanks all for the quick reviews and merge! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

SparkQA commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-887975081 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46259/ -- This is an automated message from the Apache

[GitHub] [spark] viirya closed pull request #33548: [SPARK-36314][SS] Update Sessionization examples to use native support of session window

viirya closed pull request #33548: URL: https://github.com/apache/spark/pull/33548 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] viirya commented on pull request #33548: [SPARK-36314][SS] Update Sessionization examples to use native support of session window

viirya commented on pull request #33548: URL: https://github.com/apache/spark/pull/33548#issuecomment-887974743 Thanks! Merging to master/3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #33549: [SPARK-36320][PYTHON] Fix Series/Index.copy() to drop extra columns

SparkQA commented on pull request #33549: URL: https://github.com/apache/spark/pull/33549#issuecomment-887974631 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46258/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33526: [SPARK-34952][SQL][FOLLOW-UP] DSv2 aggregate push down follow-up

SparkQA commented on pull request #33526: URL: https://github.com/apache/spark/pull/33526#issuecomment-887972507 **[Test build #141735 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141735/testReport)** for PR 33526 at commit

[GitHub] [spark] venkata91 commented on a change in pull request #33034: [SPARK-32923][CORE][SHUFFLE] Handle indeterminate stage retries for push-based shuffle

venkata91 commented on a change in pull request #33034: URL: https://github.com/apache/spark/pull/33034#discussion_r677937888 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -648,14 +809,24 @@ public void

  1   2   3   4   5   6   7   8   9   >