[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812256776 **[Test build #136831 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136831/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

SparkQA removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812256562 **[Test build #136831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136831/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812256562 **[Test build #136831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136831/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA commented on pull request #30480: [SPARK-32921][SHUFFLE] MapOutputTracker extensions to support push-based shuffle

SparkQA commented on pull request #30480: URL: https://github.com/apache/spark/pull/30480#issuecomment-812256558 **[Test build #136832 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136832/testReport)** for PR 30480 at commit

[GitHub] [spark] SparkQA commented on pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run.

SparkQA commented on pull request #32032: URL: https://github.com/apache/spark/pull/32032#issuecomment-812256299 **[Test build #136830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136830/testReport)** for PR 32032 at commit

[GitHub] [spark] imback82 commented on a change in pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis ru

imback82 commented on a change in pull request #32032: URL: https://github.com/apache/spark/pull/32032#discussion_r606017579 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -3701,6 +3703,16 @@ object CleanupAliases extends

[GitHub] [spark] SparkQA commented on pull request #32030: [WIP] Improve map children

SparkQA commented on pull request #32030: URL: https://github.com/apache/spark/pull/32030#issuecomment-812254032 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41408/ -- This is an automated message from the Apache

[GitHub] [spark] imback82 commented on a change in pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis ru

imback82 commented on a change in pull request #32032: URL: https://github.com/apache/spark/pull/32032#discussion_r606016600 ## File path: sql/core/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveSessionCatalog.scala ## @@ -482,7 +482,7 @@ class

[GitHub] [spark] imback82 commented on a change in pull request #31933: [SPARK-34701][SQL] Remove analyzing temp view again in CreateViewCommand

imback82 commented on a change in pull request #31933: URL: https://github.com/apache/spark/pull/31933#discussion_r606017149 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala ## @@ -96,11 +96,6 @@ case class CreateViewCommand( }

[GitHub] [spark] github-actions[bot] closed pull request #30725: [SPARK-33753][CORE] Reduce the memory footprint and gc of the cache (hadoopJobMetadata)

github-actions[bot] closed pull request #30725: URL: https://github.com/apache/spark/pull/30725 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [spark] github-actions[bot] closed pull request #29655: [SPARK-32806][SQL] SortMergeJoin with partial hash distribution can be optimized to remove shuffle

github-actions[bot] closed pull request #29655: URL: https://github.com/apache/spark/pull/29655 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this

[GitHub] [spark] imback82 opened a new pull request #32032: [SPARK-34701][SQL] Introduce TransformaAfterAnalysis rule that allows a logical plan to be transformed after all the analysis rules run.

imback82 opened a new pull request #32032: URL: https://github.com/apache/spark/pull/32032 ### What changes were proposed in this pull request? This PR proposes to introduce the `TransformAfterAnalysis` rule such that any logical plan that extends the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

AmplabJenkins removed a comment on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812240213 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136826/

[GitHub] [spark] AmplabJenkins commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

AmplabJenkins commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812240213 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136826/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

SparkQA removed a comment on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812164543 **[Test build #136826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136826/testReport)** for PR 31871 at commit

[GitHub] [spark] SparkQA commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

SparkQA commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812239621 **[Test build #136826 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136826/testReport)** for PR 31871 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

AmplabJenkins removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812236434 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136827/

[GitHub] [spark] SparkQA removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

SparkQA removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812236256 **[Test build #136827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136827/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

SparkQA commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-812236536 **[Test build #136829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136829/testReport)** for PR 30145 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

AmplabJenkins commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812236434 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136827/ -- This

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812236425 **[Test build #136827 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136827/testReport)** for PR 32031 at commit

[GitHub] [spark] SparkQA commented on pull request #32030: [WIP] Improve map children

SparkQA commented on pull request #32030: URL: https://github.com/apache/spark/pull/32030#issuecomment-812236261 **[Test build #136828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136828/testReport)** for PR 32030 at commit

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-812236256 **[Test build #136827 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136827/testReport)** for PR 32031 at commit

[GitHub] [spark] AngersZhuuuu commented on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

AngersZh commented on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-812234468 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AngersZhuuuu commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AngersZh commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812234168 > LGTM, @AngersZh could you update PR's description. DOne -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] hiboyang opened a new pull request #32031: [WIP ]Initial work of Remote Shuffle Service to support dynamic allocation on Kubernetes

hiboyang opened a new pull request #32031: URL: https://github.com/apache/spark/pull/32031 ### What changes were proposed in this pull request? This PR contains Remote Shuffle Service to support dynamic allocation on Kubernetes. The code is mostly copied from [Uber Remote Shuffle

[GitHub] [spark] dbaliafroozeh opened a new pull request #32030: [WIP] Improve map children

dbaliafroozeh opened a new pull request #32030: URL: https://github.com/apache/spark/pull/32030 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ###

[GitHub] [spark] wypoon commented on a change in pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan

wypoon commented on a change in pull request #31451: URL: https://github.com/apache/spark/pull/31451#discussion_r605995166 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/PartitionReader.java ## @@ -51,7 +51,8 @@ T get(); /** - * Returns

[GitHub] [spark] HeartSaVioR edited a comment on pull request #31638: [SPARK-34526][SS] Skip checking glob path in FileStreamSink.hasMetadata

HeartSaVioR edited a comment on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-812224204 I guess we still don't decide with the following: > Just a one thing I think we'd like to make clear is, the flag on behavior. As we are sure we are fixing

[GitHub] [spark] HeartSaVioR commented on pull request #31638: [SPARK-34526][SS] Skip checking glob path in FileStreamSink.hasMetadata

HeartSaVioR commented on pull request #31638: URL: https://github.com/apache/spark/pull/31638#issuecomment-812224204 I guess we still don't decide with the following: > Just a one thing I think we'd like to make clear is, the flag on behavior. As we are sure we are fixing regression

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

AmplabJenkins removed a comment on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812217240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136825/

[GitHub] [spark] AmplabJenkins commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

AmplabJenkins commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812217240 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136825/ -- This

[GitHub] [spark] HyukjinKwon commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

HyukjinKwon commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812216156  -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

[GitHub] [spark] dongjoon-hyun commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812204092 Thank you, @sarutak and @srowen . -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] dongjoon-hyun closed pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

dongjoon-hyun closed pull request #32029: URL: https://github.com/apache/spark/pull/32029 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] srowen commented on pull request #32023: [SPARK-34933][DOC][SQL] Remove the description that || and && can be used as logical operators from the document

srowen commented on pull request #32023: URL: https://github.com/apache/spark/pull/32023#issuecomment-812203203 Merged to master/3.1/3.0. 2.4 has a conflict -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] srowen closed pull request #32023: [SPARK-34933][DOC][SQL] Remove the description that || and && can be used as logical operators from the document

srowen closed pull request #32023: URL: https://github.com/apache/spark/pull/32023 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] srowen commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

srowen commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812202027 Merged to 2.4 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

SparkQA removed a comment on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812133272 **[Test build #136825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136825/testReport)** for PR 32029 at commit

[GitHub] [spark] SparkQA commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

SparkQA commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812200852 **[Test build #136825 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136825/testReport)** for PR 32029 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

AmplabJenkins removed a comment on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812198288 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41406/

[GitHub] [spark] AmplabJenkins commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

AmplabJenkins commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812198288 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41406/ --

[GitHub] [spark] SparkQA commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

SparkQA commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812198273 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41406/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

SparkQA commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812197544 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41406/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #31935: [SPARK-34789][TEST] Introduce Jetty based construct for integration tests where HTTP server is used

AmplabJenkins commented on pull request #31935: URL: https://github.com/apache/spark/pull/31935#issuecomment-812191378 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136823/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31935: [SPARK-34789][TEST] Introduce Jetty based construct for integration tests where HTTP server is used

AmplabJenkins removed a comment on pull request #31935: URL: https://github.com/apache/spark/pull/31935#issuecomment-812191378 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136823/

[GitHub] [spark] SparkQA removed a comment on pull request #31935: [SPARK-34789][TEST] Introduce Jetty based construct for integration tests where HTTP server is used

SparkQA removed a comment on pull request #31935: URL: https://github.com/apache/spark/pull/31935#issuecomment-811998813 **[Test build #136823 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136823/testReport)** for PR 31935 at commit

[GitHub] [spark] SparkQA commented on pull request #31935: [SPARK-34789][TEST] Introduce Jetty based construct for integration tests where HTTP server is used

SparkQA commented on pull request #31935: URL: https://github.com/apache/spark/pull/31935#issuecomment-812190894 **[Test build #136823 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136823/testReport)** for PR 31935 at commit

[GitHub] [spark] SparkQA commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

SparkQA commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812164543 **[Test build #136826 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136826/testReport)** for PR 31871 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

AmplabJenkins removed a comment on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812163232 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41405/

[GitHub] [spark] AmplabJenkins commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

AmplabJenkins commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812163232 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41405/ --

[GitHub] [spark] LeonardoZV edited a comment on pull request #31771: [SPARK-34652][AVRO] Support SchemaRegistry in from_avro method

LeonardoZV edited a comment on pull request #31771: URL: https://github.com/apache/spark/pull/31771#issuecomment-812063145 My humble opinion: I think Spark should full support Confluent SR. The use of event driven architecture is skyrockting and big companies like to control event

[GitHub] [spark] SparkQA commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

SparkQA commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812149015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For

[GitHub] [spark] attilapiros commented on pull request #31871: [SPARK-34779][CORE] ExecutorMetricsPoller should keep stage entry in stageTCMP until a heartbeat occurs

attilapiros commented on pull request #31871: URL: https://github.com/apache/spark/pull/31871#issuecomment-812143190 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

SparkQA commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812133272 **[Test build #136825 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136825/testReport)** for PR 32029 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32027: [SPARK-34354][SQL] Fix failure when apply CostBasedJoinReorder on self-join

AmplabJenkins removed a comment on pull request #32027: URL: https://github.com/apache/spark/pull/32027#issuecomment-812132246 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136819/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

AmplabJenkins removed a comment on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812132245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136824/

[GitHub] [spark] AmplabJenkins commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

AmplabJenkins commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812132245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136824/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32027: [SPARK-34354][SQL] Fix failure when apply CostBasedJoinReorder on self-join

AmplabJenkins commented on pull request #32027: URL: https://github.com/apache/spark/pull/32027#issuecomment-812132246 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136819/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

SparkQA removed a comment on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812070108 **[Test build #136824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136824/testReport)** for PR 32017 at commit

[GitHub] [spark] SparkQA commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

SparkQA commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812120876 **[Test build #136824 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136824/testReport)** for PR 32017 at commit

[GitHub] [spark] sarutak commented on pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

sarutak commented on pull request #32029: URL: https://github.com/apache/spark/pull/32029#issuecomment-812119509 cc: @dongjoon-hyun @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] sarutak opened a new pull request #32029: [SPARK-24931][INFRA][2.4] Fix the GA failure related to R linter

sarutak opened a new pull request #32029: URL: https://github.com/apache/spark/pull/32029 ### What changes were proposed in this pull request? This PR backports the change of #32028 . This PR fixes the GA failure related to R linter which happens on some PRs (e.g. #32023,

[GitHub] [spark] sarutak commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

sarutak commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812117248 O.K, I'll open a PR for `branch-2.4`. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812116993 It's backported to branch-3.1/3.0. For branch-2.4, there is a conflict. Could you make a backporting PR to branch-2.4 only? -- This is an automated message from the

[GitHub] [spark] sarutak commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

sarutak commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812115799 O.K, I won't. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] dongjoon-hyun commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812115568 Oh, I'll backport it then. I'm just blocked due to the JIRA issue issue. You don't need to make backporting PRs. -- This is an automated message from the Apache Git

[GitHub] [spark] dongjoon-hyun commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812115012 BTW, do we need this in old branches? I didn't see Linter failures on branch-3.1/3.0 yet. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] sarutak commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

sarutak commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812114875 Yeah, we are lucky. Sorry again and thank you for the work. BTW, this issue happens on `branch-3.1`, `branch-3.0` and `branch-2.4`. I'll open backport PRs. -- This is an

[GitHub] [spark] dongjoon-hyun commented on pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812114196 The switch is finished~ In this case, we are lucky because the old one is not resolved yet. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] sarutak commented on pull request #32028: [SPARK-34931][INFRA] Fix the GA failure related to R linter

sarutak commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812112518 Sorry for that. I've noticed the wrong JIRA ID just now... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] dongjoon-hyun commented on pull request #32028: [SPARK-34931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812111879 We cannot change the commit log. Let me switch those issues. cc @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] dongjoon-hyun commented on pull request #32028: [SPARK-34931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun commented on pull request #32028: URL: https://github.com/apache/spark/pull/32028#issuecomment-812111763 Oh, @sarutak . It seems that you gave the wrong JIRA ID. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] SparkQA removed a comment on pull request #32027: [SPARK-34354][SQL] Fix failure when apply CostBasedJoinReorder on self-join

SparkQA removed a comment on pull request #32027: URL: https://github.com/apache/spark/pull/32027#issuecomment-811959223 **[Test build #136819 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136819/testReport)** for PR 32027 at commit

[GitHub] [spark] SparkQA commented on pull request #32027: [SPARK-34354][SQL] Fix failure when apply CostBasedJoinReorder on self-join

SparkQA commented on pull request #32027: URL: https://github.com/apache/spark/pull/32027#issuecomment-812110943 **[Test build #136819 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136819/testReport)** for PR 32027 at commit

[GitHub] [spark] dongjoon-hyun closed pull request #32028: [SPARK-24931][INFRA] Fix the GA failure related to R linter

dongjoon-hyun closed pull request #32028: URL: https://github.com/apache/spark/pull/32028 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

AmplabJenkins removed a comment on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812101279 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41404/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins removed a comment on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812101280 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136817/

[GitHub] [spark] AmplabJenkins commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812101280 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136817/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

AmplabJenkins commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812101279 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/41404/ --

[GitHub] [spark] SparkQA commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

SparkQA commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812092976 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41404/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

SparkQA commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812091832 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/41404/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

SparkQA removed a comment on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-81191 **[Test build #136817 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136817/testReport)** for PR 32018 at commit

[GitHub] [spark] SparkQA commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

SparkQA commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812085515 **[Test build #136817 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136817/testReport)** for PR 32018 at commit

[GitHub] [spark] karenfeng commented on a change in pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

karenfeng commented on a change in pull request #32017: URL: https://github.com/apache/spark/pull/32017#discussion_r605847308 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -924,11 +947,6 @@ case class

[GitHub] [spark] AmplabJenkins commented on pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

AmplabJenkins commented on pull request #32026: URL: https://github.com/apache/spark/pull/32026#issuecomment-812074839 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins removed a comment on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812073418 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136816/

[GitHub] [spark] AmplabJenkins commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812073418 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136816/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

SparkQA removed a comment on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-811920107 **[Test build #136816 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136816/testReport)** for PR 32018 at commit

[GitHub] [spark] SparkQA commented on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

SparkQA commented on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812072213 **[Test build #136816 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136816/testReport)** for PR 32018 at commit

[GitHub] [spark] rdblue commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

rdblue commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812071082 I share @viirya's concern about losing `metadataOutput`. I think it is better to propagate by default than to drop by default, since most of the time the metadata output is not

[GitHub] [spark] SparkQA commented on pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

SparkQA commented on pull request #32017: URL: https://github.com/apache/spark/pull/32017#issuecomment-812070108 **[Test build #136824 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/136824/testReport)** for PR 32017 at commit

[GitHub] [spark] rdblue commented on a change in pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

rdblue commented on a change in pull request #32017: URL: https://github.com/apache/spark/pull/32017#discussion_r605840188 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -31,7 +31,7 @@ import

[GitHub] [spark] srowen commented on pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

srowen commented on pull request #31204: URL: https://github.com/apache/spark/pull/31204#issuecomment-812069527 Merged to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] srowen closed pull request #31204: [SPARK-26399][WEBUI][CORE] Add new stage-level REST APIs and parameters

srowen closed pull request #31204: URL: https://github.com/apache/spark/pull/31204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please

[GitHub] [spark] rdblue commented on a change in pull request #32017: [SPARK-34923][SQL] Metadata output should be empty by default

rdblue commented on a change in pull request #32017: URL: https://github.com/apache/spark/pull/32017#discussion_r605839093 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -924,11 +947,6 @@ case class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30145: [SPARK-33233][SQL]CUBE/ROLLUP/GROUPING SETS support GROUP BY ordinal

AmplabJenkins removed a comment on pull request #30145: URL: https://github.com/apache/spark/pull/30145#issuecomment-812068246 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136815/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31908: [SPARK-34808][SQL] Removes outer join if it only has DISTINCT on streamed side

AmplabJenkins removed a comment on pull request #31908: URL: https://github.com/apache/spark/pull/31908#issuecomment-812068243 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136814/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32018: [SPARK-34926][SQL] PartitioningUtils.getPathFragment() should respect partition value is null

AmplabJenkins removed a comment on pull request #32018: URL: https://github.com/apache/spark/pull/32018#issuecomment-812068245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136821/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32026: [SPARK-34771] Support UDT for Pandas/Spark conversion with Arrow support Enabled

AmplabJenkins removed a comment on pull request #32026: URL: https://github.com/apache/spark/pull/32026#issuecomment-812068244 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/136820/

<    1   2   3   4   5   6   >