[GitHub] [spark] SparkQA commented on pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

SparkQA commented on pull request #32563: URL: https://github.com/apache/spark/pull/32563#issuecomment-849940315 **[Test build #139027 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139027/testReport)** for PR 32563 at commit [`320e72b`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32643: [SPARK-35538][SQL] Migrate transformAllExpressions call sites to use transformAllExpressionsWithPruning

SparkQA commented on pull request #32643: URL: https://github.com/apache/spark/pull/32643#issuecomment-849940403 **[Test build #139026 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139026/testReport)** for PR 32643 at commit [`93adfb4`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #28032: [SPARK-31264][SQL] Repartition by dynamic partition columns before insert partition table

SparkQA commented on pull request #28032: URL: https://github.com/apache/spark/pull/28032#issuecomment-849941065 **[Test build #139025 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139025/testReport)** for PR 28032 at commit [`4552368`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

SparkQA removed a comment on pull request #32563: URL: https://github.com/apache/spark/pull/32563#issuecomment-849780322 **[Test build #139027 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139027/testReport)** for PR 32563 at commit [`320e72b`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #28032: [SPARK-31264][SQL] Repartition by dynamic partition columns before insert partition table

SparkQA removed a comment on pull request #28032: URL: https://github.com/apache/spark/pull/28032#issuecomment-849778592 **[Test build #139025 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139025/testReport)** for PR 28032 at commit [`4552368`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #32643: [SPARK-35538][SQL] Migrate transformAllExpressions call sites to use transformAllExpressionsWithPruning

SparkQA removed a comment on pull request #32643: URL: https://github.com/apache/spark/pull/32643#issuecomment-849780171 **[Test build #139026 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139026/testReport)** for PR 32643 at commit [`93adfb4`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

SparkQA commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-849953278 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43550/ -- This

[GitHub] [spark] SparkQA commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-849957217 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43551/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #28032: [SPARK-31264][SQL] Repartition by dynamic partition columns before insert partition table

AmplabJenkins commented on pull request #28032: URL: https://github.com/apache/spark/pull/28032#issuecomment-849957468 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139025/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32643: [SPARK-35538][SQL] Migrate transformAllExpressions call sites to use transformAllExpressionsWithPruning

AmplabJenkins commented on pull request #32643: URL: https://github.com/apache/spark/pull/32643#issuecomment-849957463 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139026/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

AmplabJenkins commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-849957464 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43550/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

AmplabJenkins commented on pull request #32563: URL: https://github.com/apache/spark/pull/32563#issuecomment-849957465 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139027/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

AmplabJenkins commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-849957466 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43548/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

AmplabJenkins removed a comment on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-849957464 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43550/

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849957914 **[Test build #139034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139034/testReport)** for PR 32686 at commit [`60fd253`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #28032: [SPARK-31264][SQL] Repartition by dynamic partition columns before insert partition table

AmplabJenkins removed a comment on pull request #28032: URL: https://github.com/apache/spark/pull/28032#issuecomment-849957468 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139025/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32643: [SPARK-35538][SQL] Migrate transformAllExpressions call sites to use transformAllExpressionsWithPruning

AmplabJenkins removed a comment on pull request #32643: URL: https://github.com/apache/spark/pull/32643#issuecomment-849957463 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139026/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32563: [SPARK-35415][SQL] Change `information` to map type for SHOW TABLE EXTENDED command

AmplabJenkins removed a comment on pull request #32563: URL: https://github.com/apache/spark/pull/32563#issuecomment-849825151 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

AmplabJenkins removed a comment on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-849957466 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43548/

[GitHub] [spark] SparkQA commented on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

SparkQA commented on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-849958280 **[Test build #139035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139035/testReport)** for PR 32301 at commit [`8a29e94`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849966932 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43547/ -- T

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849966907 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43547/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849966932 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43547/

[GitHub] [spark] zhouyejoe commented on a change in pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

zhouyejoe commented on a change in pull request #32007: URL: https://github.com/apache/spark/pull/32007#discussion_r640996792 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -336,51 +436,26 @@ public Stream

[GitHub] [spark] allisonwang-db opened a new pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

allisonwang-db opened a new pull request #32687: URL: https://github.com/apache/spark/pull/32687 ### What changes were proposed in this pull request? This PR refactors `SubqueryExpression` class. It removes the children field from SubqueryExpression's constructor and adds `outerAttrs` an

[GitHub] [spark] zhouyejoe commented on a change in pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

zhouyejoe commented on a change in pull request #32007: URL: https://github.com/apache/spark/pull/32007#discussion_r640999074 ## File path: common/network-shuffle/src/main/java/org/apache/spark/network/shuffle/RemoteBlockPushResolver.java ## @@ -117,64 +123,85 @@ public Shuffl

[GitHub] [spark] SparkQA commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

SparkQA commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-849972655 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43551/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849978232 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43552/ -- This

[GitHub] [spark] zhouyejoe commented on a change in pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

zhouyejoe commented on a change in pull request #32007: URL: https://github.com/apache/spark/pull/32007#discussion_r641010452 ## File path: core/src/test/scala/org/apache/spark/util/UtilsSuite.scala ## @@ -17,12 +17,13 @@ package org.apache.spark.util -import java.io.{Byte

[GitHub] [spark] SparkQA commented on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

SparkQA commented on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-849979996 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43553/ -- This is an automated message from the Apache

[GitHub] [spark] HeartSaVioR commented on pull request #32653: [SPARK-35312][SS] Introduce new Option in Kafka source to specify minimum number of records to read per trigger

HeartSaVioR commented on pull request #32653: URL: https://github.com/apache/spark/pull/32653#issuecomment-849982538 Let's ping once again and wait for a couple of days more. I'd like to make sure I'm not the only one thinking of such direction. Thanks for your patience. @brkyvz @boz

[GitHub] [spark] AmplabJenkins commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849985492 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43552/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins commented on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-849985493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43551/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849985492 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43552/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32611: [SPARK-35314][PYTHON] Support arithmetic operations against bool IndexOpsMixin

AmplabJenkins removed a comment on pull request #32611: URL: https://github.com/apache/spark/pull/32611#issuecomment-849985493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43551/

[GitHub] [spark] SparkQA commented on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

SparkQA commented on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-849987071 **[Test build #139036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139036/testReport)** for PR 32687 at commit [`d37d01a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

SparkQA commented on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-849993303 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43553/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

AmplabJenkins commented on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-849993315 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43553/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

AmplabJenkins removed a comment on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-849993315 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43553/

[GitHub] [spark] HyukjinKwon commented on pull request #32631: [SPARK-35483][INFRA] Add docker-integration-tests to run-tests.py and GA.

HyukjinKwon commented on pull request #32631: URL: https://github.com/apache/spark/pull/32631#issuecomment-849994570 Merged to master. I know this GA and Docker integration things are sort of tricky to handle .. Thanks @sarutak for working on this! -- This is an automated message

[GitHub] [spark] HyukjinKwon closed pull request #32631: [SPARK-35483][INFRA] Add docker-integration-tests to run-tests.py and GA.

HyukjinKwon closed pull request #32631: URL: https://github.com/apache/spark/pull/32631 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850001497 **[Test build #139034 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139034/testReport)** for PR 32686 at commit [`60fd253`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

SparkQA commented on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-850002116 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43554/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-849957914 **[Test build #139034 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139034/testReport)** for PR 32686 at commit [`60fd253`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

SparkQA commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-850004475 **[Test build #139032 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139032/testReport)** for PR 30135 at commit [`c56cb14`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

SparkQA removed a comment on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-849931685 **[Test build #139032 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139032/testReport)** for PR 30135 at commit [`c56cb14`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

AmplabJenkins commented on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-850008997 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43554/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

AmplabJenkins commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-850008994 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139032/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850008996 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139034/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

AmplabJenkins removed a comment on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-850008994 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139032/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

AmplabJenkins removed a comment on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-850008997 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43554/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850008996 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139034/ -

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850010173 **[Test build #139037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139037/testReport)** for PR 32686 at commit [`ab3b61f`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

SparkQA commented on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850011679 **[Test build #139038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139038/testReport)** for PR 32397 at commit [`b47599f`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

SparkQA commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-850012891 **[Test build #139030 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139030/testReport)** for PR 32473 at commit [`e582c26`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

SparkQA removed a comment on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-849888159 **[Test build #139030 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139030/testReport)** for PR 32473 at commit [`e582c26`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

AmplabJenkins commented on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-850013624 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139030/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32473: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

AmplabJenkins removed a comment on pull request #32473: URL: https://github.com/apache/spark/pull/32473#issuecomment-850013624 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139030/ -

[GitHub] [spark] Victsm commented on pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

Victsm commented on pull request #32007: URL: https://github.com/apache/spark/pull/32007#issuecomment-850024225 In fact, I'm leaning towards taking both handling multiple app attempts and the race conditions out of this PR and fix them in a separate jira. The original scope of this ticke

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850025721 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43555/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

SparkQA commented on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850029505 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43556/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on pull request #32631: [SPARK-35483][INFRA] Add docker-integration-tests to run-tests.py and GA.

dongjoon-hyun commented on pull request #32631: URL: https://github.com/apache/spark/pull/32631#issuecomment-850033054 Thank you, @sarutak and @HyukjinKwon . Ya, this is really nice to have. -- This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] zhengruifeng commented on a change in pull request #32673: [SPARK-35530][ML][TESTS] Fix rounding error in DifferentiableLossAggregatorSuite with Java 11

zhengruifeng commented on a change in pull request #32673: URL: https://github.com/apache/spark/pull/32673#discussion_r641077227 ## File path: mllib/src/test/scala/org/apache/spark/ml/optim/aggregator/DifferentiableLossAggregatorSuite.scala ## @@ -149,10 +152,43 @@ object Diff

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850035874 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43555/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850035894 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43555/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850035894 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/43555/

[GitHub] [spark] zhouyejoe commented on pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

zhouyejoe commented on pull request #32007: URL: https://github.com/apache/spark/pull/32007#issuecomment-850036241 Created ticket for later improvement https://issues.apache.org/jira/browse/SPARK-35546 -- This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] zhouyejoe edited a comment on pull request #32007: [SPARK-33350][SHUFFLE] Add support to DiskBlockManager to create merge directory and to get the local shuffle merged data

zhouyejoe edited a comment on pull request #32007: URL: https://github.com/apache/spark/pull/32007#issuecomment-850036241 Created ticket for later improvement [SPARK-35546](https://issues.apache.org/jira/browse/SPARK-35546) -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] sunchao commented on pull request #31998: [SPARK-34859][SQL] parquet vectorized reader - support column index with rowIndexes

sunchao commented on pull request #31998: URL: https://github.com/apache/spark/pull/31998#issuecomment-850040241 @lxian In the current approach we'd have to copy values from one vector to another. I think a better and more efficient approach may be to feed the row indexes to `VectorizedRle

[GitHub] [spark] allisonwang-db commented on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

allisonwang-db commented on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-850041129 cc @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] venkata91 commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

venkata91 commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r641094842 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2136,9 +2137,24 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] venkata91 commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

venkata91 commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r641095507 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2004,6 +2020,131 @@ private[spark] class DAGScheduler( }

[GitHub] [spark] venkata91 commented on pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

venkata91 commented on pull request #30691: URL: https://github.com/apache/spark/pull/30691#issuecomment-850045368 Addressed all the comments AFAIK, please review @mridulm @Victsm @Ngone51 @otterc -- This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850049444 **[Test build #139037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139037/testReport)** for PR 32686 at commit [`ab3b61f`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850010173 **[Test build #139037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139037/testReport)** for PR 32686 at commit [`ab3b61f`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850049662 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139037/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

AmplabJenkins removed a comment on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850049662 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139037/ -

[GitHub] [spark] otterc commented on a change in pull request #30691: [SPARK-32920][SHUFFLE] Finalization of Shuffle push/merge with Push based shuffle and preparation step for the reduce stage

otterc commented on a change in pull request #30691: URL: https://github.com/apache/spark/pull/30691#discussion_r641118134 ## File path: core/src/main/scala/org/apache/spark/scheduler/DAGScheduler.scala ## @@ -2000,6 +2023,147 @@ private[spark] class DAGScheduler( } }

[GitHub] [spark] SparkQA commented on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

SparkQA commented on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-850053105 **[Test build #139035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139035/testReport)** for PR 32301 at commit [`8a29e94`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

SparkQA removed a comment on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-849958280 **[Test build #139035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139035/testReport)** for PR 32301 at commit [`8a29e94`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

AmplabJenkins commented on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-850053730 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139035/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32301: [SPARK-35194][SQL] Refactor nested column aliasing for readability

AmplabJenkins removed a comment on pull request #32301: URL: https://github.com/apache/spark/pull/32301#issuecomment-850053730 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/139035/ -

[GitHub] [spark] SparkQA commented on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

SparkQA commented on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850056962 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43556/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

SparkQA commented on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850058005 **[Test build #139038 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139038/testReport)** for PR 32397 at commit [`b47599f`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

SparkQA removed a comment on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850011679 **[Test build #139038 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139038/testReport)** for PR 32397 at commit [`b47599f`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

AmplabJenkins commented on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850067873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32397: [SPARK-35084][CORE] Spark 3: supporting "--packages" in k8s cluster mode

AmplabJenkins removed a comment on pull request #32397: URL: https://github.com/apache/spark/pull/32397#issuecomment-850067873 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] LuciferYang opened a new pull request #32688: [SPARK-35550][BUILD] Upgrade Jackson to 2.12.3

LuciferYang opened a new pull request #32688: URL: https://github.com/apache/spark/pull/32688 ### What changes were proposed in this pull request? This pr upgrade Jackson version to 2.12.3. Jackson Release 2.12.3: [https://github.com/FasterXML/jackson/wiki/Jackson-Release-2.12.3](http

[GitHub] [spark] SparkQA commented on pull request #32688: [SPARK-35550][BUILD] Upgrade Jackson to 2.12.3

SparkQA commented on pull request #32688: URL: https://github.com/apache/spark/pull/32688#issuecomment-850068523 **[Test build #139039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139039/testReport)** for PR 32688 at commit [`72c425c`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850068550 **[Test build #139040 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139040/testReport)** for PR 32686 at commit [`408851d`](https://github.com

[GitHub] [spark] HyukjinKwon commented on pull request #32673: [SPARK-35530][ML][TESTS] Fix rounding error in DifferentiableLossAggregatorSuite with Java 11

HyukjinKwon commented on pull request #32673: URL: https://github.com/apache/spark/pull/32673#issuecomment-850068886 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon closed pull request #32673: [SPARK-35530][ML][TESTS] Fix rounding error in DifferentiableLossAggregatorSuite with Java 11

HyukjinKwon closed pull request #32673: URL: https://github.com/apache/spark/pull/32673 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pl

[GitHub] [spark] SparkQA commented on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

SparkQA commented on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-850074626 **[Test build #139036 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139036/testReport)** for PR 32687 at commit [`d37d01a`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32687: [SPARK-35545][SQL] Split SubqueryExpression's children field into outer attributes and join conditions

SparkQA removed a comment on pull request #32687: URL: https://github.com/apache/spark/pull/32687#issuecomment-849987071 **[Test build #139036 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/139036/testReport)** for PR 32687 at commit [`d37d01a`](https://gi

[GitHub] [spark] xuanyuanking commented on a change in pull request #32272: [SPARK-35172][SS] The implementation of RocksDBCheckpointMetadata

xuanyuanking commented on a change in pull request #32272: URL: https://github.com/apache/spark/pull/32272#discussion_r641196335 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBFileManager.scala ## @@ -0,0 +1,165 @@ +/* + * Licensed

[GitHub] [spark] xuanyuanking commented on pull request #32272: [SPARK-35172][SS] The implementation of RocksDBCheckpointMetadata

xuanyuanking commented on pull request #32272: URL: https://github.com/apache/spark/pull/32272#issuecomment-850078509 Thanks for the review and help! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] xuanyuanking commented on pull request #32582: [SPARK-35436] RocksDBFileManager - save checkpoint to DFS

xuanyuanking commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-850079062 As we merged #32272, after rebasing and addressing the comment, this one is ready for review. cc @viirya and @HeartSaVioR -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32686: [WIP][SPARK-35544][SQL] Add tree pattern pruning to Analyzer rules

SparkQA commented on pull request #32686: URL: https://github.com/apache/spark/pull/32686#issuecomment-850082809 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43558/ -- This is an automated message from the Apache

[GitHub] [spark] viirya commented on pull request #32582: [SPARK-35436][SS] RocksDBFileManager - save checkpoint to DFS

viirya commented on pull request #32582: URL: https://github.com/apache/spark/pull/32582#issuecomment-850083083 Thanks @xuanyuanking. I will find some time to review this. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] SparkQA commented on pull request #32688: [SPARK-35550][BUILD] Upgrade Jackson to 2.12.3

SparkQA commented on pull request #32688: URL: https://github.com/apache/spark/pull/32688#issuecomment-850083738 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/43557/ -- This

<    1   2   3   4   5   >