[GitHub] [spark] gengliangwang commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP] Increase it in build/mvn script

gengliangwang commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872738426 @dongjoon-hyun @LuciferYang Awesome, hopefully the issue is resolved this time. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] xuanyuanking commented on a change in pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

xuanyuanking commented on a change in pull request #32933: URL: https://github.com/apache/spark/pull/32933#discussion_r662755860 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala ## @@ -207,6 +273,133 @@ class RocksDBSuite

[GitHub] [spark] LuciferYang commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP] Increase it in build/mvn script

LuciferYang commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872737709 Yes, the `catalyst` module often has this problem -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

dongjoon-hyun edited a comment on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-872736822 Oh, if you are using ORC, please try to bring SPARK-35783. It's irrelevant to this Hadoop topic, but it helps you reduce the traffic. -

[GitHub] [spark] dongjoon-hyun commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

dongjoon-hyun commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-872736822 Oh, if you are using ORC, please try to bring SPARK-35783. It's irrelevant to Hadoop, but it helps you reduce the traffic. - https://github.com/apache/spark/pull/32923

[GitHub] [spark] cloud-fan commented on a change in pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

cloud-fan commented on a change in pull request #33140: URL: https://github.com/apache/spark/pull/33140#discussion_r662754669 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -122,7 +123,8 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

viirya commented on a change in pull request #32933: URL: https://github.com/apache/spark/pull/32933#discussion_r662754308 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala ## @@ -253,6 +253,13 @@ class RocksDB(

[GitHub] [spark] xuanyuanking commented on a change in pull request #32933: [SPARK-35785][SS] Cleanup support for RocksDB instance

xuanyuanking commented on a change in pull request #32933: URL: https://github.com/apache/spark/pull/32933#discussion_r662753878 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDB.scala ## @@ -253,6 +253,13 @@ class RocksDB(

[GitHub] [spark] viirya commented on pull request #29326: [WIP][SPARK-32502][BUILD] Upgrade Guava to 27.0-jre

viirya commented on pull request #29326: URL: https://github.com/apache/spark/pull/29326#issuecomment-872735918 Hmm, from the failed tests below: org.apache.spark.sql.hive.DataSourceWithHiveMetastoreCatalogSuite org.apache.spark.sql.hive.HiveExternalCatalogSuite

[GitHub] [spark] dongjoon-hyun commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP] Increase it in build/mvn script

dongjoon-hyun commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872735624 Oh! Thank you for sharing that, @LuciferYang . Ya, I saw this on `catalyst` mostly. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] LuciferYang commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP] Increase it in build/mvn script

LuciferYang commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872735039 @dongjoon-hyun @gengliangwang It seems to work, I have compile and test catalyst and related modules for many times in my compilation environment, no `StackOverflowError`

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872732859 **[Test build #140562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140562/testReport)** for PR 33172 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

AmplabJenkins removed a comment on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872731983 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45069/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins removed a comment on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872731982 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45070/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop vers

AmplabJenkins removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872731980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872731984 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140550/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

AmplabJenkins removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872711527 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

AmplabJenkins removed a comment on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872731981 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

AmplabJenkins commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872731988 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45073/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

AmplabJenkins commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872731982 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45070/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

AmplabJenkins commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872731985 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872731984 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140550/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

AmplabJenkins commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872731983 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45069/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions old

AmplabJenkins commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872731980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] arghya18 commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

arghya18 commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-872730959 @dongjoon-hyun Thanks.. I am testing more jobs for further statistics. BDW I am testing this on ORC. -- This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872729734 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45073/ --

[GitHub] [spark] SparkQA removed a comment on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions o

SparkQA removed a comment on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872708897 **[Test build #140560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140560/testReport)** for PR 33160 at commit

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older tha

SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872729629 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45072/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

dongjoon-hyun commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-872729358 Thank you for sharing, @arghya18 . It's interesting. The read statistic increase is also observed in my environment, but TPCDS 1TB on S3 parquet performance was faster

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older tha

SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872729293 **[Test build #140560 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140560/testReport)** for PR 33160 at commit

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older tha

SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872728701 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45072/ -- This is an automated message from the Apache

[GitHub] [spark] gengliangwang commented on a change in pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

gengliangwang commented on a change in pull request #33164: URL: https://github.com/apache/spark/pull/33164#discussion_r662746235 ## File path: core/src/main/java/org/apache/spark/memory/SparkOutOfMemoryError.java ## @@ -14,8 +14,11 @@ * See the License for the specific

[GitHub] [spark] cloud-fan commented on a change in pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

cloud-fan commented on a change in pull request #33164: URL: https://github.com/apache/spark/pull/33164#discussion_r662745025 ## File path: core/src/main/resources/error/error-classes.json ## @@ -15,6 +15,10 @@ "message" : [ "The second argument of '%s' function needs to

[GitHub] [spark] cloud-fan commented on a change in pull request #33164: [SPARK-35958][CORE] Refactor SparkError.scala to SparkThrowable.java

cloud-fan commented on a change in pull request #33164: URL: https://github.com/apache/spark/pull/33164#discussion_r662744867 ## File path: core/src/main/java/org/apache/spark/SparkThrowable.java ## @@ -0,0 +1,38 @@ +/* + * Licensed to the Apache Software Foundation (ASF)

[GitHub] [spark] dongjoon-hyun closed pull request #33180: [SPARK-35825][INFRA][FOLLOWUP] Increase it in build/mvn script

dongjoon-hyun closed pull request #33180: URL: https://github.com/apache/spark/pull/33180 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] dongjoon-hyun commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

dongjoon-hyun commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872724889 I'll merge this. Please let us know your result when you have some time, @LuciferYang ~ -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872723745 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45071/ -- This is an automated message from the

[GitHub] [spark] dongjoon-hyun commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

dongjoon-hyun commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872723431 Since this is a flaky compilation issue, the above two Maven runs might be insufficient for verification. However, I believe this patch is no harm for the build and only

[GitHub] [spark] dongjoon-hyun commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

dongjoon-hyun commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872722372 The above Maven run actually passed the compilation on `catalyst` and `sql` which we see the StackOverflowError frequently. Only fails with the following. It seems to be

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872717213 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45070/ -- This is an automated message from the

[GitHub] [spark] SparkQA removed a comment on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

SparkQA removed a comment on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872709101 **[Test build #140561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140561/testReport)** for PR 32816 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872634594 **[Test build #140550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140550/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872715758 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45068/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872714879 **[Test build #140550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140550/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872714477 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45069/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

AmplabJenkins commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872711527 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140561/ -- This

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872711498 **[Test build #140561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140561/testReport)** for PR 32816 at commit

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872709712 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45071/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

SparkQA commented on pull request #32816: URL: https://github.com/apache/spark/pull/32816#issuecomment-872709101 **[Test build #140561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140561/testReport)** for PR 32816 at commit

[GitHub] [spark] SparkQA commented on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older tha

SparkQA commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872708897 **[Test build #140560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140560/testReport)** for PR 33160 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

AmplabJenkins removed a comment on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872708365 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140552/

[GitHub] [spark] SparkQA removed a comment on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

SparkQA removed a comment on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872636572 **[Test build #140552 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140552/testReport)** for PR 33180 at commit

[GitHub] [spark] gengliangwang closed pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

gengliangwang closed pull request #33177: URL: https://github.com/apache/spark/pull/33177 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] AmplabJenkins commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

AmplabJenkins commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872708365 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140552/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

AmplabJenkins removed a comment on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872708148 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140559/

[GitHub] [spark] SparkQA commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

SparkQA commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872708254 **[Test build #140552 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140552/testReport)** for PR 33180 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

AmplabJenkins removed a comment on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-872708150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140547/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

AmplabJenkins removed a comment on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872708152 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140546/

[GitHub] [spark] AmplabJenkins commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

AmplabJenkins commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872708152 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140546/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

AmplabJenkins commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872708148 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140559/ -- This

[GitHub] [spark] gengliangwang commented on pull request #33177: [SPARK-35955][SQL] Check for overflow in Average in ANSI mode

gengliangwang commented on pull request #33177: URL: https://github.com/apache/spark/pull/33177#issuecomment-872708153 Thanks, merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] AmplabJenkins commented on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

AmplabJenkins commented on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-872708150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140547/ -- This

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872704800 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45070/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872704188 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45068/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872703815 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45069/ -- This is an automated message from the Apache

[GitHub] [spark] sunchao commented on pull request #33160: [SPARK-35959][BUILD][test-maven][test-hadoop3.2][test-java11] Add a new Maven profile "no-shaded-hadoop-client" for Hadoop versions older tha

sunchao commented on pull request #33160: URL: https://github.com/apache/spark/pull/33160#issuecomment-872703193 That's unfortunate... maybe for testing purpose I'll just change Hadoop version directly in the `pom.xml` to work around the sbt + maven property issue. -- This is an

[GitHub] [spark] arghya18 edited a comment on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

arghya18 edited a comment on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-872701489 @dongjoon-hyun @steveloughran I was able to test my use case with Hadoop 3.3.1 and posted the result

[GitHub] [spark] arghya18 commented on pull request #30135: [SPARK-29250][BUILD] Upgrade to Hadoop 3.3.1

arghya18 commented on pull request #30135: URL: https://github.com/apache/spark/pull/30135#issuecomment-872701489 @dongjoon-hyun I was able to test my use case with Hadoop 3.3.1 and posted the result

[GitHub] [spark] ulysses-you commented on a change in pull request #32816: [SPARK-33832][SQL] Support optimize skewed join even if introduce extra shuffle

ulysses-you commented on a change in pull request #32816: URL: https://github.com/apache/spark/pull/32816#discussion_r662725098 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -252,17 +275,26 @@ case class

[GitHub] [spark] gengliangwang commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

gengliangwang commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872698339 LGTM. @LuciferYang what is the local test result? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] SparkQA removed a comment on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

SparkQA removed a comment on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872614787 **[Test build #140546 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140546/testReport)** for PR 33070 at commit

[GitHub] [spark] SparkQA commented on pull request #33070: [SPARK-35551][SQL] Handle the COUNT bug for lateral subqueries

SparkQA commented on pull request #33070: URL: https://github.com/apache/spark/pull/33070#issuecomment-872697943 **[Test build #140546 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140546/testReport)** for PR 33070 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA removed a comment on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872694042 **[Test build #140559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140559/testReport)** for PR 33182 at commit

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872695942 **[Test build #140559 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140559/testReport)** for PR 33182 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

SparkQA removed a comment on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-872614842 **[Test build #140547 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140547/testReport)** for PR 32944 at commit

[GitHub] [spark] linhongliu-db commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

linhongliu-db commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872695771 cc @cloud-fan -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] SparkQA commented on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

SparkQA commented on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-872694868 **[Test build #140547 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140547/testReport)** for PR 32944 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32286: [SPARK-35181][CORE] Use zstd for spark.io.compression.codec by default

AmplabJenkins commented on pull request #32286: URL: https://github.com/apache/spark/pull/32286#issuecomment-872694269 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45066/ --

[GitHub] [spark] SparkQA commented on pull request #32286: [SPARK-35181][CORE] Use zstd for spark.io.compression.codec by default

SparkQA commented on pull request #32286: URL: https://github.com/apache/spark/pull/32286#issuecomment-872694251 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45066/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872694042 **[Test build #140559 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140559/testReport)** for PR 33182 at commit

[GitHub] [spark] gengliangwang closed pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

gengliangwang closed pull request #33093: URL: https://github.com/apache/spark/pull/33093 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] gengliangwang commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

gengliangwang commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872692395 Thanks, merging to master -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #33140: [SPARK-35881][SQL] Add support for columnar execution of final query stage in AdaptiveSparkPlanExec

SparkQA commented on pull request #33140: URL: https://github.com/apache/spark/pull/33140#issuecomment-872689929 **[Test build #140558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140558/testReport)** for PR 33140 at commit

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-872689863 **[Test build #140556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140556/testReport)** for PR 33182 at commit

[GitHub] [spark] SparkQA commented on pull request #33172: [SPARK-35968][SQL] Make sure partitions are not too small in AQE partition coalescing

SparkQA commented on pull request #33172: URL: https://github.com/apache/spark/pull/33172#issuecomment-872689889 **[Test build #140557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140557/testReport)** for PR 33172 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33181: [SPARK-35982][SQL] Allow from_json/to_json for map types where value types are year-month intervals

AmplabJenkins removed a comment on pull request #33181: URL: https://github.com/apache/spark/pull/33181#issuecomment-872689084 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45067/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33174: [SPARK-35721][PYTHON] Path level discover for python unittests

AmplabJenkins removed a comment on pull request #33174: URL: https://github.com/apache/spark/pull/33174#issuecomment-872689081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

AmplabJenkins removed a comment on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872689083 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140548/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872689085 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140543/

[GitHub] [spark] AmplabJenkins commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

AmplabJenkins commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872689085 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140543/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33174: [SPARK-35721][PYTHON] Path level discover for python unittests

AmplabJenkins commented on pull request #33174: URL: https://github.com/apache/spark/pull/33174#issuecomment-872689082 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33180: [SPARK-35825][INFRA][FOLLOWUP][test-maven] Increase it in build/mvn script

AmplabJenkins commented on pull request #33180: URL: https://github.com/apache/spark/pull/33180#issuecomment-872689083 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140548/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33181: [SPARK-35982][SQL] Allow from_json/to_json for map types where value types are year-month intervals

AmplabJenkins commented on pull request #33181: URL: https://github.com/apache/spark/pull/33181#issuecomment-872689084 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45067/ --

[GitHub] [spark] SparkQA commented on pull request #33181: [SPARK-35982][SQL] Allow from_json/to_json for map types where value types are year-month intervals

SparkQA commented on pull request #33181: URL: https://github.com/apache/spark/pull/33181#issuecomment-872688574 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45067/ --

[GitHub] [spark] SparkQA removed a comment on pull request #33174: [SPARK-35721][PYTHON] Path level discover for python unittests

SparkQA removed a comment on pull request #33174: URL: https://github.com/apache/spark/pull/33174#issuecomment-872652027 **[Test build #140553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140553/testReport)** for PR 33174 at commit

[GitHub] [spark] SparkQA commented on pull request #33174: [SPARK-35721][PYTHON] Path level discover for python unittests

SparkQA commented on pull request #33174: URL: https://github.com/apache/spark/pull/33174#issuecomment-872686045 **[Test build #140553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140553/testReport)** for PR 33174 at commit

[GitHub] [spark] linhongliu-db opened a new pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

linhongliu-db opened a new pull request #33182: URL: https://github.com/apache/spark/pull/33182 ### What changes were proposed in this pull request? Add a config `spark.sql.join.forceApplyShuffledHashJoin` to force applying shuffled hash join during the join selection. ###

[GitHub] [spark] SparkQA removed a comment on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA removed a comment on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872593557 **[Test build #140543 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140543/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #33093: [SPARK-35897][SS] Support user defined initial state with flatMapGroupsWithState in Structured Streaming

SparkQA commented on pull request #33093: URL: https://github.com/apache/spark/pull/33093#issuecomment-872685340 **[Test build #140543 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140543/testReport)** for PR 33093 at commit

[GitHub] [spark] SparkQA commented on pull request #32286: [SPARK-35181][CORE] Use zstd for spark.io.compression.codec by default

SparkQA commented on pull request #32286: URL: https://github.com/apache/spark/pull/32286#issuecomment-872685080 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45066/ -- This is an automated message from the Apache

  1   2   3   4   5   6   7   8   9   10   >