[GitHub] [spark] AmplabJenkins commented on pull request #30170: [SPARK-33267][SQL] Fix NPE issue on 'In' filter when one of values contains null

AmplabJenkins commented on pull request #30170: URL: https://github.com/apache/spark/pull/30170#issuecomment-717955276 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30170: [SPARK-33267][SQL] Fix NPE issue on 'In' filter when one of values contains null

AmplabJenkins removed a comment on pull request #30170: URL: https://github.com/apache/spark/pull/30170#issuecomment-717955276 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] tanelk commented on a change in pull request #30018: [SPARK-33122][SQL] Remove redundant aggregates in the Optimzier

tanelk commented on a change in pull request #30018: URL: https://github.com/apache/spark/pull/30018#discussion_r513473158 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/AliasHelper.scala ## @@ -0,0 +1,100 @@ +/* + * Licensed to the Apache

[GitHub] [spark] MaxGekk opened a new pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

MaxGekk opened a new pull request #30172: URL: https://github.com/apache/spark/pull/30172 ### What changes were proposed in this pull request? Return schema in SQL format instead of Catalog string from the SchemaOfJson` expression. ### Why are the changes needed? In some cases,

[GitHub] [spark] RussellSpitzer commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

RussellSpitzer commented on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-717965512 I think the use case you are specifying is rather unlikely compared to users who set this property and expect it to work or not work. It's hard for me to imagine someone

[GitHub] [spark] MaxGekk commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

MaxGekk commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-717965308 @HyukjinKwon @cloud-fan Could you take a look at this. This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

SparkQA commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-717968636 **[Test build #130372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130372/testReport)** for PR 30172 at commit [`df19069`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717968735 **[Test build #130373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130373/testReport)** for PR 30146 at commit [`13e62d5`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717973045 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717973045 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717973012 **[Test build #130373 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130373/testReport)** for PR 30146 at commit [`13e62d5`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717968735 **[Test build #130373 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130373/testReport)** for PR 30146 at commit [`13e62d5`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717973053 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

HyukjinKwon commented on a change in pull request #30172: URL: https://github.com/apache/spark/pull/30172#discussion_r513494007 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala ## @@ -801,7 +801,7 @@ case class SchemaOfJ

[GitHub] [spark] SparkQA commented on pull request #30170: [SPARK-33267][SQL] Fix NPE issue on 'In' filter when one of values contains null

SparkQA commented on pull request #30170: URL: https://github.com/apache/spark/pull/30170#issuecomment-717974835 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34973/ -

[GitHub] [spark] HyukjinKwon commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

HyukjinKwon commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-717975069 +1 from me. This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] SparkQA commented on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

SparkQA commented on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-717978929 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34974/ -

[GitHub] [spark] HyukjinKwon commented on pull request #30167: [SPARK-33240][SQL][3.0] Fail fast when fails to instantiate configured v2 session catalog

HyukjinKwon commented on pull request #30167: URL: https://github.com/apache/spark/pull/30167#issuecomment-717982630 I am not sure. It looks to me less usual to port back a fix for maintenance release when there's a possibility that breaks something that can make sense. I would be fine if

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717983578 **[Test build #130374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130374/testReport)** for PR 30146 at commit [`9d6e97a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30170: [SPARK-33267][SQL] Fix NPE issue on 'In' filter when one of values contains null

SparkQA commented on pull request #30170: URL: https://github.com/apache/spark/pull/30170#issuecomment-717988980 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34973/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30170: [SPARK-33267][SQL] Fix NPE issue on 'In' filter when one of values contains null

AmplabJenkins commented on pull request #30170: URL: https://github.com/apache/spark/pull/30170#issuecomment-717989007 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30170: [SPARK-33267][SQL] Fix NPE issue on 'In' filter when one of values contains null

AmplabJenkins removed a comment on pull request #30170: URL: https://github.com/apache/spark/pull/30170#issuecomment-717989007 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

AmplabJenkins commented on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-717994903 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

SparkQA commented on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-717994880 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34974/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

AmplabJenkins removed a comment on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-717994903 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

SparkQA commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-717996487 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34975/ -

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717996765 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34976/ -

[GitHub] [spark] SparkQA commented on pull request #30171: [SPARK-33269][INFRA] Ignore ".bsp/" directory in Git

SparkQA commented on pull request #30171: URL: https://github.com/apache/spark/pull/30171#issuecomment-717998983 **[Test build #130366 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130366/testReport)** for PR 30171 at commit [`3bb31f3`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30171: [SPARK-33269][INFRA] Ignore ".bsp/" directory in Git

AmplabJenkins removed a comment on pull request #30171: URL: https://github.com/apache/spark/pull/30171#issuecomment-717902402 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8

[GitHub] [spark] AmplabJenkins commented on pull request #30171: [SPARK-33269][INFRA] Ignore ".bsp/" directory in Git

AmplabJenkins commented on pull request #30171: URL: https://github.com/apache/spark/pull/30171#issuecomment-718000503 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA removed a comment on pull request #30171: [SPARK-33269][INFRA] Ignore ".bsp/" directory in Git

SparkQA removed a comment on pull request #30171: URL: https://github.com/apache/spark/pull/30171#issuecomment-717901556 **[Test build #130366 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130366/testReport)** for PR 30171 at commit [`3bb31f3`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30171: [SPARK-33269][INFRA] Ignore ".bsp/" directory in Git

AmplabJenkins removed a comment on pull request #30171: URL: https://github.com/apache/spark/pull/30171#issuecomment-718000503 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] wangyum commented on a change in pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

wangyum commented on a change in pull request #30146: URL: https://github.com/apache/spark/pull/30146#discussion_r513526708 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -245,6 +245,13 @@ object SQLConf { .stringConf .cre

[GitHub] [spark] wangyum commented on a change in pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

wangyum commented on a change in pull request #30146: URL: https://github.com/apache/spark/pull/30146#discussion_r513530157 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -282,6 +289,23 @@ object SQLConf { .booleanConf

[GitHub] [spark] dongjoon-hyun closed pull request #30169: [SPARK-33268][SQL][PYTHON] Fix bugs for casting data from/to PythonUserDefinedType

dongjoon-hyun closed pull request #30169: URL: https://github.com/apache/spark/pull/30169 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718014940 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34977/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718016092 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

AmplabJenkins removed a comment on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-718015823 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

AmplabJenkins commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-718015823 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] dongjoon-hyun commented on pull request #30169: [SPARK-33268][SQL][PYTHON] Fix bugs for casting data from/to PythonUserDefinedType

dongjoon-hyun commented on pull request #30169: URL: https://github.com/apache/spark/pull/30169#issuecomment-718015700 Merged to master. @maropu . There is a conflict on the release branches. Could you make a backporting PR to branch-3.0 and branch-2.4? ---

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718016063 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34976/ ---

[GitHub] [spark] SparkQA commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

SparkQA commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-718015793 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34975/ ---

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718016092 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] tgravescs commented on a change in pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

tgravescs commented on a change in pull request #29906: URL: https://github.com/apache/spark/pull/29906#discussion_r513545836 ## File path: core/src/main/scala/org/apache/spark/SparkConf.scala ## @@ -612,7 +612,29 @@ private[spark] object SparkConf extends Logging { Depr

[GitHub] [spark] tgravescs commented on a change in pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

tgravescs commented on a change in pull request #29906: URL: https://github.com/apache/spark/pull/29906#discussion_r513545930 ## File path: core/src/main/scala/org/apache/spark/scheduler/SparkListener.scala ## @@ -25,7 +25,7 @@ import scala.collection.Map import com.fasterxml.

[GitHub] [spark] dongjoon-hyun commented on pull request #30171: [SPARK-33269][INFRA] Ignore ".bsp/" directory in Git

dongjoon-hyun commented on pull request #30171: URL: https://github.com/apache/spark/pull/30171#issuecomment-718018330 Thank you, @gengliangwang and @HyukjinKwon . This is an automated message from the Apache Git Service. To

[GitHub] [spark] cloud-fan commented on pull request #30173: [SPARK-33272][SQL] prune the attributes mapping in QueryPlan.transformUpWithNewOutput

cloud-fan commented on pull request #30173: URL: https://github.com/apache/spark/pull/30173#issuecomment-718018397 cc @maropu @viirya This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] cloud-fan opened a new pull request #30173: [SPARK-33272][SQL] prune the attributes mapping in QueryPlan.transformUpWithNewOutput

cloud-fan opened a new pull request #30173: URL: https://github.com/apache/spark/pull/30173 ### What changes were proposed in this pull request? For complex query plans, `QueryPlan.transformUpWithNewOutput` will keep accumulating the attributes mapping to be propagated, which

[GitHub] [spark] SparkQA commented on pull request #30173: [SPARK-33272][SQL] prune the attributes mapping in QueryPlan.transformUpWithNewOutput

SparkQA commented on pull request #30173: URL: https://github.com/apache/spark/pull/30173#issuecomment-718020231 **[Test build #130375 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130375/testReport)** for PR 30173 at commit [`4f94664`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

cloud-fan commented on a change in pull request #30172: URL: https://github.com/apache/spark/pull/30172#discussion_r513550927 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala ## @@ -801,7 +801,7 @@ case class SchemaOfJso

[GitHub] [spark] SparkQA commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-718022918 **[Test build #130364 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130364/testReport)** for PR 29882 at commit [`6ff1af9`](https://github.co

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

HyukjinKwon commented on a change in pull request #30172: URL: https://github.com/apache/spark/pull/30172#discussion_r513552711 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala ## @@ -801,7 +801,7 @@ case class SchemaOfJ

[GitHub] [spark] AmplabJenkins commented on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins commented on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-718024013 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] tgravescs commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partition

tgravescs commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-718023729 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-718025479 **[Test build #130376 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130376/testReport)** for PR 30164 at commit [`2688df2`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513557495 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala ## @@ -320,15 +339,22 @@ trait DivModLike extends

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513558248 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala ## @@ -530,15 +581,22 @@ case class Pmod(left: Exp

[GitHub] [spark] MaxGekk commented on a change in pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

MaxGekk commented on a change in pull request #30172: URL: https://github.com/apache/spark/pull/30172#discussion_r513558983 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala ## @@ -801,7 +801,7 @@ case class SchemaOfJson(

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513558976 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala ## @@ -560,6 +618,17 @@ case class Pmod(left: Expr

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513559156 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala ## @@ -560,6 +618,17 @@ case class Pmod(left: Expr

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513559516 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/arithmetic.scala ## @@ -560,6 +618,17 @@ case class Pmod(left: Expr

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for mergi

AmplabJenkins removed a comment on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-717586016 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718030287 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34977/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718030318 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-718024013 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] SparkQA removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

SparkQA removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-717880163 **[Test build #130364 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130364/testReport)** for PR 29882 at commit [`6ff1af9`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

AmplabJenkins removed a comment on pull request #29882: URL: https://github.com/apache/spark/pull/29882#issuecomment-718024028 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718030318 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] warrenzhu25 commented on pull request #29247: [SPARK-32446][SHS] Add new executor metrics summary REST APIs

warrenzhu25 commented on pull request #29247: URL: https://github.com/apache/spark/pull/29247#issuecomment-718030886 @gengliangwang Could you help retest? The failed sparkr tests should be unrelated with this change. This is

[GitHub] [spark] MaxGekk commented on a change in pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

MaxGekk commented on a change in pull request #30172: URL: https://github.com/apache/spark/pull/30172#discussion_r513558983 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/jsonExpressions.scala ## @@ -801,7 +801,7 @@ case class SchemaOfJson(

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513563441 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ArithmeticExpressionSuite.scala ## @@ -500,4 +524,53 @@ class Arith

[GitHub] [spark] cloud-fan commented on a change in pull request #29882: [SPARK-33008][SQL] Division by zero on divide-like operations returns incorrect result

cloud-fan commented on a change in pull request #29882: URL: https://github.com/apache/spark/pull/29882#discussion_r513564618 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/ArithmeticExpressionSuite.scala ## @@ -500,4 +524,53 @@ class Arith

[GitHub] [spark] BigaDev opened a new pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

BigaDev opened a new pull request #30174: URL: https://github.com/apache/spark/pull/30174 **What changes were proposed in this pull request?** Use Hadoop Free Build Setup for Spark 2.4.7 on Kubernetes **Why are the changes needed?** To run the Hadoop free build of Spark

[GitHub] [spark] OYousryB commented on pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

OYousryB commented on pull request #30174: URL: https://github.com/apache/spark/pull/30174#issuecomment-718035283 This is great, I was looking for something like this for a long time 💯 💯 This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

AmplabJenkins commented on pull request #30174: URL: https://github.com/apache/spark/pull/30174#issuecomment-718035653 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

AmplabJenkins removed a comment on pull request #30174: URL: https://github.com/apache/spark/pull/30174#issuecomment-718035653 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #30174: [SPARK-33271] load HADOOP_HOME and SPARK_DIST_CLASSPATH in class path

AmplabJenkins commented on pull request #30174: URL: https://github.com/apache/spark/pull/30174#issuecomment-718036657 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

SparkQA commented on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-718039706 **[Test build #130370 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130370/testReport)** for PR 29906 at commit [`b38dd66`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

SparkQA removed a comment on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-717949324 **[Test build #130370 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130370/testReport)** for PR 29906 at commit [`b38dd66`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

AmplabJenkins commented on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-718041030 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

AmplabJenkins removed a comment on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-718041030 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #29906: [SPARK-32037][CORE] Rename blacklisting feature

AmplabJenkins removed a comment on pull request #29906: URL: https://github.com/apache/spark/pull/29906#issuecomment-718041041 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130

[GitHub] [spark] dongjoon-hyun commented on pull request #30165: [SPARK-33264][SQL][DOCS] Add a dedicated page for SQL-on-file in SQL documents

dongjoon-hyun commented on pull request #30165: URL: https://github.com/apache/spark/pull/30165#issuecomment-718044389 +1, late LGTM. Thanks! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] SparkQA commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

SparkQA commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-718051310 **[Test build #130377 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130377/testReport)** for PR 30172 at commit [`718b10e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30173: [SPARK-33272][SQL] prune the attributes mapping in QueryPlan.transformUpWithNewOutput

SparkQA commented on pull request #30173: URL: https://github.com/apache/spark/pull/30173#issuecomment-718051309 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34978/ -

[GitHub] [spark] SparkQA commented on pull request #30164: [SPARK-32919][SHUFFLE] Driver side changes for coordinating push based shuffle by selecting external shuffle services for merging partitions

SparkQA commented on pull request #30164: URL: https://github.com/apache/spark/pull/30164#issuecomment-718053697 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/34979/ -

[GitHub] [spark] viirya commented on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

viirya commented on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718054995 > While we may not want to run all stateful tests with all compression, can we make sure we run basic tests against all compressions? All tests in StateStoreSuiteBase may need to

[GitHub] [spark] viirya edited a comment on pull request #30162: [SPARK-33263][SS] Configurable StateStore compression codec

viirya edited a comment on pull request #30162: URL: https://github.com/apache/spark/pull/30162#issuecomment-718054995 > While we may not want to run all stateful tests with all compression, can we make sure we run basic tests against all compressions? All tests in StateStoreSuiteBase may

[GitHub] [spark] warrenzhu25 opened a new pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

warrenzhu25 opened a new pull request #30175: URL: https://github.com/apache/spark/pull/30175 ### What changes were proposed in this pull request? Add check for total executor cores when `SetReaderPartitions` message received. ### Why are the changes needed? Current behavior is

[GitHub] [spark] SparkQA commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718058851 **[Test build #130374 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130374/testReport)** for PR 30146 at commit [`9d6e97a`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

SparkQA removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-717983578 **[Test build #130374 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130374/testReport)** for PR 30146 at commit [`9d6e97a`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins commented on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718059084 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718059084 Merged build finished. Test FAILed. This is an automated message from the Apache Git Service. To r

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30146: [SPARK-33241][SQL] Dynamic pruning on data column

AmplabJenkins removed a comment on pull request #30146: URL: https://github.com/apache/spark/pull/30146#issuecomment-718059093 Test FAILed. Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/130

[GitHub] [spark] AmplabJenkins commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-718060586 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] otterc commented on a change in pull request #30062: [SPARK-32916][SHUFFLE] Implementation of shuffle service that leverages push-based shuffle in YARN deployment mode

otterc commented on a change in pull request #30062: URL: https://github.com/apache/spark/pull/30062#discussion_r513598673 ## File path: common/network-yarn/src/main/java/org/apache/spark/network/yarn/YarnShuffleService.java ## @@ -297,6 +306,7 @@ public void stopApplication(A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-718060586 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] AmplabJenkins commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-718061329 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

SparkQA commented on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-718062950 **[Test build #130372 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130372/testReport)** for PR 30172 at commit [`df19069`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30172: [SPARK-33270][SQL] Return SQL schema instead of Catalog string from the `SchemaOfJson` expression

SparkQA removed a comment on pull request #30172: URL: https://github.com/apache/spark/pull/30172#issuecomment-717968636 **[Test build #130372 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/130372/testReport)** for PR 30172 at commit [`df19069`](https://gi

<    1   2   3   4   5   6   7   >