[GitHub] [spark] cloud-fan commented on pull request #30870: [SPARK-33542][SQL] Group exception messages in catalyst/catalog

cloud-fan commented on pull request #30870: URL: https://github.com/apache/spark/pull/30870#issuecomment-753816984 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] yaooqinn commented on pull request #31003: [SPARK-33980][SS] invalidate char/varchar in spark.readStream.schema

yaooqinn commented on pull request #31003: URL: https://github.com/apache/spark/pull/31003#issuecomment-753815657 cc @cloud-fan @maropu @HyukjinKwon thanks~ This is an automated message from the Apache Git Service. To

[GitHub] [spark] yaooqinn opened a new pull request #31003: [SPARK-33980][SS] invalidate char/varchar in spark.readStream.schema

yaooqinn opened a new pull request #31003: URL: https://github.com/apache/spark/pull/31003 ### What changes were proposed in this pull request? invalidate char/varchar in `spark.readStream.schema` just like what we've done for `spark.read.schema` in

[GitHub] [spark] turboFei commented on a change in pull request #29982: [SPARK-33100][SQL] Ignore a semicolon inside a bracketed comment in spark-sql

turboFei commented on a change in pull request #29982: URL: https://github.com/apache/spark/pull/29982#discussion_r551158253 ## File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala ## @@ -573,4 +573,53 @@ class CliSuite extends

[GitHub] [spark] cloud-fan commented on a change in pull request #30807: [SPARK-33794][SQL]NextDay expression throw runtime IllegalArgumentException when receiving invalid input under ANSI mode

cloud-fan commented on a change in pull request #30807: URL: https://github.com/apache/spark/pull/30807#discussion_r551157960 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1170,52 +1175,73 @@ case class

[GitHub] [spark] ulysses-you commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

ulysses-you commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551157649 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -200,6 +203,7 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] viirya commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

viirya commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551156203 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

viirya commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551156203 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] viirya commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

viirya commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551156203 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #30975: [SPARK-33938][SQL] Optimize Like Any/All by LikeSimplification

cloud-fan commented on a change in pull request #30975: URL: https://github.com/apache/spark/pull/30975#discussion_r551156170 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -634,36 +636,70 @@ object LikeSimplification

[GitHub] [spark] cloud-fan commented on a change in pull request #30975: [SPARK-33938][SQL] Optimize Like Any/All by LikeSimplification

cloud-fan commented on a change in pull request #30975: URL: https://github.com/apache/spark/pull/30975#discussion_r551156002 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/expressions.scala ## @@ -634,36 +636,70 @@ object LikeSimplification

[GitHub] [spark] dongjoon-hyun commented on pull request #30962: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

dongjoon-hyun commented on pull request #30962: URL: https://github.com/apache/spark/pull/30962#issuecomment-753809646 Thank you, @zhongyu09 . This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun closed pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

dongjoon-hyun closed pull request #30738: URL: https://github.com/apache/spark/pull/30738 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #30738: [SPARK-33759][K8S] docker entrypoint should using `spark-class` for spark executor

dongjoon-hyun commented on pull request #30738: URL: https://github.com/apache/spark/pull/30738#issuecomment-753809439 I close this PR for now. Please feel free to reopen this if there is any change. Happy New Year!

[GitHub] [spark] maropu commented on pull request #29982: [SPARK-33100][SQL] Ignore a semicolon inside a bracketed comment in spark-sql

maropu commented on pull request #29982: URL: https://github.com/apache/spark/pull/29982#issuecomment-753807873 Have you checked the comment: https://github.com/apache/spark/pull/29982#discussion_r551076053 ? This is an

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551149987 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets

[GitHub] [spark] maropu commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

maropu commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551149951 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551149228 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets

[GitHub] [spark] maropu commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

maropu commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551148737 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets the

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551147824 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets

[GitHub] [spark] maropu commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

maropu commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551147461 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets the

[GitHub] [spark] chongguang commented on a change in pull request #30807: [SPARK-33794][SQL]NextDay expression throw runtime IllegalArgumentException when receiving invalid input under ANSI mode

chongguang commented on a change in pull request #30807: URL: https://github.com/apache/spark/pull/30807#discussion_r551147396 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1170,52 +1175,67 @@ case class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551146496 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala ## @@ -594,4 +594,12 @@ class

[GitHub] [spark] chongguang commented on a change in pull request #30807: [SPARK-33794][SQL]NextDay expression throw runtime IllegalArgumentException when receiving invalid input under ANSI mode

chongguang commented on a change in pull request #30807: URL: https://github.com/apache/spark/pull/30807#discussion_r551147173 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1170,52 +1175,73 @@ case class

[GitHub] [spark] dongjoon-hyun commented on pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on pull request #31002: URL: https://github.com/apache/spark/pull/31002#issuecomment-753799986 Thank you for review, @HyukjinKwon , @viirya and @maropu . After adding more new features, I'm going to update `docs/sql-data-sources-orc.md` together before Apache

[GitHub] [spark] zhongyu09 commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551146998 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala ## @@ -425,6 +425,29 @@ abstract class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551146496 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala ## @@ -594,4 +594,12 @@ class

[GitHub] [spark] zhongyu09 commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551146351 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] AngersZhuuuu commented on pull request #30957: [SPARK-31937][SQL] Support processing ArrayType/MapType/StructType data using no-serde mode script transform

AngersZh commented on pull request #30957: URL: https://github.com/apache/spark/pull/30957#issuecomment-753798073 gentle ping @HyukjinKwon @maropu Conflict solved. This is an automated message from the Apache Git

[GitHub] [spark] cloud-fan commented on pull request #30996: [SPARK-33964][SQL] Combine distinct unions in more cases

cloud-fan commented on pull request #30996: URL: https://github.com/apache/spark/pull/30996#issuecomment-753798010 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

HyukjinKwon commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551145026 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala ## @@ -594,4 +594,12 @@ class

[GitHub] [spark] cloud-fan commented on a change in pull request #30937: [SPARK-33919][SQL][TESTS] Unify v1 and v2 SHOW NAMESPACES tests

cloud-fan commented on a change in pull request #30937: URL: https://github.com/apache/spark/pull/30937#discussion_r551144655 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowNamespacesSuite.scala ## @@ -0,0 +1,72 @@ +/* + * Licensed to the

[GitHub] [spark] cloud-fan commented on a change in pull request #30937: [SPARK-33919][SQL][TESTS] Unify v1 and v2 SHOW NAMESPACES tests

cloud-fan commented on a change in pull request #30937: URL: https://github.com/apache/spark/pull/30937#discussion_r551144416 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/ShowNamespacesSuite.scala ## @@ -0,0 +1,72 @@ +/* + * Licensed to the

[GitHub] [spark] cloud-fan commented on a change in pull request #30937: [SPARK-33919][SQL][TESTS] Unify v1 and v2 SHOW NAMESPACES tests

cloud-fan commented on a change in pull request #30937: URL: https://github.com/apache/spark/pull/30937#discussion_r551144001 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/ShowNamespacesSuiteBase.scala ## @@ -0,0 +1,129 @@ +/* + * Licensed to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

HyukjinKwon commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551143994 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/orc/OrcSourceSuite.scala ## @@ -594,4 +594,12 @@ class

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551143958 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

HyukjinKwon commented on a change in pull request #31002: URL: https://github.com/apache/spark/pull/31002#discussion_r551143842 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -796,11 +796,11 @@ object SQLConf { .doc("Sets the

[GitHub] [spark] cloud-fan commented on a change in pull request #30937: [SPARK-33919][SQL][TESTS] Unify v1 and v2 SHOW NAMESPACES tests

cloud-fan commented on a change in pull request #30937: URL: https://github.com/apache/spark/pull/30937#discussion_r551143272 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/ShowNamespacesSuiteBase.scala ## @@ -0,0 +1,129 @@ +/* + * Licensed to

[GitHub] [spark] AngersZhuuuu commented on pull request #29087: [SPARK-28227][SQL] Support TRANSFORM with aggregation

AngersZh commented on pull request #29087: URL: https://github.com/apache/spark/pull/29087#issuecomment-753794280 ping @HyukjinKwon @maropu This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan edited a comment on pull request #30902: [SPARK-33888][SQL] JDBC SQL TIME type represents incorrectly as TimestampType, it should be physical Int in millis

cloud-fan edited a comment on pull request #30902: URL: https://github.com/apache/spark/pull/30902#issuecomment-753793633 thanks, merging to master! (not backporting because TIME is rarely used) This is an automated message

[GitHub] [spark] cloud-fan closed pull request #30902: [SPARK-33888][SQL] JDBC SQL TIME type represents incorrectly as TimestampType, it should be physical Int in millis

cloud-fan closed pull request #30902: URL: https://github.com/apache/spark/pull/30902 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #30902: [SPARK-33888][SQL] JDBC SQL TIME type represents incorrectly as TimestampType, it should be physical Int in millis

cloud-fan commented on pull request #30902: URL: https://github.com/apache/spark/pull/30902#issuecomment-753793633 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] HyukjinKwon closed pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

HyukjinKwon closed pull request #30973: URL: https://github.com/apache/spark/pull/30973 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

HyukjinKwon commented on pull request #30973: URL: https://github.com/apache/spark/pull/30973#issuecomment-753791010 Merged to master. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] cloud-fan commented on a change in pull request #30807: [SPARK-33794][SQL]NextDay expression throw runtime IllegalArgumentException when receiving invalid input under ANSI mode

cloud-fan commented on a change in pull request #30807: URL: https://github.com/apache/spark/pull/30807#discussion_r551138554 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1170,52 +1175,69 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #30807: [SPARK-33794][SQL]NextDay expression throw runtime IllegalArgumentException when receiving invalid input under ANSI mode

cloud-fan commented on a change in pull request #30807: URL: https://github.com/apache/spark/pull/30807#discussion_r551138222 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1170,52 +1175,73 @@ case class

[GitHub] [spark] dongjoon-hyun opened a new pull request #31002: [SPARK-33978][SQL] Support ZSTD compression in ORC data source

dongjoon-hyun opened a new pull request #31002: URL: https://github.com/apache/spark/pull/31002 ### What changes were proposed in this pull request? This PR aims to support ZSTD compression in ORC data source. ### Why are the changes needed? Apache ORC 1.6 supports ZSTD

[GitHub] [spark] AngersZhuuuu commented on pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

AngersZh commented on pull request #30973: URL: https://github.com/apache/spark/pull/30973#issuecomment-753785448 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AngersZhuuuu commented on pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

AngersZh commented on pull request #30973: URL: https://github.com/apache/spark/pull/30973#issuecomment-753784226 > Please update the PR description, too. Looks fine otherwise. Done This is an automated message

[GitHub] [spark] cloud-fan commented on pull request #30999: [SPARK-33971][SQL] Eliminate distinct from more aggregates

cloud-fan commented on pull request #30999: URL: https://github.com/apache/spark/pull/30999#issuecomment-753778464 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] beliefer commented on pull request #30982: [SPARK-33951][SQL] Distinguish the error between filter and distinct

beliefer commented on pull request #30982: URL: https://github.com/apache/spark/pull/30982#issuecomment-75346 @cloud-fan Thanks! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] beliefer commented on a change in pull request #30981: [SPARK-33947] String functions: Trim/Ltrim/Rtrim support byte array

beliefer commented on a change in pull request #30981: URL: https://github.com/apache/spark/pull/30981#discussion_r551131846 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -751,267 +751,348 @@ case class

[GitHub] [spark] beliefer commented on a change in pull request #30981: [SPARK-33947] String functions: Trim/Ltrim/Rtrim support byte array

beliefer commented on a change in pull request #30981: URL: https://github.com/apache/spark/pull/30981#discussion_r551131619 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/strings.sql.out ## @@ -977,7 +977,7 @@ struct -- !query SELECT

[GitHub] [spark] MaxGekk commented on pull request #30997: [SPARK-33965][SQL][TESTS] Recognize `spark_catalog` by `CACHE TABLE` in Hive table names

MaxGekk commented on pull request #30997: URL: https://github.com/apache/spark/pull/30997#issuecomment-753775686 @cloud-fan @HyukjinKwon Could you review this PR, please. This is an automated message from the Apache Git

[GitHub] [spark] cloud-fan commented on a change in pull request #30981: [SPARK-33947] String functions: Trim/Ltrim/Rtrim support byte array

cloud-fan commented on a change in pull request #30981: URL: https://github.com/apache/spark/pull/30981#discussion_r551130707 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/stringExpressions.scala ## @@ -751,267 +751,348 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #30981: [SPARK-33947] String functions: Trim/Ltrim/Rtrim support byte array

cloud-fan commented on a change in pull request #30981: URL: https://github.com/apache/spark/pull/30981#discussion_r551130603 ## File path: sql/core/src/test/resources/sql-tests/results/postgreSQL/strings.sql.out ## @@ -977,7 +977,7 @@ struct -- !query SELECT

[GitHub] [spark] maropu commented on pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

maropu commented on pull request #30973: URL: https://github.com/apache/spark/pull/30973#issuecomment-753774166 Please update the PR description, too. Looks fine otherwise. This is an automated message from the Apache Git

[GitHub] [spark] cloud-fan closed pull request #30987: [SPARK-33954][SQL] Some operator missing rowCount when enable CBO

cloud-fan closed pull request #30987: URL: https://github.com/apache/spark/pull/30987 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #30987: [SPARK-33954][SQL] Some operator missing rowCount when enable CBO

cloud-fan commented on pull request #30987: URL: https://github.com/apache/spark/pull/30987#issuecomment-753772579 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cloud-fan commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

cloud-fan commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551128113 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -200,6 +203,7 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] cloud-fan closed pull request #30982: [SPARK-33951][SQL] Distinguish the error between filter and distinct

cloud-fan closed pull request #30982: URL: https://github.com/apache/spark/pull/30982 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #30982: [SPARK-33951][SQL] Distinguish the error between filter and distinct

cloud-fan commented on pull request #30982: URL: https://github.com/apache/spark/pull/30982#issuecomment-753769608 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] dongjoon-hyun closed pull request #30977: [SPARK-33945][SQL][3.1] Handles a random seed consisting of an expr tree

dongjoon-hyun closed pull request #30977: URL: https://github.com/apache/spark/pull/30977 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] cloud-fan commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

cloud-fan commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551124942 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/joins/BroadcastJoinSuite.scala ## @@ -425,6 +425,29 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

cloud-fan commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551124867 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30984: [SPARK-33915] Allow json expression to be pushable column

HyukjinKwon commented on a change in pull request #30984: URL: https://github.com/apache/spark/pull/30984#discussion_r551124736 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/DataSourceStrategy.scala ## @@ -737,6 +737,10 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

cloud-fan commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551123915 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] cloud-fan commented on a change in pull request #30998: [SPARK-33933][SQL] Materialize BroadcastQueryState first to avoid broadcast timeout in AQE

cloud-fan commented on a change in pull request #30998: URL: https://github.com/apache/spark/pull/30998#discussion_r551123915 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -189,8 +189,17 @@ case class

[GitHub] [spark] Ngone51 commented on a change in pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

Ngone51 commented on a change in pull request #30650: URL: https://github.com/apache/spark/pull/30650#discussion_r551123668 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSchedulerImpl.scala ## @@ -661,35 +667,51 @@ private[spark] class TaskSchedulerImpl(

[GitHub] [spark] Ngone51 commented on a change in pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

Ngone51 commented on a change in pull request #30650: URL: https://github.com/apache/spark/pull/30650#discussion_r551123280 ## File path: core/src/main/scala/org/apache/spark/scheduler/TaskSetManager.scala ## @@ -439,77 +446,109 @@ private[spark] class TaskSetManager(

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

AngersZh commented on a change in pull request #30973: URL: https://github.com/apache/spark/pull/30973#discussion_r551119978 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -72,6 +72,10 @@ trait

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30973: [SPARK-33934][SQL] Add SparkFile's root dir to env property PATH

AngersZh commented on a change in pull request #30973: URL: https://github.com/apache/spark/pull/30973#discussion_r551119912 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala ## @@ -470,6 +470,119 @@ abstract class

[GitHub] [spark] AngersZhuuuu commented on pull request #31001: [SPARK-26399][CORE] Add new stage-level REST APIs and parameters to get stage level executor peak metrics distribution

AngersZh commented on pull request #31001: URL: https://github.com/apache/spark/pull/31001#issuecomment-753753335 FYI @ron8hu @gengliangwang @maropu @warrenzhu25 @dongjoon-hyun Since some logic is same as https://github.com/apache/spark/pull/29247, so I just use his code and I

[GitHub] [spark] AngersZhuuuu opened a new pull request #31001: [SPARK-26399][CORE] Add new stage-level REST APIs and parameters to get stage level executor peak metrics distribution

AngersZh opened a new pull request #31001: URL: https://github.com/apache/spark/pull/31001 ### What changes were proposed in this pull request? Add restful api for user to get stage level executor peak metrics distribution. - **/applications//stages//< stage_attempt_id

[GitHub] [spark] HyukjinKwon commented on pull request #30973: [SPARK-33934][SQL] Support user defined script command wrapper and handle cmd arg like hive

HyukjinKwon commented on pull request #30973: URL: https://github.com/apache/spark/pull/30973#issuecomment-753750997 LGTM otherwise This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30973: [SPARK-33934][SQL] Support user defined script command wrapper and handle cmd arg like hive

HyukjinKwon commented on a change in pull request #30973: URL: https://github.com/apache/spark/pull/30973#discussion_r551113902 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/BaseScriptTransformationSuite.scala ## @@ -470,6 +470,119 @@ abstract class

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30973: [SPARK-33934][SQL] Support user defined script command wrapper and handle cmd arg like hive

HyukjinKwon commented on a change in pull request #30973: URL: https://github.com/apache/spark/pull/30973#discussion_r551113731 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -72,6 +72,10 @@ trait

[GitHub] [spark] cloud-fan commented on pull request #30995: [SPARK-33963][SQL] Canonicalize `HiveTableRelation` w/o table stats

cloud-fan commented on pull request #30995: URL: https://github.com/apache/spark/pull/30995#issuecomment-753749400 late LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] beliefer commented on pull request #30981: [SPARK-33947] String functions: Trim/Ltrim/Rtrim support byte array

beliefer commented on pull request #30981: URL: https://github.com/apache/spark/pull/30981#issuecomment-753747597 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] beliefer commented on pull request #30982: [SPARK-33951][SQL] Distinguish the error between filter and distinct

beliefer commented on pull request #30982: URL: https://github.com/apache/spark/pull/30982#issuecomment-753747486 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] cloud-fan commented on pull request #30983: [SPARK-33950][SQL] Refresh cache in v1 `ALTER TABLE .. DROP PARTITION`

cloud-fan commented on pull request #30983: URL: https://github.com/apache/spark/pull/30983#issuecomment-753746443 @MaxGekk can you open backport PRs for 3.1/3.0/2.4? thanks! This is an automated message from the Apache Git

[GitHub] [spark] cloud-fan closed pull request #30983: [SPARK-33950][SQL] Refresh cache in v1 `ALTER TABLE .. DROP PARTITION`

cloud-fan closed pull request #30983: URL: https://github.com/apache/spark/pull/30983 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on pull request #30983: [SPARK-33950][SQL] Refresh cache in v1 `ALTER TABLE .. DROP PARTITION`

cloud-fan commented on pull request #30983: URL: https://github.com/apache/spark/pull/30983#issuecomment-753746321 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] bmarcott edited a comment on pull request #30889: [SPARK-33398] Fix loading tree models prior to Spark 3.0

bmarcott edited a comment on pull request #30889: URL: https://github.com/apache/spark/pull/30889#issuecomment-753745535 @zhengruifeng @srowen Thanks for putting in the fix! This is an automated message from the Apache Git

[GitHub] [spark] bmarcott commented on pull request #30889: [SPARK-33398] Fix loading tree models prior to Spark 3.0

bmarcott commented on pull request #30889: URL: https://github.com/apache/spark/pull/30889#issuecomment-753745535 @zhengruifeng Thanks for putting in the fix! This is an automated message from the Apache Git Service. To

[GitHub] [spark] turboFei commented on a change in pull request #29982: [SPARK-33100][SQL] Ignore a semicolon inside a bracketed comment in spark-sql

turboFei commented on a change in pull request #29982: URL: https://github.com/apache/spark/pull/29982#discussion_r551109247 ## File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala ## @@ -573,4 +573,37 @@ class CliSuite extends

[GitHub] [spark] WeichenXu123 commented on pull request #30889: [SPARK-33398] Fix loading tree models prior to Spark 3.0

WeichenXu123 commented on pull request #30889: URL: https://github.com/apache/spark/pull/30889#issuecomment-753742450 Good works! This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] ulysses-you commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named use UnresolvedAlias to assign name

ulysses-you commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551100491 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -200,6 +203,7 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] ulysses-you commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named use UnresolvedAlias to assign name

ulysses-you commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551100405 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -165,6 +165,10 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] wangyum commented on pull request #29490: [SPARK-32668][SQL] HiveGenericUDTF initialize UDTF should use StructObjectInspector method

wangyum commented on pull request #29490: URL: https://github.com/apache/spark/pull/29490#issuecomment-753729175 Do we still need this error message?

[GitHub] [spark] sunchao commented on pull request #29490: [SPARK-32668][SQL] HiveGenericUDTF initialize UDTF should use StructObjectInspector method

sunchao commented on pull request #29490: URL: https://github.com/apache/spark/pull/29490#issuecomment-753728221 @ulysses-you nvm please ignore my comment above :) I was thinking the case where Spark somehow loads the Hive 0.12 `GenericUDTF` class when executing user's custom UDTF but it

[GitHub] [spark] turboFei commented on a change in pull request #29982: [SPARK-33100][SQL] Ignore a semicolon inside a bracketed comment in spark-sql

turboFei commented on a change in pull request #29982: URL: https://github.com/apache/spark/pull/29982#discussion_r551093127 ## File path: sql/hive-thriftserver/src/test/scala/org/apache/spark/sql/hive/thriftserver/CliSuite.scala ## @@ -573,4 +573,37 @@ class CliSuite extends

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30973: [SPARK-33934][SQL] Support user defined script command wrapper and handle cmd arg like hive

AngersZh commented on a change in pull request #30973: URL: https://github.com/apache/spark/pull/30973#discussion_r551090642 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/BaseScriptTransformationExec.scala ## @@ -69,9 +75,19 @@ trait

[GitHub] [spark] ulysses-you commented on pull request #29490: [SPARK-32668][SQL] HiveGenericUDTF initialize UDTF should use StructObjectInspector method

ulysses-you commented on pull request #29490: URL: https://github.com/apache/spark/pull/29490#issuecomment-753722452 @sunchao you mean user create a permanent udf which is from Hive0.12 build-in function ? If so I believe it's really rare ..

[GitHub] [spark] AngersZhuuuu commented on pull request #30973: [SPARK-33934][SQL] Support user defined script command wrapper and handle cmd arg like hive

AngersZh commented on pull request #30973: URL: https://github.com/apache/spark/pull/30973#issuecomment-753720540 > This new feature itself looks useful. Btw, are you planning to make a PR to add a dedicated SQL document page for the `TRANSFORM`-related functionality, @AngersZh ?

[GitHub] [spark] maropu commented on pull request #30974: [SPARK-33939][SQL] Make Column.named use UnresolvedAlias to assign name

maropu commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-753720385 I left the minor comments and it looks fine otherwise. This is an automated message from the Apache Git Service.

[GitHub] [spark] maropu commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named use UnresolvedAlias to assign name

maropu commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551088050 ## File path: sql/core/src/test/scala/org/apache/spark/sql/DataFrameSuite.scala ## @@ -2613,6 +2613,19 @@ class DataFrameSuite extends QueryTest val

[GitHub] [spark] maropu commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named use UnresolvedAlias to assign name

maropu commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551087881 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -165,6 +165,10 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] maropu commented on a change in pull request #30974: [SPARK-33939][SQL] Make Column.named use UnresolvedAlias to assign name

maropu commented on a change in pull request #30974: URL: https://github.com/apache/spark/pull/30974#discussion_r551087838 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -200,6 +203,7 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] zhengruifeng commented on pull request #30889: [SPARK-33398] Fix loading tree models prior to Spark 3.0

zhengruifeng commented on pull request #30889: URL: https://github.com/apache/spark/pull/30889#issuecomment-753718038 Thanks @srowen @bmarcott for reviewing! This is an automated message from the Apache Git Service. To

  1   2   >