[GitHub] [spark] SparkQA removed a comment on pull request #30517: [DO-NOT-MERGE][test-maven] Test compatibility for Parquet 1.11.1, Avro 1.10.0 and Hive 2.3.8

SparkQA removed a comment on pull request #30517: URL: https://github.com/apache/spark/pull/30517#issuecomment-734689415 **[Test build #131867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131867/testReport)** for PR 30517 at commit [`5937a41`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30517: [DO-NOT-MERGE][test-maven] Test compatibility for Parquet 1.11.1, Avro 1.10.0 and Hive 2.3.8

SparkQA commented on pull request #30517: URL: https://github.com/apache/spark/pull/30517#issuecomment-734696370 **[Test build #131867 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131867/testReport)** for PR 30517 at commit [`5937a41`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

cloud-fan commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-734695503 Not that, before this PR, people can already create varchar type column but it has no length check during write. People can already create char type column with hive table but

[GitHub] [spark] SparkQA removed a comment on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

SparkQA removed a comment on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734625327 **[Test build #131858 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131858/testReport)** for PR 30518 at commit [`ca79a48`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

SparkQA commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734693388 **[Test build #131858 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131858/testReport)** for PR 30518 at commit [`ca79a48`](https://github.co

[GitHub] [spark] cloud-fan commented on pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

cloud-fan commented on pull request #30494: URL: https://github.com/apache/spark/pull/30494#issuecomment-734692905 then it's not a user-specified partitioning and we don't need to respect it. This is an automated message from

[GitHub] [spark] cloud-fan commented on a change in pull request #30403: [SPARK-33448][SQL] Support CACHE/UNCACHE TABLE commands for v2 tables

cloud-fan commented on a change in pull request #30403: URL: https://github.com/apache/spark/pull/30403#discussion_r531426411 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/cache.scala ## @@ -51,32 +54,35 @@ case class CacheTableCommand( if

[GitHub] [spark] LuciferYang commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

LuciferYang commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r531425281 ## File path: pom.xml ## @@ -164,6 +164,7 @@ 3.2.2 2.12.10 2.12 +-Ywarn-unused-import Review comment: OK ~ I'll collect the

[GitHub] [spark] hddong opened a new pull request #30520: [SPARK-33357][k8s]Support SparkLauncher in Kubernetes

hddong opened a new pull request #30520: URL: https://github.com/apache/spark/pull/30520 ### What changes were proposed in this pull request? ### Why are the changes needed? ### Does this PR introduce _any_ user-facing change? ### How was

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30515: [SPARK-33570][SQL][TESTS] Set the proper version of gssapi plugin automatically for MariaDBKrbIntegrationsuite

AmplabJenkins removed a comment on pull request #30515: URL: https://github.com/apache/spark/pull/30515#issuecomment-734689927 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30515: [SPARK-33570][SQL][TESTS] Set the proper version of gssapi plugin automatically for MariaDBKrbIntegrationsuite

AmplabJenkins commented on pull request #30515: URL: https://github.com/apache/spark/pull/30515#issuecomment-734689927 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

SparkQA commented on pull request #29893: URL: https://github.com/apache/spark/pull/29893#issuecomment-734689757 **[Test build #131868 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131868/testReport)** for PR 29893 at commit [`6f9b42e`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30517: [DO-NOT-MERGE][test-maven] Test compatibility for Parquet 1.11.1, Avro 1.10.0 and Hive 2.3.8

SparkQA commented on pull request #30517: URL: https://github.com/apache/spark/pull/30517#issuecomment-734689415 **[Test build #131867 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131867/testReport)** for PR 30517 at commit [`5937a41`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30517: [DO-NOT-MERGE][test-maven] Test compatibility for Parquet 1.11.1, Avro 1.10.0 and Hive 2.3.8

AmplabJenkins removed a comment on pull request #30517: URL: https://github.com/apache/spark/pull/30517#issuecomment-734358435 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531422458 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLInsertTestSuite.scala ## @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [spark] LuciferYang commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

LuciferYang commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734687799 thx @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531421673 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLInsertTestSuite.scala ## @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531421389 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLInsertTestSuite.scala ## @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [spark] yaooqinn commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

yaooqinn commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531421331 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -3104,6 +3106,62 @@ class Analyzer(override val ca

[GitHub] [spark] manuzhang commented on pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

manuzhang commented on pull request #30494: URL: https://github.com/apache/spark/pull/30494#issuecomment-734686995 @cloud-fan No explicit `repartition` but implicit through join with `spark.sql.shuffle.partitions` configuration. ---

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531420563 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- RO

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531420563 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- RO

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531420156 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLInsertTestSuite.scala ## @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531419671 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLInsertTestSuite.scala ## @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531419455 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- ROLLUP

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531419418 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLInsertTestSuite.scala ## @@ -0,0 +1,244 @@ +/* + * Licensed to the Apache Software Foundat

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531418859 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- RO

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531418779 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameWriter.scala ## @@ -522,7 +523,8 @@ final class DataFrameWriter[T] private[sql](ds:

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531418236 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- ROLLUP

[GitHub] [spark] cloud-fan commented on a change in pull request #29893: [SPARK-32976][SQL]Support column list in INSERT statement

cloud-fan commented on a change in pull request #29893: URL: https://github.com/apache/spark/pull/29893#discussion_r531418233 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -3104,6 +3106,62 @@ class Analyzer(override val c

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

AmplabJenkins removed a comment on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-734171079 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

SparkQA commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-734683690 **[Test build #131866 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131866/testReport)** for PR 30412 at commit [`69adca5`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

SparkQA commented on pull request #30442: URL: https://github.com/apache/spark/pull/30442#issuecomment-734683691 **[Test build #131865 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131865/testReport)** for PR 30442 at commit [`4706576`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

AmplabJenkins removed a comment on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734665677 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

AmplabJenkins commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734683091 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] cloud-fan commented on a change in pull request #30465: [SPARK-33045][SQL][FOLLOWUP] Support built-in function like_any and fix StackOverflowError issue.

cloud-fan commented on a change in pull request #30465: URL: https://github.com/apache/spark/pull/30465#discussion_r531417281 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/AstBuilder.scala ## @@ -1404,12 +1396,26 @@ class AstBuilder extends Sql

[GitHub] [spark] cloud-fan closed pull request #30519: [SPARK-33575][SQL] Fix misleading exception for "ANALYZE TABLE ... FOR COLUMNS" on temporary views

cloud-fan closed pull request #30519: URL: https://github.com/apache/spark/pull/30519 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] cloud-fan commented on pull request #30519: [SPARK-33575][SQL] Fix misleading exception for "ANALYZE TABLE ... FOR COLUMNS" on temporary views

cloud-fan commented on pull request #30519: URL: https://github.com/apache/spark/pull/30519#issuecomment-734682124 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] SparkQA removed a comment on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

SparkQA removed a comment on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734622708 **[Test build #131857 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131857/testReport)** for PR 30518 at commit [`1770c56`](https://gi

[GitHub] [spark] maropu commented on pull request #30412: [SPARK-33480][SQL] Support char/varchar type

maropu commented on pull request #30412: URL: https://github.com/apache/spark/pull/30412#issuecomment-734681874 okay, it looks good to me. Btw, this new feature will land in the v3.1 release? There are still some remaining works (e.g., https://github.com/apache/spark/pull/30412#discussion_

[GitHub] [spark] SparkQA commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

SparkQA commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734681712 **[Test build #131857 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131857/testReport)** for PR 30518 at commit [`1770c56`](https://github.co

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh edited a comment on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-734679145 > > In postgres sql, it support > > > select a, b, c, count(1) from t group by a, b, cube(a, b, c); > > > select a, b, c, count(1) from t group by a, c, grouping

[GitHub] [spark] AngersZhuuuu commented on pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-734679145 > > In postgres sql, it support > > > select a, b, c, count(1) from t group by a, b, cube(a, b, c); > > > select a, b, c, count(1) from t group by a, c, grouping sets (

[GitHub] [spark] cloud-fan commented on a change in pull request #30504: [SPARK-33544][SQL] Optimizer should not insert filter when explode with CreateArray/CreateMap

cloud-fan commented on a change in pull request #30504: URL: https://github.com/apache/spark/pull/30504#discussion_r531413076 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -873,24 +873,30 @@ object InferFiltersFromGener

[GitHub] [spark] leanken commented on a change in pull request #30442: [SPARK-33498][SQL] Datetime parsing should fail if the input string can't be parsed, or the pattern string is invalid

leanken commented on a change in pull request #30442: URL: https://github.com/apache/spark/pull/30442#discussion_r531412245 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/CastSuite.scala ## @@ -945,6 +934,36 @@ abstract class AnsiCastSuiteB

[GitHub] [spark] maropu commented on pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-734677252 > In postgres sql, it support >> select a, b, c, count(1) from t group by a, b, cube(a, b, c); select a, b, c, count(1) from t group by a, c, grouping sets (a, b, (a, b));

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531411187 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- RO

[GitHub] [spark] HyukjinKwon commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

HyukjinKwon commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734675416 Thanks @LuciferYang. This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] HyukjinKwon closed pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

HyukjinKwon closed pull request #30518: URL: https://github.com/apache/spark/pull/30518 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

HyukjinKwon commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734674732 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531409067 ## File path: sql/core/src/test/resources/sql-tests/inputs/group-analytics.sql ## @@ -18,9 +18,13 @@ AS courseSales(course, year, earnings); -- ROLLUP

[GitHub] [spark] maryannxue commented on pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

maryannxue commented on pull request #30494: URL: https://github.com/apache/spark/pull/30494#issuecomment-734672087 I think the case you mentioned is exactly one of the test cases. Can you find a case that is not working? And if repartition is not specified, we should not guarantee any

[GitHub] [spark] cloud-fan commented on a change in pull request #30412: [SPARK-33480][SQL] Support char/varchar type

cloud-fan commented on a change in pull request #30412: URL: https://github.com/apache/spark/pull/30412#discussion_r531406576 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ## @@ -1181,7 +1181,9 @@ class Column(val expr: Expression) extends Logging {

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531406586 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregationClause

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

AmplabJenkins removed a comment on pull request #30494: URL: https://github.com/apache/spark/pull/30494#issuecomment-733966636 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AmplabJenkins removed a comment on pull request #30212: URL: https://github.com/apache/spark/pull/30212#issuecomment-733578781 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531405446 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregation

[GitHub] [spark] cloud-fan commented on pull request #30494: [SPARK-33551][SQL] Do not use custom shuffle reader for repartition

cloud-fan commented on pull request #30494: URL: https://github.com/apache/spark/pull/30494#issuecomment-734669644 is `repartition` involved in your case? This is an automated message from the Apache Git Service. To respond t

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531404387 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregationClause

[GitHub] [spark] prakharjain09 commented on pull request #30426: [SPARK-33486][SQL] Collapse Partial and Final physical aggregation nodes together whenever possible

prakharjain09 commented on pull request #30426: URL: https://github.com/apache/spark/pull/30426#issuecomment-734668231 > So, could you give us a concrete example of how much it will improve performance? @maropu We have seen customer queries where Aggregation happens on close to prim

[GitHub] [spark] waitinfuture commented on a change in pull request #30516: [SPARK-33498][SQL] Datetime building should fail if the year, month, ..., second combination is invalid

waitinfuture commented on a change in pull request #30516: URL: https://github.com/apache/spark/pull/30516#discussion_r531403400 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1776,23 +1776,29 @@ case class

[GitHub] [spark] beliefer commented on pull request #30512: [SPARK-28645][SQL] ParseException is thrown when the window is redefined

beliefer commented on pull request #30512: URL: https://github.com/apache/spark/pull/30512#issuecomment-734666009 > The fix looks fine. In the PR description, could you add an example query output with/without this fix? OK ---

[GitHub] [spark] AmplabJenkins commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

AmplabJenkins commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734665677 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #30515: [SPARK-33570][SQL][TESTS] Set the proper version of gssapi plugin automatically for MariaDBKrbIntegrationsuite

SparkQA commented on pull request #30515: URL: https://github.com/apache/spark/pull/30515#issuecomment-734665200 **[Test build #131864 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/131864/testReport)** for PR 30515 at commit [`164e5ea`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

AmplabJenkins removed a comment on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734663481 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

AmplabJenkins commented on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734663481 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531396333 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregation

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531397913 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -628,15 +627,17 @@ class Analyzer(override val catal

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531397620 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -659,30 +660,29 @@ class Analyzer(override val

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531397445 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -659,30 +660,29 @@ class Analyzer(override val catal

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531397397 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -659,30 +660,29 @@ class Analyzer(override val catal

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531397063 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregation

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531396965 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -611,14 +609,15 @@ class Analyzer(override val

[GitHub] [spark] imback82 commented on pull request #30519: [SPARK-33575][SQL] Fix misleading exception for "ANALYZE TABLE ... FOR COLUMNS" on temporary views

imback82 commented on pull request #30519: URL: https://github.com/apache/spark/pull/30519#issuecomment-734659790 Thanks @maropu for taking a look! cc @cloud-fan This is an automated message from the Apache Git Servic

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531396840 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -611,14 +609,15 @@ class Analyzer(override val catal

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531396333 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregation

[GitHub] [spark] sarutak commented on a change in pull request #30515: [SPARK-33570][SQL][TESTS] Set the proper version of gssapi plugin automatically for MariaDBKrbIntegrationsuite

sarutak commented on a change in pull request #30515: URL: https://github.com/apache/spark/pull/30515#discussion_r531396228 ## File path: external/docker-integration-tests/src/test/resources/mariadb_docker_entrypoint.sh ## @@ -18,7 +18,7 @@ dpkg-divert --add /bin/systemctl

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531396103 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531395757 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -611,14 +609,15 @@ class Analyzer(override val catal

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531395718 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -611,14 +609,15 @@ class Analyzer(override val catal

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531394659 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r529502777 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531394329 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531394379 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregationClause

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531394329 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30351: [SPARK-33441][BUILD] Add unused-imports compilation check and remove all unused-imports

HyukjinKwon commented on a change in pull request #30351: URL: https://github.com/apache/spark/pull/30351#discussion_r531394220 ## File path: pom.xml ## @@ -164,6 +164,7 @@ 3.2.2 2.12.10 2.12 +-Ywarn-unused-import Review comment: We can give a shot. A

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531393983 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531392644 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531393607 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregationClause

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531392644 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

AngersZh commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531392644 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregation

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531390806 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregationClause

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531391985 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregationClause

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531391314 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -593,13 +593,24 @@ fromClause ; aggregationClause

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531390806 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregationClause

[GitHub] [spark] maropu commented on a change in pull request #30212: [SPARK-33308][SQL] Refract current grouping analytics

maropu commented on a change in pull request #30212: URL: https://github.com/apache/spark/pull/30212#discussion_r531389726 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -587,13 +587,24 @@ fromClause ; aggregationClause

[GitHub] [spark] AmplabJenkins commented on pull request #24173: [SPARK-27237][SS] Introduce State schema validation among query restart

AmplabJenkins commented on pull request #24173: URL: https://github.com/apache/spark/pull/24173#issuecomment-734651544 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30519: [SPARK-33575][SQL] Fix misleading exception for "ANALYZE TABLE ... FOR COLUMNS" on temporary views

AmplabJenkins commented on pull request #30519: URL: https://github.com/apache/spark/pull/30519#issuecomment-734651413 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30421: [SPARK-33474][SQL] Support TypeConstructed partition spec value

AmplabJenkins commented on pull request #30421: URL: https://github.com/apache/spark/pull/30421#issuecomment-734650163 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30518: [SPARK-33566][CORE][SQL][SS][PYTHON] Make unescapedQuoteHandling option configurable when read CSV

AmplabJenkins removed a comment on pull request #30518: URL: https://github.com/apache/spark/pull/30518#issuecomment-734637172 This is an automated message from the Apache Git Service. To respond to the message, please log on

  1   2   3   4   5   >