[GitHub] [spark] cloud-fan commented on a change in pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

cloud-fan commented on a change in pull request #33182: URL: https://github.com/apache/spark/pull/33182#discussion_r66436 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -419,6 +419,15 @@ object SQLConf { .booleanConf

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33227: [SPARK-35972][SQL][3.1] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

AmplabJenkins removed a comment on pull request #33227: URL: https://github.com/apache/spark/pull/33227#issuecomment-874600241 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45203/

[GitHub] [spark] MaxGekk commented on pull request #33221: [SPARK-35977][SQL] Support non-reserved keyword TIMESTAMP_NTZ

MaxGekk commented on pull request #33221: URL: https://github.com/apache/spark/pull/33221#issuecomment-874295346 +1, LGTM. GA passed. Merging to master/3.2. https://user-images.githubusercontent.com/1580697/124512219-847b0680-dde0-11eb-9bc5-900428cee693.png;> Thank you, @gengliangwang

[GitHub] [spark] c21 commented on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

c21 commented on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-874459010 Thank you @cloud-fan and @HyukjinKwon for review! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA removed a comment on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

SparkQA removed a comment on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-874440343 **[Test build #140679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140679/testReport)** for PR 32959 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

SparkQA removed a comment on pull request #32934: URL: https://github.com/apache/spark/pull/32934#issuecomment-874174521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33226: [SPARK-35999][SQL] Make from_csv/to_csv to handle day-time intervals properly

SparkQA commented on pull request #33226: URL: https://github.com/apache/spark/pull/33226#issuecomment-87450 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] jerqi commented on a change in pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

jerqi commented on a change in pull request #33204: URL: https://github.com/apache/spark/pull/33204#discussion_r664192279 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -910,4 +910,20 @@ abstract class SQLViewSuite extends

[GitHub] [spark] SparkQA commented on pull request #33224: [SPARK-35978][SQL] Support non-reserved keyword TIMESTAMP_LTZ

SparkQA commented on pull request #33224: URL: https://github.com/apache/spark/pull/33224#issuecomment-874460457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HeartSaVioR commented on pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

HeartSaVioR commented on pull request #33220: URL: https://github.com/apache/spark/pull/33220#issuecomment-874363986 FYI, build 140666 failed from the same test. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] AmplabJenkins commented on pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

AmplabJenkins commented on pull request #32934: URL: https://github.com/apache/spark/pull/32934#issuecomment-874218239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cloud-fan commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

cloud-fan commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-874248671 @linhongliu-db can you fix the code conflicts? thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] MaxGekk commented on a change in pull request #33217: [SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

MaxGekk commented on a change in pull request #33217: URL: https://github.com/apache/spark/pull/33217#discussion_r664074226 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/IntervalUtils.scala ## @@ -119,6 +119,39 @@ object IntervalUtils { }

[GitHub] [spark] LuciferYang commented on a change in pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

LuciferYang commented on a change in pull request #33220: URL: https://github.com/apache/spark/pull/33220#discussion_r664303141 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/streaming/state/RocksDBSuite.scala ## @@ -337,12 +338,12 @@ class RocksDBSuite

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33223: [SPARK-35684][INFRA][PYTHON] Bump up mypy version in GitHub Actions

AmplabJenkins removed a comment on pull request #33223: URL: https://github.com/apache/spark/pull/33223#issuecomment-874422408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] HeartSaVioR commented on pull request #33187: [WIP][SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

HeartSaVioR commented on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874415015 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] viirya closed pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

viirya closed pull request #33183: URL: https://github.com/apache/spark/pull/33183 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan closed pull request #33226: [SPARK-35999][SQL] Make from_csv/to_csv to handle day-time intervals properly

cloud-fan closed pull request #33226: URL: https://github.com/apache/spark/pull/33226 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] xuanyuanking commented on pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

xuanyuanking commented on pull request #33220: URL: https://github.com/apache/spark/pull/33220#issuecomment-874216392 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] sarutak commented on pull request #33226: [SPARK-35999][SQL] Make from_csv/to_csv to handle day-time intervals properly

sarutak commented on pull request #33226: URL: https://github.com/apache/spark/pull/33226#issuecomment-874497052 cc: @MaxGekk I think #33225 and this resolve the issue pointed out [here](https://github.com/apache/spark/pull/32949/files#r659884896). -- This is an automated message

[GitHub] [spark] AmplabJenkins commented on pull request #33223: [SPARK-35684][INFRA][PYTHON] Bump up mypy version in GitHub Actions

AmplabJenkins commented on pull request #33223: URL: https://github.com/apache/spark/pull/33223#issuecomment-874422408 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] Peng-Lei commented on pull request #33219: [SPARK-36012][SQL] Add null flag when SHOW CREATE TABLE in v2

Peng-Lei commented on pull request #33219: URL: https://github.com/apache/spark/pull/33219#issuecomment-874497006 @cloud-fan Could you take a look? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] MaxGekk commented on pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

MaxGekk commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874658612 +1, LGTM. Merging to master/3.2. https://user-images.githubusercontent.com/1580697/124588101-19254900-de61-11eb-8b8e-cb65b8b8f685.png;> Thank you, @AngersZh . --

[GitHub] [spark] jerqi commented on pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

jerqi commented on pull request #33204: URL: https://github.com/apache/spark/pull/33204#issuecomment-874660371 > @jerqi can you help to create backport PRs for 3.1 and 3.0? thanks! Ok, it's my pleasure. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins commented on pull request #33221: [SPARK-35977][SQL] Support non-reserved keyword TIMESTAMP_NTZ

AmplabJenkins commented on pull request #33221: URL: https://github.com/apache/spark/pull/33221#issuecomment-874303207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] sarutak edited a comment on pull request #32949: [SPARK-35749][SPARK-35773][SQL] Parse unit list interval literals as tightest year-month/day-time interval types

sarutak edited a comment on pull request #32949: URL: https://github.com/apache/spark/pull/32949#issuecomment-874467563 @cloud-fan This is in progress. SPARK-35983 and SPARK-35999 block this to fix the issue pointed out

[GitHub] [spark] SparkQA commented on pull request #33222: [SPARK-36020][SQL] Check logical link in remove redundant projects

SparkQA commented on pull request #33222: URL: https://github.com/apache/spark/pull/33222#issuecomment-874304157 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cloud-fan commented on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

cloud-fan commented on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-874250103 cc @viirya again -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] venkata91 commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

venkata91 commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r664110967 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -2225,4 +2225,14 @@ package object config { .stringConf

[GitHub] [spark] AmplabJenkins commented on pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

AmplabJenkins commented on pull request #33220: URL: https://github.com/apache/spark/pull/33220#issuecomment-874236457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33228: Update README.md

AmplabJenkins commented on pull request #33228: URL: https://github.com/apache/spark/pull/33228#issuecomment-874556258 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] Peng-Lei edited a comment on pull request #33132: [SPARK-35926][SQL] Add support YearMonthIntervalType for width_bucket

Peng-Lei edited a comment on pull request #33132: URL: https://github.com/apache/spark/pull/33132#issuecomment-874496061 > > Why are the changes needed? > > 35926 > > @Peng-Lei I would like to see why the changes are really needed. Could you explain that, please. > > Just

[GitHub] [spark] Shockang edited a comment on pull request #33101: [SPARK-35907][CORE] Instead of File#mkdirs, Files#createDirectories is expected.

Shockang edited a comment on pull request #33101: URL: https://github.com/apache/spark/pull/33101#issuecomment-874215443 > Looks fine. The tests may be overkill but OK :) > Are there any other instance of .mkdirs() out there that should use the utility method, while we're here?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33225: [SPARK-35983][SQL] Allow from_json/to_json for map types where value types are day-time intervals

AmplabJenkins removed a comment on pull request #33225: URL: https://github.com/apache/spark/pull/33225#issuecomment-874545801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] cloud-fan closed pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

cloud-fan closed pull request #33213: URL: https://github.com/apache/spark/pull/33213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on a change in pull request #33229: [SPARK-36025][SQL][TESTS] Reduce the run time of DateExpressionsSuite

cloud-fan commented on a change in pull request #33229: URL: https://github.com/apache/spark/pull/33229#discussion_r664397897 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeTestUtils.scala ## @@ -54,6 +55,13 @@ object DateTimeTestUtils {

[GitHub] [spark] cloud-fan commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

cloud-fan commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r664348918 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -662,6 +662,14 @@ case class

[GitHub] [spark] gengliangwang commented on pull request #33224: [SPARK-35978][SQL] Support non-reserved keyword TIMESTAMP_LTZ

gengliangwang commented on pull request #33224: URL: https://github.com/apache/spark/pull/33224#issuecomment-874497744 Merging to master/3.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] Peng-Lei commented on pull request #33132: [SPARK-35926][SQL] Add support YearMonthIntervalType for width_bucket

Peng-Lei commented on pull request #33132: URL: https://github.com/apache/spark/pull/33132#issuecomment-874496061 > > Why are the changes needed? > > 35926 > > @Peng-Lei I would like to see why the changes are really needed. Could you explain that, please. > > Just in

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r664372808 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -662,6 +662,14 @@ case class

[GitHub] [spark] cloud-fan closed pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

cloud-fan closed pull request #33204: URL: https://github.com/apache/spark/pull/33204 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA removed a comment on pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

SparkQA removed a comment on pull request #33220: URL: https://github.com/apache/spark/pull/33220#issuecomment-874237946 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33224: [SPARK-35978][SQL] Support non-reserved keyword TIMESTAMP_LTZ

AmplabJenkins commented on pull request #33224: URL: https://github.com/apache/spark/pull/33224#issuecomment-874462583 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33222: [SPARK-36020][SQL] Check logical link in remove redundant projects

AmplabJenkins removed a comment on pull request #33222: URL: https://github.com/apache/spark/pull/33222#issuecomment-874306368 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33187: [SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

AmplabJenkins commented on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874545800 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] viirya commented on pull request #30565: [WIP][SPARK-33625][SQL] Subexpression elimination for whole-stage codegen in Filter

viirya commented on pull request #30565: URL: https://github.com/apache/spark/pull/30565#issuecomment-874534597 Oh, seems there is still a trouble. The local inputs for subexpressions for each predicate. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA removed a comment on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

SparkQA removed a comment on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-874391662 **[Test build #140673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140673/testReport)** for PR 33183 at commit

[GitHub] [spark] xuanyuanking commented on pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

xuanyuanking commented on pull request #32934: URL: https://github.com/apache/spark/pull/32934#issuecomment-874378032 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33217: [SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

AmplabJenkins removed a comment on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874065592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] MaxGekk commented on a change in pull request #33229: [SPARK-36025][SQL][TESTS] Reduce the run time of DateExpressionsSuite

MaxGekk commented on a change in pull request #33229: URL: https://github.com/apache/spark/pull/33229#discussion_r664452157 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/expressions/DateExpressionsSuite.scala ## @@ -122,8 +124,8 @@ class

[GitHub] [spark] MaxGekk closed pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

MaxGekk closed pull request #33217: URL: https://github.com/apache/spark/pull/33217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

SparkQA commented on pull request #31763: URL: https://github.com/apache/spark/pull/31763#issuecomment-873893785 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

AmplabJenkins removed a comment on pull request #32934: URL: https://github.com/apache/spark/pull/32934#issuecomment-874218239 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #33224: [SPARK-35978][SQL] Support non-reserved keyword TIMESTAMP_LTZ

SparkQA removed a comment on pull request #33224: URL: https://github.com/apache/spark/pull/33224#issuecomment-874460457 **[Test build #140680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140680/testReport)** for PR 33224 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33226: [SPARK-35999][SQL] Make from_csv/to_csv to handle day-time intervals properly

AmplabJenkins commented on pull request #33226: URL: https://github.com/apache/spark/pull/33226#issuecomment-874586150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45196/ --

[GitHub] [spark] viirya commented on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

viirya commented on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-874322711 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-874634979 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] xuanyuanking commented on pull request #33187: [SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

xuanyuanking commented on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874491268 Yes, finally! Addressed the comments and rebased the code. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] SparkQA commented on pull request #33227: [SPARK-35972][SQL][3.1] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

SparkQA commented on pull request #33227: URL: https://github.com/apache/spark/pull/33227#issuecomment-874548375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33221: [SPARK-35977][SQL] Support non-reserved keyword TIMESTAMP_NTZ

AmplabJenkins removed a comment on pull request #33221: URL: https://github.com/apache/spark/pull/33221#issuecomment-874303207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

AngersZh commented on a change in pull request #33217: URL: https://github.com/apache/spark/pull/33217#discussion_r664201809 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/IntervalUtils.scala ## @@ -119,6 +119,39 @@ object IntervalUtils {

[GitHub] [spark] gengliangwang closed pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

gengliangwang closed pull request #33215: URL: https://github.com/apache/spark/pull/33215 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] MaxGekk commented on pull request #33132: [SPARK-35926][SQL] Add support YearMonthIntervalType for width_bucket

MaxGekk commented on pull request #33132: URL: https://github.com/apache/spark/pull/33132#issuecomment-874272032 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AngersZhuuuu commented on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

AngersZh commented on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-874390710 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HeartSaVioR edited a comment on pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

HeartSaVioR edited a comment on pull request #32934: URL: https://github.com/apache/spark/pull/32934#issuecomment-874409174 Thanks! Merging to master/3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above

[GitHub] [spark] HyukjinKwon commented on pull request #33214: [SPARK-35929][PYTHON] Support to infer nested dict as a struct when creating a DataFrame

HyukjinKwon commented on pull request #33214: URL: https://github.com/apache/spark/pull/33214#issuecomment-874563263 Can you also update the PR description? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] cloud-fan commented on pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

cloud-fan commented on pull request #33204: URL: https://github.com/apache/spark/pull/33204#issuecomment-874508946 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33183: [SPARK-35972][SQL] When replace ExtractValue in NestedColumnAliasing we should use semanticEquals

AmplabJenkins commented on pull request #33183: URL: https://github.com/apache/spark/pull/33183#issuecomment-874422409 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33216: [SPARK-36018][CORE][SQL] Some Improvement for Spark Core

AmplabJenkins commented on pull request #33216: URL: https://github.com/apache/spark/pull/33216#issuecomment-873982918 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cloud-fan commented on a change in pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

cloud-fan commented on a change in pull request #33215: URL: https://github.com/apache/spark/pull/33215#discussion_r663968258 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/parser/ExpressionParserSuite.scala ## @@ -463,6 +464,17 @@ class

[GitHub] [spark] SparkQA commented on pull request #33221: [SPARK-35977][SQL] Support non-reserved keyword TIMESTAMP_NTZ

SparkQA commented on pull request #33221: URL: https://github.com/apache/spark/pull/33221#issuecomment-874264660 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33225: [SPARK-35983][SQL] Allow from_json/to_json for map types where value types are day-time intervals

SparkQA commented on pull request #33225: URL: https://github.com/apache/spark/pull/33225#issuecomment-874483254 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cfmcgrady commented on pull request #33146: [SPARK-35912][SQL] Fix cast struct contains null value to string/struct

cfmcgrady commented on pull request #33146: URL: https://github.com/apache/spark/pull/33146#issuecomment-873926089 Create a new [PR](https://github.com/apache/spark/pull/33212) BTW, Shall we merge this PR? The cast issue may occur when the user create dataframe from API

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

AmplabJenkins removed a comment on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873888677 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140629/

[GitHub] [spark] SparkQA removed a comment on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

SparkQA removed a comment on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873889628 **[Test build #140639 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140639/testReport)** for PR 33211 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

SparkQA removed a comment on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873828692 **[Test build #140631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140631/testReport)** for PR 33206 at commit

[GitHub] [spark] cfmcgrady closed pull request #33146: [SPARK-35912][SQL] Fix cast struct contains null value to string/struct

cfmcgrady closed pull request #33146: URL: https://github.com/apache/spark/pull/33146 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] MaxGekk closed pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

MaxGekk closed pull request #33210: URL: https://github.com/apache/spark/pull/33210 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] MaxGekk commented on a change in pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

MaxGekk commented on a change in pull request #33210: URL: https://github.com/apache/spark/pull/33210#discussion_r663801703 ## File path: sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala ## @@ -279,4 +281,31 @@ class CsvFunctionsSuite extends QueryTest with

[GitHub] [spark] HyukjinKwon commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

HyukjinKwon commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-873969119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

AmplabJenkins removed a comment on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873932370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33219: [SPARK-36012][SQL] Add null flag when SHOW CREATE TABLE in v2

AmplabJenkins commented on pull request #33219: URL: https://github.com/apache/spark/pull/33219#issuecomment-874053729 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA removed a comment on pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

SparkQA removed a comment on pull request #33214: URL: https://github.com/apache/spark/pull/33214#issuecomment-873933793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

AmplabJenkins removed a comment on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873888673 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] cloud-fan commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

cloud-fan commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873899622 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33217: [SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

AmplabJenkins commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874065592 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

AmplabJenkins commented on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873902824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

AmplabJenkins commented on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873940119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] gengliangwang commented on pull request #33218: [SPARK-35664][SQL][FOLLOWUP] Fix incorrect comment for TimestampNTZType.

gengliangwang commented on pull request #33218: URL: https://github.com/apache/spark/pull/33218#issuecomment-874012741 Merging to master/branch-3.2 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] cloud-fan commented on a change in pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

cloud-fan commented on a change in pull request #33213: URL: https://github.com/apache/spark/pull/33213#discussion_r663754582 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ## @@ -372,7 +372,7 @@ case class StructType(fields:

[GitHub] [spark] SparkQA commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

SparkQA commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873902830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cfmcgrady commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

cfmcgrady commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-873921827 cc @HyukjinKwon @cloud-fan @maropu @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA commented on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

SparkQA commented on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873934801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] gengliangwang commented on a change in pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

gengliangwang commented on a change in pull request #33215: URL: https://github.com/apache/spark/pull/33215#discussion_r663837939 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -248,7 +248,7 @@ object DateTimeUtils {

[GitHub] [spark] AmplabJenkins commented on pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

AmplabJenkins commented on pull request #31763: URL: https://github.com/apache/spark/pull/31763#issuecomment-873990010 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140641/ -- This

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r663798968 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ## @@ -133,6 +144,26 @@ this.peakMemoryUsedBytes =

[GitHub] [spark] Ngone51 commented on a change in pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state

Ngone51 commented on a change in pull request #33078: URL: https://github.com/apache/spark/pull/33078#discussion_r663710964 ## File path: common/network-shuffle/src/test/java/org/apache/spark/network/shuffle/RemoteBlockPushResolverSuite.java ## @@ -219,101 +238,145 @@ public

[GitHub] [spark] cloud-fan commented on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

cloud-fan commented on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873945816 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] xuanyuanking commented on a change in pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

xuanyuanking commented on a change in pull request #32934: URL: https://github.com/apache/spark/pull/32934#discussion_r663972591 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/RocksDBFileManager.scala ## @@ -486,6 +516,23 @@ class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

AmplabJenkins removed a comment on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873888669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

<    1   2   3   4   5   6   7   8   9   10   >