[GitHub] [spark] SparkQA commented on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

SparkQA commented on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873917796 **[Test build #140630 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140630/testReport)** for PR 33209 at commit

[GitHub] [spark] gengliangwang commented on pull request #32714: [SPARK-35581][SQL] Support special datetime values in typed literals only

gengliangwang commented on pull request #32714: URL: https://github.com/apache/spark/pull/32714#issuecomment-873970851 @MaxGekk Oh, I made a mistake in the test with PostgreSQL. Sorry for that. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA removed a comment on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

SparkQA removed a comment on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873823116 **[Test build #140630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140630/testReport)** for PR 33209 at commit

[GitHub] [spark] SparkQA commented on pull request #33216: [SPARK-36018][CORE][SQL] Some Improvement for Spark Core

SparkQA commented on pull request #33216: URL: https://github.com/apache/spark/pull/33216#issuecomment-873978584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873893252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cloud-fan commented on pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

cloud-fan commented on pull request #32944: URL: https://github.com/apache/spark/pull/32944#issuecomment-873942385 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA commented on pull request #32934: [SPARK-35788][SS] Metrics support for RocksDB instance

SparkQA commented on pull request #32934: URL: https://github.com/apache/spark/pull/32934#issuecomment-874174521 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] beliefer commented on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

beliefer commented on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873945468 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33216: [SPARK-36018][CORE][SQL] Some Improvement for Spark Core

AmplabJenkins removed a comment on pull request #33216: URL: https://github.com/apache/spark/pull/33216#issuecomment-873982918 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

AmplabJenkins commented on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873979662 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] sarutak commented on a change in pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

sarutak commented on a change in pull request #33210: URL: https://github.com/apache/spark/pull/33210#discussion_r663803202 ## File path: sql/core/src/test/scala/org/apache/spark/sql/CsvFunctionsSuite.scala ## @@ -279,4 +281,31 @@ class CsvFunctionsSuite extends QueryTest with

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31948: [SPARK-34848][CORE] Add duration to TaskMetricDistributions

AngersZh commented on a change in pull request #31948: URL: https://github.com/apache/spark/pull/31948#discussion_r663948849 ## File path: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ## @@ -342,6 +342,7 @@ class ShuffleWriteMetrics private[spark]( class

[GitHub] [spark] SparkQA commented on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

SparkQA commented on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873881588 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

AmplabJenkins removed a comment on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-873769162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

AmplabJenkins commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873932370 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873881504 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HyukjinKwon commented on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

HyukjinKwon commented on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873929744 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] LuciferYang edited a comment on pull request #31036: [SPARK-34009][BUILD] To activate profile 'aarch64' based on OS settings

LuciferYang edited a comment on pull request #31036: URL: https://github.com/apache/spark/pull/31036#issuecomment-873906415 @dongjoon-hyun can we run spark UTs on M1 now ? I found that when I run `mvn clean test -Paarch64 -pl core - am` on M1 mac , the `kvstore` module will test failed as

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

HyukjinKwon commented on a change in pull request #33204: URL: https://github.com/apache/spark/pull/33204#discussion_r663792850 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -910,4 +910,20 @@ abstract class SQLViewSuite extends

[GitHub] [spark] Ngone51 commented on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873985519 > so what did the benchmarking numbers look like? Was there an average hit across or mostly just noise? @tgravescs Our internal benchmark runs between the baseline

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33220: [WIP][SPARK-35993][TESTS] Fix flaky tests for RocksDBSuite

AmplabJenkins removed a comment on pull request #33220: URL: https://github.com/apache/spark/pull/33220#issuecomment-874662508 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] sarutak closed pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

sarutak closed pull request #33208: URL: https://github.com/apache/spark/pull/33208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon commented on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

HyukjinKwon commented on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873906834 Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

HyukjinKwon commented on a change in pull request #33214: URL: https://github.com/apache/spark/pull/33214#discussion_r663757786 ## File path: python/pyspark/sql/types.py ## @@ -1020,10 +1020,19 @@ def _infer_type(obj): return dataType() if isinstance(obj,

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

AmplabJenkins removed a comment on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-874662507 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873932372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

SparkQA commented on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873979199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA removed a comment on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in

SparkQA removed a comment on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873853596 **[Test build #140633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140633/testReport)** for PR 33078 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

AmplabJenkins commented on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873932378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873888679 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] gengliangwang closed pull request #33218: [SPARK-35664][SQL][FOLLOWUP] Fix incorrect comment for TimestampNTZType.

gengliangwang closed pull request #33218: URL: https://github.com/apache/spark/pull/33218 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA removed a comment on pull request #33216: [SPARK-36018][CORE][SQL] Some Improvement for Spark Core

SparkQA removed a comment on pull request #33216: URL: https://github.com/apache/spark/pull/33216#issuecomment-873978584 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] MaxGekk commented on pull request #32714: [SPARK-35581][SQL] Support special datetime values in typed literals only

MaxGekk commented on pull request #32714: URL: https://github.com/apache/spark/pull/32714#issuecomment-873967342 > QQ: why do the special values support zone id? I tried PostgreSQL and this is not supported. @gengliangwang We follow PostgreSQL behavior strictly. Time zones are

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

AmplabJenkins removed a comment on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873769630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] srowen commented on a change in pull request #31948: [SPARK-34848][CORE] Add duration to TaskMetricDistributions

srowen commented on a change in pull request #31948: URL: https://github.com/apache/spark/pull/31948#discussion_r663914637 ## File path: core/src/main/scala/org/apache/spark/status/api/v1/api.scala ## @@ -342,6 +342,7 @@ class ShuffleWriteMetrics private[spark]( class

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

AmplabJenkins removed a comment on pull request #33215: URL: https://github.com/apache/spark/pull/33215#issuecomment-873976394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873861809 **[Test build #140638 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140638/testReport)** for PR 31517 at commit

[GitHub] [spark] beliefer commented on pull request #33216: [SPARK-36018][CORE][SQL] Some Improvement for Spark Core

beliefer commented on pull request #33216: URL: https://github.com/apache/spark/pull/33216#issuecomment-873983321 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

SparkQA commented on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873889628 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA removed a comment on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

SparkQA removed a comment on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873902830 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HyukjinKwon closed pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

HyukjinKwon closed pull request #33209: URL: https://github.com/apache/spark/pull/33209 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] SparkQA commented on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

SparkQA commented on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873927249 **[Test build #140631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140631/testReport)** for PR 33206 at commit

[GitHub] [spark] SparkQA commented on pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

SparkQA commented on pull request #33215: URL: https://github.com/apache/spark/pull/33215#issuecomment-873933792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] beliefer commented on pull request #33218: [SPARK-35664][SQL][FOLLOWUP] Fix incorrect comment for TimestampNTZType.

beliefer commented on pull request #33218: URL: https://github.com/apache/spark/pull/33218#issuecomment-874005974 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33187: [SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

AmplabJenkins removed a comment on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874662509 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

AmplabJenkins removed a comment on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873932378 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] ulysses-you commented on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

ulysses-you commented on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873885487 Do you have time to take a look ? thanks @dongjoon-hyun @holdenk @vanzin @attilapiros -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins commented on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

AmplabJenkins commented on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873932381 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140630/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

AmplabJenkins commented on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873888669 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA removed a comment on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

SparkQA removed a comment on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873979199 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

AmplabJenkins commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-873889384 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] SparkQA removed a comment on pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

SparkQA removed a comment on pull request #33215: URL: https://github.com/apache/spark/pull/33215#issuecomment-873933792 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] sarutak commented on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

sarutak commented on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873977792 I also double-chedked on my laptop. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] AmplabJenkins commented on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

AmplabJenkins commented on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873888678 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140624/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

AmplabJenkins removed a comment on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873902824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

AmplabJenkins commented on pull request #33214: URL: https://github.com/apache/spark/pull/33214#issuecomment-874016367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better

SparkQA commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873881506 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

AmplabJenkins removed a comment on pull request #33214: URL: https://github.com/apache/spark/pull/33214#issuecomment-874016367 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

SparkQA removed a comment on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873853417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

HyukjinKwon commented on a change in pull request #33215: URL: https://github.com/apache/spark/pull/33215#discussion_r663791613 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -248,7 +248,7 @@ object DateTimeUtils { *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

AmplabJenkins removed a comment on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873940119 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

SparkQA commented on pull request #33214: URL: https://github.com/apache/spark/pull/33214#issuecomment-873933793 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] LuciferYang commented on pull request #31036: [SPARK-34009][BUILD] To activate profile 'aarch64' based on OS settings

LuciferYang commented on pull request #31036: URL: https://github.com/apache/spark/pull/31036#issuecomment-873906415 @dongjoon-hyun can we run spark UTs on M1 now ? I found that when I run 'mvn clean test -Paarch64 - pl core - am' on M1 mac , the `kvstore` module will test failed as

[GitHub] [spark] SparkQA removed a comment on pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

SparkQA removed a comment on pull request #31763: URL: https://github.com/apache/spark/pull/31763#issuecomment-873893785 **[Test build #140641 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140641/testReport)** for PR 31763 at commit

[GitHub] [spark] ulysses-you commented on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

ulysses-you commented on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-874070559 thank you @cloud-fan @HyukjinKwon -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873888671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31763: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

AmplabJenkins removed a comment on pull request #31763: URL: https://github.com/apache/spark/pull/31763#issuecomment-873990010 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140641/

[GitHub] [spark] AmplabJenkins commented on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

AmplabJenkins commented on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873888677 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140629/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

AmplabJenkins commented on pull request #33215: URL: https://github.com/apache/spark/pull/33215#issuecomment-873976394 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AngersZhuuuu commented on pull request #33217: [SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

AngersZh commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874129213 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] eejbyfeldt commented on a change in pull request #33205: [SPARK-20384][SQL] Support value class in nested schema for Dataset

eejbyfeldt commented on a change in pull request #33205: URL: https://github.com/apache/spark/pull/33205#discussion_r663891046 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/ScalaReflection.scala ## @@ -155,6 +155,65 @@ object ScalaReflection extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

AmplabJenkins removed a comment on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873932375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] cloud-fan commented on a change in pull request #33113: [SPARK-34302][SQL] Migrate ALTER TABLE ... CHANGE COLUMN command to use UnresolvedTable to resolve the identifier

cloud-fan commented on a change in pull request #33113: URL: https://github.com/apache/spark/pull/33113#discussion_r663721254 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala ## @@ -1088,8 +1058,51 @@ trait CheckAnalysis

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873932372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] HyukjinKwon commented on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

HyukjinKwon commented on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873970457 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33217: [SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

SparkQA commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874018329 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873893252 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA removed a comment on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

SparkQA removed a comment on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873934801 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] linhongliu-db commented on a change in pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

linhongliu-db commented on a change in pull request #33204: URL: https://github.com/apache/spark/pull/33204#discussion_r663776384 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -910,4 +910,20 @@ abstract class SQLViewSuite

[GitHub] [spark] cloud-fan commented on pull request #33146: [SPARK-35912][SQL] Fix cast struct contains null value to string/struct

cloud-fan commented on pull request #33146: URL: https://github.com/apache/spark/pull/33146#issuecomment-873932739 `spark.internalCreateDataFrame()` is a private API and users should be responsible if they do something wrong. We can't merge this PR, because we can't sacrifice the

[GitHub] [spark] HyukjinKwon closed pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

HyukjinKwon closed pull request #33207: URL: https://github.com/apache/spark/pull/33207 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan closed pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

cloud-fan closed pull request #33188: URL: https://github.com/apache/spark/pull/33188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan closed pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

cloud-fan closed pull request #32944: URL: https://github.com/apache/spark/pull/32944 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] sarutak commented on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

sarutak commented on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873888106 I believe the failure on Jenkins is not related to this change and passed on GA so merging to `master`. -- This is an automated message from the Apache Git Service. To

[GitHub] [spark] AmplabJenkins commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a

AmplabJenkins commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873932375 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] cloud-fan commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

cloud-fan commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-873934648 > should return a default value instead of null for the primitive type I don't agree with this. It's a correctness bug if an integer value is null in the JSON file but

[GitHub] [spark] cloud-fan commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

cloud-fan commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r663765278 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -678,6 +678,14 @@ object SQLConf { .booleanConf

[GitHub] [spark] AmplabJenkins commented on pull request #33187: [SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

AmplabJenkins commented on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874662510 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45208/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

AmplabJenkins commented on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-874662507 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45211/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

AmplabJenkins commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874662505 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #33182: [SPARK-35984][SQL] Config to force applying shuffled hash join

SparkQA commented on pull request #33182: URL: https://github.com/apache/spark/pull/33182#issuecomment-874661429 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45212/ -- This is an automated message from the Apache

[GitHub] [spark] jerqi commented on pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

jerqi commented on pull request #33204: URL: https://github.com/apache/spark/pull/33204#issuecomment-874660371 > @jerqi can you help to create backport PRs for 3.1 and 3.0? thanks! Ok, it's my pleasure. -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] MaxGekk closed pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

MaxGekk closed pull request #33217: URL: https://github.com/apache/spark/pull/33217 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] MaxGekk commented on pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

MaxGekk commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874658612 +1, LGTM. Merging to master/3.2. https://user-images.githubusercontent.com/1580697/124588101-19254900-de61-11eb-8b8e-cb65b8b8f685.png;> Thank you, @AngersZh . --

[GitHub] [spark] SparkQA commented on pull request #33229: [SPARK-36025][SQL][TESTS] Reduce the run time of DateExpressionsSuite

SparkQA commented on pull request #33229: URL: https://github.com/apache/spark/pull/33229#issuecomment-874658075 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45210/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

SparkQA commented on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-874656824 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45211/ --

[GitHub] [spark] SparkQA removed a comment on pull request #33187: [SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

SparkQA removed a comment on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874586692 **[Test build #140697 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140697/testReport)** for PR 33187 at commit

[GitHub] [spark] SparkQA commented on pull request #33187: [SPARK-35988][SS] The implementation for RocksDBStateStoreProvider

SparkQA commented on pull request #33187: URL: https://github.com/apache/spark/pull/33187#issuecomment-874651648 **[Test build #140697 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140697/testReport)** for PR 33187 at commit

[GitHub] [spark] SparkQA commented on pull request #33217: [SPARK-36023][SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

SparkQA commented on pull request #33217: URL: https://github.com/apache/spark/pull/33217#issuecomment-874651491 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45209/ -- This is an automated message from the

[GitHub] [spark] HyukjinKwon commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

HyukjinKwon commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-874644274 cc @MaxGekk FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

<    2   3   4   5   6   7   8   9   10   >