[GitHub] [spark] SparkQA commented on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

SparkQA commented on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873852132 **[Test build #140628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140628/testReport)** for PR 33208 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

SparkQA removed a comment on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873783903 **[Test build #140628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140628/testReport)** for PR 33208 at commit

[GitHub] [spark] SparkQA commented on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

SparkQA commented on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873855124 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45144/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873860824 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45147/ --

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873860853 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45147/ --

[GitHub] [spark] SparkQA commented on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

SparkQA commented on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873879466 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45143/ -- This is an automated message from the

[GitHub] [spark] ulysses-you opened a new pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

ulysses-you opened a new pull request #33211: URL: https://github.com/apache/spark/pull/33211 ### What changes were proposed in this pull request? Use uuid instead of `System. currentTimeMillis` as app id in kubernetes client mode. ### Why are the changes needed?

[GitHub] [spark] ulysses-you commented on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

ulysses-you commented on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873885487 Do you have time to take a look ? thanks @dongjoon-hyun @holdenk @vanzin @attilapiros -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

AmplabJenkins removed a comment on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-873769162 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

AmplabJenkins removed a comment on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873769630 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

AmplabJenkins removed a comment on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873888673 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45143/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

AmplabJenkins removed a comment on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873888677 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140629/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873888671 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

AmplabJenkins removed a comment on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873888669 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45144/

[GitHub] [spark] AmplabJenkins commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

AmplabJenkins commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-873889384 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] sarutak closed pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

sarutak closed pull request #33208: URL: https://github.com/apache/spark/pull/33208 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873888679 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45149/ --

[GitHub] [spark] cloud-fan opened a new pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

cloud-fan opened a new pull request #33213: URL: https://github.com/apache/spark/pull/33213 ### What changes were proposed in this pull request? This is a followup of https://github.com/apache/spark/pull/33113, to do some code cleanup: 1. `UnresolvedFieldPosition` doesn't

[GitHub] [spark] cloud-fan commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

cloud-fan commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873899622 cc @imback82 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

SparkQA commented on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873905189 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45145/ -- This is an automated message from the

[GitHub] [spark] itholic opened a new pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

itholic opened a new pull request #33214: URL: https://github.com/apache/spark/pull/33214 ### What changes were proposed in this pull request? Currently, inferring nested structs is always using `MapType`. This behavior causes an issue because it infers the schema with a value

[GitHub] [spark] SparkQA commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better

SparkQA commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873904488 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45146/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873913124 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45153/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

AmplabJenkins removed a comment on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873932375 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45146/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873932372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

AmplabJenkins removed a comment on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873932381 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140630/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

AmplabJenkins removed a comment on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873932378 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45145/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873932368 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45153/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

AmplabJenkins removed a comment on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873932374 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140631/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

AmplabJenkins removed a comment on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873902824 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] cloud-fan commented on pull request #33146: [SPARK-35912][SQL] Fix cast struct contains null value to string/struct

cloud-fan commented on pull request #33146: URL: https://github.com/apache/spark/pull/33146#issuecomment-873932739 `spark.internalCreateDataFrame()` is a private API and users should be responsible if they do something wrong. We can't merge this PR, because we can't sacrifice the

[GitHub] [spark] AmplabJenkins commented on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

AmplabJenkins commented on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873932374 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140631/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

AmplabJenkins removed a comment on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873932370 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140642/

[GitHub] [spark] AmplabJenkins commented on pull request #33211: [SPARK-36014][K8S] Use uuid as app id in kubernetes client mode

AmplabJenkins commented on pull request #33211: URL: https://github.com/apache/spark/pull/33211#issuecomment-873932380 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45152/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

AmplabJenkins commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873932370 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140642/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a

AmplabJenkins commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873932375 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45146/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

AmplabJenkins commented on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873932381 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140630/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

AmplabJenkins commented on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873932378 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45145/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873932372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873932368 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45153/ --

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

HyukjinKwon commented on a change in pull request #33214: URL: https://github.com/apache/spark/pull/33214#discussion_r663761040 ## File path: python/pyspark/sql/tests/test_types.py ## @@ -196,6 +196,12 @@ def test_infer_nested_schema(self): df =

[GitHub] [spark] SparkQA commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better

SparkQA commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873936105 **[Test build #140633 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140633/testReport)** for PR 33078 at commit

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33214: [SPARK-35929][PYTHON] Schema inference of nested structs defaults to map

HyukjinKwon commented on a change in pull request #33214: URL: https://github.com/apache/spark/pull/33214#discussion_r663760264 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -4040,6 +4047,8 @@ class SQLConf extends Serializable

[GitHub] [spark] SparkQA removed a comment on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in

SparkQA removed a comment on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873853596 **[Test build #140633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140633/testReport)** for PR 33078 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

AmplabJenkins commented on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873940119 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140646/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

SparkQA removed a comment on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873934801 **[Test build #140646 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140646/testReport)** for PR 31847 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

AmplabJenkins removed a comment on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873940119 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140646/

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873941019 **[Test build #140647 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140647/testReport)** for PR 32365 at commit

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873954634 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45159/ --

[GitHub] [spark] SparkQA commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

SparkQA commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873970461 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45157/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

HyukjinKwon commented on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873970457 cc @sarutak @srowen too can you take a quick look please? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [spark] HyukjinKwon commented on pull request #33207: [SPARK-33996][BUILD][FOLLOW-UP] Match SBT's plugin checkstyle version to Maven's

HyukjinKwon commented on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873970818 this more specifically affects `./dev/sbt-checkstyle` which I checked that it passes locally -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] gengliangwang commented on pull request #32714: [SPARK-35581][SQL] Support special datetime values in typed literals only

gengliangwang commented on pull request #32714: URL: https://github.com/apache/spark/pull/32714#issuecomment-873970851 @MaxGekk Oh, I made a mistake in the test with PostgreSQL. Sorry for that. -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] Ngone51 commented on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873985519 > so what did the benchmarking numbers look like? Was there an average hit across or mostly just noise? @tgravescs Our internal benchmark runs between the baseline

[GitHub] [spark] SparkQA commented on pull request #31847: [SPARK-34755][SQL] Support the utils for transform number format

SparkQA commented on pull request #31847: URL: https://github.com/apache/spark/pull/31847#issuecomment-873996023 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45158/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33209: [SPARK-36013][BUILD] Upgrade Dropwizard Metrics to 4.2.2

SparkQA commented on pull request #33209: URL: https://github.com/apache/spark/pull/33209#issuecomment-873855820 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45143/ -- This is an automated message from the Apache

[GitHub] [spark] gengliangwang commented on pull request #32714: [SPARK-35581][SQL] Support special datetime values in typed literals only

gengliangwang commented on pull request #32714: URL: https://github.com/apache/spark/pull/32714#issuecomment-873868035 I suggest that we remove the support of zone id in the special strings to make things simple. -- This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-873867857 **[Test build #140626 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140626/testReport)** for PR 32959 at commit

[GitHub] [spark] MaxGekk closed pull request #33181: [SPARK-35982][SQL] Allow from_json/to_json for map types where value types are year-month intervals

MaxGekk closed pull request #33181: URL: https://github.com/apache/spark/pull/33181 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] gengliangwang opened a new pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

gengliangwang opened a new pull request #33215: URL: https://github.com/apache/spark/pull/33215 ### What changes were proposed in this pull request? For the timestamp literal, it should have the following behavior. 1. When `spark.sql.timestampType` is TIMESTAMP_NTZ: if

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the sta

AmplabJenkins removed a comment on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873937173 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140633/

[GitHub] [spark] AmplabJenkins commented on pull request #33078: [SPARK-35546][Shuffle] Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a

AmplabJenkins commented on pull request #33078: URL: https://github.com/apache/spark/pull/33078#issuecomment-873937173 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140633/ -- This

[GitHub] [spark] SparkQA commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

SparkQA commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873941152 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45154/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #32944: [SPARK-35794][SQL] Allow custom plugin for AQE cost evaluator

cloud-fan commented on a change in pull request #32944: URL: https://github.com/apache/spark/pull/32944#discussion_r663765278 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -678,6 +678,14 @@ object SQLConf { .booleanConf

[GitHub] [spark] cloud-fan closed pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

cloud-fan closed pull request #33188: URL: https://github.com/apache/spark/pull/33188 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] linhongliu-db commented on a change in pull request #33204: [SPARK-36011][SQL] Disallow altering permanent views based on temporary views or UDFs

linhongliu-db commented on a change in pull request #33204: URL: https://github.com/apache/spark/pull/33204#discussion_r663776384 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/SQLViewSuite.scala ## @@ -910,4 +910,20 @@ abstract class SQLViewSuite

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

HyukjinKwon commented on a change in pull request #33215: URL: https://github.com/apache/spark/pull/33215#discussion_r663791613 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -248,7 +248,7 @@ object DateTimeUtils { *

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873968232 **[Test build #140638 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140638/testReport)** for PR 31517 at commit

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r663798968 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ## @@ -133,6 +144,26 @@ this.peakMemoryUsedBytes =

[GitHub] [spark] Ngone51 commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

Ngone51 commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r663799506 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1368,6 +1368,14 @@ package object config { s"The buffer

[GitHub] [spark] SparkQA removed a comment on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

SparkQA removed a comment on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873979199 **[Test build #140652 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140652/testReport)** for PR 32401 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

AmplabJenkins removed a comment on pull request #32401: URL: https://github.com/apache/spark/pull/32401#issuecomment-873979662 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140652/

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873987636 **[Test build #140653 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140653/testReport)** for PR 32365 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873987693 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140653/ -- This

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873994111 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45163/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33206: [SPARK-36002][PYTHON] Consolidate tests for data-type-based operations of decimal Series

AmplabJenkins removed a comment on pull request #33206: URL: https://github.com/apache/spark/pull/33206#issuecomment-873798808 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45140/

[GitHub] [spark] sarutak commented on pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

sarutak commented on pull request #33210: URL: https://github.com/apache/spark/pull/33210#issuecomment-873832398 cc: @MaxGekk -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] sarutak opened a new pull request #33210: [SPARK-35998][SQL] Make from_csv/to_csv to handle year-month intervals properly

sarutak opened a new pull request #33210: URL: https://github.com/apache/spark/pull/33210 ### What changes were proposed in this pull request? This PR fixes an issue that `from_csv/to_csv` doesn't handle year-month intervals properly. `from_csv` throws exception if year-month

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873856779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873856827 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873857147 **[Test build #140635 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140635/testReport)** for PR 31517 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873857198 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140635/ -- This

[GitHub] [spark] SparkQA commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

SparkQA commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873976701 **[Test build #140648 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140648/testReport)** for PR 32365 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

AmplabJenkins removed a comment on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873976390 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45154/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873976392 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140638/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

AmplabJenkins removed a comment on pull request #33215: URL: https://github.com/apache/spark/pull/33215#issuecomment-873976394 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45155/

[GitHub] [spark] AmplabJenkins commented on pull request #33215: [SPARK-35979][SQL] Return different timestamp literals based on the default timestamp type

AmplabJenkins commented on pull request #33215: URL: https://github.com/apache/spark/pull/33215#issuecomment-873976394 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45155/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins removed a comment on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873943149 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #32365: [SPARK-35228][SQL] Add expression ToHiveString for keep consistent between hive/spark format in df.show

AmplabJenkins commented on pull request #32365: URL: https://github.com/apache/spark/pull/32365#issuecomment-873976389 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45159/ --

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-873976392 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140638/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

AmplabJenkins commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873976390 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45154/ --

[GitHub] [spark] SparkQA commented on pull request #33213: [SPARK-34302][SQL][FOLLOWUP] More code cleanup

SparkQA commented on pull request #33213: URL: https://github.com/apache/spark/pull/33213#issuecomment-873994853 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45157/ -- This is an automated message from the

[GitHub] [spark] AngersZhuuuu opened a new pull request #33217: [SPARK-35735][SPARK-35768][SQL] Refactor code about parse string to DT/YM

AngersZh opened a new pull request #33217: URL: https://github.com/apache/spark/pull/33217 ### What changes were proposed in this pull request? Refactor code about parse string to DT/YM ### Why are the changes needed? Extract common code about parse string to DT/YM

[GitHub] [spark] SparkQA commented on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

SparkQA commented on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873830258 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45141/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins commented on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

AmplabJenkins commented on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873830289 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45141/ --

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33208: [SPARK-36010][BUILD] Upgrade sbt-antlr4 from 0.8.2 to 0.8.3

AmplabJenkins removed a comment on pull request #33208: URL: https://github.com/apache/spark/pull/33208#issuecomment-873830289 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45141/

[GitHub] [spark] AmplabJenkins commented on pull request #33207: [SPARK-33996][BUILD] Match SBT's plugin checkstyle version to Maven's

AmplabJenkins commented on pull request #33207: URL: https://github.com/apache/spark/pull/33207#issuecomment-873852761 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45142/ --

[GitHub] [spark] HeartSaVioR commented on pull request #33061: [SPARK-35862][SS] Remove hardcoded time zone time format for watermark stats

HeartSaVioR commented on pull request #33061: URL: https://github.com/apache/spark/pull/33061#issuecomment-873854557 `"-MM-dd'T'HH:mm:ss.SSS'Z'"` Here the 'Z' refers to the "UTC", according to the ISO 8601. See https://en.wikipedia.org/wiki/ISO_8601 Using local timezone

[GitHub] [spark] SparkQA commented on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

SparkQA commented on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873864851 **[Test build #140624 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140624/testReport)** for PR 33188 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33188: [SPARK-35989][SQL] Only remove redundant shuffle if shuffle origin is REPARTITION_BY_COL in AQE

SparkQA removed a comment on pull request #33188: URL: https://github.com/apache/spark/pull/33188#issuecomment-873745387 **[Test build #140624 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140624/testReport)** for PR 33188 at commit

  1   2   3   4   5   6   >