[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r667672577 ## File path: core/src/test/scala/org/apache/spark/shuffle/ShuffleChecksumTester.scala ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r66767 ## File path: core/src/test/scala/org/apache/spark/shuffle/ShuffleChecksumTester.scala ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [spark] maropu commented on pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

maropu commented on pull request #32455: URL: https://github.com/apache/spark/pull/32455#issuecomment-878021318 @cloud-fan @HyukjinKwon As you know, the janino community is pretty inactive now. So, I'm wondering if it would be better to revert this commit from branch-3.2 before the QA peri

[GitHub] [spark] MaxGekk closed pull request #33278: [SPARK-36044][SQL] Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

MaxGekk closed pull request #33278: URL: https://github.com/apache/spark/pull/33278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r667669668 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1368,6 +1368,25 @@ package object config { s"The buffer

[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r667668816 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ## @@ -107,14 +110,19 @@ @Nullable private MemoryBlock cur

[GitHub] [spark] SparkQA removed a comment on pull request #33278: [SPARK-36044][SQL] Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

SparkQA removed a comment on pull request #33278: URL: https://github.com/apache/spark/pull/33278#issuecomment-877921600 **[Test build #140890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140890/testReport)** for PR 33278 at commit [`1761df9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33278: [SPARK-36044][SQL] Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

SparkQA commented on pull request #33278: URL: https://github.com/apache/spark/pull/33278#issuecomment-878017336 **[Test build #140890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140890/testReport)** for PR 33278 at commit [`1761df9`](https://github.co

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667665862 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala ## @@ -2924,6 +2924,61 @@ abstract class JsonSuite

[GitHub] [spark] HyukjinKwon commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

HyukjinKwon commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878016626 @dgd-contributor can you identify the Pr that fixed this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667665610 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonParserSuite.scala ## @@ -54,4 +55,64 @@ class JacksonParserSuite exten

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667665051 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonParserSuite.scala ## @@ -54,4 +55,64 @@ class JacksonParserSuite exten

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667664747 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala ## @@ -418,6 +426,19 @@ class JacksonParser( }

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878011972 **[Test build #140899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140899/testReport)** for PR 33295 at commit [`e5a8b4f`](https://github.co

[GitHub] [spark] dgd-contributor edited a comment on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

dgd-contributor edited a comment on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878014625 > @dgd-contributor can you create a PR against the master branch? this PR is only for branch 3.0 and 3.1. On master this issue not occur since it already wr

[GitHub] [spark] dgd-contributor commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

dgd-contributor commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878014625 > @dgd-contributor can you create a PR against the master branch? this PR is only for branch 3.0 and 3.1. On master this issue not occur since it already wrapped in a

[GitHub] [spark] SparkQA removed a comment on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

SparkQA removed a comment on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878008721 **[Test build #140902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140902/testReport)** for PR 33295 at commit [`7368b08`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878014198 **[Test build #140902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140902/testReport)** for PR 33295 at commit [`7368b08`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

AmplabJenkins removed a comment on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878012135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140899/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

AmplabJenkins commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878012135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140899/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

SparkQA removed a comment on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878006810 **[Test build #140899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140899/testReport)** for PR 33295 at commit [`e5a8b4f`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

AmplabJenkins removed a comment on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-863413159 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878008721 **[Test build #140902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140902/testReport)** for PR 33295 at commit [`7368b08`](https://github.com

[GitHub] [spark] ulysses-you commented on a change in pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

ulysses-you commented on a change in pull request #33295: URL: https://github.com/apache/spark/pull/33295#discussion_r667656971 ## File path: docs/sql-performance-tuning.md ## @@ -234,7 +234,7 @@ The "REPARTITION_BY_RANGE" hint must have column names and a partition number is

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r667656166 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -662,6 +662,16 @@ case class ShowCurrentNam

[GitHub] [spark] SparkQA commented on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

SparkQA commented on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-878007078 **[Test build #140901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140901/testReport)** for PR 32951 at commit [`ab71669`](https://github.com

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

HyukjinKwon commented on a change in pull request #33295: URL: https://github.com/apache/spark/pull/33295#discussion_r667655419 ## File path: docs/sql-performance-tuning.md ## @@ -234,7 +234,7 @@ The "REPARTITION_BY_RANGE" hint must have column names and a partition number is

[GitHub] [spark] SparkQA commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

SparkQA commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-878007045 **[Test build #140900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140900/testReport)** for PR 33038 at commit [`8b2f945`](https://github.com

[GitHub] [spark] ulysses-you commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

ulysses-you commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-878006960 thank you @cloud-fan , created https://github.com/apache/spark/pull/33295 -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878006810 **[Test build #140899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140899/testReport)** for PR 33295 at commit [`e5a8b4f`](https://github.com

[GitHub] [spark] ulysses-you opened a new pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

ulysses-you opened a new pull request #33295: URL: https://github.com/apache/spark/pull/33295 ### What changes were proposed in this pull request? Update AQE is `disabled` to `enabled` in sql-performance-tuning docs ### Why are the changes needed? Make docs corre

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

AmplabJenkins removed a comment on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-878005490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140891/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-878005489 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140893/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

AmplabJenkins removed a comment on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-878005487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140894/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

AmplabJenkins removed a comment on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-878005488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45408/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

AmplabJenkins removed a comment on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-878005486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

AmplabJenkins removed a comment on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-878005495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45410/

[GitHub] [spark] AmplabJenkins commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

AmplabJenkins commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-878005490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140891/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

AmplabJenkins commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-878005495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45410/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

AmplabJenkins commented on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-878005487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140894/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

AmplabJenkins commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-878005488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45408/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-878005489 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140893/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

AmplabJenkins commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-878005486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] gengliangwang commented on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

gengliangwang commented on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-878004446 Jenkins, retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] maropu commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

maropu commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-878000933 Could you update the title and description, too? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon closed pull request #33285: [SPARK-36003][PYTHON] Implement unary operator `invert` of integral ps.Series/Index.

HyukjinKwon closed pull request #33285: URL: https://github.com/apache/spark/pull/33285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] HyukjinKwon commented on pull request #33285: [SPARK-36003][PYTHON] Implement unary operator `invert` of integral ps.Series/Index.

HyukjinKwon commented on pull request #33285: URL: https://github.com/apache/spark/pull/33285#issuecomment-87859 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HeartSaVioR commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

HeartSaVioR commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-87811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [spark] SparkQA commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-877999358 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45410/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

SparkQA removed a comment on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-877942722 **[Test build #140894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140894/testReport)** for PR 33267 at commit [`fc14fb8`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

SparkQA commented on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-877993809 **[Test build #140894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140894/testReport)** for PR 33267 at commit [`fc14fb8`](https://github.co

[GitHub] [spark] geekyouth commented on pull request #33292: [SPARK-36069] spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

geekyouth commented on pull request #33292: URL: https://github.com/apache/spark/pull/33292#issuecomment-877993199 > @geekyouth can you create a PR against the master branch? OK, I will do it now. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] SparkQA commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

SparkQA commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-877993163 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45408/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877992146 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45409/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877940872 **[Test build #140893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140893/testReport)** for PR 31517 at commit [`ee05314`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877991274 **[Test build #140893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140893/testReport)** for PR 31517 at commit [`ee05314`](https://github.co

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33291: [SPARK-35561][SQL] Remove leading zeros from empty static number type partition

HyukjinKwon commented on a change in pull request #33291: URL: https://github.com/apache/spark/pull/33291#discussion_r667640615 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala ## @@ -351,10 +351,24 @@ object Partitioning

[GitHub] [spark] SparkQA removed a comment on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

SparkQA removed a comment on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-877921683 **[Test build #140891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140891/testReport)** for PR 33038 at commit [`8b2f945`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

SparkQA commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-877989099 **[Test build #140891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140891/testReport)** for PR 33038 at commit [`8b2f945`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877987718 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45407/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #32872: [SPARK-35639][SQL] Make hasCoalescedPartition return true if something was actually coalesced

cloud-fan commented on a change in pull request #32872: URL: https://github.com/apache/spark/pull/32872#discussion_r667637282 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/CoalesceShufflePartitionsSuite.scala ## @@ -331,7 +331,8 @@ class CoalesceShuffleP

[GitHub] [spark] HyukjinKwon commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

HyukjinKwon commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-877986460 @dgd-contributor can you create a PR against the master branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] cloud-fan commented on a change in pull request #32872: [SPARK-35639][SQL] Make hasCoalescedPartition return true if something was actually coalesced

cloud-fan commented on a change in pull request #32872: URL: https://github.com/apache/spark/pull/32872#discussion_r667635965 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -87,8 +87,15 @@ case class CustomShuf

[GitHub] [spark] cloud-fan commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

cloud-fan commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-877985052 missed one thing: `AQE is disabled by default.` Can we correct this in the doc as well? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cfmcgrady commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

cfmcgrady commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667635424 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala ## @@ -399,16 +399,24 @@ class JacksonParser( val

[GitHub] [spark] cloud-fan closed pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

cloud-fan closed pull request #32960: URL: https://github.com/apache/spark/pull/32960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] cloud-fan commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

cloud-fan commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-877984476 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] HyukjinKwon commented on pull request #33292: [SPARK-36069] spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

HyukjinKwon commented on pull request #33292: URL: https://github.com/apache/spark/pull/33292#issuecomment-877983543 @geekyouth can you create a PR against the master branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

HyukjinKwon commented on a change in pull request #33142: URL: https://github.com/apache/spark/pull/33142#discussion_r667633052 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -170,65 +167,71 @@ class Equiva

[GitHub] [spark] SparkQA commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-877981379 **[Test build #140898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140898/testReport)** for PR 32959 at commit [`e146045`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877980413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45404/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

AmplabJenkins removed a comment on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877980411 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

AmplabJenkins removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877907828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

AmplabJenkins commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877980410 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140888/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877980413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45404/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

AmplabJenkins commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877980412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] cloud-fan commented on a change in pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

cloud-fan commented on a change in pull request #33142: URL: https://github.com/apache/spark/pull/33142#discussion_r667631076 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -170,65 +167,71 @@ class Equivale

[GitHub] [spark] SparkQA commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

SparkQA commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-877979863 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45408/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877979695 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45409/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

HyukjinKwon commented on a change in pull request #33142: URL: https://github.com/apache/spark/pull/33142#discussion_r667627831 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -170,65 +167,71 @@ class Equiva

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877974240 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45407/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877969669 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45405/ -- This is an automated message from the A

[GitHub] [spark] gengliangwang commented on a change in pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

gengliangwang commented on a change in pull request #33258: URL: https://github.com/apache/spark/pull/33258#discussion_r667622706 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -200,6 +200,30 @@ case class No

[GitHub] [spark] gengliangwang commented on a change in pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

gengliangwang commented on a change in pull request #33258: URL: https://github.com/apache/spark/pull/33258#discussion_r667622302 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -968,6 +968,11 @@ object DateTimeUtils {

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877966933 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45403/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

SparkQA removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877887767 **[Test build #140888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140888/testReport)** for PR 32049 at commit [`ccdc543`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877965693 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45404/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

SparkQA commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877965636 **[Test build #140888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140888/testReport)** for PR 32049 at commit [`ccdc543`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962895 **[Test build #140897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140897/testReport)** for PR 33294 at commit [`2360d07`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

AmplabJenkins removed a comment on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962103 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140895/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA removed a comment on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877959913 **[Test build #140895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140895/testReport)** for PR 33294 at commit [`2360d07`](https://gi

[GitHub] [spark] ulysses-you commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

ulysses-you commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962597 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33263: [SPARK-35027][CORE] Close the inputStream in FileAppender when writin…

HyukjinKwon commented on a change in pull request #33263: URL: https://github.com/apache/spark/pull/33263#discussion_r667616566 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala ## @@ -185,11 +185,11 @@ private[deploy] class ExecutorRunner(

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962082 **[Test build #140895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140895/testReport)** for PR 33294 at commit [`2360d07`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

AmplabJenkins commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962103 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140895/ -- This

[GitHub] [spark] SparkQA commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

SparkQA commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-877961618 **[Test build #140896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140896/testReport)** for PR 33280 at commit [`4c11b7f`](https://github.com

[GitHub] [spark] gengliangwang commented on a change in pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

gengliangwang commented on a change in pull request #33280: URL: https://github.com/apache/spark/pull/33280#discussion_r667614979 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1008,17 +1008,17 @@ case class

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877959913 **[Test build #140895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140895/testReport)** for PR 33294 at commit [`2360d07`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

AmplabJenkins commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-877958950 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

AmplabJenkins removed a comment on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-877958525 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45402/

  1   2   3   >