[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r667672577 ## File path: core/src/test/scala/org/apache/spark/shuffle/ShuffleChecksumTester.scala ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r66767 ## File path: core/src/test/scala/org/apache/spark/shuffle/ShuffleChecksumTester.scala ## @@ -0,0 +1,76 @@ +/* + * Licensed to the Apache Software Foun

[GitHub] [spark] maropu commented on pull request #32455: [SPARK-35253][SQL][BUILD] Bump up the janino version to v3.1.4

2021-07-11 Thread GitBox
maropu commented on pull request #32455: URL: https://github.com/apache/spark/pull/32455#issuecomment-878021318 @cloud-fan @HyukjinKwon As you know, the janino community is pretty inactive now. So, I'm wondering if it would be better to revert this commit from branch-3.2 before the QA peri

[GitHub] [spark] MaxGekk closed pull request #33278: [SPARK-36044][SQL] Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

2021-07-11 Thread GitBox
MaxGekk closed pull request #33278: URL: https://github.com/apache/spark/pull/33278 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubs

[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r667669668 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1368,6 +1368,25 @@ package object config { s"The buffer

[GitHub] [spark] cloud-fan commented on a change in pull request #32401: [SPARK-35276][CORE] Calculate checksum for shuffle data and write as checksum file

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #32401: URL: https://github.com/apache/spark/pull/32401#discussion_r667668816 ## File path: core/src/main/java/org/apache/spark/shuffle/sort/ShuffleExternalSorter.java ## @@ -107,14 +110,19 @@ @Nullable private MemoryBlock cur

[GitHub] [spark] SparkQA removed a comment on pull request #33278: [SPARK-36044][SQL] Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #33278: URL: https://github.com/apache/spark/pull/33278#issuecomment-877921600 **[Test build #140890 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140890/testReport)** for PR 33278 at commit [`1761df9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33278: [SPARK-36044][SQL] Suport TimestampNTZ in functions unix_timestamp/to_unix_timestamp

2021-07-11 Thread GitBox
SparkQA commented on pull request #33278: URL: https://github.com/apache/spark/pull/33278#issuecomment-878017336 **[Test build #140890 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140890/testReport)** for PR 33278 at commit [`1761df9`](https://github.co

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

2021-07-11 Thread GitBox
maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667665862 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/datasources/json/JsonSuite.scala ## @@ -2924,6 +2924,61 @@ abstract class JsonSuite

[GitHub] [spark] HyukjinKwon commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

2021-07-11 Thread GitBox
HyukjinKwon commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878016626 @dgd-contributor can you identify the Pr that fixed this issue? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

2021-07-11 Thread GitBox
maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667665610 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonParserSuite.scala ## @@ -54,4 +55,64 @@ class JacksonParserSuite exten

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

2021-07-11 Thread GitBox
maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667665051 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/json/JacksonParserSuite.scala ## @@ -54,4 +55,64 @@ class JacksonParserSuite exten

[GitHub] [spark] maropu commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

2021-07-11 Thread GitBox
maropu commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667664747 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala ## @@ -418,6 +426,19 @@ class JacksonParser( }

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878011972 **[Test build #140899 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140899/testReport)** for PR 33295 at commit [`e5a8b4f`](https://github.co

[GitHub] [spark] dgd-contributor edited a comment on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

2021-07-11 Thread GitBox
dgd-contributor edited a comment on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878014625 > @dgd-contributor can you create a PR against the master branch? this PR is only for branch 3.0 and 3.1. On master this issue not occur since it already wr

[GitHub] [spark] dgd-contributor commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

2021-07-11 Thread GitBox
dgd-contributor commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-878014625 > @dgd-contributor can you create a PR against the master branch? this PR is only for branch 3.0 and 3.1. On master this issue not occur since it already wrapped in a

[GitHub] [spark] SparkQA removed a comment on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878008721 **[Test build #140902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140902/testReport)** for PR 33295 at commit [`7368b08`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878014198 **[Test build #140902 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140902/testReport)** for PR 33295 at commit [`7368b08`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878012135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140899/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878012135 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140899/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878006810 **[Test build #140899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140899/testReport)** for PR 33295 at commit [`e5a8b4f`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-863413159 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878008721 **[Test build #140902 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140902/testReport)** for PR 33295 at commit [`7368b08`](https://github.com

[GitHub] [spark] ulysses-you commented on a change in pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
ulysses-you commented on a change in pull request #33295: URL: https://github.com/apache/spark/pull/33295#discussion_r667656971 ## File path: docs/sql-performance-tuning.md ## @@ -234,7 +234,7 @@ The "REPARTITION_BY_RANGE" hint must have column names and a partition number is

[GitHub] [spark] Peng-Lei commented on a change in pull request #33175: [SPARK-35973][SQL] DataSourceV2: Support SHOW CATALOGS

2021-07-11 Thread GitBox
Peng-Lei commented on a change in pull request #33175: URL: https://github.com/apache/spark/pull/33175#discussion_r667656166 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/v2Commands.scala ## @@ -662,6 +662,16 @@ case class ShowCurrentNam

[GitHub] [spark] SparkQA commented on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

2021-07-11 Thread GitBox
SparkQA commented on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-878007078 **[Test build #140901 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140901/testReport)** for PR 32951 at commit [`ab71669`](https://github.com

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33295: URL: https://github.com/apache/spark/pull/33295#discussion_r667655419 ## File path: docs/sql-performance-tuning.md ## @@ -234,7 +234,7 @@ The "REPARTITION_BY_RANGE" hint must have column names and a partition number is

[GitHub] [spark] SparkQA commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
SparkQA commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-878007045 **[Test build #140900 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140900/testReport)** for PR 33038 at commit [`8b2f945`](https://github.com

[GitHub] [spark] ulysses-you commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

2021-07-11 Thread GitBox
ulysses-you commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-878006960 thank you @cloud-fan , created https://github.com/apache/spark/pull/33295 -- This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
SparkQA commented on pull request #33295: URL: https://github.com/apache/spark/pull/33295#issuecomment-878006810 **[Test build #140899 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140899/testReport)** for PR 33295 at commit [`e5a8b4f`](https://github.com

[GitHub] [spark] ulysses-you opened a new pull request #33295: [SPARK-33679][SQL][FOLLOWUP] Enable spark.sql.adaptive.enabled by default

2021-07-11 Thread GitBox
ulysses-you opened a new pull request #33295: URL: https://github.com/apache/spark/pull/33295 ### What changes were proposed in this pull request? Update AQE is `disabled` to `enabled` in sql-performance-tuning docs ### Why are the changes needed? Make docs corre

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-878005490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140891/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-878005489 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140893/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-878005487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140894/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-878005488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45408/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-878005486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-878005495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45410/

[GitHub] [spark] AmplabJenkins commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-878005490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140891/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-878005495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45410/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-878005487 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140894/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-878005488 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45408/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-878005489 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140893/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-878005486 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] gengliangwang commented on pull request #32951: [SPARK-33603][SQL] Grouping execution/command

2021-07-11 Thread GitBox
gengliangwang commented on pull request #32951: URL: https://github.com/apache/spark/pull/32951#issuecomment-878004446 Jenkins, retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] maropu commented on pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

2021-07-11 Thread GitBox
maropu commented on pull request #33212: URL: https://github.com/apache/spark/pull/33212#issuecomment-878000933 Could you update the title and description, too? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HyukjinKwon closed pull request #33285: [SPARK-36003][PYTHON] Implement unary operator `invert` of integral ps.Series/Index.

2021-07-11 Thread GitBox
HyukjinKwon closed pull request #33285: URL: https://github.com/apache/spark/pull/33285 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] HyukjinKwon commented on pull request #33285: [SPARK-36003][PYTHON] Implement unary operator `invert` of integral ps.Series/Index.

2021-07-11 Thread GitBox
HyukjinKwon commented on pull request #33285: URL: https://github.com/apache/spark/pull/33285#issuecomment-87859 Merged to master and branch-3.2. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HeartSaVioR commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
HeartSaVioR commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-87811 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsu

[GitHub] [spark] SparkQA commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-11 Thread GitBox
SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-877999358 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45410/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-877942722 **[Test build #140894 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140894/testReport)** for PR 33267 at commit [`fc14fb8`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33267: [SPARK-36053][CORE] Extract a helper method to eliminate duplicate code related to delete abnormal disk block object file

2021-07-11 Thread GitBox
SparkQA commented on pull request #33267: URL: https://github.com/apache/spark/pull/33267#issuecomment-877993809 **[Test build #140894 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140894/testReport)** for PR 33267 at commit [`fc14fb8`](https://github.co

[GitHub] [spark] geekyouth commented on pull request #33292: [SPARK-36069] spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

2021-07-11 Thread GitBox
geekyouth commented on pull request #33292: URL: https://github.com/apache/spark/pull/33292#issuecomment-877993199 > @geekyouth can you create a PR against the master branch? OK, I will do it now. -- This is an automated message from the Apache Git Service. To respond to the messag

[GitHub] [spark] SparkQA commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

2021-07-11 Thread GitBox
SparkQA commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-877993163 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45408/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877992146 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45409/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877940872 **[Test build #140893 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140893/testReport)** for PR 31517 at commit [`ee05314`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877991274 **[Test build #140893 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140893/testReport)** for PR 31517 at commit [`ee05314`](https://github.co

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33291: [SPARK-35561][SQL] Remove leading zeros from empty static number type partition

2021-07-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33291: URL: https://github.com/apache/spark/pull/33291#discussion_r667640615 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/PartitioningUtils.scala ## @@ -351,10 +351,24 @@ object Partitioning

[GitHub] [spark] SparkQA removed a comment on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-877921683 **[Test build #140891 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140891/testReport)** for PR 33038 at commit [`8b2f945`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
SparkQA commented on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-877989099 **[Test build #140891 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140891/testReport)** for PR 33038 at commit [`8b2f945`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877987718 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45407/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #32872: [SPARK-35639][SQL] Make hasCoalescedPartition return true if something was actually coalesced

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #32872: URL: https://github.com/apache/spark/pull/32872#discussion_r667637282 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/CoalesceShufflePartitionsSuite.scala ## @@ -331,7 +331,8 @@ class CoalesceShuffleP

[GitHub] [spark] HyukjinKwon commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

2021-07-11 Thread GitBox
HyukjinKwon commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-877986460 @dgd-contributor can you create a PR against the master branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

[GitHub] [spark] cloud-fan commented on a change in pull request #32872: [SPARK-35639][SQL] Make hasCoalescedPartition return true if something was actually coalesced

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #32872: URL: https://github.com/apache/spark/pull/32872#discussion_r667635965 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -87,8 +87,15 @@ case class CustomShuf

[GitHub] [spark] cloud-fan commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

2021-07-11 Thread GitBox
cloud-fan commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-877985052 missed one thing: `AQE is disabled by default.` Can we correct this in the doc as well? -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] cfmcgrady commented on a change in pull request #33212: [SPARK-35912][SQL] Fix nullability of `spark.read.json`

2021-07-11 Thread GitBox
cfmcgrady commented on a change in pull request #33212: URL: https://github.com/apache/spark/pull/33212#discussion_r667635424 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/json/JacksonParser.scala ## @@ -399,16 +399,24 @@ class JacksonParser( val

[GitHub] [spark] cloud-fan closed pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

2021-07-11 Thread GitBox
cloud-fan closed pull request #32960: URL: https://github.com/apache/spark/pull/32960 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] cloud-fan commented on pull request #32960: [SPARK-35813][SQL][DOCS] Add new adaptive config into sql-performance-tuning docs

2021-07-11 Thread GitBox
cloud-fan commented on pull request #32960: URL: https://github.com/apache/spark/pull/32960#issuecomment-877984476 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] HyukjinKwon commented on pull request #33292: [SPARK-36069] spark function from_json should output field name, field type and field value when FAILFAST mode throw exception

2021-07-11 Thread GitBox
HyukjinKwon commented on pull request #33292: URL: https://github.com/apache/spark/pull/33292#issuecomment-877983543 @geekyouth can you create a PR against the master branch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub an

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

2021-07-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33142: URL: https://github.com/apache/spark/pull/33142#discussion_r667633052 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -170,65 +167,71 @@ class Equiva

[GitHub] [spark] SparkQA commented on pull request #32959: [SPARK-35780][SQL] Support DATE/TIMESTAMP literals across the full range

2021-07-11 Thread GitBox
SparkQA commented on pull request #32959: URL: https://github.com/apache/spark/pull/32959#issuecomment-877981379 **[Test build #140898 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140898/testReport)** for PR 32959 at commit [`e146045`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877980413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45404/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877980411 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877907828 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877980410 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140888/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877980413 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45404/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877980412 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] cloud-fan commented on a change in pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

2021-07-11 Thread GitBox
cloud-fan commented on a change in pull request #33142: URL: https://github.com/apache/spark/pull/33142#discussion_r667631076 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -170,65 +167,71 @@ class Equivale

[GitHub] [spark] SparkQA commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

2021-07-11 Thread GitBox
SparkQA commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-877979863 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45408/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877979695 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45409/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33142: [SPARK-35940][SQL] Refactor EquivalentExpressions to make it more efficient

2021-07-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33142: URL: https://github.com/apache/spark/pull/33142#discussion_r667627831 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/EquivalentExpressions.scala ## @@ -170,65 +167,71 @@ class Equiva

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877974240 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45407/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

2021-07-11 Thread GitBox
SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877969669 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45405/ -- This is an automated message from the A

[GitHub] [spark] gengliangwang commented on a change in pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

2021-07-11 Thread GitBox
gengliangwang commented on a change in pull request #33258: URL: https://github.com/apache/spark/pull/33258#discussion_r667622706 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -200,6 +200,30 @@ case class No

[GitHub] [spark] gengliangwang commented on a change in pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

2021-07-11 Thread GitBox
gengliangwang commented on a change in pull request #33258: URL: https://github.com/apache/spark/pull/33258#discussion_r667622302 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/DateTimeUtils.scala ## @@ -968,6 +968,11 @@ object DateTimeUtils {

[GitHub] [spark] SparkQA commented on pull request #33258: [SPARK-36037][SQL] Support ANSI SQL LOCALTIMESTAMP datetime value function

2021-07-11 Thread GitBox
SparkQA commented on pull request #33258: URL: https://github.com/apache/spark/pull/33258#issuecomment-877966933 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45403/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877887767 **[Test build #140888 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140888/testReport)** for PR 32049 at commit [`ccdc543`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-11 Thread GitBox
SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-877965693 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/45404/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #32049: [SPARK-34952][SQL] Aggregate (Min/Max/Count) push down for Parquet

2021-07-11 Thread GitBox
SparkQA commented on pull request #32049: URL: https://github.com/apache/spark/pull/32049#issuecomment-877965636 **[Test build #140888 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140888/testReport)** for PR 32049 at commit [`ccdc543`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962895 **[Test build #140897 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140897/testReport)** for PR 33294 at commit [`2360d07`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962103 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140895/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA removed a comment on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877959913 **[Test build #140895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140895/testReport)** for PR 33294 at commit [`2360d07`](https://gi

[GitHub] [spark] ulysses-you commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
ulysses-you commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962597 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33263: [SPARK-35027][CORE] Close the inputStream in FileAppender when writin…

2021-07-11 Thread GitBox
HyukjinKwon commented on a change in pull request #33263: URL: https://github.com/apache/spark/pull/33263#discussion_r667616566 ## File path: core/src/main/scala/org/apache/spark/deploy/worker/ExecutorRunner.scala ## @@ -185,11 +185,11 @@ private[deploy] class ExecutorRunner(

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962082 **[Test build #140895 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140895/testReport)** for PR 33294 at commit [`2360d07`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877962103 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/140895/ -- This

[GitHub] [spark] SparkQA commented on pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

2021-07-11 Thread GitBox
SparkQA commented on pull request #33280: URL: https://github.com/apache/spark/pull/33280#issuecomment-877961618 **[Test build #140896 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140896/testReport)** for PR 33280 at commit [`4c11b7f`](https://github.com

[GitHub] [spark] gengliangwang commented on a change in pull request #33280: [SPARK-36072][SQL] TO_TIMESTAMP: return different results based on the default timestamp type

2021-07-11 Thread GitBox
gengliangwang commented on a change in pull request #33280: URL: https://github.com/apache/spark/pull/33280#discussion_r667614979 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/datetimeExpressions.scala ## @@ -1008,17 +1008,17 @@ case class

[GitHub] [spark] SparkQA commented on pull request #33294: [SPARK-36085][SQL] Make broadcast query stage executionContext isolation from AQE

2021-07-11 Thread GitBox
SparkQA commented on pull request #33294: URL: https://github.com/apache/spark/pull/33294#issuecomment-877959913 **[Test build #140895 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/140895/testReport)** for PR 33294 at commit [`2360d07`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #33293: [SPARK-36076][SQL][3.1] ArrayIndexOutOfBounds in Cast string to times…

2021-07-11 Thread GitBox
AmplabJenkins commented on pull request #33293: URL: https://github.com/apache/spark/pull/33293#issuecomment-877958950 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33038: [SPARK-35861][SS] Introduce "prefix match scan" feature on state store

2021-07-11 Thread GitBox
AmplabJenkins removed a comment on pull request #33038: URL: https://github.com/apache/spark/pull/33038#issuecomment-877958525 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/45402/

  1   2   3   >