[GitHub] [spark] SparkQA commented on pull request #33317: [SPARK-36095][CORE] Grouping exception in core/rdd

2021-07-26 Thread GitBox
SparkQA commented on pull request #33317: URL: https://github.com/apache/spark/pull/33317#issuecomment-886647615 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46151/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33484: [SPARK-36263][SQL][PYTHON] Add Dataframe.observation to PySpark

2021-07-26 Thread GitBox
SparkQA commented on pull request #33484: URL: https://github.com/apache/spark/pull/33484#issuecomment-886646462 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46150/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
SparkQA commented on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886641514 **[Test build #141620 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141620/testReport)** for PR 33488 at commit [`3e4cd4b`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #33459: [SPARK-36229][SQL] conv() inconsistently handles invalid strings with more than 64 invalid characters and return wrong value on

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33459: URL: https://github.com/apache/spark/pull/33459#discussion_r676539297 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/NumberConverter.scala ## @@ -89,6 +91,10 @@ object NumberConverter {

[GitHub] [spark] yoda-mon opened a new pull request #33519: [SPARK-36288][DOCS][PYTHON] Update API usage on pyspark pandas documents

2021-07-26 Thread GitBox
yoda-mon opened a new pull request #33519: URL: https://github.com/apache/spark/pull/33519 ### What changes were proposed in this pull request? Update api usage examples on PySpark pandas API documents. ### Why are the changes needed? If users try to use

[GitHub] [spark] MaxGekk edited a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
MaxGekk edited a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886628231 @cloud-fan @AngersZh @sarutak @beliefer @Peng-Lei Please, review this PR. -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [spark] MaxGekk commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
MaxGekk commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886628231 @cloud-fan @AngersZh @sarutak Please, review this PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

[GitHub] [spark] cloud-fan commented on a change in pull request #33441: [SPARK-33865][SPARK-36202][SQL] When HiveDDL, we need check avro schema too

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33441: URL: https://github.com/apache/spark/pull/33441#discussion_r676525896 ## File path: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala ## @@ -153,6 +154,27 @@ private[sql] class AvroFileFormat ext

[GitHub] [spark] HyukjinKwon commented on pull request #32332: [SPARK-35211][PYTHON] verify inferred schema for _create_dataframe

2021-07-26 Thread GitBox
HyukjinKwon commented on pull request #32332: URL: https://github.com/apache/spark/pull/32332#issuecomment-886625575 I think this looks okay (as far as I remember from the last discussion) but would be great to have a second look from @BryanCutler, @ueshin or @viirya. -- This is an autom

[GitHub] [spark] cloud-fan commented on a change in pull request #33441: [SPARK-33865][SPARK-36202][SQL] When HiveDDL, we need check avro schema too

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33441: URL: https://github.com/apache/spark/pull/33441#discussion_r676525393 ## File path: external/avro/src/main/scala/org/apache/spark/sql/avro/AvroFileFormat.scala ## @@ -153,6 +154,27 @@ private[sql] class AvroFileFormat ext

[GitHub] [spark] cloud-fan commented on pull request #33457: [SPARK-36237][UI][SQL] Attach and start handler after application started in UI

2021-07-26 Thread GitBox
cloud-fan commented on pull request #33457: URL: https://github.com/apache/spark/pull/33457#issuecomment-886624148 Shall we make the RESTFUL request hang and the web page loading if the spark application is not fully started? -- This is an automated message from the Apache Git Service. T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886622838 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141631/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886612041 **[Test build #141631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141631/testReport)** for PR 33518 at commit [`4319076`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886622838 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141631/ -- This

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886622551 **[Test build #141631 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141631/testReport)** for PR 33518 at commit [`4319076`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33429: [SPARK-36217][SQL] Rename CustomShuffleReader and OptimizeLocalShuffleReader in AQE

2021-07-26 Thread GitBox
SparkQA commented on pull request #33429: URL: https://github.com/apache/spark/pull/33429#issuecomment-886620004 **[Test build #141636 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141636/testReport)** for PR 33429 at commit [`19d6dee`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #32776: [SPARK-35639][SQL] Add metrics about coalesced partitions to CustomShuffleReader in AQE

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #32776: URL: https://github.com/apache/spark/pull/32776#discussion_r676519068 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/CustomShuffleReaderExec.scala ## @@ -182,6 +193,17 @@ case class CustomSh

[GitHub] [spark] SparkQA commented on pull request #33317: [SPARK-36095][CORE] Grouping exception in core/rdd

2021-07-26 Thread GitBox
SparkQA commented on pull request #33317: URL: https://github.com/apache/spark/pull/33317#issuecomment-886617714 **[Test build #141635 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141635/testReport)** for PR 33317 at commit [`aa9b1da`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33484: [SPARK-36263][SQL][PYTHON] Add Dataframe.observation to PySpark

2021-07-26 Thread GitBox
SparkQA commented on pull request #33484: URL: https://github.com/apache/spark/pull/33484#issuecomment-886617562 **[Test build #141634 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141634/testReport)** for PR 33484 at commit [`fa8c802`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886615514 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141616/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886615514 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141616/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33317: [SPARK-36095][CORE] Grouping exception in core/rdd

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33317: URL: https://github.com/apache/spark/pull/33317#issuecomment-878846823 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] cloud-fan commented on pull request #33317: [SPARK-36095][CORE] Grouping exception in core/rdd

2021-07-26 Thread GitBox
cloud-fan commented on pull request #33317: URL: https://github.com/apache/spark/pull/33317#issuecomment-886614956 ok to test -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] HyukjinKwon commented on pull request #33484: [SPARK-36263][SQL][PYTHON] Add Dataframe.observation to PySpark

2021-07-26 Thread GitBox
HyukjinKwon commented on pull request #33484: URL: https://github.com/apache/spark/pull/33484#issuecomment-886614529 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] SparkQA commented on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
SparkQA commented on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886614284 **[Test build #141616 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141616/testReport)** for PR 33490 at commit [`389a610`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886443993 **[Test build #141616 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141616/testReport)** for PR 33490 at commit [`389a610`](https://gi

[GitHub] [spark] cloud-fan commented on a change in pull request #33317: [SPARK-36095][CORE] Grouping exception in core/rdd

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33317: URL: https://github.com/apache/spark/pull/33317#discussion_r676514772 ## File path: core/src/main/scala/org/apache/spark/errors/SparkCoreErrors.scala ## @@ -0,0 +1,144 @@ +/* + * Licensed to the Apache Software Foundation

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886613222 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141632/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886612131 **[Test build #141632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141632/testReport)** for PR 33517 at commit [`50d8a6a`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886613222 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141632/ -- This

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886613183 **[Test build #141632 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141632/testReport)** for PR 33517 at commit [`50d8a6a`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31517: [SPARK-34309][BUILD][CORE][SQL][K8S]Use Caffeine instead of Guava Cache

2021-07-26 Thread GitBox
SparkQA commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-886613290 **[Test build #141633 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141633/testReport)** for PR 31517 at commit [`f2a656a`](https://github.com

[GitHub] [spark] HyukjinKwon commented on pull request #33482: [SPARK-36259] Expose localtimestamp in pyspark.sql.functions

2021-07-26 Thread GitBox
HyukjinKwon commented on pull request #33482: URL: https://github.com/apache/spark/pull/33482#issuecomment-886613056 Let's implement https://issues.apache.org/jira/projects/SPARK/issues/SPARK-36287 first before getting this in. -- This is an automated message from the Apache Git Service

[GitHub] [spark] cloud-fan commented on pull request #33296: [SPARK-34402][SQL] Group exception about data format schema

2021-07-26 Thread GitBox
cloud-fan commented on pull request #33296: URL: https://github.com/apache/spark/pull/33296#issuecomment-886612690 thanks, merging to master/3.2! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] cloud-fan closed pull request #33296: [SPARK-34402][SQL] Group exception about data format schema

2021-07-26 Thread GitBox
cloud-fan closed pull request #33296: URL: https://github.com/apache/spark/pull/33296 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886612131 **[Test build #141632 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141632/testReport)** for PR 33517 at commit [`50d8a6a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
SparkQA commented on pull request #33518: URL: https://github.com/apache/spark/pull/33518#issuecomment-886612041 **[Test build #141631 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141631/testReport)** for PR 33518 at commit [`4319076`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886611363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886611367 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46144/

[GitHub] [spark] AmplabJenkins commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886611363 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886611367 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46144/ -- T

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886605823 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46144/ -- This is an automated message from the A

[GitHub] [spark] MaxGekk opened a new pull request #33518: [SPARK-34619][SQL][DOCS] Describe ANSI interval types at the `Data types` page of the SQL reference

2021-07-26 Thread GitBox
MaxGekk opened a new pull request #33518: URL: https://github.com/apache/spark/pull/33518 ### What changes were proposed in this pull request? In the PR, I propose to update the page https://spark.apache.org/docs/latest/sql-ref-datatypes.html and add information about the year-month and

[GitHub] [spark] cloud-fan commented on pull request #33489: [SPARK-36269][SQL] Fix only set data columns to Hive column names config

2021-07-26 Thread GitBox
cloud-fan commented on pull request #33489: URL: https://github.com/apache/spark/pull/33489#issuecomment-886591544 thanks, merging to master/3.2/3.1/3.0! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

[GitHub] [spark] cloud-fan closed pull request #33489: [SPARK-36269][SQL] Fix only set data columns to Hive column names config

2021-07-26 Thread GitBox
cloud-fan closed pull request #33489: URL: https://github.com/apache/spark/pull/33489 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886590033 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46143/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886588641 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46145/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #33382: [SPARK-36137][SQL] HiveShim should fallback to getAllPartitionsOf even if directSQL is enabled in remote HMS

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33382: URL: https://github.com/apache/spark/pull/33382#discussion_r676488898 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveShim.scala ## @@ -876,29 +876,24 @@ private[client] class Shim_v0_13 extends

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886578997 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141619/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886578997 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141619/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886446409 **[Test build #141619 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141619/testReport)** for PR 33488 at commit [`27eb8e1`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
SparkQA commented on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886578418 **[Test build #141619 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141619/testReport)** for PR 33488 at commit [`27eb8e1`](https://github.co

[GitHub] [spark] linhongliu-db commented on a change in pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
linhongliu-db commented on a change in pull request #33490: URL: https://github.com/apache/spark/pull/33490#discussion_r676294216 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala ## @@ -289,6 +290,11 @@ class DateTimeUtilsSu

[GitHub] [spark] linhongliu-db commented on a change in pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
linhongliu-db commented on a change in pull request #33490: URL: https://github.com/apache/spark/pull/33490#discussion_r676294216 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/util/DateTimeUtilsSuite.scala ## @@ -289,6 +290,11 @@ class DateTimeUtilsSu

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886573082 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46146/

[GitHub] [spark] SparkQA commented on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
SparkQA commented on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886573046 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46146/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886573082 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46146/ -- T

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886571502 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46144/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886569731 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33516: [SPARK-34249][DOCS] Add documentation for ANSI implicit cast rules

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33516: URL: https://github.com/apache/spark/pull/33516#issuecomment-886569734 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46138/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33482: [SPARK-36259] Expose localtimestamp in pyspark.sql.functions

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33482: URL: https://github.com/apache/spark/pull/33482#issuecomment-886569730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins commented on pull request #33516: [SPARK-34249][DOCS] Add documentation for ANSI implicit cast rules

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33516: URL: https://github.com/apache/spark/pull/33516#issuecomment-886569734 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46138/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33482: [SPARK-36259] Expose localtimestamp in pyspark.sql.functions

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33482: URL: https://github.com/apache/spark/pull/33482#issuecomment-886569730 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886569732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] SparkQA commented on pull request #33482: [SPARK-36259] Expose localtimestamp in pyspark.sql.functions

2021-07-26 Thread GitBox
SparkQA commented on pull request #33482: URL: https://github.com/apache/spark/pull/33482#issuecomment-886567563 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46141/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886566782 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46143/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886564545 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46145/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #33404: [SPARK-36194][SQL] Remove the aggregation from left semi/anti join if the same aggregation has already been done on left side

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33404: URL: https://github.com/apache/spark/pull/33404#discussion_r676469338 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/DistinctAttributesVisitor.scala ## @@ -0,0 +1,87 @@ +/* + * Licen

[GitHub] [spark] cloud-fan commented on pull request #33310: [SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-26 Thread GitBox
cloud-fan commented on pull request #33310: URL: https://github.com/apache/spark/pull/33310#issuecomment-886555280 thanks, merging to master/3.2 (since AQE is turned on by default in 3.2) -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] cloud-fan closed pull request #33310: [SPARK-36105][SQL] OptimizeLocalShuffleReader support reading data of multiple mappers in one task

2021-07-26 Thread GitBox
cloud-fan closed pull request #33310: URL: https://github.com/apache/spark/pull/33310 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] SparkQA commented on pull request #33516: [SPARK-34249][DOCS] Add documentation for ANSI implicit cast rules

2021-07-26 Thread GitBox
SparkQA commented on pull request #33516: URL: https://github.com/apache/spark/pull/33516#issuecomment-886553418 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46138/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33482: [SPARK-36259] Expose localtimestamp in pyspark.sql.functions

2021-07-26 Thread GitBox
SparkQA commented on pull request #33482: URL: https://github.com/apache/spark/pull/33482#issuecomment-886552558 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46140/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886546510 **[Test build #141629 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141629/testReport)** for PR 33364 at commit [`e83f194`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886533682 **[Test build #141629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141629/testReport)** for PR 33364 at commit [`e83f194`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886530969 **[Test build #141627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141627/testReport)** for PR 33364 at commit [`6e12157`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886543634 **[Test build #141627 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141627/testReport)** for PR 33364 at commit [`6e12157`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33482: [SPARK-36259] Expose localtimestamp in pyspark.sql.functions

2021-07-26 Thread GitBox
SparkQA commented on pull request #33482: URL: https://github.com/apache/spark/pull/33482#issuecomment-886542614 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46141/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886539801 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46139/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33340: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33340: URL: https://github.com/apache/spark/pull/33340#issuecomment-886540204 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141618/ -

[GitHub] [spark] AmplabJenkins commented on pull request #33340: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33340: URL: https://github.com/apache/spark/pull/33340#issuecomment-886540204 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141618/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33340: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33340: URL: https://github.com/apache/spark/pull/33340#issuecomment-886444248 **[Test build #141618 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141618/testReport)** for PR 33340 at commit [`1d40d59`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
SparkQA commented on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886539764 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46139/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886539801 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46139/ -- T

[GitHub] [spark] SparkQA commented on pull request #33340: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

2021-07-26 Thread GitBox
SparkQA commented on pull request #33340: URL: https://github.com/apache/spark/pull/33340#issuecomment-886539498 **[Test build #141618 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141618/testReport)** for PR 33340 at commit [`1d40d59`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
SparkQA commented on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886536073 **[Test build #141630 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141630/testReport)** for PR 33490 at commit [`48e7357`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886534367 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141628/ -

[GitHub] [spark] SparkQA removed a comment on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA removed a comment on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886533481 **[Test build #141628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141628/testReport)** for PR 33517 at commit [`2502b2a`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886534367 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141628/ -- This

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886534332 **[Test build #141628 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141628/testReport)** for PR 33517 at commit [`2502b2a`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886533682 **[Test build #141629 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141629/testReport)** for PR 33364 at commit [`e83f194`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886533481 **[Test build #141628 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141628/testReport)** for PR 33517 at commit [`2502b2a`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #33200: [SPARK-36006][SQL] Migrate ALTER TABLE ... ADD/REPLACE COLUMNS commands to use UnresolvedTable to resolve the identifier

2021-07-26 Thread GitBox
cloud-fan commented on a change in pull request #33200: URL: https://github.com/apache/spark/pull/33200#discussion_r676435402 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -3574,16 +3568,67 @@ class Analyzer(override val

[GitHub] [spark] SparkQA commented on pull request #33364: [SPARK-36161][PYTHON] Add type check on dropDuplicates pyspark function

2021-07-26 Thread GitBox
SparkQA commented on pull request #33364: URL: https://github.com/apache/spark/pull/33364#issuecomment-886530969 **[Test build #141627 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/141627/testReport)** for PR 33364 at commit [`6e12157`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886529486 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46142/

[GitHub] [spark] AmplabJenkins commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
AmplabJenkins commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886529486 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46142/ -- T

[GitHub] [spark] SparkQA commented on pull request #33517: [SPARK-36287] adding TimestampNTZType

2021-07-26 Thread GitBox
SparkQA commented on pull request #33517: URL: https://github.com/apache/spark/pull/33517#issuecomment-886529455 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/46142/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33488: [SPARK-36241][SQL] Support creating tables with void column

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33488: URL: https://github.com/apache/spark/pull/33488#issuecomment-886528600 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46134/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33340: [SPARK-36266][SHUFFLE] Rename classes in shuffle RPC used for block push operations

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33340: URL: https://github.com/apache/spark/pull/33340#issuecomment-886528599 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46136/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33490: [SPARK-36286][SQL] Block some invalid datetime string

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #33490: URL: https://github.com/apache/spark/pull/33490#issuecomment-886528602 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/46133/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32332: [SPARK-35211][PYTHON] verify inferred schema for _create_dataframe

2021-07-26 Thread GitBox
AmplabJenkins removed a comment on pull request #32332: URL: https://github.com/apache/spark/pull/32332#issuecomment-886528598 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/141624/ -

<    3   4   5   6   7   8   9   10   >