[GitHub] [spark] AmplabJenkins commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-900014181 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142534/ -- This

[GitHub] [spark] SparkQA commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-900013873 **[Test build #142534 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142534/testReport)** for PR 33757 at commit

[GitHub] [spark] SparkQA commented on pull request #33723: [SPARK-36496][SQL] Remove literals from grouping expressions when using the DataFrame withColumn API

2021-08-16 Thread GitBox
SparkQA commented on pull request #33723: URL: https://github.com/apache/spark/pull/33723#issuecomment-900013805 **[Test build #142536 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142536/testReport)** for PR 33723 at commit

[GitHub] [spark] tanelk commented on pull request #33723: [SPARK-36496][SQL] Remove literals from grouping expressions when using the DataFrame withColumn API

2021-08-16 Thread GitBox
tanelk commented on pull request #33723: URL: https://github.com/apache/spark/pull/33723#issuecomment-900013436 pinging @maropu -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-900011854 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47031/ --

[GitHub] [spark] SparkQA commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-900011833 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47031/ -- This is an automated message from the

[GitHub] [spark] HyukjinKwon commented on pull request #33754: [SPARK-36526][SQL] DSV2 Index Support: Add supportsIndex interface

2021-08-16 Thread GitBox
HyukjinKwon commented on pull request #33754: URL: https://github.com/apache/spark/pull/33754#issuecomment-900011457 Oh, okay. so it really means the concept of an index in DBMS's table. -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #33758: Revert "[SPARK-35028][SQL] ANSI mode: disallow group by aliases"

2021-08-16 Thread GitBox
SparkQA commented on pull request #33758: URL: https://github.com/apache/spark/pull/33758#issuecomment-98151 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47034/ -- This is an automated message from the Apache

[GitHub] [spark] gatorsmile commented on a change in pull request #33758: Revert "[SPARK-35028][SQL] ANSI mode: disallow group by aliases"

2021-08-16 Thread GitBox
gatorsmile commented on a change in pull request #33758: URL: https://github.com/apache/spark/pull/33758#discussion_r690048714 ## File path: sql/core/src/test/resources/sql-tests/inputs/ansi/group-analytics.sql ## @@ -1 +0,0 @@ ---IMPORT group-analytics.sql Review comment:

[GitHub] [spark] tooptoop4 commented on pull request #33332: [SPARK-36147][SQL] Warn if less files visible after stats write in BasicWriteStatsTracker

2021-08-16 Thread GitBox
tooptoop4 commented on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-95346 can this log level change be merged? @steveloughran -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] tooptoop4 removed a comment on pull request #33332: [SPARK-36147][SQL] Warn if less files visible after stats write in BasicWriteStatsTracker

2021-08-16 Thread GitBox
tooptoop4 removed a comment on pull request #2: URL: https://github.com/apache/spark/pull/2#issuecomment-888246182 can this log level change be merged? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-94901 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47029/

[GitHub] [spark] AmplabJenkins commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-94901 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47029/ --

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-94831 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47029/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #33736: [SPARK-35991][SQL] Add PlanStability suite for TPCH

2021-08-16 Thread GitBox
cloud-fan commented on a change in pull request #33736: URL: https://github.com/apache/spark/pull/33736#discussion_r690046057 ## File path: sql/core/src/test/scala/org/apache/spark/sql/PlanStabilitySuite.scala ## @@ -333,3 +340,22 @@ class

[GitHub] [spark] SparkQA commented on pull request #33753: [SPARK-36524][SQL] Common class for ANSI interval types

2021-08-16 Thread GitBox
SparkQA commented on pull request #33753: URL: https://github.com/apache/spark/pull/33753#issuecomment-94377 **[Test build #142535 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142535/testReport)** for PR 33753 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #33736: [SPARK-35991][SQL] Add PlanStability suite for TPCH

2021-08-16 Thread GitBox
cloud-fan commented on a change in pull request #33736: URL: https://github.com/apache/spark/pull/33736#discussion_r690045699 ## File path: sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala ## @@ -79,6 +79,17 @@ trait TPCDSBase extends SharedSparkSession with

[GitHub] [spark] huaxingao commented on pull request #33754: [SPARK-36526][SQL] DSV2 Index Support: Add supportsIndex interface

2021-08-16 Thread GitBox
huaxingao commented on pull request #33754: URL: https://github.com/apache/spark/pull/33754#issuecomment-94097 @HyukjinKwon Sorry for the confusion. I didn't put enough explanation in the PR's description. I updated the description. Hope it's clear now. -- This is an automated

[GitHub] [spark] SparkQA commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-94006 **[Test build #142534 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142534/testReport)** for PR 33757 at commit

[GitHub] [spark] SparkQA commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-93489 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47032/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33753: [SPARK-36524][SQL] Common class for ANSI interval types

2021-08-16 Thread GitBox
HyukjinKwon commented on a change in pull request #33753: URL: https://github.com/apache/spark/pull/33753#discussion_r690044302 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala ## @@ -222,3 +222,10 @@ private[sql] object

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-92450 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142530/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-92446 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47028/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-92447 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47033/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-92448 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47027/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-92449 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142531/

[GitHub] [spark] AmplabJenkins commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-92450 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142530/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-92446 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47028/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-92448 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47027/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-92447 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47033/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-92449 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142531/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA removed a comment on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899985674 **[Test build #142531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142531/testReport)** for PR 33744 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA removed a comment on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-899985652 **[Test build #142530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142530/testReport)** for PR 33757 at commit

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-08-16 Thread GitBox
SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-90124 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47033/ --

[GitHub] [spark] SparkQA commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-89228 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47031/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
SparkQA commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-89072 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47027/ -- This is an automated message from the

[GitHub] [spark] HeartSaVioR commented on a change in pull request #33749: [SPARK-36519][SS]Store RocksDB format version in the checkpoint for streaming queries

2021-08-16 Thread GitBox
HeartSaVioR commented on a change in pull request #33749: URL: https://github.com/apache/spark/pull/33749#discussion_r690031859 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1586,6 +1586,21 @@ object SQLConf { .stringConf

[GitHub] [spark] sumeetgajjar commented on pull request #32114: [SPARK-35011][CORE] Avoid Block Manager registrations when StopExecutor msg is in-flight

2021-08-16 Thread GitBox
sumeetgajjar commented on pull request #32114: URL: https://github.com/apache/spark/pull/32114#issuecomment-87315 > I just realized this bug does cause the real problem when working in conjunction with #24533. Basically, the re-registration issue leads to the driver thinks an executor

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-84735 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47028/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-84506 **[Test build #142531 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142531/testReport)** for PR 33744 at commit

[GitHub] [spark] SparkQA commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-83976 **[Test build #142530 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142530/testReport)** for PR 33757 at commit

[GitHub] [spark] itholic edited a comment on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
itholic edited a comment on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-899987424 ~I'm fixing CategoricalIndexTest.~ Fixed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] MaxGekk commented on a change in pull request #33753: [SPARK-36524][SQL] Common class for ANSI interval types

2021-08-16 Thread GitBox
MaxGekk commented on a change in pull request #33753: URL: https://github.com/apache/spark/pull/33753#discussion_r690034234 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/AbstractDataType.scala ## @@ -222,3 +222,11 @@ private[sql] object AnyTimestampType

[GitHub] [spark] SparkQA commented on pull request #33758: Revert "[SPARK-35028][SQL] ANSI mode: disallow group by aliases"

2021-08-16 Thread GitBox
SparkQA commented on pull request #33758: URL: https://github.com/apache/spark/pull/33758#issuecomment-80871 **[Test build #142533 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142533/testReport)** for PR 33758 at commit

[GitHub] [spark] gengliangwang opened a new pull request #33758: Revert "[SPARK-35028][SQL] ANSI mode: disallow group by aliases"

2021-08-16 Thread GitBox
gengliangwang opened a new pull request #33758: URL: https://github.com/apache/spark/pull/33758 ### What changes were proposed in this pull request? Revert [[SPARK-35028][SQL] ANSI mode: disallow group by aliases ](https://github.com/apache/spark/pull/32129) ### Why

[GitHub] [spark] itholic commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
itholic commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-899987424 I'm fixing CategoricalIndexTest. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] HyukjinKwon commented on a change in pull request #33323: [SPARK-35739][SQL] Add Java-compatible Dataset.join overloads

2021-08-16 Thread GitBox
HyukjinKwon commented on a change in pull request #33323: URL: https://github.com/apache/spark/pull/33323#discussion_r690027199 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -970,7 +995,59 @@ class Dataset[T] private[sql]( } /** - *

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899986644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47030/

[GitHub] [spark] AmplabJenkins commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899986644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47030/ --

[GitHub] [spark] SparkQA commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
SparkQA commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899986625 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47030/ --

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899986237 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47029/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-08-16 Thread GitBox
SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-899986032 **[Test build #142532 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142532/testReport)** for PR 33588 at commit

[GitHub] [spark] SparkQA commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899985674 **[Test build #142531 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142531/testReport)** for PR 33744 at commit

[GitHub] [spark] SparkQA commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
SparkQA commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-899985652 **[Test build #142530 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142530/testReport)** for PR 33757 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-899410081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] Ngone51 edited a comment on pull request #32114: [SPARK-35011][CORE] Avoid Block Manager registrations when StopExecutor msg is in-flight

2021-08-16 Thread GitBox
Ngone51 edited a comment on pull request #32114: URL: https://github.com/apache/spark/pull/32114#issuecomment-899979045 I just realized this bug does cause the real problem when working in conjunction with https://github.com/apache/spark/pull/24533. Basically, the re-registration issue

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899983343 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47026/

[GitHub] [spark] AmplabJenkins commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899983343 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47026/ --

[GitHub] [spark] SparkQA commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899983260 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47026/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899983001 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47028/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899982648 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47025/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899973551 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] brandondahler commented on a change in pull request #33323: [SPARK-35739][SQL] Add Java-compatible Dataset.join overloads

2021-08-16 Thread GitBox
brandondahler commented on a change in pull request #33323: URL: https://github.com/apache/spark/pull/33323#discussion_r690022929 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -970,7 +995,59 @@ class Dataset[T] private[sql]( } /** - *

[GitHub] [spark] AmplabJenkins commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899982648 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/47025/ --

[GitHub] [spark] AmplabJenkins commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899982652 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142528/ -- This

[GitHub] [spark] SparkQA commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
SparkQA commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899980920 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47027/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA removed a comment on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899965311 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] HyukjinKwon commented on pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
HyukjinKwon commented on pull request #33757: URL: https://github.com/apache/spark/pull/33757#issuecomment-899980252 cc @xinrong-databricks @ueshin FYI -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] Ngone51 edited a comment on pull request #32114: [SPARK-35011][CORE] Avoid Block Manager registrations when StopExecutor msg is in-flight

2021-08-16 Thread GitBox
Ngone51 edited a comment on pull request #32114: URL: https://github.com/apache/spark/pull/32114#issuecomment-899979045 I just realized this bug does cause the real problem when working in conjunction with https://github.com/apache/spark/pull/24533. Basically, the re-registration issue

[GitHub] [spark] itholic opened a new pull request #33757: [SPARK-36368][PYTHON] Fix CategoricalOps.astype to follow pandas 1.3

2021-08-16 Thread GitBox
itholic opened a new pull request #33757: URL: https://github.com/apache/spark/pull/33757 ### What changes were proposed in this pull request? This PR proposes to fix the behavior of `astype` for `CategoricalDtype` to follow pandas 1.3. **Before:** ```python >>>

[GitHub] [spark] Ngone51 commented on pull request #32114: [SPARK-35011][CORE] Avoid Block Manager registrations when StopExecutor msg is in-flight

2021-08-16 Thread GitBox
Ngone51 commented on pull request #32114: URL: https://github.com/apache/spark/pull/32114#issuecomment-899979045 I just realized this bug does cause the real problem when working in conjunction with https://github.com/apache/spark/pull/24533. Could you @sumeetgajjar backport this fix to

[GitHub] [spark] kevincmchen commented on a change in pull request #33746: [SPARK-36514][SQL] Support to set the meta conf in HiveMetastoreClient

2021-08-16 Thread GitBox
kevincmchen commented on a change in pull request #33746: URL: https://github.com/apache/spark/pull/33746#discussion_r690018846 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveUtils.scala ## @@ -118,6 +118,13 @@ private[spark] object HiveUtils extends

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899977543 **[Test build #142528 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142528/testReport)** for PR 33735 at commit

[GitHub] [spark] SparkQA commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899977039 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47025/ -- This is an automated message from the

[GitHub] [spark] Peng-Lei commented on pull request #33753: [SPARK-36524][SQL] Common class for ANSI interval types

2021-08-16 Thread GitBox
Peng-Lei commented on pull request #33753: URL: https://github.com/apache/spark/pull/33753#issuecomment-899976268 +1 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] viirya commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
viirya commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690015983 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileMetaCacheManager.scala ## @@ -0,0 +1,94 @@ +/* + * Licensed to the

[GitHub] [spark] viirya commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
viirya commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690015983 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileMetaCacheManager.scala ## @@ -0,0 +1,94 @@ +/* + * Licensed to the

[GitHub] [spark] SparkQA commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
SparkQA commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899974410 **[Test build #142529 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142529/testReport)** for PR 33748 at commit

[GitHub] [spark] LuciferYang commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899974034 > BTW, is it possible to have a benchmark too? @viirya I will do this later -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899973551 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/142527/ -- This

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690014322 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -967,6 +967,28 @@ object SQLConf { .booleanConf

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899973365 **[Test build #142527 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142527/testReport)** for PR 33735 at commit

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690014322 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -967,6 +967,28 @@ object SQLConf { .booleanConf

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690011178 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r689993828 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r689993828 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690011178 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899968767 **[Test build #142528 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142528/testReport)** for PR 33735 at commit

[GitHub] [spark] dongjoon-hyun commented on pull request #33755: [SPARK-36052][K8S][FOLLOWUP] Update config version to 3.2.0

2021-08-16 Thread GitBox
dongjoon-hyun commented on pull request #33755: URL: https://github.com/apache/spark/pull/33755#issuecomment-899968756 Thank you, @HyukjinKwon and @attilapiros ! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690007555 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690007555 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

[GitHub] [spark] SparkQA commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899965557 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/47026/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33735: [SPARK-36387][PYTHON] Fix Series.astype from datetime to nullable string

2021-08-16 Thread GitBox
SparkQA commented on pull request #33735: URL: https://github.com/apache/spark/pull/33735#issuecomment-899965311 **[Test build #142527 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142527/testReport)** for PR 33735 at commit

[GitHub] [spark] SparkQA commented on pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
SparkQA commented on pull request #33748: URL: https://github.com/apache/spark/pull/33748#issuecomment-899965261 **[Test build #142526 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/142526/testReport)** for PR 33748 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899963942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33756: [SPARK-36470][PYTHON] Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread GitBox
AmplabJenkins removed a comment on pull request #33756: URL: https://github.com/apache/spark/pull/33756#issuecomment-899963943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins commented on pull request #33756: [SPARK-36470][PYTHON] Implement `CategoricalIndex.map` and `DatetimeIndx.map`

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33756: URL: https://github.com/apache/spark/pull/33756#issuecomment-899963943 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
AmplabJenkins commented on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899963942 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] SparkQA removed a comment on pull request #33744: [SPARK-36403][PYTHON] Implement Index.putmask

2021-08-16 Thread GitBox
SparkQA removed a comment on pull request #33744: URL: https://github.com/apache/spark/pull/33744#issuecomment-899944732 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] LuciferYang commented on a change in pull request #33748: [SPARK-36516][SQL] Add File Metadata cache support for Orc

2021-08-16 Thread GitBox
LuciferYang commented on a change in pull request #33748: URL: https://github.com/apache/spark/pull/33748#discussion_r690002843 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/orc/OrcFileFormat.scala ## @@ -154,11 +155,18 @@ class

  1   2   3   4   5   6   7   >