[GitHub] [spark] gaborgsomogyi commented on a change in pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

gaborgsomogyi commented on a change in pull request #29024: URL: https://github.com/apache/spark/pull/29024#discussion_r754476290 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/ConnectionProvider.scala ## @@ -18,60 +18,45 @@

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

AmplabJenkins removed a comment on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975723994 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

SparkQA removed a comment on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975607333 **[Test build #145512 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145512/testReport)** for PR 34683 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

AmplabJenkins removed a comment on pull request #34671: URL: https://github.com/apache/spark/pull/34671#issuecomment-975723995 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145514/

[GitHub] [spark] AmplabJenkins commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

AmplabJenkins commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975723994 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [spark] AmplabJenkins commented on pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

AmplabJenkins commented on pull request #34671: URL: https://github.com/apache/spark/pull/34671#issuecomment-975723995 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145514/ -- This

[GitHub] [spark] SparkQA commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

SparkQA commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975721740 **[Test build #145512 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145512/testReport)** for PR 34683 at commit

[GitHub] [spark] gengliangwang commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

gengliangwang commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754460682 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala ## @@ -1002,15 +1002,19 @@ private[hive] class

[GitHub] [spark] SparkQA commented on pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

SparkQA commented on pull request #34671: URL: https://github.com/apache/spark/pull/34671#issuecomment-975718740 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49985/ -- This is an automated message from the Apache

[GitHub] [spark] gengliangwang commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

gengliangwang commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754456952 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class

[GitHub] [spark] SparkQA commented on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

SparkQA commented on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975716293 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49986/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975714284 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49987/ -- This is an automated message from the Apache

[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

sarutak commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754268757 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase

[GitHub] [spark] srowen commented on a change in pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

srowen commented on a change in pull request #29024: URL: https://github.com/apache/spark/pull/29024#discussion_r754452221 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/ConnectionProvider.scala ## @@ -18,60 +18,45 @@ package

[GitHub] [spark] SparkQA commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

SparkQA commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975708801 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49984/ -- This is an automated message from the

[GitHub] [spark] tdg5 commented on a change in pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

tdg5 commented on a change in pull request #29024: URL: https://github.com/apache/spark/pull/29024#discussion_r754441783 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/ConnectionProvider.scala ## @@ -18,60 +18,45 @@ package

[GitHub] [spark] tdg5 commented on pull request #31622: [SPARK-34497][SQL] Fix built-in JDBC connection providers to restore JVM security context changes

tdg5 commented on pull request #31622: URL: https://github.com/apache/spark/pull/31622#issuecomment-975696894 Moved my whining over to https://github.com/apache/spark/pull/29024/files#r754441783. Sorry for the noise on this PR. -- This is an automated message from the Apache Git

[GitHub] [spark] tdg5 commented on a change in pull request #29024: [SPARK-32001][SQL]Create JDBC authentication provider developer API

tdg5 commented on a change in pull request #29024: URL: https://github.com/apache/spark/pull/29024#discussion_r754441783 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/jdbc/connection/ConnectionProvider.scala ## @@ -18,60 +18,45 @@ package

[GitHub] [spark] SparkQA removed a comment on pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

SparkQA removed a comment on pull request #34671: URL: https://github.com/apache/spark/pull/34671#issuecomment-975667464 **[Test build #145514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145514/testReport)** for PR 34671 at commit

[GitHub] [spark] SparkQA commented on pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

SparkQA commented on pull request #34671: URL: https://github.com/apache/spark/pull/34671#issuecomment-975695597 **[Test build #145514 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145514/testReport)** for PR 34671 at commit

[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

sarutak commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754268757 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase

[GitHub] [spark] tdg5 commented on pull request #31622: [SPARK-34497][SQL] Fix built-in JDBC connection providers to restore JVM security context changes

tdg5 commented on pull request #31622: URL: https://github.com/apache/spark/pull/31622#issuecomment-975686621 @srowen I believe the problem is actually introduced here: https://github.com/apache/spark/pull/29024/files#diff-345beef18081272d77d91eeca2d9b5534ff6e642245352f40f4e9c9b8922b085R58

[GitHub] [spark] srowen commented on pull request #31622: [SPARK-34497][SQL] Fix built-in JDBC connection providers to restore JVM security context changes

srowen commented on pull request #31622: URL: https://github.com/apache/spark/pull/31622#issuecomment-975678842 I'm not immediately seeing the problem. this removed a lot of synchronization and introduced it at create() only -- This is an automated message from the Apache Git Service.

[GitHub] [spark] sarutak commented on a change in pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

sarutak commented on a change in pull request #34683: URL: https://github.com/apache/spark/pull/34683#discussion_r754422263 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ## @@ -1411,6 +1411,7 @@ object HiveExternalCatalog {

[GitHub] [spark] gengliangwang commented on a change in pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

gengliangwang commented on a change in pull request #34681: URL: https://github.com/apache/spark/pull/34681#discussion_r754415776 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/AnsiTypeCoercion.scala ## @@ -348,6 +290,9 @@ object

[GitHub] [spark] SparkQA commented on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

SparkQA commented on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975667590 **[Test build #145515 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145515/testReport)** for PR 34668 at commit

[GitHub] [spark] SparkQA commented on pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

SparkQA commented on pull request #34671: URL: https://github.com/apache/spark/pull/34671#issuecomment-975667464 **[Test build #145514 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145514/testReport)** for PR 34671 at commit

[GitHub] [spark] SparkQA commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975667360 **[Test build #145513 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145513/testReport)** for PR 34681 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

AmplabJenkins removed a comment on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975664103 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145509/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins removed a comment on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975664102 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49983/

[GitHub] [spark] AmplabJenkins commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975664102 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49983/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

AmplabJenkins commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975664103 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145509/ -- This

[GitHub] [spark] cloud-fan closed pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

cloud-fan closed pull request #34666: URL: https://github.com/apache/spark/pull/34666 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

cloud-fan commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975658458 The GA job fails `org.apache.spark.ml.source.image.ImageFileFormatSuite` This is unrelated and passed in jenkins. I'm merging this to master, thanks! -- This is an

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975656323 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49983/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

cloud-fan commented on a change in pull request #34683: URL: https://github.com/apache/spark/pull/34683#discussion_r754397467 ## File path: sql/hive/src/main/scala/org/apache/spark/sql/hive/HiveExternalCatalog.scala ## @@ -1411,6 +1411,7 @@ object HiveExternalCatalog {

[GitHub] [spark] SparkQA commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

SparkQA commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975651914 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49984/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r754394988 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParseDriver.scala ## @@ -91,6 +91,23 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r754394021 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/parser/ParseDriver.scala ## @@ -91,6 +91,23 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r754392659 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -1926,7 +1926,7 @@ SIMPLE_COMMENT ;

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r754392659 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -1926,7 +1926,7 @@ SIMPLE_COMMENT ;

[GitHub] [spark] pralabhkumar commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

pralabhkumar commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975647107 @HyukjinKwon Tests are passed . Please merge the PR , if everything looks ok . -- This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] tgravescs commented on pull request #34672: [SPARK-37394][CORE] Skip registering with ESS if a customized shuffle manager is configured

tgravescs commented on pull request #34672: URL: https://github.com/apache/spark/pull/34672#issuecomment-975646985 So my first reaction is: you have a 3rd party shuffle manager that is an external shuffle service because it supports dynamic allocation, then why is it failing... is it

[GitHub] [spark] sarutak commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

sarutak commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975635620 cc: @cloud-fan too. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA removed a comment on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975362023 **[Test build #145509 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145509/testReport)** for PR 34666 at commit

[GitHub] [spark] tdg5 edited a comment on pull request #31622: [SPARK-34497][SQL] Fix built-in JDBC connection providers to restore JVM security context changes

tdg5 edited a comment on pull request #31622: URL: https://github.com/apache/spark/pull/31622#issuecomment-974350953 Help! It seems like no one considered the reality that some apps may rely on being able to establish many connections simultaneously for performance reasons. This

[GitHub] [spark] tdg5 commented on pull request #31622: [SPARK-34497][SQL] Fix built-in JDBC connection providers to restore JVM security context changes

tdg5 commented on pull request #31622: URL: https://github.com/apache/spark/pull/31622#issuecomment-975632632 @HyukjinKwon , oops, you are correct, I got confused at which PR I was looking at. The problem is actually introduced here:

[GitHub] [spark] nchammas commented on a change in pull request #34671: [SPARK-37399][SPARK-37403][PySpark][ML] Merge {ml, mllib}/common.pyi into common.py

nchammas commented on a change in pull request #34671: URL: https://github.com/apache/spark/pull/34671#discussion_r753568842 ## File path: python/mypy.ini ## @@ -41,9 +41,15 @@ disallow_untyped_defs = False [mypy-pyspark.join] disallow_untyped_defs = False

[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975631207 **[Test build #145509 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145509/testReport)** for PR 34666 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

AmplabJenkins removed a comment on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975606552 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145508/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins removed a comment on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975606553 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145511/

[GitHub] [spark] AmplabJenkins commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

AmplabJenkins commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975606552 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145508/ -- This

[GitHub] [spark] SparkQA commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

SparkQA commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975607333 **[Test build #145512 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145512/testReport)** for PR 34683 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

AmplabJenkins commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975606553 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145511/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA removed a comment on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975346763 **[Test build #145508 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145508/testReport)** for PR 34681 at commit

[GitHub] [spark] SparkQA commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975602765 **[Test build #145508 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145508/testReport)** for PR 34681 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #34683: [SPARK-37283][SQL][FOLLOWUP] Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

HyukjinKwon commented on pull request #34683: URL: https://github.com/apache/spark/pull/34683#issuecomment-975600467 cc @beliefer -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975598739 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49983/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA removed a comment on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975549058 **[Test build #145511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145511/testReport)** for PR 34401 at commit

[GitHub] [spark] sarutak commented on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

sarutak commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975586524 @LuciferYang BTW, how about SBT? I think this change doesn't care SBT. Also, it's better to note how you tested more concretely (e.g. show the command lines) to allow

[GitHub] [spark] sarutak commented on a change in pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

sarutak commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r754325315 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + +

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975579552 **[Test build #145511 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145511/testReport)** for PR 34401 at commit

[GitHub] [spark] sarutak opened a new pull request #34683: [SPARK-37283][SQL][FOLLOWQUP Avoid trying to store a table which contains timestamp_ntz types in Hive compatible format

sarutak opened a new pull request #34683: URL: https://github.com/apache/spark/pull/34683 ### What changes were proposed in this pull request? This is PR is to avoid trying to store a table which contains `timestamp_ntz` types in Hive compatible format. In the current master,

[GitHub] [spark] gengliangwang commented on a change in pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

gengliangwang commented on a change in pull request #34681: URL: https://github.com/apache/spark/pull/34681#discussion_r754304433 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala ## @@ -31,75 +31,49 @@ import

[GitHub] [spark] gengliangwang commented on a change in pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

gengliangwang commented on a change in pull request #34681: URL: https://github.com/apache/spark/pull/34681#discussion_r754303746 ## File path: sql/core/src/test/resources/sql-tests/results/ansi/string-functions.sql.out ## @@ -545,37 +539,33 @@ AABB -- !query SELECT

[GitHub] [spark] SparkQA commented on pull request #34401: [SPARK-30537][PYTHON] Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

SparkQA commented on pull request #34401: URL: https://github.com/apache/spark/pull/34401#issuecomment-975549058 **[Test build #145511 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145511/testReport)** for PR 34401 at commit

[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

sarutak commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754268757 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase

[GitHub] [spark] sarutak commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

sarutak commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754268757 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class SQLQuerySuiteBase

[GitHub] [spark] cloud-fan commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

cloud-fan commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754252588 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class

[GitHub] [spark] sarutak commented on pull request #34576: [SPARK-37282][TESTS][FOLLOWUP] Mark `YarnShuffleServiceSuite` as ExtendedLevelDBTest

sarutak commented on pull request #34576: URL: https://github.com/apache/spark/pull/34576#issuecomment-975494129 @LuciferYang Thank you for double checking! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

AmplabJenkins removed a comment on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975490386 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49982/

[GitHub] [spark] AmplabJenkins commented on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

AmplabJenkins commented on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975490386 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49982/ --

[GitHub] [spark] cloud-fan commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

cloud-fan commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754237594 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/execution/SQLQuerySuite.scala ## @@ -2659,6 +2660,19 @@ abstract class

[GitHub] [spark] cloud-fan commented on a change in pull request #34647: [SPARK-36180][SQL] Support TimestampNTZ type in Hive

cloud-fan commented on a change in pull request #34647: URL: https://github.com/apache/spark/pull/34647#discussion_r754236576 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/types/DataType.scala ## @@ -392,6 +392,8 @@ object DataType {

[GitHub] [spark] cloud-fan commented on a change in pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

cloud-fan commented on a change in pull request #34681: URL: https://github.com/apache/spark/pull/34681#discussion_r754235244 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/TypeCoercionSuite.scala ## @@ -31,75 +31,49 @@ import

[GitHub] [spark] SparkQA commented on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

SparkQA commented on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975480664 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49982/ -- This is an automated message from the

[GitHub] [spark] cloud-fan commented on a change in pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

cloud-fan commented on a change in pull request #34681: URL: https://github.com/apache/spark/pull/34681#discussion_r754234230 ## File path: sql/core/src/test/resources/sql-tests/results/ansi/string-functions.sql.out ## @@ -545,37 +539,33 @@ AABB -- !query SELECT lpad('abc',

[GitHub] [spark] beliefer commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

beliefer commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-975478373 @cloud-fan @gengliangwang Thank you for your review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] cloud-fan closed pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

cloud-fan closed pull request #34644: URL: https://github.com/apache/spark/pull/34644 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] cloud-fan commented on pull request #34644: [SPARK-36357][SQL] Support pushdown Timestamp without time zone for orc

cloud-fan commented on pull request #34644: URL: https://github.com/apache/spark/pull/34644#issuecomment-975461123 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] zero323 commented on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

zero323 commented on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975457282 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] zero323 closed pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

zero323 closed pull request #34682: URL: https://github.com/apache/spark/pull/34682 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[GitHub] [spark] HyukjinKwon edited a comment on pull request #34641: [SPARK-37368][SQL][TESTS] Explicit GC for TPC-DS query runs

HyukjinKwon edited a comment on pull request #34641: URL: https://github.com/apache/spark/pull/34641#issuecomment-972554762 Merged to master. Let's see how it gose. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

AmplabJenkins removed a comment on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975441782 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

AmplabJenkins removed a comment on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975441783 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145510/

[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

AmplabJenkins commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975442169 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49981/ --

[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975442133 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49981/ -- This is an automated message from the

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

AmplabJenkins removed a comment on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975441784 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49980/

[GitHub] [spark] AmplabJenkins commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

AmplabJenkins commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975441784 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49980/ --

[GitHub] [spark] AmplabJenkins commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

AmplabJenkins commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975441782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145499/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

AmplabJenkins commented on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975441783 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145510/ -- This

[GitHub] [spark] SparkQA commented on pull request #34681: [SPARK-37438][SQL] ANSI mode: Use store assignment rules for resolving function invocation

SparkQA commented on pull request #34681: URL: https://github.com/apache/spark/pull/34681#issuecomment-975433877 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49980/ -- This is an automated message from the

[GitHub] [spark] SparkQA commented on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

SparkQA commented on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975433196 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49982/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA removed a comment on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA removed a comment on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975181998 **[Test build #145499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145499/testReport)** for PR 34666 at commit

[GitHub] [spark] zero323 commented on a change in pull request #34674: [SPARK-37419][PYTHON][ML] Rewrite _shared_params_code_gen.py to inline type hints for ml/param/shared.py

zero323 commented on a change in pull request #34674: URL: https://github.com/apache/spark/pull/34674#discussion_r754186121 ## File path: python/pyspark/ml/param/__init__.pyi ## @@ -94,3 +94,6 @@ class Params(pyspark.ml.util.Identifiable, metaclass=abc.ABCMeta): def

[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975431880 **[Test build #145499 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145499/testReport)** for PR 34666 at commit

[GitHub] [spark] zero323 commented on a change in pull request #34674: [SPARK-37419][PYTHON][ML] Rewrite _shared_params_code_gen.py to inline type hints for ml/param/shared.py

zero323 commented on a change in pull request #34674: URL: https://github.com/apache/spark/pull/34674#discussion_r754186121 ## File path: python/pyspark/ml/param/__init__.pyi ## @@ -94,3 +94,6 @@ class Params(pyspark.ml.util.Identifiable, metaclass=abc.ABCMeta): def

[GitHub] [spark] SparkQA commented on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

SparkQA commented on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975428772 **[Test build #145510 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145510/testReport)** for PR 34682 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #34682: [MINOR][R][DOCS] Fix misplaced brackets in SparkR:::mapValues doc

SparkQA removed a comment on pull request #34682: URL: https://github.com/apache/spark/pull/34682#issuecomment-975397786 **[Test build #145510 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145510/testReport)** for PR 34682 at commit

[GitHub] [spark] zero323 commented on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

zero323 commented on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975405755 Same as https://github.com/apache/spark/pull/34680#issuecomment-975397090. Ignores here will be obsolete after SPARK-37152 is completed. -- This is an automated message from

<    1   2   3   4   5   6   >