[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-975217562 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49972/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34679: [SPARK-37437][BUILD] Remove unused hive profile

SparkQA commented on pull request #34679: URL: https://github.com/apache/spark/pull/34679#issuecomment-975217339 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49973/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

SparkQA commented on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-975216033 **[Test build #145505 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145505/testReport)** for PR 34060 at commit [`a2dd853`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

SparkQA commented on pull request #34513: URL: https://github.com/apache/spark/pull/34513#issuecomment-975215676 **[Test build #145504 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145504/testReport)** for PR 34513 at commit [`836903a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34680: [SPARK-37421][PYTHON] Inline type hints for python/pyspark/mllib/evaluation.py

SparkQA commented on pull request #34680: URL: https://github.com/apache/spark/pull/34680#issuecomment-975215469 **[Test build #145503 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145503/testReport)** for PR 34680 at commit [`e686682`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

AmplabJenkins removed a comment on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975214010 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49968/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

AmplabJenkins removed a comment on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975214012 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145492/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

AmplabJenkins removed a comment on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975214009 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145497/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

AmplabJenkins commented on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975214009 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145497/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

AmplabJenkins commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975214010 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49968/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

AmplabJenkins commented on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975214012 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145492/ -- This

[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

cloud-fan commented on a change in pull request #34666: URL: https://github.com/apache/spark/pull/34666#discussion_r754016866 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala ## @@ -56,7 +44,7 @@ trait ShowTblProperti

[GitHub] [spark] SparkQA commented on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

SparkQA commented on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975213361 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49969/ -- This is an automated message from the Apache

[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

cloud-fan commented on a change in pull request #34666: URL: https://github.com/apache/spark/pull/34666#discussion_r754016866 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala ## @@ -56,7 +44,7 @@ trait ShowTblProperti

[GitHub] [spark] cloud-fan commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

cloud-fan commented on a change in pull request #34666: URL: https://github.com/apache/spark/pull/34666#discussion_r754015958 ## File path: sql/core/src/test/resources/sql-tests/inputs/show-tblproperties.sql ## @@ -1,26 +0,0 @@ --- create a table with properties -CREATE TABLE t

[GitHub] [spark] SparkQA removed a comment on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

SparkQA removed a comment on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975022788 **[Test build #145492 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145492/testReport)** for PR 34532 at commit [`9436901`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

SparkQA commented on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975210052 **[Test build #145492 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145492/testReport)** for PR 34532 at commit [`9436901`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975209500 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49971/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34676: [SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975208219 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49968/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-975208165 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49970/ -- This is an automated message from the Apache

[GitHub] [spark] dchvn commented on a change in pull request #34680: [SPARK-37421][PYTHON] Inline type hints for python/pyspark/mllib/evaluation.py

dchvn commented on a change in pull request #34680: URL: https://github.com/apache/spark/pull/34680#discussion_r754003277 ## File path: python/pyspark/mllib/evaluation.py ## @@ -74,29 +80,31 @@ def __init__(self, scoreAndLabels): if numCol == 3: schema.add

[GitHub] [spark] dchvn commented on pull request #34680: [SPARK-37421][PYTHON] Inline type hints for python/pyspark/mllib/evaluation.py

dchvn commented on pull request #34680: URL: https://github.com/apache/spark/pull/34680#issuecomment-975196185 cc @zero323, @HyukjinKwon, @ueshin FYI. Many thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [spark] SparkQA removed a comment on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

SparkQA removed a comment on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975181888 **[Test build #145497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145497/testReport)** for PR 34678 at commit [`609831c`](https://gi

[GitHub] [spark] dchvn commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

dchvn commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r754001496 ## File path: python/pyspark/mllib/stat/_statistics.py ## @@ -16,51 +16,57 @@ # import sys +from typing import overload, List, Optional, Union +from typ

[GitHub] [spark] SparkQA commented on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

SparkQA commented on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975195667 **[Test build #145497 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145497/testReport)** for PR 34678 at commit [`609831c`](https://github.co

[GitHub] [spark] dchvn commented on a change in pull request #34680: [SPARK-37421][PYTHON] Inline type hints for python/pyspark/mllib/evaluation.py

dchvn commented on a change in pull request #34680: URL: https://github.com/apache/spark/pull/34680#discussion_r754000217 ## File path: python/pyspark/mllib/evaluation.py ## @@ -74,29 +80,31 @@ def __init__(self, scoreAndLabels): if numCol == 3: schema.add

[GitHub] [spark] dchvn commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

dchvn commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753999742 ## File path: python/pyspark/mllib/linalg/__init__.pyi ## @@ -68,6 +68,7 @@ class Vector: __UDT__: VectorUDT def toArray(self) -> ndarray: ...

[GitHub] [spark] dchvn opened a new pull request #34680: [SPARK-37421][PYTHON] Inline type hints for python/pyspark/mllib/evaluation.py

dchvn opened a new pull request #34680: URL: https://github.com/apache/spark/pull/34680 ### What changes were proposed in this pull request? Inline type hints for evaluation.py in python/pyspark/mllib/ ### Why are the changes needed? We can take advantage of static type checking wit

[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-975190436 **[Test build #145502 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145502/testReport)** for PR 34677 at commit [`091bd96`](https://github.com

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r753996710 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -73,6 +73,21 @@ grammar SqlBase; return false;

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r753996055 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -73,6 +73,21 @@ grammar SqlBase; return false;

[GitHub] [spark] cloud-fan commented on a change in pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

cloud-fan commented on a change in pull request #34668: URL: https://github.com/apache/spark/pull/34668#discussion_r753995720 ## File path: sql/catalyst/src/main/antlr4/org/apache/spark/sql/catalyst/parser/SqlBase.g4 ## @@ -73,6 +73,21 @@ grammar SqlBase; return false;

[GitHub] [spark] dchvn commented on a change in pull request #34513: [SPARK-37234][PYTHON] Inline type hints for python/pyspark/mllib/stat/_statistics.py

dchvn commented on a change in pull request #34513: URL: https://github.com/apache/spark/pull/34513#discussion_r753995410 ## File path: python/pyspark/mllib/stat/_statistics.py ## @@ -170,10 +190,29 @@ def corr(x, y=None, method=None): if not y: return cal

[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-975185264 **[Test build #145501 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145501/testReport)** for PR 34677 at commit [`bace4c0`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34679: [SPARK-37437][BUILD] remove unused hive profile

SparkQA commented on pull request #34679: URL: https://github.com/apache/spark/pull/34679#issuecomment-975185148 **[Test build #145500 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145500/testReport)** for PR 34679 at commit [`f78af65`](https://github.com

[GitHub] [spark] AngersZhuuuu opened a new pull request #34679: [SPARK-37437][BUILD] remove unused hive profile

AngersZh opened a new pull request #34679: URL: https://github.com/apache/spark/pull/34679 ### What changes were proposed in this pull request? Since we only support hive-2.3, we should remove the unused profile. ### Why are the changes needed? remove unused profile

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975183385 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49967/

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975183385 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49967/ -- T

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975183367 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49967/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

SparkQA commented on pull request #34666: URL: https://github.com/apache/spark/pull/34666#issuecomment-975181998 **[Test build #145499 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145499/testReport)** for PR 34666 at commit [`6e8ac96`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

SparkQA commented on pull request #34677: URL: https://github.com/apache/spark/pull/34677#issuecomment-975181931 **[Test build #145498 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145498/testReport)** for PR 34677 at commit [`a8c3eca`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

SparkQA commented on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975181888 **[Test build #145497 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145497/testReport)** for PR 34678 at commit [`609831c`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

AmplabJenkins removed a comment on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975180504 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145490/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

AmplabJenkins removed a comment on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975180497 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145494/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

AmplabJenkins commented on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975180504 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145490/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

AmplabJenkins commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975180497 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145494/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA removed a comment on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975049090 **[Test build #145494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145494/testReport)** for PR 34676 at commit [`b8e917f`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975178783 **[Test build #145494 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145494/testReport)** for PR 34676 at commit [`b8e917f`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975178405 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49968/ -- This is an automated message from the Apache

[GitHub] [spark] dchvn commented on pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

dchvn commented on pull request #34678: URL: https://github.com/apache/spark/pull/34678#issuecomment-975177529 cc @zero323 Please take a look when you have time! Thanks 😄 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] dchvn opened a new pull request #34678: [SPARK-37407][PYTHON] Inline type hints for python/pyspark/ml/functions.py

dchvn opened a new pull request #34678: URL: https://github.com/apache/spark/pull/34678 ### What changes were proposed in this pull request? Inline type hints for python/pyspark/ml/functions.py ### Why are the changes needed? We can take advantage of static type chec

[GitHub] [spark] beliefer commented on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

beliefer commented on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975176620 ping @cloud-fan @gengliangwang -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA removed a comment on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

SparkQA removed a comment on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975002132 **[Test build #145490 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145490/testReport)** for PR 34668 at commit [`75c6e43`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

SparkQA commented on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975174898 **[Test build #145490 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145490/testReport)** for PR 34668 at commit [`75c6e43`](https://github.co

[GitHub] [spark] HyukjinKwon opened a new pull request #34677: [WIP][SPARK-37436][PYTHON] Uses Python's standard string formatter for SQL API in pandas API on Spark

HyukjinKwon opened a new pull request #34677: URL: https://github.com/apache/spark/pull/34677 ### What changes were proposed in this pull request? This PR proposes to use [Python's standard string formatter](https://docs.python.org/3/library/string.html#custom-string-formatting) ins

[GitHub] [spark] Peng-Lei commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

Peng-Lei commented on a change in pull request #34666: URL: https://github.com/apache/spark/pull/34666#discussion_r753982833 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/ShowTblPropertiesSuiteBase.scala ## @@ -87,4 +88,25 @@ trait ShowTblPropert

[GitHub] [spark] Peng-Lei commented on a change in pull request #34666: [SPARK-37192][SQL] Migrate SHOW TBLPROPERTIES to use V2 command by default

Peng-Lei commented on a change in pull request #34666: URL: https://github.com/apache/spark/pull/34666#discussion_r753982634 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala ## @@ -56,7 +44,7 @@ trait ShowTblPropertie

[GitHub] [spark] Yikun commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-975166197 > We should probably have to bump up .. ideally we should test all the combinatins just like other python projects .. but we can't do this due to the resource problem in GA.

[GitHub] [spark] HyukjinKwon commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

HyukjinKwon commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-975165011 Testing on 1.1.x or 1.2.x should be good enough for the fix itself. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] HyukjinKwon commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

HyukjinKwon commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-975164734 It's actually documented here; https://github.com/apache/spark/blob/master/python/setup.py#L115. We should probably have to bump up .. ideally we should test all the comb

[GitHub] [spark] Yikun commented on pull request #34314: [SPARK-36231][PYTHON] Support arithmetic operations of decimal(nan) series

Yikun commented on pull request #34314: URL: https://github.com/apache/spark/pull/34314#issuecomment-975160487 @HyukjinKwon Thanks for your help, and one more question, what's the mainly version of pandas should be tested and supported? Should we announce it in somewhere, and then add the

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

HyukjinKwon commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r753967179 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + + org.apache.spark.tags.Exte

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34670: [SPARK-37388][SQL] Fix NPE in WidthBucket in WholeStageCodegenExec

AmplabJenkins removed a comment on pull request #34670: URL: https://github.com/apache/spark/pull/34670#issuecomment-975153964 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145487/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34670: [SPARK-37388][SQL] Fix NPE in WidthBucket in WholeStageCodegenExec

AmplabJenkins commented on pull request #34670: URL: https://github.com/apache/spark/pull/34670#issuecomment-975153964 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145487/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34670: [SPARK-37388][SQL] Fix NPE in WidthBucket in WholeStageCodegenExec

SparkQA removed a comment on pull request #34670: URL: https://github.com/apache/spark/pull/34670#issuecomment-974961496 **[Test build #145487 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145487/testReport)** for PR 34670 at commit [`0cba30d`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34670: [SPARK-37388][SQL] Fix NPE in WidthBucket in WholeStageCodegenExec

SparkQA commented on pull request #34670: URL: https://github.com/apache/spark/pull/34670#issuecomment-975152621 **[Test build #145487 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145487/testReport)** for PR 34670 at commit [`0cba30d`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975151828 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49967/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975150371 **[Test build #145496 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145496/testReport)** for PR 34676 at commit [`f87a633`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

AmplabJenkins removed a comment on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975149860 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145493/ -

[GitHub] [spark] LuciferYang commented on a change in pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

LuciferYang commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r753959557 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + + org.apache.spark.tags.Exte

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

AmplabJenkins removed a comment on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975149858 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49966/

[GitHub] [spark] AmplabJenkins commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

AmplabJenkins commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975149858 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49966/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

AmplabJenkins commented on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975149860 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145493/ -- This

[GitHub] [spark] LuciferYang commented on a change in pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

LuciferYang commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r753959557 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + + org.apache.spark.tags.Exte

[GitHub] [spark] SparkQA removed a comment on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

SparkQA removed a comment on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975023113 **[Test build #145493 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145493/testReport)** for PR 33628 at commit [`1eaad94`](https://gi

[GitHub] [spark] SparkQA commented on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

SparkQA commented on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975148251 **[Test build #145493 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145493/testReport)** for PR 33628 at commit [`1eaad94`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975146317 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49966/ -- This is an automated message from the A

[GitHub] [spark] LuciferYang commented on a change in pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

LuciferYang commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r753950582 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + + org.apache.spark.tags.Exte

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975127306 **[Test build #145495 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145495/testReport)** for PR 34596 at commit [`309643b`](https://github.com

[GitHub] [spark] sadikovi commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

sadikovi commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975124738 @cloud-fan @gengliangwang I updated the code, followed the advice to pass an extra boolean flag `ignoreTimeZone` in the relevant methods. Can you review the changes? Thanks!

[GitHub] [spark] cxzl25 commented on pull request #34493: [SPARK-37217][SQL] The number of dynamic partitions should early check when writing to external tables

cxzl25 commented on pull request #34493: URL: https://github.com/apache/spark/pull/34493#issuecomment-975113863 Can we continue to review this pr? @dongjoon-hyun @sunchao @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitH

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

AmplabJenkins removed a comment on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975110769 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49964/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

AmplabJenkins removed a comment on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975110770 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49965/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975110768 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49963/

[GitHub] [spark] AmplabJenkins commented on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

AmplabJenkins commented on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975110770 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49965/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

AmplabJenkins commented on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975110769 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49964/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975110768 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49963/ -- T

[GitHub] [spark] SparkQA commented on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

SparkQA commented on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975104802 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49965/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

SparkQA commented on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975102523 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49964/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975102524 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49966/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

SparkQA commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975083874 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49963/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on a change in pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

HyukjinKwon commented on a change in pull request #34676: URL: https://github.com/apache/spark/pull/34676#discussion_r753929523 ## File path: pom.xml ## @@ -3580,6 +3580,18 @@ + + mac-on-apple-silicon + + org.apache.spark.tags.Exte

[GitHub] [spark] SparkQA commented on pull request #33628: [SPARK-36406][CORE] Avoid unnecessary file operations before delete a write failed file held by DiskBlockObjectWriter

SparkQA commented on pull request #33628: URL: https://github.com/apache/spark/pull/33628#issuecomment-975054194 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49965/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34532: [SPARK-37256][SQL] Replace `ScalaObjectMapper` with `ClassTagExtensions` to fix compilation warning

SparkQA commented on pull request #34532: URL: https://github.com/apache/spark/pull/34532#issuecomment-975051861 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49964/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34676: [WIP][SPARK-37434][BUILD] Add a new profile to auto disable unsupported UTs on MacOs using Apple Silicon

SparkQA commented on pull request #34676: URL: https://github.com/apache/spark/pull/34676#issuecomment-975049090 **[Test build #145494 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145494/testReport)** for PR 34676 at commit [`b8e917f`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins removed a comment on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975048158 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145491/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

AmplabJenkins removed a comment on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975048157 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49962/

[GitHub] [spark] AmplabJenkins commented on pull request #34668: [SPARK-37389][SQL] Check unclosed bracketed comments

AmplabJenkins commented on pull request #34668: URL: https://github.com/apache/spark/pull/34668#issuecomment-975048157 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49962/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

AmplabJenkins commented on pull request #34596: URL: https://github.com/apache/spark/pull/34596#issuecomment-975048158 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145491/ -- This

[GitHub] [spark] xuechendi opened a new pull request #34396: [SPARK-37124][SQL] Support RowToColumnarExec with Arrow format

xuechendi opened a new pull request #34396: URL: https://github.com/apache/spark/pull/34396 ### What changes were proposed in this pull request? This Jira is aim to support Arrow format in RowToColumnarExec. ### Why are the changes needed? Current ArrowColumnVector is not fully

  1   2   3   4   >