[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA removed a comment on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774584708 **[Test build #134973 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134973/testReport)** for PR 31378 at commit

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774630332 **[Test build #134973 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134973/testReport)** for PR 31378 at commit

[GitHub] [spark] SparkQA commented on pull request #31504: [SPARK-34172][SQL] Add `SHOW DATABASES` as table-valued function

SparkQA commented on pull request #31504: URL: https://github.com/apache/spark/pull/31504#issuecomment-774630305 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39563/

[GitHub] [spark] dongjoon-hyun closed pull request #31502: [SPARK-34390][CORE] Enable Zstandard buffer pool by default

dongjoon-hyun closed pull request #31502: URL: https://github.com/apache/spark/pull/31502 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #31502: [SPARK-34390][CORE] Enable Zstandard buffer pool by default

dongjoon-hyun commented on pull request #31502: URL: https://github.com/apache/spark/pull/31502#issuecomment-774629886 Thank you, @HyukjinKwon . Merged to master. If there is an issue, I'll document it and switch the default before 3.2.0 release.

[GitHub] [spark] AngersZhuuuu opened a new pull request #31506: [SPARK-34171][SQL] Add 'SHOW COLUMNS' as table-valued function

AngersZh opened a new pull request #31506: URL: https://github.com/apache/spark/pull/31506 ### What changes were proposed in this pull request? Add `SHOW COLUMNS` as table-valued function 'show_columns' In origin, we can't directly analysis column metadata information in SQL,

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774629859 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39565/

[GitHub] [spark] dongjoon-hyun commented on pull request #31502: [SPARK-34390][CORE] Enable Zstandard buffer pool by default

dongjoon-hyun commented on pull request #31502: URL: https://github.com/apache/spark/pull/31502#issuecomment-774629495 Ya, it might not be a general case. For Spark side, I'm testing this with Spark shuffle io, eventlog compression and Parquet read/write. This will be tested in the

[GitHub] [spark] SparkQA commented on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

SparkQA commented on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774628552 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39560/

[GitHub] [spark] AngersZhuuuu opened a new pull request #31505: [SPARK-34174][SQL] Add 'SHOW PARTITIONS' as table-valued function

AngersZh opened a new pull request #31505: URL: https://github.com/apache/spark/pull/31505 ### What changes were proposed in this pull request? Add `SHOW PARTITIONS` as table-valued function 'show_partitions' In origin, we can't directly analysis partition metadata information

[GitHub] [spark] SparkQA removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-774583356 **[Test build #134972 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134972/testReport)** for PR 31245 at commit

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-774627571 **[Test build #134972 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134972/testReport)** for PR 31245 at commit

[GitHub] [spark] SparkQA commented on pull request #31504: [SPARK-34172][SQL] Add `SHOW DATABASES` as table-valued function

SparkQA commented on pull request #31504: URL: https://github.com/apache/spark/pull/31504#issuecomment-774627288 **[Test build #134980 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134980/testReport)** for PR 31504 at commit

[GitHub] [spark] SparkQA commented on pull request #31499: [WIP][SPARK-31891][SQL] Support MSCK REPAIR TABLE .. (ADD|DROP|SYNC) PARTITIONS

SparkQA commented on pull request #31499: URL: https://github.com/apache/spark/pull/31499#issuecomment-774626893 **[Test build #134981 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134981/testReport)** for PR 31499 at commit

[GitHub] [spark] SparkQA commented on pull request #31504: [SPARK-34172][SQL] Add `SHOW DATABASES` as table-valued function

SparkQA commented on pull request #31504: URL: https://github.com/apache/spark/pull/31504#issuecomment-774626174 **[Test build #134983 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134983/testReport)** for PR 31504 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774626019 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39562/

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774626019 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39562/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

AmplabJenkins removed a comment on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774619556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134974/

[GitHub] [spark] SparkQA removed a comment on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

SparkQA removed a comment on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774592852 **[Test build #134974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134974/testReport)** for PR 31503 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774619558 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39561/

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774620850 **[Test build #134982 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134982/testReport)** for PR 31337 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

AmplabJenkins commented on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774619556 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134974/

[GitHub] [spark] AmplabJenkins commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774619558 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39561/

[GitHub] [spark] SparkQA commented on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

SparkQA commented on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774616630 **[Test build #134974 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134974/testReport)** for PR 31503 at commit

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774615619 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39561/

[GitHub] [spark] AngersZhuuuu opened a new pull request #31504: [SPARK-34172][SQL] Add `SHOW DATABASES` as table-valued function

AngersZh opened a new pull request #31504: URL: https://github.com/apache/spark/pull/31504 ### What changes were proposed in this pull request? Add show tables as Table-valued function make user convenient to analysis db information in SQL ### Why are the changes needed?

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774614418 **[Test build #134979 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134979/testReport)** for PR 31337 at commit

[GitHub] [spark] AngersZhuuuu edited a comment on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

AngersZh edited a comment on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-774613080 @cloud-fan Should we add ``` object ShowPartitions { val OUTPUT = xxx } ``` to keep consistence?

[GitHub] [spark] AngersZhuuuu commented on pull request #31341: [SPARK-34238][SQL] Unify output of SHOW PARTITIONS and pass output attributes properly

AngersZh commented on pull request #31341: URL: https://github.com/apache/spark/pull/31341#issuecomment-774613080 @cloud-fan Should we add ``` object ShowColumns { val OUTPUT = xxx } ``` to keep consistence?

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

AmplabJenkins removed a comment on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774610475 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39557/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774610473 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774610474 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

AmplabJenkins commented on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774610475 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39557/

[GitHub] [spark] SparkQA commented on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

SparkQA commented on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774610465 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39560/

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774610085 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39561/

[GitHub] [spark] beliefer commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

beliefer commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774607653 retest this please This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774607401 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39559/

[GitHub] [spark] HyukjinKwon commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

HyukjinKwon commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774606427 cc @tgravescs and @mridulm too FYI This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774592094 **[Test build #134976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134976/testReport)** for PR 31337 at commit

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774603745 **[Test build #134976 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134976/testReport)** for PR 31337 at commit

[GitHub] [spark] HyukjinKwon commented on pull request #31502: [SPARK-34390][CORE] Enable Zstandard buffer pool by default

HyukjinKwon commented on pull request #31502: URL: https://github.com/apache/spark/pull/31502#issuecomment-774603017 I read https://github.com/luben/zstd-jni/commit/f7c8279bc162c8c8b1964948d0f3b309ad715311 and I wonder what are the "some workloads". But I think it's fine to give a try of

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774602899 **[Test build #134978 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134978/testReport)** for PR 31485 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774602619 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39558/

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774602612 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39558/

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774602619 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39558/

[GitHub] [spark] SparkQA commented on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

SparkQA commented on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774602475 **[Test build #134977 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134977/testReport)** for PR 31500 at commit

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774601422 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39559/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31502: [SPARK-34390][CORE] Enable Zstandard buffer pool by default

AmplabJenkins removed a comment on pull request #31502: URL: https://github.com/apache/spark/pull/31502#issuecomment-774598900 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins removed a comment on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-774598905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39555/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

AmplabJenkins removed a comment on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-774598906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134967/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

AmplabJenkins removed a comment on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774594283 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins removed a comment on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-774598903 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

AmplabJenkins commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-774598905 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39555/

[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

AmplabJenkins commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774598901 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39556/

[GitHub] [spark] AmplabJenkins commented on pull request #31502: [SPARK-34390][CORE] Enable Zstandard buffer pool by default

AmplabJenkins commented on pull request #31502: URL: https://github.com/apache/spark/pull/31502#issuecomment-774598900 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

AmplabJenkins commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-774598906 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134967/

[GitHub] [spark] AmplabJenkins commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

AmplabJenkins commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-774598904 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

AngersZh commented on a change in pull request #31471: URL: https://github.com/apache/spark/pull/31471#discussion_r571531720 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala ## @@ -217,8 +217,9 @@ object

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AngersZh commented on a change in pull request #31485: URL: https://github.com/apache/spark/pull/31485#discussion_r571540424 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlanVisitor.scala ## @@ -47,6 +49,15 @@ trait

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read

HyukjinKwon commented on a change in pull request #31489: URL: https://github.com/apache/spark/pull/31489#discussion_r571539866 ## File path: sql/core/src/main/scala/org/apache/spark/sql/DataFrameReader.scala ## @@ -825,6 +825,26 @@ class DataFrameReader

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

HyukjinKwon commented on a change in pull request #31485: URL: https://github.com/apache/spark/pull/31485#discussion_r571539632 ## File path: sql/core/src/test/scala/org/apache/spark/sql/StatisticsCollectionSuite.scala ## @@ -678,4 +680,50 @@ class StatisticsCollectionSuite

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

HyukjinKwon commented on a change in pull request #31485: URL: https://github.com/apache/spark/pull/31485#discussion_r571539592 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlanVisitor.scala ## @@ -47,6 +49,15 @@ trait

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774597138 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39558/

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31483: [SPARK-33434][PYTHON][DOCS] Added RuntimeConfig to PySpark docs

HyukjinKwon commented on a change in pull request #31483: URL: https://github.com/apache/spark/pull/31483#discussion_r571539164 ## File path: python/docs/source/reference/pyspark.sql.rst ## @@ -72,6 +72,11 @@ See also :class:`SparkSession`. SparkSession.udf

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-774596006 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39555/

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774595346 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39556/

[GitHub] [spark] viirya commented on a change in pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

viirya commented on a change in pull request #31476: URL: https://github.com/apache/spark/pull/31476#discussion_r571537663 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/Scan.java ## @@ -102,4 +102,13 @@ default MicroBatchStream

[GitHub] [spark] SparkQA commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

SparkQA commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-774595114 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39552/

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774594235 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39553/

[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

AmplabJenkins commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774594283 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39553/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

AmplabJenkins removed a comment on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-774593405 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39554/

[GitHub] [spark] AmplabJenkins commented on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

AmplabJenkins commented on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-774593405 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39554/

[GitHub] [spark] SparkQA commented on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

SparkQA commented on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-774593345 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39554/

[GitHub] [spark] SparkQA commented on pull request #31503: [SPARK-34391][BUILD] Upgrade commons-io to 2.8.0

SparkQA commented on pull request #31503: URL: https://github.com/apache/spark/pull/31503#issuecomment-774592852 **[Test build #134974 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134974/testReport)** for PR 31503 at commit

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774592094 **[Test build #134976 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134976/testReport)** for PR 31337 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

AmplabJenkins removed a comment on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774590784 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134963/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

AmplabJenkins removed a comment on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-774590783 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39551/

[GitHub] [spark] SparkQA removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774590876 **[Test build #134975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134975/testReport)** for PR 31337 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774591113 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134975/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774590782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39549/

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774591113 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134975/

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774591077 **[Test build #134975 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134975/testReport)** for PR 31337 at commit

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-774590876 **[Test build #134975 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134975/testReport)** for PR 31337 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

AmplabJenkins commented on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774590784 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134963/

[GitHub] [spark] AmplabJenkins commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

AmplabJenkins commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-774590783 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39551/

[GitHub] [spark] AmplabJenkins commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-774590782 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39549/

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774590677 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39553/

[GitHub] [spark] SparkQA commented on pull request #31245: [SPARK-34157][SQL] Unify output of SHOW TABLES and pass output attributes properly

SparkQA commented on pull request #31245: URL: https://github.com/apache/spark/pull/31245#issuecomment-774590444 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39555/

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31471: [SPARK-34355][SQL] Add log and time cost for commit job

AngersZh commented on a change in pull request #31471: URL: https://github.com/apache/spark/pull/31471#discussion_r571531720 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/FileFormatWriter.scala ## @@ -217,8 +217,9 @@ object

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r571531722 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala ## @@ -46,6 +47,23 @@ import

[GitHub] [spark] SparkQA commented on pull request #31480: [SPARK-32384][CORE] repartitionAndSortWithinPartitions avoid shuffle with same partitioner

SparkQA commented on pull request #31480: URL: https://github.com/apache/spark/pull/31480#issuecomment-774590110 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39552/

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-774590085 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39556/

[GitHub] [spark] SparkQA commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

SparkQA commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-774589737 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39551/

[GitHub] [spark] SparkQA removed a comment on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

SparkQA removed a comment on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774559040 **[Test build #134963 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134963/testReport)** for PR 31500 at commit

[GitHub] [spark] SparkQA commented on pull request #31500: [SPARK-34388][SQL] Propagate the registered UDF name to ScalaUDF, ScalaUDAF and ScalaAggregator

SparkQA commented on pull request #31500: URL: https://github.com/apache/spark/pull/31500#issuecomment-774589680 **[Test build #134963 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134963/testReport)** for PR 31500 at commit

[GitHub] [spark] SparkQA commented on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

SparkQA commented on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-774589600 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39554/

[GitHub] [spark] SparkQA commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

SparkQA commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-774589368 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39551/

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31455: [SPARK-34342][SQL] Format DateLiteral and TimestampLiteral toString

dongjoon-hyun edited a comment on pull request #31455: URL: https://github.com/apache/spark/pull/31455#issuecomment-774588931 @MaxGekk . Could you file a new JIRA and make a PR for your suggestion? We can check the behavior change of BINARY/MAP in that PR independently.

[GitHub] [spark] dongjoon-hyun commented on pull request #31455: [SPARK-34342][SQL] Format DateLiteral and TimestampLiteral toString

dongjoon-hyun commented on pull request #31455: URL: https://github.com/apache/spark/pull/31455#issuecomment-774588931 @MaxGekk . Could you file a new JIRA and make a PR for your suggestion? We can check the behavior change there independently.

[GitHub] [spark] dongjoon-hyun closed pull request #31455: [SPARK-34342][SQL] Format DateLiteral and TimestampLiteral toString

dongjoon-hyun closed pull request #31455: URL: https://github.com/apache/spark/pull/31455 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

  1   2   3   4   >