[GitHub] [spark] SparkQA commented on pull request #34622: [SPARK-37340][UI] Display StageIds in Operators for SQL UI

SparkQA commented on pull request #34622: URL: https://github.com/apache/spark/pull/34622#issuecomment-982266377 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50214/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

SparkQA commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982266648 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50208/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982267744 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50211/ -- This is an automated message from the Apache

[GitHub] [spark] LuciferYang commented on a change in pull request #34749: [SPARK-37493][CORE] show driver's gc time and duration time in executors page

LuciferYang commented on a change in pull request #34749: URL: https://github.com/apache/spark/pull/34749#discussion_r758917691 ## File path: core/src/main/scala/org/apache/spark/metrics/ExecutorMetricType.scala ## @@ -137,7 +138,9 @@ case object GarbageCollectionMetrics extend

[GitHub] [spark] LuciferYang commented on a change in pull request #34749: [SPARK-37493][CORE] show driver's gc time and duration time in executors page

LuciferYang commented on a change in pull request #34749: URL: https://github.com/apache/spark/pull/34749#discussion_r758918215 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusStore.scala ## @@ -89,7 +89,44 @@ private[spark] class AppStatusStore( } else

[GitHub] [spark] SparkQA commented on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

SparkQA commented on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982269964 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50212/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982270701 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50215/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

AmplabJenkins commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982271491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145744/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

AmplabJenkins commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982271490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50209/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

AmplabJenkins commented on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982271495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50208/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982271493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145741/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982271492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

AmplabJenkins removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982271490 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50209/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

AmplabJenkins removed a comment on pull request #34723: URL: https://github.com/apache/spark/pull/34723#issuecomment-982271495 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50208/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982271492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982271493 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145741/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

AmplabJenkins removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982271491 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145744/ -

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982272552 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50213/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982272743 **[Test build #145746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145746/testReport)** for PR 34746 at commit [`55bfce8`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982272845 **[Test build #145747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145747/testReport)** for PR 34657 at commit [`37d2ff4`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

SparkQA commented on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982273172 **[Test build #145748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145748/testReport)** for PR 34213 at commit [`dc20b38`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

AmplabJenkins commented on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982273957 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50210/ -- T

[GitHub] [spark] SparkQA commented on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

SparkQA commented on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982273940 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50210/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34568: [SPARK-37287][SQL] Pull out dynamic partition and bucket sort from FileFormatWriter

AmplabJenkins removed a comment on pull request #34568: URL: https://github.com/apache/spark/pull/34568#issuecomment-982273957 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50210/

[GitHub] [spark] dongjoon-hyun closed pull request #34736: [SPARK-37452][SQL][3.1] Char and Varchar break backward compatibility between v3.1 and v2

dongjoon-hyun closed pull request #34736: URL: https://github.com/apache/spark/pull/34736 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] SparkQA commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

SparkQA commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982275747 **[Test build #145739 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145739/testReport)** for PR 34578 at commit [`c30d443`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

SparkQA removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982219830 **[Test build #145739 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145739/testReport)** for PR 34578 at commit [`c30d443`](https://gi

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758922930 ## File path: docs/running-on-kubernetes.md ## @@ -591,7 +591,7 @@ See the [configuration page](configuration.html) for information on Spark config

[GitHub] [spark] AmplabJenkins commented on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

AmplabJenkins commented on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982276249 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145739/ -- This

[GitHub] [spark] cloud-fan commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

cloud-fan commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758923158 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override val ca

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34578: [SPARK-37300][CORE] TaskSchedulerImpl should ignore task finished eve…

AmplabJenkins removed a comment on pull request #34578: URL: https://github.com/apache/spark/pull/34578#issuecomment-982276249 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145739/ -

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758924421 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] cloud-fan commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

cloud-fan commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758924491 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override val ca

[GitHub] [spark] entong commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

entong commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758924799 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override val catal

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758924807 ## File path: docs/running-on-kubernetes.md ## @@ -1278,6 +1295,14 @@ See the [configuration page](configuration.html) for information on Spark con

[GitHub] [spark] cloud-fan commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

cloud-fan commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758924914 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override val ca

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925060 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925122 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925174 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925346 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925368 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925510 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758925884 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758926045 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758926255 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758926274 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34734: [SPARK-37480][K8S][DOC] Sync Kubernetes configuration to latest in running-on-k8s.md

dongjoon-hyun commented on a change in pull request #34734: URL: https://github.com/apache/spark/pull/34734#discussion_r758926832 ## File path: docs/running-on-kubernetes.md ## @@ -1322,6 +1347,144 @@ See the [configuration page](configuration.html) for information on Spark co

[GitHub] [spark] cloud-fan commented on pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

cloud-fan commented on pull request #34060: URL: https://github.com/apache/spark/pull/34060#issuecomment-982282199 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan closed pull request #34060: [SPARK-36850][SQL] Migrate CreateTableStatement to v2 command framework

cloud-fan closed pull request #34060: URL: https://github.com/apache/spark/pull/34060 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsu

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758929218 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override va

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

dongjoon-hyun commented on a change in pull request #34723: URL: https://github.com/apache/spark/pull/34723#discussion_r758929249 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcQuerySuite.scala ## @@ -288,10 +288,9 @@ class HiveOrcQuerySuite extends

[GitHub] [spark] cloud-fan commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

cloud-fan commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r758929705 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownLimit.java ## @@ -18,6 +18,7 @@ package org.apache.spark.

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

dongjoon-hyun commented on a change in pull request #34723: URL: https://github.com/apache/spark/pull/34723#discussion_r758929816 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcQuerySuite.scala ## @@ -288,10 +288,9 @@ class HiveOrcQuerySuite extends

[GitHub] [spark] cloud-fan commented on a change in pull request #34738: [SPARK-37483][SQL] Support push down top N to JDBC data source V2

cloud-fan commented on a change in pull request #34738: URL: https://github.com/apache/spark/pull/34738#discussion_r75892 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/SupportsPushDownLimit.java ## @@ -33,4 +34,9 @@ * Pushes down LIMIT to

[GitHub] [spark] dongjoon-hyun closed pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

dongjoon-hyun closed pull request #34723: URL: https://github.com/apache/spark/pull/34723 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-

[GitHub] [spark] SparkQA commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982284411 **[Test build #145747 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145747/testReport)** for PR 34657 at commit [`37d2ff4`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #34701: [SPARK-37450][SQL] Prune unnecessary fiels from Generate under count-only Aggregate

cloud-fan commented on a change in pull request #34701: URL: https://github.com/apache/spark/pull/34701#discussion_r758930961 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -2162,6 +2163,74 @@ object RemoveLiteralFromGro

[GitHub] [spark] SparkQA commented on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

SparkQA commented on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982285733 **[Test build #145748 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145748/testReport)** for PR 34213 at commit [`dc20b38`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982285782 **[Test build #145746 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145746/testReport)** for PR 34746 at commit [`55bfce8`](https://github.co

[GitHub] [spark] dongjoon-hyun commented on pull request #34635: [SPARK-37205][YARN] Introduce a new config 'spark.yarn.am.tokenConfRegex' to support renewing delegation tokens in a multi-cluster envi

dongjoon-hyun commented on pull request #34635: URL: https://github.com/apache/spark/pull/34635#issuecomment-982285975 Could you review this once more please, @mridulm ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

[GitHub] [spark] cloud-fan commented on a change in pull request #34684: [SPARK-37442][SQL] InMemoryRelation statistics bug causing broadcast join failures with AQE enabled

cloud-fan commented on a change in pull request #34684: URL: https://github.com/apache/spark/pull/34684#discussion_r758931580 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala ## @@ -237,7 +238,23 @@ case class CachedRDDBuilde

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982286223 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50211/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

SparkQA removed a comment on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982272845 **[Test build #145747 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145747/testReport)** for PR 34657 at commit [`37d2ff4`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982272743 **[Test build #145746 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145746/testReport)** for PR 34746 at commit [`55bfce8`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

SparkQA removed a comment on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982273172 **[Test build #145748 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145748/testReport)** for PR 34213 at commit [`dc20b38`](https://gi

[GitHub] [spark] dongjoon-hyun commented on pull request #34656: [SPARK-37376][SQL] Introduce a new DataSource V2 interface HasPartitionKey

dongjoon-hyun commented on pull request #34656: URL: https://github.com/apache/spark/pull/34656#issuecomment-982286689 Could you resolve the conflicts, @sunchao ? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [spark] cloud-fan commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

cloud-fan commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758932749 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override val ca

[GitHub] [spark] dongjoon-hyun commented on pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on pull request #34264: URL: https://github.com/apache/spark/pull/34264#issuecomment-982288413 Sorry for being late, @holdenk . I'll take a look at this now. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Gi

[GitHub] [spark] SparkQA commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

SparkQA commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982288646 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50215/ -- This is an automated message from the A

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758934740 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -395,6 +395,23 @@ private[spark] obje

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982291656 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50216/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34622: [SPARK-37340][UI] Display StageIds in Operators for SQL UI

SparkQA commented on pull request #34622: URL: https://github.com/apache/spark/pull/34622#issuecomment-982291693 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50214/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

SparkQA commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982291748 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50213/ -- This is an automated message from the A

[GitHub] [spark] HyukjinKwon commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

HyukjinKwon commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982292481 Merged to master. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] HyukjinKwon closed pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

HyukjinKwon closed pull request #34717: URL: https://github.com/apache/spark/pull/34717 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] SparkQA commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

SparkQA commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982293900 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50217/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758937355 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsWatchSnapshotSource.scala ## @@ -

[GitHub] [spark] SparkQA commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

SparkQA commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982294398 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50218/ -- This is an automated message from the Apache

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758938108 ## File path: resource-managers/kubernetes/core/src/test/scala/org/apache/spark/scheduler/cluster/k8s/ExecutorPodsPollingSnapshotSourceSuite.scala

[GitHub] [spark] SparkQA commented on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

SparkQA commented on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982295028 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/50212/ -- This is an automated message from the A

[GitHub] [spark] gengliangwang commented on a change in pull request #34747: [SPARK-37490][SQL] Show extra hint if analyzer fails due to ANSI type coercion

gengliangwang commented on a change in pull request #34747: URL: https://github.com/apache/spark/pull/34747#discussion_r758938379 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -198,21 +205,39 @@ class Analyzer(override va

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758938725 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -395,6 +395,23 @@ private[spark] obje

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #34264: [SPARK-36462][K8S] Add the ability to selectively disable watching or polling

dongjoon-hyun commented on a change in pull request #34264: URL: https://github.com/apache/spark/pull/34264#discussion_r758938817 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -395,6 +395,23 @@ private[spark] obje

[GitHub] [spark] AmplabJenkins commented on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins commented on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982295990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

AmplabJenkins commented on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982295991 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145748/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34622: [SPARK-37340][UI] Display StageIds in Operators for SQL UI

AmplabJenkins commented on pull request #34622: URL: https://github.com/apache/spark/pull/34622#issuecomment-982295998 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50214/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

AmplabJenkins commented on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982295992 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50212/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins commented on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982295999 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50213/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

AmplabJenkins commented on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982295994 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145747/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

AmplabJenkins commented on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982295989 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50215/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34657: [WIP] Support TimedeltaIndex in pandas API on Spark

AmplabJenkins removed a comment on pull request #34657: URL: https://github.com/apache/spark/pull/34657#issuecomment-982295994 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145747/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

AmplabJenkins removed a comment on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982295991 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145748/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34731: [SPARK-37153][PYTHON] Inline type hints for python/pyspark/profiler.py

AmplabJenkins removed a comment on pull request #34731: URL: https://github.com/apache/spark/pull/34731#issuecomment-982295999 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50213/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34717: [SPARK-37465][PYTHON] Bump minimum pandas version to 1.0.5

AmplabJenkins removed a comment on pull request #34717: URL: https://github.com/apache/spark/pull/34717#issuecomment-982295989 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50215/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34746: [SPARK-37489][PYTHON] Skip hasnans check in numops if eager_check disable

AmplabJenkins removed a comment on pull request #34746: URL: https://github.com/apache/spark/pull/34746#issuecomment-982295990 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34622: [SPARK-37340][UI] Display StageIds in Operators for SQL UI

AmplabJenkins removed a comment on pull request #34622: URL: https://github.com/apache/spark/pull/34622#issuecomment-982295998 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50214/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34732: [SPARK-37291][PYSPARK][FOLLOWUP] PySpark create SparkSession should pass initialSessionOptions

AmplabJenkins removed a comment on pull request #34732: URL: https://github.com/apache/spark/pull/34732#issuecomment-982295992 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/50212/

[GitHub] [spark] dchvn commented on pull request #34213: [SPARK-36396][PYTHON] Implement DataFrame.cov

dchvn commented on pull request #34213: URL: https://github.com/apache/spark/pull/34213#issuecomment-982303245 Ping @HyukjinKwon Thanks :D -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [spark] ChenMichael commented on a change in pull request #34684: [SPARK-37442][SQL] InMemoryRelation statistics bug causing broadcast join failures with AQE enabled

ChenMichael commented on a change in pull request #34684: URL: https://github.com/apache/spark/pull/34684#discussion_r758950802 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/columnar/InMemoryRelation.scala ## @@ -237,7 +238,23 @@ case class CachedRDDBuil

[GitHub] [spark] beliefer commented on a change in pull request #34723: [SPARK-37492][SQL] Optimize Orc test code with withAllNativeOrcReaders

beliefer commented on a change in pull request #34723: URL: https://github.com/apache/spark/pull/34723#discussion_r758951813 ## File path: sql/hive/src/test/scala/org/apache/spark/sql/hive/orc/HiveOrcQuerySuite.scala ## @@ -288,10 +288,9 @@ class HiveOrcQuerySuite extends OrcQ

<    2   3   4   5   6   7   8   9   >