[GitHub] [spark] advancedxy commented on pull request #34623: [SPARK-37347][SQL] Spark Thrift Server (STS) driver fullFC becourse of timeoutExecutor not shutdown correctly

advancedxy commented on pull request #34623: URL: https://github.com/apache/spark/pull/34623#issuecomment-971326115 > But timeoutExecutor is not shutdown correctly when statement is finished Could you elaborate more on this? Also, is it possible for you to add a unit test for thi

[GitHub] [spark] AngersZhuuuu commented on pull request #34326: [SPARK-37053][CORE] Add metrics to SparkHistoryServer

AngersZh commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971326096 @thejdeep @HyukjinKwon Update pr desc and add unit test to show the result. Hope for your review and can suggest if I have miss some thing that can be added to metrics. -

[GitHub] [spark] SparkQA commented on pull request #34594: [SPARK-35345][SQL] Add Parquet tests to BloomFilterBenchmark

SparkQA commented on pull request #34594: URL: https://github.com/apache/spark/pull/34594#issuecomment-971325622 **[Test build #145306 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145306/testReport)** for PR 34594 at commit [`bac368c`](https://github.co

[GitHub] [spark] Peng-Lei commented on a change in pull request #34476: [SPARK-37195][SQL][TESTS] Unify v1 and v2 SHOW TBLPROPERTIES tests

Peng-Lei commented on a change in pull request #34476: URL: https://github.com/apache/spark/pull/34476#discussion_r750972162 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/ShowTblPropertiesSuiteBase.scala ## @@ -0,0 +1,90 @@ +/* + * Licensed to th

[GitHub] [spark] advancedxy commented on pull request #34602: [SPARK-37328][SQL] Fix bug that OptimizeSkewedJoin may not work after it was moved from queryStageOptimizerRules to queryStagePreparationR

advancedxy commented on pull request #34602: URL: https://github.com/apache/spark/pull/34602#issuecomment-971322018 > `OptimizeSkewedJoin` is supposed to only handle materialized shuffle stages, or did I miss something? Hi @cloud-fan, `OptimizeSkewedJoin` is supposed to handle materi

[GitHub] [spark] Peng-Lei commented on a change in pull request #34476: [SPARK-37195][SQL][TESTS] Unify v1 and v2 SHOW TBLPROPERTIES tests

Peng-Lei commented on a change in pull request #34476: URL: https://github.com/apache/spark/pull/34476#discussion_r750969056 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v1/ShowTblPropertiesSuite.scala ## @@ -0,0 +1,87 @@ +/* + * Licensed to the

[GitHub] [spark] sarutak commented on a change in pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

sarutak commented on a change in pull request #34607: URL: https://github.com/apache/spark/pull/34607#discussion_r750964537 ## File path: core/src/main/resources/org/apache/spark/ui/static/stagepage.js ## @@ -652,6 +652,38 @@ $(document).ready(function () { execut

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-971315377 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145301/ -

[GitHub] [spark] wangyum commented on pull request #34603: [SPARK-37702][SQL][FOLLOWUP] Store referred temp functions for CacheTableAsSelect

wangyum commented on pull request #34603: URL: https://github.com/apache/spark/pull/34603#issuecomment-971315741 @linhongliu-db t seems this PR makes branch-3.2 test fail. ``` [info] *** 1 TEST FAILED *** [error] Failed: Total 6479, Failed 1, Errors 0, Passed 6478, Ignored 5

[GitHub] [spark] AmplabJenkins commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

AmplabJenkins commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-971315377 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145301/ -- This

[GitHub] [spark] SparkQA removed a comment on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA removed a comment on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-971118667 **[Test build #145301 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145301/testReport)** for PR 34611 at commit [`c7699da`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34326: [WIP][SPARK-37053][CORE] Add metrics to SparkHistoryServer

SparkQA commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971315087 **[Test build #145323 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145323/testReport)** for PR 34326 at commit [`c8cb0fc`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

SparkQA commented on pull request #34494: URL: https://github.com/apache/spark/pull/34494#issuecomment-971314950 **[Test build #145322 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145322/testReport)** for PR 34494 at commit [`2ac9d11`](https://github.com

[GitHub] [spark] sarutak commented on a change in pull request #34607: [SPARK-36038][CORE] Speculation metrics summary at stage level

sarutak commented on a change in pull request #34607: URL: https://github.com/apache/spark/pull/34607#discussion_r750963340 ## File path: core/src/main/scala/org/apache/spark/status/AppStatusListener.scala ## @@ -600,6 +600,12 @@ private[spark] class AppStatusListener( liv

[GitHub] [spark] SparkQA commented on pull request #34611: [SPARK-35867][SQL] Enable vectorized read for VectorizedPlainValuesReader.readBooleans

SparkQA commented on pull request #34611: URL: https://github.com/apache/spark/pull/34611#issuecomment-971314231 **[Test build #145301 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145301/testReport)** for PR 34611 at commit [`c7699da`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

SparkQA commented on pull request #34494: URL: https://github.com/apache/spark/pull/34494#issuecomment-971312880 **[Test build #145321 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145321/testReport)** for PR 34494 at commit [`276673a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

SparkQA commented on pull request #34627: URL: https://github.com/apache/spark/pull/34627#issuecomment-971311745 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49789/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

AmplabJenkins removed a comment on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971310410 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49783/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

AmplabJenkins removed a comment on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971310406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/tar

AmplabJenkins removed a comment on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971310417 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49782/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34626: [SPARK-37352][PYTHON] Silence the index_col advice in to_spark() for internal usage.

AmplabJenkins removed a comment on pull request #34626: URL: https://github.com/apache/spark/pull/34626#issuecomment-971310415 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145319/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34580: [SPARK-37270][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

AmplabJenkins removed a comment on pull request #34580: URL: https://github.com/apache/spark/pull/34580#issuecomment-971310408 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145297/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

AmplabJenkins removed a comment on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971310412 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145300/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

AmplabJenkins removed a comment on pull request #34627: URL: https://github.com/apache/spark/pull/34627#issuecomment-971310405 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145318/ -

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r750959137 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -1755,6 +1850,10 @@ abstract class CastBase extends

[GitHub] [spark] AmplabJenkins commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scal

AmplabJenkins commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971310417 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49782/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

AmplabJenkins commented on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971310406 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [spark] AmplabJenkins commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

AmplabJenkins commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971310410 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49783/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34580: [SPARK-37270][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

AmplabJenkins commented on pull request #34580: URL: https://github.com/apache/spark/pull/34580#issuecomment-971310408 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145297/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34626: [SPARK-37352][PYTHON] Silence the index_col advice in to_spark() for internal usage.

AmplabJenkins commented on pull request #34626: URL: https://github.com/apache/spark/pull/34626#issuecomment-971310415 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145319/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

AmplabJenkins commented on pull request #34627: URL: https://github.com/apache/spark/pull/34627#issuecomment-971310405 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145318/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

AmplabJenkins commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971310412 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145300/ -- This

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r750957056 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -1580,6 +1647,59 @@ abstract class CastBase extends

[GitHub] [spark] SparkQA commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971308362 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49787/ -- This is an automated message from the Apache

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r750956969 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/Cast.scala ## @@ -589,6 +594,15 @@ abstract class CastBase extends U

[GitHub] [spark] SparkQA commented on pull request #34625: [SPARK-37281][PYTHON] Support DayTimeIntervalType in Py4J

SparkQA commented on pull request #34625: URL: https://github.com/apache/spark/pull/34625#issuecomment-971307498 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49786/ -- This is an automated message from the Apache

[GitHub] [spark] Yikun commented on pull request #34599: [SPARK-37331][K8S] Add the ability to create resources before driverPod creating

Yikun commented on pull request #34599: URL: https://github.com/apache/spark/pull/34599#issuecomment-971305980 Intergration test failed is unrelated, ready for review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use t

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971304927 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49788/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

SparkQA commented on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971304795 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49785/ -- This is an automated message from the A

[GitHub] [spark] SparkQA removed a comment on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA removed a comment on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971118692 **[Test build #145300 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145300/testReport)** for PR 34614 at commit [`d81327b`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971292044 **[Test build #145300 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145300/testReport)** for PR 34614 at commit [`d81327b`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

SparkQA removed a comment on pull request #34627: URL: https://github.com/apache/spark/pull/34627#issuecomment-971245799 **[Test build #145318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145318/testReport)** for PR 34627 at commit [`67e73e9`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34626: [SPARK-37352][PYTHON] Silence the index_col advice in to_spark() for internal usage.

SparkQA removed a comment on pull request #34626: URL: https://github.com/apache/spark/pull/34626#issuecomment-971245884 **[Test build #145319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145319/testReport)** for PR 34626 at commit [`41d5b54`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #34580: [SPARK-37270][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

SparkQA removed a comment on pull request #34580: URL: https://github.com/apache/spark/pull/34580#issuecomment-971041485 **[Test build #145297 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145297/testReport)** for PR 34580 at commit [`237f8a8`](https://gi

[GitHub] [spark] Peng-Lei commented on a change in pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

Peng-Lei commented on a change in pull request #34494: URL: https://github.com/apache/spark/pull/34494#discussion_r750947143 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/util/IntervalUtils.scala ## @@ -1253,4 +1255,115 @@ object IntervalUtils {

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971277649 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49782/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

SparkQA commented on pull request #34627: URL: https://github.com/apache/spark/pull/34627#issuecomment-971276632 **[Test build #145318 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145318/testReport)** for PR 34627 at commit [`67e73e9`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34626: [SPARK-37352][PYTHON] Silence the index_col advice in to_spark() for internal usage.

SparkQA commented on pull request #34626: URL: https://github.com/apache/spark/pull/34626#issuecomment-971276586 **[Test build #145319 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145319/testReport)** for PR 34626 at commit [`41d5b54`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

SparkQA commented on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971271253 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49784/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971270235 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49783/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34580: [SPARK-37270][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

SparkQA commented on pull request #34580: URL: https://github.com/apache/spark/pull/34580#issuecomment-971264615 **[Test build #145297 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145297/testReport)** for PR 34580 at commit [`237f8a8`](https://github.co

[GitHub] [spark] hgs19921112 closed pull request #34492: [SPARK-37216][SQL] Add the Hive macro functionality to SparkSQL

hgs19921112 closed pull request #34492: URL: https://github.com/apache/spark/pull/34492 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-un

[GitHub] [spark] itholic commented on a change in pull request #34401: [SPARK-30537][PYTHON], Fix toPandas wrong dtypes when applied on empty DF when Arrow enabled

itholic commented on a change in pull request #34401: URL: https://github.com/apache/spark/pull/34401#discussion_r750928501 ## File path: python/pyspark/sql/pandas/conversion.py ## @@ -171,7 +171,18 @@ def toPandas(self) -> "PandasDataFrameLike":

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/tar

AmplabJenkins removed a comment on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971255115 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49779/

[GitHub] [spark] AmplabJenkins commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scal

AmplabJenkins commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971255115 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49779/ -- T

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971255089 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49779/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971248760 **[Test build #145320 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145320/testReport)** for PR 33588 at commit [`3b70990`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34626: [SPARK-37352][PYTHON] Silence the index_col advice in to_spark() for internal usage.

SparkQA commented on pull request #34626: URL: https://github.com/apache/spark/pull/34626#issuecomment-971245884 **[Test build #145319 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145319/testReport)** for PR 34626 at commit [`41d5b54`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

SparkQA commented on pull request #34627: URL: https://github.com/apache/spark/pull/34627#issuecomment-971245799 **[Test build #145318 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145318/testReport)** for PR 34627 at commit [`67e73e9`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

AmplabJenkins removed a comment on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971242250 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49780/

[GitHub] [spark] AmplabJenkins commented on pull request #34603: [SPARK-37702][SQL][FOLLOWUP] Store referred temp functions for CacheTableAsSelect

AmplabJenkins commented on pull request #34603: URL: https://github.com/apache/spark/pull/34603#issuecomment-971243355 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145296/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

AmplabJenkins removed a comment on pull request #34494: URL: https://github.com/apache/spark/pull/34494#issuecomment-971242245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49781/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34326: [WIP][SPARK-37053][CORE] Add metrics to SparkHistoryServer

AmplabJenkins removed a comment on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971242247 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49778/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34603: [SPARK-37702][SQL][FOLLOWUP] Store referred temp functions for CacheTableAsSelect

AmplabJenkins removed a comment on pull request #34603: URL: https://github.com/apache/spark/pull/34603#issuecomment-971243355 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145296/ -

[GitHub] [spark] AmplabJenkins commented on pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

AmplabJenkins commented on pull request #34494: URL: https://github.com/apache/spark/pull/34494#issuecomment-971242245 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49781/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

AmplabJenkins commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971242250 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49780/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #34326: [WIP][SPARK-37053][CORE] Add metrics to SparkHistoryServer

AmplabJenkins commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971242247 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/49778/ -- T

[GitHub] [spark] SparkQA removed a comment on pull request #34603: [SPARK-37702][SQL][FOLLOWUP] Store referred temp functions for CacheTableAsSelect

SparkQA removed a comment on pull request #34603: URL: https://github.com/apache/spark/pull/34603#issuecomment-971041286 **[Test build #145296 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145296/testReport)** for PR 34603 at commit [`a213518`](https://gi

[GitHub] [spark] SparkQA commented on pull request #34603: [SPARK-37702][SQL][FOLLOWUP] Store referred temp functions for CacheTableAsSelect

SparkQA commented on pull request #34603: URL: https://github.com/apache/spark/pull/34603#issuecomment-971238512 **[Test build #145296 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145296/testReport)** for PR 34603 at commit [`a213518`](https://github.co

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32340: [SPARK-35139][SQL] Support ANSI intervals as Arrow Column vectors

HyukjinKwon commented on a change in pull request #32340: URL: https://github.com/apache/spark/pull/32340#discussion_r750915520 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/arrow/ArrowWriter.scala ## @@ -394,3 +397,28 @@ private[arrow] class NullWriter(

[GitHub] [spark] wangyum commented on pull request #34580: [SPARK-37270][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

wangyum commented on pull request #34580: URL: https://github.com/apache/spark/pull/34580#issuecomment-971229509 OK. PR for 3.2: https://github.com/apache/spark/pull/34627 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and u

[GitHub] [spark] wangyum opened a new pull request #34627: [SPARK-37270][3.2][SQL] Fix push foldable into CaseWhen branches if elseValue is empty

wangyum opened a new pull request #34627: URL: https://github.com/apache/spark/pull/34627 Backport #34580 ### What changes were proposed in this pull request? This pr fix push foldable into CaseWhen branches if elseValue is empty. For example: ```scala spark.sql("CREATE

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971227202 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49782/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

SparkQA commented on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971225277 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49785/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

SparkQA commented on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971225174 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49784/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971224992 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49783/ -- This is an automated message from the Apache

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32340: [SPARK-35139][SQL] Support ANSI intervals as Arrow Column vectors

HyukjinKwon commented on a change in pull request #32340: URL: https://github.com/apache/spark/pull/32340#discussion_r750909587 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java ## @@ -172,6 +176,10 @@ public ArrowColumnVector(Valu

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32340: [SPARK-35139][SQL] Support ANSI intervals as Arrow Column vectors

HyukjinKwon commented on a change in pull request #32340: URL: https://github.com/apache/spark/pull/32340#discussion_r750909587 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java ## @@ -172,6 +176,10 @@ public ArrowColumnVector(Valu

[GitHub] [spark] HyukjinKwon commented on a change in pull request #32340: [SPARK-35139][SQL] Support ANSI intervals as Arrow Column vectors

HyukjinKwon commented on a change in pull request #32340: URL: https://github.com/apache/spark/pull/32340#discussion_r750909587 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/vectorized/ArrowColumnVector.java ## @@ -172,6 +176,10 @@ public ArrowColumnVector(Valu

[GitHub] [spark] itholic opened a new pull request #34626: [SPARK-37352] Silence the index_col advice in to_spark() for internal usage.

itholic opened a new pull request #34626: URL: https://github.com/apache/spark/pull/34626 ### What changes were proposed in this pull request? This PR proposes adding internal method `_to_spark()` to silence the advice log when it's used as an internal purpose. ### Why are the

[GitHub] [spark] SparkQA commented on pull request #34326: [WIP][SPARK-37053][CORE] Add metrics to SparkHistoryServer

SparkQA commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971221777 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49778/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34494: [SPARK-36924][SQL] CAST between ANSI intervals and IntegralType

SparkQA commented on pull request #34494: URL: https://github.com/apache/spark/pull/34494#issuecomment-971219289 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49781/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971218494 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49780/ -- This is an automated message from the A

[GitHub] [spark] SparkQA commented on pull request #33588: [SPARK-36346][SQL] Support TimestampNTZ type in Orc file source

SparkQA commented on pull request #33588: URL: https://github.com/apache/spark/pull/33588#issuecomment-971212730 **[Test build #145317 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145317/testReport)** for PR 33588 at commit [`51d7651`](https://github.com

[GitHub] [spark] wankunde edited a comment on pull request #34536: [SPARK-35011][CORE] Fix false active executor in UI that caused by BlockManager reregistration

wankunde edited a comment on pull request #34536: URL: https://github.com/apache/spark/pull/34536#issuecomment-971211171 > For such registered `BlockManager`s, fortunately, we have `HeartbeatReceiver.expireDeadHosts` to remove them in the end, which fires a `SparkListenerBlockManagerRemove

[GitHub] [spark] wankunde commented on pull request #34536: [SPARK-35011][CORE] Fix false active executor in UI that caused by BlockManager reregistration

wankunde commented on pull request #34536: URL: https://github.com/apache/spark/pull/34536#issuecomment-971211171 @Ngone51 > For such registered `BlockManager`s, fortunately, we have `HeartbeatReceiver.expireDeadHosts` to remove them in the end, which fires a `SparkListenerBloc

[GitHub] [spark] SparkQA commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971210761 **[Test build #145316 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145316/testReport)** for PR 34614 at commit [`2126ebe`](https://github.com

[GitHub] [spark] HyukjinKwon commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

HyukjinKwon commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971210334 retest this please -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] cloud-fan commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

cloud-fan commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r750896342 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala ## @@ -38,6 +39,13 @@ class CSVInferSchema(val options

[GitHub] [spark] cloud-fan commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

cloud-fan commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r750896030 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala ## @@ -38,6 +39,13 @@ class CSVInferSchema(val options

[GitHub] [spark] cloud-fan commented on a change in pull request #34596: [SPARK-37326][SQL] Support TimestampNTZ in CSV data source

cloud-fan commented on a change in pull request #34596: URL: https://github.com/apache/spark/pull/34596#discussion_r750895685 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/csv/CSVInferSchema.scala ## @@ -160,6 +169,15 @@ class CSVInferSchema(val optio

[GitHub] [spark] SparkQA commented on pull request #34326: [WIP][SPARK-37053][CORE] Add metrics to SparkHistoryServer

SparkQA commented on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971205263 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49778/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34620: [SPARK-37209][YARN][TESTS] Fix `YarnShuffleIntegrationSuite` releated UTs when using `hadoop-3.2` profile without `assembly/target/scala-%s/j

SparkQA commented on pull request #34620: URL: https://github.com/apache/spark/pull/34620#issuecomment-971207125 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/49779/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #34625: [SPARK-37281][PYTHON] Support DayTimeIntervalType in Py4J

SparkQA commented on pull request #34625: URL: https://github.com/apache/spark/pull/34625#issuecomment-971206141 **[Test build #145315 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145315/testReport)** for PR 34625 at commit [`2b9298b`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

AmplabJenkins removed a comment on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971205941 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145309/ -

[GitHub] [spark] SparkQA removed a comment on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA removed a comment on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971143004 **[Test build #145309 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145309/testReport)** for PR 34614 at commit [`2126ebe`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

AmplabJenkins commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971205941 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145309/ -- This

[GitHub] [spark] SparkQA commented on pull request #34614: [SPARK-37279][PYTHON][SQL] Support DayTimeIntervalType in createDataFrame, collect and Python UDF

SparkQA commented on pull request #34614: URL: https://github.com/apache/spark/pull/34614#issuecomment-971205818 **[Test build #145309 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/145309/testReport)** for PR 34614 at commit [`2126ebe`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34624: [SPARK-37230][DOCS][PYTHON] Document DataFrame.mapInArrow in PySpark documentation

AmplabJenkins removed a comment on pull request #34624: URL: https://github.com/apache/spark/pull/34624#issuecomment-971204986 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/145314/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #34326: [WIP][SPARK-37053][CORE] Add metrics to SparkHistoryServer

AmplabJenkins removed a comment on pull request #34326: URL: https://github.com/apache/spark/pull/34326#issuecomment-971204980 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

  1   2   3   4   5   6   7   >