[GitHub] [spark] imback82 commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

imback82 commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768544554 Thanks all! I'll close this PR and check the other one. This is an automated message from the Apache Git Servic

[GitHub] [spark] imback82 closed pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

imback82 closed pull request #31352: URL: https://github.com/apache/spark/pull/31352 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] SparkQA removed a comment on pull request #31366: [SPARK-34267][SQL] Remove `refreshTable()` from `SessionState`

SparkQA removed a comment on pull request #31366: URL: https://github.com/apache/spark/pull/31366#issuecomment-768372607 **[Test build #134556 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134556/testReport)** for PR 31366 at commit [`3f57b7c`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31366: [SPARK-34267][SQL] Remove `refreshTable()` from `SessionState`

SparkQA commented on pull request #31366: URL: https://github.com/apache/spark/pull/31366#issuecomment-768541762 **[Test build #134556 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134556/testReport)** for PR 31366 at commit [`3f57b7c`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31362: [SPARK-34263][SQL] Simplify the code for treating unicode/octal/escaped characters in string literals

SparkQA removed a comment on pull request #31362: URL: https://github.com/apache/spark/pull/31362#issuecomment-768367359 **[Test build #134557 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134557/testReport)** for PR 31362 at commit [`82b8fb9`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31362: [SPARK-34263][SQL] Simplify the code for treating unicode/octal/escaped characters in string literals

SparkQA commented on pull request #31362: URL: https://github.com/apache/spark/pull/31362#issuecomment-768540786 **[Test build #134557 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134557/testReport)** for PR 31362 at commit [`82b8fb9`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-768538356 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39149/ ---

[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

SparkQA commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768536603 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39147/ ---

[GitHub] [spark] SparkQA removed a comment on pull request #31254: [SPARK-34165][SQL] Add count_distinct as an option to Dataset#summary

SparkQA removed a comment on pull request #31254: URL: https://github.com/apache/spark/pull/31254#issuecomment-768367512 **[Test build #134558 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134558/testReport)** for PR 31254 at commit [`bc547a2`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31254: [SPARK-34165][SQL] Add count_distinct as an option to Dataset#summary

SparkQA commented on pull request #31254: URL: https://github.com/apache/spark/pull/31254#issuecomment-768533838 **[Test build #134558 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134558/testReport)** for PR 31254 at commit [`bc547a2`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

SparkQA commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768532277 **[Test build #134565 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134565/testReport)** for PR 31368 at commit [`640a36b`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-768530024 **[Test build #134564 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134564/testReport)** for PR 31249 at commit [`1d0a522`](https://github.com

[GitHub] [spark] ekoifman commented on pull request #30829: [SPARK-33832][SQL] Add an option in AQE to mitigate skew even if it c…

ekoifman commented on pull request #30829: URL: https://github.com/apache/spark/pull/30829#issuecomment-768529666 ok, let me think about this - I'll be back This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] dongjoon-hyun commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

dongjoon-hyun commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768529453 Thank you, @imback82 , @HyukjinKwon , and @cloud-fan . Ya, @cloud-fan 's new PR might be a better approach.

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768528973 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134566/ -

[GitHub] [spark] SparkQA removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768527521 **[Test build #134566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134566/testReport)** for PR 30175 at commit [`d6174fa`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768528973 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134566/ -

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768528959 **[Test build #134566 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134566/testReport)** for PR 30175 at commit [`d6174fa`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

AmplabJenkins removed a comment on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-768526293 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134553/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

AmplabJenkins removed a comment on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-718061329 Can one of the admins verify this patch? This is an automated message from the Apache Git Service.

[GitHub] [spark] SparkQA commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

SparkQA commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768527521 **[Test build #134566 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134566/testReport)** for PR 30175 at commit [`d6174fa`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31358: [SPARK-34261][SQL] Avoid side effect if create exists temporary function

AmplabJenkins removed a comment on pull request #31358: URL: https://github.com/apache/spark/pull/31358#issuecomment-768527549 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134555/ -

[GitHub] [spark] SparkQA commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

SparkQA commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-768527445 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39148/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31358: [SPARK-34261][SQL] Avoid side effect if create exists temporary function

AmplabJenkins commented on pull request #31358: URL: https://github.com/apache/spark/pull/31358#issuecomment-768527549 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134555/ -

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31355: [SPARK-34255][SQL] Support partitioning with static number on required distribution and ordering on V2 write

HeartSaVioR commented on a change in pull request #31355: URL: https://github.com/apache/spark/pull/31355#discussion_r565581188 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/connector/distributions/distributions.scala ## @@ -29,31 +29,47 @@ private[sql] object

[GitHub] [spark] AmplabJenkins commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

AmplabJenkins commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-768526293 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134553/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

AmplabJenkins commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768526213 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39151/ -

[GitHub] [spark] revans2 commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

revans2 commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768526099 > I don't know if/how we can automate a unit test for this. Let me know what you think It looks like the changes are within noise. For me I don't think we need any automa

[GitHub] [spark] SparkQA removed a comment on pull request #31358: [SPARK-34261][SQL] Avoid side effect if create exists temporary function

SparkQA removed a comment on pull request #31358: URL: https://github.com/apache/spark/pull/31358#issuecomment-768333557 **[Test build #134555 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134555/testReport)** for PR 31358 at commit [`590b0b3`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31358: [SPARK-34261][SQL] Avoid side effect if create exists temporary function

SparkQA commented on pull request #31358: URL: https://github.com/apache/spark/pull/31358#issuecomment-768525554 **[Test build #134555 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134555/testReport)** for PR 31358 at commit [`590b0b3`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

SparkQA removed a comment on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-768333446 **[Test build #134553 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134553/testReport)** for PR 31364 at commit [`c20579e`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

SparkQA commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768525194 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39147/ -

[GitHub] [spark] SparkQA commented on pull request #31364: [SPARK-34266][SQL][DOCS] Update comments for `SessionCatalog.refreshTable()` and `CatalogImpl.refreshTable()`

SparkQA commented on pull request #31364: URL: https://github.com/apache/spark/pull/31364#issuecomment-768525090 **[Test build #134553 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134553/testReport)** for PR 31364 at commit [`c20579e`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

AmplabJenkins removed a comment on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-768523389 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39146/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

AmplabJenkins removed a comment on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768523384 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134561/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

AmplabJenkins commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768523384 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134561/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

AmplabJenkins commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-768523389 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39146/ -

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-768519247 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39149/ -

[GitHub] [spark] razajafri edited a comment on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri edited a comment on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768517258 @revans2 I ran a test manually with two files with 1M records written with Spark 3.0.0. They were read in with `Spark-3.0.0`, `Spark-3.1` and with master with my fix. E

[GitHub] [spark] razajafri commented on pull request #31284: [SPARK-34167][SQL]Reading parquet with IntDecimal written as a LongDecimal blows up

razajafri commented on pull request #31284: URL: https://github.com/apache/spark/pull/31284#issuecomment-768517258 @revans2 I ran a test manually with two files with 1M records written with Spark 3.0.0. They were read in with Spark-3.0.0, Spark-3.1 and with master with my fix. Each file wa

[GitHub] [spark] viirya commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

viirya commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768514875 cc @HeartSaVioR @xuanyuanking @gaborgsomogyi This is an automated message from the Apache Git Service. To respon

[GitHub] [spark] viirya commented on pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

viirya commented on pull request #30175: URL: https://github.com/apache/spark/pull/30175#issuecomment-768514018 ok to test This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] warrenzhu25 commented on a change in pull request #30175: [SPARK-33274][SS] Fix job hang in cp mode when total cores less than total kafka partition

warrenzhu25 commented on a change in pull request #30175: URL: https://github.com/apache/spark/pull/30175#discussion_r565566452 ## File path: sql/core/src/test/scala/org/apache/spark/sql/streaming/continuous/ContinuousSuite.scala ## @@ -37,7 +37,9 @@ class ContinuousSuiteBase

[GitHub] [spark] cloud-fan commented on pull request #31352: [SPARK-34252][SQL] Subquery with view in aggregate's grouping expression fails during the analysis check

cloud-fan commented on pull request #31352: URL: https://github.com/apache/spark/pull/31352#issuecomment-768510068 @imback82 thanks for catching this issue! Since it only affects tests, I don't believe it's a real bug. I think calling a rule in `doCanonicalize` is tricky, so I opened

[GitHub] [spark] SparkQA removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

SparkQA removed a comment on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768492655 **[Test build #134561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134561/testReport)** for PR 31363 at commit [`b965b8a`](https://gi

[GitHub] [spark] cloud-fan commented on pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan commented on pull request #31368: URL: https://github.com/apache/spark/pull/31368#issuecomment-768507410 cc @linhongliu-db @imback82 @maropu @viirya This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

SparkQA commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768507222 **[Test build #134561 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134561/testReport)** for PR 31363 at commit [`b965b8a`](https://github.co

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565563247 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ## @@ -625,7 +625,8 @@ case class DescribeTableCommand(

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565562668 ## File path: sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/analysis/AnalysisSuite.scala ## @@ -655,28 +654,6 @@ class AnalysisSuite extend

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565562297 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/basicLogicalOperators.scala ## @@ -450,46 +450,22 @@ case class I

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565561910 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/LogicalPlan.scala ## @@ -230,7 +230,7 @@ object LogicalPlanIntegr

[GitHub] [spark] cloud-fan commented on a change in pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan commented on a change in pull request #31368: URL: https://github.com/apache/spark/pull/31368#discussion_r565560459 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala ## @@ -422,41 +422,6 @@ trait CheckAnalysis extend

[GitHub] [spark] dongjoon-hyun commented on pull request #31346: [SPARK-34244][SQL] Remove the Scala function version of regexp_extract_all

dongjoon-hyun commented on pull request #31346: URL: https://github.com/apache/spark/pull/31346#issuecomment-768503920 Could you send your opinion on the mailing list, @MrPowers ? Specifically, the following. Thanks! > Can we just implement all the SQL methods in Scala and Python so we h

[GitHub] [spark] dongjoon-hyun edited a comment on pull request #31346: [SPARK-34244][SQL] Remove the Scala function version of regexp_extract_all

dongjoon-hyun edited a comment on pull request #31346: URL: https://github.com/apache/spark/pull/31346#issuecomment-768503920 Could you send your opinion on the dev mailing list, @MrPowers ? Specifically, the following. Thanks! > Can we just implement all the SQL methods in Scala and Pyt

[GitHub] [spark] cloud-fan opened a new pull request #31368: [SPARK-34269][SQL] Simplify SQL view resolution

cloud-fan opened a new pull request #31368: URL: https://github.com/apache/spark/pull/31368 ### What changes were proposed in this pull request? The currently SQL (temp or permanent) view resolution is done in 2 steps: 1. In `SessionCatalog`, we get the view metadata, parse

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-768499457 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39146/ ---

[GitHub] [spark] MrPowers commented on pull request #31346: [SPARK-34244][SQL] Remove the Scala function version of regexp_extract_all

MrPowers commented on pull request #31346: URL: https://github.com/apache/spark/pull/31346#issuecomment-768499224 Can someone help me understand why this was removed? I [added this method to spark-daria a while back](https://github.com/MrPowers/spark-daria/blob/master/src/main/scala/com/g

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-768493108 **[Test build #134563 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134563/testReport)** for PR 31249 at commit [`50d3747`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

SparkQA commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768492655 **[Test build #134561 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134561/testReport)** for PR 31363 at commit [`b965b8a`](https://github.com

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r565542013 ## File path: resource-managers/kubernetes/integration-tests/src/test/scala/org/apache/spark/deploy/k8s/integrationtest/KubernetesSuite.scala ## @@ -345,

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r565541372 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -850,6 +882,22 @@ class CoarseGrainedSched

[GitHub] [spark] SparkQA commented on pull request #31349: [SPARK-34246][SQL] New type coercion syntax rules in ANSI mode

SparkQA commented on pull request #31349: URL: https://github.com/apache/spark/pull/31349#issuecomment-768486043 **[Test build #134562 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134562/testReport)** for PR 31349 at commit [`4a8ce27`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

AmplabJenkins removed a comment on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768485363 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39145/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31254: [SPARK-34165][SQL] Add count_distinct as an option to Dataset#summary

AmplabJenkins removed a comment on pull request #31254: URL: https://github.com/apache/spark/pull/31254#issuecomment-768485364 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39144/

[GitHub] [spark] AmplabJenkins commented on pull request #31254: [SPARK-34165][SQL] Add count_distinct as an option to Dataset#summary

AmplabJenkins commented on pull request #31254: URL: https://github.com/apache/spark/pull/31254#issuecomment-768485364 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39144/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31363: [SPARK-34154][YARN] Extend LocalityPlacementStrategySuite's test with a timeout

AmplabJenkins commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768485363 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39145/ -

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r565539349 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -506,6 +521,20 @@ class CoarseGrainedSched

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-768484943 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39146/ -

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r565537537 ## File path: core/src/main/scala/org/apache/spark/scheduler/cluster/CoarseGrainedSchedulerBackend.scala ## @@ -506,6 +521,20 @@ class CoarseGrainedSched

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r565535625 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -809,6 +809,12 @@ package object config { .booleanConf

[GitHub] [spark] attilapiros commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

attilapiros commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768477269 @dongjoon-hyun you are welcome. The code is ready for review / merge. This is an automated message

[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

viirya commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565521077 ## File path: sql/core/src/test/scala/org/apache/spark/sql/ColumnExpressionSuite.scala ## @@ -155,6 +155,21 @@ class ColumnExpressionSuite extends QueryTe

[GitHub] [spark] dongjoon-hyun closed pull request #31314: [SPARK-34221][WEBUI] Ensure if a stage fails in the UI page, the corresponding error message can be displayed correctly.

dongjoon-hyun closed pull request #31314: URL: https://github.com/apache/spark/pull/31314 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

viirya commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565520217 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala ## @@ -1559,6 +1559,14 @@ object SQLConf { .booleanConf

[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

viirya commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565519587 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/Analyzer.scala ## @@ -1763,6 +1763,21 @@ class Analyzer(override val cata

[GitHub] [spark] SparkQA commented on pull request #31254: [SPARK-34165][SQL] Add count_distinct as an option to Dataset#summary

SparkQA commented on pull request #31254: URL: https://github.com/apache/spark/pull/31254#issuecomment-768466190 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39144/ ---

[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

viirya commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565516787 ## File path: docs/sql-migration-guide.md ## @@ -49,6 +49,8 @@ license: | * and the method `spark.catalog.refreshTable` In Spark 3.1 and earlier, t

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31319: [SPARK-34212][SQL] Fix incorrect decimal reading from Parquet files

dongjoon-hyun commented on a change in pull request #31319: URL: https://github.com/apache/spark/pull/31319#discussion_r565516655 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQuerySuite.scala ## @@ -3868,6 +3869,57 @@ class SQLQuerySuite extends QueryTest with

[GitHub] [spark] attilapiros commented on a change in pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

attilapiros commented on a change in pull request #31363: URL: https://github.com/apache/spark/pull/31363#discussion_r565516665 ## File path: resource-managers/yarn/src/test/scala/org/apache/spark/deploy/yarn/LocalityPlacementStrategySuite.scala ## @@ -33,7 +33,7 @@ class Loca

[GitHub] [spark] viirya commented on a change in pull request #31286: [SPARK-34199][SQL] Block `table.*` inside function to follow ANSI standard and other SQL engines

viirya commented on a change in pull request #31286: URL: https://github.com/apache/spark/pull/31286#discussion_r565514250 ## File path: docs/sql-migration-guide.md ## @@ -49,6 +49,8 @@ license: | * and the method `spark.catalog.refreshTable` In Spark 3.1 and earlier, t

[GitHub] [spark] SparkQA commented on pull request #31298: [SPARK-34193][CORE] TorrentBroadcast block manager decommissioning race fix

SparkQA commented on pull request #31298: URL: https://github.com/apache/spark/pull/31298#issuecomment-768460247 **[Test build #134560 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134560/testReport)** for PR 31298 at commit [`de11694`](https://github.com

[GitHub] [spark] dongjoon-hyun closed pull request #31366: [SPARK-34267][SQL] Remove `refreshTable()` from `SessionState`

dongjoon-hyun closed pull request #31366: URL: https://github.com/apache/spark/pull/31366 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

SparkQA commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768456420 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39145/ ---

[GitHub] [spark] dongjoon-hyun closed pull request #31357: [SPARK-34212][SQL][FOLLOWUP] Refine the behavior of reading parquet non-decimal fields as decimal

dongjoon-hyun closed pull request #31357: URL: https://github.com/apache/spark/pull/31357 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on a change in pull request #31357: [SPARK-34212][SQL][FOLLOWUP] Refine the behavior of reading parquet non-decimal fields as decimal

dongjoon-hyun commented on a change in pull request #31357: URL: https://github.com/apache/spark/pull/31357#discussion_r565499039 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/parquet/ParquetRowConverter.scala ## @@ -282,20 +269,43 @@ private

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

AmplabJenkins removed a comment on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768448516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134559/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31366: [SPARK-34267][SQL] Remove `refreshTable()` from `SessionState`

AmplabJenkins removed a comment on pull request #31366: URL: https://github.com/apache/spark/pull/31366#issuecomment-768448519 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39142/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31362: [SPARK-34263][SQL] Simplify the code for treating unicode/octal/escaped characters in string literals

AmplabJenkins removed a comment on pull request #31362: URL: https://github.com/apache/spark/pull/31362#issuecomment-768448521 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39143/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31148: [SPARK-33867][SQL] Instant and LocalDate values aren't handled when generating SQL queries

AmplabJenkins removed a comment on pull request #31148: URL: https://github.com/apache/spark/pull/31148#issuecomment-768448520 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134551/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31264: [SPARK-34144][SQL] Exception thrown when trying to write LocalDate and Instant values to a table

AmplabJenkins removed a comment on pull request #31264: URL: https://github.com/apache/spark/pull/31264#issuecomment-768448523 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134550/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31362: [SPARK-34263][SQL] Simplify the code for treating unicode/octal/escaped characters in string literals

AmplabJenkins commented on pull request #31362: URL: https://github.com/apache/spark/pull/31362#issuecomment-768448521 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39143/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31148: [SPARK-33867][SQL] Instant and LocalDate values aren't handled when generating SQL queries

AmplabJenkins commented on pull request #31148: URL: https://github.com/apache/spark/pull/31148#issuecomment-768448520 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134551/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31366: [SPARK-34267][SQL] Remove `refreshTable()` from `SessionState`

AmplabJenkins commented on pull request #31366: URL: https://github.com/apache/spark/pull/31366#issuecomment-768448519 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39142/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31264: [SPARK-34144][SQL] Exception thrown when trying to write LocalDate and Instant values to a table

AmplabJenkins commented on pull request #31264: URL: https://github.com/apache/spark/pull/31264#issuecomment-768448523 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134550/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

AmplabJenkins commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768448516 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134559/ -

[GitHub] [spark] dongjoon-hyun commented on pull request #31223: [SPARK-31168][SPARK-33913][BUILD] Upgrade Scala to 2.12.13 and Kafka to 2.7.0

dongjoon-hyun commented on pull request #31223: URL: https://github.com/apache/spark/pull/31223#issuecomment-768443405 Thank you for the investigation and taking care of that, @Ngone51 and @HyukjinKwon . This is an automated

[GitHub] [spark] SparkQA commented on pull request #31363: [WIP][SPARK-34154][YARN] Fix flaky test in LocalityPlacementStrategySuite

SparkQA commented on pull request #31363: URL: https://github.com/apache/spark/pull/31363#issuecomment-768437448 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39145/ -

[GitHub] [spark] SparkQA removed a comment on pull request #31264: [SPARK-34144][SQL] Exception thrown when trying to write LocalDate and Instant values to a table

SparkQA removed a comment on pull request #31264: URL: https://github.com/apache/spark/pull/31264#issuecomment-768260152 **[Test build #134550 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134550/testReport)** for PR 31264 at commit [`307d819`](https://gi

[GitHub] [spark] SparkQA commented on pull request #31264: [SPARK-34144][SQL] Exception thrown when trying to write LocalDate and Instant values to a table

SparkQA commented on pull request #31264: URL: https://github.com/apache/spark/pull/31264#issuecomment-768431458 **[Test build #134550 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134550/testReport)** for PR 31264 at commit [`307d819`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31148: [SPARK-33867][SQL] Instant and LocalDate values aren't handled when generating SQL queries

SparkQA removed a comment on pull request #31148: URL: https://github.com/apache/spark/pull/31148#issuecomment-768260316 **[Test build #134551 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134551/testReport)** for PR 31148 at commit [`4f058af`](https://gi

<    1   2   3   4   5   6   7   8   >