[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775645976 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39640/

[GitHub] [spark] HyukjinKwon edited a comment on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

HyukjinKwon edited a comment on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-775543510 I don't think this blocks RC I am preparing now. Same here too. Hope we can do this earlier next time. @tgravescs, though, some of them like

[GitHub] [spark] SparkQA commented on pull request #31520: [SPARK-34355][CORE][SQL][FOLLOWUP] Log commit time in all File Writer

SparkQA commented on pull request #31520: URL: https://github.com/apache/spark/pull/31520#issuecomment-775644499 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39639/

[GitHub] [spark] 397090770 opened a new pull request #31532: [SPARK-34405][CORE] Fix mean value of timersLabels in the PrometheusServlet class

397090770 opened a new pull request #31532: URL: https://github.com/apache/spark/pull/31532 ### What changes were proposed in this pull request? The getMetricsSnapshot method of the PrometheusServlet class has a wrong value, It should be taking the mean value but it's taking the

[GitHub] [spark] calvin-pietersen commented on pull request #24801: [SPARK-27950][DSTREAMS][Kinesis] dynamoDBEndpointUrl and cloudWatchMetricsLevel for Kinesis

calvin-pietersen commented on pull request #24801: URL: https://github.com/apache/spark/pull/24801#issuecomment-775643984 @etspaceman any plans to try to merge this one? This is an automated message from the Apache Git

[GitHub] [spark] Ngone51 commented on pull request #30650: [SPARK-24818][CORE] Support delay scheduling for barrier execution

Ngone51 commented on pull request #30650: URL: https://github.com/apache/spark/pull/30650#issuecomment-775643884 Oh I mean, for the behavior of disabling legacy delay scheduling for the barrier stage. This is an automated

[GitHub] [spark] HyukjinKwon commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

HyukjinKwon commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-775643637 I think there's a logical conflict in branch-3.1 when we backport, see https://github.com/apache/spark/runs/1859823085. I think it doesn't matter much whether to backport

[GitHub] [spark] tgravescs commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

tgravescs commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-775643486 The API as you state is Evolving and that is on purpose so we can extend and change as people use it and we learn more. I'm happy to hear the feedback and improve the API

[GitHub] [spark] SparkQA removed a comment on pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in document

SparkQA removed a comment on pull request #31531: URL: https://github.com/apache/spark/pull/31531#issuecomment-775611852 **[Test build #135053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135053/testReport)** for PR 31531 at commit

[GitHub] [spark] SparkQA commented on pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in document

SparkQA commented on pull request #31531: URL: https://github.com/apache/spark/pull/31531#issuecomment-77563 **[Test build #135053 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135053/testReport)** for PR 31531 at commit

[GitHub] [spark] SparkQA commented on pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

SparkQA commented on pull request #31442: URL: https://github.com/apache/spark/pull/31442#issuecomment-775636311 **[Test build #135060 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135060/testReport)** for PR 31442 at commit

[GitHub] [spark] SparkQA commented on pull request #31520: [SPARK-34355][CORE][SQL][FOLLOWUP] Log commit time in all File Writer

SparkQA commented on pull request #31520: URL: https://github.com/apache/spark/pull/31520#issuecomment-775636157 **[Test build #135057 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135057/testReport)** for PR 31520 at commit

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775636072 **[Test build #135058 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135058/testReport)** for PR 31494 at commit

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775635903 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39638/

[GitHub] [spark] AmplabJenkins commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775635914 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39638/

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775635016 **[Test build #135059 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135059/testReport)** for PR 31485 at commit

[GitHub] [spark] SparkQA commented on pull request #31520: [SPARK-34355][CORE][SQL][FOLLOWUP] Log commit time in all File Writer

SparkQA commented on pull request #31520: URL: https://github.com/apache/spark/pull/31520#issuecomment-775633751 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39639/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-775632756 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135061/

[GitHub] [spark] SparkQA removed a comment on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA removed a comment on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-775630484 **[Test build #135061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135061/testReport)** for PR 31337 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

AmplabJenkins commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-775632756 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135061/

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-775632730 **[Test build #135061 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135061/testReport)** for PR 31337 at commit

[GitHub] [spark] SparkQA commented on pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in document

SparkQA commented on pull request #31531: URL: https://github.com/apache/spark/pull/31531#issuecomment-775631653 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39636/

[GitHub] [spark] HyukjinKwon commented on pull request #31530: Added KMeansSummary to __all__ in clustering.py

HyukjinKwon commented on pull request #31530: URL: https://github.com/apache/spark/pull/31530#issuecomment-775631560 Would you mind reading http://spark.apache.org/contributing.html and keeping [the GitHub PR

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

AmplabJenkins removed a comment on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775629976 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135048/

[GitHub] [spark] beliefer commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

beliefer commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-775630900 Why missing import 'org.apache.spark.TestUtils' ? This is an automated message from the Apache Git Service. To

[GitHub] [spark] SparkQA commented on pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

SparkQA commented on pull request #31337: URL: https://github.com/apache/spark/pull/31337#issuecomment-775630484 **[Test build #135061 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135061/testReport)** for PR 31337 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

AmplabJenkins commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775629976 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135048/

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775624853 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39638/

[GitHub] [spark] HyukjinKwon commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

HyukjinKwon commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-775624131 Thanks @zzcclp. I reverted it back from branch-3.1. This is an automated message from the Apache Git

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AngersZh commented on a change in pull request #31485: URL: https://github.com/apache/spark/pull/31485#discussion_r572535922 ## File path: sql/core/src/test/resources/sql-tests/results/explain-cbo.sql.out ## @@ -0,0 +1,80 @@ +-- Automatically generated by SQLQueryTestSuite

[GitHub] [spark] zzcclp commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

zzcclp commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-775618187 > Merged to master and branch-3.1. @HyukjinKwon after merged this pr into branch-3.1, there is an 'not found: value TestUtils' error in SQLQueryTestSuite, root cause is

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r572532292 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala ## @@ -46,6 +47,23 @@ import

[GitHub] [spark] SparkQA commented on pull request #31520: [SPARK-34355][CORE][SQL][FOLLOWUP] Log commit time in all File Writer

SparkQA commented on pull request #31520: URL: https://github.com/apache/spark/pull/31520#issuecomment-775616013 **[Test build #135056 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135056/testReport)** for PR 31520 at commit

[GitHub] [spark] sarutak commented on a change in pull request #31442: [SPARK-34333][SQL] Fix PostgresDialect to handle money types properly

sarutak commented on a change in pull request #31442: URL: https://github.com/apache/spark/pull/31442#discussion_r572527343 ## File path: docs/sql-migration-guide.md ## @@ -24,6 +24,8 @@ license: | ## Upgrading from Spark SQL 3.1 to 3.2 + - In Spark 3.2, money type in

[GitHub] [spark] srowen commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

srowen commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775612869 Merged to master This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] srowen closed pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

srowen closed pull request #31510: URL: https://github.com/apache/spark/pull/31510 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] Ngone51 commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

Ngone51 commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r572524552 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala ## @@ -46,6 +47,23 @@ import

[GitHub] [spark] AngersZhuuuu commented on pull request #31520: [SPARK-34355][CORE][SQL][FOLLOWUP] Log commit time in all File Writer

AngersZh commented on pull request #31520: URL: https://github.com/apache/spark/pull/31520#issuecomment-775612366 > I'm sorry I haven't considered about the impact of extracting the code - logger. I'm not sure we are leveraging the information on classname difference for tracking log

[GitHub] [spark] wangyum commented on a change in pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

wangyum commented on a change in pull request #31485: URL: https://github.com/apache/spark/pull/31485#discussion_r572524056 ## File path: sql/core/src/test/resources/sql-tests/results/explain-cbo.sql.out ## @@ -0,0 +1,80 @@ +-- Automatically generated by SQLQueryTestSuite +--

[GitHub] [spark] AmplabJenkins commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

AmplabJenkins commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-775612160 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39635/

[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-775612140 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39635/

[GitHub] [spark] SparkQA commented on pull request #31516: [SPARK-34238][SQL][FOLLOW_UP] SHOW PARTITIONS Keep consistence with other `SHOW` command

SparkQA commented on pull request #31516: URL: https://github.com/apache/spark/pull/31516#issuecomment-775612051 **[Test build #135055 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135055/testReport)** for PR 31516 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

SparkQA removed a comment on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775509993 **[Test build #135048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135048/testReport)** for PR 31510 at commit

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775611880 **[Test build #135054 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135054/testReport)** for PR 31485 at commit

[GitHub] [spark] SparkQA commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

SparkQA commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775611895 **[Test build #135048 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135048/testReport)** for PR 31510 at commit

[GitHub] [spark] SparkQA commented on pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in document

SparkQA commented on pull request #31531: URL: https://github.com/apache/spark/pull/31531#issuecomment-775611852 **[Test build #135053 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135053/testReport)** for PR 31531 at commit

[GitHub] [spark] rednaxelafx commented on pull request #31509: [SPARK-34396][SQL] Add a new build-in function delegate

rednaxelafx commented on pull request #31509: URL: https://github.com/apache/spark/pull/31509#issuecomment-775610985 Just some random thoughts: This kind of functionality is certain useful for debugging and could come in handy for writing shorthands, but I have mixed feelings about

[GitHub] [spark] beliefer commented on a change in pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

beliefer commented on a change in pull request #31337: URL: https://github.com/apache/spark/pull/31337#discussion_r572521967 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/commands.scala ## @@ -157,7 +156,7 @@ case class ExplainCommand(

[GitHub] [spark] beliefer commented on a change in pull request #31337: [SPARK-34234][SQL] Remove TreeNodeException that didn't work

beliefer commented on a change in pull request #31337: URL: https://github.com/apache/spark/pull/31337#discussion_r572520513 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala ## @@ -504,8 +504,7 @@ abstract class TreeNode[BaseType <:

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775605752 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39634/

[GitHub] [spark] LuciferYang commented on pull request #31517: [WIP][SPARK-34309][CORE][SQL] Use Caffeine instead of Guava Cache

LuciferYang commented on pull request #31517: URL: https://github.com/apache/spark/pull/31517#issuecomment-775607120 @Ngone51 I have tested some benchmark data in

[GitHub] [spark] AngersZhuuuu commented on pull request #31516: [SPARK-34238][SQL][FOLLOW_UP] SHOW PARTITIONS Keep consistence with other `SHOW` command

AngersZh commented on pull request #31516: URL: https://github.com/apache/spark/pull/31516#issuecomment-775606160 retest this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775605752 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39634/

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AngersZh commented on a change in pull request #31485: URL: https://github.com/apache/spark/pull/31485#discussion_r572518676 ## File path: sql/core/src/test/resources/sql-tests/inputs/explain-cbo.sql ## @@ -0,0 +1,25 @@ +CREATE TABLE t1(a INT, b INT) USING PARQUET; +CREATE

[GitHub] [spark] LuciferYang commented on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

LuciferYang commented on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-775605089 thx @srowen @HyukjinKwon This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-775601900 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39635/

[GitHub] [spark] mengxr edited a comment on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

mengxr edited a comment on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-775599608 @tgravescs I think there are two separate questions: 1. Does it block 3.1 release? I agree with you that it shouldn't. It is indeed too late in the release process

[GitHub] [spark] mengxr commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

mengxr commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-775599608 @tgravescs I think there are two separate questions: 1. Does it block 3.1 release? I agree with you that it shouldn't. It is indeed too late in the release process and the

[GitHub] [spark] beliefer commented on a change in pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

beliefer commented on a change in pull request #31466: URL: https://github.com/apache/spark/pull/31466#discussion_r572513892 ## File path: sql/core/src/test/scala/org/apache/spark/sql/SQLQueryTestSuite.scala ## @@ -154,10 +154,14 @@ class SQLQueryTestSuite extends QueryTest

[GitHub] [spark] beliefer commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

beliefer commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-775599266 @HyukjinKwon Thanks for your work! @cloud-fan @maropu Thanks for your review. This is an automated message

[GitHub] [spark] beliefer commented on pull request #31508: [SPARK-34393][SQL] Unify output of SHOW VIEWS and pass output attributes properly

beliefer commented on pull request #31508: URL: https://github.com/apache/spark/pull/31508#issuecomment-775597953 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] beliefer commented on pull request #31519: [SPARK-34394][SQL] Unify output of SHOW FUNCTIONS and pass output attributes properly

beliefer commented on pull request #31519: URL: https://github.com/apache/spark/pull/31519#issuecomment-775597852 cc @cloud-fan This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] HyukjinKwon closed pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

HyukjinKwon closed pull request #31466: URL: https://github.com/apache/spark/pull/31466 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] HyukjinKwon commented on pull request #31466: [SPARK-34352][SQL] Improve SQLQueryTestSuite so as could run on windows system

HyukjinKwon commented on pull request #31466: URL: https://github.com/apache/spark/pull/31466#issuecomment-775597566 Merged to master and branch-3.1. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] beliefer commented on pull request #31526: [SPARK-34157][BUILD][FOLLOW-UP] Fix Scala 2.13 compilation error via using Array.deep

beliefer commented on pull request #31526: URL: https://github.com/apache/spark/pull/31526#issuecomment-775595375 @HyukjinKwon Thanks for your fix! This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] Ngone51 commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

Ngone51 commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-775589882 LGTM This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] Ngone51 commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

Ngone51 commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r572505384 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -239,18 +239,35 @@ class HDFSMetadataLog[T <:

[GitHub] [spark] viirya commented on pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in document

viirya commented on pull request #31531: URL: https://github.com/apache/spark/pull/31531#issuecomment-775588020 cc @huaxingao @WeichenXu123 This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] viirya opened a new pull request #31531: [SPARK-34080][ML][PYTHON][FOLLOW-UP] Update score function in document

viirya opened a new pull request #31531: URL: https://github.com/apache/spark/pull/31531 ### What changes were proposed in this pull request? This follows up #31160 to update score function in the document. ### Why are the changes needed? Currently we

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775586246 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39634/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

AmplabJenkins removed a comment on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775584697 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135050/

[GitHub] [spark] AmplabJenkins commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

AmplabJenkins commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775584697 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135050/

[GitHub] [spark] AmplabJenkins commented on pull request #31530: Added KMeansSummary to __all__ in clustering.py

AmplabJenkins commented on pull request #31530: URL: https://github.com/apache/spark/pull/31530#issuecomment-775584415 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To

[GitHub] [spark] JohnHBauer opened a new pull request #31530: Added KMeansSummary to __all__ in clustering.py

JohnHBauer opened a new pull request #31530: URL: https://github.com/apache/spark/pull/31530 KMeansSummary was undocumented. The KMeansSummary class is present in the code, not exposed in __all__, and therefore not documented in the Sphinx build. Adding 'KMeansSummary' to

[GitHub] [spark] SparkQA removed a comment on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA removed a comment on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775537236 **[Test build #135050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135050/testReport)** for PR 31249 at commit

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775584209 **[Test build #135050 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135050/testReport)** for PR 31249 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

AmplabJenkins removed a comment on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775580867 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39632/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775580868 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135051/

[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-775582196 **[Test build #135052 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135052/testReport)** for PR 31495 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

AmplabJenkins commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775580867 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39632/

[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775580868 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135051/

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775574877 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39634/

[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775561922 **[Test build #135051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135051/testReport)** for PR 31529 at commit

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775571042 **[Test build #135051 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135051/testReport)** for PR 31529 at commit

[GitHub] [spark] SparkQA commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

SparkQA commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775568472 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39632/

[GitHub] [spark] github-actions[bot] closed pull request #30199: [SPARK-33296][SQL] Format exception of unsupported nested grouping analytics

github-actions[bot] closed pull request #30199: URL: https://github.com/apache/spark/pull/30199 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

[GitHub] [spark] HeartSaVioR commented on pull request #31520: [SPARK-34355][CORE][SQL][FOLLOWUP] Log commit time in all File Writer

HeartSaVioR commented on pull request #31520: URL: https://github.com/apache/spark/pull/31520#issuecomment-775566881 I'm sorry I haven't considered about the impact of extracting the code - logger. I'm not sure we are leveraging the information on classname difference for tracking log on

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r572479885 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -239,18 +239,35 @@ class

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775561922 **[Test build #135051 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135051/testReport)** for PR 31529 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

AmplabJenkins removed a comment on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775558824 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39633/

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r572473687 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -239,18 +239,35 @@ class

[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations

HeartSaVioR commented on a change in pull request #31495: URL: https://github.com/apache/spark/pull/31495#discussion_r572473687 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/HDFSMetadataLog.scala ## @@ -239,18 +239,35 @@ class

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775558803 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39633/

[GitHub] [spark] AmplabJenkins commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

AmplabJenkins commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775558824 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39633/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-77757 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135046/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

AmplabJenkins removed a comment on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-77758 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135049/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-77761 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39630/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

AmplabJenkins removed a comment on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-77759 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39631/

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775556423 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39633/

[GitHub] [spark] AmplabJenkins commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

AmplabJenkins commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-77758 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135049/

<    1   2   3   4   5   6   7   8   >