[GitHub] [spark] AmplabJenkins commented on pull request #32245: [SPARK-35142][ML] Fix incorrect return type for `rawPredictionUDF` in `OneVsRestModel`

AmplabJenkins commented on pull request #32245: URL: https://github.com/apache/spark/pull/32245#issuecomment-823026417 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For q

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32245: [SPARK-35142][ML] Fix incorrect return type for `rawPredictionUDF` in `OneVsRestModel`

AmplabJenkins removed a comment on pull request #32245: URL: https://github.com/apache/spark/pull/32245#issuecomment-823026415 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment

[GitHub] [spark] SparkQA removed a comment on pull request #32244: [SPARK-35060][SQL] Group exception messages in sql/types

SparkQA removed a comment on pull request #32244: URL: https://github.com/apache/spark/pull/32244#issuecomment-822919322 **[Test build #137659 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137659/testReport)** for PR 32244 at commit [`ee008f4`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32248: [SPARK-35143][SQL][SHELL]Add default log level config for spark-sql

AmplabJenkins commented on pull request #32248: URL: https://github.com/apache/spark/pull/32248#issuecomment-823027364 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #32244: [SPARK-35060][SQL] Group exception messages in sql/types

AmplabJenkins commented on pull request #32244: URL: https://github.com/apache/spark/pull/32244#issuecomment-823027619 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137659/ -- This

[GitHub] [spark] SparkQA commented on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite

SparkQA commented on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823027790 **[Test build #137674 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137674/testReport)** for PR 32146 at commit [`ccd4e0a`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823027862 **[Test build #137675 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137675/testReport)** for PR 32031 at commit [`ba7d5b6`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

SparkQA commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823027651 **[Test build #137673 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137673/testReport)** for PR 32247 at commit [`d636996`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan

SparkQA commented on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823028148 **[Test build #137676 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137676/testReport)** for PR 31451 at commit [`06eb9c7`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32244: [SPARK-35060][SQL] Group exception messages in sql/types

AmplabJenkins removed a comment on pull request #32244: URL: https://github.com/apache/spark/pull/32244#issuecomment-823027619 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137659/ -

[GitHub] [spark] SparkQA commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

SparkQA commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823029605 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

AmplabJenkins commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823029644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42198/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

AmplabJenkins removed a comment on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823029644 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42198/

[GitHub] [spark] SparkQA commented on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI

SparkQA commented on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823032202 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42197/ -- This is an automated message from the Apache

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

AmplabJenkins removed a comment on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-822921313 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137655/ -

[GitHub] [spark] SparkQA commented on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

SparkQA commented on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823033274 **[Test build #137677 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137677/testReport)** for PR 32242 at commit [`c9d09be`](https://github.com

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616404526 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/grouping.scala ## @@ -214,9 +214,9 @@ case class Grouping(child: Expressi

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616405930 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -761,8 +804,10 @@ case class HashAggregateExec(

[GitHub] [spark] SparkQA commented on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime

SparkQA commented on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823038086 Kubernetes integration test unable to build dist. exiting with code: 1 URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42200/ -- This

[GitHub] [spark] AmplabJenkins commented on pull request #32219: [WIP][SPARK-35097][SQL] Add column name to SparkUpgradeException about ancient datetime

AmplabJenkins commented on pull request #32219: URL: https://github.com/apache/spark/pull/32219#issuecomment-823038173 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42200/ -- T

[GitHub] [spark] SparkQA commented on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI

SparkQA commented on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823038327 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42197/ -- This is an automated message from the A

[GitHub] [spark] AmplabJenkins commented on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI

AmplabJenkins commented on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823038348 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42197/ -- T

[GitHub] [spark] grarkydev commented on pull request #32077: [SPARK-33357][K8S] Support Spark application managing with SparkAppHandle on Kubernetes

grarkydev commented on pull request #32077: URL: https://github.com/apache/spark/pull/32077#issuecomment-823038349 @dongjoon-hyun any further actions required? Co-authoring has been updated as you requested. -- This is an automated message from the Apache Git Service. To respond to t

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616409848 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala ## @@ -87,6 +87,18 @@ class RowBasedHashMapGene

[GitHub] [spark] SparkQA commented on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

SparkQA commented on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823042664 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] cloud-fan commented on pull request #32179: [SPARK-35080][SQL] Only allow a subset of correlated equality predicates when a subquery is aggregated

cloud-fan commented on pull request #32179: URL: https://github.com/apache/spark/pull/32179#issuecomment-823049437 @yaooqinn can you help to create the backport PR? I think we just need to update the golden files and the new tests added in this PR. -- This is an automated message from th

[GitHub] [spark] cloud-fan commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

cloud-fan commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-823052679 If the language import feature for symbol literal will be always there, I think we don't need any changes. We can probably wait until the Scala community decides to remove tha

[GitHub] [spark] beliefer commented on pull request #32244: [SPARK-35060][SQL] Group exception messages in sql/types

beliefer commented on pull request #32244: URL: https://github.com/apache/spark/pull/32244#issuecomment-823053881 ping @allisonwang-db -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specif

[GitHub] [spark] cloud-fan opened a new pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

cloud-fan opened a new pull request #32249: URL: https://github.com/apache/spark/pull/32249 ### What changes were proposed in this pull request? `CurrentOrigin` is a thread-local variable to track the original SQL line position in plan/expression. Usually, we set `CurrentOrig

[GitHub] [spark] cloud-fan commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

cloud-fan commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823061830 cc @maropu @viirya -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specifi

[GitHub] [spark] SparkQA commented on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

SparkQA commented on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823066540 **[Test build #137662 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137662/testReport)** for PR 32228 at commit [`0af2f3d`](https://github.co

[GitHub] [spark] AmplabJenkins commented on pull request #32245: [SPARK-35142][ML] Fix incorrect return type for `rawPredictionUDF` in `OneVsRestModel`

AmplabJenkins commented on pull request #32245: URL: https://github.com/apache/spark/pull/32245#issuecomment-823066628 Can one of the admins verify this patch? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL a

[GitHub] [spark] AmplabJenkins commented on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

AmplabJenkins commented on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823066593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42199/ -- T

[GitHub] [spark] SparkQA removed a comment on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

SparkQA removed a comment on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-822936964 **[Test build #137662 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137662/testReport)** for PR 32228 at commit [`0af2f3d`](https://gi

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

AmplabJenkins removed a comment on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823066593 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42199/

[GitHub] [spark] SparkQA commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

SparkQA commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823067877 **[Test build #137678 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137678/testReport)** for PR 32249 at commit [`b8ef281`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

AmplabJenkins commented on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823067977 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137662/ -- This

[GitHub] [spark] SparkQA commented on pull request #32201: [SPARK-35026][SQL] Support nested CUBE/ROLLUP/GROUPING SETS in GROUPING SETS

SparkQA commented on pull request #32201: URL: https://github.com/apache/spark/pull/32201#issuecomment-823068261 **[Test build #137664 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137664/testReport)** for PR 32201 at commit [`73478ca`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

AmplabJenkins removed a comment on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823067977 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137662/ -

[GitHub] [spark] SparkQA removed a comment on pull request #32201: [SPARK-35026][SQL] Support nested CUBE/ROLLUP/GROUPING SETS in GROUPING SETS

SparkQA removed a comment on pull request #32201: URL: https://github.com/apache/spark/pull/32201#issuecomment-822941577 **[Test build #137664 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137664/testReport)** for PR 32201 at commit [`73478ca`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32201: [SPARK-35026][SQL] Support nested CUBE/ROLLUP/GROUPING SETS in GROUPING SETS

AmplabJenkins commented on pull request #32201: URL: https://github.com/apache/spark/pull/32201#issuecomment-823070012 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137664/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32201: [SPARK-35026][SQL] Support nested CUBE/ROLLUP/GROUPING SETS in GROUPING SETS

AmplabJenkins removed a comment on pull request #32201: URL: https://github.com/apache/spark/pull/32201#issuecomment-823070012 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137664/ -

[GitHub] [spark] cloud-fan commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

cloud-fan commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616445190 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/RowBasedHashMapGenerator.scala ## @@ -87,6 +87,18 @@ class RowBasedHashM

[GitHub] [spark] cloud-fan commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

cloud-fan commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616447030 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -128,6 +128,16 @@ case class HashAggregate

[GitHub] [spark] attilapiros commented on pull request #32169: [SPARK-35009][CORE] Avoid creating multiple python worker monitor threads for the same worker and same task context

attilapiros commented on pull request #32169: URL: https://github.com/apache/spark/pull/32169#issuecomment-823073547 Thanks @srowen ! There is one more comment to address: https://github.com/apache/spark/pull/32169#discussion_r613924852 Let me think about it. -- This is an automat

[GitHub] [spark] LuciferYang commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

LuciferYang commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-823073871 Or we can temporarily suppress the compilation warnings to make it look cleaner -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616450151 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -128,6 +128,16 @@ case class HashAggregateExec(

[GitHub] [spark] SparkQA commented on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

SparkQA commented on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823075589 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins commented on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

AmplabJenkins commented on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823075641 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42205/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

AmplabJenkins removed a comment on pull request #32242: URL: https://github.com/apache/spark/pull/32242#issuecomment-823075641 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42205/

[GitHub] [spark] SparkQA commented on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan

SparkQA commented on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823076762 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins commented on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan

AmplabJenkins commented on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823076827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42204/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31451: [SPARK-34338][SQL] Report metrics from Datasource v2 scan

AmplabJenkins removed a comment on pull request #31451: URL: https://github.com/apache/spark/pull/31451#issuecomment-823076827 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42204/

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823077716 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42203/ -- This is an automated message from the Apache

[GitHub] [spark] SparkQA commented on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite

SparkQA commented on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823077761 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins commented on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite

AmplabJenkins commented on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823077803 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42202/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32146: [SPARK-34990][SQL][TESTS] Add ParquetEncryptionSuite

AmplabJenkins removed a comment on pull request #32146: URL: https://github.com/apache/spark/pull/32146#issuecomment-823077803 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42202/

[GitHub] [spark] AngersZhuuuu commented on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

AngersZh commented on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823080161 Gentle ping @maropu @cloud-fan This pr pass jenkins test now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823082430 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/42203/ -- This is an automated message from the A

[GitHub] [spark] cloud-fan commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

cloud-fan commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616458076 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -128,6 +128,16 @@ case class HashAggregate

[GitHub] [spark] cloud-fan commented on pull request #31569: [SPARK-34443][CORE] Replace symbol literals with Symbol constructor invocations to comply with Scala 2.13

cloud-fan commented on pull request #31569: URL: https://github.com/apache/spark/pull/31569#issuecomment-823082892 > Or we can temporarily suppress the compilation warnings to make it look cleaner +1 -- This is an automated message from the Apache Git Service. To respond to the me

[GitHub] [spark] cloud-fan commented on a change in pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

cloud-fan commented on a change in pull request #32228: URL: https://github.com/apache/spark/pull/32228#discussion_r616463214 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/ScriptTransformation.scala ## @@ -17,24 +17,22 @@ package org.

[GitHub] [spark] cloud-fan commented on a change in pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

cloud-fan commented on a change in pull request #32228: URL: https://github.com/apache/spark/pull/32228#discussion_r616463214 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/ScriptTransformation.scala ## @@ -17,24 +17,22 @@ package org.

[GitHub] [spark] viirya commented on pull request #32179: [SPARK-35080][SQL] Only allow a subset of correlated equality predicates when a subquery is aggregated

viirya commented on pull request #32179: URL: https://github.com/apache/spark/pull/32179#issuecomment-823088234 @yaooqinn please ping me if you create the backport PR. Thanks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub a

[GitHub] [spark] viirya commented on a change in pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

viirya commented on a change in pull request #32249: URL: https://github.com/apache/spark/pull/32249#discussion_r616466866 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala ## @@ -74,8 +74,11 @@ object CurrentOrigin { } def wi

[GitHub] [spark] yaooqinn commented on pull request #32179: [SPARK-35080][SQL] Only allow a subset of correlated equality predicates when a subquery is aggregated

yaooqinn commented on pull request #32179: URL: https://github.com/apache/spark/pull/32179#issuecomment-823091395 OK, I will try to raise a PR today (+08:00) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] SparkQA commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

SparkQA commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823091528 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616468793 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -128,6 +128,16 @@ case class HashAggregateExec(

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616473534 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -128,6 +128,16 @@ case class HashAggregateExec(

[GitHub] [spark] c21 commented on a change in pull request #32242: [SPARK-35141][SQL] Support two level of hash maps for final hash aggregation

c21 commented on a change in pull request #32242: URL: https://github.com/apache/spark/pull/32242#discussion_r616468793 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/HashAggregateExec.scala ## @@ -128,6 +128,16 @@ case class HashAggregateExec(

[GitHub] [spark] cloud-fan commented on a change in pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

cloud-fan commented on a change in pull request #32249: URL: https://github.com/apache/spark/pull/32249#discussion_r616475114 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/trees/TreeNode.scala ## @@ -74,8 +74,11 @@ object CurrentOrigin { } def

[GitHub] [spark] gaborgsomogyi commented on pull request #31944: [SPARK-34854][SQL][SS] Expose source metrics via progress report and add Kafka use-case to report delay.

gaborgsomogyi commented on pull request #31944: URL: https://github.com/apache/spark/pull/31944#issuecomment-823098027 I see that that the comments are resolved but no changes pushed. Are you still working on this, doing some tests or just forgotten? -- This is an automated message from

[GitHub] [spark] AngersZhuuuu opened a new pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite

AngersZh opened a new pull request #32250: URL: https://github.com/apache/spark/pull/32250 ### What changes were proposed in this pull request? After the PR https://github.com/apache/spark/pull/32209, this should be possible now. We can add test case for ANSI intervals to HiveThr

[GitHub] [spark] AngersZhuuuu commented on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite

AngersZh commented on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823106068 FYI @MaxGekk You mean add UT like this? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abo

[GitHub] [spark] AmplabJenkins commented on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

AmplabJenkins commented on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823109084 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42201/ -- T

[GitHub] [spark] AmplabJenkins commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

AmplabJenkins commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823109088 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42203/ -- T

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

AmplabJenkins removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823109088 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42203/

[GitHub] [spark] SparkQA commented on pull request #32250: [SPARK-35068][SQL] Add tests for ANSI intervals to HiveThriftBinaryServerSuite

SparkQA commented on pull request #32250: URL: https://github.com/apache/spark/pull/32250#issuecomment-823109907 **[Test build #137679 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137679/testReport)** for PR 32250 at commit [`b1a81bd`](https://github.com

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32247: [WIP][SPARK-35075] Add traversal pruning for subquery related rules

AmplabJenkins removed a comment on pull request #32247: URL: https://github.com/apache/spark/pull/32247#issuecomment-823109084 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42201/

[GitHub] [spark] SparkQA commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

SparkQA commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-82372 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries

[GitHub] [spark] AmplabJenkins commented on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

AmplabJenkins commented on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823111207 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42206/ -- T

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

AngersZh commented on a change in pull request #32228: URL: https://github.com/apache/spark/pull/32228#discussion_r616490950 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/ScriptTransformation.scala ## @@ -17,24 +17,22 @@ package o

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32249: [SPARK-35145][SQL] CurrentOrigin should support nested invoking

AmplabJenkins removed a comment on pull request #32249: URL: https://github.com/apache/spark/pull/32249#issuecomment-823111207 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/42206/

[GitHub] [spark] SparkQA commented on pull request #32228: [SPARK-34035][SQL] Refactor ScriptTransformation to remove input parameter and replace it by child.output

SparkQA commented on pull request #32228: URL: https://github.com/apache/spark/pull/32228#issuecomment-823112474 **[Test build #137680 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137680/testReport)** for PR 32228 at commit [`adea65b`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

SparkQA commented on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823114352 **[Test build #137671 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137671/testReport)** for PR 32227 at commit [`c3bb7d6`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

SparkQA removed a comment on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823005681 **[Test build #137671 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137671/testReport)** for PR 32227 at commit [`c3bb7d6`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

AmplabJenkins commented on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823116070 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137671/ -- This

[GitHub] [spark] SparkQA commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823119110 **[Test build #137675 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137675/testReport)** for PR 32031 at commit [`ba7d5b6`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

SparkQA removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823027862 **[Test build #137675 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137675/testReport)** for PR 32031 at commit [`ba7d5b6`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

AmplabJenkins commented on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823120150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137675/ -- This

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32031: [WIP] Initial work of Remote Shuffle Service on Kubernetes

AmplabJenkins removed a comment on pull request #32031: URL: https://github.com/apache/spark/pull/32031#issuecomment-823120150 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137675/ -

[GitHub] [spark] SparkQA commented on pull request #32072: [SPARK-34974][SQL] Improve subquery decorrelation framework

SparkQA commented on pull request #32072: URL: https://github.com/apache/spark/pull/32072#issuecomment-823121699 **[Test build #137667 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137667/testReport)** for PR 32072 at commit [`f665a0d`](https://github.co

[GitHub] [spark] AmplabJenkins removed a comment on pull request #32227: [SPARK-35132][BUILD][CORE] Upgrade netty-all to 4.1.63.Final

AmplabJenkins removed a comment on pull request #32227: URL: https://github.com/apache/spark/pull/32227#issuecomment-823116070 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137671/ -

[GitHub] [spark] kyoty commented on a change in pull request #32223: [WIP][SPARK-35127][UI] When we switch between different stage-detail pages, the entry item in the newly-opened page may be blank.

kyoty commented on a change in pull request #32223: URL: https://github.com/apache/spark/pull/32223#discussion_r616504336 ## File path: core/src/main/resources/org/apache/spark/ui/static/stagepage.js ## @@ -747,6 +747,7 @@ $(document).ready(function () { "i

[GitHub] [spark] AmplabJenkins commented on pull request #32072: [SPARK-34974][SQL] Improve subquery decorrelation framework

AmplabJenkins commented on pull request #32072: URL: https://github.com/apache/spark/pull/32072#issuecomment-823123266 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/137667/ -- This

[GitHub] [spark] cloud-fan commented on pull request #32072: [SPARK-34974][SQL] Improve subquery decorrelation framework

cloud-fan commented on pull request #32072: URL: https://github.com/apache/spark/pull/32072#issuecomment-823124078 thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[GitHub] [spark] cloud-fan closed pull request #32072: [SPARK-34974][SQL] Improve subquery decorrelation framework

cloud-fan closed pull request #32072: URL: https://github.com/apache/spark/pull/32072 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, plea

[GitHub] [spark] SparkQA commented on pull request #31974: [SPARK-34877][CORE][YARN]Add the code change for adding the Spark AM log link in spark UI

SparkQA commented on pull request #31974: URL: https://github.com/apache/spark/pull/31974#issuecomment-823125287 **[Test build #137669 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/137669/testReport)** for PR 31974 at commit [`eadb6da`](https://github.co

[GitHub] [spark] Peng-Lei opened a new pull request #32251: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode

Peng-Lei opened a new pull request #32251: URL: https://github.com/apache/spark/pull/32251 ### Why are the changes needed? In https://issues.apache.org/jira/browse/SPARK-35094, When use spark from_json(JsonToStruct) function in permissive mode to handle the case of contains incorrect ne

[GitHub] [spark] Peng-Lei closed pull request #32251: [SPARK-35094][SQL]Spark from_json(JsonToStruct) function return wrong value in permissive mode

Peng-Lei closed pull request #32251: URL: https://github.com/apache/spark/pull/32251 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, pleas

  1   2   3   4   5   6   >