[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745444557 **[Test build #132839 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132839/testReport)** for PR 30739 at commit [`2a73b8c`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30776: [SPARK-33787][SQL] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

SparkQA commented on pull request #30776: URL: https://github.com/apache/spark/pull/30776#issuecomment-745445534 **[Test build #132829 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132829/testReport)** for PR 30776 at commit [`2a6cbb8`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30779: [SPARK-33789][SQL][TESTS] Refactor unified V1 and V2 datasource tests

SparkQA commented on pull request #30779: URL: https://github.com/apache/spark/pull/30779#issuecomment-745445535 **[Test build #132828 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132828/testReport)** for PR 30779 at commit [`5836e54`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30776: [SPARK-33787][SQL] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

SparkQA removed a comment on pull request #30776: URL: https://github.com/apache/spark/pull/30776#issuecomment-745281011 **[Test build #132829 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132829/testReport)** for PR 30776 at commit [`2a6cbb8`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30784: [SPARK-33560][BUILD] Add "unused-import" check to Maven compilation process

SparkQA removed a comment on pull request #30784: URL: https://github.com/apache/spark/pull/30784#issuecomment-745343914 **[Test build #132835 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132835/testReport)** for PR 30784 at commit [`c6d2926`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30779: [SPARK-33789][SQL][TESTS] Refactor unified V1 and V2 datasource tests

SparkQA removed a comment on pull request #30779: URL: https://github.com/apache/spark/pull/30779#issuecomment-745280921 **[Test build #132828 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132828/testReport)** for PR 30779 at commit [`5836e54`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #29966: [SPARK-33084][CORE][SQL] Add jar support ivy path

SparkQA removed a comment on pull request #29966: URL: https://github.com/apache/spark/pull/29966#issuecomment-745282519 **[Test build #132831 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132831/testReport)** for PR 29966 at commit [`57c351d`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

SparkQA removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745437173 **[Test build #132839 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132839/testReport)** for PR 30739 at commit [`2a73b8c`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

SparkQA removed a comment on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745437041 **[Test build #132838 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132838/testReport)** for PR 30786 at commit [`946e0a8`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745446967 **[Test build #132840 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132840/testReport)** for PR 30739 at commit [`8149c34`](https://github.co

[GitHub] [spark] rdblue commented on a change in pull request #30558: [SPARK-33612][SQL] Add dataSourceRewriteRules batch to Optimizer

rdblue commented on a change in pull request #30558: URL: https://github.com/apache/spark/pull/30558#discussion_r543546112 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/optimizer/Optimizer.scala ## @@ -185,6 +185,9 @@ abstract class Optimizer(catalogM

[GitHub] [spark] SparkQA removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

SparkQA removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745439650 **[Test build #132840 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132840/testReport)** for PR 30739 at commit [`8149c34`](https://gi

[GitHub] [spark] viirya commented on a change in pull request #30774: [SPARK-33786][SQL] The storage level for a cache should be respected when a table name is altered.

viirya commented on a change in pull request #30774: URL: https://github.com/apache/spark/pull/30774#discussion_r543550354 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ## @@ -195,16 +195,21 @@ case class AlterTableRenameCommand(

[GitHub] [spark] SparkQA commented on pull request #30642: [SPARK-32991][SQL] [FOLLOWUP] Reset command relies on session initials first

SparkQA commented on pull request #30642: URL: https://github.com/apache/spark/pull/30642#issuecomment-745453172 **[Test build #132830 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132830/testReport)** for PR 30642 at commit [`8702522`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30642: [SPARK-32991][SQL] [FOLLOWUP] Reset command relies on session initials first

SparkQA removed a comment on pull request #30642: URL: https://github.com/apache/spark/pull/30642#issuecomment-745280955 **[Test build #132830 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132830/testReport)** for PR 30642 at commit [`8702522`](https://gi

[GitHub] [spark] SparkQA commented on pull request #30658: [SPARK-22769] Do not log rpc post message error when sparkEnv is already stopped

SparkQA commented on pull request #30658: URL: https://github.com/apache/spark/pull/30658#issuecomment-745459024 **[Test build #132837 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132837/testReport)** for PR 30658 at commit [`5c40030`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30728: [SPARK-33756][SQL] Make BytesToBytesMap's MapIterator idempotent

SparkQA commented on pull request #30728: URL: https://github.com/apache/spark/pull/30728#issuecomment-745459481 **[Test build #132836 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132836/testReport)** for PR 30728 at commit [`f52d851`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30759: [SPARK-33774][UI][CORE]"Back to Master" returns 500 error in Standalone cluster

SparkQA commented on pull request #30759: URL: https://github.com/apache/spark/pull/30759#issuecomment-745459738 **[Test build #132833 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132833/testReport)** for PR 30759 at commit [`8e27e06`](https://github.co

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745460124 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37440/ -

[GitHub] [spark] SparkQA commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

SparkQA commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745464152 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37441/ -

[GitHub] [spark] SparkQA commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745464549 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37442/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30759: [SPARK-33774][UI][CORE]"Back to Master" returns 500 error in Standalone cluster

AmplabJenkins commented on pull request #30759: URL: https://github.com/apache/spark/pull/30759#issuecomment-745466053 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132833/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30776: [SPARK-33787][SQL] Add the `purge` parameter to `dropPartition()` of `SupportsPartitionManagement`

AmplabJenkins commented on pull request #30776: URL: https://github.com/apache/spark/pull/30776#issuecomment-745466056 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132829/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30658: [SPARK-22769] Do not log rpc post message error when sparkEnv is already stopped

AmplabJenkins commented on pull request #30658: URL: https://github.com/apache/spark/pull/30658#issuecomment-745466054 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132837/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

AmplabJenkins commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745466049 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30728: [SPARK-33756][SQL] Make BytesToBytesMap's MapIterator idempotent

AmplabJenkins commented on pull request #30728: URL: https://github.com/apache/spark/pull/30728#issuecomment-745466060 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132836/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30779: [SPARK-33789][SQL][TESTS] Refactor unified V1 and V2 datasource tests

AmplabJenkins commented on pull request #30779: URL: https://github.com/apache/spark/pull/30779#issuecomment-745466052 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30642: [SPARK-32991][SQL] [FOLLOWUP] Reset command relies on session initials first

AmplabJenkins commented on pull request #30642: URL: https://github.com/apache/spark/pull/30642#issuecomment-745466047 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132830/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

AmplabJenkins commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745466058 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132838/ -

[GitHub] [spark] aokolnychyi opened a new pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

aokolnychyi opened a new pull request #30787: URL: https://github.com/apache/spark/pull/30787 ### What changes were proposed in this pull request? This PR adds `UpdateTable` to supported plans in `ReplaceNullWithFalseInPredicate`. ### Why are the changes needed?

[GitHub] [spark] SparkQA commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

SparkQA commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745467039 **[Test build #132842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132842/testReport)** for PR 30787 at commit [`5a8dffb`](https://github.com

[GitHub] [spark] aokolnychyi commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

aokolnychyi commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745466939 cc @dbtsai @dongjoon-hyun @viirya @sunchao @cloud-fan @HyukjinKwon This is an automated message from the Ap

[GitHub] [spark] SparkQA commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745467805 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[GitHub] [spark] dmcwhorter edited a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

dmcwhorter edited a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745435394 @srowen I believe if no config is set this change would result in mesos allocating max(384MB, driverMemory * 0.1) of extra overhead memory for the driver, whereas pre

[GitHub] [spark] dongjoon-hyun commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

dongjoon-hyun commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745470185 cc @rdblue This is an automated message from the Apache Git Service. To respond to the message, please lo

[GitHub] [spark] SparkQA commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745470475 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37444/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745470507 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37444/ -

[GitHub] [spark] dmcwhorter commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

dmcwhorter commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745472795 I don't see a migration guide for mesos, unless I'm missing it somewhere. I also don't think I have access to make those changes to the JIRA. Could someone else add the rel

[GitHub] [spark] yliou opened a new pull request #30788: Fix and Test for SPARK-33726

yliou opened a new pull request #30788: URL: https://github.com/apache/spark/pull/30788 ### What changes were proposed in this pull request? The code will iterate over the fields themselves in `org.apache.spark.sql.catalyst.expressions.RowBasedKeyValueBatch#allocate` instead of

[GitHub] [spark] AmplabJenkins commented on pull request #30788: Fix and Test for SPARK-33726

AmplabJenkins commented on pull request #30788: URL: https://github.com/apache/spark/pull/30788#issuecomment-745473938 Can one of the admins verify this patch? This is an automated message from the Apache Git Service. To resp

[GitHub] [spark] SparkQA commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

SparkQA commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745479731 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37440/ ---

[GitHub] [spark] SparkQA commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745480781 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37442/ ---

[GitHub] [spark] dbtsai commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

dbtsai commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745481133 LGTM. Pending tests. This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] SparkQA removed a comment on pull request #30728: [SPARK-33756][SQL] Make BytesToBytesMap's MapIterator idempotent

SparkQA removed a comment on pull request #30728: URL: https://github.com/apache/spark/pull/30728#issuecomment-745367358 **[Test build #132836 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132836/testReport)** for PR 30728 at commit [`f52d851`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30759: [SPARK-33774][UI][CORE]"Back to Master" returns 500 error in Standalone cluster

SparkQA removed a comment on pull request #30759: URL: https://github.com/apache/spark/pull/30759#issuecomment-745303339 **[Test build #132833 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132833/testReport)** for PR 30759 at commit [`8e27e06`](https://gi

[GitHub] [spark] SparkQA removed a comment on pull request #30658: [SPARK-22769] Do not log rpc post message error when sparkEnv is already stopped

SparkQA removed a comment on pull request #30658: URL: https://github.com/apache/spark/pull/30658#issuecomment-745367547 **[Test build #132837 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132837/testReport)** for PR 30658 at commit [`5c40030`](https://gi

[GitHub] [spark] aokolnychyi commented on a change in pull request #30706: [SPARK-33779][SQL] DataSource V2: API to request distribution and ordering on write

aokolnychyi commented on a change in pull request #30706: URL: https://github.com/apache/spark/pull/30706#discussion_r543587762 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/distributions/ClusteredDistribution.java ## @@ -0,0 +1,35 @@ +/* + * Licensed

[GitHub] [spark] aokolnychyi commented on a change in pull request #30706: [SPARK-33779][SQL] DataSource V2: API to request distribution and ordering on write

aokolnychyi commented on a change in pull request #30706: URL: https://github.com/apache/spark/pull/30706#discussion_r543587762 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/distributions/ClusteredDistribution.java ## @@ -0,0 +1,35 @@ +/* + * Licensed

[GitHub] [spark] SparkQA commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

SparkQA commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745483187 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37441/ ---

[GitHub] [spark] SparkQA commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

SparkQA commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745497135 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37445/ -

[GitHub] [spark] venkata91 commented on a change in pull request #30775: [SPARK-33778][SQL] Allow typesafe join for LeftSemi and LeftAnti

venkata91 commented on a change in pull request #30775: URL: https://github.com/apache/spark/pull/30775#discussion_r543609795 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -1240,6 +1240,40 @@ class Dataset[T] private[sql]( joinWith(other, c

[GitHub] [spark] AmplabJenkins commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

AmplabJenkins commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745501294 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37441/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

AmplabJenkins commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745501289 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHu

[GitHub] [spark] AmplabJenkins commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745501288 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37442/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

AmplabJenkins removed a comment on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745466049 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins removed a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745470507 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

AmplabJenkins removed a comment on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745466058 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] venkata91 commented on a change in pull request #30775: [SPARK-33778][SQL] Allow typesafe join for LeftSemi and LeftAnti

venkata91 commented on a change in pull request #30775: URL: https://github.com/apache/spark/pull/30775#discussion_r543610223 ## File path: sql/core/src/main/scala/org/apache/spark/sql/Dataset.scala ## @@ -1240,6 +1240,40 @@ class Dataset[T] private[sql]( joinWith(other, c

[GitHub] [spark] SparkQA commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

SparkQA commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745516122 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/37445/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

AmplabJenkins commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745535762 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37445/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

AmplabJenkins removed a comment on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745535762 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/37445/

[GitHub] [spark] memoryz commented on pull request #29068: [SPARK-27892][ML]Saving/loading stages in PipelineModel are parallel

memoryz commented on pull request #29068: URL: https://github.com/apache/spark/pull/29068#issuecomment-745550930 Can someone merge this change? It's a very straightforward change, and saves quite a bit of saving/loading time for big models.

[GitHub] [spark] SparkQA commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745586273 **[Test build #132841 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132841/testReport)** for PR 30770 at commit [`417e5c7`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA removed a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745441438 **[Test build #132841 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132841/testReport)** for PR 30770 at commit [`417e5c7`](https://gi

[GitHub] [spark] viirya commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

viirya commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745587886 cc @dongjoon-hyun @HyukjinKwon @HeartSaVioR This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] dongjoon-hyun closed pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

dongjoon-hyun closed pull request #30787: URL: https://github.com/apache/spark/pull/30787 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

dongjoon-hyun commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745590265 Merged to master for Apache Spark 3.2.0. Thanks, @aokolnychyi , @dbtsai , @viirya , @rdblue . This is an

[GitHub] [spark] dongjoon-hyun closed pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

dongjoon-hyun closed pull request #30739: URL: https://github.com/apache/spark/pull/30739 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[GitHub] [spark] dongjoon-hyun commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

dongjoon-hyun commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745595615 Thank you for your first contribution, @dmcwhorter . You are added to the Apache Spark contributor group and SPARK-22256 is assigned to you. -

[GitHub] [spark] AmplabJenkins commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745597396 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132841/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins removed a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745597396 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132841/ -

[GitHub] [spark] dmcwhorter commented on pull request #30739: [SPARK-22256][MESOS] Introduce spark.mesos.driver.memoryOverhead

dmcwhorter commented on pull request #30739: URL: https://github.com/apache/spark/pull/30739#issuecomment-745603895 Excellent, thanks to all of you for helping me to get this through the process and into spark. This is an au

[GitHub] [spark] SparkQA commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

SparkQA commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745607193 **[Test build #132842 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132842/testReport)** for PR 30787 at commit [`5a8dffb`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

SparkQA removed a comment on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745467039 **[Test build #132842 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132842/testReport)** for PR 30787 at commit [`5a8dffb`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

AmplabJenkins commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745622668 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132842/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

AmplabJenkins removed a comment on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745622668 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132842/ -

[GitHub] [spark] SparkQA commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745624498 **[Test build #132843 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132843/testReport)** for PR 30770 at commit [`417e5c7`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

SparkQA removed a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745467894 **[Test build #132843 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132843/testReport)** for PR 30770 at commit [`417e5c7`](https://gi

[GitHub] [spark] AmplabJenkins commented on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins commented on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745625224 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132843/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30770: [SPARK-33783][SS] Unload State Store Provider after configured keep alive time

AmplabJenkins removed a comment on pull request #30770: URL: https://github.com/apache/spark/pull/30770#issuecomment-745625224 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132843/ -

[GitHub] [spark] imback82 commented on a change in pull request #30774: [SPARK-33786][SQL] The storage level for a cache should be respected when a table name is altered.

imback82 commented on a change in pull request #30774: URL: https://github.com/apache/spark/pull/30774#discussion_r543780949 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala ## @@ -195,16 +195,21 @@ case class AlterTableRenameCommand(

[GitHub] [spark] viirya opened a new pull request #30789: [SPARK-33797][SS][DOCS] Update SS doc about State Store and task locality

viirya opened a new pull request #30789: URL: https://github.com/apache/spark/pull/30789 ### What changes were proposed in this pull request? This updates SS documentation to document about State Store and task locality. ### Why are the changes needed? D

[GitHub] [spark] viirya commented on pull request #30789: [SPARK-33797][SS][DOCS] Update SS doc about State Store and task locality

viirya commented on pull request #30789: URL: https://github.com/apache/spark/pull/30789#issuecomment-745691941 cc @HyukjinKwon @HeartSaVioR @dongjoon-hyun This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HyukjinKwon commented on pull request #30787: [SPARK-33735][SQL] Handle UPDATE in ReplaceNullWithFalseInPredicate

HyukjinKwon commented on pull request #30787: URL: https://github.com/apache/spark/pull/30787#issuecomment-745692436 LGTM, thanks @aokolnychyi. This is an automated message from the Apache Git Service. To respond to the messa

[GitHub] [spark] hiboyang commented on pull request #30004: [SPARK-33114][CORE] Add metadata in MapStatus to support custom shuffle manager

hiboyang commented on pull request #30004: URL: https://github.com/apache/spark/pull/30004#issuecomment-745694448 > @hiboyang for helping @mccheah who is busy with other projects I cloned his PR as #30763. > > I will keep this up-to-date with master and react to review comments.

[GitHub] [spark] HyukjinKwon commented on pull request #30778: [SPARK-33788][SQL] Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions()

HyukjinKwon commented on pull request #30778: URL: https://github.com/apache/spark/pull/30778#issuecomment-745695766 Merged to master. This is an automated message from the Apache Git Service. To respond to the message, pleas

[GitHub] [spark] HyukjinKwon closed pull request #30778: [SPARK-33788][SQL] Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions()

HyukjinKwon closed pull request #30778: URL: https://github.com/apache/spark/pull/30778 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] HyukjinKwon commented on pull request #30778: [SPARK-33788][SQL] Throw NoSuchPartitionsException from HiveExternalCatalog.dropPartitions()

HyukjinKwon commented on pull request #30778: URL: https://github.com/apache/spark/pull/30778#issuecomment-745696120 @MaxGekk, it has a conflict in other branches. do you want to backport? This is an automated message from th

[GitHub] [spark] HyukjinKwon commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

HyukjinKwon commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745696726 Thanks @gengliangwang for fixing this! This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HyukjinKwon commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

HyukjinKwon commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745696844 Merged to master, branch-3.1 and branch-3.0. This is an automated message from the Apache Git Service. To re

[GitHub] [spark] HyukjinKwon closed pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

HyukjinKwon closed pull request #30786: URL: https://github.com/apache/spark/pull/30786 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #30789: [SPARK-33797][SS][DOCS] Update SS doc about State Store and task locality

SparkQA commented on pull request #30789: URL: https://github.com/apache/spark/pull/30789#issuecomment-745697065 **[Test build #132844 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132844/testReport)** for PR 30789 at commit [`cec4be9`](https://github.com

[GitHub] [spark] HyukjinKwon edited a comment on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

HyukjinKwon edited a comment on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745696844 Merged to master and branch-3.1. This is an automated message from the Apache Git Service. To respond

[GitHub] [spark] HyukjinKwon commented on pull request #30786: [SPARK-33796][DOCS] Show hidden text from the left menu of Spark Doc

HyukjinKwon commented on pull request #30786: URL: https://github.com/apache/spark/pull/30786#issuecomment-745697332 @gengliangwang, should we fix it in branch-3.0 too? it has a conflict. This is an automated message from the

[GitHub] [spark] HyukjinKwon commented on pull request #30784: [SPARK-33560][BUILD] Add "unused-import" check to Maven compilation process

HyukjinKwon commented on pull request #30784: URL: https://github.com/apache/spark/pull/30784#issuecomment-745697818 Thank you @LuciferYang for tackling this in Maven too. @MaxGekk too FYI. This is an automated message from t

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30784: [SPARK-33560][BUILD] Add "unused-import" check to Maven compilation process

AmplabJenkins removed a comment on pull request #30784: URL: https://github.com/apache/spark/pull/30784#issuecomment-745432684 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132835/ -

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30784: [SPARK-33560][BUILD] Add "unused-import" check to Maven compilation process

HyukjinKwon commented on a change in pull request #30784: URL: https://github.com/apache/spark/pull/30784#discussion_r543803604 ## File path: pom.xml ## @@ -3220,9 +3217,34 @@ scala-2.12 + +true + + +

[GitHub] [spark] HyukjinKwon commented on a change in pull request #30789: [SPARK-33797][SS][DOCS] Update SS doc about State Store and task locality

HyukjinKwon commented on a change in pull request #30789: URL: https://github.com/apache/spark/pull/30789#discussion_r543804073 ## File path: docs/structured-streaming-programming-guide.md ## @@ -1689,6 +1689,12 @@ hence the number is not same as the number of original input r

[GitHub] [spark] SparkQA commented on pull request #29268: [SPARK-29544] Collect the row count info and optimize the skewed conditions with row count info in the OptimizeSkewedJoin rule

SparkQA commented on pull request #29268: URL: https://github.com/apache/spark/pull/29268#issuecomment-745699022 **[Test build #132845 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/132845/testReport)** for PR 29268 at commit [`d0e0084`](https://github.com

[GitHub] [spark] AmplabJenkins commented on pull request #29268: [SPARK-29544] Collect the row count info and optimize the skewed conditions with row count info in the OptimizeSkewedJoin rule

AmplabJenkins commented on pull request #29268: URL: https://github.com/apache/spark/pull/29268#issuecomment-745699575 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/132845/ -

<    1   2   3   4   5   6   7   8   >