[GitHub] [spark] SparkQA commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read
SparkQA commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774420334 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39534/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774415544 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134949/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read
SparkQA commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774415968 **[Test build #134951 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134951/testReport)** for PR 31489 at commit [`f33b5a8`](https://github.com/apache/spark/commit/f33b5a85d56ee99f98df11a2f3cac4f1f568a836). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774415544 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134949/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774380337 **[Test build #134949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134949/testReport)** for PR 24559 at commit [`15e127f`](https://github.com/apache/spark/commit/15e127f67b50f1150a5ec2a5ebf7a89dc5aa4da4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774410185 **[Test build #134949 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134949/testReport)** for PR 24559 at commit [`15e127f`](https://github.com/apache/spark/commit/15e127f67b50f1150a5ec2a5ebf7a89dc5aa4da4). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations
AmplabJenkins removed a comment on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774407572 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39533/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations
AmplabJenkins commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774407572 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39533/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations
SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774402870 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39533/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations
SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774402314 **[Test build #134950 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134950/testReport)** for PR 31495 at commit [`df4dd75`](https://github.com/apache/spark/commit/df4dd75f4741a7c3df60c2e90347fff519e0f264). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774401013 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39532/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774401013 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39532/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase via reducing cost of filesystem operations
SparkQA commented on pull request #31495: URL: https://github.com/apache/spark/pull/31495#issuecomment-774399753 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39533/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774397966 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39532/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase on SS
HeartSaVioR commented on a change in pull request #31495:
URL: https://github.com/apache/spark/pull/31495#discussion_r571355810
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala
##
@@ -46,6 +47,23 @@ import org.apache.spark.sql.connector.read.streaming.{Offset
=> OffsetV2}
class OffsetSeqLog(sparkSession: SparkSession, path: String)
extends HDFSMetadataLog[OffsetSeq](sparkSession, path) {
+ private val cachedMetadata = new ju.TreeMap[Long, OffsetSeq]()
+
+ override def add(batchId: Long, metadata: OffsetSeq): Boolean = {
+val added = super.add(batchId, metadata)
+if (added) {
+ // cache metadata as it will be read again
+ cachedMetadata.put(batchId, metadata)
+ // we don't access metadata for (batchId - 2) batches; evict them
Review comment:
Note for reviewers: see MicroBatchExecution / ContinuousExecution around
calling `source.commit()`. To provide the offset metadata to source side we
read offset metadata for previous batch, which is probably written by this
driver in previous batches. micro-batch: (batchId - 1), continuous mode:
batchId.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR commented on a change in pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase on SS
HeartSaVioR commented on a change in pull request #31495:
URL: https://github.com/apache/spark/pull/31495#discussion_r571355810
##
File path:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/OffsetSeqLog.scala
##
@@ -46,6 +47,23 @@ import org.apache.spark.sql.connector.read.streaming.{Offset
=> OffsetV2}
class OffsetSeqLog(sparkSession: SparkSession, path: String)
extends HDFSMetadataLog[OffsetSeq](sparkSession, path) {
+ private val cachedMetadata = new ju.TreeMap[Long, OffsetSeq]()
+
+ override def add(batchId: Long, metadata: OffsetSeq): Boolean = {
+val added = super.add(batchId, metadata)
+if (added) {
+ // cache metadata as it will be read again
+ cachedMetadata.put(batchId, metadata)
+ // we don't access metadata for (batchId - 2) batches; evict them
Review comment:
Note for reviewers: see MicroBatchExecution / ContinuousExecution around
calling `source.commit()`. To provide the offset metadata to source side we
read offset metadata for previous batch, which is probably written by this
driver in previous batches.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] HeartSaVioR opened a new pull request #31495: [SPARK-34383][SS] Optimize WAL commit phase on SS
HeartSaVioR opened a new pull request #31495: URL: https://github.com/apache/spark/pull/31495 ### What changes were proposed in this pull request? This PR proposes to optimize WAL commit phase via following changes: * cache offset log to avoid FS get operation per batch * use FS exist operation instead of FS list operation on purge (2 operations per batch) ### Why are the changes needed? There're inefficiency on WAL commit phase which can be easily optimized via using a small driver memory. ### Does this PR introduce _any_ user-facing change? No. ### How was this patch tested? Manually tested with debug log. (Verified that cache is used, cache keeps the size as 2, only one exist call is used instead of list call) Experimental on AWS S3 + S3 guard: > before the patch > after the patch Experimental on Azure: > before the patch > after the patch This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774394722 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134945/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774394722 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134945/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774340507 **[Test build #134945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134945/testReport)** for PR 31494 at commit [`dbbde92`](https://github.com/apache/spark/commit/dbbde92612ff4084905465174d027f8f13100679). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774388999 **[Test build #134945 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134945/testReport)** for PR 31494 at commit [`dbbde92`](https://github.com/apache/spark/commit/dbbde92612ff4084905465174d027f8f13100679). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774387534 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39532/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571342363 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -437,9 +437,8 @@ class DataSourceV2Strategy(session: SparkSession) extends Strategy with Predicat }.toSeq AlterTableExec(table.catalog, table.identifier, changes) :: Nil -// TODO: v2 `UNSET TBLPROPERTIES` should respect the ifExists flag. -case AlterTableUnsetProperties(table: ResolvedTable, keys, _) => - val changes = keys.map(key => TableChange.removeProperty(key)) +case AlterTableUnsetProperties(table: ResolvedTable, keys, ifExists) => + val changes = keys.map(key => TableChange.removeProperty(key, ifExists)) Review comment: Another approach is to do the existence check here instead of changing the public APIs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r571342363 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/v2/DataSourceV2Strategy.scala ## @@ -437,9 +437,8 @@ class DataSourceV2Strategy(session: SparkSession) extends Strategy with Predicat }.toSeq AlterTableExec(table.catalog, table.identifier, changes) :: Nil -// TODO: v2 `UNSET TBLPROPERTIES` should respect the ifExists flag. -case AlterTableUnsetProperties(table: ResolvedTable, keys, _) => - val changes = keys.map(key => TableChange.removeProperty(key)) +case AlterTableUnsetProperties(table: ResolvedTable, keys, ifExists) => + val changes = keys.map(key => TableChange.removeProperty(key, ifExists)) Review comment: Another approach is do the existence check here instead of changing the public APIs. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774380337 **[Test build #134949 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134949/testReport)** for PR 24559 at commit [`15e127f`](https://github.com/apache/spark/commit/15e127f67b50f1150a5ec2a5ebf7a89dc5aa4da4). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774379652 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134941/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774379653 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134947/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774379654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134946/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774379654 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134946/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774379653 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134947/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774379652 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134941/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
imback82 commented on a change in pull request #31494:
URL: https://github.com/apache/spark/pull/31494#discussion_r571282171
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
##
@@ -116,7 +116,11 @@ private[sql] object CatalogV2Util {
newProperties.put(set.property, set.value)
case unset: RemoveProperty =>
-newProperties.remove(unset.property)
+val prop = unset.property
+if (!unset.ifExists && !properties.containsKey(prop) && prop !=
TableCatalog.PROP_COMMENT) {
Review comment:
This aligns with:
https://github.com/apache/spark/blob/e614f34c7a538b1f2c59616689eaea95af85fd54/sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala#L307-L311
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
imback82 commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774375335 cc @cloud-fan @MaxGekk This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336759 **[Test build #134946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134946/testReport)** for PR 31493 at commit [`233ec5a`](https://github.com/apache/spark/commit/233ec5a9b76d9471463f595c2aa9064ff76cb325). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774290518 **[Test build #134941 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134941/testReport)** for PR 31494 at commit [`0065170`](https://github.com/apache/spark/commit/0065170f24a2ff00cfee920e8fd31b0574f55ac7). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774371375 **[Test build #134946 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134946/testReport)** for PR 31493 at commit [`233ec5a`](https://github.com/apache/spark/commit/233ec5a9b76d9471463f595c2aa9064ff76cb325). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774371350 **[Test build #134941 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134941/testReport)** for PR 31494 at commit [`0065170`](https://github.com/apache/spark/commit/0065170f24a2ff00cfee920e8fd31b0574f55ac7). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774370078 **[Test build #134947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134947/testReport)** for PR 24559 at commit [`86c6ce7`](https://github.com/apache/spark/commit/86c6ce7483b701b43913e43948edf005cfe3dcab). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774371188 **[Test build #134947 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134947/testReport)** for PR 24559 at commit [`86c6ce7`](https://github.com/apache/spark/commit/86c6ce7483b701b43913e43948edf005cfe3dcab). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774370078 **[Test build #134947 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134947/testReport)** for PR 24559 at commit [`86c6ce7`](https://github.com/apache/spark/commit/86c6ce7483b701b43913e43948edf005cfe3dcab). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774368595 **[Test build #134948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134948/testReport)** for PR 24559 at commit [`8f1084e`](https://github.com/apache/spark/commit/8f1084eba9a122b25e7f0d8b6649a9e77eae30ba). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369530 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134948/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369530 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134948/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369521 **[Test build #134948 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134948/testReport)** for PR 24559 at commit [`8f1084e`](https://github.com/apache/spark/commit/8f1084eba9a122b25e7f0d8b6649a9e77eae30ba). * This patch **fails to build**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369327 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39531/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774369327 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39531/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
SparkQA commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774368595 **[Test build #134948 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134948/testReport)** for PR 24559 at commit [`8f1084e`](https://github.com/apache/spark/commit/8f1084eba9a122b25e7f0d8b6649a9e77eae30ba). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774367636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39528/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins removed a comment on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774367634 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39530/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774367635 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39529/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #24559: [SPARK-27658][SQL] Add FunctionCatalog API
AmplabJenkins commented on pull request #24559: URL: https://github.com/apache/spark/pull/24559#issuecomment-774367634 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39530/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774367636 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39528/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774367635 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39529/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #30115: [WIP][SPARK-33200][CORE][K8S] Verify all shuffle blocks migrated in K8s integration testing
github-actions[bot] commented on pull request #30115: URL: https://github.com/apache/spark/pull/30115#issuecomment-774362521 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #28938: [WIP][SPARK-32118][SQL] Use fine-grained read write lock for each database in HiveExternalCatalog
github-actions[bot] closed pull request #28938: URL: https://github.com/apache/spark/pull/28938 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] github-actions[bot] commented on pull request #29677: [SPARK-32820][SQL] Remove redundant shuffle exchanges inserted by EnsureRequirements
github-actions[bot] commented on pull request #29677: URL: https://github.com/apache/spark/pull/29677#issuecomment-774362528 We're closing this PR because it hasn't been updated in a while. This isn't a judgement on the merit of the PR in any way. It's just a way of keeping the PR queue manageable. If you'd like to revive this PR, please reopen it and ask a committer to remove the Stale tag! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] github-actions[bot] closed pull request #30158: [SPARK-33249][CORE][UI] Add status plugin for live application
github-actions[bot] closed pull request #30158: URL: https://github.com/apache/spark/pull/30158 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774359284 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39529/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774357618 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39528/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774353696 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
AmplabJenkins removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774353694 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134943/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read
AmplabJenkins removed a comment on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774353695 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134940/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read
AmplabJenkins commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774353695 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134940/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
AmplabJenkins commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774353694 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134943/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774353697 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774352078 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39529/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774350161 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39528/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774346760 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39527/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
SparkQA removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774295061 **[Test build #134943 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://github.com/apache/spark/commit/d37289da58c289e3327c5e2d3ae611d4de647954). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774345612 **[Test build #134943 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134943/testReport)** for PR 31450 at commit [`d37289d`](https://github.com/apache/spark/commit/d37289da58c289e3327c5e2d3ae611d4de647954). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774340507 **[Test build #134945 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134945/testReport)** for PR 31494 at commit [`dbbde92`](https://github.com/apache/spark/commit/dbbde92612ff4084905465174d027f8f13100679). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType
AmplabJenkins removed a comment on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774339535 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134939/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType
AmplabJenkins commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774339535 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134939/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType
SparkQA removed a comment on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774230546 **[Test build #134939 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134939/testReport)** for PR 31491 at commit [`222be6e`](https://github.com/apache/spark/commit/222be6ea3e76bbd471dba88bdee6bc173826d6dc). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31491: [SPARK-34379][SQL] Map JDBC RowID to StringType rather than LongType
SparkQA commented on pull request #31491: URL: https://github.com/apache/spark/pull/31491#issuecomment-774338901 **[Test build #134939 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134939/testReport)** for PR 31491 at commit [`222be6e`](https://github.com/apache/spark/commit/222be6ea3e76bbd471dba88bdee6bc173826d6dc). * This patch passes all tests. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774296346 **[Test build #134942 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134942/testReport)** for PR 31493 at commit [`54e0bce`](https://github.com/apache/spark/commit/54e0bce5e798a9d98c779f7dc775d37c1ab2a425). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774338270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134942/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774338270 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134942/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774337794 **[Test build #134942 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134942/testReport)** for PR 31493 at commit [`54e0bce`](https://github.com/apache/spark/commit/54e0bce5e798a9d98c779f7dc775d37c1ab2a425). * This patch **fails Spark unit tests**. * This patch merges cleanly. * This patch adds no public classes. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336759 **[Test build #134946 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134946/testReport)** for PR 31493 at commit [`233ec5a`](https://github.com/apache/spark/commit/233ec5a9b76d9471463f595c2aa9064ff76cb325). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774336633 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39527/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
AmplabJenkins removed a comment on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774336116 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39526/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336114 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39525/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774336115 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39524/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
AmplabJenkins commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774336116 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39526/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774336114 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39525/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774336115 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39524/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774322983 **[Test build #134944 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134944/testReport)** for PR 31494 at commit [`c757b6d`](https://github.com/apache/spark/commit/c757b6d7f2f8590fbe5f33e3a3c86d1968a4bb65). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
imback82 commented on a change in pull request #31494:
URL: https://github.com/apache/spark/pull/31494#discussion_r571282171
##
File path:
sql/catalyst/src/main/scala/org/apache/spark/sql/connector/catalog/CatalogV2Util.scala
##
@@ -116,7 +116,11 @@ private[sql] object CatalogV2Util {
newProperties.put(set.property, set.value)
case unset: RemoveProperty =>
-newProperties.remove(unset.property)
+val prop = unset.property
+if (!unset.ifExists && !properties.containsKey(prop) && prop !=
TableCatalog.PROP_COMMENT) {
Review comment:
This is from:
https://github.com/apache/spark/blob/e614f34c7a538b1f2c59616689eaea95af85fd54/sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala#L307-L311
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
imback82 commented on a change in pull request #31494:
URL: https://github.com/apache/spark/pull/31494#discussion_r571240319
##
File path:
sql/catalyst/src/main/java/org/apache/spark/sql/connector/catalog/TableChange.java
##
@@ -64,7 +64,20 @@ static TableChange setProperty(String property, String
value) {
* @return a TableChange for the addition
*/
static TableChange removeProperty(String property) {
Review comment:
Can I directly update this API or do we need to keep the compatibility?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774319136 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39524/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774318369 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39525/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774318333 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39526/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] holdenk commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks
holdenk commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-774316948 I think the Kafka failures are unrelated. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31492: [SPARK-34346][CORE][SQL][3.0] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally
AmplabJenkins removed a comment on pull request #31492: URL: https://github.com/apache/spark/pull/31492#issuecomment-774314203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134938/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins removed a comment on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read
AmplabJenkins removed a comment on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774314202 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39523/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31492: [SPARK-34346][CORE][SQL][3.0] io.file.buffer.size set by spark.buffer.size will override by loading hive-site.xml accidentally may caus
AmplabJenkins commented on pull request #31492: URL: https://github.com/apache/spark/pull/31492#issuecomment-774314203 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134938/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] AmplabJenkins commented on pull request #31489: [SPARK-34377][SQL] Add new parquet datasource options to control datetime rebasing in read
AmplabJenkins commented on pull request #31489: URL: https://github.com/apache/spark/pull/31489#issuecomment-774314202 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39523/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command
SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-774311918 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39524/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
[GitHub] [spark] SparkQA commented on pull request #31450: [WIP][SPARK-33763] Add metrics for better tracking of dynamic allocation
SparkQA commented on pull request #31450: URL: https://github.com/apache/spark/pull/31450#issuecomment-774306946 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39526/ This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org - To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
