[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-77757 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135046/

[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-77761 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39630/

[GitHub] [spark] AmplabJenkins commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

AmplabJenkins commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-77759 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39631/

[GitHub] [spark] MaxGekk commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

MaxGekk commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775553799 jenkins, retest this, please This is an automated message from the Apache Git Service. To respond to the

[GitHub] [spark] SparkQA commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

SparkQA commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775552827 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39632/

[GitHub] [spark] HyukjinKwon commented on pull request #31496: [SPARK-34384][CORE] API cleanup for ResourceProfile

HyukjinKwon commented on pull request #31496: URL: https://github.com/apache/spark/pull/31496#issuecomment-775543510 I don't think blocks RC I am preparing now. Same here too. Hope we can do this earlier next time. @tgravescs, though, some of them like

[GitHub] [spark] SparkQA removed a comment on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

SparkQA removed a comment on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775536734 **[Test build #135049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135049/testReport)** for PR 31445 at commit

[GitHub] [spark] SparkQA commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

SparkQA commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775541886 **[Test build #135049 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135049/testReport)** for PR 31445 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775470550 **[Test build #135046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135046/testReport)** for PR 31493 at commit

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775537415 **[Test build #135046 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135046/testReport)** for PR 31493 at commit

[GitHub] [spark] SparkQA commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

SparkQA commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775537236 **[Test build #135050 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135050/testReport)** for PR 31249 at commit

[GitHub] [spark] SparkQA commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

SparkQA commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775536734 **[Test build #135049 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135049/testReport)** for PR 31445 at commit

[GitHub] [spark] SparkQA commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

SparkQA commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775536471 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39631/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775528864 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39629/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775528858 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775528860 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135043/

[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775528859 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775528864 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39629/

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775528860 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135043/

[GitHub] [spark] SparkQA removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775390813 **[Test build #135043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135043/testReport)** for PR 31494 at commit

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775526942 **[Test build #135043 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135043/testReport)** for PR 31494 at commit

[GitHub] [spark] SparkQA commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

SparkQA commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775525435 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39631/

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775517184 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39628/

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775514042 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39629/

[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775503416 **[Test build #135047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135047/testReport)** for PR 31529 at commit

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775510620 **[Test build #135047 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135047/testReport)** for PR 31529 at commit

[GitHub] [spark] SparkQA commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

SparkQA commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775509993 **[Test build #135048 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135048/testReport)** for PR 31510 at commit

[GitHub] [spark] holdenk commented on pull request #31445: [SPARK-34334][K8S] Correctly identify timed out pending pod requests as excess request

holdenk commented on pull request #31445: URL: https://github.com/apache/spark/pull/31445#issuecomment-775509482 LGTM pending CI. Jenkins failure is unrelated but to be safe, Jenkins retest this please. This is an

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775503100 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135044/

[GitHub] [spark] SparkQA removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775386321 **[Test build #135044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135044/testReport)** for PR 31494 at commit

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775503416 **[Test build #135047 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135047/testReport)** for PR 31529 at commit

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775503100 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135044/

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775502521 **[Test build #135044 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135044/testReport)** for PR 31494 at commit

[GitHub] [spark] viirya commented on a change in pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

viirya commented on a change in pull request #31476: URL: https://github.com/apache/spark/pull/31476#discussion_r572414083 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/Scan.java ## @@ -102,4 +102,13 @@ default MicroBatchStream

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775497614 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39629/

[GitHub] [spark] holdenk commented on pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on pull request #31249: URL: https://github.com/apache/spark/pull/31249#issuecomment-775491530 Thanks @Ngone51 :) This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] rdblue commented on pull request #31475: [SPARK-34360][SQL] Support table truncation by v2 Table Catalogs

rdblue commented on pull request #31475: URL: https://github.com/apache/spark/pull/31475#issuecomment-775490751 @MaxGekk, can you share the use case that you have for this? You mentioned truncation-specific optimizations. I think working with concrete use cases is usually a good idea. If

[GitHub] [spark] srowen commented on pull request #31510: [SPARK-34395][SQL]Clean up unused code for code simplifications.

srowen commented on pull request #31510: URL: https://github.com/apache/spark/pull/31510#issuecomment-775483346 Jenkins test this please This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] srowen commented on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

srowen commented on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-775482881 Merged to master This is an automated message from the Apache Git Service. To respond to the message, please log

[GitHub] [spark] srowen closed pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

srowen closed pull request #31484: URL: https://github.com/apache/spark/pull/31484 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [spark] dongjoon-hyun commented on pull request #31527: [MINOR] Add a note about pip installation test in RC for release vote template

dongjoon-hyun commented on pull request #31527: URL: https://github.com/apache/spark/pull/31527#issuecomment-775482765 +1, late LGTM. This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [spark] sunchao commented on a change in pull request #31524: [SPARK-34401][SQL][DOCS] Update docs about altering cached tables/views

sunchao commented on a change in pull request #31524: URL: https://github.com/apache/spark/pull/31524#discussion_r572383191 ## File path: docs/sql-ref-syntax-ddl-drop-table.md ## @@ -26,6 +26,8 @@ if the table is not `EXTERNAL` table. If the table is not present it throws an

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775468802 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135045/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775468806 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135042/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

AmplabJenkins removed a comment on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775468804 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

AmplabJenkins removed a comment on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-775468801 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135039/

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775470550 **[Test build #135046 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135046/testReport)** for PR 31493 at commit

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775469874 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39628/

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775468806 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135042/

[GitHub] [spark] AmplabJenkins commented on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

AmplabJenkins commented on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-775468801 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135039/

[GitHub] [spark] AmplabJenkins commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

AmplabJenkins commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775468804 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

AmplabJenkins commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775468802 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135045/

[GitHub] [spark] SparkQA removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775355318 **[Test build #135042 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135042/testReport)** for PR 31493 at commit

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775457133 **[Test build #135042 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135042/testReport)** for PR 31493 at commit

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775452893 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39627/

[GitHub] [spark] SparkQA removed a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA removed a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775436704 **[Test build #135045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135045/testReport)** for PR 31529 at commit

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775449509 **[Test build #135045 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135045/testReport)** for PR 31529 at commit

[GitHub] [spark] SparkQA removed a comment on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

SparkQA removed a comment on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-775264831 **[Test build #135039 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135039/testReport)** for PR 31484 at commit

[GitHub] [spark] SparkQA commented on pull request #31484: [SPARK-34374][SQL][DSTREAM] Use standard methods to extract keys or values from a Map

SparkQA commented on pull request #31484: URL: https://github.com/apache/spark/pull/31484#issuecomment-775440637 **[Test build #135039 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135039/testReport)** for PR 31484 at commit

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572353356 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -1141,6 +1141,36 @@ trait AlterTableTests extends

[GitHub] [spark] SparkQA commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

SparkQA commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775436704 **[Test build #135045 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135045/testReport)** for PR 31529 at commit

[GitHub] [spark] MaxGekk edited a comment on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

MaxGekk edited a comment on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775424148 @cloud-fan @gengliangwang @HyukjinKwon Could you review this PR, please. This is an automated message

[GitHub] [spark] sunchao commented on pull request #31462: [SPARK-34347][SQL] CatalogImpl.uncacheTable should invalidate in cascade for temp views

sunchao commented on pull request #31462: URL: https://github.com/apache/spark/pull/31462#issuecomment-775429482 Any more comments on this @viirya ? cc @dongjoon-hyun too. This is an automated message from the Apache Git

[GitHub] [spark] MaxGekk commented on pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

MaxGekk commented on pull request #31529: URL: https://github.com/apache/spark/pull/31529#issuecomment-775424148 @cloud-fan @HyukjinKwon Could you review this PR, please. This is an automated message from the Apache Git

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

AmplabJenkins removed a comment on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-775419237 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135037/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775419235 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775420690 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39627/

[GitHub] [spark] AmplabJenkins commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

AmplabJenkins commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-775419237 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135037/

[GitHub] [spark] AmplabJenkins commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775419236 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] MaxGekk opened a new pull request #31529: [SPARK-34404][SQL] Add new Avro datasource options to control datetime rebasing in read

MaxGekk opened a new pull request #31529: URL: https://github.com/apache/spark/pull/31529 ### What changes were proposed in this pull request? In the PR, I propose new option `datetimeRebaseMode` for the Avro datasource. The option influences on loading ancient dates and timestamps

[GitHub] [spark] cloud-fan commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

cloud-fan commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572333847 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -1141,6 +1141,36 @@ trait AlterTableTests extends

[GitHub] [spark] cloud-fan commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

cloud-fan commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r57265 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -1141,6 +1141,36 @@ trait AlterTableTests extends

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775401954 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39624/

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r572326065 ## File path: core/src/test/scala/org/apache/spark/scheduler/HealthTrackerSuite.scala ## @@ -554,6 +554,50 @@ class HealthTrackerSuite extends

[GitHub] [spark] SparkQA removed a comment on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA removed a comment on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-775221906 **[Test build #135037 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135037/testReport)** for PR 31378 at commit

[GitHub] [spark] SparkQA commented on pull request #31378: [SPARK-34240][SQL] Unify output of `SHOW TBLPROPERTIES` clause's output attribute's schema and ExprID

SparkQA commented on pull request #31378: URL: https://github.com/apache/spark/pull/31378#issuecomment-775398487 **[Test build #135037 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135037/testReport)** for PR 31378 at commit

[GitHub] [spark] holdenk commented on a change in pull request #31249: [SPARK-34104][SPARK-34105][CORE][K8S] Maximum decommissioning time & allow decommissioning for excludes

holdenk commented on a change in pull request #31249: URL: https://github.com/apache/spark/pull/31249#discussion_r572321583 ## File path: core/src/main/scala/org/apache/spark/internal/config/package.scala ## @@ -1942,6 +1950,15 @@ package object config {

[GitHub] [spark] SparkQA removed a comment on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775231660 **[Test build #135035 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135035/testReport)** for PR 31485 at commit

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775392322 **[Test build #135035 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135035/testReport)** for PR 31485 at commit

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775390813 **[Test build #135043 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135043/testReport)** for PR 31494 at commit

[GitHub] [spark] SparkQA commented on pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

SparkQA commented on pull request #31494: URL: https://github.com/apache/spark/pull/31494#issuecomment-775386321 **[Test build #135044 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135044/testReport)** for PR 31494 at commit

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins removed a comment on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775380295 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39625/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

AmplabJenkins removed a comment on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-775380293 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135033/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775380294 This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572305204 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -1141,6 +1141,36 @@ trait AlterTableTests extends

[GitHub] [spark] AmplabJenkins commented on pull request #31258: [SPARK-34168] [SQL] Support DPP in AQE when the join is Broadcast hash join at the beginning

AmplabJenkins commented on pull request #31258: URL: https://github.com/apache/spark/pull/31258#issuecomment-775380293 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/135033/

[GitHub] [spark] AmplabJenkins commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

AmplabJenkins commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775380294 This is an automated message from the Apache Git Service. To respond to the message, please log on to

[GitHub] [spark] AmplabJenkins commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

AmplabJenkins commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775380295 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/39625/

[GitHub] [spark] SparkQA removed a comment on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA removed a comment on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775315610 **[Test build #135041 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135041/testReport)** for PR 31485 at commit

[GitHub] [spark] SparkQA commented on pull request #31485: [SPARK-34137][SQL] Update suquery's stats when build LogicalPlan's stats

SparkQA commented on pull request #31485: URL: https://github.com/apache/spark/pull/31485#issuecomment-775377002 **[Test build #135041 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/135041/testReport)** for PR 31485 at commit

[GitHub] [spark] cloud-fan commented on a change in pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

cloud-fan commented on a change in pull request #31476: URL: https://github.com/apache/spark/pull/31476#discussion_r572300669 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/SupportsReportMetrics.java ## @@ -0,0 +1,39 @@ +/* + * Licensed to the Apache

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775376682 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39625/

[GitHub] [spark] cloud-fan commented on a change in pull request #31476: [SPARK-34366][SQL] Add interface for DS v2 metrics

cloud-fan commented on a change in pull request #31476: URL: https://github.com/apache/spark/pull/31476#discussion_r572298710 ## File path: sql/catalyst/src/main/java/org/apache/spark/sql/connector/read/Scan.java ## @@ -102,4 +102,13 @@ default MicroBatchStream

[GitHub] [spark] SparkQA commented on pull request #31493: [SPARK-34363][CORE] Add an option for limiting storage for migrated shuffle blocks

SparkQA commented on pull request #31493: URL: https://github.com/apache/spark/pull/31493#issuecomment-775374134 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/39625/

[GitHub] [spark] MaxGekk commented on pull request #31475: [SPARK-34360][SQL] Support table truncation by v2 Table Catalogs

MaxGekk commented on pull request #31475: URL: https://github.com/apache/spark/pull/31475#issuecomment-775373033 > ... why is this necessary instead of deleting from the table or overwriting everything with no new records? 1. By emulating table truncation via the insertion of no

[GitHub] [spark] cloud-fan commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

cloud-fan commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572293623 ## File path: sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala ## @@ -1141,6 +1141,36 @@ trait AlterTableTests extends

[GitHub] [spark] cloud-fan commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

cloud-fan commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572292880 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableProperties.scala ## @@ -0,0 +1,43 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

cloud-fan commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572292383 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableProperties.scala ## @@ -0,0 +1,43 @@ +/* + * Licensed to

[GitHub] [spark] imback82 commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

imback82 commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572291602 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableProperties.scala ## @@ -0,0 +1,43 @@ +/* + * Licensed to

[GitHub] [spark] cloud-fan commented on a change in pull request #31494: [SPARK-34380][SQL] Support ifExists for ALTER TABLE ... UNSET TBLPROPERTIES for v2 command

cloud-fan commented on a change in pull request #31494: URL: https://github.com/apache/spark/pull/31494#discussion_r572291464 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/ResolveTableProperties.scala ## @@ -0,0 +1,43 @@ +/* + * Licensed to

<    1   2   3   4   5   6   7   8   >