[GitHub] [spark] MaxGekk commented on a change in pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

MaxGekk commented on a change in pull request #31229: URL: https://github.com/apache/spark/pull/31229#discussion_r559977670 ## File path: sql/core/src/test/scala/org/apache/spark/sql/execution/command/v2/AlterTableAddPartitionSuite.scala ## @@ -53,4 +53,19 @@ class AlterTableA

[GitHub] [spark] SparkQA commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

SparkQA commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762674060 **[Test build #134219 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134219/testReport)** for PR 31229 at commit [`3df29aa`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

SparkQA commented on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762674434 **[Test build #134218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134218/testReport)** for PR 31242 at commit [`0e5e7de`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

SparkQA commented on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762676991 Kubernetes integration test status success URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38802/ ---

[GitHub] [spark] SparkQA commented on pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

SparkQA commented on pull request #31240: URL: https://github.com/apache/spark/pull/31240#issuecomment-762678690 **[Test build #134212 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134212/testReport)** for PR 31240 at commit [`0575c51`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

SparkQA removed a comment on pull request #31240: URL: https://github.com/apache/spark/pull/31240#issuecomment-762589102 **[Test build #134212 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134212/testReport)** for PR 31240 at commit [`0575c51`](https://gi

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r559988554 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r559988554 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] viirya commented on a change in pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

viirya commented on a change in pull request #31107: URL: https://github.com/apache/spark/pull/31107#discussion_r559990533 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ## @@ -417,17 +411,25 @@ class CatalogImpl(sparkSession: SparkSessio

[GitHub] [spark] viirya commented on a change in pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

viirya commented on a change in pull request #31107: URL: https://github.com/apache/spark/pull/31107#discussion_r559991821 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ## @@ -417,17 +411,25 @@ class CatalogImpl(sparkSession: SparkSessio

[GitHub] [spark] SparkQA commented on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

SparkQA commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-762684895 **[Test build #134214 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134214/testReport)** for PR 30974 at commit [`1b00b85`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

SparkQA removed a comment on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-762588959 **[Test build #134214 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134214/testReport)** for PR 30974 at commit [`1b00b85`](https://gi

[GitHub] [spark] viirya commented on a change in pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

viirya commented on a change in pull request #31107: URL: https://github.com/apache/spark/pull/31107#discussion_r559991821 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ## @@ -417,17 +411,25 @@ class CatalogImpl(sparkSession: SparkSessio

[GitHub] [spark] SparkQA commented on pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

SparkQA commented on pull request #31107: URL: https://github.com/apache/spark/pull/31107#issuecomment-762690616 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38801/ ---

[GitHub] [spark] SparkQA commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

SparkQA commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762693055 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38804/ -

[GitHub] [spark] viirya commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

viirya commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560003976 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSpark

[GitHub] [spark] cloud-fan commented on a change in pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

cloud-fan commented on a change in pull request #31107: URL: https://github.com/apache/spark/pull/31107#discussion_r560004830 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ## @@ -417,17 +411,25 @@ class CatalogImpl(sparkSession: SparkSes

[GitHub] [spark] viirya commented on a change in pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

viirya commented on a change in pull request #31107: URL: https://github.com/apache/spark/pull/31107#discussion_r560007174 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ## @@ -417,17 +411,25 @@ class CatalogImpl(sparkSession: SparkSessio

[GitHub] [spark] AmplabJenkins commented on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins commented on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762699388 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38802/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

AmplabJenkins commented on pull request #31240: URL: https://github.com/apache/spark/pull/31240#issuecomment-762699395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134212/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins commented on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762699393 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38803/ -

[GitHub] [spark] AmplabJenkins commented on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

AmplabJenkins commented on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762699390 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38805/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

AmplabJenkins removed a comment on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762699390 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38805/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

AmplabJenkins removed a comment on pull request #31107: URL: https://github.com/apache/spark/pull/31107#issuecomment-762699394 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38801/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

AmplabJenkins removed a comment on pull request #31240: URL: https://github.com/apache/spark/pull/31240#issuecomment-762699395 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134212/ -

[GitHub] [spark] AmplabJenkins commented on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

AmplabJenkins commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-762699387 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134214/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

AmplabJenkins removed a comment on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-762699387 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134214/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins removed a comment on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762699388 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38802/

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins removed a comment on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762699393 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38803/

[GitHub] [spark] AmplabJenkins commented on pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

AmplabJenkins commented on pull request #31107: URL: https://github.com/apache/spark/pull/31107#issuecomment-762699394 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38801/ -

[GitHub] [spark] HyukjinKwon commented on pull request #31199: [SPARK-34130][SQL] Impove preformace for char varchar padding and length check with StaticInvoke

HyukjinKwon commented on pull request #31199: URL: https://github.com/apache/spark/pull/31199#issuecomment-762701071 Yes, I think it's fine to port back. It's not very common case but we can think it's like a perf regression for varchar cases compared to Spark 3.1 in a way. -

[GitHub] [spark] cloud-fan commented on pull request #31199: [SPARK-34130][SQL] Impove preformace for char varchar padding and length check with StaticInvoke

cloud-fan commented on pull request #31199: URL: https://github.com/apache/spark/pull/31199#issuecomment-762702242 thanks, merging to master/3.1! This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [spark] cloud-fan closed pull request #31199: [SPARK-34130][SQL] Impove preformace for char varchar padding and length check with StaticInvoke

cloud-fan closed pull request #31199: URL: https://github.com/apache/spark/pull/31199 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] SparkQA commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

SparkQA commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762703010 Kubernetes integration test status failure URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38804/ ---

[GitHub] [spark] AmplabJenkins commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

AmplabJenkins commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762703029 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38804/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

AmplabJenkins removed a comment on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762703029 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder-K8s/38804/

[GitHub] [spark] SparkQA commented on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

SparkQA commented on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762704238 **[Test build #134220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134220/testReport)** for PR 31228 at commit [`bfa56b5`](https://github.com

[GitHub] [spark] HyukjinKwon commented on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

HyukjinKwon commented on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762706105 Merged to branch-2.4. This is an automated message from the Apache Git Service. To respond to the message, p

[GitHub] [spark] HyukjinKwon closed pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

HyukjinKwon closed pull request #31242: URL: https://github.com/apache/spark/pull/31242 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] yaooqinn commented on pull request #31199: [SPARK-34130][SQL] Impove preformace for char varchar padding and length check with StaticInvoke

yaooqinn commented on pull request #31199: URL: https://github.com/apache/spark/pull/31199#issuecomment-762706758 thanks for merging This is an automated message from the Apache Git Service. To respond to the message, please

[GitHub] [spark] HyukjinKwon closed pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

HyukjinKwon closed pull request #31241: URL: https://github.com/apache/spark/pull/31241 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

[GitHub] [spark] SparkQA commented on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

SparkQA commented on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762706986 **[Test build #134220 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134220/testReport)** for PR 31228 at commit [`bfa56b5`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

SparkQA removed a comment on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762704238 **[Test build #134220 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134220/testReport)** for PR 31228 at commit [`bfa56b5`](https://gi

[GitHub] [spark] HyukjinKwon commented on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

HyukjinKwon commented on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762707745 Merged to branch-3.1 and branch-3.0. This is an automated message from the Apache Git Service. To respond to

[GitHub] [spark] SparkQA commented on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

SparkQA commented on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762712288 **[Test build #134217 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134217/testReport)** for PR 31241 at commit [`ea081d5`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

SparkQA removed a comment on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762649516 **[Test build #134217 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134217/testReport)** for PR 31241 at commit [`ea081d5`](https://gi

[GitHub] [spark] iemejia commented on a change in pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

iemejia commented on a change in pull request #31232: URL: https://github.com/apache/spark/pull/31232#discussion_r560029136 ## File path: .github/PULL_REQUEST_TEMPLATE ## @@ -7,7 +7,7 @@ Thanks for sending a pull request! Here are some tips for you: 5. Please write your PR

[GitHub] [spark] cloud-fan commented on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

cloud-fan commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-762721109 thanks, merging to master! This is an automated message from the Apache Git Service. To respond to the message

[GitHub] [spark] cloud-fan closed pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

cloud-fan closed pull request #30974: URL: https://github.com/apache/spark/pull/30974 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] iemejia commented on a change in pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

iemejia commented on a change in pull request #31232: URL: https://github.com/apache/spark/pull/31232#discussion_r560037772 ## File path: dev/deps/spark-deps-hadoop-2.7-hive-2.3 ## @@ -22,9 +22,10 @@ arrow-memory-netty/2.0.0//arrow-memory-netty-2.0.0.jar arrow-vector/2.0.0//ar

[GitHub] [spark] iemejia commented on a change in pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

iemejia commented on a change in pull request #31232: URL: https://github.com/apache/spark/pull/31232#discussion_r560038223 ## File path: dev/deps/spark-deps-hadoop-2.7-hive-2.3 ## @@ -125,6 +126,7 @@ jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar jakarta.xml.bind-api/2.

[GitHub] [spark] iemejia commented on a change in pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

iemejia commented on a change in pull request #31232: URL: https://github.com/apache/spark/pull/31232#discussion_r560040952 ## File path: dev/deps/spark-deps-hadoop-2.7-hive-2.3 ## @@ -233,12 +235,13 @@ super-csv/2.2.0//super-csv-2.2.0.jar threeten-extra/1.5.0//threeten-extra-

[GitHub] [spark] cloud-fan commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

cloud-fan commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762724582 GA passed, merging to master! This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [spark] cloud-fan closed pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

cloud-fan closed pull request #31229: URL: https://github.com/apache/spark/pull/31229 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go t

[GitHub] [spark] iemejia commented on a change in pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

iemejia commented on a change in pull request #31232: URL: https://github.com/apache/spark/pull/31232#discussion_r560038223 ## File path: dev/deps/spark-deps-hadoop-2.7-hive-2.3 ## @@ -125,6 +126,7 @@ jakarta.ws.rs-api/2.1.6//jakarta.ws.rs-api-2.1.6.jar jakarta.xml.bind-api/2.

[GitHub] [spark] cloud-fan commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

cloud-fan commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762725739 @MaxGekk let's also have a backport PR for 3.1 (the partition API v2 was added in 3.1), thanks! This is an au

[GitHub] [spark] cloud-fan commented on a change in pull request #31107: [SPARK-34052][SQL] A cached view should become invalid after the source table is dropped

cloud-fan commented on a change in pull request #31107: URL: https://github.com/apache/spark/pull/31107#discussion_r560047601 ## File path: sql/core/src/main/scala/org/apache/spark/sql/internal/CatalogImpl.scala ## @@ -417,17 +411,25 @@ class CatalogImpl(sparkSession: SparkSes

[GitHub] [spark] SparkQA commented on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

SparkQA commented on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762732359 **[Test build #134218 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134218/testReport)** for PR 31242 at commit [`0e5e7de`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

SparkQA removed a comment on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762674434 **[Test build #134218 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134218/testReport)** for PR 31242 at commit [`0e5e7de`](https://gi

[GitHub] [spark] yaooqinn commented on a change in pull request #31012: [SPARK-34083][SQL] Using TPCDS original definitions for char/varchar columns

yaooqinn commented on a change in pull request #31012: URL: https://github.com/apache/spark/pull/31012#discussion_r560050954 ## File path: sql/core/src/test/resources/tpcds-plan-stability/approved-plans-modified/q10.sf100/simplified.txt ## @@ -1,81 +1,100 @@ TakeOrderedAndPro

[GitHub] [spark] MaxGekk commented on pull request #31229: [SPARK-34149][SQL] Refresh cache in v2 `ALTER TABLE .. ADD PARTITION`

MaxGekk commented on pull request #31229: URL: https://github.com/apache/spark/pull/31229#issuecomment-762733140 @cloud-fan Since https://github.com/apache/spark/pull/31216 wasn't merged to 3.1, I can backport this only w/o test. Is it ok? -

[GitHub] [spark] yaooqinn commented on a change in pull request #31012: [SPARK-34083][SQL] Using TPCDS original definitions for char/varchar columns

yaooqinn commented on a change in pull request #31012: URL: https://github.com/apache/spark/pull/31012#discussion_r560052173 ## File path: sql/core/src/test/resources/tpcds-plan-stability/approved-plans-modified/q10.sf100/explain.txt ## @@ -200,87 +207,115 @@ Left keys [1]: [c

[GitHub] [spark] AmplabJenkins commented on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

AmplabJenkins commented on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762734082 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134220/ -

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560052568 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] AmplabJenkins commented on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins commented on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762734081 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134218/ -

[GitHub] [spark] hddong commented on a change in pull request #30684: [SPARK-33720][K8S] Support submit to k8s only with token

hddong commented on a change in pull request #30684: URL: https://github.com/apache/spark/pull/30684#discussion_r560052449 ## File path: resource-managers/kubernetes/core/src/main/scala/org/apache/spark/deploy/k8s/Config.scala ## @@ -392,6 +392,13 @@ private[spark] object Conf

[GitHub] [spark] AmplabJenkins commented on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins commented on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762734085 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134217/ -

[GitHub] [spark] MaxGekk commented on pull request #31216: [SPARK-34143][SQL][TESTS] Fix adding partitions to fully partitioned v2 tables

MaxGekk commented on pull request #31216: URL: https://github.com/apache/spark/pull/31216#issuecomment-762734088 Can we backport this to 3.1 to be able to backport other changes like https://github.com/apache/spark/pull/31229 ? -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31242: [SPARK-34153][SQL][2.4] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins removed a comment on pull request #31242: URL: https://github.com/apache/spark/pull/31242#issuecomment-762734081 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134218/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31228: [SPARK-33541][SQL] Group exception messages in catalyst/expressions

AmplabJenkins removed a comment on pull request #31228: URL: https://github.com/apache/spark/pull/31228#issuecomment-762734082 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134220/ -

[GitHub] [spark] AmplabJenkins removed a comment on pull request #31241: [SPARK-34153][SQL][3.1][3.0] Remove unused `getRawTable()` from `HiveExternalCatalog.alterPartitions()`

AmplabJenkins removed a comment on pull request #31241: URL: https://github.com/apache/spark/pull/31241#issuecomment-762734085 Refer to this link for build results (access rights to CI server needed): https://amplab.cs.berkeley.edu/jenkins//job/SparkPullRequestBuilder/134217/ -

[GitHub] [spark] yaooqinn commented on a change in pull request #31012: [SPARK-34083][SQL] Using TPCDS original definitions for char/varchar columns

yaooqinn commented on a change in pull request #31012: URL: https://github.com/apache/spark/pull/31012#discussion_r560053593 ## File path: sql/core/src/test/resources/tpcds-plan-stability/approved-plans-modified/q10.sf100/explain.txt ## @@ -200,87 +207,115 @@ Left keys [1]: [c

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560052568 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] hddong commented on pull request #30684: [SPARK-33720][K8S] Support submit to k8s only with token

hddong commented on pull request #30684: URL: https://github.com/apache/spark/pull/30684#issuecomment-762736450 @dongjoon-hyun : sorry for feedback so long time later, `insecure-skip-tls-verify` is a chose when have `.kubeconfig`. Use oauthToken, we can submit without `.kubeconfig`, and su

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560052568 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560052568 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560052568 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] hddong edited a comment on pull request #30684: [SPARK-33720][K8S] Support submit to k8s only with token

hddong edited a comment on pull request #30684: URL: https://github.com/apache/spark/pull/30684#issuecomment-762736450 @dongjoon-hyun : sorry for feedback so long time later, `insecure-skip-tls-verify` is a chose when have `.kubeconfig`. Use `oauthToken`, we can submit without `.kubeconfig

[GitHub] [spark] SparkQA commented on pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

SparkQA commented on pull request #31232: URL: https://github.com/apache/spark/pull/31232#issuecomment-762739409 **[Test build #134221 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134221/testReport)** for PR 31232 at commit [`6c6b137`](https://github.com

[GitHub] [spark] SparkQA commented on pull request #31012: [SPARK-34083][SQL] Using TPCDS original definitions for char/varchar columns

SparkQA commented on pull request #31012: URL: https://github.com/apache/spark/pull/31012#issuecomment-762739599 **[Test build #134222 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134222/testReport)** for PR 31012 at commit [`5817a66`](https://github.com

[GitHub] [spark] wangyum opened a new pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

wangyum opened a new pull request #31243: URL: https://github.com/apache/spark/pull/31243 ### What changes were proposed in this pull request? This pr add partition columns for TPCDS tables. The partition column is consistent with the [TPCDSTables](https://github.com/databricks/spar

[GitHub] [spark] hddong edited a comment on pull request #30684: [SPARK-33720][K8S] Support submit to k8s only with token

hddong edited a comment on pull request #30684: URL: https://github.com/apache/spark/pull/30684#issuecomment-762736450 @dongjoon-hyun : sorry for feedback so long time later, `insecure-skip-tls-verify` is a chose when have `.kubeconfig`. Use `oauthToken`, we can submit without `.kubeconfig

[GitHub] [spark] zhongyu09 commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

zhongyu09 commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560061600 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] wangyum commented on pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

wangyum commented on pull request #31243: URL: https://github.com/apache/spark/pull/31243#issuecomment-762744325 The actual change is: ```diff diff --git a/sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala b/sql/core/src/test/scala/org/apache/spark/sql/TPCDSBase.scala i

[GitHub] [spark] SparkQA commented on pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

SparkQA commented on pull request #31243: URL: https://github.com/apache/spark/pull/31243#issuecomment-762744524 **[Test build #134223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134223/testReport)** for PR 31243 at commit [`3fe1a93`](https://github.com

[GitHub] [spark] cloud-fan commented on pull request #31216: [SPARK-34143][SQL][TESTS] Fix adding partitions to fully partitioned v2 tables

cloud-fan commented on pull request #31216: URL: https://github.com/apache/spark/pull/31216#issuecomment-762745966 yea sure, as it's test-only. @MaxGekk can you open a PR? This is an automated message from the Apache Git Serv

[GitHub] [spark] nob13 opened a new pull request #31244: [WIP][SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

nob13 opened a new pull request #31244: URL: https://github.com/apache/spark/pull/31244 ### What changes were proposed in this pull request? Check SPARK_TESTING as lazy val to avoid slow down when there are many environment variables ### Why are the changes needed? If there are

[GitHub] [spark] wangyum commented on a change in pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

wangyum commented on a change in pull request #31240: URL: https://github.com/apache/spark/pull/31240#discussion_r560068147 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/BasicStatsPlanVisitor.scala ## @@ -57,7 +57,15 @@ o

[GitHub] [spark] cloud-fan commented on a change in pull request #31167: [SPARK-33933][SQL] Materialize BroadcastQueryStage first to avoid broadcast timeout in AQE

cloud-fan commented on a change in pull request #31167: URL: https://github.com/apache/spark/pull/31167#discussion_r560071651 ## File path: sql/core/src/main/scala/org/apache/spark/sql/execution/adaptive/AdaptiveSparkPlanExec.scala ## @@ -190,7 +191,36 @@ case class AdaptiveSp

[GitHub] [spark] ulysses-you commented on pull request #30974: [SPARK-33939][SQL] Make Column.named UnresolvedExtractValue use UnresolvedAlias to assign name

ulysses-you commented on pull request #30974: URL: https://github.com/apache/spark/pull/30974#issuecomment-762752049 thanks for merging ! This is an automated message from the Apache Git Service. To respond to the message, pl

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31244: [SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

HyukjinKwon commented on a change in pull request #31244: URL: https://github.com/apache/spark/pull/31244#discussion_r560076501 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -1939,7 +1939,15 @@ private[spark] object Utils extends Logging { * Indi

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31244: [SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

HyukjinKwon commented on a change in pull request #31244: URL: https://github.com/apache/spark/pull/31244#discussion_r560077870 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -1939,7 +1939,15 @@ private[spark] object Utils extends Logging { * Indi

[GitHub] [spark] HyukjinKwon commented on pull request #31244: [SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

HyukjinKwon commented on pull request #31244: URL: https://github.com/apache/spark/pull/31244#issuecomment-762756702 cc @dongjoon-hyun and @holdenk FYI. seems like this affects particularly K8S that automatically creates many envs according to the JIRA. ---

[GitHub] [spark] SparkQA commented on pull request #31232: [SPARK-27733][CORE] Upgrade Avro to version 1.10.1

SparkQA commented on pull request #31232: URL: https://github.com/apache/spark/pull/31232#issuecomment-762759975 Kubernetes integration test starting URL: https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder-K8s/38806/ -

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31244: [SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

HyukjinKwon commented on a change in pull request #31244: URL: https://github.com/apache/spark/pull/31244#discussion_r560082930 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -1939,7 +1939,15 @@ private[spark] object Utils extends Logging { * Indi

[GitHub] [spark] nob13 commented on a change in pull request #31244: [SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

nob13 commented on a change in pull request #31244: URL: https://github.com/apache/spark/pull/31244#discussion_r560083952 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -1939,7 +1939,15 @@ private[spark] object Utils extends Logging { * Indicates

[GitHub] [spark] AngersZhuuuu commented on a change in pull request #31240: [SPARK-34121][SQL] Intersect operator missing rowCount when CBO enabled

AngersZh commented on a change in pull request #31240: URL: https://github.com/apache/spark/pull/31240#discussion_r560085505 ## File path: sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/plans/logical/statsEstimation/BasicStatsPlanVisitor.scala ## @@ -57,7 +57,15

[GitHub] [spark] HyukjinKwon commented on a change in pull request #31244: [SPARK-34115][CORE] Check SPARK_TESTING as lazy val to avoid slowdown

HyukjinKwon commented on a change in pull request #31244: URL: https://github.com/apache/spark/pull/31244#discussion_r560086529 ## File path: core/src/main/scala/org/apache/spark/util/Utils.scala ## @@ -1939,7 +1939,15 @@ private[spark] object Utils extends Logging { * Indi

[GitHub] [spark] SparkQA commented on pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

SparkQA commented on pull request #31243: URL: https://github.com/apache/spark/pull/31243#issuecomment-762764349 **[Test build #134223 has finished](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134223/testReport)** for PR 31243 at commit [`3fe1a93`](https://github.co

[GitHub] [spark] SparkQA removed a comment on pull request #31243: [SPARK-34155][SQL][TEST] Add partition columns for TPCDS tables

SparkQA removed a comment on pull request #31243: URL: https://github.com/apache/spark/pull/31243#issuecomment-762744524 **[Test build #134223 has started](https://amplab.cs.berkeley.edu/jenkins/job/SparkPullRequestBuilder/134223/testReport)** for PR 31243 at commit [`3fe1a93`](https://gi

  1   2   3   4   5   6   >