Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

HyukjinKwon commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1602870592 ## sql/catalyst/src/test/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalogSuite.scala: ## @@ -742,7 +742,7 @@ abstract class SessionCatalogSuite extends An

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan closed pull request #46267: [SPARK-48031][SQL] Support view schema evolution URL: https://github.com/apache/spark/pull/46267 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific co

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on PR #46267: URL: https://github.com/apache/spark/pull/46267#issuecomment-2107718906 the k8s test failure is unrelated, thanks, merging to master! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on PR #46267: URL: https://github.com/apache/spark/pull/46267#issuecomment-2107356801 @cloud-fan @gengliangwang All of @cloud-fan comments have been accepted and addressed. Shall we merge it? -- This is an automated message from the Apache Git Service. To respond to t

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1598332828 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,65 @@ object CollationCheck extends (LogicalPlan => Unit) { priv

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1598272715 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,65 @@ object CollationCheck extends (LogicalPlan => Unit) { priv

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1597322015 ## sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala: ## @@ -224,6 +224,7 @@ abstract class BaseSessionStateBuilder( TableCap

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596850227 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala: ## @@ -35,6 +35,56 @@ object EliminateView extends Rule[LogicalPlan] with CastSuppo

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596873275 ## sql/core/src/test/resources/sql-tests/results/view-schema-binding-config.sql.out: ## @@ -0,0 +1,794 @@ +-- Automatically generated by SQLQueryTestSuite +-- !query

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596873275 ## sql/core/src/test/resources/sql-tests/results/view-schema-binding-config.sql.out: ## @@ -0,0 +1,794 @@ +-- Automatically generated by SQLQueryTestSuite +-- !query

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596870581 ## sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala: ## @@ -224,6 +224,7 @@ abstract class BaseSessionStateBuilder( TableC

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596869407 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,65 @@ object CollationCheck extends (LogicalPlan => Unit) { pr

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596861036 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,65 @@ object CollationCheck extends (LogicalPlan => Unit) { pr

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596852480 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala: ## @@ -945,54 +945,87 @@ class SessionCatalog( throw QueryCompil

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596850227 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala: ## @@ -35,6 +35,56 @@ object EliminateView extends Rule[LogicalPlan] with CastSuppo

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596849473 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala: ## @@ -35,6 +35,56 @@ object EliminateView extends Rule[LogicalPlan] with CastSuppo

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596184905 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala: ## @@ -945,54 +945,73 @@ class SessionCatalog( throw QueryCompilat

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596184739 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala: ## @@ -945,54 +945,73 @@ class SessionCatalog( throw QueryCompilat

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596180917 ## sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala: ## @@ -224,6 +224,7 @@ abstract class BaseSessionStateBuilder( TableCap

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596169480 ## sql/core/src/main/scala/org/apache/spark/sql/internal/BaseSessionStateBuilder.scala: ## @@ -224,6 +224,7 @@ abstract class BaseSessionStateBuilder( TableC

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

gengliangwang commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1596122256 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/catalog/SessionCatalog.scala: ## @@ -945,54 +945,73 @@ class SessionCatalog( throw QueryCo

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on PR #46267: URL: https://github.com/apache/spark/pull/46267#issuecomment-2103503061 @cloud-fan @gengliangwang I have addressed all comments (except the "override" on the, which intelij and Wenchen appear to be of different opinions). The testsuite has been split up in

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595880193 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(true)

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

gengliangwang commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595854690 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(t

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

gengliangwang commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595853192 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(t

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

mridulm commented on PR #46267: URL: https://github.com/apache/spark/pull/46267#issuecomment-2103095339 +CC @shardulm94, @robreeves -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific c

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595711075 ## sql/core/src/test/resources/sql-tests/results/view-schema-binding.sql.out: ## @@ -0,0 +1,3081 @@ +-- Automatically generated by SQLQueryTestSuite +-- !query +SET spa

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595698980 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,66 @@ object CollationCheck extends (LogicalPlan => Unit) { priv

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595689617 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala: ## @@ -437,6 +493,21 @@ object ViewHelper extends SQLConfHelper with Logging { }

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595683366 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala: ## @@ -318,6 +329,51 @@ case class AlterViewAsCommand( } } +/** + * Alter a view wi

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595681121 ## sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala: ## @@ -546,6 +551,13 @@ class SparkSqlAstBuilder extends AstBuilder { val origina

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595675243 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(true)

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595672298 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala: ## @@ -35,6 +35,56 @@ object EliminateView extends Rule[LogicalPlan] with CastSupport

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595668170 ## docs/sql-ref-syntax-ddl-create-view.md: ## @@ -61,6 +61,17 @@ CREATE [ OR REPLACE ] [ [ GLOBAL ] TEMPORARY ] VIEW [ IF NOT EXISTS ] view_ident * `[ ( column_na

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595668170 ## docs/sql-ref-syntax-ddl-create-view.md: ## @@ -61,6 +61,17 @@ CREATE [ OR REPLACE ] [ [ GLOBAL ] TEMPORARY ] VIEW [ IF NOT EXISTS ] view_ident * `[ ( column_na

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595660959 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(true)

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

srielau commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595660959 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(true)

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595534768 ## sql/core/src/test/resources/sql-tests/results/view-schema-binding.sql.out: ## @@ -0,0 +1,3081 @@ +-- Automatically generated by SQLQueryTestSuite +-- !query +SET s

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595529865 ## sql/core/src/test/resources/sql-tests/inputs/view-schema-binding.sql: ## @@ -0,0 +1,429 @@ +-- This test suits check the WITH SCHEMA clause +-- Make sure we use AN

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595522181 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,66 @@ object CollationCheck extends (LogicalPlan => Unit) { pr

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595519795 ## sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/rules.scala: ## @@ -620,3 +621,66 @@ object CollationCheck extends (LogicalPlan => Unit) { pr

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595515280 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala: ## @@ -437,6 +493,21 @@ object ViewHelper extends SQLConfHelper with Logging { }

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595513707 ## sql/core/src/main/scala/org/apache/spark/sql/execution/command/views.scala: ## @@ -318,6 +329,51 @@ case class AlterViewAsCommand( } } +/** + * Alter a view

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595505686 ## sql/core/src/main/scala/org/apache/spark/sql/execution/SparkSqlParser.scala: ## @@ -546,6 +551,13 @@ class SparkSqlAstBuilder extends AstBuilder { val origi

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595489391 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(true)

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595486032 ## sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/view.scala: ## @@ -35,6 +35,56 @@ object EliminateView extends Rule[LogicalPlan] with CastSuppo

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595484586 ## docs/sql-ref-syntax-ddl-create-view.md: ## @@ -80,6 +91,10 @@ CREATE GLOBAL TEMPORARY VIEW IF NOT EXISTS subscribed_movies AS SELECT mo.member_id, mb.full_nam

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595483982 ## docs/sql-ref-syntax-ddl-create-view.md: ## @@ -61,6 +61,17 @@ CREATE [ OR REPLACE ] [ [ GLOBAL ] TEMPORARY ] VIEW [ IF NOT EXISTS ] view_ident * `[ ( column_

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595479851 ## sql/catalyst/src/main/scala/org/apache/spark/sql/internal/SQLConf.scala: ## @@ -1700,6 +1700,16 @@ object SQLConf { .booleanConf .createWithDefault(true)

Re: [PR] [SPARK-48031][SQL] Support view schema evolution [spark]

cloud-fan commented on code in PR #46267: URL: https://github.com/apache/spark/pull/46267#discussion_r1595468220 ## common/utils/src/main/resources/error/error-conditions.json: ## @@ -4481,6 +4487,11 @@ "Table does not support . Please check the current catalog and