ottomata commented on PR #21012: URL: https://github.com/apache/spark/pull/21012#issuecomment-1856336189
Hi, I'd like to reopen this issue. Back in 2019, @cloud-fan said that v2 datasource: > already support changing nested fields with command AlterTable However, there is an [explicit check](https://github.com/apache/spark/blob/master/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/analysis/CheckAnalysis.scala#L1441) and [test](https://github.com/apache/spark/blob/e3f46ed57dc063566cdb9425b4d5e02c65332df1/sql/core/src/test/scala/org/apache/spark/sql/connector/AlterTableTests.scala#L583) that prevents this from happening. We need to be able to evolve tables in a backwards compatible way. This safe and possible with top level fields, and it should be safe and possible with struct column fields too, as long as we are only adding new fields. cc @MaxGekk @beliefer @brkyvz Thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org --------------------------------------------------------------------- To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org For additional commands, e-mail: reviews-h...@spark.apache.org
