Sudhanva Huruli created SPARK-62033:
---------------------------------------

             Summary: COMMENT ON TABLE clears the default collation, and ALTER 
TABLE ... DEFAULT COLLATION clears the comment, on the session catalog
                 Key: SPARK-62033
                 URL: https://issues.apache.org/jira/browse/SPARK-62033
             Project: Spark
          Issue Type: Bug
          Components: SQL
    Affects Versions: 4.3.0
            Reporter: Sudhanva Huruli


A table in the session catalog keeps its comment and its default collation in 
dedicated fields, not in its table properties. V2SessionCatalog.alterTable 
reads both values back from the table properties after applying the changes, so 
a change that sets one of them clears the other.

Steps (spark-sql, default in-memory session catalog; the Hive-backed session 
catalog behaves the same):

CREATE TABLE t (a STRING) USING parquet COMMENT 'table comment' DEFAULT 
COLLATION UTF8_LCASE;
COMMENT ON TABLE t IS 'new comment';
DESCRIBE TABLE EXTENDED t;

Expected: Comment is 'new comment' and the default collation is still 
UTF8_LCASE. Actual: the collation is gone.

The other direction: after the same CREATE, ALTER TABLE t DEFAULT COLLATION 
UTF8_BINARY; leaves the collation as UTF8_BINARY but removes the comment. 
COMMENT ON TABLE t IS NULL also removes the collation.

ALTER TABLE ... SET TBLPROPERTIES keeps both values, because it runs as the v1 
command, which already keeps fields a change does not name. Expected rule for 
alterTable: recompute the comment or the collation only when a change sets or 
removes that property, and otherwise keep the existing value.



--
This message was sent by Atlassian Jira
(v10.3.26#10030026)

---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]

Reply via email to