Aleksei Shishkin created SPARK-50470:
----------------------------------------
Summary: Block usage of collations for maps' key
Key: SPARK-50470
URL: https://issues.apache.org/jira/browse/SPARK-50470
Project: Spark
Issue Type: Bug
Components: Input/Output
Affects Versions: 4.0.0, 3.5.4
Reporter: Aleksei Shishkin
Now it is possible to set collations on keys of map type. But it could lead to
unexpected bugs related to uniqueness of keys, so after collation change
existing data may become unreadable. For example, it happens when table is
created with
{code:java}
map('A' -> 1, 'a' -> 2){code}
and then collation switched to UTF8_LCASE.
I propose to block collations on keys for now. Other systems rejects such
changes as well.
Probably int he future we could discuss about blocking changing collation but
not blocking creation tables with collated keys.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
