Jungtaek Lim created SPARK-38412:
------------------------------------
Summary: `from` and `to` is swapped in the
StateSchemaCompatibilityChecker
Key: SPARK-38412
URL: https://issues.apache.org/jira/browse/SPARK-38412
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.2.1, 3.1.2, 3.3.0
Reporter: Jungtaek Lim
During the work in SPARK-38024, I figured out the parameter is swapped which
led to test failure on new test (I disabled the schema check for now in the PR
of SPARK-38024).
That allows nullable column to be stored into non-nullable column, which should
be prohibited. This is less likely making runtime problem since state schema is
conceptual one and row can be stored even not respecting the state schema.
The problem is opposite, that disallows non-nullable column to be stored into
nullable column, which should be allowed. Spark fails the query for this case.
We should fix this to allow the case properly.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
