Jungtaek Lim created SPARK-38206:
------------------------------------
Summary: Relax the requirement of data type comparison for keys in
stream-stream join
Key: SPARK-38206
URL: https://issues.apache.org/jira/browse/SPARK-38206
Project: Spark
Issue Type: Bug
Components: Structured Streaming
Affects Versions: 3.2.1, 3.1.2, 3.3.0
Reporter: Jungtaek Lim
Currently, stream-stream join checks for the data type compatible between left
keys and right keys. It is done as "strict" checking, requiring nullability as
same for both sides. This leads to throw assertion error if optimizer turns
some columns in one side from nullable to non-nullable but not touching
opposite side.
If it is logically correct to relax the nullability check (with deciding proper
type on output schema), we should do it to avoid any possible issue from
optimization.
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
