Weiyi Kong created SPARK-38056:
----------------------------------
Summary: Structured streaming not working in history server when
using LevelDB
Key: SPARK-38056
URL: https://issues.apache.org/jira/browse/SPARK-38056
Project: Spark
Issue Type: Bug
Components: Structured Streaming, Web UI
Affects Versions: 3.2.0, 3.1.2
Reporter: Weiyi Kong
Attachments: local-1643373518829
In
[SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68],
structured streaming support is added to history server. However this does not
work when spark.history.store.path is set to save app info using LevelDB.
This is because one of the keys of StreamingQueryData, runId, is UUID type,
which is not supported by LevelDB. When replaying event log file in history
server, StreamingQueryStatusListener will throw an exception when writing info
to the store, saying "java.lang.IllegalArgumentException: Type java.util.UUID
not allowed as key.".
--
This message was sent by Atlassian Jira
(v8.20.1#820001)
---------------------------------------------------------------------
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
