[ https://issues.apache.org/jira/browse/SPARK-38056?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Jungtaek Lim resolved SPARK-38056. ---------------------------------- Fix Version/s: 3.3.0 Resolution: Fixed Issue resolved by pull request 35356 [https://github.com/apache/spark/pull/35356] > Structured streaming not working in history server when using LevelDB > --------------------------------------------------------------------- > > Key: SPARK-38056 > URL: https://issues.apache.org/jira/browse/SPARK-38056 > Project: Spark > Issue Type: Bug > Components: Structured Streaming, Web UI > Affects Versions: 3.1.2, 3.2.0 > Reporter: wy > Assignee: wy > Priority: Major > Fix For: 3.3.0 > > Attachments: local-1643373518829 > > > In > [SPARK-31953|https://github.com/apache/spark/commit/4f9667035886a67e6c9a4e8fad2efa390e87ca68], > structured streaming support is added to history server. However this does > not work when spark.history.store.path is set to save app info using LevelDB. > This is because one of the keys of StreamingQueryData, runId, is UUID type, > which is not supported by LevelDB. When replaying event log file in history > server, StreamingQueryStatusListener will throw an exception when writing > info to the store, saying "java.lang.IllegalArgumentException: Type > java.util.UUID not allowed as key.". > Example event log is provided in attachments. When opening it in history > server with spark.history.store.path set to somewhere, no structured > streaming info is available. -- This message was sent by Atlassian Jira (v8.20.1#820001) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org