[ https://issues.apache.org/jira/browse/SPARK-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16760071#comment-16760071 ]
Shixiong Zhu commented on SPARK-26824: -------------------------------------- This will need a release note. After the fix, the paths to store metadata will be changed. The user needs to move the metadata files which are in a wrong place to the right location manually. Otherwise, their query will not pick up the old metadata. > Streaming queries may store checkpoint data in a wrong directory > ---------------------------------------------------------------- > > Key: SPARK-26824 > URL: https://issues.apache.org/jira/browse/SPARK-26824 > Project: Spark > Issue Type: Bug > Components: Structured Streaming > Affects Versions: 2.4.0 > Reporter: Shixiong Zhu > Assignee: Shixiong Zhu > Priority: Major > Labels: release-notes > > When a user specifies a checkpoint location containing special chars that > need to be escaped in a path, the streaming query will store checkpoint in a > wrong place. For example, if you use "/chk chk", the metadata will be stored > in "/chk%20chk". File sink's "_spark_metadata" directory has the same issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org