[jira] [Updated] (SPARK-26824) Streaming queries may store checkpoint data in a wrong directory
[ https://issues.apache.org/jira/browse/SPARK-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26824: - Docs Text: Earlier version of Spark incorrectly escaped paths when writing out checkpoints and "_spark_metadata" for structured streaming. Queries affected by this issue will fail when running in Spark 3.0. It will report an instruction about how to migrate your queries. > Streaming queries may store checkpoint data in a wrong directory > > > Key: SPARK-26824 > URL: https://issues.apache.org/jira/browse/SPARK-26824 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0 >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu >Priority: Major > Labels: release-notes > Fix For: 3.0.0 > > > When a user specifies a checkpoint location containing special chars that > need to be escaped in a path, the streaming query will store checkpoint in a > wrong place. For example, if you use "/chk chk", the metadata will be stored > in "/chk%20chk". File sink's "_spark_metadata" directory has the same issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-26824) Streaming queries may store checkpoint data in a wrong directory
[ https://issues.apache.org/jira/browse/SPARK-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26824: - Affects Version/s: 2.0.0 2.1.0 2.2.0 2.3.0 > Streaming queries may store checkpoint data in a wrong directory > > > Key: SPARK-26824 > URL: https://issues.apache.org/jira/browse/SPARK-26824 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 2.0.0, 2.1.0, 2.2.0, 2.3.0, 2.4.0 >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu >Priority: Major > Labels: release-notes > > When a user specifies a checkpoint location containing special chars that > need to be escaped in a path, the streaming query will store checkpoint in a > wrong place. For example, if you use "/chk chk", the metadata will be stored > in "/chk%20chk". File sink's "_spark_metadata" directory has the same issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-26824) Streaming queries may store checkpoint data in a wrong directory
[ https://issues.apache.org/jira/browse/SPARK-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26824: - Description: When a user specifies a checkpoint location containing special chars that need to be escaped in a path, the streaming query will store checkpoint in a wrong place. For example, if you use "/chk chk", the metadata will be stored in "/chk%20chk". File sink's "_spark_metadata" directory has the same issue. (was: When a user specifies a checkpoint location containing special chars that need to be escaped in a path, the streaming query will store checkpoint in a wrong place. File sink's "_spark_metadata" directory has the same issue. ) > Streaming queries may store checkpoint data in a wrong directory > > > Key: SPARK-26824 > URL: https://issues.apache.org/jira/browse/SPARK-26824 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 2.4.0 >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu >Priority: Major > Labels: release-notes > > When a user specifies a checkpoint location containing special chars that > need to be escaped in a path, the streaming query will store checkpoint in a > wrong place. For example, if you use "/chk chk", the metadata will be stored > in "/chk%20chk". File sink's "_spark_metadata" directory has the same issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-26824) Streaming queries may store checkpoint data in a wrong directory
[ https://issues.apache.org/jira/browse/SPARK-26824?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-26824: - Labels: release-notes (was: ) > Streaming queries may store checkpoint data in a wrong directory > > > Key: SPARK-26824 > URL: https://issues.apache.org/jira/browse/SPARK-26824 > Project: Spark > Issue Type: Bug > Components: Structured Streaming >Affects Versions: 2.4.0 >Reporter: Shixiong Zhu >Assignee: Shixiong Zhu >Priority: Major > Labels: release-notes > > When a user specifies a checkpoint location containing special chars that > need to be escaped in a path, the streaming query will store checkpoint in a > wrong place. File sink's "_spark_metadata" directory has the same issue. -- This message was sent by Atlassian JIRA (v7.6.3#76005) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
