[ https://issues.apache.org/jira/browse/SPARK-8630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14728680#comment-14728680 ]
Andrei Dragomir commented on SPARK-8630: ---------------------------------------- Totally agree with [~asimjalis] comments; We have some integration tests set up that use QueueInputDStream, as well as a processing topology that has updateStateByKey (which implies checkpoints). This patch means that that test suite no longer runs. Is there a work around ? What is the preferred way to push test data into a spark job which has checkpointing ? > Prevent from checkpointing QueueInputDStream > -------------------------------------------- > > Key: SPARK-8630 > URL: https://issues.apache.org/jira/browse/SPARK-8630 > Project: Spark > Issue Type: Improvement > Components: Streaming > Reporter: Shixiong Zhu > Assignee: Shixiong Zhu > Fix For: 1.4.1, 1.5.0 > > > It's better to prevent from checkpointing QueueInputDStream rather than > failing the application when recovering `QueueInputDStream`, so that people > can find the issue as soon as possible. See SPARK-8553 for example. -- This message was sent by Atlassian JIRA (v6.3.4#6332) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org