[
https://issues.apache.org/jira/browse/FLINK-4075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15356825#comment-15356825
]
ASF GitHub Bot commented on FLINK-4075:
---------------------------------------
Github user rmetzger commented on a diff in the pull request:
https://github.com/apache/flink/pull/2174#discussion_r69102339
--- Diff:
flink-streaming-java/src/main/java/org/apache/flink/streaming/api/functions/source/ContinuousFileReaderOperator.java
---
@@ -334,9 +342,11 @@ public void run() {
this.pendingSplits.remove();
}
- if (this.format instanceof CheckpointableInputFormat &&
this.isSplitOpen) {
- S formatState = (S)
((CheckpointableInputFormat) format).getCurrentState();
- return new Tuple3<>(snapshot, currentSplit,
currentSplit == null ? null : formatState);
+ if (this.format instanceof CheckpointableInputFormat &&
this.currentSplit != null) {
+ S formatState = this.isSplitOpen ?
+ (S) ((CheckpointableInputFormat)
format).getCurrentState() :
+ restoredFormatState;
+ return new Tuple3<>(snapshot, currentSplit,
formatState);
} else {
LOG.info("The format used is not
checkpointable. The current input split will be restarted upon recovery.");
--- End diff --
Monitoring an empty directory looks like this:
```
11:36:42,704 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
checkpoint 1 @ 1467279402703
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,709 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,710 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:42,823 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
checkpoint 1 (in 120 ms)
11:36:44,703 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
checkpoint 2 @ 1467279404703
11:36:44,705 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,706 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,706 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,706 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,706 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,706 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,707 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,710 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:44,745 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
checkpoint 2 (in 18 ms)
11:36:46,703 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
checkpoint 3 @ 1467279406703
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,705 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:46,723 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
checkpoint 3 (in 8 ms)
11:36:48,703 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Triggering
checkpoint 4 @ 1467279408703
11:36:48,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,704 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,705 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,705 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,705 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,705 INFO
org.apache.flink.streaming.api.functions.source.ContinuousFileReaderOperator -
The format used is not checkpointable. The current input split will be
restarted upon recovery.
11:36:48,729 INFO
org.apache.flink.runtime.checkpoint.CheckpointCoordinator - Completed
checkpoint 4 (in 10 ms)
```
> ContinuousFileProcessingCheckpointITCase failed on Travis
> ---------------------------------------------------------
>
> Key: FLINK-4075
> URL: https://issues.apache.org/jira/browse/FLINK-4075
> Project: Flink
> Issue Type: Bug
> Components: Tests
> Affects Versions: 1.1.0
> Reporter: Till Rohrmann
> Assignee: Kostas Kloudas
> Priority: Critical
> Labels: test-stability
>
> The test case {{ContinuousFileProcessingCheckpointITCase}} failed on Travis.
> https://s3.amazonaws.com/archive.travis-ci.org/jobs/137748004/log.txt
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
