[
https://issues.apache.org/jira/browse/FLINK-37870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rui Fan updated FLINK-37870:
----------------------------
Description:
FLINK-36287 would disable unaligned checkpoint between {color:#172b4d}the sink
writer and committer, but it disabled unaligned checkpoint for all connections
of the entire DAG.{color}
h2. {color:#172b4d}Reason:{color}
[https://github.com/apache/flink/blob/a4e3084ab9220b02bc3eefc70ea372251aec7f07/flink-runtime/src/main/java/org/apache/flink/streaming/runtime/translators/SinkTransformationTranslator.java#L226]
*_Set<Integer> seen = new HashSet<>(writer.getId());_* creates a HashSet to
record all seen transformations. It expects to create a HashSet and put the
first writer transformation into the set to end the recursive loop (to prevent
traversing the previous transformation of the writer).
But the constructor parameter of new HashSet is initialCapacity, so it only
created a HashSet, and didn't put the first writer transformation into the set.
was:
FLINK-36287 would disable unaligned checkpoint between {color:#172b4d}the sink
writer and committer, but it disabled unaligned checkpoint for whole DAG.{color}
h2. {color:#172b4d}Reason:{color}
[https://github.com/apache/flink/blob/a4e3084ab9220b02bc3eefc70ea372251aec7f07/flink-runtime/src/main/java/org/apache/flink/streaming/runtime/translators/SinkTransformationTranslator.java#L226]
*_Set<Integer> seen = new HashSet<>(writer.getId());_* creates a HashSet to
record all seen transformations. It expects to create a HashSet and put the
first writer transformation into the set to end the recursive loop (to prevent
traversing the previous transformation of the writer).
But the constructor parameter of new HashSet is initialCapacity, so it only
created a HashSet, and didn't put the first writer transformation into the set.
> Unaligned checkpoint is disabled for all connections unexpectedly
> -----------------------------------------------------------------
>
> Key: FLINK-37870
> URL: https://issues.apache.org/jira/browse/FLINK-37870
> Project: Flink
> Issue Type: Bug
> Components: Runtime / Checkpointing
> Affects Versions: 2.0.0, 1.19.2, 1.20.1
> Reporter: Rui Fan
> Assignee: Rui Fan
> Priority: Major
>
> FLINK-36287 would disable unaligned checkpoint between {color:#172b4d}the
> sink writer and committer, but it disabled unaligned checkpoint for all
> connections of the entire DAG.{color}
> h2. {color:#172b4d}Reason:{color}
> [https://github.com/apache/flink/blob/a4e3084ab9220b02bc3eefc70ea372251aec7f07/flink-runtime/src/main/java/org/apache/flink/streaming/runtime/translators/SinkTransformationTranslator.java#L226]
> *_Set<Integer> seen = new HashSet<>(writer.getId());_* creates a HashSet to
> record all seen transformations. It expects to create a HashSet and put the
> first writer transformation into the set to end the recursive loop (to
> prevent traversing the previous transformation of the writer).
> But the constructor parameter of new HashSet is initialCapacity, so it only
> created a HashSet, and didn't put the first writer transformation into the
> set.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
