[ https://issues.apache.org/jira/browse/FLINK-4534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16596464#comment-16596464 ]
ASF GitHub Bot commented on FLINK-4534: --------------------------------------- zentol closed pull request #4482: [FLINK-4534] Fix synchronization issue in BucketingSink URL: https://github.com/apache/flink/pull/4482 This is a PR merged from a forked repository. As GitHub hides the original diff on merge, it is displayed below for the sake of provenance: As this is a foreign pull request (from a fork), the diff is supplied below (as it won't show otherwise due to GitHub magic): diff --git a/flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java b/flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java index faf3c566803..2472387ed44 100644 --- a/flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java +++ b/flink-connectors/flink-connector-filesystem/src/main/java/org/apache/flink/streaming/connectors/fs/bucketing/BucketingSink.java @@ -422,9 +422,11 @@ private void initFileSystem() throws IOException { @Override public void close() throws Exception { - if (state != null) { - for (Map.Entry<String, BucketState<T>> entry : state.bucketStates.entrySet()) { - closeCurrentPartFile(entry.getValue()); + synchronized (state.bucketStates) { + if (state != null) { + for (Map.Entry<String, BucketState<T>> entry : state.bucketStates.entrySet()) { + closeCurrentPartFile(entry.getValue()); + } } } } ---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org > Lack of synchronization in BucketingSink#restoreState() > ------------------------------------------------------- > > Key: FLINK-4534 > URL: https://issues.apache.org/jira/browse/FLINK-4534 > Project: Flink > Issue Type: Bug > Components: Streaming Connectors > Reporter: Ted Yu > Assignee: zhangminglei > Priority: Major > Labels: pull-request-available > > Iteration over state.bucketStates is protected by synchronization in other > methods, except for the following in restoreState(): > {code} > for (BucketState<T> bucketState : state.bucketStates.values()) { > {code} > and following in close(): > {code} > for (Map.Entry<String, BucketState<T>> entry : > state.bucketStates.entrySet()) { > closeCurrentPartFile(entry.getValue()); > {code} > w.r.t. bucketState.pendingFilesPerCheckpoint , there is similar issue > starting line 752: > {code} > Set<Long> pastCheckpointIds = > bucketState.pendingFilesPerCheckpoint.keySet(); > LOG.debug("Moving pending files to final location on restore."); > for (Long pastCheckpointId : pastCheckpointIds) { > {code} -- This message was sent by Atlassian JIRA (v7.6.3#76005)