[ https://issues.apache.org/jira/browse/FLINK-25429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Yun Tang resolved FLINK-25429. ------------------------------ Resolution: Fixed > Avoid to close output streams twice during uploading changelogs > --------------------------------------------------------------- > > Key: FLINK-25429 > URL: https://issues.apache.org/jira/browse/FLINK-25429 > Project: Flink > Issue Type: Improvement > Components: Runtime / State Backends > Reporter: Yun Tang > Assignee: Yun Tang > Priority: Major > Labels: pull-request-available > Fix For: 1.15.0 > > > Current uploader implementation would close {{stream}} and {{fsStream}} one > by one, which lead to {{fsStream}} closed twice. > {code:java} > try (FSDataOutputStream fsStream = fileSystem.create(path, > NO_OVERWRITE)) { > fsStream.write(compression ? 1 : 0); > try (OutputStreamWithPos stream = wrap(fsStream); ) { > final Map<UploadTask, Map<StateChangeSet, Long>> tasksOffsets > = new HashMap<>(); > for (UploadTask task : tasks) { > tasksOffsets.put(task, format.write(stream, > task.changeSets)); > } > FileStateHandle handle = new FileStateHandle(path, > stream.getPos()); > // WARN: streams have to be closed before returning the > results > // otherwise JM may receive invalid handles > return new LocalResult(tasksOffsets, handle); > } > } > {code} > Not all file system supports to close same stream twice. -- This message was sent by Atlassian Jira (v8.20.1#820001)