stankiewicz commented on code in PR #38149:
URL: https://github.com/apache/beam/pull/38149#discussion_r3074175409
##########
sdks/java/io/iceberg/src/main/java/org/apache/beam/sdk/io/iceberg/RecordWriterManager.java:
##########
@@ -434,20 +431,6 @@ public void close() throws IOException {
state.dataFiles.clear();
}
} finally {
- // Close unique FileIO instances now that all writers are done.
- // table.io() may return a shared FileIO; we deduplicate by identity
- // so we close each underlying connection pool exactly once.
- Set<FileIO> closedIOs = new HashSet<>();
- for (DestinationState state : destinations.values()) {
- FileIO io = state.table.io();
- if (io != null && closedIOs.add(io)) {
- try {
- io.close();
Review Comment:
are there any issues if io is not closed? Dataflow invokes teardown only
when doFn needs to be recycled due to exception. call of teardown is best
effort, do not use for side effects. are there side effects of closing on those
io?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
