cshuo commented on code in PR #19202:
URL: https://github.com/apache/hudi/pull/19202#discussion_r3569099157
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/table/format/cdc/CdcIterators.java:
##########
@@ -135,14 +153,25 @@ public boolean hasNext() {
}
@Override
- public RowData next() {
+ public synchronized RowData next() {
Review Comment:
+1 to keep the iterator-side synchronization as the hot fix.
For a longer-term fix, we should remove the lazy read from `BatchRecords`:
`SplitReader.fetch()` should materialize a bounded mini-batch of records, and
`BatchRecords` should carry that batch rather than a live iterator. This keeps
the iterator, `CdcImageManager`, and file-reader lifecycle entirely in the
split reader function / fetcher thread, where they can be released
deterministically on split completion or cancellation.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
