SEZ9 commented on PR #11819: URL: https://github.com/apache/seatunnel/pull/11819#issuecomment-5673764693
Thanks @goutamadwant — noted that 5dbfb8966 is a dev sync only and that LocalFile source behavior is unchanged. Since none of the earlier points were touched in this update, here is what is still open from the previous review: - **Fail-the-reader after post-read mutation (F1/F2/F5):** I'm fine treating this as a design-review discussion, but it needs a decision before merge. My concern remains that a routine rotation/deletion during a read becomes a whole-reader failure rather than an isolated skip, and that rows are already handed to the collector before the identity re-check, so bytes from the replacing file can reach the sink. Please either (a) move the identity check ahead of emission and downgrade the mismatch to a per-split skip + log/event, or (b) write up in the PR description why fail-the-reader is the intended contract and how operators are expected to recover. - **Serialized field growth (F3):** `FileSplitFinishedEvent`, `FileSourceSplit` and `FileSourceState` gain fields — please confirm (ideally with a test) that a checkpoint taken by a pre-upgrade job restores cleanly. - **Identity stability (F4):** `fileKey.toString()` (dev+inode) is not stable across reboot/remount and is subject to inode reuse. Please describe how restored state avoids misclassifying tracked files, or add a secondary signal (e.g. size/mtime or a content fingerprint) to the identity. - **Only pre-read `NoSuchFileException` tolerated (F6):** other transient I/O errors on a single tailed file still crash-loop the streaming job — please widen the handling or explain why that is acceptable. - **Stale fingerprint on skipped tail splits (F7):** skipped splits should not report the previous read's content fingerprint in `FileSplitFinishedEvent`. - **`processedBytes` / `getLastReadBytes()` sentinels (F8):** please add Javadoc defining the -1 vs 0 contract. Happy to look again once these are addressed or you've replied on each point. <!-- streview-comment:1052 --> -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
