pkuzmickas opened a new pull request, #40354:
URL: https://github.com/apache/beam/pull/40354
## Summary
Restore the source split logs in the Flink DataStream runner that #39874
removed when it moved splitting into `FlinkSourceSplitUtils`.
Before #39874, the static and lazy enumerators logged:
- `Split bounded source {} in {} splits`
- `Split source {} to {} splits` (unbounded)
After the refactor, only the opt-in size-based enumerator logged split
counts. The default lazy and static paths logged nothing about splitting.
## Why
These logs are the main way to diagnose source skew in batch jobs. For
example, comparing the split count with the source parallelism shows whether a
source produces too few splits to distribute work evenly.
## Changes
- Log in `FlinkSourceSplitUtils.splitBoundedSource` /
`splitUnboundedSource`, so every enumerator emits the log.
- Keep the original message prefixes so existing log searches still match.
The bounded log also includes the estimated source size and desired split size.
- Remove the size-based enumerator's post-split log, which would now be a
duplicate. Its existing assignment-mode log still reports the selected mode.
------------------------
- [x] Logging-only change; no `CHANGES.md` entry needed.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
