cshuo commented on code in PR #18406:
URL: https://github.com/apache/hudi/pull/18406#discussion_r3008061964
##########
hudi-flink-datasource/hudi-flink/src/main/java/org/apache/hudi/source/reader/HoodieSourceSplitReader.java:
##########
@@ -54,21 +56,30 @@
public class HoodieSourceSplitReader<T> implements
SplitReader<HoodieRecordWithPosition<T>, HoodieSourceSplit> {
private static final Logger LOG =
LoggerFactory.getLogger(HoodieSourceSplitReader.class);
+ /** Sentinel value indicating that no row limit has been pushed down. */
+ public static final long NO_LIMIT = -1L;
+
private final SerializableComparator<HoodieSourceSplit> splitComparator;
private final Queue<HoodieSourceSplit> splits;
private final FlinkStreamReadMetrics readerMetrics;
private final SplitReaderFunction<T> readerFunction;
+ /** Total row limit pushed down from the planner; {@code NO_LIMIT} means
unlimited. */
+ private final long limit;
+ /** Number of records emitted so far across all splits. */
+ private long totalReadCount = 0;
Review Comment:
Can we add an `RecordLimiter` utility class, which provides methods like
`increment()` and `reachLimit()`, so that limit-related variables and logic can
be gathered within this class.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
