xiangfu0 commented on code in PR #19770:
URL: https://github.com/apache/pinot/pull/19770#discussion_r4214759560
##########
pinot-spi/src/main/java/org/apache/pinot/spi/config/table/SegmentsValidationAndRetentionConfig.java:
##########
@@ -88,6 +89,55 @@ public void setRetentionTimeValue(String retentionTimeValue)
{
_retentionTimeValue = retentionTimeValue;
}
+ /// Returns the size retention limit, expressed as a positive size
understood by
+ /// [DataSizeUtils#toBytes(String)], for example `"100G"`.
+ ///
+ /// The limit sums compressed archive bytes for active, live completed
segments, counted once regardless of
+ /// replication. Live-segment lineage filtering excludes sources of
COMPLETED entries and destinations of other
+ /// entries to avoid counting shadow copies. OFFLINE APPEND and REALTIME
limits are enforced independently, including
+ /// hybrid tables.
+ ///
+ /// Segments are ordered ascending by the first nonnegative end time,
creation time, or push time, in that priority,
+ /// then by segment name. Eviction removes only an old prefix before the
first active segment listed as a source or
+ /// destination in any retained lineage entry. Every listed segment is
protected until its lineage entry is removed,
+ /// regardless of entry state, age, or the lineage-exclusive deletion
setting. Listed shadow copies remain stop
+ /// positions even when their bytes are excluded. Missing live metadata or
unknown live completed-segment sizes cause
+ /// the pass to skip safely. Missing metadata or timestamps for an active
lineage stop prevent eviction when over cap.
+ ///
+ /// With the default lineage manager, COMPLETED sources wait for
[#getReplacedSegmentsRetentionPeriod()] (4h for
Review Comment:
Verified this paragraph line-by-line against `DefaultLineageManager` and
`ControllerConf`: the 4h/24h defaults, the 1d stale-IN_PROGRESS window,
immediate REVERTED eligibility, the 6h schedule delay, and the two-pass
COMPLETED-entry removal (sources deleted on one pass, entry removed on a later
pass once they've left IdealState) are all accurate. This is exactly the
operator-facing context the gauge needed — someone seeing
`sizeRetentionBlocked=1` on a merge-heavy table can now tell expected pinning
from a real problem.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
