shauryachats commented on PR #18855:
URL: https://github.com/apache/pinot/pull/18855#issuecomment-5641241690
@Jackie-Jiang sorry for the delay in replying.
**Why local should be unaffected:**
- `SegmentZkMetadataFetcher` and `TimeSegmentPruner` have no
local-vs-remote branching — same code path either way.
- What actually differs is *how* a local broker learns about a commit: it
gets a direct Helix UDM that calls `refreshSegment()` immediately, which
completely bypasses `_onlineSegmentsCached` and the EV check this PR changes.
That path isn't touched here.
- So for local clusters, the fast/authoritative update keeps working
exactly as before — the `onAssignmentChange`/EV-watch logic we're changing is
really just a fallback there, which is also why this bug was invisible locally
in the first place.
- Remote/spectator brokers don't get that UDM, so they're fully dependent
on the path we're fixing — which is why it only showed up for them.
**Tests added to back that up:**
-
`SegmentZkMetadataFetcherTest#testSegmentZkMetadataFetcherShouldNotCacheConsumingSegmentUntilCommitted`
— a segment marked CONSUMING in the EV is skipped by `onAssignmentChange` (no
ZK read, listener never called), gets fetched and cached once it flips to
ONLINE, and isn't re-fetched after that. Tests the caching logic in isolation,
nothing local/remote-specific about it.
- `SegmentPrunerTest#testTimeSegmentPrunerConsumingSegmentCommit` —
end-to-end through both `SegmentZkMetadataFetcher` and `TimeSegmentPruner`:
segment starts CONSUMING at `DEFAULT_INTERVAL` (unprunable, matching the bug),
commits with a real time range, flips to ONLINE, and I assert it's correctly
prunable afterward.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
