xiangfu0 commented on code in PR #18761:
URL: https://github.com/apache/pinot/pull/18761#discussion_r3435689413
##########
pinot-core/src/main/java/org/apache/pinot/core/data/manager/BaseTableDataManager.java:
##########
@@ -1043,6 +1051,12 @@ protected void reloadSegments(List<SegmentDataManager>
segmentDataManagers, Inde
protected void reloadSegment(SegmentDataManager segmentDataManager,
IndexLoadingConfig indexLoadingConfig,
boolean forceDownload)
throws Exception {
+ // Reload path — always run full preprocess against the latest table
config, regardless of the persisted
+ // tableIndexConfig.skipSegmentPreprocess. The override lives on this
transient IndexLoadingConfig only and
+ // is discarded when the reload finishes. Centralizing it here means every
public reload entry point
+ // (reloadSegment / reloadAllSegments / reloadSegments) gets the reload
semantics for free; boot /
+ // state-transition loads continue to use a plain
fetchIndexLoadingConfig() and honor skipSegmentPreprocess.
+ indexLoadingConfig.setOverrideSkipSegmentPreprocess(true);
Review Comment:
This changes the contract of the existing
`tableIndexConfig.skipSegmentPreprocess` flag for every reload path, not just
the async index-materialization flow. Today a table that sets this flag can
rely on reloads honoring the persisted forward-only load policy; after this
override, any manual/controller reload will silently build all secondary
indexes instead. That is a behavior change for an existing config with real
CPU/disk blast radius for current deployments. Please gate the bypass behind a
new explicit reload API/flag instead of unconditionally overriding the
persisted setting here.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
