shounakmk219 opened a new pull request, #18294: URL: https://github.com/apache/pinot/pull/18294
## Summary - Timestamp-index derived columns (`$col$GRANULARITY`) are materialized as physical segment columns when a `TimestampConfig` is active, but are not present in the user-facing schema. When the timestamp index is later removed from the table config, `applyTimestampIndex` no longer injects the derived column into the `IndexLoadingConfig` schema, causing `isSegmentStale` to fire `"column deleted: $col$GRANULARITY"` and trigger an unnecessary segment reload. - The fix skips the "column deleted" verdict for `$col$GRANULARITY` columns when the **base column is still in the schema** (confirming this is a timestamp-derived orphan, not a genuine user-schema deletion). If the base column is also absent the guard does not activate, so user-defined columns whose names happen to match the pattern are still correctly caught. - Six tests added: no-change, index added (stale), index removed (not stale), granularity added (stale), granularity removed (not stale), and a boundary test for a user-defined `$col$GRANULARITY`-named column that is genuinely deleted. ## Root cause `FieldSpec.isVirtualColumn()` returns `true` only when `_virtualColumnProvider` is set — a `transformFunction` alone does not make a column virtual. So `$col$DAY` added by `applyTimestampIndex` lands in `schema.getPhysicalColumnNames()`, and the segment also stores it as a physical column (with `isAutoGenerated=false`). When the `TimestampConfig` is removed, the schema no longer contains the derived column but the segment still does, triggering the false stale verdict. ## Test plan - [x] `BaseTableDataManagerNeedRefreshTest` — all 43 tests pass (`./mvnw -pl pinot-core -am -Dtest=BaseTableDataManagerNeedRefreshTest -Dsurefire.failIfNoSpecifiedTests=false test`) - [x] `spotless:apply`, `license:format`, `checkstyle:check`, `license:check` all pass on `pinot-core` 🤖 Generated with [Claude Code](https://claude.com/claude-code) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
