shounakmk219 commented on PR #18294: URL: https://github.com/apache/pinot/pull/18294#issuecomment-4302750069
@Jackie-Jiang @noob-se7en that makes sense. Let me take another look. My initial take was that the stale reason should be flagged by the timestamp index check and not come up as a column removed operation. Its confusing as the schema will not have the column. Also I ran few flows around the timestamp index and the stale reason is hard to understand as well as for few cases it is not fixed even after a segment reload. Steps: 1. Add a timestamp index 2. check for stale API -> fails with reason "column added" 3. reload segments 4. check for stake API -> fails with reason "range index changed: $mtime$HOUR" 5. add another granularity MINUTE to the index 6. check for stale API -> fails with reason "column added" 7. reload segments 8. check for stake API -> still fails with reason "range index changed: $mtime$HOUR" 9. remove the HOUR granularity 10. check for stake API -> fails with reason "column deleted: $mtime$HOUR" 11. reload segments 12. check for stake API -> fails with reason "range index changed: $mtime$MINUTE" Looks like the right fix would be to add a dedicated timestamp index validation and emit the right reason of staleness But its also worth looking into why the check complains about range index even after we reload the segments. Goal is to make the stale API meaningful as well as actionable through something like segment reload. If a stale reason cannot be addressed by reload we should have a way to skip these as typical usecase involve hitting this stale check and proceed to run reload if needed to handle things like preventing server restart time, etc. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
