englefly commented on PR #68282: URL: https://github.com/apache/doris/pull/68282#issuecomment-5790668126
Handled in 19756ddc9de. Fixed: the truncate statistics transition is now journal-reproducible instead of guessed. The truncate entry carries what the truncate did to the record (`TruncateTableInfo.tableStatsRecordCreated`, written from `AnalysisManager.resetTableStats()`, which now reports whether it had to create the record), and the replay applies exactly that (`replayResetTableStats(table, recordCreated)`): reset when the record exists, create only when the truncate created it. A truncate entry therefore no longer resurrects a record which a concurrent journaled `DROP STATS`/analyzer cleanup/statistics cleaner removed after it, while a table which had no statistics record still gets one on every frontend so that the rows loaded after the truncation stay accounted for. Entries written before this change carry no flag and only reset an existing record. Deterministic replay-order coverage, as requested: `AnalysisManagerTest.testReplayOfTruncateFollowsTheRecordedStatsTransition` applies the deletion first and then replays truncate entries for the three cases (did not create -> record stays absent; created -> record exists with `updatedRows = 0`; did not create on a table without a record -> no record). Residual, stated explicitly: the two `(mutate record, write journal entry)` pairs of a truncate and of a deletion are still not protected by a common lock, so an interleaving remains in which their apply order and journal order disagree (it now needs the record to be absent when the truncate creates it, plus the deletion applied after that creation and journaled before the truncate entry; the master then ends absent while the replay creates the record). Closing that requires writing both journal entries under the same lock as their record mutations, which touches the deletion paths of `DROP STATS`, the analyzer's empty table cleanup and the statistics cleaner and their journaling. I would rather land that as its own change than fold it into this PR — say the word if you want it here. Re-verified on this revision (local FE + BE cluster): `AnalysisManagerTest` 17/17 (with the new case), `TableStatsMetaTest` 8/8, `test_row_count_after_truncate` (including the aggregate rollup case) and `test_drop_stats_and_truncate` pass, and the end to end case still reports 3 rows after `TRUNCATE TABLE` + immediate load + immediate query instead of 1. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
