matanper opened a new issue, #69861: URL: https://github.com/apache/doris/issues/69861
### Search before asking - [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues. ### Version 4.1.4 (cloud mode, packed files). master is the same: `cloud/src/recycler/recycler.cpp:1487` reads the recycle and tmp rowset keys in one transaction (`check_recycle_and_tmp_rowset_exists`), and `:1503` reads the formal rowset key in another (`check_rowset_exists`). ### Description When the recycler corrects a packed file, it decides for each small file (slice) whether its rowset is still alive: ```cpp if (check_recycle_and_tmp_rowset_exists(tablet_id, rowset_id, txn_id, &recycle_exists, &tmp_exists) != 0) ... ... } else { // neither recycle nor tmp key seen bool rowset_exists = false; if (check_rowset_exists(tablet_id, rowset_id, &rowset_exists, stats) != 0) ... if (!rowset_exists) { small_file->set_deleted(true); small_file->set_corrected(true); ... } ``` The two checks are separate transactions. If a compaction that consumes the rowset commits between them, the first read sees the formal key (no recycle key yet) and the second sees neither the formal key (removed by the commit) nor looks again at the recycle key (just written). The live, just-recycled rowset is taken as gone, its slice is marked deleted, and the packed file can be deleted before the compacted rowset's retention ends. Readers still on the old version (or anything relying on the retention window) then hit missing data. PR #68153 makes the packed-file *checker* read within one snapshot; this recycler correction path is separate. ### Solution Read the recycle, tmp and formal rowset keys of a slice in one transaction / snapshot (e.g. one `check_rowset_keys(...)` that returns all three), so a concurrent compaction commit is seen either entirely before or entirely after. ### How to Reproduce? With packed files enabled, inject a sync point between `check_recycle_and_tmp_rowset_exists` and `check_rowset_exists` in packed file correction, commit a compaction whose input includes the slice's rowset there, and observe the slice marked `deleted` while the rowset's recycle key exists. ### Anything Else? The race window is small, but the result is premature deletion of data still inside its retention window. ### Are you willing to submit PR? - [x] Yes I am willing to submit a PR! ### Code of Conduct - [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
