JeonDaehong opened a new pull request, #1148: URL: https://github.com/apache/paimon-rust/pull/1148
### Purpose Linked issue: close #1146 Turning a deletion vector into non-deleted row ranges walked it one deleted row at a time, so deleting one big block cost as much as deleting the same number of scattered rows. ### Brief change log - `DeletionVector::deleted_ranges()` returns deleted runs. Short runs are walked with `next`; after 4 adjacent positions it switches to roaring's `next_range` for the rest of the run, joining runs split at container boundaries. `next_range` alone was ~6x slower on scattered deletes. - `dv_to_non_deleted_ranges` (data_file_reader.rs) and `non_deleted_local_ranges` (data_evolution_reader.rs) use it. ### Tests - `test_deleted_ranges_edges`: empty, runs of exactly 4 and 5, a run across several containers, runs ending at `u32::MAX`. - `test_deleted_ranges_match_model_on_random_bitmaps`: compares against grouping `bitmap.iter()`. - `dv_to_non_deleted_ranges_handles_long_runs`: runs across containers and past `row_count`. Ran the same steps as ci.yml on Linux: fmt, clippy (1.98.0, `fulltext,vortex`), msrv check, `paimon` unit tests, rest-server, and after `make docker-up` the rust, datafusion, vortex, lumina, python and go suites. All pass. I did not run the jindo and oss-cpp suites. Read timings are in #1146 (4M-row file: 15.4 → 2.6 ms with half the rows deleted in one block, no change for scattered deletes). ### API and Format No API or format change. `deleted_ranges` is `pub(crate)`. ### Documentation None. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
