fallintoplace opened a new pull request, #1753: URL: https://github.com/apache/iceberg-go/pull/1753
Position delete planning currently sorts all delete files by sequence, then builds a file_path metrics evaluator and scans the remaining suffix for every data file. This adds a focused positional delete index that: - groups single-file deletes by referenced data path, using referenced_data_file or equal file_path bounds - groups the remaining deletes by spec and partition - sequence-sorts each bucket and binary-searches the first applicable delete - keeps the existing deletion vector path index and suppression behavior unchanged This follows #1752 but does not depend on it. Benchmarks on Apple M1 Pro: - 20,000 data files and 5,000 sparse path deletes: about 1.3 ms per indexed plan - 2,000 data files and 500 sparse path deletes: about 0.14 ms indexed vs 160 ms with the metrics suffix scan Tests: - go test ./... - go test -race ./table/... - go vet ./table/... - go test ./table -run ^$ -bench BenchmarkPositionalDelete -benchmem -benchtime=3x -count=3 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
