laskoviymishka opened a new pull request, #852:
URL: https://github.com/apache/iceberg-go/pull/852
…essure
Add TestReadPerfUnderDeletePressure and BenchmarkScanWithEqualityDeletes
that measure how Scan.ToArrowRecords latency scales with the number of equality
delete files. Each delete file is committed individually to simulate real CDC
workloads where deletes accumulate over time.
```
Results on Apple M3 (1000 data rows):
0 delete files: 1.8ms (baseline)
10 delete files: 3.3ms (1.8x)
50 delete files: 10.2ms (5.8x)
100 delete files: 18.5ms (10.4x)
200 delete files: 34.4ms (19.5x)
```
Degradation is ~linear with delete file count. Each delete file requires a
separate hash-set anti-join during scan. This documents the baseline that
compaction (#832) and deletion vectors (#589) will improve.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
