JeonDaehong commented on PR #18027: URL: https://github.com/apache/iceberg/pull/18027#issuecomment-5664923219
@pvary > Could you please add JMH test to show the gains? Added PositionDeleteIndexBenchmark in core/src/jmh. It reads 5M positions in 5,000-position batches, the way the vectorized reader does, at densities on either side of the array/bitmap boundary. JMH 1.37, JDK 21.0.10: | Benchmark | (density) | Mode | Cnt | Score | Error | Units | | :--- | :---: | :---: | :---: | :---: | :---: | :---: | | `PositionDeleteIndexBenchmark.probePerPosition` | 0.5 | ss | 5 | 82.893 | ± 13.257 | ms/op | | `PositionDeleteIndexBenchmark.probePerPosition` | 6.1 | ss | 5 | 111.101 | ± 43.291 | ms/op | | `PositionDeleteIndexBenchmark.probePerPosition` | 12.0 | ss | 5 | 68.924 | ± 18.444 | ms/op | | `PositionDeleteIndexBenchmark.traverseRange` | 0.5 | ss | 5 | 0.369 | ± 0.378 | ms/op | | `PositionDeleteIndexBenchmark.traverseRange` | 6.1 | ss | 5 | 1.304 | ± 0.891 | ms/op | | `PositionDeleteIndexBenchmark.traverseRange` | 12.0 | ss | 5 | 2.456 | ± 1.660 | ms/op | The spread is wide at five single-shot iterations, but the two arms are two orders of magnitude apart, so the comparison holds either way. Two things are worth noting here: - **First, the per-position cost is not monotonic in density:** 6.1% is the worst case because the container is still a sorted array near its deepest, and 12% is cheaper because it has flipped to a bitmap. - **Second, this ratio is much larger than the 2.6–9.3x quoted for the Spark path:** This is because this benchmark measures only the index lookup performance, whereas the delete-check subtree in a real scan also contains DV deserialization, the mapping array write, and the index wrapper, none of which this change touches. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
