Hello Everyone, I have implemented a compaction benchmark with some parameters and results Would appreciate feedback on it, Maybe some better way to do anything there. Attached is results of the same total data scaling on a number of files comparing s3 and local fs.
Benchmark configs: Machine: MacBook Pro, Apple M4 (10 cores, 16 GB RAM), macOS 26.3.1, OpenJDK 21.0.10 Benchmark: JMH SingleShotTime, 3 warmup + 10 measurement iterations, 1 thread, 1 fork Data: 2,000,000 rows (fixed, 14.5Mb each), repartitioned into 250 / 500 / 1,000 / 2,000 files Storage(cloud): Amazon S3 (ap-south-1), FileIO: S3FileIO, rewrite strategy: rewrite-all=true PR - https://github.com/apache/iceberg/pull/16219 [image: image.png]
