vaquarkhan commented on PR #17284: URL: https://github.com/apache/iceberg/pull/17284#issuecomment-5123857471
Thanks @varun-lakhyani for the thorough benchmark, the graph, and especially the S3 GET breakdown (3 -> 1). That request count is the strongest part of the story since it does not depend on the machine. I reproduced this on my own account (EC2 r7i.2xlarge, us-east-1, S3, JDK 17, same JMH warmup and measurement settings) and the direction holds cleanly: - Parquet v4, numCols=50, non-partitioned: 0.141 -> 0.063 s/op with eager on. - Avro v4: 0.060 -> 0.061 s/op, no change. My baseline matches yours closely (0.141 vs your 0.134). My eager number is lower (0.063 vs 0.100),most likely because of the newer instance: the eager path reads the whole file into memory and serves from RAM, so it gains from faster CPU and memory, while the network-bound default path stays about the same. Different hardware, same conclusion. <img width="980" height="630" alt="image" src="https://github.com/user-attachments/assets/023bd920-74f1-4b72-9aa6-48acef3e0b2a"; /> I also ran one extra benchmark, same read path, adding a local filesystem and a size sweep across the 1 MB threshold: 1. Local filesystem: eager gives no benefit, and for a small Parquet manifest it looked slightly slower (within noise). No round trip to collapse locally, so this is expected. This is why I would keep it behind the flag rather than always on. 2. Size sweep: below 1 MB the S3 Parquet win holds (~52-53%); above 1 MB it correctly does not fire. The 1 MB cap behaves as intended. 3. Avro vs Parquet baseline (eager off): on S3, a Parquet manifest read is about 1.9x-2.4x slower than an Avro manifest of the same size (non-overlapping error bars). On local disk there is no such gap. With eager on, the S3 Parquet read comes back down close to Avro. So eager fetch is really closing the object-store Parquet penalty, which lines up with your 3 -> 1 GET result. <img width="1400" height="728" alt="image" src="https://github.com/user-attachments/assets/58a61374-8b9c-44e2-9e94-f5640f9890b5"; /> Net: a solid win for Parquet manifests on object storage under the 1 MB cap, little or nothing for Avro or on local storage. I would keep the default off, or scope any future default to object store plus Parquet under the cap, rather than always on. One reproducibility note: the benchmark branch did not build/run for me as-is. The iceberg-core jmh source set has no iceberg-aws dependency and the AWS SDK is compile-only, so :iceberg-core:jmh fails to compile and then at runtime with missing AWS classes. I added iceberg-aws plus the SDK to the jmh classpath to get it running; might be worth including so others can reproduce. I'll share a Google Doc with the full numbers and charts, plus my benchmark class, this weekend -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
