varun-lakhyani commented on PR #17284: URL: https://github.com/apache/iceberg/pull/17284#issuecomment-5133489386
Thanks @vaquarkhan for the review and extensive Benchmark run. Agreed! numbers are almost similar to my run, although I ran for v1-v4 Avro and v4 Parquet but not for local. I posted my JMH results and s3 bucket logs which conclusively shows 3 -> 1 GET calls on S3 for reference https://github.com/varun-lakhyani/iceberg-manifest-eagerpath-benchmark/blob/main/README.md I am hoping the see community's view on keeping it default true. Avro path showed no benefit as expected and local run also showed no result or little overhead as expected. Let's see if we need some checks along with flag so it doesn't use this eagerPath in local and or for Avro Also, I pushed you commit to the [benchmark branch ](https://github.com/varun-lakhyani/iceberg/pull/1) anyone can fork it and run the benchmark as is, included the aws dependencies so its easy for folks to just run the benchmarks as is. Thanks -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
