This is awesome and really valuable, request others to review and conclude ,if needs plz perform own benchmark.
Regards, Viquar Khan On Mon, Aug 3, 2026, 11:15 AM Kevin Liu <[email protected]> wrote: > Thanks for the great work, this is super exciting. I've been looking at > object storage optimizations lately, I'll find some time this week to take > a look. > > On Mon, Aug 3, 2026 at 8:19 AM Varun Lakhyani <[email protected]> > wrote: > >> I am looking for views on keeping the one flag true by default to use >> EagerInputFile in ManifestFiles read path (Major benefits in v4 Parquet >> Manifests). >> We have ready to run benchmark [1] and two independent sets of results >> showing similar benefits ~25 - 55% reduction in reading Parquet manifest >> depending on machine and file. >> >> I would appreciate the community's look and feedback on this - PR having >> changes [2] and whether we can enable this path by default. >> >> [1] https://github.com/varun-lakhyani/iceberg/pull/1 >> [2] https://github.com/apache/iceberg/pull/17284 >> >> On Wed, Jul 29, 2026 at 6:29 AM Russell Spitzer < >> [email protected]> wrote: >> >>> I'm not sure if anyone else has checked this out yet, but I think it's a >>> really exciting improvement. We should seriously consider making this a >>> default feature in the next release. Or potentially just have it always be >>> on. >>> >>> On Tue, Jul 28, 2026 at 4:58 PM Varun Lakhyani < >>> [email protected]> wrote: >>> >>>> I think we can flag property to enable this as default true, It would >>>> be a great improvement for v4 parquet manifest. >>>> Tried to do benchmarkings as extensive as possible. >>>> I would love to hear thoughts on this. >>>> >>>> Please review the PR once and would appreciate feedback. >>>> >>>> On Mon, Jul 20, 2026 at 10:02 PM Varun Lakhyani < >>>> [email protected]> wrote: >>>> >>>>> Hello Everyone, >>>>> >>>>> I evaluated using EagerInputFile in v4 manifest reader path (parquet) >>>>> using ManifestBenchmark and S3 as object store and EC2 machine for jmh >>>>> benchmarking. >>>>> >>>>> This is to make sure we are doing a proper checklist while wiring this >>>>> EagerInputFile path here, >>>>> Detailed benchmarking setup and results are commented on in PR[1]. >>>>> Using an EC2 machine in the same region (ap-south-1) gives *~28.4%* >>>>> to *~42.7%* betterment while comparing against baseline default while >>>>> benchmarking. >>>>> >>>>> Please provide review and feedback. >>>>> Thanks >>>>> >>>>> [1] https://github.com/apache/iceberg/pull/17284 >>>>> -- >>>>> Lakhyani Varun >>>>> Indian Institute of Technology Roorkee >>>>> Contact: +91 96246 46174 >>>>> >>>>>
