Thanks for the great work, this is super exciting. I've been looking at object storage optimizations lately, I'll find some time this week to take a look.
On Mon, Aug 3, 2026 at 8:19 AM Varun Lakhyani <[email protected]> wrote: > I am looking for views on keeping the one flag true by default to use > EagerInputFile in ManifestFiles read path (Major benefits in v4 Parquet > Manifests). > We have ready to run benchmark [1] and two independent sets of results > showing similar benefits ~25 - 55% reduction in reading Parquet manifest > depending on machine and file. > > I would appreciate the community's look and feedback on this - PR having > changes [2] and whether we can enable this path by default. > > [1] https://github.com/varun-lakhyani/iceberg/pull/1 > [2] https://github.com/apache/iceberg/pull/17284 > > On Wed, Jul 29, 2026 at 6:29 AM Russell Spitzer <[email protected]> > wrote: > >> I'm not sure if anyone else has checked this out yet, but I think it's a >> really exciting improvement. We should seriously consider making this a >> default feature in the next release. Or potentially just have it always be >> on. >> >> On Tue, Jul 28, 2026 at 4:58 PM Varun Lakhyani <[email protected]> >> wrote: >> >>> I think we can flag property to enable this as default true, It would be >>> a great improvement for v4 parquet manifest. >>> Tried to do benchmarkings as extensive as possible. >>> I would love to hear thoughts on this. >>> >>> Please review the PR once and would appreciate feedback. >>> >>> On Mon, Jul 20, 2026 at 10:02 PM Varun Lakhyani < >>> [email protected]> wrote: >>> >>>> Hello Everyone, >>>> >>>> I evaluated using EagerInputFile in v4 manifest reader path (parquet) >>>> using ManifestBenchmark and S3 as object store and EC2 machine for jmh >>>> benchmarking. >>>> >>>> This is to make sure we are doing a proper checklist while wiring this >>>> EagerInputFile path here, >>>> Detailed benchmarking setup and results are commented on in PR[1]. >>>> Using an EC2 machine in the same region (ap-south-1) gives *~28.4%* to >>>> *~42.7%* betterment while comparing against baseline default while >>>> benchmarking. >>>> >>>> Please provide review and feedback. >>>> Thanks >>>> >>>> [1] https://github.com/apache/iceberg/pull/17284 >>>> -- >>>> Lakhyani Varun >>>> Indian Institute of Technology Roorkee >>>> Contact: +91 96246 46174 >>>> >>>>
