> I recall a separate discussion in our community sync meetings about deprecating ORC as well
I want to clarify that the purpose of this thread is only about manifest files. I like to keep the data file formats discussion separate (adding new or deprecating old). In v1-v3, manifest files are only in Avro format. As Anoop said, an upgraded v4 table can still contain pre-v4 Avro manifest files and read support for them is still needed. Here, we propose limiting v4 manifest files to Parquet format only, due to the benefits mentioned in this thread. On Wed, Aug 12, 2026 at 7:02 AM Péter Váry <[email protected]> wrote: > I think that, for both metadata and data files, we should keep the > infrastructure open to experimenting with and if we decide so, adopting new > file formats. File formats are a foundational layer of the ecosystem, and > tying ourselves to a single downstream project creates unnecessary risk. > > If that project is unwilling to accept features that are critical for our > use cases, or slow to adopt new technological advances, or even abandoned, > our ability to evolve would be constrained by decisions outside our > control. Maintaining format flexibility helps preserve innovation, > interoperability, and long-term resilience of Iceberg. > > Manu Zhang <[email protected]> ezt írta (időpont: 2026. aug. 12., > Sze, 5:20): > >> +1. I do not see any downside to making V4 manifests Parquet-only. Agree >> with Russell that we should converge on a single file format if possible (I >> recall a separate discussion in our community sync meetings about >> deprecating ORC as well). >> >> Thanks, >> Manu >> >> >> On Wed, Aug 12, 2026 at 3:30 AM Russell Spitzer < >> [email protected]> wrote: >> >>> I'm just pretty neutral here with a slight bias towards Parquet only. I >>> don't think supporting Avro is much more difficult, but that's mostly my >>> prejudice from the Java library. It's probably better for us to converge on >>> a single file format if possible so I think Parquet only is a good step in >>> that direction. >>> >>> On Tue, Aug 11, 2026 at 1:42 PM Anoop Johnson <[email protected]> wrote: >>> >>>> It would be great to make V4 manifest Parquet-only. This simplifies the >>>> spec and the implementation complexity and should improve read performance >>>> as Steven noted. Also we don't currently keep track of the root manifest >>>> format currently. So if we really want to support Avro root manifests, we >>>> need to start tracking the format. The alternative is to require writers to >>>> always produce the root manifest in Parquet. But then there is no point in >>>> supporting Avro leaf manifests if writers are required to produce root in >>>> Parquet anyway. >>>> >>>> Note that for v4 table upgrades we will support v3 leaf manifests which >>>> are in Avro - so the Parquet restriction applies to new V4 manifests only. >>>> >>>> Best, >>>> Anoop >>>> >>>> On Tue, Aug 11, 2026 at 11:29 AM Steven Wu <[email protected]> >>>> wrote: >>>> >>>>> Hi, >>>>> >>>>> During today’s column update sync, we discussed whether to keep the >>>>> Avro format option for V4 manifest files. The initial inclination was to >>>>> keep it because it already exists, even though Avro won’t support >>>>> projection reads on manifest files (including column stats). >>>>> >>>>> With both Avro and Parquet available, integrations and engines would >>>>> need to choose. We expect most will pick Parquet for projection-read >>>>> support. That raises a question: should we instead require Parquet for V4 >>>>> manifests to reduce the cognitive and decision burden on integrations? >>>>> >>>>> This also aligns with Iceberg’s usual priority of read >>>>> performance—especially query scan planning, where projecting column stats >>>>> from manifests matters. >>>>> >>>>> In the sync, most people generally agreed with making V4 manifests >>>>> *Parquet-only*. I’m sending this to collect any feedback or concerns >>>>> from the broader community before we settle on that direction. >>>>> Thanks, >>>>> Steven >>>>> >>>>> P.S. I uploaded the recording to the YouTube channel here >>>>> <https://youtu.be/dDZKzz3PKIk?t=2925> (relevant discussion started at >>>>> 48:45). >>>>> >>>>
