Hi all, This proposal already had reviews from four different committers on GitHub and I think it's in good shape to call for a vote. As was noted, it closely matches the Parquet file logical type and I think it makes sense to add it to canonical extension types in Arrow.
Best, Rok On Wed, Nov 4, 2026 at 7:10 PM Mandukhai Alimaa <[email protected]> wrote: > Hi everyone, > > Bumping this thread to see if anyone has any additional feedback or > thoughts on format PR #51248 [1]. > > If there are no further comments or objections, we will start a vote soon. > > Thanks! > > [1] https://github.com/apache/arrow/pull/51248 > > On Thu, Sep 17, 2026 at 2:22 PM Mandukhai Alimaa > <[email protected]> wrote: > > > > Hi everyone, > > > > Thanks for the feedback. Following the precedent set by > > "arrow.parquet.variant", the extension name will be > > "arrow.parquet.file". I have opened a format change PR [1], please > > take a look. Thanks. > > > > [1] https://github.com/apache/arrow/pull/51248 > > > > On Wed, Sep 2, 2026 at 9:43 PM Neelesh Salian <[email protected]> > wrote: > > > > > > +1 to use "parquet.file" as well. > > > > > > On Wed, Sep 2, 2026 at 7:29 PM Matt Topol <[email protected]> > wrote: > > > > > > > We ended up doing "arrow.parquet.variant" so should we do > > > > "arrow.parquet.file" for symmetry? > > > > > > > > On Wed, Sep 2, 2026, 10:20 PM Gang Wu <[email protected]> wrote: > > > > > > > > > +1 for "parquet.file" > > > > > > > > > > I would expect it to be very similar to how we define > "parquet.variant". > > > > > > > > > > On Thu, Sep 3, 2026 at 5:22 AM Rok Mihevc <[email protected]> > wrote: > > > > > > > > > > > > Shouldn't the extension name be `parquet.file`? It's really a > direct > > > > > > > mapping of the Parquet FILE type, not something we decided to > > > > > > > standardize independently. > > > > > > > > > > > > parquet.file sounds good in that it makes it clear where the > > > > > specification > > > > > > comes from. > > > > > > > > > > > > > Also I wonder if it's the Arrow community's role to > standardize such > > > > > > > extension types, or if we should let the Parquet community do > it? > > > > > > > > > > > > We have non-arrow namespaced types in geoarrow [1], but they are > > > > > specified > > > > > > in geoarrow repo. Are you proposing that? > > > > > > > > > > > > Parquet already specified the file type, we'd just be adding a > mirror > > > > > type > > > > > > in Arrow to make handling it practical. > > > > > > > > > > > > [1] > https://github.com/geoarrow/geoarrow/blob/main/extension-types.md > > > > > > > > > > > > Rok > > > > > > > > > > > > On Wed, Sep 2, 2026 at 11:05 PM Antoine Pitrou < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > > > > Shouldn't the extension name be `parquet.file`? It's really a > direct > > > > > > > mapping of the Parquet FILE type, not something we decided to > > > > > > > standardize independently. > > > > > > > > > > > > > > Also I wonder if it's the Arrow community's role to > standardize such > > > > > > > extension types, or if we should let the Parquet community do > it? > > > > > > > > > > > > > > Regards > > > > > > > > > > > > > > Antoine. > > > > > > > > > > > > > > > > > > > > > > > > > > > > Le 02/09/2026 à 22:51, Mandukhai Alimaa a écrit : > > > > > > > > Hi everyone, > > > > > > > > > > > > > > > > Following the recent addition and subsequent refinement of > the FILE > > > > > > > > LogicalType to the Parquet format specification [1, 2] and > the > > > > > > > > corresponding Parquet C++ > > > > > > > > reader/writer support currently underway [3], we (@rok and > I) would > > > > > > > > like to propose defining a matching Canonical Extension Type > for > > > > the > > > > > > > > Apache Arrow specification: arrow.file. > > > > > > > > > > > > > > > > Motivation > > > > > > > > As storage formats standardize file-level references, Arrow > needs a > > > > > > > > canonical in-memory representation to preserve these > semantics > > > > across > > > > > > > > IPC streams, the C Data Interface, Arrow Flight, and language > > > > > > > > bindings. Without a canonical extension type, file fields > lose > > > > their > > > > > > > > semantic metadata during serialization. > > > > > > > > > > > > > > > > Proposed High-Level Design: > > > > > > > > The intended design is to directly align with the finalized > Parquet > > > > > > spec > > > > > > > [4]. > > > > > > > > > > > > > > > > * Extension name: ``arrow.file``. > > > > > > > > * Storage Type: StructType containing standard optional > fields: > > > > > > > > * ``uri`` (Utf8) > > > > > > > > * ``offset`` (Int64) > > > > > > > > * `` size`` (Int64) > > > > > > > > * ``content_type`` (Utf8) > > > > > > > > * `` checksum`` (Utf8) > > > > > > > > * ``inline`` (Binary) > > > > > > > > > > > > > > > > I have created a GitHub issue to track this [5]. Please > share your > > > > > > > > thoughts! Thanks! > > > > > > > > > > > > > > > > [1] https://github.com/apache/parquet-format/pull/585 > > > > > > > > [2] https://github.com/apache/parquet-format/pull/603 > > > > > > > > [3] https://github.com/apache/arrow/issues/50891 > > > > > > > > [4] > > > > > > > > > > > > > > > > > > > > > > > https://github.com/apache/parquet-format/blob/master/LogicalTypes.md#file > > > > > > > > [5] https://github.com/apache/arrow/issues/51143 > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
