Re: [DISCUSS] Parquet Java 1.16.0 release

[Aihua Xu]

Hi Gabor and all,

Here’s my current understanding of the progress on the *Variant* support in
Parquet:

   -

   Per Parquet's requirements, we need at least two reference
   implementations to finalize the Variant logical type specification.
   -

   The community is actively working on Java, Go, and Rust implementations:
   -

      Java already has the encoding and shredding implementations in place:
      -

         Variant Decoding <https://github.com/apache/parquet-java/pull/3197>
         -

         Variant Encoding <https://github.com/apache/parquet-java/pull/3202>
         -

         Variant Shredding Writer
         <https://github.com/apache/parquet-java/issues/3223>
         -

         Variant Shredding Reader
         <https://github.com/apache/parquet-java/issues/3211>
         -

      Go also includes encoding and shredding support:
      -

         Variant Encoding/Decoding
         <https://github.com/apache/arrow-go/pull/344>
         -

         Variant Shredding <https://github.com/apache/arrow-go/pull/434>
         -

      Rust is currently working on the shredding implementation.

In addition to these, we already have a full Variant implementation in
Apache Iceberg, as well as in some closed-source engines.

At this point, I’d like to check if we have enough implementation coverage
to move forward with finalizing the Variant spec. Would it make sense to
start a vote thread at this stage?

Ultimately, our goal is to release a new version of parquet-format and
parquet-java that includes the Variant logical type, so that Iceberg and
other engines can officially depend on it and proceed with further
implementation.

Let me know your thoughts and how we should proceed.

Thanks,

Aihua

On Sun, Jul 13, 2025 at 10:08 PM Gábor Szádovszky <ga...@apache.org> wrote:

> Hi,
>
> I was not able to open the recordings of the last meeting because of
> permission issues. (Shouldn't these be accessible for anyone?)
> So, I'm not sure if you have talked about this, but the Variant spec is
> still not final. Since parquet-java already has Variant support, how do we
> prevent writing potentially invalid Variant data with the proper logical
> types we will use for the finalized spec? Is it behind a feature flag?
>
> Cheers,
> Gabor
>
> Aihua Xu <aihu...@gmail.com> ezt írta (időpont: 2025. júl. 11., P, 19:33):
>
> > Hi community,
> >
> > As discussed in the last community sync-up meeting, I'd like to proceed
> > with releasing *Parquet-Java 1.16.0*, which will include support for
> > *geo-type* and *variant*.
> >
> > Please let me know if you have any objections or if you have any upcoming
> > changes you'd like to include in this release.
> > Thanks,
> > Aihua
> >
>