Hi, After discussing with Micah, I've split out the spec change to describe how unsupported logical/physical type combinations should be handled by readers (TLDR the logical type and stats should be dropped). I've opened a parquet-format PR [1] and parquet-java PR [2] for the spec change and reference implementation. The previous spec change [3] and implementation are still open and ready for review. Please take a look if interested.
Best, Div [1] https://github.com/apache/parquet-format/pull/606 [2] https://github.com/apache/parquet-java/pull/3711 [3] https://github.com/apache/parquet-format/pull/601 [4] https://github.com/apache/parquet-java/pull/3680 On Tue, Jul 21, 2026 at 7:42 PM Divjot Arora <[email protected]> wrote: > Hi folks, > > I've opened PRs for parquet-format [1] and parquet-java [2] to support the > FLBA(12) physical type for the TimestampType logical annotation. Please > take a look if interested and comment. > > Best, > Div Arora > > [1] https://github.com/apache/parquet-format/pull/601 > [2] https://github.com/apache/parquet-java/pull/3680 > > On Thu, Jul 16, 2026 at 4:16 PM Russell Spitzer <[email protected]> > wrote: > >> I'd lean toward choosing a bit width and going forward for similar >> reasons. >> I don't think a lot >> of engines/users are going to custom-define their bit-width ahead of time. >> It seems more >> likely if we wanted to benefit readers, we would just store the max >> used-bit width as a stat. >> >> On Thu, Jul 16, 2026 at 12:30 AM Micah Kornfield <[email protected]> >> wrote: >> >> > Being able to use compact data types is a good thing, no? :-) >> > >> > >> > Generally yes, but I think there is a difference here between Decimal >> and >> > Timestamp. Decimal a user is specifying precision and scale up front, >> so we >> > can calculate the exact FLBA width. The closest thing we have with >> > timestamp is for SQL purposes it has to cover a 10K year range. But in >> > general, we won't know the exact min and max value up front. >> > >> > For existing granularities of milliseconds and microseconds there is not >> > any benefit to using FLBA (users should use existing physical integer >> > values). Extra engineering effort for these granularities doesn't seem >> > worthwhile. >> > >> > For nanoseconds, I believe we could use the FLBA<8> type. However, some >> > systems are starting to move toward picoseconds [1][2] (and I've heard >> > femtoseconds discussed) so we will eventually want to support all of >> those >> > ranges. FLBA<9> allows us to do it all with the same code. >> > >> > I think extending the spec to support different size types is certainly >> > something we could do in the future, but doing it now seems like it is >> just >> > more code to maintain without a concrete use-case. >> > >> > Cheers, >> > Micah >> > >> > [1] >> > >> > >> https://docs.cloud.google.com/bigquery/docs/reference/storage/rpc/google.cloud.bigquery.storage.v1?content_ref=read%20api%20will%20return%20full%20precision%20picosecond%20value%20the%20value%20will%20be%20encoded%20as%20a%20string%20which%20conforms%20to%20iso%208601%20format#picostimestampprecision >> > [2] https://trino.io/docs/current/language/types.html#timestamp-p >> > >> > On Fri, Jul 10, 2026 at 2:38 AM Antoine Pitrou <[email protected]> >> wrote: >> > >> > > >> > > Le 10/07/2026 à 11:27, Divjot Arora via dev a écrit : >> > > >> Yeah, well, regardless, this is derailing the discussion quite a >> bit. >> > > > This proposal is not about extending DBP to FLBA. Can we keep it on >> > > topic? >> > > > >> > > > Agreed, let's keep this discussion focused to extended-precision >> > > > timestamps. We can have separate discussions on encodings.. >> > > > >> > > >> And conversely, I do not see what we get out of mandating FLBA<12>. >> > :-) >> > > > >> > > > FLBA<N> works, but I would be concerned about added complexity on >> the >> > > > reader side as I assume engines will want to use compact data types >> to >> > > > represent timestamps as much as possible, so they'd have to manage >> > > > different data types for different values of N. >> > > >> > > Being able to use compact data types is a good thing, no? :-) >> > > >> > > The good news is that some readers already have similar code for >> reading >> > > Decimals from arbitrary FLBA<N>. For example, Parquet C++ can read >> > > Decimal-annotated FLBA<N> into Arrow Decimal128 or Decimal256, >> depending >> > > on the advertised precision. >> > > >> > > (and the code to do that is reasonably simple: you're really >> > > sign-extending from one integer width to another) >> > > >> > > Regards >> > > >> > > Antoine. >> > > >> > > >> > > >> > >> >
