Le 03/09/2026 à 01:08, Andrew Lamb a écrit :
I am not sure it is proof, but here is my logic:
Given any particular Parquet file, there is no simple, practical way to
understand what versions of what products/projects can read it.
I understand we have the compatibility chart[1] (which I helped to create),
but that doesn't have entries for end user systems (e.g. Spark version 3.5)
and I believe there is a widely held belief that writers must stick to very
conservative feature lists (see blogs below)
In my mind, having a single clear version in the file (this file is written
with version X Parquet features), makes it clear, unambiguous, and easy to
verify what systems can read it (e.g. DB X can read parquet files with
version Y).
My underlying question is: why is it easier if the version number is
written in the file, than simply by trying to read the file?
(you have to read the file anyway to get the version number that's
written in it :-))
Regards
Antoine.
