Le 03/09/2026 à 12:31, Andrew Lamb a écrit :
why is it easier if the version number is written in the file, than
simply by trying to read the file?
In my opinion, it is easier because you can reason/communicate about a
single thing (version number) and compare that quickly to documented
compatibility levels in projects/systems rather than having to empirically
test your file with all your potential readers.
I do agree that such a version could technically be achieved without
changing what is written in the file (e.g. to make some tool to check what
features are used), but the check would need to be more complicated than
simply looking at a version
I think there are two limitations here:
1) readers don't necessarily support "only V3" or "all of V3", so a
simple version flag in the footer is not an exact answer to the question
"will the reader be able to read the file"?
2) users can ask for a subset of a file's contents (especially columns),
such that, for example, a ALP-encoded float column in a Parquet "V3"
file need not stop a user who only wants to read int and string columns
from that file with a pre-"V3" Parquet reader.
So, in the end, writing this version number in the footer would mostly
be informational. I guess that can make it useful anyway, but less than
seems to be believed by the proponents of this feature :-)
Regards
Antoine.
