> Personally, I don't think readers should up front refuse reading a file > that has a version number higher than their official compatibility > level. Other people (Micah, Will, at least) see to be of the same opinion.
In my mind, writing a version number into a file does not obligate any particular behavior in the reader; It does not have to refuse reading any files. A version number summarizes, in a single number, what features may be encountered in that file. The reader is perfectly free to try and read files for which it may not support all features. The only obligation to readers would be that if they say "supports Apache Parquet V3" they do indeed support all features of V3 (or whatever version) Andrew p.s. I really like Will's formulation of the tension between long term stable storage and platform vendor upgrade cycles. On Fri, Sep 4, 2026 at 5:20 PM Russell Spitzer <[email protected]> wrote: > Let me try to clarify, as I still think we are talking about different > things. > > The version marker on a file is the oldest format version whose structure > that file actually uses. A file labeled V3 contains a structure a V2 reader > cannot correctly parse, or a semantic change it will get wrong. There is no > partial-success path for a V2 reader on that file. (I believe this is the > point Micah was trying to probe.) If a V2 reader could read it at all, > labeling it V3 would be incorrect; the writer should have labeled it V2. > The file becomes V3 when it contains a breaking change that older readers > cannot handle, causes incorrect results, or that they would mishandle > without a clear error. That is why a new encoding does not, by itself, bump > the file version. A V2 reader can see an encoding or logical type it does > not understand and fail on that column. > > Separately, “this engine supports V3” is a claim about the reader, not a > stamp on every file that uses a V3-owned feature. A V3 reader is expected > to handle the V3 bundle (including encodings that may also appear as > preview in V2 files) by default. The encoding is owned by V3; the file is > only V3 if it also contains the breaking change. > > If anyone has had different interpretations please let me know, but this > has been my understanding > > > On Fri, Sep 4, 2026 at 3:32 PM Antoine Pitrou <[email protected]> wrote: > > > > > Hey Ryan, > > > > Let's get the process issue straight. We voted A, not A + B. Perhaps you > > hoped that A implies B, but that is your personal opinion on the topic. > > > > (as a reminder, you were the one who insisted on a first vote on a very > > minimal proposition, before discussing the specifics) > > > > Having the community accept B implies another discussion and a vote. And > > that's the discussion we're having currently, actually, so I don't even > > know why you seem to be objecting. > > > > Regards > > > > Antoine. > > > > > > > > Le 04/09/2026 à 22:02, Ryan Blue a écrit : > > > From the doc, here's the point that we've been discussing: > > > > > >> Readers need a simple mechanism to know whether they support all > > features > > > they may encounter when reading a specific file. > > > > > > This is a follow-up to decide between options: > > > - Set footer version to 3 (for example) > > > - Set footer version to 3+ when preview features are present > > > - Set footer version to 3/3+ and record preview feature names > > > - Use magic bytes for version - PAR3 > > > > > > From this, it appears to me that most of us (at least those of us > > > discussing it) have been working under the assumption that our vote > meant > > > using the version by writing it into files. Personally, I think this is > > > also a natural conclusion. A reader needs to know what version a file > was > > > written with in order to decide whether it has guarantees (which is > > another > > > way to say what's in the versioning doc). > > > > > > I don't think that this is a dishonest or bad faith argument. > > > > > > My suggestion to ask the community is really to clarify what we all > think > > > we voted on. If there was genuine confusion about this then let's talk > > > through it. But, again, I don't think that this was unclear to most > > people. > > > > > > Do you agree that it's reasonable to ask what people thought they were > > > agreeing to? > > > > > > Ryan > > > > > > On Fri, Sep 4, 2026 at 12:26 PM Antoine Pitrou <[email protected]> > > wrote: > > > > > >> > > >> Le 04/09/2026 à 20:35, Ryan Blue a écrit : > > >>> It sounds like you're suggesting that we don't version data files. I > > >> don't > > >>> think that makes sense with versioning and I'm pretty sure most > people > > >> that > > >>> voted to use versions would agree with me. > > >> > > >> The *only* implication of the vote was to move forward with the usage > of > > >> community-agreed versions (rather than, say, mechanistic > calendar-based > > >> presets) to signal levels of compatibility between readers and > writers. > > >> > > >> If you wanted the vote to mean something more, then you should have > had > > >> the vote explicitly mention it. Anything else is dishonest. > > >> > > >>> Should I start a thread dedicated to asking the > > >>> community whether most people thought that "using versions" would > mean > > >>> writing the version used for a file in the file metadata? > > >> > > >> No, you should wait for this discussion to happen in the hope of > > >> reaching a consensus. > > >> > > >> Personally, I don't think readers should up front refuse reading a > file > > >> that has a version number higher than their official compatibility > > >> level. Other people (Micah, Will, at least) see to be of the same > > opinion. > > >> > > >> Regards > > >> > > >> Antoine. > > >> > > >> > > >> > > > > > > > > > >
