Hmmm. I am pretty conservative in not wanting to make any changes, ever. That's the easiest way to offer great support between artifacts, versions and languages!
For this example (extending the spec to permit JSON number and string for floating point defaults), there's absolutely no impact on binary Avro encoding, which is where we should really be most change-resistant! There's an impact on Schemas, which means impact on Avro files. It's possible for Avro 1.11.0 to write a file or schema that would cause an error when being parsed by previous versions. However, we can currently turn off default value validation during parsing. The default value is not used when reading a file (as the actual/writer schema), but could also cause an error when used with schema evolution (as the expected/reader schema). We've made backwards-compatible changes to the schema format in the past (notably adding defaults for Schema evolution of enums). There's no impact to JSON encoding if you're only using finite doubles. Unfortunately, JSON encoding is defined in terms of defaults, so Avro 1.11.0 (with the extension) could create JSON encoded data that is unreadable by previous versions. I just checked, and the Java SDK is currently producing invalid JSON for non-finite double values, and it can't deserialize the JSON it produces. So this could be considered more of a bugfix for the spec than an extension. I'm not sure what the right answer is! Maybe we should just say non-finite floating points are undefined for default values and JSON encoding (which is pretty much the current state). We probably should write down some place what level of support we have for backwards/forwards compatible changes for different things in the spec and language SDKs. I'm more interested in having that clear in my mind ! All my best, Ryan On Thu, Jun 11, 2020 at 7:29 PM Sean Busbey <bus...@apache.org> wrote: > > Sounds like a reasonable spec change. I think this would be our first > breaking spec change? does that mean this is the time for us to hash > out versioning the spec, or are we going to update all of our versions > to 2.x? > > Can we get an idea on failure expectations for the SDKs we maintain? > > On Wed, Jun 10, 2020 at 9:49 AM Ryan Skraba <r...@skraba.com> wrote: > > > > Hello! > > > > There's this problem and fix that would require a spec change: > > > > https://issues.apache.org/jira/browse/AVRO-2858 > > > > Currently NaN and Infinity are valid IEEE 754 floating point values, > > but only finite values are permitted in JSON numbers. > > > > What do you think about encoding Avro numeric types either as a JSON > > number or a JSON string? I think this is probably more in line with > > the intention of default values than failing when "123.45" is > > encountered, or excluding NaN / Infinity from possible default values. > > > > The consequence of making this official is that new schemas might > > cause an existing SDK for a language to fail. I'm not sure this is > > currently the case... to verify. > > > > Best regards, Ryan > > > > -- > Sean
