Hi, For 1.12, I would really like to include a change to the (new) SchemaParser API that was merged previously. The current version can return invalid Schemata (in case of missing references), which can be resolved with a special resolve(…) method. I’t like to return a ParseResult instead that enforces this, and thus ensures no invalid schemata can be returned.
I’ll make a new PR, a subset of #2642 <https://github.com/apache/avro/pull/2642>, to include just that. Kind regards, Oscar -- Oscar Westra van Holthe - Kind <[email protected]> > On 12 Feb 2024, at 22:53, Fokko Driesprong <[email protected]> wrote: > > I compiled the latest snapshot against Iceberg and Spark today, and it all > succeeded. I think it is good to also extend the BlockingDirectBinaryEncode > <https://github.com/apache/avro/pull/2732>r to support nested structures in > case we want to use it to write arbitrary Avro as well (next to the > metadata which is the first objective). Let me know if there is anything > that you would like to get into 1.12. > > Kind regards, > Fokko > > Op ma 12 feb 2024 om 13:57 schreef Martin Grigorov <[email protected]>: > >> On Mon, Feb 12, 2024 at 11:59 AM Fokko Driesprong <[email protected]> >> wrote: >> >>> Hey everyone, >>> >>> I wanted to bring the release up again. We now have Nanosecond support, >>> Fixed bytes encoded UUIDs, and the directBlockingBinaryEncoders that >> we're >>> on the Iceberg side very much looking forward to. >>> >>> How about running a 1.12.0 release? After that, we can discuss the 2.0 >>> release with major new features (dropping Java 8, new schema parser, >> etc). >>> WDYT? >>> >> >> Agreed! >> >> >>> >>> Kind regards, >>> Fokko Driesprong >>> >>> Op do 28 dec 2023 om 14:22 schreef Fokko Driesprong <[email protected]>: >>> >>>> Hey everyone, >>>> >>>> Hope everyone had a great Christmas, and thanks for all the replies >> here. >>>> >>>> Good idea. I would really like to include the changes necessary to >>>>> complete AVRO-3666 (the new SchemaParser class). >>>> >>>> >>>> I started a review on this one, but haven't finished yet. My main >> concern >>>> was around API compatibility. I noticed that a public API's changed, >> and >>>> tried to run the Iceberg tests against the PR to catch any breaking >>>> changes. When doing so, I noticed that the checks around logical types >>>> became more strict. In Iceberg we extend the Avro specification by >>> storing UUIDs >>>> as fixed binary <https://iceberg.apache.org/spec/#avro>. I've raised a >>>> discussion >>>> <https://lists.apache.org/thread/h4lvc2lk194om6cjm76wzosm22cop2xw> on >>> the >>>> mailing-list, and also Christophe created a PR >>>> <https://github.com/apache/avro/pull/2652> to add this to the Java SDK >>>> (thank you for that!). I'll finish the PR of the new parser, and I >> would >>>> also love to get the UUID support in there. >>>> >>>> Ideally, I'd also like to include a counterpart for formatting schemas, >>> by that >>>>> can also wait until 1.12.1. >>>> >>>> >>>> Typically we don't backport features to earlier versions. 1.11 was >>>> released in October 2021 which is way back. I'm in favor of doing a >> major >>>> release more often. >>>> >>>> We don't have a strict policy for new features in patch releases so why >>> don't >>>>> we get out a new 1.11.x release with the latest changes >>> (DirectBlockingBinaryEncoder >>>>> + nanoseconds + others) and wait for the 1.12.0 release for the >> plugin + >>>>> any maintenance policy changes (which versions will be maintained, >> etc) >>>>> and other eventual backward incompatible changes. >>>> >>>> >>>> Adding nanoseconds in a patch version sounds awkward to me, and I don't >>>> think this aligns with the public consensus around what should be >>> included >>>> in a patch release. I would just cut master as 1.12.0 and then once we >>> get >>>> the plugin in we can do a 1.13 release. WDYT? >>>> >>>> Any volunteer to be the release manager? >>>> >>>> >>>> I'm happy to run the release! >>>> >>>> It's really irritating that we don't have a "clean" plan for updating >> the >>>>> website with new releases :O I'm really pleased with the new >> website, >>>>> and I think we have almost all of the elements in place to get this >>>>> right. >>>>> >>>> >>>> Ryan S., can you elaborate on the gaps here? >>>> >>>> Thanks everyone, and wish y'all a great 2024! >>>> >>>> Kind regards, >>>> Fokko >>>> >>>> >>>> >>>> Op do 28 dec 2023 om 01:22 schreef David M. Carr <[email protected]>: >>>> >>>>> Just noticed this thread. >>>>> >>>>>> I would **love** to get the avro-gradle-plugin donation rolling so >>>>> it would be off the plate of the current maintainer (and donator). >>>>> >>>>> I'd love that too (being the maintainer/donator mentioned). That >> said, >>> I >>>>> agree that it probably doesn't make sense to block 1.12.0 for it. >> That >>>>> said... any thoughts on how we could accelerate getting it ready? I'm >>>>> happy to contribute; just need some direction on what needs to be done >>> and >>>>> coordination. >>>>> >>>>> On Mon, Dec 18, 2023 at 9:34 AM Ismaël Mejía <[email protected]> >> wrote: >>>>> >>>>>> +1 to do a release. >>>>>> >>>>>> If we want to have a release 'soon' it is probably a good idea to >> not >>>>> wait >>>>>> for the `avro-gradle-plugin` to be included since there is still a >>> good >>>>>> amount of work to be done to get it merged, so it will end up >> delaying >>>>> the >>>>>> release. >>>>>> >>>>>> We don't have a strict policy for new features in patch releases so >>> why >>>>>> don't we get out a new 1.11.x release with the latest changes >>>>>> (DirectBlockingBinaryEncoder + nanoseconds + others) and wait for >> the >>>>>> 1.12.0 release for the plugin + any maintenance policy changes >> (which >>>>>> versions will be maintained, etc) and other eventual backward >>>>> incompatible >>>>>> changes. >>>>>> >>>>>> Any volunteer to be the release manager? >>>>>> >>>>>> Regards, >>>>>> Ismaël >>>>>> >>>>>> >>>>>> On Fri, Dec 15, 2023 at 4:36 PM Ryan Skraba <[email protected]> >> wrote: >>>>>> >>>>>>> Hey -- I think a 1.12.0 is a great idea! >>>>>>> >>>>>>> There's two things I'd love to see included, but aren't >> finished... >>>>>>> >>>>>>> 1. I would **love** to get the avro-gradle-plugin donation rolling >>> so >>>>>>> it would be off the plate of the current maintainer (and donator). >>>>>>> >>>>>>> 2. It's really irritating that we don't have a "clean" plan for >>>>>>> updating the website with new releases :O I'm really pleased >> with >>>>>>> the new website, and I think we have almost all of the elements in >>>>>>> place to get this right. >>>>>>> >>>>>>> All my best, Ryan >>>>>>> >>>>>>> On Wed, Dec 13, 2023 at 10:09 AM Martin Grigorov < >>>>> [email protected]> >>>>>>> wrote: >>>>>>>> >>>>>>>> +1 to release 1.12.0 ! >>>>>>>> >>>>>>>> BTW there were few other discussions at dev@ about how many >>>>> branches >>>>>> to >>>>>>>> maintain once 1.12.0 is released, for how long, etc. I think >> this >>> is >>>>>> also >>>>>>>> an important topic to conclude on! >>>>>>>> >>>>>>>> Regards, >>>>>>>> Martin >>>>>>>> >>>>>>>> On Tue, Dec 12, 2023 at 11:21 AM Fokko Driesprong < >>> [email protected] >>>>>> >>>>>>> wrote: >>>>>>>> >>>>>>>>> Hi everyone, >>>>>>>>> >>>>>>>>> I would like to discuss the Apache Avro 1.12.0 release. >> Recently >>>>> for >>>>>>> Java, >>>>>>>>> we have merged the DirectBlockingBinaryEncoder >>>>>>>>> <https://github.com/apache/avro/pull/2521> and the support >> for >>>>>>> nanosecond >>>>>>>>> timestamps <https://github.com/apache/avro/pull/2608>. Next >> to >>>>> that, >>>>>>> there >>>>>>>>> is much more that I think would be great to get out to the >>>>> public. I >>>>>>> would >>>>>>>>> like to know if there is anything that you think should be >>>>> included >>>>>> in >>>>>>> the >>>>>>>>> release, so we can start planning the next release. Thoughts? >>>>>>>>> >>>>>>>>> Kind regards, >>>>>>>>> Fokko >>>>>>>>> >>>>>>> >>>>>> >>>>> >>>>> >>>>> -- >>>>> David M. Carr >>>>> [email protected] >>>>> >>>> >>> >>
