I compiled the latest snapshot against Iceberg and Spark today, and it all succeeded. I think it is good to also extend the BlockingDirectBinaryEncode <https://github.com/apache/avro/pull/2732>r to support nested structures in case we want to use it to write arbitrary Avro as well (next to the metadata which is the first objective). Let me know if there is anything that you would like to get into 1.12.
Kind regards, Fokko Op ma 12 feb 2024 om 13:57 schreef Martin Grigorov <mgrigo...@apache.org>: > On Mon, Feb 12, 2024 at 11:59 AM Fokko Driesprong <fo...@apache.org> > wrote: > > > Hey everyone, > > > > I wanted to bring the release up again. We now have Nanosecond support, > > Fixed bytes encoded UUIDs, and the directBlockingBinaryEncoders that > we're > > on the Iceberg side very much looking forward to. > > > > How about running a 1.12.0 release? After that, we can discuss the 2.0 > > release with major new features (dropping Java 8, new schema parser, > etc). > > WDYT? > > > > Agreed! > > > > > > Kind regards, > > Fokko Driesprong > > > > Op do 28 dec 2023 om 14:22 schreef Fokko Driesprong <fo...@apache.org>: > > > > > Hey everyone, > > > > > > Hope everyone had a great Christmas, and thanks for all the replies > here. > > > > > > Good idea. I would really like to include the changes necessary to > > >> complete AVRO-3666 (the new SchemaParser class). > > > > > > > > > I started a review on this one, but haven't finished yet. My main > concern > > > was around API compatibility. I noticed that a public API's changed, > and > > > tried to run the Iceberg tests against the PR to catch any breaking > > > changes. When doing so, I noticed that the checks around logical types > > > became more strict. In Iceberg we extend the Avro specification by > > storing UUIDs > > > as fixed binary <https://iceberg.apache.org/spec/#avro>. I've raised a > > > discussion > > > <https://lists.apache.org/thread/h4lvc2lk194om6cjm76wzosm22cop2xw> on > > the > > > mailing-list, and also Christophe created a PR > > > <https://github.com/apache/avro/pull/2652> to add this to the Java SDK > > > (thank you for that!). I'll finish the PR of the new parser, and I > would > > > also love to get the UUID support in there. > > > > > > Ideally, I'd also like to include a counterpart for formatting schemas, > > by that > > >> can also wait until 1.12.1. > > > > > > > > > Typically we don't backport features to earlier versions. 1.11 was > > > released in October 2021 which is way back. I'm in favor of doing a > major > > > release more often. > > > > > > We don't have a strict policy for new features in patch releases so why > > don't > > >> we get out a new 1.11.x release with the latest changes > > (DirectBlockingBinaryEncoder > > >> + nanoseconds + others) and wait for the 1.12.0 release for the > plugin + > > >> any maintenance policy changes (which versions will be maintained, > etc) > > >> and other eventual backward incompatible changes. > > > > > > > > > Adding nanoseconds in a patch version sounds awkward to me, and I don't > > > think this aligns with the public consensus around what should be > > included > > > in a patch release. I would just cut master as 1.12.0 and then once we > > get > > > the plugin in we can do a 1.13 release. WDYT? > > > > > > Any volunteer to be the release manager? > > > > > > > > > I'm happy to run the release! > > > > > > It's really irritating that we don't have a "clean" plan for updating > the > > >> website with new releases :O I'm really pleased with the new > website, > > >> and I think we have almost all of the elements in place to get this > > >> right. > > >> > > > > > > Ryan S., can you elaborate on the gaps here? > > > > > > Thanks everyone, and wish y'all a great 2024! > > > > > > Kind regards, > > > Fokko > > > > > > > > > > > > Op do 28 dec 2023 om 01:22 schreef David M. Carr <da...@carrclan.us>: > > > > > >> Just noticed this thread. > > >> > > >> > I would **love** to get the avro-gradle-plugin donation rolling so > > >> it would be off the plate of the current maintainer (and donator). > > >> > > >> I'd love that too (being the maintainer/donator mentioned). That > said, > > I > > >> agree that it probably doesn't make sense to block 1.12.0 for it. > That > > >> said... any thoughts on how we could accelerate getting it ready? I'm > > >> happy to contribute; just need some direction on what needs to be done > > and > > >> coordination. > > >> > > >> On Mon, Dec 18, 2023 at 9:34 AM Ismaël Mejía <ieme...@gmail.com> > wrote: > > >> > > >> > +1 to do a release. > > >> > > > >> > If we want to have a release 'soon' it is probably a good idea to > not > > >> wait > > >> > for the `avro-gradle-plugin` to be included since there is still a > > good > > >> > amount of work to be done to get it merged, so it will end up > delaying > > >> the > > >> > release. > > >> > > > >> > We don't have a strict policy for new features in patch releases so > > why > > >> > don't we get out a new 1.11.x release with the latest changes > > >> > (DirectBlockingBinaryEncoder + nanoseconds + others) and wait for > the > > >> > 1.12.0 release for the plugin + any maintenance policy changes > (which > > >> > versions will be maintained, etc) and other eventual backward > > >> incompatible > > >> > changes. > > >> > > > >> > Any volunteer to be the release manager? > > >> > > > >> > Regards, > > >> > Ismaël > > >> > > > >> > > > >> > On Fri, Dec 15, 2023 at 4:36 PM Ryan Skraba <r...@skraba.com> > wrote: > > >> > > > >> > > Hey -- I think a 1.12.0 is a great idea! > > >> > > > > >> > > There's two things I'd love to see included, but aren't > finished... > > >> > > > > >> > > 1. I would **love** to get the avro-gradle-plugin donation rolling > > so > > >> > > it would be off the plate of the current maintainer (and donator). > > >> > > > > >> > > 2. It's really irritating that we don't have a "clean" plan for > > >> > > updating the website with new releases :O I'm really pleased > with > > >> > > the new website, and I think we have almost all of the elements in > > >> > > place to get this right. > > >> > > > > >> > > All my best, Ryan > > >> > > > > >> > > On Wed, Dec 13, 2023 at 10:09 AM Martin Grigorov < > > >> mgrigo...@apache.org> > > >> > > wrote: > > >> > > > > > >> > > > +1 to release 1.12.0 ! > > >> > > > > > >> > > > BTW there were few other discussions at dev@ about how many > > >> branches > > >> > to > > >> > > > maintain once 1.12.0 is released, for how long, etc. I think > this > > is > > >> > also > > >> > > > an important topic to conclude on! > > >> > > > > > >> > > > Regards, > > >> > > > Martin > > >> > > > > > >> > > > On Tue, Dec 12, 2023 at 11:21 AM Fokko Driesprong < > > fo...@apache.org > > >> > > > >> > > wrote: > > >> > > > > > >> > > > > Hi everyone, > > >> > > > > > > >> > > > > I would like to discuss the Apache Avro 1.12.0 release. > Recently > > >> for > > >> > > Java, > > >> > > > > we have merged the DirectBlockingBinaryEncoder > > >> > > > > <https://github.com/apache/avro/pull/2521> and the support > for > > >> > > nanosecond > > >> > > > > timestamps <https://github.com/apache/avro/pull/2608>. Next > to > > >> that, > > >> > > there > > >> > > > > is much more that I think would be great to get out to the > > >> public. I > > >> > > would > > >> > > > > like to know if there is anything that you think should be > > >> included > > >> > in > > >> > > the > > >> > > > > release, so we can start planning the next release. Thoughts? > > >> > > > > > > >> > > > > Kind regards, > > >> > > > > Fokko > > >> > > > > > > >> > > > > >> > > > >> > > >> > > >> -- > > >> David M. Carr > > >> da...@carrclan.us > > >> > > > > > >
