I think its immature to start a new major version at this point in time. * Apache Beam 2.x series is less then a year old. * Many features that users want can be built on top of the existing APIs.
On Wed, Mar 21, 2018 at 1:31 AM Jean-Baptiste Onofré <[email protected]> wrote: > Hi, > > Starting from scratch is an option, but don't you think it's a huge effort > ? > Anyway, we will reuse part of the existing codebase. > > Let's see what the team is thinking. > > Regards > JB > > On 03/21/2018 09:26 AM, Romain Manni-Bucau wrote: > > > > > > 2018-03-21 9:13 GMT+01:00 Jean-Baptiste Onofré <[email protected] > > <mailto:[email protected]>>: > > > > Hi Romain, > > > > We didn't define a date yet. > > > > However, I think it makes lot of sense to think about that. > > > > What about creating a beam-2.x branch and move master version to > > 3.0.0-SNAPSHOT ? > > > > > > Do we want to "move" master or start fresh to avoid to pay again the > legacy > > which prevents us to move correctly forward ATM? > > I really wonder if starting from scratch and only bringing stable and > well > > defined API wouldn't be saner after 10 months working with beam. > > > > Most of the actual logic will be importable easily when needed (I'm > thinking to > > the runner) but just bumping the major will keep the same pitfall in the > > codebase which are very basic design issues IMHO. > > > > > > > > I almost agree with your point even if I would suggest you to use a > more > > positive tone: being sharp never encourage the community, > contribution and don't > > motivate people. You can say things but with some friendly form. > > > > > > Read it as it is "I'm tired to always workaround the API" ;). Generally > I send a > > mail after some hard fight and disappointment so mea culpa for this one. > > > > > > > > I would add: > > > > - Schema or PCollection: it's already started but I think we could > do some > > improvements (potentially introducing some API change) > > - Hints/Annotations on PCollection: it's something we discussed > during Beam > > Summit with Tyler and others. The idea is to mimic the Message > Headers in Apache > > Camel. It would allow us to have more dynamic IOs and transforms, > and give some > > additional statements to the runners. > > > > > > +1 > > > > > > > > I'm proposing to start a vote to create the 2.x branch and move > master to Beam > > 3.0.0-SNAPSHOT as join effort. > > > > Regards > > JB > > > > On 03/21/2018 08:36 AM, Romain Manni-Bucau wrote: > > > Hi guys, > > > > > > it got mentionned but without any concrete dates: when beam 3 work > will be started? > > > > > > I'm very interested in: > > > > > > 1. reworking the whole DAG API to ensure it is instrumentable > (today the dag > > > uses a tons of static utilities and internals which makes it not > > > industrializable at all as soon as you are just on top of beam) > > > 2. reworking the API definition in its own module not coupled to > any > > > implementation details (api/provider design) and 100% based on the > sdf > > > 3. rework the overall serialization (coders + transform > serialization which is > > > hardcoded today and not portable or industrializable at all) > > > 4. make runners decorable properly and not just forked each time > you need to > > > modify some behavior for a particular case > > > (+ indeed all the issues we hit and saw on the list) > > > > > > Romain Manni-Bucau > > > @rmannibucau <https://twitter.com/rmannibucau > > <https://twitter.com/rmannibucau>> | Blog > > > <https://rmannibucau.metawerx.net/ < > https://rmannibucau.metawerx.net/>> | > > Old Blog > > > <http://rmannibucau.wordpress.com < > http://rmannibucau.wordpress.com>> > > | Github <https://github.com/rmannibucau < > https://github.com/rmannibucau>> | > > > LinkedIn <https://www.linkedin.com/in/rmannibucau > > <https://www.linkedin.com/in/rmannibucau>> | Book > > > > > < > https://www.packtpub.com/application-development/java-ee-8-high-performance > > < > https://www.packtpub.com/application-development/java-ee-8-high-performance > >> > > > > -- > > Jean-Baptiste Onofré > > [email protected] <mailto:[email protected]> > > http://blog.nanthrax.net > > Talend - http://www.talend.com > > > > > > -- > Jean-Baptiste Onofré > [email protected] > http://blog.nanthrax.net > Talend - http://www.talend.com >
