Hi, Starting from scratch is an option, but don't you think it's a huge effort ? Anyway, we will reuse part of the existing codebase.
Let's see what the team is thinking. Regards JB On 03/21/2018 09:26 AM, Romain Manni-Bucau wrote: > > > 2018-03-21 9:13 GMT+01:00 Jean-Baptiste Onofré <[email protected] > <mailto:[email protected]>>: > > Hi Romain, > > We didn't define a date yet. > > However, I think it makes lot of sense to think about that. > > What about creating a beam-2.x branch and move master version to > 3.0.0-SNAPSHOT ? > > > Do we want to "move" master or start fresh to avoid to pay again the legacy > which prevents us to move correctly forward ATM? > I really wonder if starting from scratch and only bringing stable and well > defined API wouldn't be saner after 10 months working with beam. > > Most of the actual logic will be importable easily when needed (I'm thinking > to > the runner) but just bumping the major will keep the same pitfall in the > codebase which are very basic design issues IMHO. > > > > I almost agree with your point even if I would suggest you to use a more > positive tone: being sharp never encourage the community, contribution > and don't > motivate people. You can say things but with some friendly form. > > > Read it as it is "I'm tired to always workaround the API" ;). Generally I > send a > mail after some hard fight and disappointment so mea culpa for this one. > > > > I would add: > > - Schema or PCollection: it's already started but I think we could do some > improvements (potentially introducing some API change) > - Hints/Annotations on PCollection: it's something we discussed during > Beam > Summit with Tyler and others. The idea is to mimic the Message Headers in > Apache > Camel. It would allow us to have more dynamic IOs and transforms, and > give some > additional statements to the runners. > > > +1 > > > > I'm proposing to start a vote to create the 2.x branch and move master to > Beam > 3.0.0-SNAPSHOT as join effort. > > Regards > JB > > On 03/21/2018 08:36 AM, Romain Manni-Bucau wrote: > > Hi guys, > > > > it got mentionned but without any concrete dates: when beam 3 work will > be started? > > > > I'm very interested in: > > > > 1. reworking the whole DAG API to ensure it is instrumentable (today > the dag > > uses a tons of static utilities and internals which makes it not > > industrializable at all as soon as you are just on top of beam) > > 2. reworking the API definition in its own module not coupled to any > > implementation details (api/provider design) and 100% based on the sdf > > 3. rework the overall serialization (coders + transform serialization > which is > > hardcoded today and not portable or industrializable at all) > > 4. make runners decorable properly and not just forked each time you > need to > > modify some behavior for a particular case > > (+ indeed all the issues we hit and saw on the list) > > > > Romain Manni-Bucau > > @rmannibucau <https://twitter.com/rmannibucau > <https://twitter.com/rmannibucau>> | Blog > > <https://rmannibucau.metawerx.net/ <https://rmannibucau.metawerx.net/>> > | > Old Blog > > <http://rmannibucau.wordpress.com <http://rmannibucau.wordpress.com>> > | Github <https://github.com/rmannibucau > <https://github.com/rmannibucau>> | > > LinkedIn <https://www.linkedin.com/in/rmannibucau > <https://www.linkedin.com/in/rmannibucau>> | Book > > > > <https://www.packtpub.com/application-development/java-ee-8-high-performance > > <https://www.packtpub.com/application-development/java-ee-8-high-performance>> > > -- > Jean-Baptiste Onofré > [email protected] <mailto:[email protected]> > http://blog.nanthrax.net > Talend - http://www.talend.com > > -- Jean-Baptiste Onofré [email protected] http://blog.nanthrax.net Talend - http://www.talend.com
