Done, thanks.
Romain Manni-Bucau @rmannibucau <https://twitter.com/rmannibucau> | Blog <https://rmannibucau.metawerx.net/> | Old Blog <http://rmannibucau.wordpress.com> | Github <https://github.com/rmannibucau> | LinkedIn <https://www.linkedin.com/in/rmannibucau> | Book <https://www.packtpub.com/application-development/java-ee-8-high-performance> 2018-03-15 2:26 GMT+01:00 Jean-Baptiste Onofré <[email protected]>: > Fully agree. > > If nobody jumps on it, I will tackle that tomorrow. > > Regards > JB > Le 14 mars 2018, à 18:03, Reuven Lax <[email protected]> a écrit: >> >> Can you add a description and assign a reviewer (using R:). I think this >> was basically ready to merge before modulo breaking compilation. >> >> >> On Wed, Mar 14, 2018 at 10:01 AM Romain Manni-Bucau < >> [email protected]> wrote: >> >>> up, know it missed the 2.4 but can https://github.com/apache/ >>> beam/pull/4790 have some love? it really makes beam pretty unusable >>> with direct runner, >>> I start to have "// workaround for BEAM-3409" everywhere in my codebase >>> which is quite bothering after 3 months :( >>> >>> >>> Romain Manni-Bucau >>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>> <https://rmannibucau.metawerx.net/> | Old Blog >>> <http://rmannibucau.wordpress.com> | Github >>> <https://github.com/rmannibucau> | LinkedIn >>> <https://www.linkedin.com/in/rmannibucau> | Book >>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>> >>> 2018-03-06 14:44 GMT+01:00 Romain Manni-Bucau <[email protected]>: >>> >>>> Hi guys, >>>> >>>> tried to reapply the waitUntilFinish fix - without breaking the >>>> compilation this time ;) - in https://github.com/apache/beam/pull/4790, >>>> anyone able to help to review and move that forward? >>>> >>>> >>>> Romain Manni-Bucau >>>> @rmannibucau <https://twitter.com/rmannibucau> | Blog >>>> <https://rmannibucau.metawerx.net/> | Old Blog >>>> <http://rmannibucau.wordpress.com> | Github >>>> <https://github.com/rmannibucau> | LinkedIn >>>> <https://www.linkedin.com/in/rmannibucau> | Book >>>> <https://www.packtpub.com/application-development/java-ee-8-high-performance> >>>> >>>> 2018-03-02 9:28 GMT+01:00 Robert Bradshaw <[email protected]>: >>>> >>>>> On Fri, Mar 2, 2018 at 12:01 AM Jean-Baptiste Onofré <[email protected]> >>>>> wrote: >>>>> >>>>>> Hi Ismaël, >>>>>> >>>>>> that's a good idea to show history. >>>>>> >>>>>> For me, the vote duration doesn't matter as we are in the release >>>>>> process already. >>>>>> >>>>> >>>>> A more relevant duration to track would probably be cut to final >>>>> release. This both measures our investment in the release process, as well >>>>> as how behind head is when we finally get the release out. >>>>> >>>>> >>>>>> The gap between two releases is more significant. >>>>> >>>>> >>>>> +1, this is what users see. (To clarify terminology, the "time between >>>>> release" is the time between actually releasing x.y and x.y+1 that is most >>>>> visible to end users, regardless of intermediate process like cutting and >>>>> voting that we have.) Of course this gets thrown off if our >>>>> time-to-prepare-a-release becomes a significant fraction of the desired >>>>> time-between-releases. >>>>> >>>>> And clearly with an average of >>>>>> 80 days (~ 3 months) it's two long. The idea is to reduce this >>>>>> clearly. I >>>>>> propose two months previously (including the vote period), so meaning >>>>>> 6 weeks >>>>>> between releases. >>>>>> >>>>> >>>>> It seems there have been proposals for monthly, every 6 weeks >>>>> (sesquimonthly?), and bimonthly. >>>>> >>>>> >>>>>> Regarding the time we take for the PR review and merge, I think it's >>>>>> a fair time >>>>>> to give us time to include new improvements and features, but also to >>>>>> fix bugs. >>>>>> >>>>> >>>>> Concrete deadlines can provide good motivation to get around to doing >>>>> reviews, cleaning up PRs, fixing bugs, etc. that you've been meaning to do >>>>> but for whatever reason keep putting off. So I think it's still good >>>>> practice to have some lead time that a release is coming for a chance for >>>>> folks to get stuff in, while still being clear that we're not holding >>>>> things back for new features and if you don't make the cut another one is >>>>> close behind. >>>>> >>>>> Regards >>>>>> JB >>>>>> >>>>>> On 03/01/2018 06:17 PM, Ismaël Mejía wrote: >>>>>> > The average time just in the vote process for Beam since we are out >>>>>> of the >>>>>> > incubator is 17.5 days with an average of 75 days between versions. >>>>>> > >>>>>> > Version Vote Period No. days >>>>>> > 2.3.0 30/01-16/02 17 days (83 days since last) >>>>>> > 2.2.0 27/10-25/11 29 days (101 days since last) >>>>>> > 2.1.0 11/07-16/08 36 days (92 days since last) >>>>>> > 2.0.0 08/05-16/05 8 days (62 days since last) >>>>>> > 0.6.0 10/03-15/03 5 days (37 days since last) >>>>>> > 0.5.0 27/01-06/02 10 days >>>>>> > >>>>>> > I think we should have these numbers into account to refine the >>>>>> distance between >>>>>> > releases. If we want to follow strict time-based releases, what we >>>>>> can probably >>>>>> > refine is how we cut the release so we try to reduce release >>>>>> overlaps and avoid >>>>>> > rushing unnecessarily. >>>>>> > >>>>>> > Maybe we should follow the proposed 6 weeks for the next release >>>>>> like this: >>>>>> > >>>>>> > - 4 weeks let’s say just after succesful vote and then cut the >>>>>> release branch. >>>>>> > - 1 week to burn the blocker list (good to have ones that don’t >>>>>> make will be >>>>>> > moved to the next release). >>>>>> > - 1 week for the vote + RCs (in case the vote takes longer at least >>>>>> the overlap >>>>>> > between vote + next dev cycle will be smaller). >>>>>> > >>>>>> > If we do this for the next cycle we will have a 6 week ‘dev’ period >>>>>> and then we >>>>>> > will have optimistically an average of 2 weeks of ‘releasing’ and 6 >>>>>> weeks ‘dev’ >>>>>> > cycles. >>>>>> > >>>>>> > On Thu, Mar 1, 2018 at 6:46 AM, Jean-Baptiste Onofré < >>>>>> [email protected]> wrote: >>>>>> >> About BEAM-3409, I did a review yesterday and it looks good to me. >>>>>> We are >>>>>> >> waiting for Thomas' feedback. >>>>>> >> >>>>>> >> Regards >>>>>> >> JB >>>>>> >> Le 1 mars 2018, à 09:38, Robert Bradshaw <[email protected]> a >>>>>> écrit: >>>>>> >>> >>>>>> >>> Looking at the burn-down list, we have 5 remaining issues. None >>>>>> of these >>>>>> >>> are blockers, but all look like they're really close (reviewed, >>>>>> review >>>>>> >>> comments were addressed, waiting for a final LGTM). Specifically: >>>>>> >>> >>>>>> >>> BEAM-3409 (teardown issues): Thomas Groh had some concerns, could >>>>>> you >>>>>> >>> verify they have all been addressed? >>>>>> >>> BEAM-3479 (DoFn classloader regression test): Kenn Knowles had >>>>>> minor >>>>>> >>> comments, looks like they were addressed, could you confirm? >>>>>> >>> BEAM-3735 (Missing gaming release archetypes): Lukasz Cwik had >>>>>> minor >>>>>> >>> comments, looks like they were addressed, could you confirm? >>>>>> >>> BEAM-3611 (KafkaIO.java splitting): Looks like this was resolved. >>>>>> >>> BEAM-3762 (unlimited JCE for Dataflow Worker): LGTM pending >>>>>> (currently >>>>>> >>> running) tests passing. >>>>>> >>> >>>>>> >>> Let's see how many of these we can get in by, say, noon PST >>>>>> tomorrow. >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> >>>>>> >>> On Wed, Feb 28, 2018 at 9:26 PM Robert Bradshaw < >>>>>> [email protected]> >>>>>> >>> wrote: >>>>>> >>>> >>>>>> >>>> I tend to fall into the "release early, release often" camp in >>>>>> general, >>>>>> >>>> but for this one I'm particularly anxious to get the faster >>>>>> Python direct >>>>>> >>>> runner out in the hands of TFT/TFX users (and in particular have >>>>>> an eye on >>>>>> >>>> https://www.tensorflow.org/dev-summit/ which I think can be a >>>>>> healthy source >>>>>> >>>> of Beam users). >>>>>> >>>> >>>>>> >>>> >>>>>> >>>> On Wed, Feb 28, 2018 at 7:01 PM Jean-Baptiste Onofré < >>>>>> [email protected]> >>>>>> >>>> wrote: >>>>>> >>>>> >>>>>> >>>>> Hi Gus, >>>>>> >>>>> >>>>>> >>>>> Thanks for the update, it makes sense. >>>>>> >>>>> >>>>>> >>>>> Regards >>>>>> >>>>> JB >>>>>> >>>>> >>>>>> >>>>> On 03/01/2018 02:59 AM, Konstantinos Katsiapis wrote: >>>>>> >>>>>> Hi Jean-Baptiste, >>>>>> >>>>>> >>>>>> >>>>>> I can speak from the perspective of tf.transform >>>>>> >>>>>> < https://github.com/tensorflow/transform> (TFT) in >>>>>> particular and TFX >>>>>> >>>>>> < https://research.google.com/pubs/pub46484.html> libs in >>>>>> general, in >>>>>> >>>>>> case it is >>>>>> >>>>>> useful. >>>>>> >>>>>> >>>>>> >>>>>> TFX distributed computation has 2 "large" dependencies, namely >>>>>> >>>>>> TensorFlow and >>>>>> >>>>>> Apache Beam, each on their own release schedule. >>>>>> >>>>>> As such, releasing of new TFX functionality often (but not >>>>>> always) >>>>>> >>>>>> depends on >>>>>> >>>>>> (and is blocked by) releases of *both* TensorFlow *and* Apache >>>>>> Beam. >>>>>> >>>>>> >>>>>> >>>>>> Synchronizing releases across such large projects and >>>>>> organizations is >>>>>> >>>>>> likely >>>>>> >>>>>> hard, so from our perspective having *frequent* releases of >>>>>> Tensorflow >>>>>> >>>>>> or Apache >>>>>> >>>>>> Beam (and better yet both) decreases the time for which we are >>>>>> blocked >>>>>> >>>>>> on >>>>>> >>>>>> releasing our features. >>>>>> >>>>>> >>>>>> >>>>>> In light of this, I would vote for more frequent releases in >>>>>> general, >>>>>> >>>>>> and for a >>>>>> >>>>>> Beam 2.4 release soon in particular (as TFT 0.6 depends on it). >>>>>> >>>>>> >>>>>> >>>>>> Thanks, >>>>>> >>>>>> Gus >>>>>> >>>>>> >>>>>> >>>>>> On Tue, Feb 27, 2018 at 10:29 PM, Jean-Baptiste Onofré < >>>>>> >>>>>> [email protected] >>>>>> >>>>>> <mailto: [email protected]>> wrote: >>>>>> >>>>>> >>>>>> >>>>>> By the way, if third party projects based on Beam (Google >>>>>> >>>>>> Dataflow, Talend >>>>>> >>>>>> DataStream, and others) need a release (including some >>>>>> features), >>>>>> >>>>>> it's better to >>>>>> >>>>>> clearly state this on the mailing list. >>>>>> >>>>>> >>>>>> >>>>>> At Apache Karaf, I have lot of projects based on it >>>>>> (OpenDaylight, >>>>>> >>>>>> OpenHAB, >>>>>> >>>>>> Websphere, ...). They just ask for the release schedule >>>>>> and they >>>>>> >>>>>> align with >>>>>> >>>>>> these release. As a best effort, I'm always trying to move >>>>>> fast >>>>>> >>>>>> when a release >>>>>> >>>>>> is asked. >>>>>> >>>>>> >>>>>> >>>>>> So, if 2.4.0 is required by third party, no problem to >>>>>> "ask for a >>>>>> >>>>>> release". >>>>>> >>>>>> >>>>>> >>>>>> Regards >>>>>> >>>>>> JB >>>>>> >>>>>> >>>>>> >>>>>> On 02/28/2018 04:17 AM, Reuven Lax wrote: >>>>>> >>>>>> > It's been six weeks since you proposed beam 2.3.0. so >>>>>> assuming >>>>>> >>>>>> the same time >>>>>> >>>>>> > scale for this release, that's 1.5 months between >>>>>> releases. >>>>>> >>>>>> Slightly faster than >>>>>> >>>>>> > 2 months, but not by much. >>>>>> >>>>>> > >>>>>> >>>>>> > I do seem to remember that the original goal for beam was >>>>>> >>>>>> monthly releases though. >>>>>> >>>>>> > >>>>>> >>>>>> > Reuven >>>>>> >>>>>> > >>>>>> >>>>>> > On Tue, Feb 27, 2018, 9:12 PM Jean-Baptiste Onofré < >>>>>> >>>>>> [email protected] <mailto: [email protected]> >>>>>> >>>>>> > <mailto: [email protected] <mailto: [email protected]>>> >>>>>> wrote: >>>>>> >>>>>> > >>>>>> >>>>>> > Hi Reuven, >>>>>> >>>>>> > >>>>>> >>>>>> > In a previous thread (about Beam project execution), >>>>>> I >>>>>> >>>>>> proposed a release every >>>>>> >>>>>> > two months (as a best effort), I will find the >>>>>> e-mail. >>>>>> >>>>>> > >>>>>> >>>>>> > Beam 2.3.0 has been released "officially" on >>>>>> February 16th, >>>>>> >>>>>> so two week ago >>>>>> >>>>>> > roughly. I would have expected 2.4.0 not before end >>>>>> of >>>>>> >>>>>> March. >>>>>> >>>>>> > >>>>>> >>>>>> > If we have issue we want to fix fast, then 2.3.1 is >>>>>> good. If >>>>>> >>>>>> it's a new release >>>>>> >>>>>> > in the pace, it's pretty fast and might "confuse" >>>>>> our users. >>>>>> >>>>>> > >>>>>> >>>>>> > That's why I'm curious ;) >>>>>> >>>>>> > >>>>>> >>>>>> > Regards >>>>>> >>>>>> > JB >>>>>> >>>>>> > >>>>>> >>>>>> > On 02/28/2018 03:50 AM, Reuven Lax wrote: >>>>>> >>>>>> > > Wasn't the original statement monthly releases? >>>>>> We've >>>>>> >>>>>> never realistically >>>>>> >>>>>> > > managed that, but Robert's proposed cut will be on >>>>>> a >>>>>> >>>>>> 6-week pace. >>>>>> >>>>>> > > >>>>>> >>>>>> > > On Tue, Feb 27, 2018, 8:48 PM Jean-Baptiste Onofré >>>>>> < >>>>>> >>>>>> [email protected] <mailto: [email protected]> >>>>>> >>>>>> > <mailto: [email protected] <mailto: [email protected]>> >>>>>> >>>>>> > > <mailto: [email protected] <mailto: [email protected]> >>>>>> >>>>>> <mailto: [email protected] >>>>>> >>>>>> <mailto: [email protected]>>>> wrote: >>>>>> >>>>>> > > >>>>>> >>>>>> > > Hi Robert, >>>>>> >>>>>> > > >>>>>> >>>>>> > > I'm just curious: it's pretty fast compared to >>>>>> the >>>>>> >>>>>> original plan of a >>>>>> >>>>>> > release >>>>>> >>>>>> > > every two months. What's the reason to cut >>>>>> 2.4.0 now >>>>>> >>>>>> instead of end of >>>>>> >>>>>> > March ? >>>>>> >>>>>> > > >>>>>> >>>>>> > > I will do the Jira triage and update today. >>>>>> >>>>>> > > >>>>>> >>>>>> > > Regards >>>>>> >>>>>> > > JB >>>>>> >>>>>> > > >>>>>> >>>>>> > > On 02/27/2018 09:21 PM, Robert Bradshaw wrote: >>>>>> >>>>>> > > > I'm planning on cutting the 2.4.0 release >>>>>> branch >>>>>> >>>>>> soon (tomorrow?). I >>>>>> >>>>>> > see 13 >>>>>> >>>>>> > > > open issues on JIRA [1], none of which are >>>>>> labeled >>>>>> >>>>>> as blockers. If there >>>>>> >>>>>> > > > are any that cannot be bumped to the next >>>>>> release, >>>>>> >>>>>> let me know soon. >>>>>> >>>>>> > > > >>>>>> >>>>>> > > > - Robert >>>>>> >>>>>> > > > >>>>>> >>>>>> > > > >>>>>> >>>>>> > > > [1] >>>>>> >>>>>> > > > >>>>>> >>>>>> > > >>>>>> >>>>>> > >>>>>> >>>>>> https://issues.apache.org/jira/browse/BEAM-3749?jql= >>>>>> project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20% >>>>>> 22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%202.4.0 >>>>>> >>>>>> < >>>>>> >>>>>> https://issues.apache.org/jira/browse/BEAM-3749?jql= >>>>>> project%20%3D%20BEAM%20AND%20status%20in%20(Open%2C%20% >>>>>> 22In%20Progress%22%2C%20Reopened)%20AND%20fixVersion%20%3D%202.4.0> >>>>>> >>>>>> > > > >>>>>> >>>>>> > > >>>>>> >>>>>> > > -- >>>>>> >>>>>> > > Jean-Baptiste Onofré >>>>>> >>>>>> > > [email protected] <mailto: >>>>>> [email protected]> >>>>>> >>>>>> <mailto: [email protected] >>>>>> >>>>>> <mailto: [email protected]>> >>>>>> >>>>>> > <mailto: [email protected] <mailto: >>>>>> [email protected]> >>>>>> >>>>>> <mailto: [email protected] <mailto: [email protected] >>>>>> >>> >>>>>> >>>>>> > > http://blog.nanthrax.net >>>>>> >>>>>> > > Talend - http://www.talend.com >>>>>> >>>>>> > > >>>>>> >>>>>> > >>>>>> >>>>>> > -- >>>>>> >>>>>> > Jean-Baptiste Onofré >>>>>> >>>>>> > [email protected] <mailto: [email protected]> >>>>>> >>>>>> <mailto: [email protected] <mailto: [email protected] >>>>>> >> >>>>>> >>>>>> > http://blog.nanthrax.net >>>>>> >>>>>> > Talend - http://www.talend.com >>>>>> >>>>>> > >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Jean-Baptiste Onofré >>>>>> >>>>>> [email protected] <mailto: [email protected]> >>>>>> >>>>>> http://blog.nanthrax.net >>>>>> >>>>>> Talend - http://www.talend.com >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> >>>>>> -- >>>>>> >>>>>> Gus Katsiapis | Software Engineer | [email protected] >>>>>> >>>>>> <mailto: [email protected]> | 650-918-7487 >>>>>> >>>>> >>>>>> >>>>> -- >>>>>> >>>>> Jean-Baptiste Onofré >>>>>> >>>>> [email protected] >>>>>> >>>>> http://blog.nanthrax.net >>>>>> >>>>> Talend - http://www.talend.com >>>>>> >>>>>> -- >>>>>> Jean-Baptiste Onofré >>>>>> [email protected] >>>>>> http://blog.nanthrax.net >>>>>> Talend - http://www.talend.com >>>>>> >>>>> >>>> >>>
