I agree with what Lukasz Gajowy mentioned. I find that Jenkins is fine when your developing at HEAD but as soon as you cut a branch, the jenkins configuration starts to drift as it keeps getting updated to HEAD with the seed job. I was always thinking that the Jenkins configurations would be as thin as possible to invoke a single gradle target that would be in the repo. This would allow for less drift since most of the content would be coming from the repo itself.
On Fri, Sep 20, 2019 at 3:38 AM Łukasz Gajowy <lgaj...@apache.org> wrote: > And, yes, the fact that Jenkins jobs are separately evolving but pretty > tightly coupled to the repo contents is a serious problem that I wish we > had fixed. So verification of each PR was manual. > > Could you tell a little bit more about what exactly were the problems back > then? Was that due to incompatible docker images used in portability tests > maybe? Any other issues? > > My thoughts about Jenkins revolved around decoupling from Jenkins plugins, > Groovy DSL etc and replace as much as possible with more universal tools > (bash, Gradle). The main drivers were to: (1) be able to run the same thing > that runs on Jenkins using bash/Gradle (same scripts), (2) potentially be > able to replace Jenkins more easily with some more modern/better CI/CD tool > in the future (Github Actions/Gitlab or simply newer Jenkins with > Jenkinsfiles). I don't understand yet what was the problem cited above (I > didn't work on the LTS back then) so I'm not sure it would help with > releasing LTS versions with backports. > > Łukasz > > pt., 20 wrz 2019 o 02:01 Ahmet Altay <al...@google.com> napisał(a): > >> I agree with retiring 2.7 as the LTS family. Based on my experience with >> users 2.7 does not have a particularly high adoption and as pointed out has >> known critical issues. Declaring another LTS pending demand sounds >> reasonable but how are we going to gauge this demand? >> >> +Yifan Zou <yifan...@google.com> +Alan Myrvold <amyrv...@google.com> on >> the tooling question as well. Unless we address the tooling problem it >> seems difficult to feasibly maintain LTS versions over time. >> >> On Thu, Sep 19, 2019 at 3:45 PM Austin Bennett < >> whatwouldausti...@gmail.com> wrote: >> >>> To be clear, I was picking on - or reminding us of - the promise: I >>> don't have a strong personal need/desire (at least currently) for LTS to >>> exist. Though, worth ensuring we live up to what we keep on the website. >>> And, without an active LTS, probably something we should take off the site? >>> >>> >>> On Thu, Sep 19, 2019 at 1:33 PM Pablo Estrada <pabl...@google.com> >>> wrote: >>> >>>> +Łukasz Gajowy <lukasz.gaj...@polidea.com> had at some point thought >>>> of setting up jenkins jobs without coupling them to the state of the repo >>>> during the last Seed Job. It may be that that improvement can help test >>>> older LTS-type releases? >>>> >>>> On Thu, Sep 19, 2019 at 1:11 PM Robert Bradshaw <rober...@google.com> >>>> wrote: >>>> >>>>> In many ways the 2.7 LTS was trying to flesh out the process. I think >>>>> we learned some valuable lessons. It would have been good to push out >>>>> something (even if it didn't have everything we wanted) but that is >>>>> unlikely to be worth pursuing now (and 2.7 should probably be retired >>>>> as LTS and no longer recommended). >>>>> >>>>> I agree that it does not seem there is strong demand for an LTS at >>>>> this point. I would propose that we keep 2.16, etc. as potential >>>>> candidates, but only declare one as LTS pending demand. The question >>>>> of how to keep our tooling stable (or backwards/forwards compatible) >>>>> is a good one, especially as we move to drop Python 2.7 in 2020 (which >>>>> could itself be a driver for an LTS). >>>>> >>>>> On Thu, Sep 19, 2019 at 12:27 PM Kenneth Knowles <k...@apache.org> >>>>> wrote: >>>>> > >>>>> > Yes, I pretty much dropped 2.7.1 release process due to lack of >>>>> interest. >>>>> > >>>>> > There are known problems so that I cannot recommend anyone to use >>>>> 2.7.0, yet 2.7 it is the current LTS family. So my work on 2.7.1 was >>>>> philosophical. I did not like the fact that we had a designated LTS family >>>>> with no usable releases. >>>>> > >>>>> > But many backports were proposed to block 2.7.1 and took a very long >>>>> time to get contirbutors to implement the backports. I ended up doing many >>>>> of them just to move it along. This indicates a lack of interest to me. >>>>> The >>>>> problem is that we cannot really use a strict cut off date as a way to >>>>> ensure people do the important things and skip the unimportant things, >>>>> because we do know that the issues are critical. >>>>> > >>>>> > And, yes, the fact that Jenkins jobs are separately evolving but >>>>> pretty tightly coupled to the repo contents is a serious problem that I >>>>> wish we had fixed. So verification of each PR was manual. >>>>> > >>>>> > Altogether, I still think LTS is valuable to have as a promise to >>>>> users that we will backport critical fixes. I would like to keep that >>>>> promise and continue to try. Things that are rapidly changing (which >>>>> something always will be) just won't have fixes backported, and that seems >>>>> OK. >>>>> > >>>>> > Kenn >>>>> > >>>>> > On Thu, Sep 19, 2019 at 10:59 AM Maximilian Michels <m...@apache.org> >>>>> wrote: >>>>> >> >>>>> >> An LTS only makes sense if we end up patching the LTS, which so far >>>>> we >>>>> >> have never done. There has been work done in backporting fixes, see >>>>> >> https://github.com/apache/beam/commits/release-2.7.1 but the >>>>> effort was >>>>> >> never completed. The main reason I believe were complications with >>>>> >> running the evolved release scripts against old Beam versions. >>>>> >> >>>>> >> Now that the portability layer keeps maturing, it makes me >>>>> optimistic >>>>> >> that we might have a maintained LTS in the future. >>>>> >> >>>>> >> -Max >>>>> >> >>>>> >> On 19.09.19 08:40, Ismaël Mejía wrote: >>>>> >> > The fact that end users never asked AFAIK in the ML for an LTS >>>>> and for >>>>> >> > a subsequent minor release of the existing LTS shows IMO the low >>>>> >> > interest on having a LTS. >>>>> >> > >>>>> >> > We still are heavily iterating in many areas (portability/schema) >>>>> and >>>>> >> > I am not sure users (and in particular users of open source >>>>> runners) >>>>> >> > get a big benefit of relying on an old version. Maybe this is the >>>>> >> > moment to reconsider if having a LTS does even make sense given >>>>> (1) >>>>> >> > that our end user facing APIs are 'mostly' stable (even if many >>>>> still >>>>> >> > called @Experimental). (2) that users get mostly improvements on >>>>> >> > runners translation and newer APIs with a low cost just by >>>>> updating >>>>> >> > the version number, and (3) that in case of any regression in an >>>>> >> > intermediary release we still can do a minor release even if we >>>>> have >>>>> >> > not yet done so, let's not forget that the only thing we need to >>>>> do >>>>> >> > this is enough interest to do the release from the maintainers. >>>>> >> > >>>>> >> > >>>>> >> > On Tue, Sep 17, 2019 at 12:00 AM Valentyn Tymofieiev >>>>> >> > <valen...@google.com> wrote: >>>>> >> >> >>>>> >> >> I support nominating 2.16.0 as LTS release since in has robust >>>>> Python 3 support compared with prior releases, and also for reasons of >>>>> pending Python 2 deprecation. This has been discussed before [1]. As >>>>> Robert >>>>> pointed out in that thread, LTS nomination in Beam is currently >>>>> retroactive. If we keep the retroactive policy, the question is how long >>>>> we >>>>> should wait for a release to be considered "safe" for nomination. Looks >>>>> like in case of 2.7.0 we waited a month, see [2,3]. >>>>> >> >> >>>>> >> >> Thanks, >>>>> >> >> Valentyn >>>>> >> >> >>>>> >> >> [1] >>>>> https://lists.apache.org/thread.html/eba6caa58ea79a7ecbc8560d1c680a366b44c531d96ce5c699d41535@%3Cdev.beam.apache.org%3E >>>>> >> >> [2] https://beam.apache.org/blog/2018/10/03/beam-2.7.0.html >>>>> >> >> [3] >>>>> https://lists.apache.org/thread.html/896cbc9fef2e60f19b466d6b1e12ce1aeda49ce5065a0b1156233f01@%3Cdev.beam.apache.org%3E >>>>> >> >> >>>>> >> >> On Mon, Sep 16, 2019 at 2:46 PM Austin Bennett < >>>>> whatwouldausti...@gmail.com> wrote: >>>>> >> >>> >>>>> >> >>> Hi All, >>>>> >> >>> >>>>> >> >>> According to our policies page [1]: "There will be at least one >>>>> new LTS release in a 12 month period, and LTS releases are considered >>>>> deprecated after 12 months" >>>>> >> >>> >>>>> >> >>> The last LTS was released 2018-10-02 [2]. >>>>> >> >>> >>>>> >> >>> Does that mean the next release (2.16) should be the next LTS? >>>>> It looks like we are in danger of not living up to that promise. >>>>> >> >>> >>>>> >> >>> Cheers, >>>>> >> >>> Austin >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> >>>>> >> >>> [1] https://beam.apache.org/community/policies/ >>>>> >> >>> >>>>> >> >>> [2] https://beam.apache.org/get-started/downloads/ >>>>> >>>>
