Thanks, Matthias. Big +1 from me. On Tue, Nov 28, 2023 at 5:30 PM Matthias Pohl <matthias.p...@aiven.io.invalid> wrote:
> Thanks for the pointer. I'm planning to join that meeting. > > On Tue, Nov 28, 2023 at 4:16 PM Etienne Chauchot <echauc...@apache.org> > wrote: > > > Hi all, > > > > FYI there is the ASF infra roundtable soon. One of the subjects for this > > session is GitHub Actions. It could be worth passing by: > > > > December 6th, 2023 at 1700 UTC on the #Roundtablechannel on Slack. > > > > For information about theroundtables, and about how to join, > > see:https://infra.apache.org/roundtable.html > > <https://infra.apache.org/roundtable.html> > > > > Best > > > > Etienne > > > > Le 24/11/2023 à 14:16, Maximilian Michels a écrit : > > > Thanks for reviving the efforts here Matthias! +1 for the transition > > > to GitHub Actions. > > > > > > As for ASF Infra Jenkins, it works fine. Jenkins is extremely > > > feature-rich. Not sure about the spare capacity though. I know that > > > for Apache Beam, Google donated a bunch of servers to get additional > > > build capacity. > > > > > > -Max > > > > > > > > > On Thu, Nov 23, 2023 at 10:30 AM Matthias Pohl > > > <matthias.p...@aiven.io.invalid> wrote: > > >> Btw. even though we've been focusing on GitHub Actions with this FLIP, > > I'm > > >> curious whether somebody has experience with Apache Infra's Jenkins > > >> deployment. The discussion I found about Jenkins [1] is quite > out-dated > > >> (2014). I haven't worked with it myself but could imagine that there > are > > >> some features provided through plugins which are missing in GitHub > > Actions. > > >> > > >> [1]https://lists.apache.org/thread/vs81xdhn3q777r7x9k7wd4dyl9kvoqn4 > > >> > > >> On Tue, Nov 21, 2023 at 4:19 PM Matthias Pohl<matthias.p...@aiven.io> > > >> wrote: > > >> > > >>> That's a valid point. I updated the FLIP accordingly: > > >>> > > >>>> Currently, the secrets (e.g. for S3 access tokens) are maintained by > > >>>> certain PMC members with access to the corresponding configuration > in > > the > > >>>> Azure CI project. This responsibility will be moved to Apache Infra. > > They > > >>>> are in charge of handling secrets in the Apache organization. As a > > >>>> consequence, updating secrets is becoming a bit more complicated. > > This can > > >>>> be still considered an improvement from a legal standpoint because > the > > >>>> responsibility is transferred from an individual company (i.e. > > Ververica > > >>>> who's the maintainer of the Azure CI project) to the Apache > > Foundation. > > >>> > > >>> On Tue, Nov 21, 2023 at 3:37 PM Martijn Visser< > > martijnvis...@apache.org> > > >>> wrote: > > >>> > > >>>> Hi Matthias, > > >>>> > > >>>> Thanks for the write-up and for the efforts on this. I really hope > > >>>> that we can move away from Azure towards GHA for a better > integration > > >>>> as well (directly seeing if a PR can be merged due to CI passing for > > >>>> example). > > >>>> > > >>>> The one thing I'm missing in the FLIP is how we would setup the > > >>>> secrets for the nightly runs (for the S3 tests, potential tests with > > >>>> external services etc). My guess is we need to provide the secret to > > >>>> ASF Infra and then we would be able to refer to them in a pipeline? > > >>>> > > >>>> Best regards, > > >>>> > > >>>> Martijn > > >>>> > > >>>> On Tue, Nov 21, 2023 at 3:05 PM Matthias Pohl > > >>>> <matthias.p...@aiven.io.invalid> wrote: > > >>>>> I realized that I mixed up FLIP IDs. FLIP-395 is already reserved > > [1]. I > > >>>>> switched to FLIP-396 [2] for the sake of consistency. 8) > > >>>>> > > >>>>> [1] > https://lists.apache.org/thread/wjd3nbvg6nt93lb0sd52f0lzls6559tv > > >>>>> [2] > > >>>>> > > >>>> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Migration+to+GitHub+Actions > > >>>>> On Tue, Nov 21, 2023 at 2:58 PM Matthias Pohl< > matthias.p...@aiven.io > > > > > >>>>> wrote: > > >>>>> > > >>>>>> Hi everyone, > > >>>>>> > > >>>>>> The Flink community discussed migrating from Azure CI to GitHub > > >>>> Actions > > >>>>>> quite some time ago [1]. The efforts around that stalled due to > > >>>> limitations > > >>>>>> around self-hosted runner support from Apache Infra’s side. There > > >>>> were some > > >>>>>> recent developments on that topic. Apache Infra is experimenting > > with > > >>>>>> ephemeral runners now which might enable us to move ahead with > > GitHub > > >>>>>> Actions. > > >>>>>> > > >>>>>> The goal is to join the trial phase for ephemeral runners and > > >>>> experiment > > >>>>>> with our CI workflows in terms of stability and performance. At > the > > >>>> end we > > >>>>>> can decide whether we want to abandon Azure CI and move to GitHub > > >>>> Actions > > >>>>>> or stick to the former one. > > >>>>>> > > >>>>>> Nico Weidner and Chesnay laid the groundwork on this topic in the > > >>>> past. I > > >>>>>> picked up the work they did and continued experimenting with it in > > my > > >>>> own > > >>>>>> fork XComp/flink [2] the past few weeks. The workflows are in a > > state > > >>>> where > > >>>>>> I think that we start moving the relevant code into Flink’s > > >>>> repository. > > >>>>>> Example runs for the basic workflow [3] and the extended (nightly) > > >>>> workflow > > >>>>>> [4] are provided. > > >>>>>> > > >>>>>> This will bring a few more changes to the Flink contributors. That > > is > > >>>> why > > >>>>>> I wanted to bring this discussion to the mailing list first. I > did a > > >>>> write > > >>>>>> up on (hopefully) all related topics in FLIP-395 [5]. > > >>>>>> > > >>>>>> I’m looking forward to your feedback. > > >>>>>> > > >>>>>> Matthias > > >>>>>> > > >>>>>> [1] > https://lists.apache.org/thread/vcyx2nx0mhklqwm827vgykv8pc54gg3k > > >>>>>> > > >>>>>> [2]https://github.com/XComp/flink/actions > > >>>>>> > > >>>>>> [3]https://github.com/XComp/flink/actions/runs/6926309782 > > >>>>>> > > >>>>>> [4]https://github.com/XComp/flink/actions/runs/6927443941 > > >>>>>> > > >>>>>> [5] > > >>>>>> > > >>>> > > > https://cwiki.apache.org/confluence/display/FLINK/FLIP-395%3A+Migration+to+GitHub+Actions > > >>>>>> > > >>>>>> -- > > >>>>>> > > >>>>>> [image: Aiven]<https://www.aiven.io> > > >>>>>> > > >>>>>> *Matthias Pohl* > > >>>>>> Opensource Software Engineer, *Aiven* > > >>>>>> matthias.p...@aiven.io <i...@aiven.io> | +49 170 9869525 > > >>>>>> aiven.io<https://www.aiven.io> | > > >>>>>> <https://www.facebook.com/aivencloud> > > >>>>>> <https://www.linkedin.com/company/aiven/> < > > >>>> https://twitter.com/aiven_io> > > >>>>>> *Aiven Deutschland GmbH* > > >>>>>> Alexanderufer 3-7, 10117 Berlin > > >>>>>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > > >>>>>> Amtsgericht Charlottenburg, HRB 209739 B > > >>>>>> >
