Thanks for the pointer. I'm planning to join that meeting. On Tue, Nov 28, 2023 at 4:16 PM Etienne Chauchot <echauc...@apache.org> wrote:
> Hi all, > > FYI there is the ASF infra roundtable soon. One of the subjects for this > session is GitHub Actions. It could be worth passing by: > > December 6th, 2023 at 1700 UTC on the #Roundtablechannel on Slack. > > For information about theroundtables, and about how to join, > see:https://infra.apache.org/roundtable.html > <https://infra.apache.org/roundtable.html> > > Best > > Etienne > > Le 24/11/2023 à 14:16, Maximilian Michels a écrit : > > Thanks for reviving the efforts here Matthias! +1 for the transition > > to GitHub Actions. > > > > As for ASF Infra Jenkins, it works fine. Jenkins is extremely > > feature-rich. Not sure about the spare capacity though. I know that > > for Apache Beam, Google donated a bunch of servers to get additional > > build capacity. > > > > -Max > > > > > > On Thu, Nov 23, 2023 at 10:30 AM Matthias Pohl > > <matthias.p...@aiven.io.invalid> wrote: > >> Btw. even though we've been focusing on GitHub Actions with this FLIP, > I'm > >> curious whether somebody has experience with Apache Infra's Jenkins > >> deployment. The discussion I found about Jenkins [1] is quite out-dated > >> (2014). I haven't worked with it myself but could imagine that there are > >> some features provided through plugins which are missing in GitHub > Actions. > >> > >> [1]https://lists.apache.org/thread/vs81xdhn3q777r7x9k7wd4dyl9kvoqn4 > >> > >> On Tue, Nov 21, 2023 at 4:19 PM Matthias Pohl<matthias.p...@aiven.io> > >> wrote: > >> > >>> That's a valid point. I updated the FLIP accordingly: > >>> > >>>> Currently, the secrets (e.g. for S3 access tokens) are maintained by > >>>> certain PMC members with access to the corresponding configuration in > the > >>>> Azure CI project. This responsibility will be moved to Apache Infra. > They > >>>> are in charge of handling secrets in the Apache organization. As a > >>>> consequence, updating secrets is becoming a bit more complicated. > This can > >>>> be still considered an improvement from a legal standpoint because the > >>>> responsibility is transferred from an individual company (i.e. > Ververica > >>>> who's the maintainer of the Azure CI project) to the Apache > Foundation. > >>> > >>> On Tue, Nov 21, 2023 at 3:37 PM Martijn Visser< > martijnvis...@apache.org> > >>> wrote: > >>> > >>>> Hi Matthias, > >>>> > >>>> Thanks for the write-up and for the efforts on this. I really hope > >>>> that we can move away from Azure towards GHA for a better integration > >>>> as well (directly seeing if a PR can be merged due to CI passing for > >>>> example). > >>>> > >>>> The one thing I'm missing in the FLIP is how we would setup the > >>>> secrets for the nightly runs (for the S3 tests, potential tests with > >>>> external services etc). My guess is we need to provide the secret to > >>>> ASF Infra and then we would be able to refer to them in a pipeline? > >>>> > >>>> Best regards, > >>>> > >>>> Martijn > >>>> > >>>> On Tue, Nov 21, 2023 at 3:05 PM Matthias Pohl > >>>> <matthias.p...@aiven.io.invalid> wrote: > >>>>> I realized that I mixed up FLIP IDs. FLIP-395 is already reserved > [1]. I > >>>>> switched to FLIP-396 [2] for the sake of consistency. 8) > >>>>> > >>>>> [1]https://lists.apache.org/thread/wjd3nbvg6nt93lb0sd52f0lzls6559tv > >>>>> [2] > >>>>> > >>>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-396%3A+Migration+to+GitHub+Actions > >>>>> On Tue, Nov 21, 2023 at 2:58 PM Matthias Pohl<matthias.p...@aiven.io > > > >>>>> wrote: > >>>>> > >>>>>> Hi everyone, > >>>>>> > >>>>>> The Flink community discussed migrating from Azure CI to GitHub > >>>> Actions > >>>>>> quite some time ago [1]. The efforts around that stalled due to > >>>> limitations > >>>>>> around self-hosted runner support from Apache Infra’s side. There > >>>> were some > >>>>>> recent developments on that topic. Apache Infra is experimenting > with > >>>>>> ephemeral runners now which might enable us to move ahead with > GitHub > >>>>>> Actions. > >>>>>> > >>>>>> The goal is to join the trial phase for ephemeral runners and > >>>> experiment > >>>>>> with our CI workflows in terms of stability and performance. At the > >>>> end we > >>>>>> can decide whether we want to abandon Azure CI and move to GitHub > >>>> Actions > >>>>>> or stick to the former one. > >>>>>> > >>>>>> Nico Weidner and Chesnay laid the groundwork on this topic in the > >>>> past. I > >>>>>> picked up the work they did and continued experimenting with it in > my > >>>> own > >>>>>> fork XComp/flink [2] the past few weeks. The workflows are in a > state > >>>> where > >>>>>> I think that we start moving the relevant code into Flink’s > >>>> repository. > >>>>>> Example runs for the basic workflow [3] and the extended (nightly) > >>>> workflow > >>>>>> [4] are provided. > >>>>>> > >>>>>> This will bring a few more changes to the Flink contributors. That > is > >>>> why > >>>>>> I wanted to bring this discussion to the mailing list first. I did a > >>>> write > >>>>>> up on (hopefully) all related topics in FLIP-395 [5]. > >>>>>> > >>>>>> I’m looking forward to your feedback. > >>>>>> > >>>>>> Matthias > >>>>>> > >>>>>> [1]https://lists.apache.org/thread/vcyx2nx0mhklqwm827vgykv8pc54gg3k > >>>>>> > >>>>>> [2]https://github.com/XComp/flink/actions > >>>>>> > >>>>>> [3]https://github.com/XComp/flink/actions/runs/6926309782 > >>>>>> > >>>>>> [4]https://github.com/XComp/flink/actions/runs/6927443941 > >>>>>> > >>>>>> [5] > >>>>>> > >>>> > https://cwiki.apache.org/confluence/display/FLINK/FLIP-395%3A+Migration+to+GitHub+Actions > >>>>>> > >>>>>> -- > >>>>>> > >>>>>> [image: Aiven]<https://www.aiven.io> > >>>>>> > >>>>>> *Matthias Pohl* > >>>>>> Opensource Software Engineer, *Aiven* > >>>>>> matthias.p...@aiven.io <i...@aiven.io> | +49 170 9869525 > >>>>>> aiven.io<https://www.aiven.io> | > >>>>>> <https://www.facebook.com/aivencloud> > >>>>>> <https://www.linkedin.com/company/aiven/> < > >>>> https://twitter.com/aiven_io> > >>>>>> *Aiven Deutschland GmbH* > >>>>>> Alexanderufer 3-7, 10117 Berlin > >>>>>> Geschäftsführer: Oskari Saarenmaa & Hannu Valtonen > >>>>>> Amtsgericht Charlottenburg, HRB 209739 B > >>>>>>
