Commented on the Jira. Seems like it happens somewhat rarely, but is sometimes resolved by a restart, and sometimes has to do with some version mismatch issues. I did not find anything like a real root cause, just trial-and-error fixes. I'm not certain what may have occurred around May 19 to this infrastructure. Hoping INFRA can help us sort it out.
Kenn On Thu, Jun 2, 2022 at 9:15 AM Alexey Romanenko <[email protected]> wrote: > Thanks Ahmet and Ryan for taking a look! > > I agree that referenced commits seems are not related, that us why I was > puzzled with this. > > On 2 Jun 2022, at 17:56, Ahmet Altay <[email protected]> wrote: > > > > On Thu, Jun 2, 2022 at 8:53 AM Ryan Thompson <[email protected]> > wrote: > >> I asked in the slack channel to restart jenkins. I'm looking through the >> past messages to see if there's someone there I can tag. >> > > Thank you. Unfortunately, I do not believe we have a single expert we can > tag. > > >> Am I right in understanding this service is managed by someone in apache >> and not us? >> > > It is mixed. ASF Infra works with a vendor and they run a hosted jenkins > instance. We control the configuration of the instance, and the worker > nodes. > > >> >> On Thu, Jun 2, 2022 at 11:47 AM Ahmet Altay <[email protected]> wrote: >> >>> /cc @Kenneth Knowles <[email protected]> >>> >>> On Thu, Jun 2, 2022 at 8:44 AM Ahmet Altay <[email protected]> wrote: >>> >>>> I do not have a great idea but googling about the error, similar errors >>>> were resolved by restarting jenkins. We could try that. We may need to ask >>>> infra. ( @Ryan Thompson <[email protected]> - could you please >>>> infra to restart jenkins?) >>>> >>>> I do not think this issue is related to any change in the source >>>> control. The commit referenced in the first failed job is an unrelated doc >>>> change. >>>> >>>> >>>> On Thu, Jun 2, 2022 at 7:57 AM Alexey Romanenko < >>>> [email protected]> wrote: >>>> >>>>> I created a jira for this (not sure if it’s P0, but P1 for sure): >>>>> https://issues.apache.org/jira/browse/BEAM-14548 >>>>> >>>>> Could someone, who has more knowledge than me in Beam >>>>> infrastructure/Jenkins, take a look, please? >>>>> >>>>> — >>>>> Alexey >>>>> >>>>> On 31 May 2022, at 17:08, Alexey Romanenko <[email protected]> >>>>> wrote: >>>>> >>>>> The first failed job is >>>>> https://ci-beam.apache.org/job/beam_SeedJob/9696/ >>>>> It fails with this error (that says not so much): >>>>> >>>>> Processing DSL script .test-infra/jenkins/job_00_seed.groovy >>>>> Processing DSL script >>>>> .test-infra/jenkins/job_CancelStaleDataflowJobs.groovy >>>>> Processing DSL script >>>>> .test-infra/jenkins/job_CleanUpPrebuiltSDKImages.groovy >>>>> Processing DSL script .test-infra/jenkins/job_Dependency_Check.groovy >>>>> ERROR: java.io.IOException: Failed to persist config.xml >>>>> >>>>> and I don’t see any recent changes for these files. >>>>> >>>>> On 31 May 2022, at 16:24, Alexey Romanenko <[email protected]> >>>>> wrote: >>>>> >>>>> Hi everyone, >>>>> >>>>> Jenkins job `beam_SeedJob` keeps failing starting from May 19th. The >>>>> last successful build [2] was 12 days ago. >>>>> Does anyone know the reasons of this? >>>>> >>>>> — >>>>> Alexey >>>>> >>>>> >>>>> [1] https://ci-beam.apache.org/job/beam_SeedJob/ >>>>> [2] https://ci-beam.apache.org/job/beam_SeedJob/lastSuccessfulBuild/ >>>>> >>>>> >>>>> >>>>> >
