Hi all, ARIN found the issue and fixed it: https://lists.arin.net/pipermail/arin-tech-discuss/2020-November/000870.html
In short the ARIN manifest no longer included the delegated certificates so they were ignored by RPKI validators. This would have led to announcements becoming not found, rather than invalid. Still, we hope this won't happen again. Kind regards, Tim > On 22 Nov 2020, at 05:44, Honghao Zeng via RPKI <[email protected]> > wrote: > > Hi all, > > ARIN has confirmed that the issue is on their side: > >> Begin forwarded message: >> >> From: Mark Kosters <[email protected]> >> Subject: [arin-tech-discuss] Issue for Delegated Users within ARIN's RPKI >> Repository >> Date: November 21, 2020 at 11:32:19 PM EST >> To: "[email protected]" <[email protected]> >> >> Hi >> >> It was reported to us late this evening (11/21) that there is an issue >> ARIN’s RPKI repository that affects organizations that use delegated mode. >> This issue does not affect RPKI users who use the hosted mode. We are in the >> process of identifying the cause and will have a fix out shortly. >> >> Regards, >> Mark >> _______________________________________________ >> arin-tech-discuss mailing list >> [email protected] >> https://lists.arin.net/mailman/listinfo/arin-tech-discuss > > It was some very nice timing; this happens right after we upgraded Krill to > 0.8.1 yesterday. > > Sorry for the noise. > > Regards, > Honghao Zeng > >> On Nov 21, 2020, at 9:23 PM, Honghao Zeng <[email protected]> wrote: >> >> Hi all, >> >> It appears that this issue applies to all delegated RPKI CA under ARIN: >> >> rsync://rpki.multacom.com/repo/MCOMCA/0/ >> rsync://rpki.multacom.com/repo/MCOMCA/5/ >> rsync://nostromo.heficed.net/repo/1123832/0/ >> rsync://rpki.multacom.com/repo/MCOMCA/2/ >> rsync://rpki.multacom.com/repo/MCOMCA/3/ >> rsync://rpki.multacom.com/repo/MCOMCA/4/ >> rsync://rpki.tools.westconnect.ca/repo/WestConnect-CA/0/ >> rsync://rpki.qs.nu/repo/qsnu/0/ >> rsync://sakuya.nat.moe/repo/NATOCA/0/ >> rsync://rpki.admin.freerangecloud.com/repo/FRC-CA/0/ >> >> None of the above is working right now. Cloudflare's RPKI statistics [1] >> also shows a huge dip (180) in the number of ROAs under ARIN on Nov 20, >> compares to a normal < 10 ROA removals per day. >> >> Regards, >> Honghao Zeng >> >> [1] https://rpki.cloudflare.com/?ohlcTa=ARIN&ohlcDate=18586 >> >>> On Nov 21, 2020, at 6:18 PM, Honghao Zeng via RPKI >>> <[email protected]> wrote: >>> >>> Hi all, >>> >>> We operate our own RPKI CA at `sakuya.nat.moe.' It has a child CA >>> `ca.nat.moe.' Both CAs are using Krill. We recently upgraded Krill to 0.8.1 >>> and noticed that `ca.nat.moe' stopped working for some RPKI validators. >>> >>> Quick debug shows that the entitlement [1] and manifest [2] looks fine. >>> However, Cloudflare and RIPE's RPKI validator appears to ignore the >>> `ca.nat.moe' repo. Our local rpki-client also refuses to load the repo and >>> reports no error. >>> >>> Any idea what can be causing this? Also, `jdr.nlnetlabs.nl' appears to be >>> down. >>> >>> Best regards, >>> Honghao Zeng >>> >>> [1] >>> http://console.rpki-client.org/sakuya.nat.moe/repo/NATOCA/0/0108398CA988382C2A509BFDB39E146A76CF9DE0.cer.html >>> [2] >>> http://console.rpki-client.org/ca.nat.moe/repo/NATOLAB/0/0108398CA988382C2A509BFDB39E146A76CF9DE0.mft.html >>> -- >>> RPKI mailing list >>> [email protected] >>> https://lists.nlnetlabs.nl/mailman/listinfo/rpki >> > > -- > RPKI mailing list > [email protected] > https://lists.nlnetlabs.nl/mailman/listinfo/rpki -- RPKI mailing list [email protected] https://lists.nlnetlabs.nl/mailman/listinfo/rpki
