Hi Robert, Thanks a lot for the update and good look with the review.
Moritz > On 3 Oct 2019, at 11:45, Robert Kisteleki <[email protected]> wrote: > > > On 2019-10-03 08:16, Moritz Muller wrote: >> Hi, >> >> In our experiment we’re trying to assign certain probes to a ping >> measurement but until now we always get the error message "NO SUITABLE >> PROBES”. >> According to the documentation, this is a sign that a probe might not have >> enough resources. >> However, when I check on the measurement a few hours later I do see results, >> but its state is still "NO SUITABLE PROBES”. >> See https://atlas.ripe.net/measurements/23016956/#!probes >> >> Is that a common problem when selecting certain probes for measurements? >> >> Moritz >> > > Hello, > > Yesterday afternoon we had an operational problem within RIPE Atlas that > had consequences visible to users. I strongly suspect the above is a > side-effect of this. > > Due to a combination of two configuration errors and a spike in requests > from users, the core infrastructure received an unreasonably high amount > of measurement requests from an internal process related to IPmap. The > measurements scheduler and participant management subsystems struggled > to keep up with this load and eventually things started piling up. > > The issue started at approximately 12 UTC. We identified the root cause > about an hour later. Processes started to normalise late in the > afternoon, and processing the backlog finished sometime after midnight. > > We're working on a post-mortem, and reviewing the code and configuration > in order to prevent this error from happening again. > > Apologies for the inconvenience, > Robert >
signature.asc
Description: Message signed with OpenPGP
