[atlas]Re: API Not Working

[Robert Kisteleki]

Hi,

I'm not aware of issues recently, data is generally flowing as
expected. It's hard to judge why you may have problems without knowing
the details, but from your description it could be that the probes you
selected were unavailable, or it took them longer than what you
expected to do the measurement.

Please open a ticket with us (using at...@ripe.net) if you'd like
further investigation.

Regards,
Robert

On Sat, Jul 20, 2024 at 10:23 AM MARC BOSCH MARTINEZ
<100408...@alumnos.uc3m.es> wrote:
>
> Hi all!
>
> Is anyone having problems issuing SSL measurements from the API? Seems that 
> issuing them via the website works fine, but I've been having issues lately 
> with the ones that I created using the API, leading to measurements in which 
> all or most probes state "No Report Available".
>
> Best,
> Marc
> -
> To unsubscribe from this mailing list or change your subscription options, 
> please visit: https://mailman.ripe.net/mailman3/lists/ripe-atlas.ripe.net/
> As we have migrated to Mailman 3, you will need to create an account with the 
> email matching your subscription before you can change your settings.
> More details at: https://www.ripe.net/membership/mail/mailman-3-migration/
-
To unsubscribe from this mailing list or change your subscription options, 
please visit: https://mailman.ripe.net/mailman3/lists/ripe-atlas.ripe.net/
As we have migrated to Mailman 3, you will need to create an account with the 
email matching your subscription before you can change your settings. 
More details at: https://www.ripe.net/membership/mail/mailman-3-migration/

[atlas]Re: results page empty

[Robert Kisteleki]

Hello,

The page itself works (tried Firefox and Chrome) - I'm not sure why it
would not work in Edge or Pale Moon - maybe some extensions/blockers
are in the way?

Regards,
Robert

On Tue, Jul 16, 2024 at 8:16 AM Marco Moock  wrote:
>
> Hello!
>
> https://atlas.ripe.net/measurements/75882685/results
>
> If I try to access the results, the page is empty. I tried Pale Moon
> and MS Edge. In Tor Browser, I can access it.
>
> Is something faulty at my side or is that a general problem?
>
> --
> kind regards
> Marco
> -
> To unsubscribe from this mailing list or change your subscription options, 
> please visit: https://mailman.ripe.net/mailman3/lists/ripe-atlas.ripe.net/
> As we have migrated to Mailman 3, you will need to create an account with the 
> email matching your subscription before you can change your settings.
> More details at: https://www.ripe.net/membership/mail/mailman-3-migration/
-
To unsubscribe from this mailing list or change your subscription options, 
please visit: https://mailman.ripe.net/mailman3/lists/ripe-atlas.ripe.net/
As we have migrated to Mailman 3, you will need to create an account with the 
email matching your subscription before you can change your settings. 
More details at: https://www.ripe.net/membership/mail/mailman-3-migration/

[atlas]Re: Can not target google.com with periodic measurement

[Robert Kisteleki]

Hi,

I understand your point and recognise this is inconvenient at the
moment. I'd say in the short term we can increase the limit - 25 is
just a number like any other :-) - and perhaps administratively close
the failing measurements. While I can't make promises about when, but
in the longer term we can address this by looking at the the total
results towards a target per time interval, instead of total
measurements.

Cheers,
Robert

On Thu, Jul 11, 2024 at 11:19 AM Malte Tashiro via ripe-atlas
 wrote:
>
> Hi Robert,
>
> thanks for the clarification, now I know it is not a bug.
>
> With that said, I think this necessitates some form of house-keeping task 
> seeing how long Atlas has been active.
>
> I looked at the 25 running measurements (attached), assuming [0] is the right 
> request:
>
> - 23 ping measurements, 1 traceroute, 1 SSL
> - 7 are dead (-> /latest endpoint returns no results)
> - Average participant count of 5 probes
> - None have a scheduled stop time
> - The last one was started on 2023-01-11, so it was not possible to schedule 
> measurements targeting google.com since then?
>
> I know the Atlas developers always have a lot on their plate, so I won't ask 
> for anything, but maybe it would make sense to have a process that notifies 
> users of long-running / infinite measurements that did not return results 
> since x amount of time due to probes getting disconnected if they want to add 
> new probes or cancel the measurement. And if they don't reply for y time, 
> force cancel the measurement (or exclude them from the limit).
>
> Anyways, at least now it is clear to me what is happening, so thank you.
>
> Best,
> Malte
>
> [0] https://atlas.ripe.net/api/v2/measurements/?status=2=google.com
> --
> ripe-atlas mailing list -- ripe-atlas@ripe.net
> To unsubscribe send an email to ripe-atlas-le...@ripe.net
-- 
ripe-atlas mailing list -- ripe-atlas@ripe.net
To unsubscribe send an email to ripe-atlas-le...@ripe.net

[atlas]Re: Can not target google.com with periodic measurement

[Robert Kisteleki]

Hi,

Indeed, this is a global maximum against each target. We have two
major reasons for this:
* some targets can stand "any amount" of measuring, but most can't /
shouldn't have to
* once there are $thismany measurements against a particular target,
there's a good chance the new ones are redundant and results from
existing ones can be reused. Granted, this is not *always* the case.

Technically this is just a configuration variable, we can change it.
We can also add IPs / prefixes on a "bring it on!" list (allowlist).
We also have a feature request to put entire ASNs on this list. In the
meantime, I'd encourage you to look at the currently running
measurements, to see if you can use their data.

Regards,
Robert

On Wed, Jul 10, 2024 at 12:45 PM Ernst J. Oud  wrote:
>
> Same is true for 9.9.9.9 and others, too many measurements already running 
> against that target. I reckon RIPE does not want to be accused of 
> spamming/ddos so there is a maximum.
>
> Regards,
>
> Ernst J. Oud
>
> > On 10 Jul 2024, at 11:55, Malte Tashiro via ripe-atlas 
> >  wrote:
> >
> > On 7/10/24 18:09, Mark Santcroos wrote:
> >> Educated guess: this is a limit for a total over all users?
> >
> > I am aware of the quotas described here [0], which describe the 25 
> > measurement limit, but I understood them to be per user. If not, somebody 
> > could easily prevent others from performing measurements to a destination 
> > by scheduling 25 period measurements with one probe and an interval of a 
> > week or so.
> >
> > Btw. I had also two other users confirm to me that they have the same 
> > problem.
> >
> > [0] 
> > https://atlas.ripe.net/docs/getting-started/user-defined-measurements.html#quotas
> > 
> > --
> > ripe-atlas mailing list -- ripe-atlas@ripe.net
> > To unsubscribe send an email to ripe-atlas-le...@ripe.net
> --
> ripe-atlas mailing list -- ripe-atlas@ripe.net
> To unsubscribe send an email to ripe-atlas-le...@ripe.net
-- 
ripe-atlas mailing list -- ripe-atlas@ripe.net
To unsubscribe send an email to ripe-atlas-le...@ripe.net

Re: [atlas] Bug reports about Atlas

[Robert Kisteleki]

Hi Stephane,

Our 1st line support handles most of the issues/reports we receive,
and this is really helpful. Mistakes and misunderstandings can happen
of course. Please let me/us know what ticket of yours you have the
comment about, and we'll look at how we can improve our responses.

Cheers,
Robert

On Thu, Jul 4, 2024 at 5:18 PM Stephane Bortzmeyer  wrote:
>
> It seems that writing to at...@ripe.net does not go to the Atlas team,
> but to a generic RIPE-NCC helpdesk, which knows noting about Atlas and
> sends back funny, but unrelated answers. What is the preferred way to
> report bugs about Atlas?
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] [HTTP measurements] Units of timing?

[Robert Kisteleki]

Hello again,

According to the documentation
(https://atlas.ripe.net/docs/apis/result-format/#version-5000) those
fields are in milliseconds. I believe this to be correct; your
referenced measurement uses resolve-on-probe: false, therefore the DNS
resolution is done at specification time and it's a no-op on the
probe.

Indeed

On Tue, Jun 25, 2024 at 3:41 PM Stephane Bortzmeyer  wrote:
>
> When using "extended_timing" with HTTP measurements, in what units are
> tt*?
>
> I suspect that ttr is in seconds and the other two in milli-seconds but
> I want to be sure.
>
> (The Web interface does not display these values, it seems, but see
> measurement #74293060 if you want an example.)
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] [HTTP measurements] What is the use of query_string?

[Robert Kisteleki]

Hello,

On Tue, Jun 25, 2024 at 2:04 PM Stephane Bortzmeyer  wrote:
>
> On Tue, Jun 25, 2024 at 11:32:50AM +0200,
>  Stephane Bortzmeyer  wrote
>  a message of 10 lines which said:
>
> > The documentation says that you can add a "query_string" parameter in
> > a HTTP measurement but the anchor seems to ignore it?

The anchors' HTTP service is described here:
https://atlas.ripe.net/docs/howtos/anchors.html#ripe-atlas-anchor-services
- basically agreeing with you, it ignores query parameters.

> A colleague suggests that it can be useful to disable caching if the
> poor probe is locked behing a transparent HTTP proxy with caching.
>
> {'is_oneoff': True, 'definitions': [{'description': 'HTTP GET to 
> fr-par-as2486.anchors.atlas.ripe.net', 'af': 6, 'type': 'http', 'target': 
> 'fr-par-as2486.anchors.atlas.ripe.net', 'method': 'GET', 'path': '/', 
> 'query_string': 'disablecaching=123456789', 'https': False, 
> 'more_extended_timing': True}], 'probes': [{'requested': 1, 'type': 'area', 
> 'value': 'WW', 'tags': {'include': ['system-ipv6-works']}}]}

It is hard to judge a particular HTTP proxy's behaviour, but indeed
adding some variance to the URL via the query string likely helps
here.

Regards,
Robert

> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] multiple probes down since Jun 24th?

[Robert Kisteleki]

Hello,

Indeed we are experiencing some trouble with one of our controllers.
This can affect probes and anchors, seemingly up to 10% of the probe
population. The team is investigating the issue.

Regards,
Robert

On Tue, Jun 25, 2024 at 8:36 AM Andreas S. Kerber via ripe-atlas
 wrote:
>
> Hi,
>
> our probe is reported down since yesterday: 
> https://atlas.ripe.net/probes/7054/
>
> Since I couldn't find a problem on our side and rebooting didn't change the 
> status, I checked other probes (not mine) and they are reported down since 
> the exact downtime as mine. Is there any kind of known problem at the moment?
>
>
> Other probes with the same downtime:
>
> https://atlas.ripe.net/probes/7040
> https://atlas.ripe.net/probes/7052
> https://atlas.ripe.net/probes/7063
> https://atlas.ripe.net/probes/7067
> https://atlas.ripe.net/probes/7068
>
> --
> Andreas Kerber (Senior IT-Engineer/Consultant)
>
> a...@idkom.de
> ___
> IDKOM Networks GmbH
> Dieselstraße 1 * D-87437 Kempten (Allgäu)
>
> Tel.: 0831 59090-200
> Fax: 0831 59090-90
> www.idkom.de
>
> Registergericht: Amtsgericht Kempten (HRB 6359)
> Geschäftsführer: Benjamin Mayer * Klaus Giehl
>
> Diese E-Mail und alle mitgesendeten Dateien sind vertraulich und 
> ausschließlich für den Gebrauch durch den Empfänger bestimmt!
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] repetitive 2fa authentication on every login

[Robert Kisteleki]

Hello,

Thank you for this note, we'll take a look at the possible options.

Regards,
Robert

On Wed, Jun 12, 2024 at 3:50 PM  wrote:
>
> Hi team,
> https://atlas.ripe.net/probes/mine requires new login upon every browser 
> open. It even requires 2FA every time. Is it possible to keep the session 
> active for maybe a month, or at least do not ask 2FA every single time?  I 
> would think as services like gmail.com, portal.azure.com, 
> account.synology.com, etc., do not require 2fa on every browser close and 
> reopen, it may not be as crucial to perform 2fa every time I need to check if 
> my probes are online? It becomes a bit annoying, especially me using mobile 
> browser.
> Please consider.
>
> Thank you.
>
> Jiri
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas infrastructure maintenance on 11 June 2024

[Robert Kisteleki]

Dear RIPE Atlas users,

On 11 June 2024, between 8:30-11:30 (CEST) we will carry out
maintenance of the central RIPE Atlas infrastructure. The UI, APIs,
access to stored results and metadata, as well as measurement
scheduling will be unavailable. During this time results will still be
still collected and stored.

We apologise for the inconvenience this may cause.

Regards,
Robert Kisteleki, for the RIPE Atlas team

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] How long is a year?

[Robert Kisteleki]

Hello,

That calculation is simple: sum_of_connected_time / (now() -
first_connection_time)

Does this help answering your question?

Regards,
Robert

On Wed, Jun 5, 2024 at 9:28 PM Edward Lewis  wrote:
>
> On my probe’s status page, there is an “All Time” “Time Connected” of 8y 343d 
> 15h 30m.  How many minutes are in a “y”?
>
> Context: in trying to illustrate uptime (number of 9’s) I wanted to show how 
> long it would take to achieve different number of 9’s based on my probe’s 
> history.  (It’s only at one ‘9’ - 97%.)
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Summary of the discussions about previous proposals

[Robert Kisteleki]

Dear all,

In response to community feedback, as well as to the observed
behaviour of RIPE Atlas, earlier we put out some proposals about some
aspects of future behaviour of RIPE Atlas. We’d like to summarise what
we’ve heard so far (written responses as well as verbal discussions we
participated in), and the path we plan to take based on these.


Probe farming:
There were many voices who agreed with some form of limiting this
behaviour. We were also asked to reach out to users who are using many
probes this way now. We did and the short summary is that most (but
not all) answers give reasonable justification.
Conclusion: we’ll add the relatively simple proposed rules (in short:
limit the number of software probes from the same IP/prefix). Later on
we can consider softer approaches, e.g. reducing the number of credits
given to hosts with excess probes or using probe similarity metrics.

Measurement aggregators:
We proposed to embrace this existing phenomenon, ask these users to in
the future give the system an indication about the distinct clients
they serve, and perhaps step up as sponsors in such cases. We received
many responses to this proposal. For the most part, current
aggregators who responded stated that they have no significant issues
with providing such client information – in an anonymised way of
course. At the same time some of them stated that adding a “mandatory
sponsorship” would discourage them from using RIPE Atlas in the
future. Some argued that by having enough credits to do so, they were
in fact contributing positively to the network already, and in fact
some of them are or were sponsors.
Conclusion: we’ll add the features needed to implement such
aggregations, and work on incentives for these users to become RIPE
Atlas sponsors. We’ll also add the need for proper attribution to the
service, similar to what the commercial use policy describes. We may
also look at additional requirements to ensure proper funding for our
membership services.

Data retention principles:
This proposal was a high level one and we received limited feedback.
One discussion raised how “financially sustainable” is defined;
perhaps the best summary of this is the recent RIPE Labs article
published by our CTO Felipe:
https://labs.ripe.net/author/felipe_victolla_silveira/reducing-the-ripe-nccs-data-centre-footprint/
Conclusion: we are implementing our data storage and access features
along the principles we laid out.


For transparency, we’ll also include summaries of proposals we put out
earlier (see: 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/),
for which we haven’t provided a written conclusion.

Remove per-hop “late” packets from traceroute [LATE-PACKETS]: we only
heard support for this, therefore it’s in our plans now.

Measure well-known CDNs [CDN-HTTP]: Multiple responses argued that by
selecting a “short list” of CDNs that are included by default, there’s
an implicit statement and/or bias in the system towards bigger
providers, contributing to some aspects of centralisation. Others
pointed out that this approach reflects the reality of generic users,
since the majority of the content they consume is provided by said
CDNs. In addition, there were some voices supporting this proposal
because of the value it provides to (smaller) CDNs or because it
incentivises probe hosts to join or to keep their probes connected.
Based on these discussions we’ll not go ahead with a full set of CDN
measurements; however we’ll try a smaller scale experiment to discover
the potential benefits to all users.

Generic HTTP measurements [GENERIC-HTTP]: the discussions centered
around some reiterations of the previous arguments (i.e. the safety of
both the probe host and the providers’ sides) and if/how the proposed
opt-in activity of the probe host would help this. Given these
arguments, and that only a few voices expressed their explicit support
for this, we’ll put this activity on hold; we can revisit this if/when
the community wishes so.

Add support for STARTTLS measurements [STARTTLS]: there were lengthy
discussions about how certain SMTP servers may react to such incoming
measurements. The general agreement was that this measurement type can
be useful, with the confirmation that these checks will explicitly
stop after STARTTLS (i.e. they will not have the ability to even try
initiating sending mails or such).

Remove support for non-public measurements [ONLY-PUBLIC]: the
discussion raised both arguments for and against changing the system’s
behaviour. Some people also raised the possibility of making private
measurements more costly (in terms of credits). We’d also like to note
the reference to the data retention principles (private measurement
results are not available to all users, therefore we’ll only store
them for a limited time). All in all, for the time being we do not
plan to make behavioural changes about this feature.


Regards,
Robert Kisteleki
RIPE NCC

-- 
ripe-atlas mailing

Re: [atlas] Meaning and the future of the project

[Robert Kisteleki]

Dear Petr and others,

Thank you for your questions, and apologies for our delayed response. I’ve
provided some brief answers below, let me know if you have any further
questions.

Besides keeping up with the scalability aspects of growth of the network
(probes, anchors and in particular results collected by the system), and
the increasing volume of correlated support needs, here is what we’ve been
working on:

- Stability and availability of the collected results. The current focus
includes reducing costs by re-working our storage infrastructure.

- The underlying infrastructure components are being overhauled using cloud
technologies where possible to increase robustness and decrease costs where
applicable.

- The User Interface is undergoing major changes; you’ve probably seen the
effects on some major pages which are now fully API based and more will be
released soon.

- We’re re-working the probe firmware release process to make it easier for
developers to support the increasing number of architectures, and to make
users’ tasks easier when managing software probes.

In the short term, we’re focusing on stable operations. With the conclusion
of these items, which occupy a significant portion of our development
capacity, we plan to focus again on the most requested features. We’re also
considering the best future model for our hardware probe distribution and
will share more details on this when ready.

We publish our quarterly plans for RIPE Atlas here and you are invited to
provide input on these at any time:
https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas/

For 2024 we have budgeted EUR 1.35M for this activity and we have 7.9 FTEs
working on it. This information is always published in our Activity Plan
and Budget:
https://www.ripe.net/publications/docs/ripe-814/#2.2%20RIPE%20Atlas

We currently have around 800 weekly and 2500 monthly users of the system.
Aside from the intended use by network operators and researchers, the RIPE
NCC itself uses the system extensively for outage reports and
country/regional reports.

We are also actively seeking sponsorship for RIPE Atlas and more details
can be found here:
https://www.ripe.net/analyse/data-and-measurements-sponsorship/

Finally, I’d like to note that we will soon be starting our activity
planning for 2025. Any high level input on this list about desired
directions would be helpful for this process.

Regards,

Robert Kisteleki



On Tue, Feb 6, 2024 at 12:45 AM Petr Kutalek via ripe-atlas <
ripe-atlas@ripe.net> wrote:

> Hello,
>
> I have been involved in this project for over 12 years. For the last few
> weeks I have been wondering what the point of
> this is. OK, I have given out credits to students for research, though
> sometimes they do not even say thank you. The
> firmware on the probes has not changed for years, the front-end except for
> cosmetic changes also does not go through
> shocking developments (still the same bugs), new features are not
> implemented, just discussed (e-mail notifications) etc.
>
> So I would actually just like to ask:
> - How many people are working full time on this project at RIPE?
> - What is the priority for RIPE?
> - What is the future? Any realistic plans?
>
> Thank you,
>
> Petr
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas
>
>
-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Possible software probe "farming" on AS47583

[Robert Kisteleki]

Dear Ben,

In short: yes. There were a couple of responses to the approach/limits
proposed. We reached out to the users who are over those limits and have
gotten some replies with justifications. The implementation of said rules
is in in the plans now.

Regards,
Robert


On Thu, May 2, 2024 at 1:44 PM Ben Cartwright-Cox 
wrote:

> Did anything come of this? I just ran a measurement to spot check
> something and the same AS47583 probes make up a "hilarious" amount of
> results: https://atlas.ripe.net/measurements/70655999#probes
>
> On Tue, 20 Feb 2024 at 23:16, Ben Cartwright-Cox 
> wrote:
> >
> > Those limits seem reasonable enough,
> >
> > My own intuition would suggest values of:
> >
> > X=2
> > Y=3
> > Z=5
> >
> > But otherwise, I welcome such a change being implemented!
> >
> > On Tue, 20 Feb 2024 at 15:43, Robert Kisteleki  wrote:
> > >
> > >
> > > > Is there an immediate way to report these probes other than this
> > > > mailing list?  I don't know of one and so I'm here :)
> > >
> > >
> > > Dear Ben and others,
> > >
> > > Each new, connected RIPE Atlas probe provides incremental value to the
> > > system and its users, but this value decreases with similarity to
> > > existing probes ("diminishing returns"). At the same time connecting a
> > > probe and processing results from it has some costs, so we'd like to be
> > > conscious of the cost/benefit ratio.
> > >
> > > Since the potential pool of software probes is almost infinite, in
> > > response to the highlighted case, we'd like to propose the following
> > > mid-term approach:
> > >
> > > * No user/account should be allowed to run more than X SW probes from
> > > the same IP (X=3 ?)
> > >
> > > * No user/account should be allowed to run more than Y SW probes from
> > > the same IPv4/24 IPv6/48 (Y=5 ?)
> > >
> > > * Regardless of the user/account, no more than Z SW probes should be
> > > allowed from the same IPv4/24 IPv6/48 (Z=10 ?)
> > >
> > > X, Y and Z are defaults, can be changed per account. This is done in
> > > order to facilitate corner cases and overstepping the limits, if this
> is
> > > warranted (given a good explanation). We are also reaching out to the
> > > current "peak users" to understand their use cases and motivations -
> the
> > > above limits can be enforced depending on the responses.
> > >
> > > In the longer term we believe a more flexible approach is to base this
> > > on what has been termed "probe similarity metrics": a new probe that is
> > > really similar to some existing one has less value to the system,
> > > therefore after reaching a low enough limit it can be refused, or
> > > alternatively, simply not gaining credits for its owner. This
> > > diincentivises creating "probe farms".
> > >
> > > Regards,
> > >
>
-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Operational work on RIPE Atlas

[Robert Kisteleki]

Dear all,

A heads-up: we are just about to start scheduled maintenance work on RIPE Atlas.

During the maintenance we expect scheduling of new measurements will
be delayed and searching for measurements will not yield up-to-date
information.

Regards,
Robert

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Confused by the documentation

[Robert Kisteleki]

Hi,

You can find this in the documentation at:
https://atlas.ripe.net/docs/howtos/anchors.html#http-s

Cheers,
Robert

On Fri, Apr 5, 2024 at 4:33 PM Stephane Bortzmeyer  wrote:
>
> On Fri, Apr 05, 2024 at 10:13:14AM +0200,
>  Christopher Amin  wrote
>  a message of 44 lines which said:
>
> > As a workaround, you can look at the response documentation for "GET
> > /api/v2/measurements" -- all of the type-specific fields that are
> > marked with .e.g [http] or [dns] are options which can be included
> > when creating the measurement.
>
> OK, I did not look at the right place. Speaking of these options, what
> are the possible paths for a HTTP requests at an anchor? I assume the
> only supported one is "/"?
>
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Empty responses for HTTP tests with a non-existing path?

[Robert Kisteleki]

Dear Stephane,

Thank you for pointing this out, it has been fixed: the anchors
respond with a proper 404 now.

Regards,
Robert, reporting for the people who fixed it

On Thu, Apr 4, 2024 at 6:48 PM Stephane Bortzmeyer  wrote:
>
> It seems the anchors, when receiving a HTTP request for a non-existing
> path, instead of a 404, send a broken response:
>
> % curl -v http://fr-par-as2486.anchors.atlas.ripe.net/nonono
> *   Trying 2001:67c:217c:4::2:80...
> * Connected to fr-par-as2486.anchors.atlas.ripe.net (2001:67c:217c:4::2) port 
> 80 (#0)
> > GET /nonono HTTP/1.1
> > Host: fr-par-as2486.anchors.atlas.ripe.net
> > User-Agent: curl/7.81.0
> > Accept: */*
> >
> * Empty reply from server
> * Closing connection 0
> curl: (52) Empty reply from server
>
> --
> ripe-atlas mailing list
> ripe-atlas@ripe.net
> https://lists.ripe.net/mailman/listinfo/ripe-atlas

-- 
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Fwd: [ncc-announce] [News] 2FA Mandatory on RIPE NCC Access accounts

[Robert Kisteleki]

Dear RIPE Atlas users,

In case you're not following the official RIPE NCC communication 
channels: this is a heads up regarding 2FA use in RIPE Access, the 
single sign-on mechanism behind various RIPE NCC services including RIPE 
Atlas.


Action needed from you: please make sure you enable 2FA on your account.

Regards,
Robert Kisteleki
RIPE NCC


 Forwarded Message 
Subject: [ncc-announce] [News] 2FA Mandatory on RIPE NCC Access accounts
Date: Tue, 26 Mar 2024 12:12:12 +0100
From: Felipe Victolla Silveira 
To: ncc-annou...@ripe.net

Dear colleagues,

This week, two-factor authentication (2FA) will become mandatory on RIPE 
NCC Access accounts. If you have not already done so,  enable 2FA on 
your account.
For accounts that do not have two-factor authentication, when you try to 
log in from Wednesday 27 March, you will be prompted to check your inbox 
and follow the instructions in the email sent to you by the RIPE NCC. 
Once you have enabled 2FA, you can log in as you usually would.


We will continue to improve the security and usability of RIPE NCC 
Access accounts over the next few months. Next week, I will also publish 
a RIPE Labs article explaining our roadmap and main improvements planned 
in more detail. We welcome your feedback and suggestions on which 
features are the most important for you on the RIPE NCC Services Working 
Group (ncc-services...@ripe.net) mailing list.


If you have any issues setting up 2FA on your account, please email 
n...@ripe.net.

Kind regards,

Felipe Victolla Silveira
Chief Technology Officer
--
Click here to unsubscribe or change your email settings: 
https://lists.ripe.net/mailman/options/ncc-announce



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Possible software probe "farming" on AS47583

[Robert Kisteleki]

Is there an immediate way to report these probes other than this
mailing list?  I don't know of one and so I'm here :)



Dear Ben and others,

Each new, connected RIPE Atlas probe provides incremental value to the 
system and its users, but this value decreases with similarity to 
existing probes ("diminishing returns"). At the same time connecting a 
probe and processing results from it has some costs, so we'd like to be 
conscious of the cost/benefit ratio.


Since the potential pool of software probes is almost infinite, in 
response to the highlighted case, we'd like to propose the following 
mid-term approach:


* No user/account should be allowed to run more than X SW probes from 
the same IP (X=3 ?)


* No user/account should be allowed to run more than Y SW probes from 
the same IPv4/24 IPv6/48 (Y=5 ?)


* Regardless of the user/account, no more than Z SW probes should be 
allowed from the same IPv4/24 IPv6/48 (Z=10 ?)


X, Y and Z are defaults, can be changed per account. This is done in 
order to facilitate corner cases and overstepping the limits, if this is 
warranted (given a good explanation). We are also reaching out to the 
current "peak users" to understand their use cases and motivations - the 
above limits can be enforced depending on the responses.


In the longer term we believe a more flexible approach is to base this 
on what has been termed "probe similarity metrics": a new probe that is 
really similar to some existing one has less value to the system, 
therefore after reaching a low enough limit it can be refused, or 
alternatively, simply not gaining credits for its owner. This 
diincentivises creating "probe farms".


Regards,


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Possible software probe "farming" on AS47583

[Robert Kisteleki]

Hello,

On 2024-02-05 16:11, Ben Cartwright-Cox via ripe-atlas wrote:

Hi everybody,

While doing some measurement work  earlier this weekend I discovered
that there are a large amount of probes on AS47583  that look like
they are being set up to farm credits. With a suspiciously high amount
of purely software probes on the same IP prefix and sequential IPv6
addresses.


...

Thank you for the report. We are looking at ways to limit this in a way 
that doesn't hurt benevolent users (e.g. ones coming from behind the 
same CGN) and that is not a whack-a-mole exercise.



Now while I myself don't particularly care about credit farming on
RIPE Atlas (After all RIPE Atlas credit is kind of like Monopoly
money), I do care about things becoming biased towards these software
probes.

For example, when I had selected for a selection of probes in Brazil I
ended up with several of these software probes giving me the exact
same results.  this is not very useful and this has the potential to
bias academic measurements,  had I not looked at the probe list
results this would not have been immediately obvious.


One crude short-term solution to this is to exclude software probes 
(system-sw). A longer term -- nicer -- one is the "probe similarity" 
idea, where you can (will be able to) ask for dissimilar probes. 
Unfortunately that's not something we have yet...



Is there an immediate way to report these probes other than this
mailing list?  I don't know of one and so I'm here :)


You can always send mail to at...@ripe.net - which is the general 
support address for RIPE Atlas.


Regards,
Robert


Cheers
Ben



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Retiring old ("legacy") UI pages and deprecated API calls

[Robert Kisteleki]

As part of the RIPE Atlas user interface renewal last year, we released 
a new measurement scheduling form and new probe and measurement lists. 
Since these changes the old pages were still accessible via changed 
URLs. By now the use of these pages all but disappeared, and we'll 
remove these from the UI in the coming weeks.


This work is now done.

Regards,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] 4 probes on different networks suddenly down?

[Robert Kisteleki]

Status update: we identified and fixed the underlying issue, and are 
recovering now. Normal status, including processing backlogs will likely 
take some time.


We'll post updates on the RIPE NCC status page: https://status.ripe.net/

Regards,
Robert


On 2024-01-30 14:44, Robert Kisteleki wrote:

Hello,

Quick note: we are experiencing issues with the controlling 
infrastructure, trying to determine the root cause. Please stand by.


Cheers,
Robert


On 2024-01-30 14:23, Ernst J. Oud wrote:

Hi,

At noon today 4 of my probes disconnected.

1005229
61071
55795
1005072

These are on different ISP’s and one of these is in Oracle’s cloud.
So this is not a problem on my side.

Restarted the 61071 since that one is nearby and easily restarted 
since it is a hardware probe, but that does not work. Flashing red LED.


What is going on?

Regards,

Ernst J. Oud




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] 4 probes on different networks suddenly down?

[Robert Kisteleki]

Hello,

Quick note: we are experiencing issues with the controlling 
infrastructure, trying to determine the root cause. Please stand by.


Cheers,
Robert


On 2024-01-30 14:23, Ernst J. Oud wrote:

Hi,

At noon today 4 of my probes disconnected.

1005229
61071
55795
1005072

These are on different ISP’s and one of these is in Oracle’s cloud.
So this is not a problem on my side.

Restarted the 61071 since that one is nearby and easily restarted since it is a 
hardware probe, but that does not work. Flashing red LED.

What is going on?

Regards,

Ernst J. Oud


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Retiring old ("legacy") UI pages and deprecated API calls

[Robert Kisteleki]

Dear all,

As part of the RIPE Atlas user interface renewal last year, we released 
a new measurement scheduling form and new probe and measurement lists. 
Since these changes the old pages were still accessible via changed 
URLs. By now the use of these pages all but disappeared, and we'll 
remove these from the UI in the coming weeks.


On a similar note, we'll also remove support for the API calls that took 
the form /api/v2/measurements// [*]. These have been deprecated 
for a long time, and since the last active user moved way most queries 
ending up here either come from the legacy pages or from hacking attempts.


[*] in fact this has effectively been an alias for 
/api/v2/measurements/?type= so no functionality is lost and it is 
trivial to change code that otherwise used these calls


Please reach out to us if you believe the legacy pages provided some 
functionality you think you'll miss so we can help you.


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] built-in measurement id 12023

[Robert Kisteleki]

Dear Xiao,

You are absolutely right - the documentation (and the API metadata) 
incorrectly states 900 seconds, in reality it is once a day. Also for 
measurement 13023 over IPv6.


(It would be possible to adjust reality to match the docs instead and 
adjust this measurement's frequency to 900 seconds, but considering the 
meaning of the measurement and that it has been running with said 
frequency all along, I'd prefer to fix the documentation.)


Thank you for reporting this, we'll fix it shortly.

Regards,
Robert


On 2024-01-23 02:06, Xiao Song wrote:

Hi all,

Hope this email finds you well.

The document says that this built-in measurement #12023 requires all 
available probes to send http requests to the target every 900s, but 
actually it seems like all probes send requests once a day.
I was wondering if there exists inconsistency between the document and 
the actual measurements?


Any suggestions would be appreciated.

Thank you!

Best,
Xiao



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Fwd: Your credits are decreasing dramatically!

[Robert Kisteleki]

Hi Lukas,

The intention of this mail is to warn you in case the system has reasons 
to believe that you'll run out of credits "soon". A burst of one-off 
measurements changes the credit consumption rate temporarily, so it kind 
of makes sense that you see correlation between when you get these mails 
and when you schedule new measurements.


However, I cannot claim the calculation behind it is flawless; based on 
your report we'll do a sanity check of what it did in your cases.


Thank you for flagging this!

Regards,
Robert


On 2024-01-11 20:55, Lukas Tribus wrote:

Hello,


the "Your credits are decreasing dramatically!" emails have always
been a mystery to me. I always get them a few days after a
"troubleshooting session" with a number of one-off measurements.


Here's the latest one; on January 6th, between 14:00 and 16:00 UTC I
made a number of one-off Atlas measurements. The youngest (latest)
Requested Stop Time which matches Systems Stop Time is:
2024-01-06T14:35:02Z

I did not request any other measurements, and I have no "continuous"
measurements running currently.



And yet, on January 10th 00:48 UTC I get an warning:


Our records show that you've consumed 40500 RIPE Atlas credits in the last 48 
hours, or approximately 14.06 credits/minute.




On January 11th, 01:56 UTC I get the same exact warning again:


Our records show that you've consumed 40500 RIPE Atlas credits in the last 48 
hours, or approximately 14.06 credits/minute.




The data in those emails is just wrong, I did not use any credits for
at least 72 hours (first email) and at least 96 hours (second email).

It looks like already stopped one-off measurements are used for
predictions not only of the future but for the past 48 hours as well?

This is not a first, I got many of those erroneous messages in the past.




Thanks,

Lukas


-- Forwarded message -
From: RIPE Atlas 
Date: Thu, 11 Jan 2024 at 02:56
Subject: Your credits are decreasing dramatically!
To: Lukas Tribus 



Dear RIPE Atlas probe host,

Our records show that you've consumed 40500 RIPE Atlas credits in the
last 48 hours, or approximately 14.06 credits/minute. We're pleased
that you find the user-defined measurements useful and encourage you
to keep conducting them!

However, we'd like to give you a heads up that you have 2d 16h 6m
before you will run out of credits at this rate. We recommend you stop
some of your current measurements before this happens. If you do run
out of credits, we will have to automatically stop running some of
your measurements so that you do not reach a negative balance.

If you would like to continue running a large number of user-defined
measurements, you might want to consider applying for another probe or
sponsoring a block of probes in order to earn more credits. Learn more
about sponsoring probes at
https://atlas.ripe.net/get-involved/become-a-sponsor/

Thank you for participating in RIPE Atlas!

Kind regards,

RIPE Atlas Team



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] auto probe renewal

[Robert Kisteleki]

Hello,

Left unattended, over longer periods of time, measurements lose probes. 
The RIPE Atlas API includes a call that can be used to add or remove 
probes or groups of probes in an existing measurement, see:


https://atlas.ripe.net/docs/apis/rest-api-manual/participation_requests.html#creating-a-participation-change-request

We started working on a feature where you'll be able to ask the system 
itself to remove disconnected probes, and to "top it up" with other 
probes (similar to the original ones. I cannot tell when this will be 
available though.


Regards,
Robert


On 2023-12-11 17:44, POULET Benoit wrote:

Hello,

When I start a never-ending ping measurement with several probes like 20.

What happened if several probes are going off-line ?

Will Atlas remove the probes who are dead, and assign new to the 
measurement ? Or will the measurement die slowly ?


I tried to find the answer in the documentation, without success.

Thanks.




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] The role of aggregators in RIPE Atlas

[Robert Kisteleki]

Dear all,

We've just published a proposal about recognising the role of 
"aggregators" in RIPE Atlas: 
https://labs.ripe.net/author/kistel/the-role-of-aggregators-in-ripe-atlas/


We would be very happy to see discussions about this here on the mailing 
list, on the RIPE NCC Forum, or live at RIPE87.


Regards,
Robert Kisteleki
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE NCC measurement data retention

[Robert Kisteleki]

Dear all,

We've just published a proposal about establishing principles around how 
the RIPE NCC retains and publishes Internet measurement data, 
specifically in RIS and RIPE Atlas: 
https://labs.ripe.net/author/kistel/ripe-ncc-measurement-data-retention-principles/


We would be very happy to see discussions about this here on the mailing 
list, on the RIPE NCC Forum, or live at RIPE87.


Regards,
Robert Kisteleki
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] decommission of virtual probes

[Robert Kisteleki]

Hello,

If you resurrect / reinstall / relocate a probe and you expect to have 
different network conditions in its new life, then it is better to 
install it as a new one. If however you are just changing the hardware 
or just need to replace it's key, then it's better to reuse the ID (the 
host can always re-register the same probe with a different key).


Your case of no longer using a VPS is probably the first of these two.

Every disconnected probe is automatically labeled as such, and 
eventually becomes "abandoned". You can signal to the world if a 
particular probe is not expected to come back at all by "writing it 
off". We'll add this as a self-service feature in the UI, in the 
meantime you can just let us know (at at...@ripe.net) and we'll mark it 
for you.


Cheers,
Robert


On 2023-11-06 20:53, Simon Brandt via ripe-atlas wrote:

Hello folks, dear Atlas team,

I would like to discuss the case, where a virtual probe is 
decommissioned because the place where you ran it will no longer be 
available. For example, a VPS which you have rented somewhere, but 
you've decided to cancel that service.


Does it make sense to create and keep a backup of the virtual probe, so 
that you can re-deploy it later (with another AS)? Or is it better to 
delete the virtual probe and create a new one?


Also: is there an official way to decommission a virtual probe permanently?

I felt like this could be interesting for other people too, so i decided 
to discuss this question publicly.


BR,
Simon




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] HE "Supertraceroute" exposes ATLAS traceroute functionality to the public

[Robert Kisteleki]

Hi,

Yes, we are aware and we're following up with them.

Regards,
Robert


On 2023-10-19 12:04, Lukas Tribus wrote:

Hello,

FYI HE's new "supertraceroute" exposes ATLAS traceroute functionality
to the public:

https://bgp.he.net/traceroute/


Other sources are (NLNOG) RING nodes and other projects.


Interesting approach, not sure if I should be happy about this, but I
guess HE is paying for it with Atlas credits.




Lukas



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] NTP empty results ('result': [{'x': '*'}])

[Robert Kisteleki]

Hi,

This means there probe didn't get a useful answer; there was a timeout, 
the connection failed or similar reasons.


Regards,
Robert


On 2023-10-10 12:41, Giovane C. M. Moura via ripe-atlas wrote:

Hi folks,

I am getting some NTP results in the following format:

   'result': [{'x': '*'}],

Does anyone know waht is this supposed to mean? is it like the 
measurement did not run?


See blob below:

{'fw': 5080,
   'mver': '2.6.2',
   'lts': 20,
   'dst_name': '186.155.28.147',
   'dst_addr': '186.155.28.147',
   'src_addr': '10.10.31.19',
   'proto': 'UDP',
   'af': 4,
   'result': [{'x': '*'}],
   'msm_id': 47867316,
   'prb_id': 33745,
   'timestamp': 1671799154,
   'msm_name': 'Ntp',
   'from': '85.238.41.3',
   'type': 'ntp',
   'group_id': 47867316,
   'stored_timestamp': 1671799249}]

thanks,

/giovane



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Changes to RIPE Atlas API keys

[Robert Kisteleki]

Dear all,

These changes have now been implemented.

Regards,
Robert


On 2023-09-19 12:11, Robert Kisteleki wrote:


Dear RIPE Atlas users,

We'd like to update you on some upcoming changes regarding API keys in 
RIPE Atlas.


TL;DR nothing changes regarding how you can use your API keys in the 
short term - as long as you're actually using them. However, we'll 
change how unused or forgotten keys are handled as well as remove the 
less secure in-URL use of them.



At the moment RIPE Atlas users can query their existing API keys via the 
UI and API, including the possibility to retrieve old keys. In order to 
improve the security of how we handle these, we'll introduce the 
following changes in October 2023:


* The listing (retrieval) of keys will only reveal parts of the keys 
(enough to identify them) in the API as well as in the UI.


* We'll add the ability to "regenerate" an API key, which will replace 
the secret UUID of the key while keeping exactly the same permissions.


* Unused API keys will automatically be frozen after 1 year of not being 
used. Active keys (i.e. the ones that have been used at least once) will 
not be frozen.


You still have the ability to save your keys until these changes are 
done and, as written above, you will be able to regenerate them later. 
We'll notify this list when the changes are about to be done.



In addition, in order to further increase the security of our system, in 
the long run we'll make changes about how these API keys are 
communicated to the API:


* At the moment the API accepts these either in HTTP headers 
("Authorization" header) or in the URL (?key=xyz), although the 
Authorization header version has been documented as the preferred 
version for some time.


* We'll deprecate and remove the ability to use the URL form in about a 
year (around October 2024).


* We plan to send further reminders about this change over time, as well 
as reaching out to heavy users of the to-be-removed format.


Regards,
Robert Kisteleki
RIPE Atlas team






--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q4 2023

[Robert Kisteleki]

Dear colleagues,


We have published our next quarterly plans for RIPE Atlas on our 
website: 
https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas


This quarter, we'll continue discussing the five proposals for changing 
the RIPE Atlas behaviour and working on renewing the big data backend.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
Principal Engineer
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Will probes catch up?

[Robert Kisteleki]

On 2023-09-21 12:07, Ernst J. Oud wrote:

Hi,

Some of my probes, all on different providers, (for instance: 
https://atlas.ripe.net/probes/1005104/#tab-builtins 
) have not catched 
up after the recent disruption.


Will this happen automatically? They are connected, streaming results 
are ok., so I guess I don’t need to restart them?


Regards,

Ernst J. Oud


Hello,

The situation is improving, but we haven't recovered yet. There is no 
reason to reboot your probe(s). Once the system processes the the 
results that is already buffered, the graphs should look fine as well.


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas fully down..

[Robert Kisteleki]

Hi,

On 2023-09-20 11:47, Ernst J. Oud wrote:

Robert et al,

In contrast to your statement below, streaming of results from new or existing 
measurements using Magellan currently does *not* work… no results are obtained, 
see below.

—-
Looking good! Measurement 60182213 was created and details about it can be 
found here:

   https://atlas.ripe.net/measurements/60182213/

Connecting to stream...

Disconnected from stream


The answer is the same here as for Stephane - no probes were selected 
for the measurement, hence no answers are coming back on the streaming 
interface either.


Regards,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas fully down..

[Robert Kisteleki]

Hi,

On 2023-09-20 11:56, Stephane Bortzmeyer wrote:

On Wed, Sep 20, 2023 at 07:43:20AM +0200,
  Robert Kisteleki  wrote
  a message of 42 lines which said:


All else (continuing to run existing measurements, creating new ones,
real-time streaming of the results, APIs, UI, ...) are running undisturbed.


This is not what I observe. For instance, asking for 100 probes in
France yields a "No suitable probes" (see measurement #60181887). So,
"completely down" seems a fair summary to me.


In that measurement you're asking specifically for probes tagged with 
"system-ipv4-works" - which, as you can read in the related thread this 
morning, is a causality of the current issue we're facing. We have a fix 
for this particular slice of the problem that we can apply once the 
system has recovered.


Regards,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] One-off measurements stopped after 5-6 minutes without results

[Robert Kisteleki]

Hello,

As you've probably seen in the other thread, we have a problem in 
storing/processing new results. We're working on fixing this.


I wrote "all else is working fine" there - which I have to correct now: 
the above error affects the probe tagging, where because of the lack of 
"good data" the tagging process removes some tags (as you note below). 
Even though the lack of new data is not the expected scenario, this is 
something we will fix.


Regards,
Robert


On 2023-09-19 20:03, Gerdriaan Mulder wrote:

Hi list,

A few hours ago (around 15:00Z), I launched some one-off 
measurements[1][2][3] on the platform for a select number of probes 
(based on ASN and region). The target in all three cases was 
2a02:1807:1020:700::1 (no hostname, purely the address). Although the 
number of requested and participating probes varied a bit, it seems none 
of the measurements yielded any results.


At the time I didn't look at recent posts on this mailing list, nor was 
I aware of this ongoing incident[4] on the Atlas backend. It seems like 
that incident might be the cause of the (current) unavailability of 
these results.


It would be great if anyone can confirm that (viewing results of) 
user-defined measurements are affected as well.


Aside, I suppose the following messages on "My RIPE Atlas Dashboard" are 
also caused by the mentioned incident[4]?


--->8---
2023-09-18 16:40 UTC Your probe #11nnn was automatically untagged as 
"system: Resolves  Correctly"
2023-09-18 16:40 UTC Your probe #11nnn was automatically untagged as 
"system: Resolves A Correctly"
2023-09-18 16:20 UTC Your probe #11nnn was automatically untagged as 
"system: IPv6 Works"
2023-09-18 16:20 UTC Your probe #11nnn was automatically untagged as 
"system: IPv4 Works"

---8<---

Perhaps the incident page[4] can include a few lines of the current 
(estimated) impact, as well.


Best regards,
Gerdriaan Mulder

[1] https://atlas.ripe.net/frames/measurements/60161706/ (default IPv6 
traceroute)
[2] https://atlas.ripe.net/frames/measurements/60162230/ (IPv6 
traceroute, paris=0)
[3] https://atlas.ripe.net/frames/measurements/60162753/ (default IPv6 
ping)

[4] https://status.ripe.net/incidents/wfd7qywz3v68



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas fully down..

[Robert Kisteleki]

On 2023-09-19 23:04, Ernst J. Oud wrote:

Considering the fact that all of Atlas is completely down for more than 24 
hrs., I find the silence a bit deafening. No status updates, nothing. Weird.

Not up to RIPE standards.

Regards,

Ernst J. Oud


Good morning,

I'm sad to report that indeed there's still an issue with result 
processing - which is still reflected on the status page.


Specifically, the HBase backend that is responsible for storing and 
retrieving the new (and historic) results is struggling to store the 
data form the last ~24 hours. The teams have been working on solving 
this basically 24/7 since the issue occurred but haven't been successful 
yet.


All else (continuing to run existing measurements, creating new ones, 
real-time streaming of the results, APIs, UI, ...) are running undisturbed.


I hope this helps understanding the extent of the problem, and we'll of 
course let you know when there's progress.


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Changes to RIPE Atlas API keys

[Robert Kisteleki]

Dear RIPE Atlas users,

We'd like to update you on some upcoming changes regarding API keys in 
RIPE Atlas.


TL;DR nothing changes regarding how you can use your API keys in the 
short term - as long as you're actually using them. However, we'll 
change how unused or forgotten keys are handled as well as remove the 
less secure in-URL use of them.



At the moment RIPE Atlas users can query their existing API keys via the 
UI and API, including the possibility to retrieve old keys. In order to 
improve the security of how we handle these, we'll introduce the 
following changes in October 2023:


* The listing (retrieval) of keys will only reveal parts of the keys 
(enough to identify them) in the API as well as in the UI.


* We'll add the ability to "regenerate" an API key, which will replace 
the secret UUID of the key while keeping exactly the same permissions.


* Unused API keys will automatically be frozen after 1 year of not being 
used. Active keys (i.e. the ones that have been used at least once) will 
not be frozen.


You still have the ability to save your keys until these changes are 
done and, as written above, you will be able to regenerate them later. 
We'll notify this list when the changes are about to be done.



In addition, in order to further increase the security of our system, in 
the long run we'll make changes about how these API keys are 
communicated to the API:


* At the moment the API accepts these either in HTTP headers 
("Authorization" header) or in the URL (?key=xyz), although the 
Authorization header version has been documented as the preferred 
version for some time.


* We'll deprecate and remove the ability to use the URL form in about a 
year (around October 2024).


* We plan to send further reminders about this change over time, as well 
as reaching out to heavy users of the to-be-removed format.


Regards,
Robert Kisteleki
RIPE Atlas team




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Issues with Atlas back-end

[Robert Kisteleki]

Hi,

The issues with RIPEstat have been resolved in about an hour, the 
problem with the Atlas data backend is unfortunately still ongoing.


Regards,
Robert


On 2023-09-19 00:03, Ernst J. Oud wrote:


Hi,

 From September 18th there is an issue with the Atlas back-end, already 
9 hours delay. Built-ins show data until around 14:00 UTC. It is now 
midnight.


I am confused, the status page (https://status.ripe.net/ 
) at the bottom says:


  “Resolved - The backend team could restore the service level and with 
that issues in RIPEstat stopped as well.

Sep 18, 15:11 UTC”

It clearly is not resolved. On the top of that status page it says:

“Investigating - We are experiencing issues with the RIPE Atlas backend 
and currently are investigating this issue.

Sep 18, 2023 - 14:07 UTC”

So what is the status?

Regards,

Ernst J. Oud



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Built-ins tab in Atlas web interface does not respond

[Robert Kisteleki]

On 2023-09-08 18:21, Ernst J. Oud wrote:

It appears to work again. Thanks!


Yes, we deployed a fix sometime in the afternoon.


And it looks better than before, or am I hallucinating here?


AFAIK there were no real changes, maybe some tweaks.

Regards,
Robert


Regards,

Ernst



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Questions with regard to the measurement status and running time

[Robert Kisteleki]

Jerome,


My questions are:
1. Why is the measurement "ongoing" even if the results have already 
been made available?


One-off measurements are automatically marked "stopped" after 15 minutes 
by the system. This is mostly a book-keeping exercise to know what is 
active and what is not; it does not affect the flow of results.


2. What is the meaning of the "requested start time" and "requested stop 
time" on the RIPE Atlas portal?


The intention is to be able to pre-schedule a meeting into the future 
(for whatever reason you may have) instead of starting it asap, or to 
pre-determine its end if you already know when that should be. I believe 
the "start in future" is applicable to one-offs as well, stop time not 
so much (see above, they are auto-stopped).


3. Will a one-off measurement release one slot from the measurement rate 
limit once its result becomes available even if it’s still in "ongoing" 
state?


No, the measurement status is what counts. So such a slot is freed up 
after the msm is marked as stopped. We are considering a different 
book-keeping mechanism where the amount of results (activity) is used 
for quotas instead of number of measurements -- but this is not the case 
yet.


We can adjust these limits, so if you have good reasons to do more (e.g. 
measurement campaign, bursts because of some research, etc.) then just 
let us know and we can help!


Regards,
Robert


Thanks!
Best,
Jerome



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] What is the trick with "+Reuse a set from a measurement" ?????

[Robert Kisteleki]

Hello,

The UI at the moment does not offer a lookup based on measurement ID for 
this feature; mostly because in the end one would submit the same ID 
back to the API, so this is not perceived to be a critical feature. 
Lookups based on measurement descriptions (the intended use case) work 
as expected.


As part of the UI modernisation we're getting close to publishing a 
renewed "new measurement" page, where we'll also improve the behaviour 
of this function. I hope this will properly address your question.


Regards,
Robert



On 2023-08-18 12:44, Haake, Oliver via ripe-atlas wrote:

Moin,

Any updates here?

To me it looks like as if this very useful feature is still not working.



Cheers,
Olli


 Original Message 
From: Malte Tashiro via ripe-atlas [mailto:ripe-atlas@ripe.net]
Sent: Monday, November 21, 2022 at 4:17 AM
To: ripe-atlas@ripe.net
Subject: [atlas] What is the trick with "+Reuse a set from a 
measurement" ?


Hi Barry,

this function does indeed still work, but it appears that the web
interface for it is somewhat broken?
If I search for "traceroute" one of the results is measurement 1001589
[0], which has this term in the description, but searching for that
measurement id gives no result. And in any case, the only available
results seem to be very old looking at the ids.

What _does_ work is using the measurement API, for example:

"probes": [
    {
     "type": "msm",
     "value": your_msm_id_here,
     "requested": number_of_probes_here
    }
   ]

Not sure this is the answer you wanted to hear, but maybe someone from
the Atlas development team can figure out what's up with the web interface.

Best,
Malte

[0] https://atlas.ripe.net/measurements/1001589

On 11/21/22 03:16, Barry Greene wrote:
Hi Team,

I’m trying to use “+Reuse a set from a measurement.” (See
https://atlas.ripe.net/docs/getting-started/user-defined-measurements.html 
)

I’m going back to my own measurements and trying different Measurement
IDs. None of them work. I try from someone else’s test. None of them
work.

Q. Does this function work anymore?

Barry







--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Built-ins tab in Atlas web interface does not respond

[Robert Kisteleki]

On 2023-09-07 09:53, Ernst J. Oud wrote:

Hi,

In the web interface clicking the tab “Built-ins” does not work, it shows 
“loading” and nothing happens. Tried different browsers.

Regards,

Ernst J. Oud


Hi,

Thank you for reporting, indeed something is not right there. We are 
looking into it.


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Probe with ever changing upstreams/ASs?

[Robert Kisteleki]

Hi Carsten,

Anyhow: as part of this tunnel setup, the other end of the tunnel 
changes every 24 to 48 hours - and so does the upstream and therefore 
also the AS of the upstream provider.


I think the short answer is: probably not highly useful. But of course 
it depends :-)


Each result is tagged with the (then) source IP of the probe, as far as 
the infrastructure knows that. For argument's sake let's say that is the 
IP where the probe is otherwise connecting from.


If that IP, because of the tunnels, changes regularly, then it's ... 
kind of ok, results from different days will vary both in RTT and in 
source IP (and in actual content. eg. DNS or TLS results). However, if 
the probe's IP stays stable but the results vary each day, then it's may 
be confusing to interpret.


At the end of the day, measurements involving this probe would give 
varying results depending on time. It's totally real, but is it useful?


Cheers,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Easy way to view DNS results?

[Robert Kisteleki]

Hello,

We're not parsing and inserting all the bits from the raw response into 
the JSON mostly because it would inflate the size of the results 
enormously, while users are generally only interested in specific bits.


The UI gives the basics (success, RTT) to show the big picture but 
exposing all the details would overload the interface, especially on mobile.


The "abuf" is always in there, so consumers of the results can parse 
that (as you already know, and as Seth does below). There are abuf 
parsers out there, and if you're using Python then I'd recommend using 
Sagan (https://github.com/RIPE-NCC/ripe-atlas-sagan/)


Cheers,
Robert


On 2023-07-25 07:31, Seth David Schoen wrote:

Micha Bailey writes:


Hi, I’m wondering if I’m missing something - I ran a DNS measurement with
110 probes, and the results are in, but the site UI seems to only show the
result status and time elapsed until response, but not the results
themselves. Looking at the JSON from the result endpoint I’m seeing that it
seems to return the raw binary response - is there a particular reason it
can’t be parsed and displayed in a more human-readable format? I know there
are libraries and tools that do result parsing, but being on mobile at the
moment (and often) means those are somewhat less accessible as far as I’m
aware.


Hi Micha,

Do you mean the way that the JSON contains the "abuf" field with the
base64-encoded DNS response?

I have actually not worked with these before, but I was able to make some
progress in Python as follows (using the python3-dnspython package).
You could modify the "to_text()" call to print out some more specific
information from the DNS reply of interest to you, or modify the enclosing
print() to save it elsewhere.

I don't know why there isn't a parsed version of the reply included in
the JSON, but perhaps the idea is that the literal details are of
interest to some researchers.  One example that I happened to notice
in trying to answer your question: in parsing a sample DNS measurement
this way, I notice the use of DNS case randomization (also called "0x20
randomization") in some replies but not in others.  Having the literal
DNS query reply could help with analyzing the prevalence of this
mechanism, whereas it might be obscured by a parser that was written by
someone who believed that DNS replies are not case insensitive (which
is true from one point of view, but not from another point of view!).


#!/usr/bin/env python3

import json
import base64
from dns import message
import sys

measurements = json.load(open(sys.argv[1]))

for measurement in measurements:
 if "resultset" in measurement:
 for resultsetitem in measurement["resultset"]:
 if "result" in resultsetitem:
 abuf = resultsetitem["result"]["abuf"]
 msg = message.from_wire(base64.b64decode(abuf))
 print(msg.to_text())
 print()



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Issues downloading large result files

[Robert Kisteleki]

Hello,

We've seen periods where someone was downloading 10-30GB result blobs 
(years' worth of results), multiple downloads in parallel, and while it 
stressed the service, it worked for the most part. This *could* work, 
but it's suboptimal.


My advice would be to download in chunks (perhaps daily or weekly ones, 
depending on how much data that is). You can do this via the 
start=TIMESTAMP and stop=TIMESTAMP parameters, eg.


https://atlas.ripe.net/api/v2/measurements//results/?start=1688947200=1689033599=txt

Hope this helps,
Robert


On 2023-07-19 11:48, Ana wrote:

Hi everyone,

I'm seeing something very strange when trying to download the result 
files for some large measurements.


The download is a different size each time (sometimes 1.4GB, then 3.2GB, 
then back to 1 GB again) and checking the end of the file I can always 
see it's not complete as it stops in the middle of a measurement, 
closing brackets aren't there etc.


I guess this is because of the size of the file, not sure if there's 
some sort of limit on RIPE Atlas?


If anyone here has seen anything like this or has any advice on how to 
fully download the result that would be much appreciated!



Thank you,

Ana




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Delays in backend…

[Robert Kisteleki]

I forgot to mention that the result stream 
(https://atlas-stream.ripe.net/) does not share fate with the data 
backend, it is delivering results as expected.


Cheers,
Robert


On 2023-07-01 13:06, Robert Kisteleki wrote:

Hello,

I can confirm that the data storage backend has been struggling in the 
last couple of days. Our operations people are trying to come up with a 
permanent solution, but the problem keeps reappearing.


The status page has entries about these issues - though admittedly not 
about all the hickups. Apologies for that, we'll add one for the current 
instance.


We have a publicly available delay indicator on the front page of Atlas, 
that helps if you're unsure.


Regards,
Robert


On 2023-07-01 12:32, Ernst J. Oud wrote:

L.S.

Saturday July 1st. Just did a test, first result reported in the web 
interface after 25 minutes! Results of the other probes in this test 
still unreported.


Come on guys. There is something terribly wrong in the back-end for 
three days now. I know it is a free service but is there no monitoring 
on this?


Status page still mentions no issues. There clearly is.

Met vriendelijke groet / Regards,

Ernst J. Oud


On 30 Jun 2023, at 13:24, Ernst J. Oud  wrote:

Hi,

Measurements stream fine in Magellan, but results only show up in the 
web interface after - in extreme cases - hours.


The RIPE status info says:
Resolved
This incident has been resolved.
Posted 1 day ago. Jun 29, 2023 - 10:11 UTC
Investigating
We are experiencing issues with the RIPE Atlas backend and currently 
are investigating this issue.

Posted 1 day ago. Jun 29, 2023 - 08:55 UTC






But clearly this has not been resolved!

Regards,

Ernst J. Oud
--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas






--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Delays in backend…

[Robert Kisteleki]

Hello,

I can confirm that the data storage backend has been struggling in the 
last couple of days. Our operations people are trying to come up with a 
permanent solution, but the problem keeps reappearing.


The status page has entries about these issues - though admittedly not 
about all the hickups. Apologies for that, we'll add one for the current 
instance.


We have a publicly available delay indicator on the front page of Atlas, 
that helps if you're unsure.


Regards,
Robert


On 2023-07-01 12:32, Ernst J. Oud wrote:

L.S.

Saturday July 1st. Just did a test, first result reported in the web 
interface after 25 minutes! Results of the other probes in this test 
still unreported.


Come on guys. There is something terribly wrong in the back-end for 
three days now. I know it is a free service but is there no monitoring 
on this?


Status page still mentions no issues. There clearly is.

Met vriendelijke groet / Regards,

Ernst J. Oud


On 30 Jun 2023, at 13:24, Ernst J. Oud  wrote:

Hi,

Measurements stream fine in Magellan, but results only show up in the 
web interface after - in extreme cases - hours.


The RIPE status info says:
Resolved
This incident has been resolved.
Posted 1 day ago. Jun 29, 2023 - 10:11 UTC
Investigating
We are experiencing issues with the RIPE Atlas backend and currently 
are investigating this issue.

Posted 1 day ago. Jun 29, 2023 - 08:55 UTC






But clearly this has not been resolved!

Regards,

Ernst J. Oud
--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q3 2023

[Robert Kisteleki]

Dear colleagues,


We have published our next quarterly plans for RIPE Atlas on our website:
https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas

This quarter, we'll continue discussing the five proposals for changing 
the RIPE Atlas behaviour and start working on renewing the big data backend.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q2 2023

[Robert Kisteleki]

Dear colleagues,

We’ve now published our quarterly planning for RIPE Atlas. These are 
available at:

https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas

This quarter, we'll continue the discussion on five proposals for 
changing RIPE Atlas behaviour and work on restructuring and renewing 
pages of the user interface.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Missing ping and traceroute dumps for 2023-01-23

[Robert Kisteleki]

Just an update on this: the issue has been solved in the meantime, the 
missing dumps have been backfilled.


Regards,
Robert


On 2023-01-31 13:18, Robert Kisteleki wrote:



On 2023-01-31 11:40, Maxime Mouchet wrote:

Hi,

I noticed that there is no ping and traceroute dumps for 2023-01-23: 
https://data-store.ripe.net/datasets/atlas-daily-dumps/2023-01-23/

The other days are fine though.


Ooops. We'll check!


Is there any chance to get these dumps? Or are they just lost?


We have the data and should be able to create those dumps. Please stand 
by :-)


Cheers,
Robert


Thanks!
Maxime






--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Rate limiting on the APIs

[Robert Kisteleki]

Update: we applied this change a few minutes ago. We'll keep observing 
the system for ill effects.


Regards,
Robert


On 2023-01-30 14:57, Robert Kisteleki wrote:

Dear all,

We have seen in the past that even with the best intentions, every now 
and then some users are using excessive amounts of requests to the RIPE 
Atlas APIs, in some cases resulting in degraded performance to all 
users. An example for the curious: some AWS account asking the probe API 
for live queries once a day at 8am like clockwork; over a million 
queries for probe statuses (note: we don't have a million probes - every 
query is repeated hundreds or thousands of times) [1]


We will apply rate limiting to protect from (some of) these. In particular:

* The API in general will impose a limit of no more than 300 
queries/second (if done over multiple connections)


* The measurement API in particular will impose a limit of 150 
queries/second (if done over multiple connections)


* When reaching the limit, an HTTP response code of 429 (Too Many 
Requests; https://www.rfc-editor.org/rfc/rfc6585#section-4) will be given.


According to our logs these limits will in general not be visible to 
users. Users going over these limits will see the 429 response code; 
client libraries should be able to handle this. We will observe the 
effect of these measures to determine if we need to tighten further - or 
perhaps relax them.


We plan to apply these changes sometime next week; I'll let you know 
once they are in place.


Regards,
Robert


[1] a much better way of doing this is to use the probe archive 
(https://ftp.ripe.net/ripe/atlas/probes/archive/); this is one-time 
fetch of 2MB or so and probe statuses are pretty stable in general





--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Missing ping and traceroute dumps for 2023-01-23

[Robert Kisteleki]

On 2023-01-31 11:40, Maxime Mouchet wrote:

Hi,

I noticed that there is no ping and traceroute dumps for 2023-01-23: 
https://data-store.ripe.net/datasets/atlas-daily-dumps/2023-01-23/
The other days are fine though.


Ooops. We'll check!


Is there any chance to get these dumps? Or are they just lost?


We have the data and should be able to create those dumps. Please stand 
by :-)


Cheers,
Robert


Thanks!
Maxime




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Rate limiting on the APIs

[Robert Kisteleki]

Dear all,

We have seen in the past that even with the best intentions, every now 
and then some users are using excessive amounts of requests to the RIPE 
Atlas APIs, in some cases resulting in degraded performance to all 
users. An example for the curious: some AWS account asking the probe API 
for live queries once a day at 8am like clockwork; over a million 
queries for probe statuses (note: we don't have a million probes - every 
query is repeated hundreds or thousands of times) [1]


We will apply rate limiting to protect from (some of) these. In particular:

* The API in general will impose a limit of no more than 300 
queries/second (if done over multiple connections)


* The measurement API in particular will impose a limit of 150 
queries/second (if done over multiple connections)


* When reaching the limit, an HTTP response code of 429 (Too Many 
Requests; https://www.rfc-editor.org/rfc/rfc6585#section-4) will be given.


According to our logs these limits will in general not be visible to 
users. Users going over these limits will see the 429 response code; 
client libraries should be able to handle this. We will observe the 
effect of these measures to determine if we need to tighten further - or 
perhaps relax them.


We plan to apply these changes sometime next week; I'll let you know 
once they are in place.


Regards,
Robert


[1] a much better way of doing this is to use the probe archive 
(https://ftp.ripe.net/ripe/atlas/probes/archive/); this is one-time 
fetch of 2MB or so and probe statuses are pretty stable in general



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Encouraging people to upgrade software probe versions

[Robert Kisteleki]

Hi,

To me it seems that there are oh-so-many ways of packaging and 
distributing this software to the match the multitude of needs (RPMs, 
DEBs, openwrt, docker, VMs, ...) and us giving support to multitude of 
these stretches our resources thin. As I wrote before, such packaging 
would be preferably achieved via professional maintainers.


> Is there an actual reason, why it was decided to let users manage the
> software probe installation?

The intention here is/was that many users already have their own machine 
(VM or server or home router or such) that can be used as the platform. 
One can also easily spin up and dedicate a new HW, like a lingering 
Raspberry Pi, to this.


Cheers,
Robert


On 2023-01-27 10:43, Simon Brandt via ripe-atlas wrote:

Hi Robert,

The existence of software probes is great, but instead (or besides) of 
providing packages or source code, why not distribute a prebuild VM as 
OVF file?


Advantages:
- The RIPE Atlas team manages the whole OS, like it's doing for the 
hardware probes. Thus, updates can be deployed whenever needed.
- You can even use OpenWRT as VM operatingsystem. This means all the 
same premises/conditions as for hardware probes.

- an OVF file is easier to deploy, for the community
- RXTXRPT switch is obsolete
- No more false RXTXRPT data, since the report counts all traffic of the 
host, not only the traffic that is generated by the SW probe application.


Is there an actual reason, why it was decided to let users manage the 
software probe installation?
Please consider to distribute a prebuild VM *additionally* to the 
existing ways and see what happens. I'm sure, most new users will choose 
a prebuild VM.



BR,
Simon


On 19.01.23 12:48, Robert Kisteleki wrote:


That is reasonable; the difference is that we are not in control; the 
host OS is. Redhat/Fedora/derivatives as well as Debian/derivatives 
have an official solution to this via their package management 
services and I believe this is the standard way (surely with 
exceptions :-) ) of handling these matters. We are in the process of 
adopting these.







--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Encouraging people to upgrade software probe versions

[Robert Kisteleki]

Hi,

On 2023-01-29 20:11, Gert Doering wrote:

Hi,

On Fri, Jan 27, 2023 at 02:15:44AM +0100, Robert Scheck wrote:

On Mon, 09 Jan 2023, Gert Doering wrote:

So don't do OS updates.  But *do* update the probe software, automatically,
and in normal intervals.


writing while wearing my hat as a RPM packager: I'm not really sure how
this shall work practically: If the probe software is run on regular (non-
hardware probe) systems, there are runtime dependencies to other software,
e.g. due to dynamic linking. Latter is encouraged by Linux distributions,

[..]

This is all good and true - so, do not distribute as RPM.


We produce the ("self signed") RPMs because we need them internally for 
the anchors anyway, so figured we may as well publish them, plus some 
people actually need/prefer binary RPMs.


Otherwise we're heading to a direction where we act as the upstream for 
the software, and professional package maintainers do the proper 
signing, distribution and updates (like Robert S. could become one for 
RPMs).



Distribute as Docker image, or Flatpack, or whatever all these new
"encapsulate a program and all its dependencies" approaches are called,
and I'm sure some of them have mechanisms to enforce timely updates.


Docker files exist, though we are not maintaining official images. That 
may change in the future. As said before, "have mechanisms to enforce 
timely updates" indeed has proper solutions for basically all platforms.



First and foremost, a RIPE Atlas probe is a centrally managed piece of
software, and this is what it needs to be: centrally managed, centrally
upgraded, always at the upgrade level the central controller wants it
to be.


As I tried to explain before, this is true for the HW probes, not so 
much for SW.


Cheers,
Robert


Gert Doering
 -- NetMaster




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas anchors as ping targets

[Robert Kisteleki]

The idea here is

  - I have a host in a "home network" that has 2 ISPs, with a global IPv6
address from each of them

  - one of the ISPs fails, but the host does not know, and keeps using
the "broken" IPv6 source address

  - by cyclic measurements, Brian's tool will see "nah, that address stopped
working" and will de-preference it on the host -> host starts using the
other IPv6 source address -> connectivity restored (over the other ISP)

and the test envisioned would be "ping something willing to be pinged" :-)


Yes that's fine, there could be $reasons to do more measurements! :)

Cheers,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Access to probes and documents

[Robert Kisteleki]

Hello,

On 2023-01-29 11:08, Mccherry, Paul (Postgraduate Researcher) wrote:
Althiough I am logged in when trying to access various probes I do not 
seem to have the rights, probes such as 91,353,365,3094….


Those probes are (except 3094) marked as private, so authentication is 
not enough. You can still find basic information about them in the API, 
such as https://atlas.ripe.net/api/v2/probes/365/


Additionally I noticed I cannot access any documents on RIPE labs such 
as ‘RIPE IPmap: geolocating routes across the internet’ or ‘Five 
proposals for a better RIPE Atlas.’


That is a weird one - RIPE Labs articles are very public. Are you saying 
you cannot access these?


* 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/
* 
https://labs.ripe.net/author/alun_davies/ripe-ipmap-geolocating-routes-across-the-internet/


Cheers,
Robert


Can anyone resolve these access rights please ?

Regards

Paul McCherry

Phd Researcher

Lancaster University




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas anchors as ping targets

[Robert Kisteleki]

Hi,

On 2023-01-28 02:24, Brian E Carpenter wrote:

TL;DR: Is it OK to use Atlas anchors as random ping targets?


RIPE Atlas anchors are known and willing targets for measurements - so 
I'd say yes, definitely!




Would it be OK to choose the probe target by picking an Atlas anchor at 
random? (I have running code for that, but will not post it to github if 
there are objections.)


The anchors are in a full measurement mesh - meaning they ping and trace 
each other. Also, all anchors are assigned some probes to measure them. 
You can find these measurements on the anchors' pages.


This may mean you don't need to set up new measurements. If so, it's 
always nicer to reuse existing data :-)


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Encouraging people to upgrade software probe versions

[Robert Kisteleki]

Hello,

On 2023-01-09 10:52, Gert Doering wrote:

Hi,

On Mon, Jan 09, 2023 at 10:19:34AM +0100, Michel Stam wrote:

The automatic update for the CentOS package was removed as of 5080. This means 
that the update to 5080 will still be automatic, but not any more after this 
(say 5090 onwards).
This was a request by several users because Atlas forced the update, which 
violated their sysadmin policies.


I find this surprising, to say the least.

The whole point about ATLAS Probes is "RIPE NCC manages them" - as host
of a handful of hardware probes, I have no say in whether I want them
upgraded or not, or what they should do or not do.


That is indeed true for hardware probes - the host has little influence 
on the firmware upgrade. It is intended to be plug and play. It is also 
true for anchors.



So I would expect all software probes to always upgrade themselves,
and this be clearly communicated to SW probe hosts.


That is reasonable; the difference is that we are not in control; the 
host OS is. Redhat/Fedora/derivatives as well as Debian/derivatives have 
an official solution to this via their package management services and I 
believe this is the standard way (surely with exceptions :-) ) of 
handling these matters. We are in the process of adopting these.



If this violates your sysadmin policies, you shouldn't run 3rd-party
maintained *and operated* software.


The intention here is to do less special handling, and conform more to 
the operational practices. The case is entirely solvable using those, 
and this direction is expected to make deployment and maintenance easier 
on hosts (as well as the Atlas team) in the long run.


I hope this explains the motivation,
Robert



Gert Doering
 -- NetMaster



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] ripe-atlas Digest, Vol 137, Issue 8

[Robert Kisteleki]

Hello,

On 2023-01-09 22:59, Geert Jan de Groot wrote:
One question perhaps is if a probe running very obsoleted probe code, is 
useful or perhaps at some time probes older than version 1234 should not 
be used for pool measurements?


For instance, release 2345 enables measurements on 240/12 while older 
releases, including release 2344, 240/12 is broken.
Then, having probes with software 2344 or lower will give bias on 
measurements for 240/12.


Let me offer some background information here.

For the most part, newer firmware versions provide bugfixes and/or new 
functionality. If a particular feature already exists in version 
$OLD_VERSION which is running on some probes, there's no reason to 
exclude probes from measurements that need this feature just because 
it's not the $NEWEST_VERSION.


On the flip side, the measurement scheduler is _mostly_ aware of new 
features in the firmware, in particular new measurement types and 
options and such. The we-no-longer-exclude-240/4 change is not one where 
we added this logic; mostly because it would be an exceptional case for 
a transitional period, ie. until probes upgrade to the version that has 
this change.


The exception here is the v1/v2 probes; these are not useful for these 
measurements. Because of this I'd recommend excluding these, either in 
the measurement specification, or while processing the results.


And for that reason I wonder if it makes sense to have a policy to 
"phase out" probes that are running obsoleted code, however well intended?


Indeed, that is a direction we are / will be headed. We plan to add 
notifications for software probe hosts if their probe firmware is not 
up-to-date. In addition, eventually we'll "phase out" (as you say) 
probes that have too old firmware - where the definition of "too old" 
can change over time.


Cheers,
Robert


Geert Jan



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Ability to traceroute to 240/4 - software vs. hardware probes

[Robert Kisteleki]

What could account for the difference in how hardware and software
probes handle this task?  I would have thought that the change from
last November would make software probes willing to attempt these.


I'd believe it's the combination of what versions these probe run (i.e.. 
is it at least 5060?) plus the properties of the network they are in. 
The code is otherwise the same.


Cheers,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Positively meant critique …

[Robert Kisteleki]

Hi,

On 2023-01-07 18:07, Ernst J. Oud wrote:

LS.

Is there a kind of common understanding what this mailing list is about? What 
its purpose is?


This is the general discussion list for topics related to RIPE Atlas.


I see important discussions such as those on private measurements but no 
conclusions… I ask questions that lead to response from a RIPE employee but not 
to answers/solutions…


We put forward some proposals about changing properties of RIPE Atlas 
behaviour, and there are ongoing discussions. We (the development team) 
are supporting that discussion, though during the holiday period that 
surely was not at full swing...



(For instance on why results from first hop measurement to ping.ripe.net cannot 
be reported on by Magellan.)


(Will answer that separately.)


Can RIPE give some explanation what the use for them is of this mailing list?


The mailing list page describes it as: "This mailing list is open to 
anyone with an interest in sharing experiences and information about the 
project."



Are there other entries into RIPE for support questions?


Many questions are answered on this list in a "community self support" 
fashion. If you want / need to write to support, then you should address 
at...@ripe.net - which is ticketised.



Would some kind of forum not be a much better solution to discuss matters with 
probe “owners” instead of using old fashioned unstructured mailing lists… the 
way we did things when the internet was in its infancy?


The RIPE NCC introduced a forum (forum.ripe.net) a short while ago. 
There are some RIPE Atlas related topics there too.


Regards,
Robert


Regards,

Ernst J. Oud



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] A number of built-ins stopped?

[Robert Kisteleki]

Hi,

On 2022-12-23 18:25, Ernst J. Oud wrote:

L.S.

All the probes in NL that I had a look at, in a variety of different ASN’s, 
have no data available via the Atlas web of via the api (JSON, Magellan) for 
a.o. built-in measurement 1009 (ping to a.root-servers.net) and several others 
after December 22nd, around 18:00 hrs. UTC.

When I have a look at German probes there is data available. This thus appears 
to be a problem with all and only Dutch probes’ data.

A normal ping via a Windows command line works fine, i.e. a.root-servers.net is 
reachable.

Creating a ping measurement via Magellan to a.root-servers.net using Dutch 
probes generates results, i.e. the probe infrastructure works but data storage 
does not, for data from probes in NL.

Rather serious problem I guess. No mentioning on status.ripe.net though.


I just updated the status page with an entry about this.


Had an email conversation with Robert from the Atlas team today. No solution 
yet.


We believe we've found the reason [1], and we expect these measurements 
to go back to normal in the next couple of hours.


[1] from the status page: "the infrastructure declared some built-in 
measurements finished, therefore the probes stopped executing them"


Happy holidays!
Robert


Regards,

Ernst J. Oud


Met vriendelijke groet,

Ernst J. Oud


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC]

[Robert Kisteleki]

Hi,


The definition of a "small" test could be something like;
- maximum 20 probes
- runs for maximum 60 minutes
- is repeated maximum 10 times
   (if you run every 300sec you have to limit the endtime to 50 minutes)


What would you propose the API to do if this rule is violated? Should it 
outright refuse the measurement, or should it silently turn on the 
public flag (silently, because even if we emit a warning, it is probably 
never seen by a human...)?


With such limits you can troubleshoot things (non-publicly) but you 
can't build your monitoring system on top of that.


I guess this hinges on what level of support (legal, technical or other) 
we're aiming for when it comes to others building services on top of 
RIPE Atlas.


Regards,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q1 2023

[Robert Kisteleki]

Dear colleagues,

The quarterly plans for RIPE Atlas can now be found at:
https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas

In the last quarter, we worked on several proposals to change the RIPE 
Atlas behaviour, including implementing general availability of HTTP 
measurements and removing the ability to schedule “non-public” 
measurements.


This quarter, we’ll continue with the distribution of the v5 RIPE Atlas 
hardware probes to hosts, ambassadors and sponsors as well as work on 
other items proposed by the community.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
Research and Development Manager
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC]

[Robert Kisteleki]

If you can count on one hand the number of users using >90% of the
private measurements over a longer timeframe than two weeks, then I
submit that the choice is clear.


This information is somewhat harder to extract, but we'll try and get 
back to you!


Cheers,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Probe status pages -- was: Re: Non-public probes?

[Robert Kisteleki]

On 2022-12-15 14:33, ripe@toppas.net wrote:

Hi Chris,

sorry for "stealing" this conversation, but it's interesting to hear 
that there will be a redesign of the probe page coming up soon. Can we 
have a discussion about that? There are several things, that bother me a 
bit...


BR,
Simon


Hi,

The team started thinking about how to improve these pages. We're using 
already collected user input, our own observations and including 
technical changes that are looming anyway.


We're early in this work but are happy to take your input already :-) It 
can be a thread you start here (e.g. what do you like now? what do you 
dislike? what is missing? ... about probe pages) or in a "private 
interview" of some kind.


Cheers,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Proposal: Remove support for non-public measurements [ONLY-PUBLIC]

[Robert Kisteleki]

Dear RIPE Atlas users,

We recently published a RIPE Labs article containing a few proposals: 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. 
We'd like to encourage you to express your comments about this proposal 
(if you'd like to share them) here.


Regards,
Robert Kisteleki
For the RIPE Atlas team

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Proposal: Add support for STARTTLS measurements [STARTTLS]

[Robert Kisteleki]

Dear RIPE Atlas users,

We recently published a RIPE Labs article containing a few proposals: 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. 
We'd like to encourage you to express your comments about this proposal 
(if you'd like to share them) here.


Regards,
Robert Kisteleki
For the RIPE Atlas team

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Proposal: Generic HTTP measurements [GENERIC-HTTP]

[Robert Kisteleki]

Dear RIPE Atlas users,

We recently published a RIPE Labs article containing a few proposals: 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. 
We'd like to encourage you to express your comments about this proposal 
(if you'd like to share them) here.


Regards,
Robert Kisteleki
For the RIPE Atlas team

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Proposal: Measure well-known CDNs,[CDN-HTTP]

[Robert Kisteleki]

Dear RIPE Atlas users,

We recently published a RIPE Labs article containing a few proposals: 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. 
We'd like to encourage you to express your comments about this proposal 
(if you'd like to share them) here.


Regards,
Robert Kisteleki
For the RIPE Atlas team

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Proposal: remove per-hop “late” packets from traceroute [LATE-PACKETS]

[Robert Kisteleki]

Dear RIPE Atlas users,

We recently published a RIPE Labs article containing a few proposals: 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/. 
We'd like to encourage you to express your comments about this proposal 
(if you'd like to share them) here.


Regards,
Robert Kisteleki
For the RIPE Atlas team

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas api down

[Robert Kisteleki]

On 2022-12-14 10:03, Alexander Burke via ripe-atlas wrote:

Hi Robert,

If the status page is manually updated, it should probably show a prominent 
note to that effect so that it is consumed well-salted.


Hi,

The event shows up in the "past incidents" section of the page. Is that 
what you're looking for?


Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas api down

[Robert Kisteleki]

Hello,

Our data storage backend experienced two separate issues during the 
night: one between around 18:30-20:15 (CET) and another one around 
1:00-1:45 (CET)


I am updating the status page now.

Apologies for the issues,
Robert



On 2022-12-13 19:54, Ernst J. Oud wrote:


Hi,

As of around 17:30 UTC on December 13th, all measurements other than built-ins, 
from the website or via Magellan, report no results. The website just hangs and 
Magellan  times out.

I believe all access via the api to be broken.

I read that maintenance is pending, but that is scheduled the 14th…

What’s up?

Regards,

Ernst J. Oud


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] atlas related mails come 4 times

[Robert Kisteleki]

Hi,

We checked on our end, and it seems your server for some reason did not 
properly accept the mail from ours, so it was tried multiple times. 
Peter's answer is likely correct.


Cheers,
Robert


On 2022-12-02 11:16, Endre Szabo wrote:

Hi chaps,

Just recently any mail from atlas monitoring shows up 4 times in my inbox. 4 
exact copies of a mail (Message-Id is also the same), in the same time, in 4 
different SMTP sessions. Anyone else noticed such behavior?

Also, I've just subscribed to this list and the subscription confirmation 
request mail came twice :D One via IPv4 and one via IPv6. Apparently no other 
senders have this issue on my end.

--Endre



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Removing the "technical updates" RSS feed

[Robert Kisteleki]

Dear all,

As of yesterday there's a new section in the documentation where we will 
publish information about the new releases for probe firmware and 
important UI/API changes: https://atlas.ripe.net/docs/releases/


With this, we plan to stop producing the RSS feed about "technical 
updates" - which had the same information until now. There's some use of 
this still, presumably from RSS readers that have been configured some 
time ago and fetch the feed ever since. (The top 5 clients accessing 
this correspond to 70% of the queries to this URL.)


In theory we could retrofit the creation of the RSS feed into the new 
way of publishing the release news, but we would only consider this if 
there's a strong enough demand for it. So please let us know if you have 
reasons to insist on keeping the feed alive!


Thank you,
Robert Kisteleki
RIPE Atlas


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Issue to create measurement on the webpage

[Robert Kisteleki]

On 2022-12-07 00:14, Alexis MARCOU wrote:

Hi All,

I'm trying to create a measure on the website : 
https://atlas.ripe.net/measurements/form/ 



But, when I click on the "Ping", a query is sent, but the interface is 
not updated with the parameters received ...


Am I the only one with this issue ?

Alexis


Hi,

I'm afraid "it's only you" :-) To my knowledge we haven't received other 
reports, and the interface works as expected for me.


(Please try a hard reload? Apple-R or shift-ctrl-R or such in your browser.)

If the issue persists, please reach out to us at at...@ripe.net 
(ticketised) with as much detail as you can - OS, browser, what you see 
on the browser console.


Thank you,
Robert



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Format change for index pages of RIS MRT data files and RIPE Atlas daily dataset dumps

[Robert Kisteleki]

Dear RIS and RIPE Atlas users,

On Wednesday, 30 November 2022 we are planning to migrate the web server 
at <https://data.ris.ripe.net/> as well as <https://data-store.ripe.net> 
from Apache httpd to nginx.


This will affect how the file listings in HTML index pages are 
generated, including the following:

* RIS MRT data files (updates and dumps)
* RIPE Atlas daily dataset dumps

If you rely on the format of these pages, you need to update your 
software. Before we do the upgrade, you could check the new format of 
the index pages at:


 https://test-data.ris.ripe.net/rrc00/

We would like to ask you to avoid relying on the specific format of the 
index.html files.


If you are fetching BGP dump files, please use the REST API that is 
described at:


 https://ris.ripe.net/docs/40_Prototypes/15_per_rrc_dumps.html

You might also consider using per-peer dump files instead:

 https://ris.ripe.net/docs/40_Prototypes/10_per_peer_dumps.html

We want to encourage you to provide your feedback on the API, so that we 
could improve it and build a similar API for the update files.


If you are fetching update files, please use direct URLs to files as 
described at:


 https://ris.ripe.net/docs/20_raw_data_mrt.html#name-and-location


Kind regards,

Oleg Muravskiy, Robert Kisteleki
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Can't register software probe

[Robert Kisteleki]

Hi,

This is strange as this should work (I just tried with a newly generated 
random key).


Are you sure you're pasting the public part? ;-) Together with the 
prefix "ssh-rsa WHATEVERBITS"? If so, please send me the (again, public) 
key and I'll check what's wrong with it.


Cheers,
Robert


On 2022-11-15 10:32, roberto.lucign...@caleidos.it wrote:

Hi all,

I download the probe software image 
from:https://github.com/Jamesits/docker-ripe-atlas 



Changed the RSA key in /var/atlas-probe/etc generating a new one with: 
ssh-keygen -t rsa -b 4096


Trying to register the software probe 
at:https://atlas.ripe.net/apply/swprobe/ 
I get the following error:


  * The public key does not match our regular expression for this key type

I tried generating other keys, but I always get the same error.

Regards,

Roberto

--

This message contains confidential information and is intended only for 
the individual named. If you are not the named addressee, you should not 
disseminate, distribute or copy this e-mail. Please notify the sender 
immediately by e-mail if you have received this message by mistake and 
delete this message from your system. E-mail transmission cannot be 
guaranteed to be secure or error-free, because information could be 
intercepted, corrupted, lost, destroyed, arrive late or incomplete, or 
contain viruses. The sender therefore does not accept liability for any 
errors or omissions in the contents of this message that arise as a 
result of e-mail transmission.





--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] placing a probe into wireless networks (not terrestrial)

[Robert Kisteleki]

On 2022-10-28 10:04, Kurt Kayser wrote:

I would be offering support to try to establish such a probe and help to 
setup a project defition.


Any thoughts or feedback to this idea?


In addition to what's already been said: it's useful for the whole 
community to increase the diversity of the network. So, the more 
networks we can have probes in, the merrier!


While doing so I find it highly useful to add proper meta-data as well 
(ie. probe tags) to aid people understanding what they observe. For 
example it is confusing to see high RTTs for a probe, unless it's 
documented that it's on a (non-LEO) satellite link for example.


Side-note: at the moment we have 31 probes connected on SPACEX-STARLINK 
(AS14593), see 
https://atlas.ripe.net/results/maps/network-coverage/?filter=AS14593


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Shared probes not shown anymore in "my atlas"

[Robert Kisteleki]

On 2022-10-19 01:09, Dave wrote:

Hi,

In the past the probes that had shared management were shown below the 
owned probes. This is not the case anymore. Is this intentional? Can it 
be restored?


Thanks,
Dave


Hi,

I'll reach out to you for details privately.

Cheers,
Robert


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Tagging probes which are using the ISP's DNS server?

[Robert Kisteleki]

I believe it would be informative if we assembled some statistics about 
the current state, ie. how many probes would get what kind of tags if we 
applied them today. Perhaps Max already has some up his sleeve?


Here's a quick-and-dirty answer to this: what the tags could look like 
if we applied them. This is about the ~12K connected probes, which 
collectively have ~23K resolver entries. Note: in this experiment the 
"more specific tags win", i.e. if quad1 is applied, then outside-asn tag 
is not.


6718 system-resolv-v4-private
4502 system-resolv-v4-inside-asn
2834 system-resolv-v4-google
1247 system-resolv-v4-outside-asn
1020 system-resolv-v4-loopback
 933 system-resolv-v4-quad1
 338 system-resolv-v4-quad9
  31 system-resolv-v4-1-24

1745 system-resolv-v6-inside-asn
 968 system-resolv-v6-ula
 560 system-resolv-v6-loopback
 473 system-resolv-v6-quad9
 461 system-resolv-v6-google
 335 system-resolv-v6-outside-asn
 319 system-resolv-v6-linklocal
 187 system-resolv-v6-quad1

Cheers,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Revive an abandoned probe?

[Robert Kisteleki]

On 2022-10-06 21:11, Ernst J. Oud wrote:

Hi,

As a result of my experiments some three moments ago I have an abandoned probe 
(OpenWRT). If I power it again will it start working again or do I need te 
re-apply for a new probe?

Regards,

Ernst J. Oud


Met vriendelijke groet,

Ernst J. Oud


Hello,

If the probe ever existed then it should be able to reconnect. Just turn 
it on, no other action is needed.


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Tagging probes which are using the ISP's DNS server?

[Robert Kisteleki]

Inside-AS DNS
Outside-AS DNS
RFC1918 DNS 


We probably need a "localhost DNS" too

I would still argue that the "big" public resolvers warrant their own tags.


Apart from that, should there be separate tags for RFC1918
and ULA?


Paraphrasing a quote from may decades ago: "probe DNS tagging is like 
math - a simple idea but it can get complicated" :-)


We could separate IPv4 and IPv6 tags in this context. Which is clearer 
but also more complicated to process. Still, for each af, probes could 
get multiple of these tags.


I believe it would be informative if we assembled some statistics about 
the current state, ie. how many probes would get what kind of tags if we 
applied them today. Perhaps Max already has some up his sleeve?


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Tagging probes which are using the ISP's DNS server?

[Robert Kisteleki]

Hello,

This seems to be an interesting question.

We can certainly apply some (system) tags for probes that have the 
popular resolvers 8.8.8.8, 9.9.9.9 and so on in the resolver 
configuration. This would allow users like you to easily filter for, or 
filter out, probes that do this.


One complication is that in many cases probes (an by extension, the 
users) have such a public resolver *in addition to* whatever else they 
use - which complicates the semantics of what resolver was actually 
used. But I guess one can accept that as a fact and still consider the 
feature to be useful.


As an extension, we can, if that's deemed useful, tag other resolvers, 
along the lines of:

* resolvers on private IPs (ie. on-net in the home?)
* mixed private-and-quadX
* mixed private-and-public

If we go this far, a probe could have multiple tags, eg. uses- + 
uses-private + mixed-private-and-quad. This may be overdoing it...


We'd be curious about what you think.

Regards,
Robert


On 2022-10-06 03:38, Max Grobecker wrote:

Hi,

a few days ago I wanted to debug a name resolution problem of one of our 
domains.
For this reason, I wanted to test if probes inside a specific ASN are 
having difficulties to resolve a specific name (because only customers 
of this ISP were complaining).
This lead to very mixed results, mostly because some of the selected 
probes did queries to a public DNS service like Google, Quad9 and so on.

The problem existed only with the provider's DNS servers for some reason.


It did take some time to make a script which tried to filter out these 
probes, so I wondered if anyone else had the same use-case and problem.
Is there a way to automatically tag probes, which are (seemingly) using 
the ISP's own DNS servers, or, at least, not a well-known public service?
This could be done maybe by querying a special DNS name which returns 
the IP address from where the query was received (like 
"whoami.akamai.net").
By comparing the ASN of the probe and the ASN of the IP address returned 
by the DNS query, one could determine, if the ISP's servers are used.
This would also be true for people running their own recursor, but this 
could be filtered as well very easy.
If an ISP is using multiple ASN, this could be a problem. Maybe there's 
an easy solution for this as well.


Probes which pass this test, could then be tagged with 
"DNS-using-ISP-server" or something like that and explicitly be selected 
for specific DNS resolution tests.



Greetings,
  Max



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] V5 probes

[Robert Kisteleki]

On 2022-10-06 09:29, Daniel AJ Sokolov wrote:
Out of curiosity: How many active V1 probes are still out there? How 
many were distributed?


BR
Daniel AJ



The maximum number of v1 probes ever alive at the same time was about 
1000 (in 2012). We made about 1500 of these.
The maximum number of v2 probes ever alive at the same time was about 
1800 (i 2013). We made about 3000 of these.


(v1 and v2 are almost the same device; both have 16MB flash, v1 has 8MB 
RAM, v2 has 16MB RAM)


Today we have about 440 v1 and 690 v2 connected. These have proven to be 
way more stable than we anticipated; as you can see many have been 
running for over 10 years now!


(We will publish graphs to illustrate this, please stand by :-)

Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] V5 probes

[Robert Kisteleki]

On 2022-10-03 12:37, Simon Brandt via ripe-atlas wrote:

Hi Robert,

i noticed that v1 probes are stuck with Firmware Version 4790. Does that 
mean, they are missing crucial functionalities of the later firmwares?


BR,
Simon


Indeed v1-v2 probes have not received new functionality since then. 
Therefore they don't do more modern measurements or options we added 
since (essentially because they don't have enough capacity to handle 
more code). Their numbers are diminishing though, and for the most part 
our infrastructure selects other probes if a measurement depends on such 
new, "crucial" features.


Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] V5 probes

[Robert Kisteleki]

Deal All,

Let me provide some information about the v5 probes, as it seems some of 
you are interested in the details :-)


The technical bits: the probes are derived from Turris Mox, "simplified" 
to match our need (e.g. no WiFi). The specs include: Marvell Armada 3720 
CPU, 4GB eMMC, 512MB RAM, 1Gb ethernet, microUSB power.


A few of these are up and running already, hosted by testers (incl. some 
staff). We are gearing up to start distributing them via the usual 
channels (applications, ambassadors, sponsors, ...)


We do not (yet) recommend "upgrading" form earlier probes, as long as 
those still work. Of course we're always interested in new 
locations/networks to cover, so feel free to propose that! :-)


Regards,
Robert Kisteleki
for the RIPE Atlas team


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q4 2022

[Robert Kisteleki]

Dear colleagues,

The quarterly plans for RIPE Atlas can now be found at:
https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas

In this quarter, we will start shipping the v5 RIPE Atlas hardware 
probes to hosts, ambassadors and sponsors as well as continue working on 
other items proposed by the community.


We hope that by providing these plans, we can better inform the 
community of the ongoing developments on RIPE Atlas and get feedback on 
our work.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
Research and Development Manager
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q3 2022

[Robert Kisteleki]

Dear colleagues,

The quarterly plans for RIPE Atlas can now be found at:
https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas

In this quarter, we will continue with the development of the v5 RIPE 
Atlas hardware probes among other items as well as start working on two 
proposals to implement general availability of HTTP measurements and 
restricting the ability to schedule "non-public measurements".


The goal of our quarterly planning is to communicate the work we’re 
doing for the coming months, get your input and start a discussion on 
other developments we should work on.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
Research and Development Manager
RIPE NCC


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] DNS issues

[Robert Kisteleki]

Hi,

There is a short description about how these tags are applied at 
https://atlas.ripe.net/docs/probe-tags/ "DNS resolution" section.


Basically, the system looks at results of DNS measurements delivered by 
the probe and applies/removes tags based on how successful those were. I 
believe this is done daily, perhaps more often. We are reworking the 
documentation so I'll make a note to add more details on how this 
exactly work.


I'll reach out to you privately about what may be going on on your probe.

Regards,
Robert


On 2022-06-23 21:36, Phillip Remaker wrote:
I have installed a probe (/14226) at a rural fixed wireless location 
which has had intermittent issues from the carrier (AT). I got the tag 
about a DNS issue (Doesn't Resolve A, DNS Problem Suspected). Is it 
possible to get logs for when the DNS issue happened so I can take it up 
with the carrier?


Also, when does the DNS problem tage clear?

Thanks!




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] RIPE Atlas down?

[Robert Kisteleki]

Hello,

The data back-end behind RIPE Atlas, RIS and RIPEstat experienced issues 
yesterday evening; some of these are still affecting the data processing.


In particular RIPE Atlas data is experiencing result delays since around 
03:30 (CEST). The system is catching up now, but it hasn't recovered 
fully yet.


https://status.ripe.net/ has an entry about the generic problem and 
we'll put up a specific entry about this specific delay.


We apologise for the inconvenience,
Robert


On 2022-06-22 11:34, Ernst J. Oud wrote:

Hi,

I noticed that of the probes I checked here in The Netherlands, all logging 
stops at around 03:00 today (June, 22nd).

Checked a probe in Germany, same thing but stopped a couple of hours later it 
seems.

Is there some problem?

Regards,

Ernst J. Oud



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Looking for a hardware probe

[Robert Kisteleki]

Can the probe ambassador page be updated to display the probe version 
handed out next to each probe?


Thanks,
Hank


Yes, that sounds useful and it's certainly something that we can do!

Regards,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] RIPE-Atlas measurements showed "No recent report available"

[Robert Kisteleki]

On 2022-05-24 21:21, Tingting Lu wrote:

Hi all,

I am using RIPE-Atlas to generate some ping measurements for my testing 
ip recently. It worked very well until yesterday. Starting from May 
23rd, most of my measurements only have results for a few probes. Most 
probes showed "No recent report available". Does anyone have the same 
problem or know what's going on there?


--
Regards,
Tingting Lu


Hello,

I can confirm this; the data storage backend had hardware problems 
recently and it is still processing its backlog. No data is lost but 
some of it is not available in real-time. The result streaming interface 
is not affected by this issue.


We put out a status update to https://status.ripe.net/ which is 
admittedly new, not everyone is aware yet.


We apologise for the inconvenience!
Robert Kisteleki


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Overuse of software probes

[Robert Kisteleki]

Hello All,

I think we can separate out two issues here:

1. I believe we should keep incentivising (for the lack of a better 
word) "benevolent" probe hosts - ie. if someone would like to host a 
probe, either hw or sw, then they should be able to do so and enjoy the 
benefits. Having said this, with the hw probe distribution we prioritise 
yet-uncovered networks. This is much harder to do with sw probes...


2. We should dis-incentivise "farming", if that happens. The issue is 
that we can only recognise this once it happens. Coming up with hard 
policies on how to regulate this in advance is not easy, but one can 
come up with some guidelines on how to recognise such cases and what to 
do with them when they are found.



There's of course a benefit in having multiple probes in a network, and 
presumably the bigger said network is the more probes it could have. 
Surely constructs like CGNATs make all kinds of corner cases...



I'm very happy to see proposals on how to improve the built-in probe 
selection process. We'll evaluate these and of course implement ones 
that make sense within the system. At the end of the day there will 
always be cases that can only be done "manually", ie. the user selecting 
particular probes by whatever criteria they want to use.


Cheers,
Robert


On 2022-03-31 20:43, Andreas Härpfer wrote:

Of course I am wildly guessing here, but the 15 probes in
Hostinger Sao Paulo actually look a bit fishy to me.  Consecutive
probe numbers, all created at roughly the same time, all in the
same v4 /24 and v6 /64.  To me this looks like an "Atlas credit
mining farm" … so more a mis-use than an overuse.

The 4 probes in Johannesburg are different.  Same ASN but at
least different /24s and different ages, so likely also different
owners.

Just my 2¢

Cheers
-Andi




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] RIPE Atlas Quarterly Planning Q2 2022

[Robert Kisteleki]

Dear colleagues,

We are now publishing our quarterly planning for RIPE Atlas. This is 
available at:

https://www.ripe.net/support/documentation/quarterly-planning/ripe-atlas

The goals of publishing our planning are to be transparent about the 
work we will be carrying out in the coming months, to get your input on 
those plans, and hear what you would like to see the RIPE NCC work on 
going forward.


We will provide status updates as the work progresses and add input from 
the community to the pages. We will also archive previous plans so you 
can track how we progress with our work.


If you have any input on our plans or want to let us know what you would 
like to see us work on, please let us know.


Kind regards,

Robert Kisteleki
Research and Development Manager
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Location of probe controllers

[Robert Kisteleki]

Hello,

On 2022-01-28 09:55, Malte Appel wrote:

My guesses from the DNS names are based on IATA airport codes:

ctr-ams02.atlas.ripe.net (Amsterdam, NL)
ctr-ams03.atlas.ripe.net (Amsterdam, NL)
ctr-ewr01.atlas.ripe.net (Newark, NJ, US)
ctr-fnc01.atlas.ripe.net (Funchal, PT)
ctr-nue13.atlas.ripe.net (Nürnberg, DE)
ctr-nue17.atlas.ripe.net (Nürnberg, DE)
ctr-sin02.atlas.ripe.net (Singapore, SG)

Can anyone confirm or correct these locations?


It is mostly correct: fnc is Fremont, US west coast.

Cheers,
Robert

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas