Re: [atlas] Changes to RIPE Atlas API auth status codes on 2 Oct

[Chris Amin]

This change has now been made, so some endpoints will return a 401 
status code instead of 403.


As a reminder, you can keep the previous behaviour for the rest of this 
year by including the following HTTP header in your requests:


X-Compat: auth-2023

or alternatively, thanks to my generalized calendar confusion:

X-Compat: auth-2022

This migration measure will be dropped some time in January (of whatever 
year comes after this one).


Regards,
Chris


On 19/09/2023 10:38, Chris Amin wrote:

Dear colleagues,

Currently the RIPE Atlas REST API (https://atlas.ripe.net/api/v2/) 
returns a 403 Forbidden status code in two cases:


* When a request requires authentication but the user has not provided 
any credentials, or has provided incorrect credentials
* When a user has authenticated correctly, but they or their API key 
lacks the permissions needed for a particular request


Distinguishing between these two cases is important because in the first 
case the client can potentially get access by authenticating, and in the 
second case there is no point in retrying authentication with the same 
credentials.


In order to enable this distinction, and to generally conform to web 
standards and best practices, on Monday, 2nd October we will change the 
REST API so that a completely unauthenticated request will receive a 
response with a 401 Unauthorized status code. The 403 Forbidden status 
code will still be returned for users and API keys that are 
authenticated but lack the necessary permissions for the request.


As a temporary migration measure, the API will keep the same behaviour 
(always return 403) if either:


* The "Referer" header contains "RIPE Atlas Tools" and a version string 
<= 3.1.1, or

* An "X-Compat" header is set and contains the string "auth-2022"

This temporary measure is guaranteed to work for the rest of 2022, after 
which it will be removed and the API will always make the 401/403 
distinction.


Kind regards,
Chris Amin
RIPE Atlas team



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Atlas fully down..

[Chris Amin]

On 20/09/2023 12:57, Ernst J. Oud wrote:


I checked my code that calls Magellan. It does not add the “system-ipv4-works“ 
tag to the request for probes in measurements.

Does Magellan add that tag automatically?

Hi Ernst,

The system-ipv4-works (and system-ipv6-works) tags are specified in the 
configuration in the ripe-atlas-tools settings file. If you run:


ripe-atlas configure --editor

you can find a tags block with various default tag sets to use for 
different measurement types.


*However*,

as a workaround we have re-populated the system-ipv4-works and 
system-ipv6-works tags, so that the default behaviour of Magellan should 
be sane once again. This should allow scheduling of new measurements and 
streaming the results.


Note: The probe sets for these tags are currently just copied from the 
system-ipv4-stable-1d and system-ipv6-stable-1d tags, respectively, 
which were not affected by this issue because they use a different data 
backend. The actual probe sets are therefore not exactly the same, but 
they are functionally very similar if your goal is "restrict the 
participating probes to ones that are likely to work".


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Changes to RIPE Atlas API auth status codes on 2 Oct

[Chris Amin]

On 19/09/2023 11:11, Gert Doering wrote:


This temporary measure is guaranteed to work for the rest of 2022,


So which iteration of 2022 would that be?


Thanks Gert. This was of course deliberate to make see whether people 
were paying attention.


The temporary measure is *also* guaranteed to work for the rest of 2023, 
and the X-Compat header may contain either "auth-2022" or "auth-2023" to 
maintain the old behaviour until the end of this year.



Chris

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Changes to RIPE Atlas API auth status codes on 2 Oct

[Chris Amin]

On 19/09/2023 10:38, Chris Amin wrote:

As a temporary migration measure, the API will keep the same behaviour 
(always return 403) if either:


* The "Referer" header contains "RIPE Atlas Tools" and a version string 
<= 3.1.1, or


Apologies, this should refer to the "User-Agent" header and not the 
"Referer" header.



* An "X-Compat" header is set and contains the string "auth-2022"

This temporary measure is guaranteed to work for the rest of 2022, after 
which it will be removed and the API will always make the 401/403 
distinction.


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Changes to RIPE Atlas API auth status codes on 2 Oct

[Chris Amin]

Dear colleagues,

Currently the RIPE Atlas REST API (https://atlas.ripe.net/api/v2/) 
returns a 403 Forbidden status code in two cases:


* When a request requires authentication but the user has not provided 
any credentials, or has provided incorrect credentials
* When a user has authenticated correctly, but they or their API key 
lacks the permissions needed for a particular request


Distinguishing between these two cases is important because in the first 
case the client can potentially get access by authenticating, and in the 
second case there is no point in retrying authentication with the same 
credentials.


In order to enable this distinction, and to generally conform to web 
standards and best practices, on Monday, 2nd October we will change the 
REST API so that a completely unauthenticated request will receive a 
response with a 401 Unauthorized status code. The 403 Forbidden status 
code will still be returned for users and API keys that are 
authenticated but lack the necessary permissions for the request.


As a temporary migration measure, the API will keep the same behaviour 
(always return 403) if either:


* The "Referer" header contains "RIPE Atlas Tools" and a version string 
<= 3.1.1, or

* An "X-Compat" header is set and contains the string "auth-2022"

This temporary measure is guaranteed to work for the rest of 2022, after 
which it will be removed and the API will always make the 401/403 
distinction.


Kind regards,
Chris Amin
RIPE Atlas team

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] System tag

[Chris Amin]

Just to add to this, the full explanation of the "IPv#-Stable-#d" tags 
can be found here:


https://atlas.ripe.net/docs/getting-started/probe-tags.html#stability

On 04/08/2023 10:48, Michel Stam wrote:

Hello Ernst,

 From the top of my head, the system-generated tags are done based on 
internal measurements the probe does.


CTR-HEL09 has had a reboot about 3 days ago, and your probe seems to 
connect to CTR-HEL09 mostly, so that would explain that disconnect. I am 
not sure the tagging is related to the connection with the controller 
being lost, it’s more likely that one of the internal measurements did 
not get the expected response and concluded IPv4 wasn’t stable enough. 
It seems to have resolved, just now the probe showed as being stable again.


Hope this helps. Let me know if you need me to dig a bit further.

Regards,

Michel


On 3 Aug 2023, at 22:43, Ernst J. Oud  wrote:

Hi,

Further to my question; checked my logs and on at the date/time the 
connection history of this probe is reported as “disconnected” 
(2023-08-01 06:41:54 UTC) there was no problem whatsoever on my 
internet connection.


So why was the probe disconnected? I see that status for my probes 
quite a lot, is it the controller (ctr-hel09) that needs to go get a 
beer or what?


Regards,

Ernst J. Oud


On 3 Aug 2023, at 22:33, Ernst J. Oud  wrote:

I noticed that the system tag “IPv4 stable 1d” is gone from my probe at:

https://atlas.ripe.net/probes/1005104/ 



I use that tag as a criteria to select active probes for 
measurements. But I noticed that my own probe was not included in my 
measurements.


Which is weird since my probe has been online for 2.5 days (normally 
it is more stable, but my ISP had a small downtime).


So my question is what triggers this tag? The tag is self explanatory 
but what does “stable” mean?


Regards,

Ernst J. Oud

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas





--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Magellan

[Chris Amin]

Hi Ernst,

There is no way to know for absolute certain if a probe will be able to 
carry out a measurement and deliver results. There are a number of 
factors that prevent a measurement being scheduled on a probe (e.g. it 
is too busy, it is running an old firmware that doesn't support your 
measurement options). There are also factors that prevent it delivering 
results even if it is scheduled (e.g. it disconnects in the meanwhile, 
it has an issue with its hardware).


Many of these issues can be prevented most of the time by requiring one 
of the so-called stability tags when selecting your probes. These are 
tags which mark a probe as recently performing mostly successful 
measurements e.g. for an IPv4 measurement you could do:


$ ripe-atlas probe-search --asnv4 15435 --status 1 --format csv --field 
id --limit 50 --no-header --tag system-ipv4-stable-1d


to get a list of probes that have generally had good success with IPv4 
measurements in the last day.


In the case you gave, probe 2565 would have been excluded from this list 
because it has not generally been delivering results for a long while, 
which you can see here: https://atlas.ripe.net/probes/2565/#tab-builtins


For more info on the various probe tags you can look at the docs here: 
https://atlas.ripe.net/docs/getting-started/probe-tags.html#system-tags


I hope this helped, please let me know if you have any more questions.

Regards,
Chris

On 25/06/2023 17:14, Ernst J. Oud wrote:


Perhaps someone can help me.

I use Magellan to create a measurement using connected probes on AS15435.

First I search for probes using Magellan and then I supply the ID’s returned of 
those probes reported as “connected” when creating the measurement.

I don’t want to wait for the stream to time-out so I use the “stream-limit” 
parameter, equal to the number of supplied connected probes. So when that 
number of connected probes have supplied data the streaming is complete.

However, it appears that probes are reported as connected by Magellan but do 
not supply data when specified for a measurement, for instance probe 2565. That 
probe is Reported by Magellan as connected but when I use it in a measurement 
it does not supply data. Thus streaming will only time-out and does not stop 
not when spevified connected probes have supplied data. Apparantly “connected” 
does not imply “can be used in a measurement”.

How can I supply probe ID’s in a “create measurement” command using Magellan 
knowing for sure that these probes will supply data?

Regards,

Ernst J. Oud


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] New ripe-atlas-tools and ripe-atlas-cousteau

[Chris Amin]

Dear colleagues,

I'm pleased to announce the release of version 3.1.0 of the RIPE Atlas 
command-line tools, which allow you to use RIPE Atlas from the comfort 
of your shell:


https://github.com/RIPE-NCC/ripe-atlas-tools/

The changes for this release include:

* Improved probe-search and measurement-search, including 
machine-readable "csv" and "tab" output formats

* --stream-timeout and --stream-limit added to measure command

In addition, this release depends on the new ripe-atlas-cousteau==2.0.0, 
which uses the new WebSocket-based RIPE Atlas streaming API.


Regards,
Chris Amin
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Magellan report on built-in measurement 1

[Chris Amin]

Hi Ernst,

Sorry for the late response on this...

On 14/12/2022 12:05, Ernst J. Oud wrote:

Asked this question before but that post did not make it to the mailing list:

Why can’t Magellan report on built-in measurement 1 (first hop)? It always 
shows “No packets found”.



There seems to be a problem with the renderer code specifically when 
trying to render results from measurements 1 and 2. I'll look into this 
and let you know when there's a fix ready.


Cheers,
Chris

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] New RIPE Atlas stream release

[Chris Amin]

Dear colleagues,

The new version of the RIPE Atlas streaming API is now available.

As mentioned in the previous e-mail, the API can now be accessed using 
either plain WebSockets or plain GET requests.


Socket.IO support is being maintained for now, so you don't have to 
immediately change your installations, but the new interfaces should be 
preferred wherever possible.


The documentation can be found at:

https://atlas.ripe.net/docs/apis/streaming-api/

Kind regards,
Chris Amin
RIPE NCC

On 30/11/2022 10:31, Chris Amin wrote:

In around three weeks we will be releasing a new version of the RIPE 
Atlas stream (https://atlas-stream.ripe.net) with an API based on 
WebSockets and plain HTTP requests.


The current Socket.IO interface will be maintained for backwards 
compatibility for the near future, but some subscription parameters will 
be dropped due to extremely low or no usage. These are:


* Historic playback (startTime, endTime) — note that this does not 
include sendBacklog, which will still work as now
* Arbitrary server-side filtering of fields (equalsTo, greaterThan, 
lessThan) — note that filtering by "prb", "msm", "type" etc will work as 
now


We can see from the service logs that each of these parameters has 
either not been used at all or used by a single IP address in the past 
month, so the impact should be minimal. However, if you happen to be 
using one of these parameters then please get in touch with me.


Aside from these changes, existing use cases should continue to work as 
they do now, and we will be monitoring closely for any issues.


There will be more information on the new API, including complete 
documentation, closer to the time.


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Non-public probes?

[Chris Amin]

You're right that the probe page is not shown, however the public 
details are available at https://atlas.ripe.net/api/v2/probes/1003690


The important point there is that the system has not granted the 
"system: IPv4 Works" tag, so is not available for IPv4 measurements. In 
general the scheduler doesn't know/care about a probe being marked as 
public or not.


Can you confirm whether the same measurement request works for IPv6 
(af=6) measurements? Note that if you schedule measurements together 
they must all be IPv6 in that case.


In the meanwhile, we'll think about including the non-public probes 
(albeit with their somewhat restricted details) as part of a redesign of 
the probe page coming up soon.


Cheers,
Chris

On 15/12/2022 13:12, Ernst J. Oud wrote:

But what is going on then with:

https://atlas.ripe.net/frames/probes/1003690 



This probe is within AS15435 and when I list all probes in that AS it is 
shown, but I cannot add it to a measurement, it is rejected.


Using a curl request to:

stat.ripe.net/data/atlas-probes/data.json?resouce=15435

shows this probe, with a tag “is_public” : false

and I cannot access this probe’s info via the link above, I cannot add 
it to a measurement and I cannot access any data it collects.


How does this rhyme to that non-public “probes are therefore 
contributing very nearly

as much to the network as everybody else”?




--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Non-public probes?

[Chris Amin]

Hi Ernst,


Just recently I realized that a probe can be labeled as “non-public”. Why is 
this allowed? Atlas is a community effort. Why are probe “owners” allowed to 
have benefits from using Atlas without sharing data?


Probes marked as as non-public have some of their details hidden, 
notably it is not easy to find their exact IP address. However, they are 
still participating in all available built-in measurements and are 
available for scheduling for user-defined measurements, and all data are 
available publicly for public measurements (but see the recent proposal 
regarding non-public *measurements* at 
https://labs.ripe.net/author/kistel/five-proposals-for-a-better-ripe-atlas/ 
.


The hosts of "non-public" probes are therefore contributing very nearly 
as much to the network as everybody else. :-)


Regards,
Chris Amin
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Upcoming RIPE Atlas stream release in December

[Chris Amin]

Dear colleagues,

In around three weeks we will be releasing a new version of the RIPE 
Atlas stream (https://atlas-stream.ripe.net) with an API based on 
WebSockets and plain HTTP requests.


The current Socket.IO interface will be maintained for backwards 
compatibility for the near future, but some subscription parameters will 
be dropped due to extremely low or no usage. These are:


* Historic playback (startTime, endTime) — note that this does not 
include sendBacklog, which will still work as now
* Arbitrary server-side filtering of fields (equalsTo, greaterThan, 
lessThan) — note that filtering by "prb", "msm", "type" etc will work as now


We can see from the service logs that each of these parameters has 
either not been used at all or used by a single IP address in the past 
month, so the impact should be minimal. However, if you happen to be 
using one of these parameters then please get in touch with me.


Aside from these changes, existing use cases should continue to work as 
they do now, and we will be monitoring closely for any issues.


There will be more information on the new API, including complete 
documentation, closer to the time.


Kind regards,
Chris Amin
RIPE NCC

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Magellan

[Chris Amin]

Hi Ernst,

I can confirm these two bugs in the behaviour of the ripe-atlas command 
when waiting for the stream. In the first case, the command is 
hopelessly waiting for (numProbes + 1) results, which of course never 
come. In this second case, the command gives up on reading the stream 
but doesn't disconnect so the process stays open.


The fixes for both of these issues are available in v3.0.3 of the 
ripe-atlas-tools python package, which has just been released. Please 
give this a try when you get the chance and let me know if there are any 
outstanding issues.


Thanks for your mails and apologies for the inconvenience.

Regards,
Chris Amin
RIPE NCC

On 16/11/2022 19:24, Ernst J. Oud wrote:

L.S.

In the meantime I have found that there is a hardcoded 5 minute timeout for 
streams to end.

However, after this timeout the message “Disconnected from stream” appears and 
the ripe-atlas process appears to “hang”, a ctrl-c appears to be the only 
option, python then interrupts within threading.py and client.py. What is it 
doing, what is it waiting for?

Any solution?

Regards,

Ernst J. Oud


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Breaking API change?

[Chris Amin]

Hi Ray,

On 24/10/2022 14:05, Ray Bellis wrote:

I'll give that a test later and let you know.  I did manage to work 
around the new structure on the live site.


BTW, I also found (by accident) that in order to get the probe ID I had 
to use field=probe_id, not prb_id.  Is that intended
Yes, the field names actually use the ripe-atlas-sagan parser 
conventions rather than the standard measurement result keys, so 
"probe_id" rather than "prb_id".


One note: the "fields=" parameter is not actually a publicly documented 
feature of the API, which is the main reason why it was changed in a 
backwards-incompatible way. However we have no immediate plans to drop 
support for it, and I'll let you know if we're considering it. 
"use_keys" is also not currently documented, but we will upgrade that to 
be fully documented/supported in the near future.


Cheers,
Chris

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Breaking API change?

[Chris Amin]

Hi Ray,

This is indeed a breaking change, although the new behaviour is arguably 
more logical. The missing link here is the "use_keys" parameter, which 
was always supported but used to default to true when "fields" was 
present, and false otherwise. It now defaults to false unless explicitly 
specified.


You can get the previous behaviour with the following URL:

<${apiUrl}/measurements/${m}/latest/?fields=responses.0.response_time,responses.0.abuf.answers.0.data.0=1800_keys=true>

Does this work for you?

Kind regards,
Chris Amin
RIPE NCC

On 22/10/2022 15:37, Ray Bellis wrote:


ISC's Root System Atlas visualiser used to use this API call to access 
the built-in root system measurements:


<${apiUrl}/measurements/${m}/latest/?fields=responses.0.response_time,responses.0.abuf.answers.0.data.0=1800>

where ${m} is the measurement number.

It used to return this object:

{
   probe_id1: [ [ latency, site1 ], ... ]
   probe_id2: [ [ latency, site1 ], ... ]
   ...
}

It now returns this array instead:

[
   [probe_id1, latency, site],
   ...
   [probe_id1, latency, site],

   [probe_id2, latency, site],
   ...

]

and I had to explicitly request the probe_id field in order to get it.

Was this change intentional, a regression, or did I not use the API 
right in the first place?


cheers,

Ray





--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] API error

[Chris Amin]

Dear colleagues,

The issue with the latest and results endpoints not working for 
restricted built-in measurements has now been resolved. You should now 
be able to see results for public probes and probes hosted by you on 
these endpoints.


Apologies again for the inconvenience.

Regards,
Chris

On 26/09/2022 15:58, Ernst J. Oud wrote:


Hi,

Since a short while (1 or 2 days) I get an authorization error when 
downloading the data from the built-in first and second hop to 
ping.ripe.net.


Thus:

https://atlas.ripe.net/api/v2/measurements/1/results/?probe_ids=1004467=1664150400=1664236799=json


Gives:

{"error":{"detail":"Authentication credentials were not 
provided.","status":403,"title":"Forbidden","code":104}}

If I logon with my credentials via “My Atlas”, I get:

{"error":{"detail":"You do not have permission to perform this 
action.","status":403,"title":"Forbidden","code":104}}


All other tests let me download the data fine, logged in or not.

Same happens to download first and second hop data of other public probes.

Any clues?

Regards,

Ernst J. Oud



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] API error

[Chris Amin]

Hi Dimeji,

The fix for this issue is currently being tested, I will update the list 
when it's ready. There is no workaround for the issue, but you can 
expect it to be resolved this week.


Regards,
Chris

On 05/10/2022 06:32, Dimeji Fayomi via ripe-atlas wrote:

Hi,
I have also been experiencing this authorization error when I attempt to 
download first and second hop ping data of public probes.

I'd please appreciate any tips or pointers to dealing with this.

Thanks
Dimeji

--
Dimeji Fayomi



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] API error

[Chris Amin]

Hi Ernst,

Thank you for reporting this behaviour, it seems that there was a 
regression. We will take a look and update the list when there is progress.


Kind regards,
Chris Amin
RIPE NCC

On 26/09/2022 15:58, Ernst J. Oud wrote:


Hi,

Since a short while (1 or 2 days) I get an authorization error when 
downloading the data from the built-in first and second hop to 
ping.ripe.net.


Thus:

https://atlas.ripe.net/api/v2/measurements/1/results/?probe_ids=1004467=1664150400=1664236799=json


Gives:

{"error":{"detail":"Authentication credentials were not 
provided.","status":403,"title":"Forbidden","code":104}}

If I logon with my credentials via “My Atlas”, I get:

{"error":{"detail":"You do not have permission to perform this 
action.","status":403,"title":"Forbidden","code":104}}


All other tests let me download the data fine, logged in or not.

Same happens to download first and second hop data of other public probes.

Any clues?

Regards,

Ernst J. Oud



--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Updated RIPE Atlas Documentation

[Chris Amin]

The VM and V3 installation instructions have been added to the new docs 
site (and therefore the docs search functionality), and the old URLs 
still work. The references to "docs-old" on the anchors page have been 
removed. Redirects for many other pages have also been fixed, but if 
there are any others that you notice then please let me know.


Thanks again,
Chris

On 04/08/2022 12:33, Chris Amin wrote:

Hi Simon,

Sorry for this, and thanks for pointing it out. These particular pages 
should indeed have been moved to the new docs site. Additionally there 
is an issue with the new docs framework which stops redirects for old 
URLs from working properly.


This is being looked at now, I will respond to the list when it's fixed.

Kind regards,
Chris

On 04/08/2022 00:24, Simon Brandt via ripe-atlas wrote:
Alright, i just found out that the old documentation is still 
available, it has been renamed to "docs-old":

https://atlas.ripe.net/docs-old/anchor-installation-vm/

So maybe update all affected labs.ripe.net articles? I guess it's 
still better to refer to docs-old pages than "Document Not Found".


BR,
Simon



On 04.08.22 00:14, Simon Brandt via ripe-atlas wrote:

Hi,

a lot of previously published URLs are not working anymore: "Document 
Not Found". For example, links that have been posted in labs.ripe.net 
articles. Those articles are now kind of... incomplete. Are there any 
plans to fix this? Maybe mod_rewrite or something?


Here's an example: 
https://labs.ripe.net/author/alun_davies/announcing-ripe-atlas-vm-anchors/
The URL "installation instructions for VM anchors 
<https://atlas.ripe.net/docs/anchor-installation-vm/>" does not work 
anymore. Also: i wasn't able to find that article by using the search 
bar in the new documentation.


BR,
Simon


On 15.07.22 16:44, camin wrote:

Dear colleagues,

A new version of the RIPE Atlas documentation has been published. We 
have consolidated the text in places and improved the structure to 
make it easier to find content.


The new version can be found at:

https://atlas.ripe.net/docs/

Regards,
Chris Amin











--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Updated RIPE Atlas Documentation

[Chris Amin]

Hi Simon,

Sorry for this, and thanks for pointing it out. These particular pages 
should indeed have been moved to the new docs site. Additionally there 
is an issue with the new docs framework which stops redirects for old 
URLs from working properly.


This is being looked at now, I will respond to the list when it's fixed.

Kind regards,
Chris

On 04/08/2022 00:24, Simon Brandt via ripe-atlas wrote:
Alright, i just found out that the old documentation is still available, 
it has been renamed to "docs-old":

https://atlas.ripe.net/docs-old/anchor-installation-vm/

So maybe update all affected labs.ripe.net articles? I guess it's still 
better to refer to docs-old pages than "Document Not Found".


BR,
Simon



On 04.08.22 00:14, Simon Brandt via ripe-atlas wrote:

Hi,

a lot of previously published URLs are not working anymore: "Document 
Not Found". For example, links that have been posted in labs.ripe.net 
articles. Those articles are now kind of... incomplete. Are there any 
plans to fix this? Maybe mod_rewrite or something?


Here's an example: 
https://labs.ripe.net/author/alun_davies/announcing-ripe-atlas-vm-anchors/
The URL "installation instructions for VM anchors 
<https://atlas.ripe.net/docs/anchor-installation-vm/>" does not work 
anymore. Also: i wasn't able to find that article by using the search 
bar in the new documentation.


BR,
Simon


On 15.07.22 16:44, camin wrote:

Dear colleagues,

A new version of the RIPE Atlas documentation has been published. We 
have consolidated the text in places and improved the structure to 
make it easier to find content.


The new version can be found at:

https://atlas.ripe.net/docs/

Regards,
Chris Amin









--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] Discrepancy in documentation?

[Chris Amin]

Hi Stephane,


On 08/06/2022 16:58, Stephane Bortzmeyer wrote:


In the documentation
, various
HTTP methods are used (for instance DELETE to delete a measurement),
which is nice, but the examples ("Try it") always show the method GET.


You are right, this is incorrect currently on the "beta-docs" site. In 
fact, the Try it functionality doesn't work at all for  It will be fixed 
in the new "docs" site, which will be available fairly soon.


Regards,
Chris

--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] broken atlas streams

[Chris Amin]

Hi Teun,

I found a possible cause of the issue and have deployed a fix. Could you 
tell me if it (continues to) work?


Regards,
Chris

On 01/03/2022 22:35, Teun Vink wrote:

Hi Chris,

Thanks, we still see issues receiving measurements. I attached a 
stripped down version of the script we’ve been using for a number of 
years. The code basically sets up a stream and starts listening, 
printing all measurements it recieves. It has some threading left (we 
usually run this for a number of measurements). When starting this 
script and doing a tcpdump, I see nearly no data at all being received 
from the Atlas servers.


I hope this helps, let me know if you need any additional info.

Thanks!


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

Re: [atlas] broken atlas streams

[Chris Amin]

Hi Teun,

I know that my colleagues have been in contact with you but I'm writing 
here in case others are having the same issue.


There were issues with the service when you sent this original mail, but 
I can no longer detect them and I understand you are still having the 
issue. If that's the case could you please send me the cousteau code you 
are using?


Thanks,
Chris

On 23/02/2022 08:34, Teun Vink via ripe-atlas wrote:

Hi,

We’re seeing some issues with RIPE Atlas streams using the 
ripe.atlas.cousteau websocket library. Sockets seem to connect properly, 
but no data is received. tcpdumps show very little traffic. One of the 
already running receiver application (monitoring the exact same set of 
measurements) is receiving data, but any newly started daemon fails to 
receive data.


Is there an issue on the RIPE Atlas infrastructure?

Thanks,


--
ripe-atlas mailing list
ripe-atlas@ripe.net
https://lists.ripe.net/mailman/listinfo/ripe-atlas

[atlas] Scheduler adding extra probes for some measurements

[Chris Amin]

Dear colleagues,

We have fixed a bug in the RIPE Atlas scheduler which caused extra 
probes to be added to some ongoing measurements. This bug affected many 
users who submitted "removal" requests using the REST API from 
2021-03-23, and in some cases caused a large increase in the consumption 
of credits due to the extra generated measurement results.


This bug has now been fixed, and the affected accounts have been 
credited with approximately 125% of the erroneous overpayment.


My apologies for any inconvenience caused. Please let me know if you 
have any further questions or issues.


Kind regards,
Chris Amin
RIPE NCC

Re: [atlas] Rapidly decreasing credits

[Chris Amin]

Hi ms,

This was caused by a bug in the scheduler which we are currently fixing. 
I will update the list when the issue is resolved.


Regards,
Chris

On 27/04/2021 10:23, ms wrote:

Hi,

Recently, looks like since beginning of April, my credits started 
dropping exponentially. I'm only running a ping measurement at the moment.


Any changes to the credit system I missed?

Cheers,
  ms

Re: [atlas] atlas streaming data broken?

[Chris Amin]

Mistake, please ignore!

On 20/01/2021 14:25, Chris Amin wrote:



On 19/01/2021 15:28, Teun Vink wrote:

Hi,

RIPE Atlas API streams seem to have broken down today at ~12.05h CET. 
Is this a known problem?


Best regards,

Re: [atlas] atlas streaming data broken?

[Chris Amin]

On 19/01/2021 15:28, Teun Vink wrote:

Hi,

RIPE Atlas API streams seem to have broken down today at ~12.05h CET. Is 
this a known problem?


Best regards,

Re: [atlas] atlas streaming data broken?

[Chris Amin]

Hi Teun,

I will take a look into this and get back to you.

Chris

On 19/01/2021 16:32, Teun Vink wrote:

Hi Chris,

Thanks for the quick response. I don’t see any data coming in on any of 
our streams yet, even after restarting the collectors. So maybe there 
still is something broken there. You can check https://bit.org/network, 
which uses this data.


Best regards,

Re: [atlas] atlas streaming data broken?

[Chris Amin]

Hi Teun,

There were indeed issues with the stream for the past couple of hours. 
An upgrade of one of the backend servers caused an unexpected outage. 
Everything should be back to normal now, please let me know if there are 
any more issues.


Sorry for the inconvenience.

Regards,
Chris Amin
RIPE NCC

On 19/01/2021 15:28, Teun Vink wrote:

Hi,

RIPE Atlas API streams seem to have broken down today at ~12.05h CET. Is 
this a known problem?


Best regards,

[atlas] atlas-stream on HTTP

[Chris Amin]

This issue is now resolved, atlas-stream.ripe.net should work correctly 
again over plain HTTP.


On 01/09/2020 15:26, Chris Amin wrote:


On 01/09/2020 14:07, Teun Vink wrote:

Hi Robert,

atlas-stream.ripe.net seems to be broken on HTTP. Is that related to 
this change?


Hi Teun,

This is an unrelated issue that we are looking into. I'll let you know 
when the solution is ready.


Regards,

Chris

Re: [atlas] Disabling TLS 1.0 and 1.1 on RIPE Atlas anchors

[Chris Amin]

On 01/09/2020 14:07, Teun Vink wrote:

Hi Robert,

atlas-stream.ripe.net seems to be broken on HTTP. Is that related to 
this change?


Hi Teun,

This is an unrelated issue that we are looking into. I'll let you know 
when the solution is ready.


Regards,

Chris

Re: [atlas] status-check redirects to itself?

[Chris Amin]

Hi all,

As Robert mentioned, this change was unintentional.

The issue has now been fixed, so both the with and without slash 
versions of API endpoints work as before.


Apologies for the inconvenience.

Regards,

Chris

On 30/04/2020 10:25, Robert Kisteleki wrote:


On 2020-04-30 08:50, Benjamin Collet wrote:

Hi,

On Thu, Apr 30, 2020 at 08:25:26AM +0200, Stephane Bortzmeyer wrote:

The redirect is to a slightly different URL.

OK, thanks, I did not notice the detail. But it worked before (I use
it in a monitoring script), and the behavior changed somewhere around
29 april, 13:20 UTC.

Same behaviour here. The fix is easy enough but it would be nice to be
warned when API endpoints change, or did I miss something?

Hello,

We deployed some changes around the date Stephane mentioned. This was
not intended to affect the URL behaviour, and we're investigating why
this is happening. We'll get back to you soon.

In the meantime: api/xxx/?parameters (ie. with a slash at the end of the
URL before the potential parameters) works as intended so you can use
this form, at least for now.

Regards,
Robert

Re: [atlas] Probes were incorrectly not tagged with the stability tags

[Chris Amin]

On 21/02/2020 13:32, Chris Amin wrote:

> Over the past month there have been issues with the
> "system-ipv[45]-stable-{1,30,90}d" tags

Just for clarity, we do not yet tag probes as being stable over IPv5.

[atlas] Probes were incorrectly not tagged with the stability tags

[Chris Amin]

Dear colleagues,

Over the past month there have been issues with the
"system-ipv[45]-stable-{1,30,90}d" tags, which were not applied to many
of the probes that should have had them. This was caused by accidental
data format changes from a backend migration.

The issue should now be fixed, so probes will have the correct tags
applied. If you notice any persisting issues then please let us know!

Apologies for any inconvenience.

Kind regards,
Chris

Re: [atlas] Atlas API stream broken?

[Chris Amin]

Hi Teun,

Yes indeed, there were some backend issues between the times you stated.
However, I don't know about any issues from 13.30 on Friday. Do you
still see these issues? If so, could you please provide me with your
subscription parameters (in a private email if you prefer)?

Kind regards,
Chris Amin

On 07/02/2020 16:59, Teun Vink wrote:
> Hi,
> 
> Since ~13.30h CET this afternoon our Atlas data stream subscriptions
> (using the ripe.atlas.cousteau python library) stopped receiving
> updates.
> 
> Earlier today we (at 11.27h) observed some issues with the backend
> where we received redis errors, which were resolved at ~12.09h CET. I
> heard these were related to some firewall issues, could these problem
> be related?
> 
> Regards,
> 

[atlas] Probe stability tags

[Chris Amin]

Dear RIPE Atlas users,

Due to an error in our backend systems, the probe stability tags
("system: IPv4 Stable 30d" etc) had been gradually becoming stale over
the past three months, meaning that probes were not gaining or losing
these tags when they should.

This has now been fixed, so probes are once again tagged appropriately.
We have improved our internal monitoring so this kind of issue will be
noticed more quickly next time. Apologies for any inconvenience caused.

Kind regards,
Chris Amin
RIPE NCC

Re: [atlas] One-off measurements not terminating

[Chris Amin]

Hi Steve,

There was indeed a problem where measurements were not being
automatically updated with a "stopped" status. This should now be fixed,
but please let me know if you notice any lingering issues.

Can you confirm that the issue with manually DELETEing not having an
effect still persists? If so, can you give me an example measurement ID?

Thanks,
Chris Amin
RIPE NCC

On 30/12/2019 06:53, Steve Gibbard wrote:
> An update:   I was able to ‘delete' my stuck measurements via the API, so 
> they’re stopped now and I’m back up and running for the moment.
> 
> I also added an API command to my code to ‘delete’ measurements as soon as 
> the results have been picked up, which I hoped would make this fix 
> sustainable, but so far that doesn’t seem to be doing anything.  Perhaps a 
> longer delay is required between creating the measurement and sending the 
> ‘delete’ command?
> 
> Thanks,
> Steve
> 
>> On Dec 28, 2019, at 3:20 PM, Steve Gibbard  wrote:
>>
>> Hi Atlas folks,
>>
>> I hope you’re having a good holiday season.  Sorry to interrupt it by 
>> complaining about issues.
>>
>> On Christmas Eve my time (early Christmas morning your time) there was an 
>> Atlas issue where any attempt at reading measurements failed with an HTTP 
>> 500 status error.  That appears to have gotten fixed on Christmas (a really 
>> big thank you to whoever worked on that) but since then it appears that 
>> while most of the one-off measurements we’ve created have delivered results 
>> very quickly, none of the measurements created since 17:00 UTC on 2019-12-25 
>> have stopped running.  As shown in the Atlas portal:
>>
>>
>> 23722197 Traceroute  www.globaltraceroute.com (AS13335)  Test 
>> Traceroute 1   one-off 2019-12-25 22:24
>> Never
>> 23722089 Traceroute  archive.ubuntu.com (AS41231)Test Traceroute 
>> 1   one-off 2019-12-25 19:16
>> Never
>> 23722088 Traceroute  sps.prima.com.ar (AS10318)  Test Traceroute 
>> 1   one-off 2019-12-25 19:14
>> Never
>> 23721915 Traceroute  www.globaltraceroute.com (AS13335)  Test 
>> Traceroute 1   one-off 2019-12-25 17:00
>> Never
>>
>> And on for every measurement between then and now.
>>
>> Previously, the typical one-off measurement was listed with start and stop 
>> times less than 10 minutes apart.
>>
>> When a user has 100 measurements running concurrently, creation of new 
>> measurements fails, which is happening for me now.
>>
>> If somebody could take a look at this, I’d really appreciate it.
>>
>> Thanks,
>> Steve
>>
> 
> 
> 
> 
> 
> 
> 

Re: [atlas] Measurements "synchronizing"

[Chris Amin]

Sorry, I *do* see such delays. Looking into it now!

On 15/11/2019 16:02, Chris Amin wrote:
> Hi Stephane,
> 
> This status has been a possibility since at least June, but it should
> appear only very briefly while our backend systems synchronize. If you
> are seeing it for significant amounts of time then it could indicate
> that there are delays in processing measurements (although I don't see
> such delays right now).
> 
> In general if you see this status you should retry shortly afterwards.
> 
> Regards,
> Chris
> 
> On 15/11/2019 15:49, Stephane Bortzmeyer wrote:
>> It seems that, today, when querying the API about an existing
>> measurement, I always get:
>>
>> {'id': None, 'name': 'synchronizing'}
>>
>> Which I've never seen. (Measurement #23238335 but it's not the only
>> one.) A recent change? A temporay problem?
>>
>>
> 
> 
> 

Re: [atlas] Measurements "synchronizing"

[Chris Amin]

Hi Stephane,

This status has been a possibility since at least June, but it should
appear only very briefly while our backend systems synchronize. If you
are seeing it for significant amounts of time then it could indicate
that there are delays in processing measurements (although I don't see
such delays right now).

In general if you see this status you should retry shortly afterwards.

Regards,
Chris

On 15/11/2019 15:49, Stephane Bortzmeyer wrote:
> It seems that, today, when querying the API about an existing
> measurement, I always get:
> 
> {'id': None, 'name': 'synchronizing'}
> 
> Which I've never seen. (Measurement #23238335 but it's not the only
> one.) A recent change? A temporay problem?
> 
> 

Re: [atlas] RIPE Atlas API Swagger 2.0

[Chris Amin]

Hello,

Although there are no current plans to provide a Swagger/OpenAPI 2.0 (or
3.0.0) interface, it is something that we have been considering for
various reasons, so you may hear more on this in the future.

Thanks for the suggestion!

Regards,
Chris Amin
RIPE NCC

On 04/11/2019 08:54, a...@ipmn.io wrote:
> Hey guys,
> 
> Are there any plans to migrate API spec from swagger 1.2 to 2.0?
> 
> It will let us use more modern code generators...
> 
> 

Re: [atlas] User Tags missing [ProbeTag object]

[Chris Amin]

Dear Jiri,

Thanks for reporting this. It has now been fixed, so tags should display
as normal!

Regards,
Chris Amin
RIPE NCC

On 27/08/2019 13:16, r...@brite.cz wrote:
> Hi team,
> all User Tags on all probes I checked are showing string "ProbeTag object" 
> only.
> Please have a look.
> 
> Thank you
> Jiri
> 
> 

Re: [atlas] Question about built-in first/second hop latency measurements

[Chris Amin]

Hi Joel,

Results from measurements 1 and 2 are really the first and second hops
of a single traceroute measurement with a max_hops value of 2.

Regards,
Chris Amin
RIPE NCC

On 22/07/2019 00:16, Joel Sommers wrote:
> Hello all,
> 
> For the built-in latency measurements to first/second hops, my assumption has 
> pretty much always been that these are done with hop-limited probes (i.e., 
> with TTL or hop limit set to 1 or 2).  Is that true, or are these 
> measurements taken using “direct” pings?
> 
> Thanks!
> 
> Joel
> 
> 
> 

Re: [atlas] Scheduled work on RIPE Atlas

[Chris Amin]

Hi Grzegorz,

Apologies for this, it was actually an unrelated issue to do with
synchronizing certain DNS measurements. Those measurements should now
work as normal.

Regards,
Chris Amin
RIPE NCC

On 18/07/2019 12:05, Ponikierski, Grzegorz wrote:
> Hi!
> 
>  
> 
> Does this maintenance was related with DNS measurements? I generated
> today 60 DNS, got their IDs, I see in web UI that they are running but
> when I want to open them or search for them I get nulls or messages like
> "This measurement most probably does not exist (yet).". Can you for
> example check measurement *22378601*?
> 
>  
> 
> Regards,
> 
> Grzegorz
> 
>  
> 
> *From: *Robert Kisteleki 
> *Organization: *RIPE NCC
> *Date: *Tuesday 2019-07-16 at 13:47
> *To: *"ripe-atlas@ripe.net" 
> *Subject: *Re: [atlas] Scheduled work on RIPE Atlas
> 
>  
> 
>  
> 
> Dear All,
> 
>  
> 
> The publicly visible part of this work finished around noon (local time)
> 
> and all functions should be back to normal.
> 
>  
> 
> Regards,
> 
> Robert
> 
>  
> 
>  
> 
> On 2019-07-15 12:00, Robert Kisteleki wrote:
> 
> Dear RIPE Atlas users,
> 
> We have planned maintenance on the main RIPE Atlas backend database
> 
> tomorrow (Tuesday, 16 July) starting at 08:00 UTC, 10:00 Amsterdam local
> 
> time. We expect complete unavailability of the service for a short while
> 
> and further intermittent delays during the day.
> 
> During this time the website (UI and APIs) will not be responding
> 
> (properly). This also means that requests for new measurements will be
> 
> denied or delayed. Data from existing measurements will still be
> 
> collected normally, though access to them will be impacted.
> 
> Our apologies for the inconvenience.
> 
> Regards,
> 
> Robert Kisteleki
> 
>  
> 
>  
> 

Re: [atlas] Probes fluctuation

[Chris Amin]

On 30/04/2019 10:54, Daniel Karrenberg wrote:

> I have come across this question a number of times over the years.
> Wouldn't it be relatively straightforward to provide more information to
> the user about these conditions and decisions of the probe selection
> process?

There is an ongoing work item that will achieve this. It isn't
completely straightforward because probe selection is distributed and
has multiple stages, but it is doable and would indeed be useful in
various cases.

Chris

Re: [atlas] Scheduled RIPE Atlas maintenance 2019-04-10 09:00 UTC

[Chris Amin]

Hi all,

We are beginning this work now.

Regards,
Chris

On 09/04/2019 15:38, Chris Amin wrote:
> Dear RIPE Atlas users,
> 
> We are planning on making a behind-the-scenes change to the RIPE Atlas
> database tomorrow in order to make the system more robust. Measurements
> scheduled during this time may be either rejected or delayed. Probes
> will continue to carry out measurements as usual so there won't be a gap
> in scheduled measurement results.
> 
> The window for the maintenance is 2019-04-10 09:00 UTC until 12:00 UTC.
> We hope that it will be quicker than this and will keep you updated.
> 
> Kind regards,
> Chris Amin
> RIPE NCC
> 

[atlas] Temporary problem with RIPE Atlas measurement creation

[Chris Amin]

Dear RIPE Atlas users,

Due to a complication during a deploy, there was a problem scheduling
RIPE Atlas measurements from approximately 12:50 to 14:14 UTC. Most new
measurements created during this time will not have been scheduled properly.

The issue is now fixed for new measurements, so if your measurements
didn't work during this time you can try to create them again and it
should work. Apologies for any inconvenience caused.

Regard,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] DomainMON question

[Chris Amin]

Hi Marek,

This issue should now be fixed for you. Apologies for the inconvenience.
Please do let us know if you see any further problems.

Regards,
Chris Amin
RIPE NCC

On 07/09/2018 10:45, Massimo Candela wrote:

> We are currently working on fixing this issue.
> We will let you know when the problem is fixed.
> 
> Thanks for the email and sorry for the inconvenience.

> On 07/09/2018 09:05, Marek Zarychta wrote:
>> Dear subscribers,
>>
>> is DomainMON service (https://atlas.ripe.net/domainmon/) still available
>> and supported?
>> For a week or two I see no charts, only "There is no data for this
>> measurement yet." for my domains. Corresponding measurements are running
>> with default interval and spread.
>>
>> Regards,
>>
> 
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: [atlas] Is there a max running measurement number for a user

[Chris Amin]

Hi Yihao,

On 20/07/2018 07:50, Yihao Jia wrote:

> Is there a max running measurement number for a user?
> 
> I try to launch about 800 measurements, but only about 150 of them
> succeed and keep running, while another 650 fail with the description
> "overlimited denied"
> one of the fail measurement
> is: https://atlas.ripe.net/measurements/15302198/

There are indeed limits on the number of concurrent measurements,
amongst other kinds of limits. They are documented here:

https://atlas.ripe.net/docs/udm/#rate-limits

It is possible for limits to be temporarily increased to allow people to
perform research. If you'd like to email me directly with the outline of
your proposed experiment then I should be able to either increase your
limits temporarily and/or advise you on using RIPE Atlas more efficiently.

> Do you know what's the reason I failed?
> (btw, I am wondering how to check the reason why a measurement is failed
> by the measurement id?)

You can see the status of the measurement using the API:

https://atlas.ripe.net/api/v2/measurements/15302198/

In this case the status is "Scheduling denied", which is the delayed
version of the "overlimited denied" message that you saw with the other
measurements. This is because we do an initial check when you POST and a
second check when we are about to allocate probes.

The other measurement that you mentioned:

https://atlas.ripe.net/api/v2/measurements/15282285/

has a status of "Failed". In this case it is probably because the stop
time is too close to the time that you scheduled the measurement so it
didn't have a chance to run. Try increasing the stop time to a point
further in the future.

The statuses of measurements are not currently displayed clearly on the
web site, but that will be changing in the near future as we are
overhauling the measurement details pages.

Kind regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

[atlas] RIPE Atlas probe status issues

[Chris Amin]

Dear RIPE Atlas users,

There were various issues relating to the recorded status of RIPE Atlas
probes over the weekend. This was brought to our attention by internal
monitoring and information provided by users on the mailing list.

Throughout this period most probes did actually remain connected to
controllers, and measurement results were collected as normal. The side
effects included:

* the number of probes reported as connected by the system was lower
than it should have been
* the status (connected/disconnected) of many probes was incorrect
* new measurements took longer than usual to start
* fewer probes than usual were available for new measurements, leading
in some cases to “no suitable” probes messages when trying to schedule
new measurements
* various system tags were incorrectly applied, including many probes
being marked as having USB problems when this was not the case
* temporary discrepancies with crediting/debiting of RIPE Atlas credits
for the
connected time of probes

The issues were caused by a bug fix deployment at Friday 9AM UTC where a
package was accidentally downgraded causing a regression to an old bug
in the task handling of the central system. This bug caused a backlog of
messages to build, slowing down or stopping the registering of various
status messages in the system. Problems built up gradually as the
backlog increased, until the root cause was identified on Sunday
morning. The issue was then fixed and the system stabilized completely
by about 10AM UTC. We have identified procedural and technical solutions
that will stop this problem happening again, and are looking at ways to
improve our monitoring of these kinds of issues.

We apologise for any inconvenience or confusion caused by this event and
would like to thank all of you who took the time to notify us of what
you were seeing.

Kind regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] how to mark probe as dormant

[Chris Amin]

Hi Randy,

On 15/04/2018 16:56, Randy Bush wrote:
> i have a probe, 2283, which has unplugged from its old home and is in
> suspended animation in a suitcase until it reaches its new home in a
> month or two.
> 
> the location map will not let me pur it in the middle of the pacific.
> and i can not see how to mark it as dormant/down.  clue bat please.

There's nothing you have to do when a probe is down; it is automatically
marked as disconnected almost immediately, and if it isn't plugged back
in  then it is marked as "abandoned" after 90 days and "written off"
after 180 days (both of which are reversible states, so don't worry too
much).

We have generally tried to move away from using remote areas to signify
particular probe statuses as it confuses various visualisations. Feel
free to leave the location as it is, or use the location of the
suitcase, or whatever else you feel to be appropriate.

If you are concerned about being auto-nagged about the probe being
disconnected, we get that and there is an upcoming feature that will
enable you to switch off notifications for a particular probe when you
know that it will be disconnected for a while.

Finally, if you really feel like it then you're free to edit either the
probe description or the tags to signify whatever you like, so you could
make a note for your own purposes or for anybody who's curious.

Regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] tag "system: IPv6 Works"

[Chris Amin]

Hi Wolfgang, welcome to the community!

On 28/01/2018 01:20, Wolfgang S Rupprecht wrote:

> I was surprised to see that IPv4-only probes would be scheduled to
> send IPv6 pings, traceroutes, nslookups etc.  Even when I added
> "system: IPv6 Works" to the include tags and "system: IPv6 Doesn't
> Work" to the exclude tags I still got some IPv4-only probes assigned.
> Are those tags for future use and not wired up yet?  Are the IPv4-only
> probes I'm seeing in my test runs there because their status changed
> recently and previously they were IPv6 capable?

These tags are currently reapplied every 4 hours, so there is indeed the
possibility that a probe is no longer capable of carrying out
measurements with a particular address family even though it has an
"IPvX Works" tag applied.

If you have a particular example or examples then please send them to me
directly and I will take a look to see if there's anything strange going on.

Regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

[atlas] Missing servers in DNSMON

[Chris Amin]

Dear colleagues,

It was recently brought to our attention that a small number of
name servers were not being included in the visualisations on DNSMON.
This issue has now been fixed, but you may notice a gap between
mid-September and mid-December when visualising large time ranges for
the affected servers.

Apologies for any confusion or inconvenience caused. The affected zones
and nameservers are:

hk. - 203.119.87.218
xn--j6w193g.- 203.119.87.218
il. - 128.139.35.5
il. - 194.146.106.122
il. - 2001:67c:1010:31::53
mx. - 2001:13c7:7000::1
pl. - 2001:1a68:0:17::238
pl. - 2001:a10:121:1::156
pl. - 2a00:4120:8000:2::186

Kind regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] Problem in status checks?

[Chris Amin]

Hi Stephane,

We are currently experiencing some problems with one of our backends
which is affecting various API endpoints, including status checks. We
are looking into the root cause now.

Regards,
Chris

On 14/10/2017 09:11, Stephane Bortzmeyer wrote:
> Error 500 when trying to retrieve status checks:
> 
> % curl -v 
> 'https://atlas.ripe.net/api/v2/measurements/2060427/status-check?permitted_total_alerts=3'
> ...
>> GET /api/v2/measurements/2060427/status-check?permitted_total_alerts=3 
>> HTTP/1.1
>> Host: atlas.ripe.net
>> User-Agent: curl/7.52.1
>> Accept: */*
>>
> 
> < HTTP/1.1 500 Internal Server Error
> < Server: nginx/1.10.2
> < Date: Sat, 14 Oct 2017 07:09:10 GMT
> < Content-Type: text/html; charset=utf-8
> < Content-Length: 38403
> < Connection: keep-alive
> < Vary: Accept-Encoding, Cookie
> < X-Frame-Options: SAMEORIGIN
> < Strict-Transport-Security: max-age=15768000
> 
> 

Re: [atlas] Infrastructure upgrades

[Chris Amin]

Dear John,

You are correct, we worked on one of the anchor controllers today and we
have identified an issue that caused more of an impact than expected.

We are in the process of fixing this: you should already see some of the
broken anchors stabilizing, and you can expect to see the remaining ones
fixed shortly.

We'll provide an update when everything is back to normal.

Regards,
Chris


On 06/09/2017 17:28, ripe-at...@johnbond.org wrote:
> Hello Robert,
> 
> I just wanted to confirm that this work is why we see missing data in DNSMON 
> for a number of the anchors
> 
> Thanks John
> 
>> On Aug 24, 2017, at 3:47 PM, Robert Kisteleki  wrote:
>>
>> Dear all,
>>
>> Yesterday we began infrastructure (OS) upgrades on our controllers. As a
>> consequence each probe will be disconnected for an hour or two. This can
>> trigger notifications, if you have that configured to a sensitive enough
>> setting.
>>
>> Regards,
>> Robert Kisteleki
>> RIPE NCC
>>
>>
> 
> 
> 

[atlas] NSID option on the RIPE Atlas SOA measurements of the root servers

[Chris Amin]

Dear colleagues,

RIPE Atlas is currently running a series of DNS SOA built-in
measurements* towards all of the root servers from all probes. During
the recent DNS measurements hackathon** it became apparent that for some
use cases it would be useful to have SOA queries from all probes with
the NSID EDNS option set, in order to be able to match up responses with
the particular responding instances in an anycasted (or load balanced)
setup.

We would like to ask for feedback on two alternative options for
implementing this change. They are:

1) Enable the NSID option for the existing built-in measurements towards
the nine root servers which support it.

2) Start a new set of built-in measurements towards the nine root
servers which support NSID.

The advantages of option (1) are that it will be possible to compare and
contrast the two sets of results, and that historical data for the
existing built-ins will remain consistent with the current results. A
very simple analysis shows that there is no overall increase in query
error rates through enabling NSID, but there are bound to be individual
marginal cases where queries fail or produce different results with NSID
set but succeed without it.

The advantages of option (2) are that there will be no increase in
result storage usage -- the current IPv4+IPv6 UDP SOA built-ins towards
the nine supporting root servers adds up to about 80 results per second
(~1.5% of the total results in the system).
This could potentially be mitigated by reducing the frequency for these
new measurements, but perhaps more important than the slightly increased
load is the
potential user confusion caused by generating and providing two very
similar sets of
measurements.

Please let us know if you have any preference for which way we go on
this, particularly if you have a (current or future) use case for this
kind of data.

Kind regards,
Chris Amin
RIPE NCC

* https://atlas.ripe.net/docs/built-in/
** https://labs.ripe.net/Members/becha/results-dns-measurements-hackathon





signature.asc
Description: OpenPGP digital signature

[atlas] Improvements to the RIPE Atlas measurement creation form

[Chris Amin]

Dear all,

We’ve made some changes to the interface for creating measurements on
the RIPE Atlas website (https://atlas.ripe.net/measurements/form/) that
will make it both faster and more responsive. Users should also see
similar improvements in the user interface for monitoring and
configuring domains with DomainMON (https://atlas.ripe.net/domainmon/).

The changes were made possible by switching to the public measurement
API (https://atlas.ripe.net/docs/api/v2/manual/). Concurrently with
this, we also improved the error reporting in the API to make it easier
to find out the cause of errors. This API change is backwards compatible
and is documented in the API manual
(https://atlas.ripe.net/docs/api/v2/manual/overview/error_messages.html).

Whilst we do not expect the transition to cause any issues, we kindly
ask that users let us know if they experience anything out of the
ordinary that might be related to the change.

Best regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] status-check misbehaving

[Chris Amin]

Dear Georg,

The current behaviour is for all probes that have returned results in
the past three days to be included in the status-check report. That's
why removing the probes from the measurement did not immediately remove
them from the status-check.

We will review this behaviour because I can see how it is desirable to
be able to immediately remove probes from the status check, and users
will generally not care about results from probes that they have
explicitly removed. For now your status-check does not include the old
probes since they were removed more than three days ago -- if you have
to remove more probes just keep in mind that the effect is not currently
immediate.

Kind regards,
Chris Amin
RIPE NCC

On 13/04/2017 11:08, Georg Kahest wrote:
> Hello,
> 
> Am I misunderstanding something or the status-checks are kind of broken:
> 
> Yesterday i made new measurement:
> 
> https://atlas.ripe.net/measurements/8038852/
> 
> I noticed some probes werent reaching my ipv6 host, so decided to remove
> them like you can see from the Probe specifications section.
> 
> Now when i goto
> https://atlas.ripe.net/api/v2/measurements/8038852/status-check/
> 
> I can still see the removed probes in status-check page (which contains
> 17 errors ) in reality if i'm lookin at measurement probe page i should
> have only two errors.
> 
> So it seems that status-check isnt updated once probes are removed from
> a measurement.
> 




signature.asc
Description: OpenPGP digital signature

Re: [atlas] Behind the scenes changes to measurement creation

[Chris Amin]

Dear Alex,

Apologies, thank you for picking up on this. The stop_time and
start_time lt, lte, gt and gte fields were incorrectly working like
their exact match counterpart. This has now been fixed and these filters
should work as expected.

Kind regards,
Chris


On 18/04/2017 14:04, Alexandros Milolidakis wrote:
> Hi Chris ,
> It seems that stop_time , start_time etc don't work anymore.
> For example picking all trace route measurements which stopped before 
> yesterday yields no results.
> https://atlas.ripe.net:443/api/v2/measurements/?stop_time__lt=1492469625=traceroute
> Could you take  look?
> Thanks
> Alex
> 
> Sent via RIPE Forum -- https://www.ripe.net/participate/mail/forum
> 
> 




signature.asc
Description: OpenPGP digital signature

Re: [atlas] Feature request: API call in results page

[Chris Amin]

On 01/03/2017 15:18, Hugo Salgado wrote:
> Hi RIPE Atlas people.
> I wonder if it could be possible to add an option on the "measurements"
> webpage of a previous stopped or running measurement, to be able to see
> the API call that created it. Just like the "Measurement API Compatible
> Specification" section, when you create one. It could be available just
> to the user who created it... and even with API keys obscured.
> 
> That will be very handy to be able to replay and exact measurement, or
> to have the list of probes in a ready-to-use format.

Hi Hugo,

Replicating existing measurements is something which has been on our
radar for a while, and we are considering options similar to that which
you propose.

In the meanwhile, you can get a lot of the way towards replicating a
measurement by basing your POST data on the JSON measurement spec from
the API (/api/v2/measurements/{existing_msm_id}/) and using the "msm"
type of probe selection
(https://atlas.ripe.net/docs/api/v2/manual/measurements/types/in_detail.html)
to run the measurement on the same set of probes (if desired).

Thanks for the feedback.

Regards,
Chris Amin



signature.asc
Description: OpenPGP digital signature

Re: [atlas] Maximum (unknown) spending limit?

[Chris Amin]

Hi Sebastian,

Thanks for noticing this, it has now been fixed. Indeed it was a small
templating bug.

Happy new year,
Chris Amin
RIPE NCC

On 11/01/2017 05:45, Sebastian Castro wrote:
> Happy New Year RIPE Atlas Users:
> 
> Executing some measurements today, it started rejecting them with the
> message:
> 
> Executing this measurement request would violate your maximum daily
> spending limit of {max} credits. Please stop some of your currently
> running measurements and try again.
> 
> It's startling the {max} credits, like a template that didn't get
> replaced when processing the message, not the message itself as it's
> documented. A little 2017 bug perhaps?
> 
> Cheers!
> 




signature.asc
Description: OpenPGP digital signature

Re: [atlas] Emails Regarding Probe Connection Issues

[Chris Amin]

Dear Povl Ole,

Thank you for pointing this out. This was caused by an unrelated issue
with the templating system, which has now been resolved. My apologies
for the incomplete email.

Regards,
Chris Amin
RIPE NCC


On 17/11/2016 03:02, Povl Ole Haarlev Olsen wrote:

> Are you still working on this issue?
> 
> I got one of the new emails earlier today, but it was a bit less
> informative than the previous emails:
> 
> [- Quote -]
> Subject: Probe  is disconnected ()
> 
> ...
> 
> Your probe  has been disconnected from the RIPE Atlas infrastructure
> since .
> 
> ...
> [- End quote -]
> 
> It looks like there's something wrong with the template system you're
> using.




signature.asc
Description: OpenPGP digital signature

[atlas] Out of date system probe tags

[Chris Amin]

Dear all,

Some of you have noticed that certain system tags (see below) remained
applied to probes when they should have been removed. This is because
the process that adds and removes these tags was temporarily disabled
following the service issues that we experienced earlier in the month.
This process has now been re-enabled after verifying that it was not a
cause of the service downtime, and these tags have been applied correctly.

The affected tags were:

* Flaky connection
* Firewall problem suspected
* DNS problem suspected
* No flash drive
* Readonly flash drive
* Hardware problem suspected

Kind regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] Connect/disconnect timestamps on probe page disappeared

[Chris Amin]

On 13/10/2016 09:46, Marat Khalili wrote:
> Connect/disconnect timestamps on a probe page like
> https://atlas.ripe.net/probes/26656/#!tab-network disappeared this
> morning; durations still shown. Probably some problem with server side
> scripts?

Dear Marat,

Many thanks for pointing this issue out. There was indeed a problem with
the presentation layer on this page, although the actual
connect/disconnect events were properly recorded.

We have now fixed this issue so you can see connect/disconnect
timestamps again. Apologies for any inconvenience caused!

Kind regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] *-capable ≠ *-works + *-doesnt-work

[Chris Amin]

On 19/04/2016 11:07, Bajpai, Vaibhav wrote:
> Hello,
> 
> I am looking at the probe API data for connected probes (status = 1) for day 
> = 20160418
> 
> system-ipv6-capable = 3556
> system-ipv6-works   = 2995
> system-ipv6-doesnt-work = 710
> 
> Why is system-ipv6-works + system-ipv6-doesnt-work > system-ipv6-capable?
> 
> system-ipv4-capable = 9336
> system-ipv4-works   = 9187
> system-ipv4-doesnt-work = 67
> 
> Why is system-ipv4-works + system-ipv4-doesnt-work < system-ipv4-capable?

Dear Vaibhav,

You are right that system-ipv6-works + system-ipv6-doesnt-work should be
a subset of system-ipv6-capable. The -works and -doesnt-work tags are
assigned based on measurement results from the past few hours, and there
appears to be a bug whereby probes are continuing to carry out IPv6
measurements even when they no longer have IPv6 capability. This
understandably results in unsuccessful results, which triggers the
"doesnt-work" tag. Thank you for pointing this out to us, we are working
on a fix now.

The other case (system-ipv4-works + system-ipv4-doesnt-work <
system-ipv4-capable) makes more sense, because it's possible that a
probe is marked as "capable" but doesn't actually return any measurement
results for some reason -- for instance, it may be able to connect to
the controller over IPv4 -- so we know for sure that it is "capable" of
IPv4 -- but isn't able to report any IPv4 measurement results because of
a faulty SD card, which isn't reflective of an IPv4 issue per se. We
only mark a probe as "ipvX-doesnt-work" when we actually get
unsuccessful results back from it, so in some cases a probe will have
neither "works" nor "doesnt-work" tags.

Kind regards,
Chris Amin



signature.asc
Description: OpenPGP digital signature

Re: [atlas] Display issues on the RIPE Atlas measurements listing

[Chris Amin]

On 20/01/2016 14:57, Chris Amin wrote:

> Some of you may be noticing issues displaying measurements on the RIPE
> Atlas UI if you look at Public measurements or have lots of measurements
> of your own. We're currently working on this issue and apologise for any
> inconvenience caused.

Any issues with the measurement listing should now be resolved.

Thanks,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

[atlas] Display issues on the RIPE Atlas measurements listing

[Chris Amin]

Dear RIPE Atlas users,

Some of you may be noticing issues displaying measurements on the RIPE
Atlas UI if you look at Public measurements or have lots of measurements
of your own. We're currently working on this issue and apologise for any
inconvenience caused.

Kind regards,
Chris Amin
RIPE NCC



signature.asc
Description: OpenPGP digital signature

Re: [atlas] [Request] System tags for DNS resolution

[Chris Amin]

On 12/11/2015 22:45, Stephane Bortzmeyer wrote:

>>> For instance, probes 16659, 17072, 17620, 17854 use a resolver
>>> which yields a REFUSED for any request and probe 19948, 20065 one
>>> with systematic SERVFAIL.
> ...
>> The above probes are mostly tagged as "resolves-a[aaa]-correctly"
>> because they managed to resolve certain A records (e.g. MSM ID
>> 1698856).  Can you provide the measurement ID(s) where you see the
>> REFUSED and SERVFAIL responses?
> 
> Probes 19948, 20065 now seems OK. But 16659, 17072, 17620, 17854 still
> return REFUSED: measurements #2928193 and #2928199.

I think what is happening here is that all of those probes except for
17854 have at least *one* resolver which does respond to queries. 17854
seems to have no resolvers responding successfully, which is why it has
the tag "Doesn't Resolve A".

This is because the "Resolves A Correctly" tag is really "Resolves A
Correctly (for at least one resolver) (for at least one pre-designated
stable target)". We have discussed internally creating another set of
tags which say something about the reliability and stability of DNS
resolution, rather than our current liberal tags which basically claim
something like "you will probably get *some* usable results from this
probe".

I would be interested to hear your (or anybody else's) thoughts on
whether you would use such a reliable/stable DNS resolution tag, and
what kind of criteria you would expect for it to be applied.

Cheers,
Chris



signature.asc
Description: OpenPGP digital signature

Re: [atlas] [Request] System tags for DNS resolution

[Chris Amin]

Hi Stephane,

On 11/11/2015 17:36, Stephane Bortzmeyer wrote:
> [Is there a better way to record enhancement requests for Atlas? Atlas
> has no public issue tracker, if I'm correct?]

You can also send requests to at...@ripe.net, where they will be
automatically added to our issue queue. There's nothing intrinsically
wrong with sending things to the list though, especially because it
allows other users to chime in and say that they've noticed similar
things before/they have an explanation/a workaround/etc.

> It would be cool to have system (automatic) tags for DNS resolution
> because some probes are using broken DNS resolvers. For instance,
> 
> system-dns-works (modeled on IPv6's system tag) would be fine. Or may
> be system-dns-resolver-works (longer but more accurate).
> 
> For instance, probes 16659, 17072, 17620, 17854 use a resolver which
> yields a REFUSED for any request and probe 19948, 20065 one with
> systematic SERVFAIL.

You may have noticed that we apply "system-resolves-a[aaa]-correctly" to
probes. This happens when we find that at least one of their DNS
resolvers correctly resolves one of a few specific records at least
partially over a 4 hour period. We decided to be specific and shy away
from claims like "DNS works" because that could be considered a rather
strong claim.

The above probes are mostly tagged as "resolves-a[aaa]-correctly"
because they managed to resolve certain A records (e.g. MSM ID 1698856).
Can you provide the measurement ID(s) where you see the REFUSED and
SERVFAIL responses?

> [Is the maintainer of the probe automatically notified when there is
> such an issue?]

If there is a tagging change, then we send an in-site RIPE Atlas message
to the user. We don't currently provide the option of emailing users on
such changes, but it's something that we would consider if there is
support for such an idea.

Regards,
Chris



signature.asc
Description: OpenPGP digital signature

[atlas] RIPE Atlas probe archive not currently working

[Chris Amin]

Dear RIPE Atlas users,

We are aware that the archive of RIPE Atlas probe metadata has not been
available since code changes on Wednesday afternoon. This affects both
the archive API at /api/v1/probe-archive/ and recent days in the FTP
archive at http://ftp.ripe.net/ripe/atlas/probes/. If you only require
old archives (Tuesday 29th September and older), you can fetch them from
the FTP site as usual.

We are working on restoring this functionality as quickly as possible.
Please accept our apologies for any inconvenience that this may have caused.

Kind regards,
Chris Amin
RIPE NCC