Re: DNS issue affecting gnu.org (and subdomains)

2023-03-26 Thread Thomas Schmitt
Hi,

i have no idea how free it is, but this web site works without
Javascript and with a quite old browser, although it complains that
Javascript is off:

  https://www.isitdownrightnow.com/gnu.org.html

The outage of saturday is still visible but will soon be pushed aside
by success reports.


Have a nice day :)

Thomas




Re: DNS issue affecting gnu.org (and subdomains)

2023-03-26 Thread Ineiev
On Sun, Mar 26, 2023 at 12:55:56PM -0400, James Cloos wrote:
> BP> +1 for the https://hostux.social/@fsfstatus status page.
>
> try it w/ links or the like.
>
> useless.
>
> ecmascript should *never* be required.
>
> especially for something like a status page.

+1.

Also, it just replied with "403 Forbidden" when I accessed it via Tor.



Re: DNS issue affecting gnu.org (and subdomains)

2023-03-26 Thread James Cloos
> "BP" == Bob Proulx  writes:

BP> +1 for the https://hostux.social/@fsfstatus status page.

try it w/ links or the like.

useless.

ecmascript should *never* be required.

especially for something like a status page.

-JimC
-- 
James Cloos  OpenPGP: 0x997A9F17ED7DAEA6



Re: DNS issue affecting gnu.org (and subdomains)

2023-03-26 Thread Sebastian Tennant
Quoth Ar Rakin 
on Sat, 25 Mar 2023 23:38:29 +0600:
> Hello Sebastian,
>
> I'm also unable to access gnu.org. I don't exactly know why this is
> happening, though I've encountered this type of issues before, with
> my own domains. But it got fixed automatically after a few
> days. Hopefully, it will be fixed automatically after some time.
>
> What I can see:
>
> $ host gnu.org
> ;; connection timed out; no servers could be reached
>
> $ ping gnu.org
> ping: gnu.org: Temporary failure in name resolution
>
> Thanks,
>
> Rakin

Hello Rakin.  Thanks for the corroboration.


Quoth Eli Zaretskii 
on Sat, 25 Mar 2023 20:52:00 +0300:
>> […]
>
> You will find the information here:
>
>   https://hostux.social/@fsfstatus
>
> That place is always good to look at when such issues occur.

Hello Eli.  Thanks for the link.  Noted.


Quoth Bob Proulx 
on Sat, 25 Mar 2023 15:05:44 -0600:
>> […]
>
> +1 for the https://hostux.social/@fsfstatus status page.  The FSF
> sysadmins post information there (sometimes terse) when there are
> problems seen that affect systems.  It's something everyone should
> bookmark where they can find it easily.
>
>>  $ host gnu.org 8.8.8.8
>>  [...]
>>  Host gnu.org not found: 2(SERVFAIL)
>>
>> Nope, Google's resolver can't resolve gnu.org either.
>
> The authoritative nameservers (a fancy title for the upstream ones)
> are getting DDoS'd off the net.  Which means that all resolution by
> downstream nameservers, even Google ones, are timing out.

Hello Bob.  A DDoS attack.  I see.

> Compounded by the very short 300 second TTL on the gnu.org records
> mean that even if a lookup is successful that it can only be cached
> for five minutes and then discarded.  Upon which then it needs to be
> looked up again and the query will have to fight its way through the
> DDoS in a mixed martial arts cage fight arena to get the data again.

What's the thinking behind the short TTL?

>> […]
>
> The nameservers are overwhelmed making them slow to respond.  And
> then additionally I am seeing a very high packet loss across the
> network into the Boston machines.  That high packet loss means
> retries at the network protocol level making things slow.  I have
> seen 30-45 seconds on average here looking up DNS for a while.

Understood.

>> […]
>
> There is really nothing special about the Google resolver.  If the
> upstream ns*.gnu.org nameservers can't receive and can't send data
> then gnu.org names cannot be resolved.

Yup.  Understood.  I know there's nothing special about Google's
nameservers.  They have an easy-to-remember IP address, that's all.

>> I fetch from git.sv.gnu.org every 30 minutes and the fetch beagn to
>> fail two days ago (on 23rd March) at around 10pm GMT.  It has been
>> failing much more often than not since then.

> Yes.  That's about when the attack started.  I assume it is an
> attack.  That's what sysadmin said about it.  I have no special
> ability to observe this particular attack and am suffering through
> the packet loss of it along with the rest of you.

:-) It seems the worst is over now.  (Until the next time, in any
case).


Quoth Ian Kelling 
on Sat, 25 Mar 2023 18:51:48 -0400:
> Update: We think we've got things working now.

Hello Ian.  Thanks for the update.  And for your efforts restoring
normal service.



Re: DNS issue affecting gnu.org (and subdomains)

2023-03-25 Thread Ian Kelling
Update: We think we've got things working now.

-- 
Ian Kelling | Senior Systems Administrator, Free Software Foundation
GPG Key: B125 F60B 7B28 7FF6 A2B7  DF8F 170A F0E2 9542 95DF
https://fsf.org | https://gnu.org



Re: DNS issue affecting gnu.org (and subdomains)

2023-03-25 Thread Bob Proulx
Eli Zaretskii wrote:
> > Ar Rakin wrote:
> >$ host gnu.org
> > ;; connection timed out; no servers could be reached

> You will find the information here:
>
>   https://hostux.social/@fsfstatus
>
> That place is always good to look at when such issues occur.

+1 for the https://hostux.social/@fsfstatus status page.  The FSF
sysadmins post information there (sometimes terse) when there are
problems seen that affect systems.  It's something everyone should
bookmark where they can find it easily.

>  $ host gnu.org 8.8.8.8
>  [...]
>  Host gnu.org not found: 2(SERVFAIL)
>
> Nope, Google's resolver can't resolve gnu.org either.

The authoritative nameservers (a fancy title for the upstream ones)
are getting DDoS'd off the net.  Which means that all resolution by
downstream nameservers, even Google ones, are timing out.

Compounded by the very short 300 second TTL on the gnu.org records
mean that even if a lookup is successful that it can only be cached
for five minutes and then discarded.  Upon which then it needs to be
looked up again and the query will have to fight its way through the
DDoS in a mixed martial arts cage fight arena to get the data again.

> How about, making the same queries on a VPS in the US:
>
>  $ host gnu.org
>  gnu.org has address 209.51.188.116
>  gnu.org has IPv6 address 2001:470:142:5::116
>  Host gnu.org not found: 2(SERVFAIL)
>
> Hmm, that worked, just, but it was very slow (~ 8 secs).

The nameservers are overwhelmed making them slow to respond.  And then
additionally I am seeing a very high packet loss across the network
into the Boston machines.  That high packet loss means retries at the
network protocol level making things slow.  I have seen 30-45 seconds
on average here looking up DNS for a while.

>  $ host gnu.org 8.8.8.8
>  [...]
>  Host gnu.org not found: 2(SERVFAIL)
>
> Google's resolver fails again.

There is really nothing special about the Google resolver.  If the
upstream ns*.gnu.org nameservers can't receive and can't send data
then gnu.org names cannot be resolved.

> I fetch from git.sv.gnu.org every 30 minutes and the fetch beagn to
> fail two days ago (on 23rd March) at around 10pm GMT.  It has been
> failing much more often than not since then.

Yes.  That's about when the attack started.  I assume it is an
attack.  That's what sysadmin said about it.  I have no special
ability to observe this particular attack and am suffering through the
packet loss of it along with the rest of you.

Bob



Re: DNS issue affecting gnu.org (and subdomains)

2023-03-25 Thread Eli Zaretskii
> Date: Sat, 25 Mar 2023 23:38:29 +0600
> Cc: savannah-users@gnu.org
> From: Ar Rakin 
> 
> Hello Sebastian,
> 
> I'm also unable to access gnu.org. I don't exactly know why this is 
> happening, though I've encountered this type of issues before, with my 
> own domains. But it got fixed automatically after a few days. Hopefully, 
> it will be fixed automatically after some time.
> 
> What I can see:
> 
>    $ host gnu.org
>     ;; connection timed out; no servers could be reached
> 
>    $ ping gnu.org
>    ping: gnu.org: Temporary failure in name resolution

You will find the information here:

  https://hostux.social/@fsfstatus

That place is always good to look at when such issues occur.



Re: DNS issue affecting gnu.org (and subdomains)

2023-03-25 Thread Ar Rakin

Hello Sebastian,

I'm also unable to access gnu.org. I don't exactly know why this is 
happening, though I've encountered this type of issues before, with my 
own domains. But it got fixed automatically after a few days. Hopefully, 
it will be fixed automatically after some time.


What I can see:

  $ host gnu.org
   ;; connection timed out; no servers could be reached

  $ ping gnu.org
  ping: gnu.org: Temporary failure in name resolution

Thanks,
Rakin