Yes, section 8.1 overlaps somewhat with this discussion. But section 8.1 appears to be written for the network admin configuring a firewall or similar device. I am looking for recommendation to implementers of clients and servers. My personal summary of the discussion so far is:

* the list of "suspicious port numbers" is not fixed. This goes both ways, as shown in the memcached example: almost nobody heard about it before the attacks, so it was not blocked. As soon as the attacks started, it became urgent to block that port. But after the code was updated to prevent reflection, it might be time to take it off the list and avoid blocking legitimate services.

* the "preferred address abuse" attack cannot be mitigated by firewalls. Section 21.5.3 of RFC 9000 describes the attack, and points to section 21.5.6 for generic responses, which says that "Endpoints MAY choose to avoid sending datagrams to these [suspicious] ports or not send datagrams that match these [suspicious] patterns prior to validating the destination address." The discussion points to something a bit more specific, a recommendation to implement a list of suspicious ports and a requirement that this list be part of configuration.

* blocking ports is likely to cause random failures, e.g., if the client's source port is rewritten by a NAT to match a suspicious port. Rather than blocking ports outright, implementations should consider a gradual response. For example, if an Initial packets arrives from a suspicious port, trigger a Retry to validate the address rather than merely drop the packet. This might be combined with the other recommendations in section 21.5.6, e.g., filtering by address scope or filtering for suspicious patterns.

* there is not much benefit in trying to prevent volumetric attacks in the application, but there may be some. If servers do that, they could implement the gradual response mentioned above, keep track of ports recently used in failed handshakes, and move from soft response such as Retry to hard response, drop the incoming packets if some threshold is reached.

* other discussions on the the developers' chat suggest that the same issue probably applies to the "amplification limit". A factor 3 is better than 10 or 1000, but it is still more than 1. If servers notice too much traffic triggering the limit and ultimately failing, they should start requiring source address validation (Retry), and possibly move to harder response after some point.

-- Christian Huitema

On 5/5/2022 5:49 AM, Mirja Kuehlewind wrote:
Hi, as you mention the manageability draft... there is quite some relevant text 
in there.

E.g. there is a section on first packet identification:
https://www.ietf.org/archive/id/draft-ietf-quic-manageability-16.html#name-first-packet-identification
which mainly says there is not much you can do but it's discussed.

Also note that recommendations how to deploy QUIC are in scope for the 
applicability statement (not manageability) and that document e.g. as a section 
source part selection:
https://www.ietf.org/archive/id/draft-ietf-quic-applicability-16.html#name-source-port-selection

However, I think there are also various things you have discussed below that 
are not QUIC specific and therefore probably don't really belong into any of 
the two docs (but there is always a change to write another one).

Mirja



On 04.05.22, 17:51, "QUIC on behalf of Christian Huitema" <[email protected] 
on behalf of [email protected]> wrote:


     On 5/3/2022 11:53 PM, Willy Tarreau wrote:
     > On Tue, May 03, 2022 at 11:00:11PM -0700, Christian Huitema wrote:
     >> QUIC includes a feature to facilitate moving a connection from an 
anycast
     >> address to a unicast address. The client finds the anycast address 
through
     >> the DNS, contacts the server, learns a "preferred server address" 
during the
     >> handshake, and tries validating that preferred address before using it 
for
     >> the rest of the connection duration, without risk losing the connection 
to a
     >> routing event that redirects anycast to a different server. Which is 
nice,
     >> but of course a villainous server could feed a bogus preferred address.
     >> That's the client bounce for you. All the server packets before the 
bounce
     >> are sent with the expected 5 tuple. And of course the "preferred 
address" is
     >> sent using encrypted packets.
     > Ah, I wasn't aware, seen like this, that's fun. Well, I guess the client
     > will try to connect to the new address with some info allowing to 
identify
     > the origin server (e.g. SNI), thus allowing a QUIC decoder/server at that
     > address to spot where the client found the address, no ?

     Everything is encrypted, except for a few bytes in the packet header
     that carry a connection identifier, i.e., an octet string chosen by the
     server. The server provides the client with a list of those; if needed,
     that list is known by the load balancer at the server side, so the LB
     can forward the packet to the right server in the farm. The encryption
     key is specific to the connection, so only the right server can respond.

     >>> There *will* inevitably be some problems at the beginning, but one good
     >>> thing is that solutions are reasonably simple, can instantly be 
deployed
     >>> in field using port filtering, and will quickly land in the code itself
     >>> because the various stack authors have full control of their code.
     >> Port filtering at the edge will not catch the "preferred address 
attack" in
     >> which the server proposes a non-anycast address to the client, so I 
guess
     >> QUIC stacks will have to protect that. As in, if a server says "please 
move
     >> the connection to 10.0.0.1:53", the client should just decline and 
stick to
     >> the anycast address. And yes, using a configurable list makes sense.
     > Yes, that's a good point, I agree. I think quite a bit of the burden will
     > be on the clients' shoulder. It reminds me the good joke we had in the 
90s,
     > creating a "warez.<domain>" DNS entry pointing to 127.0.0.1. I've 
personally
     > seen someone trapped and saying "what, they stole'my data!" after ftping
     > there :-)
     >
     >>>> Yes. Should that really be "per source port" or "per source IP + 
port"?
     >>> Per port should be sufficient. There's no reason from the server side 
that
     >>> multiple clients collectively decide to use the same source port to 
send
     >>> legit traffic. And using the port alone will allow to quicky detect a 
new
     >>> internet-wide attack coming from a randomly vulnerable service (IoT
     >>> gateways, ToIP etc) regardless of their address. I don't know how this
     >>> would be distributed, but I guess that if a source port is responsible
     >>> for 100 times more traffic than all other one combined, it will usually
     >>> be a good indicator that you don't want to deal with that traffic 
anymore.
     >> I would expect quite a bit of legit server-to-server traffic on ports 
443 or
     >> 853,
     > I think that's exactly the thing we must avoid. Encouraging (or even
     > allowing) this is exactly what will make it impossible for infrastructure
     > to defend itself against massive attacks. There's no value in using the
     > same port on both sides. The savings of sockets and source ports are
     > already enormous, going from N to 1 from TCP to QUIC, if an agent needs
     > to make outgoing connections, it certainly can afford a second socket,
     > which is nothing compared to the hundreds of thousands it would have used
     > in TCP. Not being willing to do that is putting the internet at risk. For
     > example among the protections you usually find at the edge is the 
blocking
     > of a TCP SYN packet from a source port below 1024. A few services in the
     > past (rlogin, rsh) used to bind to ports 512-1023 to "prove" that the
     > user ID was authenticated by the local OS, but that's never seen over the
     > net. One service has caused trouble to this, active FTP with source port
     > 20, and rules which were a bit too lax used to allow it to connect to any
     > service (e.g. SMTP) thereby allowing to use FTP servers to send spams by
     > retrieving files that contained SMTP protocol contents. This has
     > contributed to making active FTP unpopular, and nowadays it has become
     > safe to block SYN from sources < 1024 at the edge. UDP doesn't have such
     > a thing as a SYN flag and it's critical that traffic cannot be made
     > symmetrical, or there's no more infrastructure filtering and only
     > application level filtering.

     Too bad that nobody discussed that during the review of
     https://datatracker.ietf.org/doc/draft-ietf-quic-manageability/, that
     could have been a nice place for such considerations.

     It seems that we really need to write all these recommendations in a
     draft. Services that are peer-to-peer in nature would tend to use a
     single port.Same for QUIC stacks. It is a little bit easier to use a
     single socket than to force outgoing connections on a single socket. If
     we want them to use two separate ports, we have to explain convincingly.

     On the other hand, I think we are past the "port below 1024" rules. For
     example, memcached is on port 11211, MDNS on port 5353, and there are
     many more like that.

     >> but yes, keeping a dynamic list of most abused ports makes sense.
     >> Although that's mostly an issue for volumetric attacks, and host based
     >> defenses are not too efficient there, but still it feels better than 
just
     >> receiving the attack and waiting until it stops.
     > Definitely.
     >
     >>>>> I think we'll discover new fun things over time and will learn new
     >>>>> attacks and workarounds as deployments grow.
     >>>> I have not heard of actual attacks "in the wild", but who knows...
     >>> It's too early, stacks are still evolving and the level of deployment
     >>> is not high yet. That doesn't mean nobody is trying to elaborate new
     >>> attacks. For example yesterday on my home servers which do not 
advertise
     >>> QUIC I received one short packet from port 443 to port 443 from a host
     >>> named "scanners.labs.something", and a 1238-byte one from a random port
     >>> to 443 from a host named "scan-19a.somethingelse". Thus it shows that
     >>> studies are already in progress. We just need to be patient :-)
     >>
     >> We probably will not have to wait too long...
     > I hope so. It's important to get real-world feedback on protocols early
     > enough to allow them to improve. It's especially important right now that
     > any QUIC site can fall back to H1/H2 over TCP so it's easy to just block
     > it in case of massive failure. If big problems are reported only 10 years
     > after deployment it will be much harder to disable it in emergency.

     Yes, but out of our control...

     -- Christian Huitema


Reply via email to