Re: Ephemeral port range (patch)
At 03:37 a.m. 04/03/2008, Mike Silbersack wrote:
While I haven't look match at the scheme proposed by Amit, I think
there's a "flaw" with the algorithm: IP IDs need to be unique for
{source IP, des IP, Protocol}. And the algorithm still keeps a
*global* IP ID. That means you'll cycle through the whole IP ID
space when you probably didn't need to.
That is true. I think we have a time/space tradeoff here, with
Amit's algorithm taking more memory and less time than a hash-based
algorithm. But I haven't benchmarked one against the other, so it is
possible that a double-hash might win in both categories.
(Thinking out loud)
Note that in the case of implementing the double-hash scheme for
connection-oriented protocols, once you compute the hash for the
first IP ID to be used for a connection, you could store the result
of the hash in the TCB, and thus you wouldn't need to recompute this
"expensive" hash every time you send a packet.
I think Robert Watson said something about investigating the issue
of IP IDs more in the near future. What I'd like to see (if
possible) is that we use Amit's algorithm until we've established a
connection with a host, then switch to per-IP state and just use
linear IP IDs. That would seem to provide the least overhead for
high speed connections.
I haven't yet looked that much at Amit's approach but, from what I
have seen, your suggestion makes sense.
That said, at least theoretically speaking, one could argue that
there shouldn't be a problem with simply randomizing the IP ID
number. For connection-oriented protocols, you should be doing
PMTUD, and thus will not care about the IP ID. If your packets are
doing fragmentation, then on links will large bandwidth-delay
products you're already in trouble. For connection-less transport
protocols (e.g., UDP), while they usually do not implement PMTUD,
they also do not implement flow-control or congestion control. So
you are either sending data to a local system (e.g., in a LAN), or
you probably shouldn't be sending data that fast (and then you
shouldn't have problems with trivially randomizing the IP ID).
I have attempted to make that argument before, and it did not go
over well with most people. :)
I think the counter-argument was primarily centered around UDP NFS,
which, as you pointed out, is almost always a losing case.
Relying on IP fragmentation for anything that is supposed to be
reliable and that should work at high speed is...mmm... probably not
the best idea. ;-) Other than the classic "fragmentation considered
harmful", there's a more recent id (RFC?) entitled "fragmentation
considered very harmful" which shows the problems that may arise due
to fragmentation.
So the thing here is that people want to do the wrong thing, and then
blame the IP ID generator. ;-)
The double-hash concept sounds pretty good, but there's a major
problem with it. If an application does a bind() to get a local
port before doing a connect(), you don't know the remote IP or the remote port.
Yes, this is described in Section 3.5 of our id
(http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt).
Our take is that in that scenario you could simply randomize the
local port. (i.e., implement the double-hash scheme, and fall-back
to trivial randomization when you face this scenario).
Doh, I will try to read the ENTIRE paper next time before commenting.
No worries.
There's a related "feature" in the BSD TCP stack that all local
ports are considered equal; even for applications that do a
connect() call and specify a remote IP/port, we do not let them
use the same local port to two different remote IPs at the same
time. This puts a limit on the total number of outgoing
connections that one machine can have.
mmm... I see. So this could limit the number of outgoing
connections to about (ephemeral_ports/TIME_WAIT). Any objections
against changing this? At least for outgoing connections (i.e.,
non-listening sockets), this shouldn't be the case. I'd be
interested in working on this issue...
I don't think anyone is actively working on that problem, so you
won't be stepping on anyone's toes by looking into it. Bring on the patches!
Great! Will do.
There's a piece of low hanging fruit also in that area - we add
incoming connections to the local port hash table, even though it
seems unlikely that you are going to receive a connection from
1.1.1.1:5->1.1.1.2:80 and then connect from
1.1.1.2:80->1.1.1.1:5. Those unnecessary additions to the local
port hash time would be nice to remove if you're investigating the
related issues.
Ok.
One thing you may or may not have noticed is that FreeBSD keeps
TIME_WAIT sockets in a seperate zone which has a limit size, so you
will not have to worry too much about them clogging up all ephemeral ports.
I had not... but will have a look at it.
Thanks!
--
Fernando Gont
e-mail: [EMAIL PROTECTED] |
Re: Ephemeral port range (patch)
On Mon, 3 Mar 2008, Fernando Gont wrote:
(Shame on me... somehow you mail got stuck in my queue, and I didn't respond
to it).
No sweat, I've taken far longer to reply to your e-mails!
While I haven't look match at the scheme proposed by Amit, I think there's a
"flaw" with the algorithm: IP IDs need to be unique for {source IP, des IP,
Protocol}. And the algorithm still keeps a *global* IP ID. That means you'll
cycle through the whole IP ID space when you probably didn't need to.
That is true. I think we have a time/space tradeoff here, with Amit's
algorithm taking more memory and less time than a hash-based algorithm.
But I haven't benchmarked one against the other, so it is possible that a
double-hash might win in both categories.
I think Robert Watson said something about investigating the issue of IP
IDs more in the near future. What I'd like to see (if possible) is that
we use Amit's algorithm until we've established a connection with a host,
then switch to per-IP state and just use linear IP IDs. That would seem
to provide the least overhead for high speed connections.
That said, at least theoretically speaking, one could argue that there
shouldn't be a problem with simply randomizing the IP ID number. For
connection-oriented protocols, you should be doing PMTUD, and thus will not
care about the IP ID. If your packets are doing fragmentation, then on links
will large bandwidth-delay products you're already in trouble. For
connection-less transport protocols (e.g., UDP), while they usually do not
implement PMTUD, they also do not implement flow-control or congestion
control. So you are either sending data to a local system (e.g., in a LAN),
or you probably shouldn't be sending data that fast (and then you shouldn't
have problems with trivially randomizing the IP ID).
I have attempted to make that argument before, and it did not go over well
with most people. :)
I think the counter-argument was primarily centered around UDP NFS, which,
as you pointed out, is almost always a losing case.
The double-hash concept sounds pretty good, but there's a major problem
with it. If an application does a bind() to get a local port before doing
a connect(), you don't know the remote IP or the remote port.
Yes, this is described in Section 3.5 of our id
(http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt).
Our take is that in that scenario you could simply randomize the local port.
(i.e., implement the double-hash scheme, and fall-back to trivial
randomization when you face this scenario).
Doh, I will try to read the ENTIRE paper next time before commenting.
There's a related "feature" in the BSD TCP stack that all local ports are
considered equal; even for applications that do a connect() call and
specify a remote IP/port, we do not let them use the same local port to two
different remote IPs at the same time. This puts a limit on the total
number of outgoing connections that one machine can have.
mmm... I see. So this could limit the number of outgoing connections to about
(ephemeral_ports/TIME_WAIT). Any objections against changing this? At least
for outgoing connections (i.e., non-listening sockets), this shouldn't be the
case. I'd be interested in working on this issue...
I don't think anyone is actively working on that problem, so you won't be
stepping on anyone's toes by looking into it. Bring on the patches!
There's a piece of low hanging fruit also in that area - we add incoming
connections to the local port hash table, even though it seems unlikely
that you are going to receive a connection from 1.1.1.1:5->1.1.1.2:80
and then connect from 1.1.1.2:80->1.1.1.1:5. Those unnecessary
additions to the local port hash time would be nice to remove if you're
investigating the related issues.
One thing you may or may not have noticed is that FreeBSD keeps TIME_WAIT
sockets in a seperate zone which has a limit size, so you will not have to
worry too much about them clogging up all ephemeral ports.
-Mike
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
At 04:43 a.m. 03/03/2008, Mike Silbersack wrote:
Earlier in the week, I had commented (via private e-mail?) that I
thought that Amit Klein's algorithm which I recently implemented in
ip_id.c might be adapted to serve as an ephemeral port
allocator. Now that I've thought more about it, I'm not as certain
that it would fit well. I'll try to sketch out my ideas and see if
I can figure out how it could fit.
(Shame on me... somehow you mail got stuck in my queue, and I didn't
respond to it).
While I haven't look match at the scheme proposed by Amit, I think
there's a "flaw" with the algorithm: IP IDs need to be unique for
{source IP, des IP, Protocol}. And the algorithm still keeps a
*global* IP ID. That means you'll cycle through the whole IP ID space
when you probably didn't need to.
Here, two, a double-hash based scheme (a la RFC1948) will do. It
would basically separate the IP ID space for every {source IP, dest
IP, Protocol} tuple, and thus you'll cycle through the IP ID space
only as fast as needed.
What's interesting is that when it comes to port randomization, IP ID
randomization, and even timestamp randomization, the double-hash
scheme seems to be the right solution.
That said, at least theoretically speaking, one could argue that
there shouldn't be a problem with simply randomizing the IP ID
number. For connection-oriented protocols, you should be doing PMTUD,
and thus will not care about the IP ID. If your packets are doing
fragmentation, then on links will large bandwidth-delay products
you're already in trouble. For connection-less transport protocols
(e.g., UDP), while they usually do not implement PMTUD, they also do
not implement flow-control or congestion control. So you are either
sending data to a local system (e.g., in a LAN), or you probably
shouldn't be sending data that fast (and then you shouldn't have
problems with trivially randomizing the IP ID).
The double-hash concept sounds pretty good, but there's a major
problem with it. If an application does a bind() to get a local
port before doing a connect(), you don't know the remote IP or the remote port.
Yes, this is described in Section 3.5 of our id
(http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt).
Our take is that in that scenario you could simply randomize the
local port. (i.e., implement the double-hash scheme, and fall-back to
trivial randomization when you face this scenario).
There's a related "feature" in the BSD TCP stack that all local
ports are considered equal; even for applications that do a
connect() call and specify a remote IP/port, we do not let them use
the same local port to two different remote IPs at the same
time. This puts a limit on the total number of outgoing connections
that one machine can have.
mmm... I see. So this could limit the number of outgoing connections
to about (ephemeral_ports/TIME_WAIT). Any objections against changing
this? At least for outgoing connections (i.e., non-listening
sockets), this shouldn't be the case. I'd be interested in working on
this issue...
Kind regards,
--
Fernando Gont
e-mail: [EMAIL PROTECTED] || [EMAIL PROTECTED]
PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1
___
freebsd-net@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-net
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
On Sat, 1 Mar 2008, Fernando Gont wrote: I will also start working on the double-hash ephemeral port selection algorithm described in the draft (this is, IMHO, the right approach to ephemeral port randomization) Kind regards, -- Fernando Gont Earlier in the week, I had commented (via private e-mail?) that I thought that Amit Klein's algorithm which I recently implemented in ip_id.c might be adapted to serve as an ephemeral port allocator. Now that I've thought more about it, I'm not as certain that it would fit well. I'll try to sketch out my ideas and see if I can figure out how it could fit. The double-hash concept sounds pretty good, but there's a major problem with it. If an application does a bind() to get a local port before doing a connect(), you don't know the remote IP or the remote port. There's a related "feature" in the BSD TCP stack that all local ports are considered equal; even for applications that do a connect() call and specify a remote IP/port, we do not let them use the same local port to two different remote IPs at the same time. This puts a limit on the total number of outgoing connections that one machine can have. -Mike ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
At 09:49 p.m. 02/03/2008, you wrote: +1 on increasing the threshold, 1024 is way too low. With the current patch, I agree. I'm planning to implement the scheme described in the port randomization internt-draft I referenced, and implement the array-of-bits thing. That way you can exclude whichever ports you want, without "wasting" the 1024- port range. Also consider the folk who depend on the existing behaviour: a predictable ephemeral port range is useful, if for some reason you need to apply a NAT policy to that traffic, with no other knowledge about how the applications you must NAT actually behave. You can still set porthi or portlow to select whichever port range you want. The patch just changes the default case. As noted in one of the sections of the draft I referenced, turns put that each TCP/IP stack chooses its own range for the ephemeral ports. So unless you're tweaking the configuration of each of the systems you have behind the NAT, I'm afraid you won't be able to implement such a policy. FWIW, Windows used the range 1024-4999 or something... at least W95 and XP. Vista probably still does the same thing. Kind regards, -- Fernando Gont e-mail: [EMAIL PROTECTED] || [EMAIL PROTECTED] PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1 ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
+1 on increasing the threshold, 1024 is way too low. Also consider the folk who depend on the existing behaviour: a predictable ephemeral port range is useful, if for some reason you need to apply a NAT policy to that traffic, with no other knowledge about how the applications you must NAT actually behave. later BMS ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
Mike Silbersack wrote: On Sat, 1 Mar 2008, Fernando Gont wrote: Folks, This patch changes the default ephemeral port range from 49152-65535 to 1024-65535. This makes it harder for an attacker to guess the ephemeral ports (as the port number space is larger). Also, it makes the chances of port number collisions smaller. (http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt) There are a number of commonly used ports above 1000, such as nfs and x11. I think OpenBSD uses 1-65535, maybe that's a safer choice to go with. Agreed about 1-65535. -- Andre ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
On Sat, 1 Mar 2008, Mike Silbersack wrote: On Sat, 1 Mar 2008, Fernando Gont wrote: This patch changes the default ephemeral port range from 49152-65535 to 1024-65535. This makes it harder for an attacker to guess the ephemeral ports (as the port number space is larger). Also, it makes the chances of port number collisions smaller. (http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt) There are a number of commonly used ports above 1000, such as nfs and x11. I think OpenBSD uses 1-65535, maybe that's a safer choice to go with. In order to get acceptable open connection counts with 10gbps ethernet, I've needed to run with a significantly lower starting portrange. In practice, the following seems to do the trick for me: sysctl net.inet.ip.portrange.first=1 Of course, I only run into this if I also increase maxsockets: sysctl kern.ipc.maxsockets=3 Lowering the lower end of the ephemeral range to 10,000 would do the trick for me, anyway. Robert N M Watson Computer Laboratory University of Cambridge ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
At 08:42 p.m. 01/03/2008, Kevin Oberman wrote: > This patch changes the default ephemeral port range from 49152-65535 > to 1024-65535. This makes it harder for an attacker to guess the > ephemeral ports (as the port number space is larger). Also, it makes > the chances of port number collisions smaller. > (http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt) > > This patch also includes my previous patch that eliminated duplicated > code in in_pcb_bind(). The idea is good, but 1024 is way too low. Things like rpc and the like use ports well above 1024. Notably, 6000 and above are used by X. Maybe 1 would be OK. Maybe not, though. I see that gnuserv and gkrellmd both use ports about 1000. (gnuserv uses 30871 and gkrellmd uses 19150.) Other UNIX-like systems use that "low" port range. e.g., OpenBSD uses the range 1024-49151. The idea is would be to define a bit string in which you can specify those ports that should not be used as ephemeral ports (I will send this patch soon). (This is described in the IETF internet-draft I referenced, too). I will also start working on the double-hash ephemeral port selection algorithm described in the draft (this is, IMHO, the right approach to ephemeral port randomization) Kind regards, -- Fernando Gont e-mail: [EMAIL PROTECTED] || [EMAIL PROTECTED] PGP Fingerprint: 7809 84F5 322E 45C7 F1C9 3945 96EE A9EF D076 FFF1 ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Ephemeral port range (patch)
> Date: Sat, 01 Mar 2008 11:34:27 -0200 > From: Fernando Gont <[EMAIL PROTECTED]> > Sender: [EMAIL PROTECTED] > > Folks, > > This patch changes the default ephemeral port range from 49152-65535 > to 1024-65535. This makes it harder for an attacker to guess the > ephemeral ports (as the port number space is larger). Also, it makes > the chances of port number collisions smaller. > (http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt) > > This patch also includes my previous patch that eliminated duplicated > code in in_pcb_bind(). The idea is good, but 1024 is way too low. Things like rpc and the like use ports well above 1024. Notably, 6000 and above are used by X. Maybe 1 would be OK. Maybe not, though. I see that gnuserv and gkrellmd both use ports about 1000. (gnuserv uses 30871 and gkrellmd uses 19150.) -- R. Kevin Oberman, Network Engineer Energy Sciences Network (ESnet) Ernest O. Lawrence Berkeley National Laboratory (Berkeley Lab) E-mail: [EMAIL PROTECTED] Phone: +1 510 486-8634 Key fingerprint:059B 2DDF 031C 9BA3 14A4 EADA 927D EBB3 987B 3751 pgpcoGVxWd5BV.pgp Description: PGP signature
Re: Ephemeral port range (patch)
On Sat, 1 Mar 2008, Fernando Gont wrote: Folks, This patch changes the default ephemeral port range from 49152-65535 to 1024-65535. This makes it harder for an attacker to guess the ephemeral ports (as the port number space is larger). Also, it makes the chances of port number collisions smaller. (http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt) There are a number of commonly used ports above 1000, such as nfs and x11. I think OpenBSD uses 1-65535, maybe that's a safer choice to go with. -Mike ___ freebsd-net@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-net To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Ephemeral port range (patch)
Folks,
This patch changes the default ephemeral port range from 49152-65535
to 1024-65535. This makes it harder for an attacker to guess the
ephemeral ports (as the port number space is larger). Also, it makes
the chances of port number collisions smaller.
(http://www.ietf.org/internet-drafts/draft-ietf-tsvwg-port-randomization-01.txt)
This patch also includes my previous patch that eliminated duplicated
code in in_pcb_bind().
Index: in.h
===
RCS file: /home/ncvs/src/sys/netinet/in.h,v
retrieving revision 1.100
diff -u -r1.100 in.h
--- in.h12 Jun 2007 16:24:53 - 1.100
+++ in.h1 Mar 2008 09:00:10 -
@@ -293,8 +293,7 @@
*
* The value IP_PORTRANGE_HIGH changes the range of candidate port numbers
* into the "high" range. These are reserved for client outbound connections
- * which do not want to be filtered by any firewalls. Note that by default
- * this is the same as IP_PORTRANGE_DEFAULT.
+ * which do not want to be filtered by any firewalls.
*
* The value IP_PORTRANGE_LOW changes the range to the "low" are
* that is (by convention) restricted to privileged processes. This
@@ -331,8 +330,13 @@
#defineIPPORT_RESERVED 1024
/*
- * Default local port range, used by both IP_PORTRANGE_DEFAULT
- * and IP_PORTRANGE_HIGH.
+ * Default local port range, used by IP_PORTRANGE_DEFAULT
+ */
+#define IPPORT_EPHEMERALFIRST 1024
+#define IPPORT_EPHEMERALLAST 65535
+
+/*
+ * Dynamic port range, used by IP_PORTRANGE_HIGH.
*/
#defineIPPORT_HIFIRSTAUTO 49152
#defineIPPORT_HILASTAUTO 65535
Index: in_pcb.c
===
RCS file: /home/ncvs/src/sys/netinet/in_pcb.c,v
retrieving revision 1.198
diff -u -r1.198 in_pcb.c
--- in_pcb.c22 Dec 2007 10:06:11 - 1.198
+++ in_pcb.c1 Mar 2008 09:00:11 -
@@ -89,8 +89,8 @@
*/
intipport_lowfirstauto = IPPORT_RESERVED - 1; /* 1023 */
intipport_lowlastauto = IPPORT_RESERVEDSTART; /* 600 */
-intipport_firstauto = IPPORT_HIFIRSTAUTO; /* 49152 */
-intipport_lastauto = IPPORT_HILASTAUTO; /* 65535 */
+intipport_firstauto = IPPORT_EPHEMERALFIRST; /* 1024 */
+intipport_lastauto = IPPORT_EPHEMERALLAST;/* 65535 */
intipport_hifirstauto = IPPORT_HIFIRSTAUTO;/* 49152 */
intipport_hilastauto = IPPORT_HILASTAUTO; /* 65535 */
@@ -393,7 +393,7 @@
if (*lportp != 0)
lport = *lportp;
if (lport == 0) {
- u_short first, last;
+ u_short first, last, aux;
int count;
if (laddr.s_addr != INADDR_ANY)
@@ -440,47 +440,28 @@
/*
* Simple check to ensure all ports are not used up causing
* a deadlock here.
-*
-* We split the two cases (up and down) so that the direction
-* is not being tested on each round of the loop.
*/
if (first > last) {
- /*
-* counting down
-*/
- if (dorandom)
- *lastport = first -
- (arc4random() % (first - last));
- count = first - last;
+ aux = first;
+ first = last;
+ last = aux;
+ }
- do {
- if (count-- < 0)/* completely used? */
- return (EADDRNOTAVAIL);
- --*lastport;
- if (*lastport > first || *lastport < last)
- *lastport = first;
- lport = htons(*lastport);
- } while (in_pcblookup_local(pcbinfo, laddr, lport,
- wild));
- } else {
- /*
-* counting up
-*/
- if (dorandom)
- *lastport = first +
- (arc4random() % (last - first));
- count = last - first;
+ if (dorandom)
+ *lastport = first +
+ (arc4random() % (last - first));
- do {
- if (count-- < 0)/* completely used? */
- return (EADDRNOTAVAIL);
- ++*lastport;
- if (*lastport < first || *lastport > last)
- *lastport = first;
- lport = htons(*lastport);
-
