+1
Thanks for the tip, this looks very useful.
Looks like it was only introduced in 2.6.35, we're still on 2.6.32 ...
might be worth the upgrade, it just takes so long to test new kernel
versions in this application.
We ended up dropping TCP_TIMEWAIT_LEN to 30 seconds as a band-aid for
now, alon
On Thu, Dec 06, 2012 at 08:58:10AM -0500, Ray Soucy wrote:
> > net.ipv4.tcp_keepalive_intvl = 15
> > net.ipv4.tcp_keepalive_probes = 3
> > net.ipv4.tcp_keepalive_time = 90
> > net.ipv4.tcp_fin_timeout = 30
>
> As discussed, those do not affect TCP_TIMEWAIT_LEN.
>
> There is a lot of misinformatio
This issue is for really for connections that close properly and
without any issue.
The application closes the socket and doesn't care about it; but the
OS keeps it in the TIME_WAIT state as required by the RFC for TCP in
case data tries to be sent after the connection has closed (out of
order tra
Question:
If a TCP connection is left hanging and continues to hoard the port for
some time before it times out, shouldn't the work to be focused on
finding out why the connection is not properly closed instead of trying
to support a greater number of hung connections waiting to time out ?
On 12/6/12 10:20 AM, Kyrian wrote:
Also, if you are going to hack the kernel to make that change, I urge you to
make it part of the sysctl mechanism as well, and to send a patch back to the
kernel developers to help out others who might be in a similar situation to
you. This is both to help
th
Quoting Ray Soucy :
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_keepalive_probes = 3
net.ipv4.tcp_keepalive_time = 90
net.ipv4.tcp_fin_timeout = 30
As discussed, those do not affect TCP_TIMEWAIT_LEN.
There is a lot of misinformation out there on this subject so please
don't just Google for
> net.ipv4.tcp_keepalive_intvl = 15
> net.ipv4.tcp_keepalive_probes = 3
> net.ipv4.tcp_keepalive_time = 90
> net.ipv4.tcp_fin_timeout = 30
As discussed, those do not affect TCP_TIMEWAIT_LEN.
There is a lot of misinformation out there on this subject so please
don't just Google for 5 min. and chim
This tunes conntrack, not local TCP on the server itself.
On Wed, Dec 5, 2012 at 4:18 PM, Cyril Bouthors wrote:
> On 5 Dec 2012, r...@maine.edu wrote:
>
>> Where there is no way to change this though /proc
>
> 10:17PM lenovo:~% sudo sysctl -a |grep wait
> net.netfilter.nf_conntrack_tcp_timeout_f
It does require a fixed source address. The box is also a router and
firewall, so it has many IP addresses available to it.
On Wed, Dec 5, 2012 at 5:24 PM, William Herrin wrote:
> On Wed, Dec 5, 2012 at 5:01 PM, Mark Andrews wrote:
>> In message
>> ,
>> William Herrin writes:
>>> The thing is
On 5 Dec 2012, r...@maine.edu wrote:
> Where there is no way to change this though /proc
...
Those netfilter connection tracking tunables have nothing to do with the
kernel's TCP socket handling.
No, but these do...
net.ipv4.tcp_keepalive_intvl = 15
net.ipv4.tcp_keepalive_probes = 3
net.i
On Wed, Dec 5, 2012 at 7:49 PM, Mark Andrews wrote:
> counter: dest address in hex
>
> 16376: 1a003ff9
> 16377: 1a003ffa
> bind: before bind: Can't assign requested address
> 16378: 1a003ffb
> connect: Can't assign requested address
> bind: before bind: Can't assign requested address
>
> and if yo
In message <20121206004909.b302f2ca2...@drugs.dv.isc.org>, Mark Andrews writes:
>
> In message <201212052325.qb5nprze005...@xs8.xs4all.nl>, "Miquel van
> Smoorenburg"
> writes:
> > In article you write:
> > >
> > >In message
> > > > m>,
> > > William Herrin writes:
> > >> The thing is, Linux
On Wed, 5 Dec 2012, Cyril Bouthors wrote:
On 5 Dec 2012, r...@maine.edu wrote:
Where there is no way to change this though /proc
10:17PM lenovo:~% sudo sysctl -a |grep wait
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfi
In message <201212052325.qb5nprze005...@xs8.xs4all.nl>, "Miquel van Smoorenburg"
writes:
> In article you write:
> >
> >In message m>,
> > William Herrin writes:
> >> The thing is, Linux doesn't behave quite that way.
> >>
> >> If you do an anonymous connect(), that is you socket() and then
>
In article you write:
>
>In message
>,
> William Herrin writes:
>> The thing is, Linux doesn't behave quite that way.
>>
>> If you do an anonymous connect(), that is you socket() and then
>> connect() without a bind() in the middle, then the limit applies *per
>> destination IP:port pair*. So, y
On Dec 5, 2012, at 2:06 PM, Fred Baker (fred) wrote:
> If you want to get into software rewriting, the simplest thing I might come
> up with would be to put TCBs in some form of LRU list and, at a point where
> you need a port back, close the TCB that least recently did anything. My
> understan
Mark Andrews writes:
>
> In message >,
> William Herrin writes:
> > On Wed, Dec 5, 2012 at 5:01 PM, Mark Andrews wrote:
> > > In message .c
> > om>,
> > > William Herrin writes:
> > >> The thing is, Linux doesn't behave quite that way.
> > >>
> > >> If you do an anonymous connect(), that is
In message ,
William Herrin writes:
> On Wed, Dec 5, 2012 at 5:01 PM, Mark Andrews wrote:
> > In message om>,
> > William Herrin writes:
> >> The thing is, Linux doesn't behave quite that way.
> >>
> >> If you do an anonymous connect(), that is you socket() and then
> >> connect() without a bi
On Wed, 5 Dec 2012, Ray Soucy wrote:
> My question is basically, "how low can you go?"
>
> There seems to be consensus around 20 seconds being safe,
> 15 being a 99% OK, and 10 or less being problematic.
I'm trying to imagine how even 10 could be problematic nowadays. Have you
found people repo
On Wed, Dec 5, 2012 at 5:01 PM, Mark Andrews wrote:
> In message
> ,
> William Herrin writes:
>> The thing is, Linux doesn't behave quite that way.
>>
>> If you do an anonymous connect(), that is you socket() and then
>> connect() without a bind() in the middle, then the limit applies *per
>> de
On 5 Dec 2012, r...@maine.edu wrote:
> Where there is no way to change this though /proc
10:17PM lenovo:~% sudo sysctl -a |grep wait
net.netfilter.nf_conntrack_tcp_timeout_fin_wait = 120
net.netfilter.nf_conntrack_tcp_timeout_close_wait = 60
net.netfilter.nf_conntrack_tcp_timeout_time_wait = 120
If you want to get into software rewriting, the simplest thing I might come up
with would be to put TCBs in some form of LRU list and, at a point where you
need a port back, close the TCB that least recently did anything. My
understanding is that this was implemented 15 years ago to manage SYN a
In message ,
William Herrin writes:
> On Wed, Dec 5, 2012 at 2:55 PM, Ray Soucy wrote:
> > For each second that goes by you remove X addresses from the available
> > pool of ports for new connections for whatever the TCP_TIMEWAIT_LEN is
> > set to (60 seconds by default in Linux).
> >
> > In thi
On Wed, 5 Dec 2012, Ray Soucy wrote:
So if I rebuild the kernel to use a 20 second timeout, then that 3
port pool can sustain 1500, and a 6 port pool can sustain 3000
connections per second.
The software could be re-written to round-robin though IP addresses
for outgoing requests, but t
On Wed, Dec 5, 2012 at 2:55 PM, Ray Soucy wrote:
> For each second that goes by you remove X addresses from the available
> pool of ports for new connections for whatever the TCP_TIMEWAIT_LEN is
> set to (60 seconds by default in Linux).
>
> In this case it's making quick connections for HTTP requ
There is an extra 7 on that number, it was 48194 (was sitting on a
different PC so I typed it instead of copy-paste).
On Wed, Dec 5, 2012 at 1:58 PM, William Herrin wrote:
> On Wed, Dec 5, 2012 at 12:09 PM, Ray Soucy wrote:
>> Like most web traffic, the majority of these connections open and
>>
For each second that goes by you remove X addresses from the available
pool of ports for new connections for whatever the TCP_TIMEWAIT_LEN is
set to (60 seconds by default in Linux).
In this case it's making quick connections for HTTP requests (most of
which finish in less than a second).
Say you
On Dec 5, 2012, at 10:58 AM, William Herrin wrote:
> On Wed, Dec 5, 2012 at 12:09 PM, Ray Soucy wrote:
>> Like most web traffic, the majority of these connections open and
>> close in under a second. When we get to a point that there is enough
>> traffic from users behind the proxy to be gener
On Wed, Dec 5, 2012 at 12:09 PM, Ray Soucy wrote:
> Like most web traffic, the majority of these connections open and
> close in under a second. When we get to a point that there is enough
> traffic from users behind the proxy to be generating over 500 new
> outgoing connections per second, susta
You could simply add another IP address to the servers's source-
address pool, which effectively gives you another 32K (or whatever
value you have for the local port range) identifiers.
Owen
On Dec 5, 2012, at 7:59 AM, Ray Soucy wrote:
> RFC 793 arbitrarily defines 2MSL (how long to hold a sock
On 12/5/12 9:09 AM, Ray Soucy wrote:
This would be outgoing connections sourced from the IP of the proxy,
destined to whatever remote website (so 80 or 443) requested by the
user.
Essentially it's a modified Squid service that is used to filter HTTP
for CIPA compliance (required by the governme
This would be outgoing connections sourced from the IP of the proxy,
destined to whatever remote website (so 80 or 443) requested by the
user.
Essentially it's a modified Squid service that is used to filter HTTP
for CIPA compliance (required by the government) for keep children in
public schools
Ray,
> With a 60 second timeout on TIME_WAIT, local port identifiers are tied
> up from being used for new outgoing connections (in this case a proxy
> server). The default local port range on Linux can easily be
> adjusted; but even when bumped up to a range of 32K ports, the 60
> second timeou
RFC 793 arbitrarily defines 2MSL (how long to hold a socket in
TIME_WAIT state before cleaning up) as 4 min.
Linux is a little more reasonable in this and has it baked into the
source as 60 seconds in "/usr/src/linux/include/net/tcp.h":
#define TCP_TIMEWAIT_LEN (60*HZ)
Where there is no way to ch
34 matches
Mail list logo
