** Description changed:
- The following iptables connlimit rule can be breached
- with a multithreaded client and network device driver,
- due to a race in the conncount/connlimit code:
+ [Impact]
- # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
- -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
- -j DROP
+ * The iptables connection count/limit rules can be breached
+ with multithreaded network driver/server/client (common)
+ due to a race in the conncount/connlimit code.
- NOTE: Patches will be sent to the kernel-team mailing list
- and more details/testing will be provided later today.
+ * For example:
+
+ # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
+ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
+ -j DROP
+
+ * The fix is a backport from an upstream commit that resolves
+ the problem (plus dependencies for a cleaner backport) that
+ address the race condition:
+
+ commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage
+ collection confirm race").
+
+ [Test Case]
+
+ * Server-side: (relevant kernel side)
+ (limit TCP port 7777 to only 2000 connections)
+
+ # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
+ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
+ -j DROP
+
+ # ulimit -SHn 65000 # increase number of open files
+ # ruby server.rb # multi-threaded server
+
+ * Client-side:
+
+ # ulimit -SHn 65000
+ # ruby client.rb <server ip> <port> <target # connections> <# threads>
+ <test output>
+
+ * Results with Original kernel:
+ (client achieves target of 6000 connections > limit of 2000 connections)
+
+ # ruby client.rb 10.230.56.100 7777 6000 3
+ 1
+ 2
+ 3
+ <...>
+ 6000
+ Target reached. Thread finishing
+ 6001
+ Target reached. Thread finishing
+ 6002
+ Target reached. Thread finishing
+ Threads done. 6002 connections
+ press enter to exit
+
+ * Results with Modified kernel:
+ (client is limited to 2000 connections, and times out afterward)
+
+ # ruby client.rb 10.230.56.100 7777 6000 3
+ 1
+ 2
+ 3
+ <...>
+ 2000
+ <... blocks for a few minutes ...>
+ failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
+ failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
+ failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
+ Threads done. 2000 connections
+ press enter to exit
+
+ * Test cases possibly available upon request,
+ depending on original author's permission.
+
+ [Regression Potential]
+
+ * The patchset has been reviewed by a netfilter maintainer [1] in
+ stable mailing list, and was considered OK for 4.14, and that's
+ essentially the same backport for 4.15 and 4.4.
+
+ * The changes are limited to netfilter conncount/connlimit (names
+ change between older/newer kernel versions).
+
+ [Other Info]
+
+ * The backport for 4.14 [2] is applied as of 4.14.92.
+
+ [1] https://www.spinics.net/lists/stable/msg276883.html
+ [2] https://www.spinics.net/lists/stable/msg276910.html
** Description changed:
[Impact]
- * The iptables connection count/limit rules can be breached
- with multithreaded network driver/server/client (common)
- due to a race in the conncount/connlimit code.
+ * The iptables connection count/limit rules can be breached
+ with multithreaded network driver/server/client (common)
+ due to a race in the conncount/connlimit code.
- * For example:
+ * For example:
- # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
- -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
- -j DROP
+ # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
+ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
+ -j DROP
- * The fix is a backport from an upstream commit that resolves
- the problem (plus dependencies for a cleaner backport) that
- address the race condition:
+ * The fix is a backport from an upstream commit that resolves
+ the problem (plus dependencies for a cleaner backport) that
+ address the race condition:
- commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage
- collection confirm race").
+ commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage
+ collection confirm race").
[Test Case]
- * Server-side: (relevant kernel side)
- (limit TCP port 7777 to only 2000 connections)
+ * Server-side: (relevant kernel side)
+ (limit TCP port 7777 to only 2000 connections)
- # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
- -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
- -j DROP
+ # iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
+ -m connlimit --connlimit-above 2000 --connlimit-mask 0 \
+ -j DROP
- # ulimit -SHn 65000 # increase number of open files
- # ruby server.rb # multi-threaded server
+ # ulimit -SHn 65000 # increase number of open files
+ # ruby server.rb # multi-threaded server
- * Client-side:
+ * Client-side:
- # ulimit -SHn 65000
- # ruby client.rb <server ip> <port> <target # connections> <# threads>
- <test output>
+ # ulimit -SHn 65000
+ # ruby client.rb <server ip> <port> <target # connections> <# threads>
+ <test output>
- * Results with Original kernel:
- (client achieves target of 6000 connections > limit of 2000 connections)
+ * Results with Original kernel:
+ (client achieves target of 6000 connections > limit of 2000 connections)
- # ruby client.rb 10.230.56.100 7777 6000 3
- 1
- 2
- 3
- <...>
- 6000
- Target reached. Thread finishing
- 6001
- Target reached. Thread finishing
- 6002
- Target reached. Thread finishing
- Threads done. 6002 connections
- press enter to exit
+ # ruby client.rb 10.230.56.100 7777 6000 3
+ 1
+ 2
+ 3
+ <...>
+ 6000
+ Target reached. Thread finishing
+ 6001
+ Target reached. Thread finishing
+ 6002
+ Target reached. Thread finishing
+ Threads done. 6002 connections
+ press enter to exit
- * Results with Modified kernel:
- (client is limited to 2000 connections, and times out afterward)
+ * Results with Modified kernel:
+ (client is limited to 2000 connections, and times out afterward)
- # ruby client.rb 10.230.56.100 7777 6000 3
- 1
- 2
- 3
- <...>
- 2000
- <... blocks for a few minutes ...>
- failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
- failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
- failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
- Threads done. 2000 connections
- press enter to exit
+ # ruby client.rb 10.230.56.100 7777 6000 3
+ 1
+ 2
+ 3
+ <...>
+ 2000
+ <... blocks for a few minutes ...>
+ failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
+ failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
+ failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
+ Threads done. 2000 connections
+ press enter to exit
- * Test cases possibly available upon request,
- depending on original author's permission.
+ * Test cases possibly available upon request,
+ depending on original author's permission.
[Regression Potential]
- * The patchset has been reviewed by a netfilter maintainer [1] in
- stable mailing list, and was considered OK for 4.14, and that's
- essentially the same backport for 4.15 and 4.4.
+ * The patchset has been reviewed by a netfilter maintainer [1] in
+ stable mailing list, and was considered OK for 4.14, and that's
+ essentially the same backport for 4.15 and 4.4.
- * The changes are limited to netfilter conncount/connlimit (names
- change between older/newer kernel versions).
+ * The changes are limited to netfilter connlimit/conncount (names
+ change between older/newer kernel versions).
[Other Info]
-
- * The backport for 4.14 [2] is applied as of 4.14.92.
+
+ * The backport for 4.14 [2] is applied as of 4.14.92.
[1] https://www.spinics.net/lists/stable/msg276883.html
[2] https://www.spinics.net/lists/stable/msg276910.html
--
You received this bug notification because you are a member of Kernel
Packages, which is subscribed to linux in Ubuntu.
https://bugs.launchpad.net/bugs/1811094
Title:
iptables connlimit allows more connections than the limit when using
multiple CPUs
Status in linux package in Ubuntu:
Confirmed
Bug description:
[Impact]
* The iptables connection count/limit rules can be breached
with multithreaded network driver/server/client (common)
due to a race in the conncount/connlimit code.
* For example:
# iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
-m connlimit --connlimit-above 2000 --connlimit-mask 0 \
-j DROP
* The fix is a backport from an upstream commit that resolves
the problem (plus dependencies for a cleaner backport) that
address the race condition:
commit b36e4523d4d5 ("netfilter: nf_conncount: fix garbage
collection confirm race").
[Test Case]
* Server-side: (relevant kernel side)
(limit TCP port 7777 to only 2000 connections)
# iptables -A INPUT -p tcp -m tcp --syn --dport 7777 \
-m connlimit --connlimit-above 2000 --connlimit-mask 0 \
-j DROP
# ulimit -SHn 65000 # increase number of open files
# ruby server.rb # multi-threaded server
* Client-side:
# ulimit -SHn 65000
# ruby client.rb <server ip> <port> <target # connections> <# threads>
<test output>
* Results with Original kernel:
(client achieves target of 6000 connections > limit of 2000 connections)
# ruby client.rb 10.230.56.100 7777 6000 3
1
2
3
<...>
6000
Target reached. Thread finishing
6001
Target reached. Thread finishing
6002
Target reached. Thread finishing
Threads done. 6002 connections
press enter to exit
* Results with Modified kernel:
(client is limited to 2000 connections, and times out afterward)
# ruby client.rb 10.230.56.100 7777 6000 3
1
2
3
<...>
2000
<... blocks for a few minutes ...>
failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
failed to create connection: Connection timed out - connect(2) for
"10.230.56.100" port 7777
Threads done. 2000 connections
press enter to exit
* Test cases possibly available upon request,
depending on original author's permission.
[Regression Potential]
* The patchset has been reviewed by a netfilter maintainer [1] in
stable mailing list, and was considered OK for 4.14, and that's
essentially the same backport for 4.15 and 4.4.
* The changes are limited to netfilter connlimit/conncount (names
change between older/newer kernel versions).
[Other Info]
* The backport for 4.14 [2] is applied as of 4.14.92.
[1] https://www.spinics.net/lists/stable/msg276883.html
[2] https://www.spinics.net/lists/stable/msg276910.html
To manage notifications about this bug go to:
https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1811094/+subscriptions
--
Mailing list: https://launchpad.net/~kernel-packages
Post to : kernel-packages@lists.launchpad.net
Unsubscribe : https://launchpad.net/~kernel-packages
More help : https://help.launchpad.net/ListHelp
