Re: Fw: [Fwd: [Bug 5644] New: NFS v3 TCP 3-way handshake incorrect, iptables blocks access]
Hi,
On Fri, 25 Nov 2005, Jozsef Kadlecsik wrote:
On Thu, 24 Nov 2005, Olaf Kirch wrote:
On Thu, Nov 24, 2005 at 03:08:27PM +0100, Harald Welte wrote:
Jozsef Kadlecsik doesn't recall those patches/changes (even though he's
our Mr. TCP state tracking and is indicated as the author of one of
the two patches.
I also don't recall having seen any of those patches before. But that
doesn't mean all too much, my brain is like a sieve some times.
Those patches came out of a discussion on netfilter-devel. Sorry,
I don't know exactly when but looking at our CVS log it was Dec 2004.
Yes, it was about a year ago - finally I could dig out the patches from my
mail archives. I'll prepare an updated version on the weekend and send it
out to netfilter-devel for reviewing.
Attached is the updated patch. Unfortunately nfsim currently is broken so
I could not test it against the testsuite, but actually it's identical
with the original, not applied patches. The patch takes care both
ip_conntrack and nf_conntrack.
Best regards,
Jozsef
-
E-mail : [EMAIL PROTECTED], [EMAIL PROTECTED]
PGP key : http://www.kfki.hu/~kadlec/pgp_public_key.txt
Address : KFKI Research Institute for Particle and Nuclear Physics
H-1525 Budapest 114, POB. 49, HungaryMounting NFS file systems after a (warm) reboot could take a long time if
firewalling and connection tracking was enabled.
The reason is that the NFS clients tends to use the same ports (800 and
counting down). Now on reboot, the server would still have a TCB for an
existing TCP connection client:800 - server:2049. The client sends a
SYN from port 800 to server:2049, which elicits an ACK from the server.
The firewall on the client drops the ACK because (from its point of
view) the connection is still in half-open state, and it expects to see
a SYNACK.
The client will eventually time out after several minutes.
The following patch corrects this, by accepting ACKs on half open connections
as well.
Signed-off-by: Jozsef Kadlecsik [EMAIL PROTECTED]
diff -urN --exclude-from=/usr/src/diff.exclude
linux-2.6.15-rc2-orig/net/ipv4/netfilter/ip_conntrack_proto_tcp.c
linux-2.6.15-rc2-tcp-win/net/ipv4/netfilter/ip_conntrack_proto_tcp.c
--- linux-2.6.15-rc2-orig/net/ipv4/netfilter/ip_conntrack_proto_tcp.c
2005-11-20 04:25:03.0 +0100
+++ linux-2.6.15-rc2-tcp-win/net/ipv4/netfilter/ip_conntrack_proto_tcp.c
2005-11-29 11:01:25.0 +0100
@@ -272,9 +272,9 @@
* sCL - sCL
*/
/* sNO, sSS, sSR, sES, sFW, sCW, sLA, sTW, sCL, sLI */
-/*ack*/ { sIV, sIV, sSR, sES, sCW, sCW, sTW, sTW, sCL, sIV },
+/*ack*/ { sIV, sIG, sSR, sES, sCW, sCW, sTW, sTW, sCL, sIV },
/*
- * sSS - sIV Might be a half-open connection.
+ * sSS - sIG Might be a half-open connection.
* sSR - sSR Might answer late resent SYN.
* sES - sES :-)
* sFW - sCW Normal close request answered by ACK.
@@ -917,8 +917,12 @@
switch (new_state) {
case TCP_CONNTRACK_IGNORE:
- /* Either SYN in ORIGINAL
-* or SYN/ACK in REPLY. */
+ /* Ignored packets:
+*
+* a) SYN in ORIGINAL
+* b) SYN/ACK in REPLY
+* c) ACK in reply direction after initial SYN in original.
+*/
if (index == TCP_SYNACK_SET
conntrack-proto.tcp.last_index == TCP_SYN_SET
conntrack-proto.tcp.last_dir != dir
@@ -985,13 +989,20 @@
}
case TCP_CONNTRACK_CLOSE:
if (index == TCP_RST_SET
-test_bit(IPS_SEEN_REPLY_BIT, conntrack-status)
-conntrack-proto.tcp.last_index == TCP_SYN_SET
+((test_bit(IPS_SEEN_REPLY_BIT, conntrack-status)
+ conntrack-proto.tcp.last_index == TCP_SYN_SET)
+ || (!test_bit(IPS_ASSURED_BIT, conntrack-status)
+conntrack-proto.tcp.last_index == TCP_ACK_SET))
ntohl(th-ack_seq) == conntrack-proto.tcp.last_end) {
- /* RST sent to invalid SYN we had let trough
-* SYN was in window then, tear down connection.
+ /* RST sent to invalid SYN or ACK we had let trough
+* at a) and c) above:
+*
+* a) SYN was in window then
+* c) we hold a half-open connection.
+*
+* Delete our connection entry.
* We skip window checking, because packet might ACK
-* segments we ignored in the SYN. */
+* segments we ignored. */
goto in_window;
}
/* Just fall trough */
diff -urN --exclude-from=/usr/src/diff.exclude
Re: Fw: [Fwd: [Bug 5644] New: NFS v3 TCP 3-way handshake incorrect, iptables blocks access]
Jozsef Kadlecsik wrote: Mounting NFS file systems after a (warm) reboot could take a long time if firewalling and connection tracking was enabled. The reason is that the NFS clients tends to use the same ports (800 and counting down). Now on reboot, the server would still have a TCB for an existing TCP connection client:800 - server:2049. The client sends a SYN from port 800 to server:2049, which elicits an ACK from the server. The firewall on the client drops the ACK because (from its point of view) the connection is still in half-open state, and it expects to see a SYNACK. The client will eventually time out after several minutes. The following patch corrects this, by accepting ACKs on half open connections as well. Thanks Jozsef, I'll pass it on to Dave tommorrow. - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Fw: [Fwd: [Bug 5644] New: NFS v3 TCP 3-way handshake incorrect, iptables blocks access]
On Wed, Nov 23, 2005 at 02:44:19PM -0800, David S. Miller wrote: Please make sure this gets looked at, and at least reviewed. Jozsef Kadlecsik doesn't recall those patches/changes (even though he's our Mr. TCP state tracking and is indicated as the author of one of the two patches. I also don't recall having seen any of those patches before. But that doesn't mean all too much, my brain is like a sieve some times. Jozsef is most familiar with that code, he said he'll cook up a new patch and do nfsim testing over the next days. Also, it was indicated that those fixes did go mainline at some point? Can soembody please indicate when that was supposed to happen? I don't really recall any of this, and I would be surprised if we deliberately backed out such changes at a later point. Thanks! -- - Harald Welte [EMAIL PROTECTED] http://netfilter.org/ Fragmentation is like classful addressing -- an interesting early architectural error that shows how much experimentation was going on while IP was being designed.-- Paul Vixie pgpQbFIBHhzlb.pgp Description: PGP signature
Re: Fw: [Fwd: [Bug 5644] New: NFS v3 TCP 3-way handshake incorrect, iptables blocks access]
On Thu, Nov 24, 2005 at 03:08:27PM +0100, Harald Welte wrote: Jozsef Kadlecsik doesn't recall those patches/changes (even though he's our Mr. TCP state tracking and is indicated as the author of one of the two patches. I also don't recall having seen any of those patches before. But that doesn't mean all too much, my brain is like a sieve some times. Those patches came out of a discussion on netfilter-devel. Sorry, I don't know exactly when but looking at our CVS log it was Dec 2004. Also, it was indicated that those fixes did go mainline at some point? Can soembody please indicate when that was supposed to happen? I don't really recall any of this, and I would be surprised if we deliberately backed out such changes at a later point. I wasn't involved in the netfilter discussion; I just did the initial debugging. But it seems the first patch went into 2.6.10, the other into 2.6.11-rc1. Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play [EMAIL PROTECTED] |/ | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
[Fwd: [Bug 5644] New: NFS v3 TCP 3-way handshake incorrect, iptables blocks access]
Sorry to be cross-posting, but does this bug ring any bells? I'm having trouble seeing how the sunrpc server code could be at fault. Cheers, Trond ---BeginMessage--- http://bugzilla.kernel.org/show_bug.cgi?id=5644 Summary: NFS v3 TCP 3-way handshake incorrect, iptables blocks access Kernel Version: 2.6.14 Status: NEW Severity: blocking Owner: [EMAIL PROTECTED] Submitter: [EMAIL PROTECTED] Most recent kernel where this bug did not occur: Distribution: Can't remember, possibly FC2. Hardware Environment: Software Environment: Problem Description: Steps to reproduce: 1. Boot NFS v3 TCP client running iptables mount NFS filesystem 2. Do a normal NFS client reboot try mounting the same filesystem again 3. Experience intermittent failure to read superblock The cause of this problem is NFS server's improper response to SYN packet sent by the client. This occurs *after* successful client authorization, when the client tries to open the connection (i.e. sends SYN to the server's nfs port) to read the superblock. The server (sometimes) responds with a pure ACK without the SYN bit set. This is blocked by iptables -- thus, mount fails with a could not read superblock message. Here is an excerpt from ethereal log: 3 0.021733client SERVER TCP 800 nfs [SYN] Seq=0 Ack=0 Win=5840 Len=0 MSS=1460 TSV=24095 TSER=0 WS=2 4 0.021846SERVER client TCP nfs 800 [ACK] Seq=9138391 Ack=3580883479 Win=16022 Len=0 TSV=244936050 TSER=1149400 5 0.021864client SERVER ICMP Destination unreachable (Host administratively prohibited) The above problem occurs with a very simple default+ssh iptables configuration. Disabling iptables on the client makes the problem go away. Even with iptables active, there is no problem when nfsd responds with a proper [SYN,ACK] instead of just pure ACK (this happens intermittently after the client reboot). Please fix nfsd so that it reliably responds to SYN packets with proper [SYN,ACK] packets instead of just ACK packets. Apparently, nfsd state doesn't get properly reset on client reboots. Other people have reported autofs failures which may be related (e.g. on remounts). --- You are receiving this mail because: --- You are the assignee for the bug, or are watching the assignee. ---End Message---
Re: [Fwd: [Bug 5644] New: NFS v3 TCP 3-way handshake incorrect, iptables blocks access]
On Wed, Nov 23, 2005 at 12:53:43PM -0500, Trond Myklebust wrote: Sorry to be cross-posting, but does this bug ring any bells? I'm having trouble seeing how the sunrpc server code could be at fault. We've seen this previously, and submitted a fix to netfilter which supposedly went into mainline at some point. It seems to be gone from 2.6.14 though. The problem is with conntrack, and filtering on RELATED (I assume your netfilter config does that) What happens is that the client reboots, opens a new TCP connection with the same port as last time (say 800), sends SYN. Server still has an active TCB for this, and thus replies with an ACK containing its current sequence numbers. Now the client is supposed to RST the connection. Unfortunately, conntrack does not expect a lone ACK in this state and ignores it. So the client will retransmit the SYN until timeout. Then it picks a new port, and succeeds (maybe). Olaf -- Olaf Kirch | --- o --- Nous sommes du soleil we love when we play [EMAIL PROTECTED] |/ | \ sol.dhoop.naytheet.ah kin.ir.samse.qurax - To unsubscribe from this list: send the line unsubscribe netdev in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordomo-info.html
