Re: Supporting 4 way connections in LKSCTP
So, can I get confirmation that whether we can enhance to support the scenarios or any resolution on providing the correct routing? On Tue, Nov 26, 2013 at 9:03 AM, Sun Paul wrote: > Hi > > we have a problem on using LKSCTP to form a 4 ways multi-homing network. > > Configuration > - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), > IP-B (eth2) > - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), > IP-Y (eth2) > > the four way paths are shown below. > 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) > 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) > 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) > 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) > > the HB/HB_ACK is normal for the paths " IP-A to IP-X" and "IP-B to > IP-Y", but it is not correct for the rest of two. > > First of all, we are using iproute2 to form 2 table such that when > IP-B arrives on IP-X, it will know how to route back to IP-B on the > same interface, i.e (eth1). Same logic for the path "IP-A to IP-X". > > What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, > LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not > using the IP 11.1.1.11. > > The above operation makes the subsequence HB/HB_ACK in using wrong IP address. > > TCP trace on eth1 > 18:02:41.058640 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] > [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 18:02:41.061634 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [COOKIE ECHO] > 18:02:41.062642 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.062846 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.361811 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.661791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.961791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > > TCP trace on eth2 > 18:02:41.058755 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: > 3340756356] > 18:02:41.061696 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 18:02:41.062663 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.062791 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.361777 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.661772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.961772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:42.161771 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:42.461770 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:42.675770 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > > > If we are using single homing, there is no problem on the SCTP > communication. Below is the TCP trace on eth1 using sctp_test > > 18:09:55.356727 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] > [init tag: 32516609] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 18:09:55.356811 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 3168861995] [rwnd: 131072] [OS: 10] [MIS: 16] [init TSN: > 1877695021] > 18:09:55.357727 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [COOKIE ECHO] > 18:09:55.357788 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 18:09:55.358724 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > 18:09:55.358740 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 18:09:55.379715 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [DATA] > (B)(E) [TSN: 0] [SID: 0] [SSEQ 0] [PPID 0x3] > 18:09:55.379735 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [SACK] > [cum ack 0] [a_rwnd 131064] [#gap acks 0] [#dup tsns 0] > 18:09:55.657716 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > 18:09:55.657732 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > > From the observations, it seems that the LKSCTP library is not able to > use the original local address when multi-homing is being used. Is > there anyway can be resolved it? > > Thanks > > PS -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 5, 2013, at 10:35 AM, David Laight wrote: the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. > ... >> With that, Sys A talking to Sys C will get an abort >> from Sys B when trying to talk to 10.10.0.2. With /8, it'll be >> even worse since SysB and SysC will have duplicate addresses >> within the subnet. :) >> >> The point is that you don't always know that the same private subnet >> is in reality 2 different subnets with duplicate addresses. >> >> I've had to debug an actual production issue similar to this where >> customer had a very similar configuration to above, and their >> associations kept getting aborted. When I tried accessing the >> system that kept sending aborts, I found it was some windows >> server and not a Diameter station they were expecting. > > Does seem that the addresses listed in INIT and INIT_ACK chunks > should be ignored until a valid HB response has been received. You are not allowed to send DATA to them until they are confirmed. > If an abort is received before a valid HB response then the > address should be ignored rather than the connection aborted. No. > Then it wouldn't matter anywhere near as much if addresses are > advertised that are not reachable from the remote system. > > All this should have been thought about when the original RFC > was written. Scoping wasn't considered that much, same as NAT traversal. The assumption was that in SIGTRAN networks no NAT boxes are deployed and you use appropriate addressing (for redundancy). Best regards Michael > > David > > > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
> >> the configured addresses could be: > >> System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y > >> System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y > >> System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z > >> > >> Same problem will occur. ... > With that, Sys A talking to Sys C will get an abort > from Sys B when trying to talk to 10.10.0.2. With /8, it'll be > even worse since SysB and SysC will have duplicate addresses > within the subnet. :) > > The point is that you don't always know that the same private subnet > is in reality 2 different subnets with duplicate addresses. > > I've had to debug an actual production issue similar to this where > customer had a very similar configuration to above, and their > associations kept getting aborted. When I tried accessing the > system that kept sending aborts, I found it was some windows > server and not a Diameter station they were expecting. Does seem that the addresses listed in INIT and INIT_ACK chunks should be ignored until a valid HB response has been received. If an abort is received before a valid HB response then the address should be ignored rather than the connection aborted. Then it wouldn't matter anywhere near as much if addresses are advertised that are not reachable from the remote system. All this should have been thought about when the original RFC was written. David -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. ... With that, Sys A talking to Sys C will get an abort from Sys B when trying to talk to 10.10.0.2. With /8, it'll be even worse since SysB and SysC will have duplicate addresses within the subnet. :) The point is that you don't always know that the same private subnet is in reality 2 different subnets with duplicate addresses. I've had to debug an actual production issue similar to this where customer had a very similar configuration to above, and their associations kept getting aborted. When I tried accessing the system that kept sending aborts, I found it was some windows server and not a Diameter station they were expecting. Does seem that the addresses listed in INIT and INIT_ACK chunks should be ignored until a valid HB response has been received. If an abort is received before a valid HB response then the address should be ignored rather than the connection aborted. Then it wouldn't matter anywhere near as much if addresses are advertised that are not reachable from the remote system. All this should have been thought about when the original RFC was written. David -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 5, 2013, at 10:35 AM, David Laight david.lai...@aculab.com wrote: the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. ... With that, Sys A talking to Sys C will get an abort from Sys B when trying to talk to 10.10.0.2. With /8, it'll be even worse since SysB and SysC will have duplicate addresses within the subnet. :) The point is that you don't always know that the same private subnet is in reality 2 different subnets with duplicate addresses. I've had to debug an actual production issue similar to this where customer had a very similar configuration to above, and their associations kept getting aborted. When I tried accessing the system that kept sending aborts, I found it was some windows server and not a Diameter station they were expecting. Does seem that the addresses listed in INIT and INIT_ACK chunks should be ignored until a valid HB response has been received. You are not allowed to send DATA to them until they are confirmed. If an abort is received before a valid HB response then the address should be ignored rather than the connection aborted. No. Then it wouldn't matter anywhere near as much if addresses are advertised that are not reachable from the remote system. All this should have been thought about when the original RFC was written. Scoping wasn't considered that much, same as NAT traversal. The assumption was that in SIGTRAN networks no NAT boxes are deployed and you use appropriate addressing (for redundancy). Best regards Michael David -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
So, can I get confirmation that whether we can enhance to support the scenarios or any resolution on providing the correct routing? On Tue, Nov 26, 2013 at 9:03 AM, Sun Paul paul...@gmail.com wrote: Hi we have a problem on using LKSCTP to form a 4 ways multi-homing network. Configuration - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), IP-B (eth2) - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), IP-Y (eth2) the four way paths are shown below. 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) the HB/HB_ACK is normal for the paths IP-A to IP-X and IP-B to IP-Y, but it is not correct for the rest of two. First of all, we are using iproute2 to form 2 table such that when IP-B arrives on IP-X, it will know how to route back to IP-B on the same interface, i.e (eth1). Same logic for the path IP-A to IP-X. What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not using the IP 11.1.1.11. The above operation makes the subsequence HB/HB_ACK in using wrong IP address. TCP trace on eth1 18:02:41.058640 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:02:41.061634 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:02:41.062642 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.062846 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.361811 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.661791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.961791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] TCP trace on eth2 18:02:41.058755 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 3340756356] 18:02:41.061696 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:02:41.062663 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.062791 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.361777 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.661772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.961772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.161771 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.461770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.675770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] If we are using single homing, there is no problem on the SCTP communication. Below is the TCP trace on eth1 using sctp_test 18:09:55.356727 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 32516609] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:09:55.356811 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3168861995] [rwnd: 131072] [OS: 10] [MIS: 16] [init TSN: 1877695021] 18:09:55.357727 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:09:55.357788 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:09:55.358724 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:09:55.358740 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] 18:09:55.379715 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [DATA] (B)(E) [TSN: 0] [SID: 0] [SSEQ 0] [PPID 0x3] 18:09:55.379735 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [SACK] [cum ack 0] [a_rwnd 131064] [#gap acks 0] [#dup tsns 0] 18:09:55.657716 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:09:55.657732 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the observations, it seems that the LKSCTP library is not able to use the original local address when multi-homing is being used. Is there anyway can be resolved it? Thanks PS -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 7:23 PM, Vlad Yasevich wrote: > On 12/04/2013 11:25 AM, Michael Tuexen wrote: >> On Dec 4, 2013, at 5:12 PM, Vlad Yasevich wrote: >> >>> On 12/04/2013 11:01 AM, Michael Tuexen wrote: On Dec 4, 2013, at 4:41 PM, Vlad Yasevich wrote: > On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? >>> >>> As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about >>> both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT >>> and INIT-ACK), then this situation is perfectly valid. In fact, this >>> has been tested an multiple interops. >> >> There are some network configurations that do cause problems. >> Consider 4 systems with 3 LAN segments: >> A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. >> B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. >> C) 10.10.10.3 on LAN X. >> D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. >> There are no routers between the networks (and none of the systems >> are running IP forwarding). >> >> If A connects to B everything is fine - traffic can use either LAN. >> >> Connections from A to C are problematic if C tries to send anything >> (except a HB) to 192.168.1.1 before receiving a HB response. >> One of the SCTP stacks we've used did send messages to an >> inappropriate address, but I've forgotten which one. > > I guess that would be problematic if A can not receive traffic for > 192.168.1.1 on the interface connected to LAN X. I shouldn't > technically be a problem for C as it should mark the path to 192.168.1.1 > as down. For A, as long as it doesn't decide to ABORT the association, > it shouldn't be a problem either. It would be interesting to know more > about what problems you've observed. > >> >> Connections between A and D fail unless the HB errors A receives >> for 192.168.1.2 are ignored. > > Yes, this configuration is very error prone, especially if system B and > system D are up at the same time. Any attempts by system A to use > LAN Y will result in an ABORT generated by system B. I have seen > this issue well in production and we had to renumber system D to solve > it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not "be in the same scope". We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... >>> >>> But these 2 are technically in the same scope. They are both private >>> address types. Also, this will not solve the problem either since >> That is correct. But I think you should not transfer a private address >> to another private address belonging to a different network. >> I don't think this was specified in the older IDs... >>> the configured addresses could be: >>> System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y >>> System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y >>> System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z >>> >>> Same problem will occur. >> Depending on the subnet masks, it might work not not. Are you >> configuring them with /8? > > No, /16 :). With that, Sys A talking to Sys C will get an abort > from Sys B when trying to talk to 10.10.0.2. With /8, it'll be > even worse since SysB and SysC will have duplicate addresses > within the subnet. :) > > The point is that you don't always know that the same private subnet > is in reality 2 different subnets with duplicate addresses. I agree, you can't do it perfectly right. But you can provide some protection. > > I've had to debug an actual production issue similar to this where > customer had a very similar configuration to above, and their > associations kept getting aborted. When I tried accessing the > system that kept sending aborts, I found it was some windows > server and not a Diameter station they were expecting. Interesting... Availability of SCTP on Windows is quite limited... But people seem to use SCTP on Windows. Best regards Michael > >>> >>> Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? >> Yes, one for IPv6. >>
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 11:25 AM, Michael Tuexen wrote: > On Dec 4, 2013, at 5:12 PM, Vlad Yasevich wrote: > >> On 12/04/2013 11:01 AM, Michael Tuexen wrote: >>> On Dec 4, 2013, at 4:41 PM, Vlad Yasevich wrote: >>> On 12/04/2013 09:50 AM, David Laight wrote: >>> In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to >>> IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In >>> the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. >>> >>> In case of the path between IP-A and IP-X is broken, IP-B sends INIT >>> to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends >>> HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB >>> communication between IP-B and IP-Y follows the normal flow. >>> >>> Can I confirm, is it really valid? >> >> As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about >> both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT >> and INIT-ACK), then this situation is perfectly valid. In fact, this >> has been tested an multiple interops. > > There are some network configurations that do cause problems. > Consider 4 systems with 3 LAN segments: > A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. > B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. > C) 10.10.10.3 on LAN X. > D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. > There are no routers between the networks (and none of the systems > are running IP forwarding). > > If A connects to B everything is fine - traffic can use either LAN. > > Connections from A to C are problematic if C tries to send anything > (except a HB) to 192.168.1.1 before receiving a HB response. > One of the SCTP stacks we've used did send messages to an > inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. > > Connections between A and D fail unless the HB errors A receives > for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. >>> The point is that address scoping should be used. When sending an >>> INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, >>> since you are transmitting an address to a node which might or might >>> not "be in the same scope". We had IDs for that in the past, but >>> they never made it to RFC state, because they were not progressed enough >>> by the authors. Maybe we should push them again... >> >> But these 2 are technically in the same scope. They are both private >> address types. Also, this will not solve the problem either since > That is correct. But I think you should not transfer a private address > to another private address belonging to a different network. > I don't think this was specified in the older IDs... >> the configured addresses could be: >> System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y >> System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y >> System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z >> >> Same problem will occur. > Depending on the subnet masks, it might work not not. Are you > configuring them with /8? No, /16 :). With that, Sys A talking to Sys C will get an abort from Sys B when trying to talk to 10.10.0.2. With /8, it'll be even worse since SysB and SysC will have duplicate addresses within the subnet. :) The point is that you don't always know that the same private subnet is in reality 2 different subnets with duplicate addresses. I've had to debug an actual production issue similar to this where customer had a very similar configuration to above, and their associations kept getting aborted. When I tried accessing the system that kept sending aborts, I found it was some windows server and not a Diameter station they were expecting. >> >> Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? > Yes, one for IPv6. > http://tools.ietf.org/html/draft-stewart-tsvwg-sctpipv6-01 > They need to be integrated and improved... > Ok. I'll take a look. Thanks -vlad > Best regards > Michael >> >> Thanks >> -vlad >> >>> >>> Best regards >>> Michael -vlad > > Of course the application could explicitly bind to only the 10.x address > but that requires the application know the exact network topology > and may be difficult for
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 12:57 PM, Sun Paul wrote: > As I know, the A to C and A to D case must have a router in between to form > SCTP multihome topology. Not necessary. I've produced proper multihoming topologies with just VLANs and different subnet assignment. You can even remove VLANs if you correctly set your arp_ignore and arp_announce values. -vlad > On Dec 4, 2013 10:51 PM, "David Laight" wrote: > In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? >>> >>> As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about >>> both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT >>> and INIT-ACK), then this situation is perfectly valid. In fact, this >>> has been tested an multiple interops. >> >> There are some network configurations that do cause problems. >> Consider 4 systems with 3 LAN segments: >> A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. >> B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. >> C) 10.10.10.3 on LAN X. >> D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. >> There are no routers between the networks (and none of the systems >> are running IP forwarding). >> >> If A connects to B everything is fine - traffic can use either LAN. >> >> Connections from A to C are problematic if C tries to send anything >> (except a HB) to 192.168.1.1 before receiving a HB response. >> One of the SCTP stacks we've used did send messages to an >> inappropriate address, but I've forgotten which one. >> >> Connections between A and D fail unless the HB errors A receives >> for 192.168.1.2 are ignored. >> >> Of course the application could explicitly bind to only the 10.x address >> but that requires the application know the exact network topology >> and may be difficult for incoming calls. >> >> David >> >> > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 5:48 PM, David Laight wrote: >> The point is that address scoping should be used. When sending an >> INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, >> since you are transmitting an address to a node which might or might >> not "be in the same scope". > > You might have two machines that are connected via the public > internet and also via a private network. > The two sets of cabling being completely separate giving you > resilience to network failure. > In which case you definitely don't want address scoping. Well, if you give the SCTP stack a hint when initiating the association, it can do the right thing. Calling sctp_connect(private_address) should work. It will list the public address without any problems. One can debate that sctp_connectx(private_address, public_address) will result in sending an INIT to the public_address listing the private one. However, calling sctp_connect(public_address) should not list the private_address. Best regards Michael > > While you may not want the SCTP traffic on the public network > itself, it could easily be routed separately. > > We have systems that 'sort of' designate one interface for SIP/RTP > and the other for 'management'. They might run M3UA/SCTP but no one > has really thought enough about which interface(s) the M3UA traffic > should use. > (Think of an ISUP/SIP gateway using M3UA for ISUP signalling.) > > David > > > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
> The point is that address scoping should be used. When sending an > INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, > since you are transmitting an address to a node which might or might > not "be in the same scope". You might have two machines that are connected via the public internet and also via a private network. The two sets of cabling being completely separate giving you resilience to network failure. In which case you definitely don't want address scoping. While you may not want the SCTP traffic on the public network itself, it could easily be routed separately. We have systems that 'sort of' designate one interface for SIP/RTP and the other for 'management'. They might run M3UA/SCTP but no one has really thought enough about which interface(s) the M3UA traffic should use. (Think of an ISUP/SIP gateway using M3UA for ISUP signalling.) David -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 4:41 PM, Vlad Yasevich wrote: > On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? >>> >>> As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about >>> both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT >>> and INIT-ACK), then this situation is perfectly valid. In fact, this >>> has been tested an multiple interops. >> >> There are some network configurations that do cause problems. >> Consider 4 systems with 3 LAN segments: >> A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. >> B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. >> C) 10.10.10.3 on LAN X. >> D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. >> There are no routers between the networks (and none of the systems >> are running IP forwarding). >> >> If A connects to B everything is fine - traffic can use either LAN. >> >> Connections from A to C are problematic if C tries to send anything >> (except a HB) to 192.168.1.1 before receiving a HB response. >> One of the SCTP stacks we've used did send messages to an >> inappropriate address, but I've forgotten which one. > > I guess that would be problematic if A can not receive traffic for > 192.168.1.1 on the interface connected to LAN X. I shouldn't > technically be a problem for C as it should mark the path to 192.168.1.1 > as down. For A, as long as it doesn't decide to ABORT the association, > it shouldn't be a problem either. It would be interesting to know more > about what problems you've observed. > >> >> Connections between A and D fail unless the HB errors A receives >> for 192.168.1.2 are ignored. > > Yes, this configuration is very error prone, especially if system B and > system D are up at the same time. Any attempts by system A to use > LAN Y will result in an ABORT generated by system B. I have seen > this issue well in production and we had to renumber system D to solve > it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not "be in the same scope". We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... Best regards Michael > > -vlad >> >> Of course the application could explicitly bind to only the 10.x address >> but that requires the application know the exact network topology >> and may be difficult for incoming calls. >> >> David >> > > -- > To unsubscribe from this list: send the line "unsubscribe linux-sctp" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 5:12 PM, Vlad Yasevich wrote: > On 12/04/2013 11:01 AM, Michael Tuexen wrote: >> On Dec 4, 2013, at 4:41 PM, Vlad Yasevich wrote: >> >>> On 12/04/2013 09:50 AM, David Laight wrote: >> In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to >> IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In >> the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. >> >> In case of the path between IP-A and IP-X is broken, IP-B sends INIT >> to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends >> HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB >> communication between IP-B and IP-Y follows the normal flow. >> >> Can I confirm, is it really valid? > > As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about > both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT > and INIT-ACK), then this situation is perfectly valid. In fact, this > has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. >>> >>> I guess that would be problematic if A can not receive traffic for >>> 192.168.1.1 on the interface connected to LAN X. I shouldn't >>> technically be a problem for C as it should mark the path to 192.168.1.1 >>> as down. For A, as long as it doesn't decide to ABORT the association, >>> it shouldn't be a problem either. It would be interesting to know more >>> about what problems you've observed. >>> Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. >>> >>> Yes, this configuration is very error prone, especially if system B and >>> system D are up at the same time. Any attempts by system A to use >>> LAN Y will result in an ABORT generated by system B. I have seen >>> this issue well in production and we had to renumber system D to solve >>> it. >> The point is that address scoping should be used. When sending an >> INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, >> since you are transmitting an address to a node which might or might >> not "be in the same scope". We had IDs for that in the past, but >> they never made it to RFC state, because they were not progressed enough >> by the authors. Maybe we should push them again... > > But these 2 are technically in the same scope. They are both private > address types. Also, this will not solve the problem either since That is correct. But I think you should not transfer a private address to another private address belonging to a different network. I don't think this was specified in the older IDs... > the configured addresses could be: > System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y > System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y > System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z > > Same problem will occur. Depending on the subnet masks, it might work not not. Are you configuring them with /8? > > Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? Yes, one for IPv6. http://tools.ietf.org/html/draft-stewart-tsvwg-sctpipv6-01 They need to be integrated and improved... Best regards Michael > > Thanks > -vlad > >> >> Best regards >> Michael >>> >>> -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David >>> >>> -- >>> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in >>> the body of a message to majord...@vger.kernel.org >>> More majordomo info at http://vger.kernel.org/majordomo-info.html >>> >> > > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
> > There are some network configurations that do cause problems. > > Consider 4 systems with 3 LAN segments: > > A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. > > B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. > > C) 10.10.10.3 on LAN X. > > D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. > > There are no routers between the networks (and none of the systems > > are running IP forwarding). > > > > If A connects to B everything is fine - traffic can use either LAN. > > > > Connections from A to C are problematic if C tries to send anything > > (except a HB) to 192.168.1.1 before receiving a HB response. > > One of the SCTP stacks we've used did send messages to an > > inappropriate address, but I've forgotten which one. > > I guess that would be problematic if A can not receive traffic for > 192.168.1.1 on the interface connected to LAN X. I shouldn't > technically be a problem for C as it should mark the path to 192.168.1.1 > as down. For A, as long as it doesn't decide to ABORT the association, > it shouldn't be a problem either. It would be interesting to know more > about what problems you've observed. It was a long time ago, we don't actually do much SCTP testing even though our product (SS7 M3UA) uses it. Mostly because we don't want to find bugs that are hard to fix! What we saw was C using the 192.x address (as supplied in the INIT) and these not being routable to A (and not getting a response from a 3rd system). So the application layer immediately timed everything out. What I can't remember is whether this was Linux, Solaris or the SCTP stack we took from freebsd and rammed into the windows kernel. David
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 11:01 AM, Michael Tuexen wrote: > On Dec 4, 2013, at 4:41 PM, Vlad Yasevich wrote: > >> On 12/04/2013 09:50 AM, David Laight wrote: > In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to > IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In > the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. > > In case of the path between IP-A and IP-X is broken, IP-B sends INIT > to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends > HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB > communication between IP-B and IP-Y follows the normal flow. > > Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. >>> >>> There are some network configurations that do cause problems. >>> Consider 4 systems with 3 LAN segments: >>> A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. >>> B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. >>> C) 10.10.10.3 on LAN X. >>> D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. >>> There are no routers between the networks (and none of the systems >>> are running IP forwarding). >>> >>> If A connects to B everything is fine - traffic can use either LAN. >>> >>> Connections from A to C are problematic if C tries to send anything >>> (except a HB) to 192.168.1.1 before receiving a HB response. >>> One of the SCTP stacks we've used did send messages to an >>> inappropriate address, but I've forgotten which one. >> >> I guess that would be problematic if A can not receive traffic for >> 192.168.1.1 on the interface connected to LAN X. I shouldn't >> technically be a problem for C as it should mark the path to 192.168.1.1 >> as down. For A, as long as it doesn't decide to ABORT the association, >> it shouldn't be a problem either. It would be interesting to know more >> about what problems you've observed. >> >>> >>> Connections between A and D fail unless the HB errors A receives >>> for 192.168.1.2 are ignored. >> >> Yes, this configuration is very error prone, especially if system B and >> system D are up at the same time. Any attempts by system A to use >> LAN Y will result in an ABORT generated by system B. I have seen >> this issue well in production and we had to renumber system D to solve >> it. > The point is that address scoping should be used. When sending an > INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, > since you are transmitting an address to a node which might or might > not "be in the same scope". We had IDs for that in the past, but > they never made it to RFC state, because they were not progressed enough > by the authors. Maybe we should push them again... But these 2 are technically in the same scope. They are both private address types. Also, this will not solve the problem either since the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? Thanks -vlad > > Best regards > Michael >> >> -vlad >>> >>> Of course the application could explicitly bind to only the 10.x address >>> but that requires the application know the exact network topology >>> and may be difficult for incoming calls. >>> >>> David >>> >> >> -- >> To unsubscribe from this list: send the line "unsubscribe linux-sctp" in >> the body of a message to majord...@vger.kernel.org >> More majordomo info at http://vger.kernel.org/majordomo-info.html >> > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 09:50 AM, David Laight wrote: >>> In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to >>> IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In >>> the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. >>> >>> In case of the path between IP-A and IP-X is broken, IP-B sends INIT >>> to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends >>> HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB >>> communication between IP-B and IP-Y follows the normal flow. >>> >>> Can I confirm, is it really valid? >> >> As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about >> both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT >> and INIT-ACK), then this situation is perfectly valid. In fact, this >> has been tested an multiple interops. > > There are some network configurations that do cause problems. > Consider 4 systems with 3 LAN segments: > A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. > B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. > C) 10.10.10.3 on LAN X. > D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. > There are no routers between the networks (and none of the systems > are running IP forwarding). > > If A connects to B everything is fine - traffic can use either LAN. > > Connections from A to C are problematic if C tries to send anything > (except a HB) to 192.168.1.1 before receiving a HB response. > One of the SCTP stacks we've used did send messages to an > inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. > > Connections between A and D fail unless the HB errors A receives > for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. -vlad > > Of course the application could explicitly bind to only the 10.x address > but that requires the application know the exact network topology > and may be difficult for incoming calls. > > David > -- To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
> > In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to > > IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In > > the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. > > > > In case of the path between IP-A and IP-X is broken, IP-B sends INIT > > to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends > > HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB > > communication between IP-B and IP-Y follows the normal flow. > > > > Can I confirm, is it really valid? > > As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about > both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT > and INIT-ACK), then this situation is perfectly valid. In fact, this > has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David
Re: Supporting 4 way connections in LKSCTP
On 12/03/2013 08:59 PM, Sun Paul wrote: > This is the most puzzling area. I really not sure whether it is really > valid or not. Is there any documented statement supporting this? > > Let me summarize the behavior again. > > NODE-A > eth1: IP-A > eth2: IP-B > > NODE-B > eth1: IP-X > eth2: IP-Y > > In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to > IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In > the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. > > In case of the path between IP-A and IP-X is broken, IP-B sends INIT > to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends > HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB > communication between IP-B and IP-Y follows the normal flow. > > Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. Now, not all implementations will pick the source address you seem to expect. Linux, does not do source based transport selection. It only looks at the destination and picks the best source address based on routing table decision. So, in the case of Linux (nodeA), there will only be 2 transports: one with a dest of IP-X and one with dest of IP-Y. If it determined at start-up that IP-A is the source to set to IP-X, it will never try to use IP-B to send to IP-X. If it detects that destination IP-X is unreachable, then it will keep probing, but it will not change the source address. Remember, HB-ACKs do not have to be returned from the same address that HB were sent to. This is because HB contains a nonce and that nonce is used to locate the correct transport that this HB-ACK belongs to. So, yes, the above communication is a valid SCTP exchange. -vlad > > On Tue, Dec 3, 2013 at 11:22 PM, Vlad Yasevich wrote: >> On 12/03/2013 08:11 AM, Sun Paul wrote: >>> But how about the HB and HB_ACK? Still valid? >> >> As long as the source address is part of the association, then yes >> it is perfectly valid. >> >> -vlad >> >>> On Dec 3, 2013 8:32 PM, "Vlad Yasevich" wrote: >>> On 12/02/2013 09:19 PM, Sun Paul wrote: > so in this case, says > > (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) > returns INIT_ACK to IP-B (NODE-A) > > this is also treated as a valid, am I correct? As long as IP-X (Node-B) is present in the address list of the INIT-ACK chunk, yes. There is the code in __sctp_rcv_lookup_harder() that looks for other adddresses in the INIT and INIT-ACK chunks. -vlad > > > On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich wrote: >> On 12/02/2013 08:39 PM, Sun Paul wrote: >>> Another question >>> >>> if a wrong source IP is used, does the association still classified as normal? >> >> What do you mean my wrong source IP? As long as the address is part of >> the association, it can be used. >> >> -vlad >> >>> >>> On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: > On 12/02/2013 10:45 AM, Karl Heiss wrote: >> On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: >>> On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB
Re: Supporting 4 way connections in LKSCTP
On 12/03/2013 08:59 PM, Sun Paul wrote: This is the most puzzling area. I really not sure whether it is really valid or not. Is there any documented statement supporting this? Let me summarize the behavior again. NODE-A eth1: IP-A eth2: IP-B NODE-B eth1: IP-X eth2: IP-Y In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. Now, not all implementations will pick the source address you seem to expect. Linux, does not do source based transport selection. It only looks at the destination and picks the best source address based on routing table decision. So, in the case of Linux (nodeA), there will only be 2 transports: one with a dest of IP-X and one with dest of IP-Y. If it determined at start-up that IP-A is the source to set to IP-X, it will never try to use IP-B to send to IP-X. If it detects that destination IP-X is unreachable, then it will keep probing, but it will not change the source address. Remember, HB-ACKs do not have to be returned from the same address that HB were sent to. This is because HB contains a nonce and that nonce is used to locate the correct transport that this HB-ACK belongs to. So, yes, the above communication is a valid SCTP exchange. -vlad On Tue, Dec 3, 2013 at 11:22 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/03/2013 08:11 AM, Sun Paul wrote: But how about the HB and HB_ACK? Still valid? As long as the source address is part of the association, then yes it is perfectly valid. -vlad On Dec 3, 2013 8:32 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 09:19 PM, Sun Paul wrote: so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? As long as IP-X (Node-B) is present in the address list of the INIT-ACK chunk, yes. There is the code in __sctp_rcv_lookup_harder() that looks for other adddresses in the INIT and INIT-ACK chunks. -vlad On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 08:39 PM, Sun Paul wrote: Another question if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2
RE: Supporting 4 way connections in LKSCTP
In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 11:01 AM, Michael Tuexen wrote: On Dec 4, 2013, at 4:41 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... But these 2 are technically in the same scope. They are both private address types. Also, this will not solve the problem either since the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? Thanks -vlad Best regards Michael -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David -- To unsubscribe from this list: send the line unsubscribe linux-sctp in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. It was a long time ago, we don't actually do much SCTP testing even though our product (SS7 M3UA) uses it. Mostly because we don't want to find bugs that are hard to fix! What we saw was C using the 192.x address (as supplied in the INIT) and these not being routable to A (and not getting a response from a 3rd system). So the application layer immediately timed everything out. What I can't remember is whether this was Linux, Solaris or the SCTP stack we took from freebsd and rammed into the windows kernel. David
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 5:12 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 11:01 AM, Michael Tuexen wrote: On Dec 4, 2013, at 4:41 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... But these 2 are technically in the same scope. They are both private address types. Also, this will not solve the problem either since That is correct. But I think you should not transfer a private address to another private address belonging to a different network. I don't think this was specified in the older IDs... the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. Depending on the subnet masks, it might work not not. Are you configuring them with /8? Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? Yes, one for IPv6. http://tools.ietf.org/html/draft-stewart-tsvwg-sctpipv6-01 They need to be integrated and improved... Best regards Michael Thanks -vlad Best regards Michael -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David -- To unsubscribe from this list: send the line unsubscribe linux-sctp in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 4:41 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... Best regards Michael -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David -- To unsubscribe from this list: send the line unsubscribe linux-sctp in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
RE: Supporting 4 way connections in LKSCTP
The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. You might have two machines that are connected via the public internet and also via a private network. The two sets of cabling being completely separate giving you resilience to network failure. In which case you definitely don't want address scoping. While you may not want the SCTP traffic on the public network itself, it could easily be routed separately. We have systems that 'sort of' designate one interface for SIP/RTP and the other for 'management'. They might run M3UA/SCTP but no one has really thought enough about which interface(s) the M3UA traffic should use. (Think of an ISUP/SIP gateway using M3UA for ISUP signalling.) David -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 5:48 PM, David Laight david.lai...@aculab.com wrote: The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. You might have two machines that are connected via the public internet and also via a private network. The two sets of cabling being completely separate giving you resilience to network failure. In which case you definitely don't want address scoping. Well, if you give the SCTP stack a hint when initiating the association, it can do the right thing. Calling sctp_connect(private_address) should work. It will list the public address without any problems. One can debate that sctp_connectx(private_address, public_address) will result in sending an INIT to the public_address listing the private one. However, calling sctp_connect(public_address) should not list the private_address. Best regards Michael While you may not want the SCTP traffic on the public network itself, it could easily be routed separately. We have systems that 'sort of' designate one interface for SIP/RTP and the other for 'management'. They might run M3UA/SCTP but no one has really thought enough about which interface(s) the M3UA traffic should use. (Think of an ISUP/SIP gateway using M3UA for ISUP signalling.) David -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 12:57 PM, Sun Paul wrote: As I know, the A to C and A to D case must have a router in between to form SCTP multihome topology. Not necessary. I've produced proper multihoming topologies with just VLANs and different subnet assignment. You can even remove VLANs if you correctly set your arp_ignore and arp_announce values. -vlad On Dec 4, 2013 10:51 PM, David Laight david.lai...@aculab.com wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html Please read the FAQ at http://www.tux.org/lkml/
Re: Supporting 4 way connections in LKSCTP
On 12/04/2013 11:25 AM, Michael Tuexen wrote: On Dec 4, 2013, at 5:12 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 11:01 AM, Michael Tuexen wrote: On Dec 4, 2013, at 4:41 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... But these 2 are technically in the same scope. They are both private address types. Also, this will not solve the problem either since That is correct. But I think you should not transfer a private address to another private address belonging to a different network. I don't think this was specified in the older IDs... the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. Depending on the subnet masks, it might work not not. Are you configuring them with /8? No, /16 :). With that, Sys A talking to Sys C will get an abort from Sys B when trying to talk to 10.10.0.2. With /8, it'll be even worse since SysB and SysC will have duplicate addresses within the subnet. :) The point is that you don't always know that the same private subnet is in reality 2 different subnets with duplicate addresses. I've had to debug an actual production issue similar to this where customer had a very similar configuration to above, and their associations kept getting aborted. When I tried accessing the system that kept sending aborts, I found it was some windows server and not a Diameter station they were expecting. Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? Yes, one for IPv6. http://tools.ietf.org/html/draft-stewart-tsvwg-sctpipv6-01 They need to be integrated and improved... Ok. I'll take a look. Thanks -vlad Best regards Michael Thanks -vlad Best regards Michael -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for incoming calls. David -- To unsubscribe from this list: send the line unsubscribe linux-sctp in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a
Re: Supporting 4 way connections in LKSCTP
On Dec 4, 2013, at 7:23 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 11:25 AM, Michael Tuexen wrote: On Dec 4, 2013, at 5:12 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 11:01 AM, Michael Tuexen wrote: On Dec 4, 2013, at 4:41 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/04/2013 09:50 AM, David Laight wrote: In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? As long as NODE-B knows about both IP-A and IP-B, and NODE-A knows about both IP-X and IP-Y (meaning all the addresses were exchanged inside INIT and INIT-ACK), then this situation is perfectly valid. In fact, this has been tested an multiple interops. There are some network configurations that do cause problems. Consider 4 systems with 3 LAN segments: A) 10.10.10.1 on LAN X and 192.168.1.1 on LAN Y. B) 10.10.10.2 on LAN X and 192.168.1.2 on LAN Y. C) 10.10.10.3 on LAN X. D) 10.10.10.4 on LAN X and 192.168.1.2 on LAN Z. There are no routers between the networks (and none of the systems are running IP forwarding). If A connects to B everything is fine - traffic can use either LAN. Connections from A to C are problematic if C tries to send anything (except a HB) to 192.168.1.1 before receiving a HB response. One of the SCTP stacks we've used did send messages to an inappropriate address, but I've forgotten which one. I guess that would be problematic if A can not receive traffic for 192.168.1.1 on the interface connected to LAN X. I shouldn't technically be a problem for C as it should mark the path to 192.168.1.1 as down. For A, as long as it doesn't decide to ABORT the association, it shouldn't be a problem either. It would be interesting to know more about what problems you've observed. Connections between A and D fail unless the HB errors A receives for 192.168.1.2 are ignored. Yes, this configuration is very error prone, especially if system B and system D are up at the same time. Any attempts by system A to use LAN Y will result in an ABORT generated by system B. I have seen this issue well in production and we had to renumber system D to solve it. The point is that address scoping should be used. When sending an INIT from 10.10.10.1 to 10.10.10.4 you should not list 192.168.1.1, since you are transmitting an address to a node which might or might not be in the same scope. We had IDs for that in the past, but they never made it to RFC state, because they were not progressed enough by the authors. Maybe we should push them again... But these 2 are technically in the same scope. They are both private address types. Also, this will not solve the problem either since That is correct. But I think you should not transfer a private address to another private address belonging to a different network. I don't think this was specified in the older IDs... the configured addresses could be: System A) 10.0.0.1 on Lan X, 10.10.0.1 on Lan Y System B) 10.0.0.2 on Lan X, 10.10.0.2 on Lan Y System C) 10.0.0.3 on Lan X, 10.10.0.2 on Lan Z Same problem will occur. Depending on the subnet masks, it might work not not. Are you configuring them with /8? No, /16 :). With that, Sys A talking to Sys C will get an abort from Sys B when trying to talk to 10.10.0.2. With /8, it'll be even worse since SysB and SysC will have duplicate addresses within the subnet. :) The point is that you don't always know that the same private subnet is in reality 2 different subnets with duplicate addresses. I agree, you can't do it perfectly right. But you can provide some protection. I've had to debug an actual production issue similar to this where customer had a very similar configuration to above, and their associations kept getting aborted. When I tried accessing the system that kept sending aborts, I found it was some windows server and not a Diameter station they were expecting. Interesting... Availability of SCTP on Windows is quite limited... But people seem to use SCTP on Windows. Best regards Michael Btw, were there any IDs other then draft-stewart-tsvwg-sctp-ipv4? Yes, one for IPv6. http://tools.ietf.org/html/draft-stewart-tsvwg-sctpipv6-01 They need to be integrated and improved... Ok. I'll take a look. Thanks -vlad Best regards Michael Thanks -vlad Best regards Michael -vlad Of course the application could explicitly bind to only the 10.x address but that requires the application know the exact network topology and may be difficult for
Re: Supporting 4 way connections in LKSCTP
This is the most puzzling area. I really not sure whether it is really valid or not. Is there any documented statement supporting this? Let me summarize the behavior again. NODE-A eth1: IP-A eth2: IP-B NODE-B eth1: IP-X eth2: IP-Y In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? On Tue, Dec 3, 2013 at 11:22 PM, Vlad Yasevich wrote: > On 12/03/2013 08:11 AM, Sun Paul wrote: >> But how about the HB and HB_ACK? Still valid? > > As long as the source address is part of the association, then yes > it is perfectly valid. > > -vlad > >> On Dec 3, 2013 8:32 PM, "Vlad Yasevich" wrote: >> >>> On 12/02/2013 09:19 PM, Sun Paul wrote: so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? >>> >>> As long as IP-X (Node-B) is present in the address list of the INIT-ACK >>> chunk, yes. >>> >>> There is the code in __sctp_rcv_lookup_harder() that looks for other >>> adddresses in the INIT and INIT-ACK chunks. >>> >>> -vlad On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich >>> wrote: > On 12/02/2013 08:39 PM, Sun Paul wrote: >> Another question >> >> if a wrong source IP is used, does the association still classified as >>> normal? > > What do you mean my wrong source IP? As long as the address is part of > the association, it can be used. > > -vlad > >> >> On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: >>> Thanks Vlad >>> >>> I checked on the route, and it looks correct. >>> >>> [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 >>> 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 >>> 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 >>> 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 >>> 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> so, if this is not being handled in LKSCTP, is it possible to suggest >>> a way how we can achieve it? >>> >>> On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich >>> wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: > On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich >>> wrote: >> On 11/27/2013 11:03 PM, Sun Paul wrote: >>> How LKSCTP select which source address to use for the INIT_ACK or >>> HB_ACK? below is the testing result where a router is located in >>> the >>> middle. >>> >>> Before starting the application. the packet on eth1 and eth2 are >>> >>> eth1 >>> 0 packets dropped by kernel >>> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >>> tcpdump: verbose output suppressed, use -v or -vv for full >>> protocol decode >>> listening on eth1, link-type EN10MB (Ethernet), capture size >>> 65535 bytes >>> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: >>> 0] >>> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) >>> [ABORT] >>> 11:24:14.539486 >>> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: >>> 0] >>> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) >>> [ABORT] >>> >>> eth2 >>> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >>> tcpdump: verbose output suppressed, use -v or -vv for full >>> protocol decode >>> listening on eth2, link-type EN10MB (Ethernet), capture size >>> 65535 bytes >>> >>> When starting the application. the packet show as below. >>> >>> eth1 >>> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: >>> 0] >>> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) >>> [COOKIE ECHO] >>> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB >>> REQ] >>>
Re: Supporting 4 way connections in LKSCTP
On 12/03/2013 08:11 AM, Sun Paul wrote: > But how about the HB and HB_ACK? Still valid? As long as the source address is part of the association, then yes it is perfectly valid. -vlad > On Dec 3, 2013 8:32 PM, "Vlad Yasevich" wrote: > >> On 12/02/2013 09:19 PM, Sun Paul wrote: >>> so in this case, says >>> >>> (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) >>> returns INIT_ACK to IP-B (NODE-A) >>> >>> this is also treated as a valid, am I correct? >> >> As long as IP-X (Node-B) is present in the address list of the INIT-ACK >> chunk, yes. >> >> There is the code in __sctp_rcv_lookup_harder() that looks for other >> adddresses in the INIT and INIT-ACK chunks. >> >> -vlad >>> >>> >>> On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich >> wrote: On 12/02/2013 08:39 PM, Sun Paul wrote: > Another question > > if a wrong source IP is used, does the association still classified as >> normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad > > On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: >> Thanks Vlad >> >> I checked on the route, and it looks correct. >> >> [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 >> 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 >> 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 >> 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 >> 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> so, if this is not being handled in LKSCTP, is it possible to suggest >> a way how we can achieve it? >> >> On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich >> wrote: >>> On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich >> wrote: > On 11/27/2013 11:03 PM, Sun Paul wrote: >> How LKSCTP select which source address to use for the INIT_ACK or >> HB_ACK? below is the testing result where a router is located in >> the >> middle. >> >> Before starting the application. the packet on eth1 and eth2 are >> >> eth1 >> 0 packets dropped by kernel >> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full >> protocol decode >> listening on eth1, link-type EN10MB (Ethernet), capture size >> 65535 bytes >> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: >> 0] >> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) >> [ABORT] >> 11:24:14.539486 >> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: >> 0] >> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) >> [ABORT] >> >> eth2 >> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full >> protocol decode >> listening on eth2, link-type EN10MB (Ethernet), capture size >> 65535 bytes >> >> When starting the application. the packet show as below. >> >> eth1 >> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: >> 0] >> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) >> [COOKIE ECHO] >> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB >> REQ] >> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB >> REQ] >> >> eth2 >> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT >> ACK] >> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >> 2330749678] >> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) >> [COOKIE ACK] >> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB >> ACK] >> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB >> REQ] >> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB >> ACK] >> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB >> ACK] >> >> From the above result, you can see that the INIT, COOKIE ECHO and >> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, >> COOKIE_ACK, HB_ACK) are returned on eth2 using
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 09:19 PM, Sun Paul wrote: > so in this case, says > > (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) > returns INIT_ACK to IP-B (NODE-A) > > this is also treated as a valid, am I correct? As long as IP-X (Node-B) is present in the address list of the INIT-ACK chunk, yes. There is the code in __sctp_rcv_lookup_harder() that looks for other adddresses in the INIT and INIT-ACK chunks. -vlad > > > On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich wrote: >> On 12/02/2013 08:39 PM, Sun Paul wrote: >>> Another question >>> >>> if a wrong source IP is used, does the association still classified as >>> normal? >> >> What do you mean my wrong source IP? As long as the address is part of >> the association, it can be used. >> >> -vlad >> >>> >>> On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: > On 12/02/2013 10:45 AM, Karl Heiss wrote: >> On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich >> wrote: >>> On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 09:19 PM, Sun Paul wrote: so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? As long as IP-X (Node-B) is present in the address list of the INIT-ACK chunk, yes. There is the code in __sctp_rcv_lookup_harder() that looks for other adddresses in the INIT and INIT-ACK chunks. -vlad On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 08:39 PM, Sun Paul wrote: Another question if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination
Re: Supporting 4 way connections in LKSCTP
On 12/03/2013 08:11 AM, Sun Paul wrote: But how about the HB and HB_ACK? Still valid? As long as the source address is part of the association, then yes it is perfectly valid. -vlad On Dec 3, 2013 8:32 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 09:19 PM, Sun Paul wrote: so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? As long as IP-X (Node-B) is present in the address list of the INIT-ACK chunk, yes. There is the code in __sctp_rcv_lookup_harder() that looks for other adddresses in the INIT and INIT-ACK chunks. -vlad On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 08:39 PM, Sun Paul wrote: Another question if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq
Re: Supporting 4 way connections in LKSCTP
This is the most puzzling area. I really not sure whether it is really valid or not. Is there any documented statement supporting this? Let me summarize the behavior again. NODE-A eth1: IP-A eth2: IP-B NODE-B eth1: IP-X eth2: IP-Y In normal operation, IP-A sends INIT to IP-X, IP-X returns INIT_ACK to IP-A. IP-A then sends HB to IP-X, IP-X then returns HB_ACK to IP-A. In the meantime, IP-B sends HB to IP-Y and IPY returns HB_ACK. In case of the path between IP-A and IP-X is broken, IP-B sends INIT to IP-X, NODE-B uses IP-Y to return INIT_ACK to IP-B. Then IP-B sends HB to IP-X, and IP-Y returns HB_ACK to IP-B. In the meantime, the HB communication between IP-B and IP-Y follows the normal flow. Can I confirm, is it really valid? On Tue, Dec 3, 2013 at 11:22 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/03/2013 08:11 AM, Sun Paul wrote: But how about the HB and HB_ACK? Still valid? As long as the source address is part of the association, then yes it is perfectly valid. -vlad On Dec 3, 2013 8:32 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 09:19 PM, Sun Paul wrote: so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? As long as IP-X (Node-B) is present in the address list of the INIT-ACK chunk, yes. There is the code in __sctp_rcv_lookup_harder() that looks for other adddresses in the INIT and INIT-ACK chunks. -vlad On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 08:39 PM, Sun Paul wrote: Another question if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE
Re: Supporting 4 way connections in LKSCTP
if we do not specific the gateway, it will be a problem. One of the reason why it is like that is due to that the server do not know which source IP should be used to send out the query. [root@localhost ~]# ip route get 12.1.1.1 RTNETLINK answers: Network is unreachable [root@localhost ~]# ip route get 11.1.1.1 RTNETLINK answers: Network is unreachable On Tue, Dec 3, 2013 at 10:02 AM, Vlad Yasevich wrote: > On 12/02/2013 08:31 PM, Sun Paul wrote: >> Thanks Vlad >> >> I checked on the route, and it looks correct. >> >> [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 >> 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 >> 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 >> 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 >> 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> so, if this is not being handled in LKSCTP, is it possible to suggest >> a way how we can achieve it? > > Like I said before lksctp only iterates over the bound address > list if the source address from the route returned by the kernel > does not match the bound address list. > > To simulate the algorithm with with ip route get, you'd > get something like this: > >ip route get 11.1.1.1 >if from_address not in bound address list >for each bound_addr in bound address list >ip route get 11.1.1.1 from bound_addr > > The kernel will almost always give you a valid route even if the > from address does not belong to the expected interface. > > So, in your case, what does a simple ip route get 11.1.1.1 return? > That's the address that will be used to talk to 11.1.1.1 by default. > > -vlad >> >> On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: >>> On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: > On 11/27/2013 11:03 PM, Sun Paul wrote: >> How LKSCTP select which source address to use for the INIT_ACK or >> HB_ACK? below is the testing result where a router is located in the >> middle. >> >> Before starting the application. the packet on eth1 and eth2 are >> >> eth1 >> 0 packets dropped by kernel >> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes >> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >> 11:24:14.539486 >> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >> >> eth2 >> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes >> >> When starting the application. the packet show as below. >> >> eth1 >> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] >> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >> >> eth2 >> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >> 2330749678] >> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] >> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] >> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >> >> From the above result, you can see that the INIT, COOKIE ECHO and >> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, >> COOKIE_ACK, HB_ACK) are returned on eth2 using source address >> 120.1.1.1 instead of 110.1.1.1. >> >> Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? >> >> For simple ICMP ping test, it is normal, but not for SCTP. >> >> eth1 >> 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >> seq
Re: Supporting 4 way connections in LKSCTP
so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich wrote: > On 12/02/2013 08:39 PM, Sun Paul wrote: >> Another question >> >> if a wrong source IP is used, does the association still classified as >> normal? > > What do you mean my wrong source IP? As long as the address is part of > the association, it can be used. > > -vlad > >> >> On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: >>> Thanks Vlad >>> >>> I checked on the route, and it looks correct. >>> >>> [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 >>> 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 >>> 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 >>> 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 >>> 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >>> cache mtu 1500 advmss 1460 hoplimit 64 >>> >>> so, if this is not being handled in LKSCTP, is it possible to suggest >>> a way how we can achieve it? >>> >>> On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: > On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: >> On 11/27/2013 11:03 PM, Sun Paul wrote: >>> How LKSCTP select which source address to use for the INIT_ACK or >>> HB_ACK? below is the testing result where a router is located in the >>> middle. >>> >>> Before starting the application. the packet on eth1 and eth2 are >>> >>> eth1 >>> 0 packets dropped by kernel >>> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes >>> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >>> 11:24:14.539486 >>> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >>> >>> eth2 >>> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol >>> decode >>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes >>> >>> When starting the application. the packet show as below. >>> >>> eth1 >>> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE >>> ECHO] >>> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >>> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >>> >>> eth2 >>> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >>> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >>> 2330749678] >>> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] >>> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >>> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] >>> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >>> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >>> >>> From the above result, you can see that the INIT, COOKIE ECHO and >>> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, >>> COOKIE_ACK, HB_ACK) are returned on eth2 using source address >>> 120.1.1.1 instead of 110.1.1.1. >>> >>> Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? >>> >>> For simple ICMP ping test, it is normal, but not for SCTP. >>> >>> eth1 >>> 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >>> seq 12, length 64 >>> 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >>> seq 12, length 64 >>> 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >>> seq 13, length 64 >>> 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >>> seq 13, length 64 >>> >>> eth2 >>> 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >>> seq 2, length 64 >>> 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138,
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 08:39 PM, Sun Paul wrote: > Another question > > if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad > > On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: >> Thanks Vlad >> >> I checked on the route, and it looks correct. >> >> [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 >> 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 >> 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 >> 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 >> 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 >> cache mtu 1500 advmss 1460 hoplimit 64 >> >> so, if this is not being handled in LKSCTP, is it possible to suggest >> a way how we can achieve it? >> >> On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: >>> On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: > On 11/27/2013 11:03 PM, Sun Paul wrote: >> How LKSCTP select which source address to use for the INIT_ACK or >> HB_ACK? below is the testing result where a router is located in the >> middle. >> >> Before starting the application. the packet on eth1 and eth2 are >> >> eth1 >> 0 packets dropped by kernel >> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes >> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >> 11:24:14.539486 >> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >> >> eth2 >> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full protocol >> decode >> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes >> >> When starting the application. the packet show as below. >> >> eth1 >> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] >> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >> >> eth2 >> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >> 2330749678] >> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] >> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] >> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >> >> From the above result, you can see that the INIT, COOKIE ECHO and >> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, >> COOKIE_ACK, HB_ACK) are returned on eth2 using source address >> 120.1.1.1 instead of 110.1.1.1. >> >> Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? >> >> For simple ICMP ping test, it is normal, but not for SCTP. >> >> eth1 >> 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >> seq 12, length 64 >> 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >> seq 12, length 64 >> 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >> seq 13, length 64 >> 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >> seq 13, length 64 >> >> eth2 >> 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >> seq 2, length 64 >> 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, >> seq 2, length 64 >> 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >> seq 3, length 64 >> 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, >> seq 3, length 64 >> >> Below is the route information >> #route -n >> Kernel IP routing table >> Destination Gateway
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 08:31 PM, Sun Paul wrote: > Thanks Vlad > > I checked on the route, and it looks correct. > > [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 > 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 > cache mtu 1500 advmss 1460 hoplimit 64 > > [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 > 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 > cache mtu 1500 advmss 1460 hoplimit 64 > > [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 > 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 > cache mtu 1500 advmss 1460 hoplimit 64 > > [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 > 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 > cache mtu 1500 advmss 1460 hoplimit 64 > > so, if this is not being handled in LKSCTP, is it possible to suggest > a way how we can achieve it? Like I said before lksctp only iterates over the bound address list if the source address from the route returned by the kernel does not match the bound address list. To simulate the algorithm with with ip route get, you'd get something like this: ip route get 11.1.1.1 if from_address not in bound address list for each bound_addr in bound address list ip route get 11.1.1.1 from bound_addr The kernel will almost always give you a valid route even if the from address does not belong to the expected interface. So, in your case, what does a simple ip route get 11.1.1.1 return? That's the address that will be used to talk to 11.1.1.1 by default. -vlad > > On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: >> On 12/02/2013 10:45 AM, Karl Heiss wrote: >>> On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: > How LKSCTP select which source address to use for the INIT_ACK or > HB_ACK? below is the testing result where a router is located in the > middle. > > Before starting the application. the packet on eth1 and eth2 are > > eth1 > 0 packets dropped by kernel > [root@localhost ~]# tcpdump -i eth1 -s 0 -nn > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes > 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] > 11:24:14.539486 > 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] > > eth2 > [root@localhost ~]# tcpdump -i eth2 -s 0 -nn > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes > > When starting the application. the packet show as below. > > eth1 > 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] > 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] > 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] > > eth2 > 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: > 2330749678] > 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] > 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > > From the above result, you can see that the INIT, COOKIE ECHO and > HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, > COOKIE_ACK, HB_ACK) are returned on eth2 using source address > 120.1.1.1 instead of 110.1.1.1. > > Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? > > For simple ICMP ping test, it is normal, but not for SCTP. > > eth1 > 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, > seq 12, length 64 > 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, > seq 12, length 64 > 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, > seq 13, length 64 > 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, > seq 13, length 64 > > eth2 > 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, > seq 2, length 64 > 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, > seq 2, length 64 >
Re: Supporting 4 way connections in LKSCTP
Another question if a wrong source IP is used, does the association still classified as normal? On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul wrote: > Thanks Vlad > > I checked on the route, and it looks correct. > > [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 > 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 > cache mtu 1500 advmss 1460 hoplimit 64 > > [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 > 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 > cache mtu 1500 advmss 1460 hoplimit 64 > > [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 > 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 > cache mtu 1500 advmss 1460 hoplimit 64 > > [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 > 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 > cache mtu 1500 advmss 1460 hoplimit 64 > > so, if this is not being handled in LKSCTP, is it possible to suggest > a way how we can achieve it? > > On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: >> On 12/02/2013 10:45 AM, Karl Heiss wrote: >>> On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: > How LKSCTP select which source address to use for the INIT_ACK or > HB_ACK? below is the testing result where a router is located in the > middle. > > Before starting the application. the packet on eth1 and eth2 are > > eth1 > 0 packets dropped by kernel > [root@localhost ~]# tcpdump -i eth1 -s 0 -nn > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes > 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] > 11:24:14.539486 > 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] > > eth2 > [root@localhost ~]# tcpdump -i eth2 -s 0 -nn > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes > > When starting the application. the packet show as below. > > eth1 > 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] > 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] > 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] > > eth2 > 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: > 2330749678] > 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] > 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > > From the above result, you can see that the INIT, COOKIE ECHO and > HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, > COOKIE_ACK, HB_ACK) are returned on eth2 using source address > 120.1.1.1 instead of 110.1.1.1. > > Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? > > For simple ICMP ping test, it is normal, but not for SCTP. > > eth1 > 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, > seq 12, length 64 > 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, > seq 12, length 64 > 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, > seq 13, length 64 > 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, > seq 13, length 64 > > eth2 > 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, > seq 2, length 64 > 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, > seq 2, length 64 > 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, > seq 3, length 64 > 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, > seq 3, length 64 > > Below is the route information > #route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric RefUse > Iface > 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth1 > 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 > eth2 > > # ip route show > 110.1.1.0/24
Re: Supporting 4 way connections in LKSCTP
Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich wrote: > On 12/02/2013 10:45 AM, Karl Heiss wrote: >> On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: >>> On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via
Re: Supporting 4 way connections in LKSCTP
On Mon, Dec 2, 2013 at 11:42 AM, Vlad Yasevich wrote: > On 12/02/2013 10:45 AM, Karl Heiss wrote: >> On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: >>> On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. >>> >>> LKSCTP will prefer the address returned from the routing table as long >>> as it is one of the addresses that is bound by the socket and are usable >>> by the association. >>> >>> If the address returned from the route lookup is not part of the >>> association, then lksctp attempts to lookup routes using one of the >>> source addresses it has
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 10:45 AM, Karl Heiss wrote: > On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: >> On 11/27/2013 11:03 PM, Sun Paul wrote: >>> How LKSCTP select which source address to use for the INIT_ACK or >>> HB_ACK? below is the testing result where a router is located in the >>> middle. >>> >>> Before starting the application. the packet on eth1 and eth2 are >>> >>> eth1 >>> 0 packets dropped by kernel >>> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >>> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes >>> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >>> 11:24:14.539486 >>> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >>> >>> eth2 >>> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes >>> >>> When starting the application. the packet show as below. >>> >>> eth1 >>> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >>> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] >>> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >>> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >>> >>> eth2 >>> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >>> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >>> 2330749678] >>> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] >>> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >>> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] >>> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >>> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >>> >>> From the above result, you can see that the INIT, COOKIE ECHO and >>> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, >>> COOKIE_ACK, HB_ACK) are returned on eth2 using source address >>> 120.1.1.1 instead of 110.1.1.1. >>> >>> Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? >>> >>> For simple ICMP ping test, it is normal, but not for SCTP. >>> >>> eth1 >>> 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >>> seq 12, length 64 >>> 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >>> seq 12, length 64 >>> 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >>> seq 13, length 64 >>> 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >>> seq 13, length 64 >>> >>> eth2 >>> 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >>> seq 2, length 64 >>> 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, >>> seq 2, length 64 >>> 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >>> seq 3, length 64 >>> 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, >>> seq 3, length 64 >>> >>> Below is the route information >>> #route -n >>> Kernel IP routing table >>> Destination Gateway Genmask Flags Metric RefUse >>> Iface >>> 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 >>> 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 >>> >>> # ip route show >>> 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 >>> 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 >>> >>> Since we are using iproute2, so we will have dedicate routing table >>> per interface >>> >>> # ip route show table SCTP1 >>> default via 110.1.1.254 dev eth1 >>> >>> # ip route show table SCTP2 >>> default via 120.1.1.254 dev eth2 >>> >>> # ip rule ls >>> 0: from all lookup local >>> 101: from 110.1.1.1 lookup SCTP1 >>> 102: from 120.1.1.1 lookup SCTP2 >>> 32766: from all lookup main >>> 32767: from all lookup default >>> >>> How LKSCTP select source address to reply? If we know how it works, >>> then we may know what is going wrong. >> >> LKSCTP will prefer the address returned from the routing table as long >> as it is one of the addresses that is bound by the socket and are usable >> by the association. >> >> If the address returned from the route lookup is not part of the >> association, then lksctp attempts to lookup routes using one of the >> source addresses it has available. Usually the first lookup succeeds >> due to the host-model implementation in linux. >> >> You may want to change your rule set to be destination based. Then >> in the
Re: Supporting 4 way connections in LKSCTP
On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote: > On 11/27/2013 11:03 PM, Sun Paul wrote: >> How LKSCTP select which source address to use for the INIT_ACK or >> HB_ACK? below is the testing result where a router is located in the >> middle. >> >> Before starting the application. the packet on eth1 and eth2 are >> >> eth1 >> 0 packets dropped by kernel >> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes >> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >> 11:24:14.539486 >> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] >> >> eth2 >> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn >> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode >> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes >> >> When starting the application. the packet show as below. >> >> eth1 >> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] >> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] >> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] >> >> eth2 >> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >> 2330749678] >> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] >> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] >> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >> >> From the above result, you can see that the INIT, COOKIE ECHO and >> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, >> COOKIE_ACK, HB_ACK) are returned on eth2 using source address >> 120.1.1.1 instead of 110.1.1.1. >> >> Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? >> >> For simple ICMP ping test, it is normal, but not for SCTP. >> >> eth1 >> 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >> seq 12, length 64 >> 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >> seq 12, length 64 >> 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, >> seq 13, length 64 >> 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, >> seq 13, length 64 >> >> eth2 >> 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >> seq 2, length 64 >> 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, >> seq 2, length 64 >> 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, >> seq 3, length 64 >> 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, >> seq 3, length 64 >> >> Below is the route information >> #route -n >> Kernel IP routing table >> Destination Gateway Genmask Flags Metric RefUse Iface >> 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 >> 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 >> >> # ip route show >> 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 >> 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 >> >> Since we are using iproute2, so we will have dedicate routing table >> per interface >> >> # ip route show table SCTP1 >> default via 110.1.1.254 dev eth1 >> >> # ip route show table SCTP2 >> default via 120.1.1.254 dev eth2 >> >> # ip rule ls >> 0: from all lookup local >> 101: from 110.1.1.1 lookup SCTP1 >> 102: from 120.1.1.1 lookup SCTP2 >> 32766: from all lookup main >> 32767: from all lookup default >> >> How LKSCTP select source address to reply? If we know how it works, >> then we may know what is going wrong. > > LKSCTP will prefer the address returned from the routing table as long > as it is one of the addresses that is bound by the socket and are usable > by the association. > > If the address returned from the route lookup is not part of the > association, then lksctp attempts to lookup routes using one of the > source addresses it has available. Usually the first lookup succeeds > due to the host-model implementation in linux. > > You may want to change your rule set to be destination based. Then > in the table associated with the rule, specify the source address > you want to be used. > > -vlad I have had similar qualms myself about this behavior, and I honestly
Re: Supporting 4 way connections in LKSCTP
On 11/27/2013 11:03 PM, Sun Paul wrote: > How LKSCTP select which source address to use for the INIT_ACK or > HB_ACK? below is the testing result where a router is located in the > middle. > > Before starting the application. the packet on eth1 and eth2 are > > eth1 > 0 packets dropped by kernel > [root@localhost ~]# tcpdump -i eth1 -s 0 -nn > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes > 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] > 11:24:14.539486 > 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] > > eth2 > [root@localhost ~]# tcpdump -i eth2 -s 0 -nn > tcpdump: verbose output suppressed, use -v or -vv for full protocol decode > listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes > > When starting the application. the packet show as below. > > eth1 > 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] > [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] > 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] > 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] > > eth2 > 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: > 2330749678] > 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] > 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > > From the above result, you can see that the INIT, COOKIE ECHO and > HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, > COOKIE_ACK, HB_ACK) are returned on eth2 using source address > 120.1.1.1 instead of 110.1.1.1. > > Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? > > For simple ICMP ping test, it is normal, but not for SCTP. > > eth1 > 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, > seq 12, length 64 > 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, > seq 12, length 64 > 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, > seq 13, length 64 > 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, > seq 13, length 64 > > eth2 > 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, > seq 2, length 64 > 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, > seq 2, length 64 > 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, > seq 3, length 64 > 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, > seq 3, length 64 > > Below is the route information > #route -n > Kernel IP routing table > Destination Gateway Genmask Flags Metric RefUse Iface > 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 > 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 > > # ip route show > 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 > 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 > > Since we are using iproute2, so we will have dedicate routing table > per interface > > # ip route show table SCTP1 > default via 110.1.1.254 dev eth1 > > # ip route show table SCTP2 > default via 120.1.1.254 dev eth2 > > # ip rule ls > 0: from all lookup local > 101: from 110.1.1.1 lookup SCTP1 > 102: from 120.1.1.1 lookup SCTP2 > 32766: from all lookup main > 32767: from all lookup default > > How LKSCTP select source address to reply? If we know how it works, > then we may know what is going wrong. LKSCTP will prefer the address returned from the routing table as long as it is one of the addresses that is bound by the socket and are usable by the association. If the address returned from the route lookup is not part of the association, then lksctp attempts to lookup routes using one of the source addresses it has available. Usually the first lookup succeeds due to the host-model implementation in linux. You may want to change your rule set to be destination based. Then in the table associated with the rule, specify the source address you want to be used. -vlad > > On Wed, Nov 27, 2013 at 8:45 PM, Neil Horman wrote: >> On Wed, Nov 27, 2013 at 07:10:49AM +0800, Sun Paul wrote: >>> Hi Vlad >>> >>> Thank for your reply. If it is based on the destination IP to find the >>> best route, why the
Re: Supporting 4 way connections in LKSCTP
On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. LKSCTP will prefer the address returned from the routing table as long as it is one of the addresses that is bound by the socket and are usable by the association. If the address returned from the route lookup is not part of the association, then lksctp attempts to lookup routes using one of the source addresses it has available. Usually the first lookup succeeds due to the host-model implementation in linux. You may want to change your rule set to be destination based. Then in the table associated with the rule, specify the source address you want to be used. -vlad On Wed, Nov 27, 2013 at 8:45 PM, Neil Horman nhor...@tuxdriver.com wrote: On Wed, Nov 27, 2013 at 07:10:49AM +0800, Sun Paul wrote: Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? Because You only ever use one address from NODE A (12.1.1.1) In the
Re: Supporting 4 way connections in LKSCTP
On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. LKSCTP will prefer the address returned from the routing table as long as it is one of the addresses that is bound by the socket and are usable by the association. If the address returned from the route lookup is not part of the association, then lksctp attempts to lookup routes using one of the source addresses it has available. Usually the first lookup succeeds due to the host-model implementation in linux. You may want to change your rule set to be destination based. Then in the table associated with the rule, specify the source address you want to be used. -vlad I have had similar qualms myself about this behavior, and I honestly don't know what the correct answer should be... In my opinion, shouldn't the source address just work for acknowledgements? If the spec explicitly states that the ACK should have a source address that matches the
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. LKSCTP will prefer the address returned from the routing table as long as it is one of the addresses that is bound by the socket and are usable by the association. If the address returned from the route lookup is not part of the association, then lksctp attempts to lookup routes using one of the source addresses it has available. Usually the first lookup succeeds due to the host-model implementation in linux. You may want to change your rule set to be destination based. Then in the table associated with the rule, specify the source address you want to be used. -vlad I have had similar qualms myself about this behavior, and I honestly don't know what the correct answer should be... In my opinion, shouldn't the source address just work for acknowledgements? If the spec explicitly states that the ACK
Re: Supporting 4 way connections in LKSCTP
On Mon, Dec 2, 2013 at 11:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. LKSCTP will prefer the address returned from the routing table as long as it is one of the addresses that is bound by the socket and are usable by the association. If the address returned from the route lookup is not part of the association, then lksctp attempts to lookup routes using one of the source addresses it has available. Usually the first lookup succeeds due to the host-model implementation in linux. You may want to change your rule set to be destination based. Then in the table associated with the rule, specify the source address you want to be used. -vlad I have had similar qualms myself about this behavior, and I honestly don't know what the correct answer should be... In my opinion, shouldn't the source address
Re: Supporting 4 way connections in LKSCTP
Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going
Re: Supporting 4 way connections in LKSCTP
Another question if a wrong source IP is used, does the association still classified as normal? On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 08:31 PM, Sun Paul wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? Like I said before lksctp only iterates over the bound address list if the source address from the route returned by the kernel does not match the bound address list. To simulate the algorithm with with ip route get, you'd get something like this: ip route get 11.1.1.1 if from_address not in bound address list for each bound_addr in bound address list ip route get 11.1.1.1 from bound_addr The kernel will almost always give you a valid route even if the from address does not belong to the expected interface. So, in your case, what does a simple ip route get 11.1.1.1 return? That's the address that will be used to talk to 11.1.1.1 by default. -vlad On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0
Re: Supporting 4 way connections in LKSCTP
On 12/02/2013 08:39 PM, Sun Paul wrote: Another question if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254
Re: Supporting 4 way connections in LKSCTP
so in this case, says (NODE-A) IP-B send INIT to IP-X (NODE-B), and then IP-Y (NODE-B) returns INIT_ACK to IP-B (NODE-A) this is also treated as a valid, am I correct? On Tue, Dec 3, 2013 at 10:03 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 08:39 PM, Sun Paul wrote: Another question if a wrong source IP is used, does the association still classified as normal? What do you mean my wrong source IP? As long as the address is part of the association, it can be used. -vlad On Tue, Dec 3, 2013 at 9:31 AM, Sun Paul paul...@gmail.com wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev
Re: Supporting 4 way connections in LKSCTP
if we do not specific the gateway, it will be a problem. One of the reason why it is like that is due to that the server do not know which source IP should be used to send out the query. [root@localhost ~]# ip route get 12.1.1.1 RTNETLINK answers: Network is unreachable [root@localhost ~]# ip route get 11.1.1.1 RTNETLINK answers: Network is unreachable On Tue, Dec 3, 2013 at 10:02 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 08:31 PM, Sun Paul wrote: Thanks Vlad I checked on the route, and it looks correct. [root@localhost ~]# ip route get 11.1.1.1 from 110.1.1.1 11.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 11.1.1.1 from 120.1.1.1 11.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 120.1.1.1 12.1.1.1 from 120.1.1.1 via 120.1.1.254 dev eth2 cache mtu 1500 advmss 1460 hoplimit 64 [root@localhost ~]# ip route get 12.1.1.1 from 110.1.1.1 12.1.1.1 from 110.1.1.1 via 110.1.1.254 dev eth1 cache mtu 1500 advmss 1460 hoplimit 64 so, if this is not being handled in LKSCTP, is it possible to suggest a way how we can achieve it? Like I said before lksctp only iterates over the bound address list if the source address from the route returned by the kernel does not match the bound address list. To simulate the algorithm with with ip route get, you'd get something like this: ip route get 11.1.1.1 if from_address not in bound address list for each bound_addr in bound address list ip route get 11.1.1.1 from bound_addr The kernel will almost always give you a valid route even if the from address does not belong to the expected interface. So, in your case, what does a simple ip route get 11.1.1.1 return? That's the address that will be used to talk to 11.1.1.1 by default. -vlad On Tue, Dec 3, 2013 at 12:42 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 12/02/2013 10:45 AM, Karl Heiss wrote: On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/27/2013 11:03 PM, Sun Paul wrote: How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64
Re: Supporting 4 way connections in LKSCTP
How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. On Wed, Nov 27, 2013 at 8:45 PM, Neil Horman wrote: > On Wed, Nov 27, 2013 at 07:10:49AM +0800, Sun Paul wrote: >> Hi Vlad >> >> Thank for your reply. If it is based on the destination IP to find the >> best route, why the problem didn't happen on single-homing sample? >> > Because You only ever use one address from NODE A (12.1.1.1) > >> In the single-homing sample that provided in the original email, both >> of the interfaces (eth1 and eth2) are presented on NODE-B during the >> test. However, the LKSCTP library know to use the interface eth1 to >> respond to the SCTP request. >> > Yes, because it does a route lookup to each of the two ip addresses to NODE B, > and in both lookups, the route indicates that only one source address should > be > used (12.1.1.1). If you issue a ip route show command, you'll see that routes > to both address on NODE B match on a rule that specifies the same src address > and interface be used. > > Neil > >> - PS >> >> On Wed, Nov 27, 2013 at 7:09 AM, Sun Paul wrote: >> > Hi Vlad >> > >> > Thank for your
Re: Supporting 4 way connections in LKSCTP
On Wed, Nov 27, 2013 at 07:10:49AM +0800, Sun Paul wrote: > Hi Vlad > > Thank for your reply. If it is based on the destination IP to find the > best route, why the problem didn't happen on single-homing sample? > Because You only ever use one address from NODE A (12.1.1.1) > In the single-homing sample that provided in the original email, both > of the interfaces (eth1 and eth2) are presented on NODE-B during the > test. However, the LKSCTP library know to use the interface eth1 to > respond to the SCTP request. > Yes, because it does a route lookup to each of the two ip addresses to NODE B, and in both lookups, the route indicates that only one source address should be used (12.1.1.1). If you issue a ip route show command, you'll see that routes to both address on NODE B match on a rule that specifies the same src address and interface be used. Neil > - PS > > On Wed, Nov 27, 2013 at 7:09 AM, Sun Paul wrote: > > Hi Vlad > > > > Thank for your reply. If it is based on the destination IP to find the > > best route, why the problem didn't happen on single-homing sample? > > > > In the single-homing sample that provided in the original email, both > > of the interfaces (eth1 and eth2) are presented on NODE-B during the > > test. However, the LKSCTP library know to use the interface eth1 to > > respond to the SCTP request. > > > > - PS > > > > On Tue, Nov 26, 2013 at 11:19 PM, Vlad Yasevich wrote: > >> On 11/25/2013 08:03 PM, Sun Paul wrote: > >>> Hi > >>> > >>> we have a problem on using LKSCTP to form a 4 ways multi-homing network. > >>> > >>> Configuration > >>> - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), > >>> IP-B (eth2) > >>> - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), > >>> IP-Y (eth2) > >>> > >> > >> First of all, this is not a 4 way multi-homed network. As far as each > >> SCTP association is concerned, it has only 2 destinations to send to > >> so it has only 2 ways to get there. The fact that you have multiple > >> local addresses doesn't mean that every local address can and should > >> be used to connect to the remote. > >> > >>> the four way paths are shown below. > >>> 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) > >>> 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) > >>> 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) > >>> 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) > >> > >> No, actually you only have 2 paths: one to IPX and one to IP-Y. > >> Which source address you choose is based on routing policy > >> decisions and is outside the scope of SCTP. > >> > >>> > >>> the HB/HB_ACK is normal for the paths " IP-A to IP-X" and "IP-B to > >>> IP-Y", but it is not correct for the rest of two. > >> > >> Right, because linux is using a host addressing model, not an interface > >> addressing model. SCTP stack simply finds the best source address > >> that can be used to reach IP-X and it happens to be IP-A. So that > >> is what it is going to use. > >> > >> The above explains why you are seeing what you describe below. > >> > >> In the end, linux SCTP implementation determines paths solely based > >> on the destination address. > >> > >> -vlad > >> > >>> > >>> First of all, we are using iproute2 to form 2 table such that when > >>> IP-B arrives on IP-X, it will know how to route back to IP-B on the > >>> same interface, i.e (eth1). Same logic for the path "IP-A to IP-X". > >>> > >>> What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, > >>> LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not > >>> using the IP 11.1.1.11. > >>> > >>> The above operation makes the subsequence HB/HB_ACK in using wrong IP > >>> address. > >>> > >>> TCP trace on eth1 > >>> 18:02:41.058640 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] > >>> [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > >>> 18:02:41.061634 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [COOKIE ECHO] > >>> 18:02:41.062642 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > >>> 18:02:41.062846 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > >>> 18:02:41.361811 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > >>> 18:02:41.661791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > >>> 18:02:41.961791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > >>> > >>> TCP trace on eth2 > >>> 18:02:41.058755 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > >>> [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: > >>> 3340756356] > >>> 18:02:41.061696 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > >>> 18:02:41.062663 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > >>> 18:02:41.062791 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > >>> 18:02:41.361777 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > >>> 18:02:41.661772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > >>> 18:02:41.961772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > >>> 18:02:42.161771 IP 11.1.1.1.2905 >
Re: Supporting 4 way connections in LKSCTP
On Wed, Nov 27, 2013 at 07:10:49AM +0800, Sun Paul wrote: Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? Because You only ever use one address from NODE A (12.1.1.1) In the single-homing sample that provided in the original email, both of the interfaces (eth1 and eth2) are presented on NODE-B during the test. However, the LKSCTP library know to use the interface eth1 to respond to the SCTP request. Yes, because it does a route lookup to each of the two ip addresses to NODE B, and in both lookups, the route indicates that only one source address should be used (12.1.1.1). If you issue a ip route show command, you'll see that routes to both address on NODE B match on a rule that specifies the same src address and interface be used. Neil - PS On Wed, Nov 27, 2013 at 7:09 AM, Sun Paul paul...@gmail.com wrote: Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? In the single-homing sample that provided in the original email, both of the interfaces (eth1 and eth2) are presented on NODE-B during the test. However, the LKSCTP library know to use the interface eth1 to respond to the SCTP request. - PS On Tue, Nov 26, 2013 at 11:19 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/25/2013 08:03 PM, Sun Paul wrote: Hi we have a problem on using LKSCTP to form a 4 ways multi-homing network. Configuration - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), IP-B (eth2) - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), IP-Y (eth2) First of all, this is not a 4 way multi-homed network. As far as each SCTP association is concerned, it has only 2 destinations to send to so it has only 2 ways to get there. The fact that you have multiple local addresses doesn't mean that every local address can and should be used to connect to the remote. the four way paths are shown below. 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) No, actually you only have 2 paths: one to IPX and one to IP-Y. Which source address you choose is based on routing policy decisions and is outside the scope of SCTP. the HB/HB_ACK is normal for the paths IP-A to IP-X and IP-B to IP-Y, but it is not correct for the rest of two. Right, because linux is using a host addressing model, not an interface addressing model. SCTP stack simply finds the best source address that can be used to reach IP-X and it happens to be IP-A. So that is what it is going to use. The above explains why you are seeing what you describe below. In the end, linux SCTP implementation determines paths solely based on the destination address. -vlad First of all, we are using iproute2 to form 2 table such that when IP-B arrives on IP-X, it will know how to route back to IP-B on the same interface, i.e (eth1). Same logic for the path IP-A to IP-X. What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not using the IP 11.1.1.11. The above operation makes the subsequence HB/HB_ACK in using wrong IP address. TCP trace on eth1 18:02:41.058640 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:02:41.061634 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:02:41.062642 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.062846 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.361811 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.661791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.961791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] TCP trace on eth2 18:02:41.058755 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 3340756356] 18:02:41.061696 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:02:41.062663 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.062791 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.361777 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.661772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.961772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.161771 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.461770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.675770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] If we are using single homing, there is no problem on the SCTP communication. Below is the TCP trace on eth1 using sctp_test
Re: Supporting 4 way connections in LKSCTP
How LKSCTP select which source address to use for the INIT_ACK or HB_ACK? below is the testing result where a router is located in the middle. Before starting the application. the packet on eth1 and eth2 are eth1 0 packets dropped by kernel [root@localhost ~]# tcpdump -i eth1 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes 11:24:14.262489 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:14.262522 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] 11:24:14.539486 11:24:16.262488 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:24:16.262520 IP 110.1.1.1.2905 12.1.1.1.2905: sctp (1) [ABORT] eth2 [root@localhost ~]# tcpdump -i eth2 -s 0 -nn tcpdump: verbose output suppressed, use -v or -vv for full protocol decode listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes When starting the application. the packet show as below. eth1 11:26:02.261511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [INIT] [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 11:26:02.263513 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [COOKIE ECHO] 11:26:02.264518 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.563511 IP 12.1.1.1.2905 110.1.1.1.2905: sctp (1) [HB REQ] eth2 11:26:02.261604 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 2330749678] 11:26:02.263583 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 11:26:02.264548 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.264652 IP 11.1.1.1.2905 120.1.1.1.2905: sctp (1) [HB REQ] 11:26:02.264705 IP 120.1.1.1.2905 11.1.1.1.2905: sctp (1) [HB ACK] 11:26:02.563543 IP 120.1.1.1.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the above result, you can see that the INIT, COOKIE ECHO and HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK, COOKIE_ACK, HB_ACK) are returned on eth2 using source address 120.1.1.1 instead of 110.1.1.1. Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1? For simple ICMP ping test, it is normal, but not for SCTP. eth1 11:30:02.824548 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 12, length 64 11:30:02.824559 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 12, length 64 11:30:03.825551 IP 12.1.1.1 110.1.1.1: ICMP echo request, id 37178, seq 13, length 64 11:30:03.825561 IP 110.1.1.1 12.1.1.1: ICMP echo reply, id 37178, seq 13, length 64 eth2 11:30:34.027687 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 2, length 64 11:30:34.027697 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 2, length 64 11:30:35.027686 IP 11.1.1.1 120.1.1.1: ICMP echo request, id 46138, seq 3, length 64 11:30:35.027694 IP 120.1.1.1 11.1.1.1: ICMP echo reply, id 46138, seq 3, length 64 Below is the route information #route -n Kernel IP routing table Destination Gateway Genmask Flags Metric RefUse Iface 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2 # ip route show 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1 Since we are using iproute2, so we will have dedicate routing table per interface # ip route show table SCTP1 default via 110.1.1.254 dev eth1 # ip route show table SCTP2 default via 120.1.1.254 dev eth2 # ip rule ls 0: from all lookup local 101: from 110.1.1.1 lookup SCTP1 102: from 120.1.1.1 lookup SCTP2 32766: from all lookup main 32767: from all lookup default How LKSCTP select source address to reply? If we know how it works, then we may know what is going wrong. On Wed, Nov 27, 2013 at 8:45 PM, Neil Horman nhor...@tuxdriver.com wrote: On Wed, Nov 27, 2013 at 07:10:49AM +0800, Sun Paul wrote: Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? Because You only ever use one address from NODE A (12.1.1.1) In the single-homing sample that provided in the original email, both of the interfaces (eth1 and eth2) are presented on NODE-B during the test. However, the LKSCTP library know to use the interface eth1 to respond to the SCTP request. Yes, because it does a route lookup to each of the two ip addresses to NODE B, and in both lookups, the route indicates that only one source address should be used (12.1.1.1). If you issue a ip route show command, you'll see that routes to both address on NODE B match on a rule that specifies the same src address and interface be used. Neil - PS On Wed, Nov 27, 2013 at 7:09 AM, Sun Paul paul...@gmail.com wrote: Hi Vlad Thank for your reply. If it is based on the
Re: Supporting 4 way connections in LKSCTP
Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? In the single-homing sample that provided in the original email, both of the interfaces (eth1 and eth2) are presented on NODE-B during the test. However, the LKSCTP library know to use the interface eth1 to respond to the SCTP request. - PS On Wed, Nov 27, 2013 at 7:09 AM, Sun Paul wrote: > Hi Vlad > > Thank for your reply. If it is based on the destination IP to find the > best route, why the problem didn't happen on single-homing sample? > > In the single-homing sample that provided in the original email, both > of the interfaces (eth1 and eth2) are presented on NODE-B during the > test. However, the LKSCTP library know to use the interface eth1 to > respond to the SCTP request. > > - PS > > On Tue, Nov 26, 2013 at 11:19 PM, Vlad Yasevich wrote: >> On 11/25/2013 08:03 PM, Sun Paul wrote: >>> Hi >>> >>> we have a problem on using LKSCTP to form a 4 ways multi-homing network. >>> >>> Configuration >>> - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), >>> IP-B (eth2) >>> - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), >>> IP-Y (eth2) >>> >> >> First of all, this is not a 4 way multi-homed network. As far as each >> SCTP association is concerned, it has only 2 destinations to send to >> so it has only 2 ways to get there. The fact that you have multiple >> local addresses doesn't mean that every local address can and should >> be used to connect to the remote. >> >>> the four way paths are shown below. >>> 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) >>> 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) >>> 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) >>> 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) >> >> No, actually you only have 2 paths: one to IPX and one to IP-Y. >> Which source address you choose is based on routing policy >> decisions and is outside the scope of SCTP. >> >>> >>> the HB/HB_ACK is normal for the paths " IP-A to IP-X" and "IP-B to >>> IP-Y", but it is not correct for the rest of two. >> >> Right, because linux is using a host addressing model, not an interface >> addressing model. SCTP stack simply finds the best source address >> that can be used to reach IP-X and it happens to be IP-A. So that >> is what it is going to use. >> >> The above explains why you are seeing what you describe below. >> >> In the end, linux SCTP implementation determines paths solely based >> on the destination address. >> >> -vlad >> >>> >>> First of all, we are using iproute2 to form 2 table such that when >>> IP-B arrives on IP-X, it will know how to route back to IP-B on the >>> same interface, i.e (eth1). Same logic for the path "IP-A to IP-X". >>> >>> What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, >>> LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not >>> using the IP 11.1.1.11. >>> >>> The above operation makes the subsequence HB/HB_ACK in using wrong IP >>> address. >>> >>> TCP trace on eth1 >>> 18:02:41.058640 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] >>> [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 18:02:41.061634 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [COOKIE ECHO] >>> 18:02:41.062642 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:41.062846 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >>> 18:02:41.361811 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >>> 18:02:41.661791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >>> 18:02:41.961791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] >>> >>> TCP trace on eth2 >>> 18:02:41.058755 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >>> [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: >>> 3340756356] >>> 18:02:41.061696 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] >>> 18:02:41.062663 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] >>> 18:02:41.062791 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:41.361777 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:41.661772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:41.961772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:42.161771 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:42.461770 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> 18:02:42.675770 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] >>> >>> >>> If we are using single homing, there is no problem on the SCTP >>> communication. Below is the TCP trace on eth1 using sctp_test >>> >>> 18:09:55.356727 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] >>> [init tag: 32516609] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] >>> 18:09:55.356811 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] >>> [init tag: 3168861995] [rwnd: 131072] [OS: 10] [MIS: 16] [init TSN: >>> 1877695021] >>> 18:09:55.357727 IP 12.1.1.1.2905 >
Re: Supporting 4 way connections in LKSCTP
On 11/25/2013 08:03 PM, Sun Paul wrote: > Hi > > we have a problem on using LKSCTP to form a 4 ways multi-homing network. > > Configuration > - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), > IP-B (eth2) > - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), > IP-Y (eth2) > First of all, this is not a 4 way multi-homed network. As far as each SCTP association is concerned, it has only 2 destinations to send to so it has only 2 ways to get there. The fact that you have multiple local addresses doesn't mean that every local address can and should be used to connect to the remote. > the four way paths are shown below. > 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) > 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) > 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) > 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) No, actually you only have 2 paths: one to IPX and one to IP-Y. Which source address you choose is based on routing policy decisions and is outside the scope of SCTP. > > the HB/HB_ACK is normal for the paths " IP-A to IP-X" and "IP-B to > IP-Y", but it is not correct for the rest of two. Right, because linux is using a host addressing model, not an interface addressing model. SCTP stack simply finds the best source address that can be used to reach IP-X and it happens to be IP-A. So that is what it is going to use. The above explains why you are seeing what you describe below. In the end, linux SCTP implementation determines paths solely based on the destination address. -vlad > > First of all, we are using iproute2 to form 2 table such that when > IP-B arrives on IP-X, it will know how to route back to IP-B on the > same interface, i.e (eth1). Same logic for the path "IP-A to IP-X". > > What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, > LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not > using the IP 11.1.1.11. > > The above operation makes the subsequence HB/HB_ACK in using wrong IP address. > > TCP trace on eth1 > 18:02:41.058640 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] > [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 18:02:41.061634 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [COOKIE ECHO] > 18:02:41.062642 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.062846 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.361811 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.661791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.961791 IP 11.1.1.11.2905 > 11.1.1.1.2905: sctp (1) [HB ACK] > > TCP trace on eth2 > 18:02:41.058755 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: > 3340756356] > 18:02:41.061696 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 18:02:41.062663 IP 12.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 18:02:41.062791 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.361777 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.661772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:41.961772 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:42.161771 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:42.461770 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > 18:02:42.675770 IP 11.1.1.1.2905 > 12.1.1.11.2905: sctp (1) [HB REQ] > > > If we are using single homing, there is no problem on the SCTP > communication. Below is the TCP trace on eth1 using sctp_test > > 18:09:55.356727 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [INIT] > [init tag: 32516609] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] > 18:09:55.356811 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK] > [init tag: 3168861995] [rwnd: 131072] [OS: 10] [MIS: 16] [init TSN: > 1877695021] > 18:09:55.357727 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [COOKIE ECHO] > 18:09:55.357788 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK] > 18:09:55.358724 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > 18:09:55.358740 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > 18:09:55.379715 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [DATA] > (B)(E) [TSN: 0] [SID: 0] [SSEQ 0] [PPID 0x3] > 18:09:55.379735 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [SACK] > [cum ack 0] [a_rwnd 131064] [#gap acks 0] [#dup tsns 0] > 18:09:55.657716 IP 12.1.1.1.2905 > 11.1.1.11.2905: sctp (1) [HB REQ] > 18:09:55.657732 IP 11.1.1.11.2905 > 12.1.1.1.2905: sctp (1) [HB ACK] > > From the observations, it seems that the LKSCTP library is not able to > use the original local address when multi-homing is being used. Is > there anyway can be resolved it? > > Thanks > > PS > -- > To unsubscribe from this list: send the line "unsubscribe linux-sctp" in > the body of a message to majord...@vger.kernel.org > More majordomo info at http://vger.kernel.org/majordomo-info.html > -- To unsubscribe from this list: send the line "unsubscribe
Re: Supporting 4 way connections in LKSCTP
On 11/25/2013 08:03 PM, Sun Paul wrote: Hi we have a problem on using LKSCTP to form a 4 ways multi-homing network. Configuration - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), IP-B (eth2) - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), IP-Y (eth2) First of all, this is not a 4 way multi-homed network. As far as each SCTP association is concerned, it has only 2 destinations to send to so it has only 2 ways to get there. The fact that you have multiple local addresses doesn't mean that every local address can and should be used to connect to the remote. the four way paths are shown below. 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) No, actually you only have 2 paths: one to IPX and one to IP-Y. Which source address you choose is based on routing policy decisions and is outside the scope of SCTP. the HB/HB_ACK is normal for the paths IP-A to IP-X and IP-B to IP-Y, but it is not correct for the rest of two. Right, because linux is using a host addressing model, not an interface addressing model. SCTP stack simply finds the best source address that can be used to reach IP-X and it happens to be IP-A. So that is what it is going to use. The above explains why you are seeing what you describe below. In the end, linux SCTP implementation determines paths solely based on the destination address. -vlad First of all, we are using iproute2 to form 2 table such that when IP-B arrives on IP-X, it will know how to route back to IP-B on the same interface, i.e (eth1). Same logic for the path IP-A to IP-X. What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not using the IP 11.1.1.11. The above operation makes the subsequence HB/HB_ACK in using wrong IP address. TCP trace on eth1 18:02:41.058640 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:02:41.061634 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:02:41.062642 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.062846 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.361811 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.661791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.961791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] TCP trace on eth2 18:02:41.058755 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 3340756356] 18:02:41.061696 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:02:41.062663 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.062791 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.361777 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.661772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.961772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.161771 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.461770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.675770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] If we are using single homing, there is no problem on the SCTP communication. Below is the TCP trace on eth1 using sctp_test 18:09:55.356727 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 32516609] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:09:55.356811 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3168861995] [rwnd: 131072] [OS: 10] [MIS: 16] [init TSN: 1877695021] 18:09:55.357727 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:09:55.357788 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:09:55.358724 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:09:55.358740 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] 18:09:55.379715 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [DATA] (B)(E) [TSN: 0] [SID: 0] [SSEQ 0] [PPID 0x3] 18:09:55.379735 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [SACK] [cum ack 0] [a_rwnd 131064] [#gap acks 0] [#dup tsns 0] 18:09:55.657716 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:09:55.657732 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] From the observations, it seems that the LKSCTP library is not able to use the original local address when multi-homing is being used. Is there anyway can be resolved it? Thanks PS -- To unsubscribe from this list: send the line unsubscribe linux-sctp in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-kernel in the body of a message to majord...@vger.kernel.org More majordomo info at
Re: Supporting 4 way connections in LKSCTP
Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? In the single-homing sample that provided in the original email, both of the interfaces (eth1 and eth2) are presented on NODE-B during the test. However, the LKSCTP library know to use the interface eth1 to respond to the SCTP request. - PS On Wed, Nov 27, 2013 at 7:09 AM, Sun Paul paul...@gmail.com wrote: Hi Vlad Thank for your reply. If it is based on the destination IP to find the best route, why the problem didn't happen on single-homing sample? In the single-homing sample that provided in the original email, both of the interfaces (eth1 and eth2) are presented on NODE-B during the test. However, the LKSCTP library know to use the interface eth1 to respond to the SCTP request. - PS On Tue, Nov 26, 2013 at 11:19 PM, Vlad Yasevich vyasev...@gmail.com wrote: On 11/25/2013 08:03 PM, Sun Paul wrote: Hi we have a problem on using LKSCTP to form a 4 ways multi-homing network. Configuration - Node-A has 2 IP addresses in different subnets, known as IP-A (eth1), IP-B (eth2) - Node-B has 2 IP addresses in different subnets, known as IP-X (eth1), IP-Y (eth2) First of all, this is not a 4 way multi-homed network. As far as each SCTP association is concerned, it has only 2 destinations to send to so it has only 2 ways to get there. The fact that you have multiple local addresses doesn't mean that every local address can and should be used to connect to the remote. the four way paths are shown below. 1. IP-A (11.1.1.1) to IP-X (11.1.1.11) 2. IP-B (12.1.1.1) to IP-Y (12.1.1.11) 3. IP-A (11.1.1.1) to IP-Y (12.1.1.11) 4. IP-B (12.1.1.1) to IP-X (11.1.1.11) No, actually you only have 2 paths: one to IPX and one to IP-Y. Which source address you choose is based on routing policy decisions and is outside the scope of SCTP. the HB/HB_ACK is normal for the paths IP-A to IP-X and IP-B to IP-Y, but it is not correct for the rest of two. Right, because linux is using a host addressing model, not an interface addressing model. SCTP stack simply finds the best source address that can be used to reach IP-X and it happens to be IP-A. So that is what it is going to use. The above explains why you are seeing what you describe below. In the end, linux SCTP implementation determines paths solely based on the destination address. -vlad First of all, we are using iproute2 to form 2 table such that when IP-B arrives on IP-X, it will know how to route back to IP-B on the same interface, i.e (eth1). Same logic for the path IP-A to IP-X. What we observed here is that when 12.1.1.1 sends INIT to 11.1.1.11, LKSCTP will send back the INIT_ACK to 12.1.1.1 using 12.1.1.11 but not using the IP 11.1.1.11. The above operation makes the subsequence HB/HB_ACK in using wrong IP address. TCP trace on eth1 18:02:41.058640 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 19933036] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:02:41.061634 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:02:41.062642 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.062846 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.361811 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.661791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.961791 IP 11.1.1.11.2905 11.1.1.1.2905: sctp (1) [HB ACK] TCP trace on eth2 18:02:41.058755 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 424726157] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN: 3340756356] 18:02:41.061696 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:02:41.062663 IP 12.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK] 18:02:41.062791 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.361777 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.661772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:41.961772 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.161771 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.461770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] 18:02:42.675770 IP 11.1.1.1.2905 12.1.1.11.2905: sctp (1) [HB REQ] If we are using single homing, there is no problem on the SCTP communication. Below is the TCP trace on eth1 using sctp_test 18:09:55.356727 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [INIT] [init tag: 32516609] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0] 18:09:55.356811 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [INIT ACK] [init tag: 3168861995] [rwnd: 131072] [OS: 10] [MIS: 16] [init TSN: 1877695021] 18:09:55.357727 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [COOKIE ECHO] 18:09:55.357788 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [COOKIE ACK] 18:09:55.358724 IP 12.1.1.1.2905 11.1.1.11.2905: sctp (1) [HB REQ] 18:09:55.358740 IP 11.1.1.11.2905 12.1.1.1.2905: sctp (1) [HB ACK]
