On Mon, Dec 2, 2013 at 9:38 AM, Vlad Yasevich wrote:
> On 11/27/2013 11:03 PM, Sun Paul wrote:
>> How LKSCTP select which source address to use for the INIT_ACK or
>> HB_ACK? below is the testing result where a router is located in the
>> middle.
>>
>> Before starting the application. the packet on eth1 and eth2 are
>>
>> eth1
>> 0 packets dropped by kernel
>> [root@localhost ~]# tcpdump -i eth1 -s 0 -nn
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on eth1, link-type EN10MB (Ethernet), capture size 65535 bytes
>> 11:24:14.262489 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT]
>> [init tag: 28362903] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0]
>> 11:24:14.262522 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT]
>> 11:24:14.539486
>> 11:24:16.262488 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT]
>> [init tag: 29391734] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0]
>> 11:24:16.262520 IP 110.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [ABORT]
>>
>> eth2
>> [root@localhost ~]# tcpdump -i eth2 -s 0 -nn
>> tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
>> listening on eth2, link-type EN10MB (Ethernet), capture size 65535 bytes
>>
>> When starting the application. the packet show as below.
>>
>> eth1
>> 11:26:02.261511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [INIT]
>> [init tag: 26256828] [rwnd: 102400] [OS: 16] [MIS: 16] [init TSN: 0]
>> 11:26:02.263513 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [COOKIE ECHO]
>> 11:26:02.264518 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ]
>> 11:26:02.563511 IP 12.1.1.1.2905 > 110.1.1.1.2905: sctp (1) [HB REQ]
>>
>> eth2
>> 11:26:02.261604 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [INIT ACK]
>> [init tag: 3478239387] [rwnd: 131072] [OS: 5] [MIS: 5] [init TSN:
>> 2330749678]
>> 11:26:02.263583 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [COOKIE ACK]
>> 11:26:02.264548 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK]
>> 11:26:02.264652 IP 11.1.1.1.2905 > 120.1.1.1.2905: sctp (1) [HB REQ]
>> 11:26:02.264705 IP 120.1.1.1.2905 > 11.1.1.1.2905: sctp (1) [HB ACK]
>> 11:26:02.563543 IP 120.1.1.1.2905 > 12.1.1.1.2905: sctp (1) [HB ACK]
>>
>> From the above result, you can see that the INIT, COOKIE ECHO and
>> HB_REQ originated from 12.1.1.1 on eth1, but the ACK (INIT_ACK,
>> COOKIE_ACK, HB_ACK) are returned on eth2 using source address
>> 120.1.1.1 instead of 110.1.1.1.
>>
>> Why LKSCTP use 120.1.1.1 as source instead of 110.1.1.1?
>>
>> For simple ICMP ping test, it is normal, but not for SCTP.
>>
>> eth1
>> 11:30:02.824548 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178,
>> seq 12, length 64
>> 11:30:02.824559 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178,
>> seq 12, length 64
>> 11:30:03.825551 IP 12.1.1.1 > 110.1.1.1: ICMP echo request, id 37178,
>> seq 13, length 64
>> 11:30:03.825561 IP 110.1.1.1 > 12.1.1.1: ICMP echo reply, id 37178,
>> seq 13, length 64
>>
>> eth2
>> 11:30:34.027687 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138,
>> seq 2, length 64
>> 11:30:34.027697 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138,
>> seq 2, length 64
>> 11:30:35.027686 IP 11.1.1.1 > 120.1.1.1: ICMP echo request, id 46138,
>> seq 3, length 64
>> 11:30:35.027694 IP 120.1.1.1 > 11.1.1.1: ICMP echo reply, id 46138,
>> seq 3, length 64
>>
>> Below is the route information
>> #route -n
>> Kernel IP routing table
>> Destination Gateway Genmask Flags Metric RefUse Iface
>> 110.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth1
>> 120.1.1.0 0.0.0.0 255.255.255.0 U 0 00 eth2
>>
>> # ip route show
>> 110.1.1.0/24 dev eth1 proto kernel scope link src 110.1.1.1
>> 120.1.1.0/24 dev eth2 proto kernel scope link src 120.1.1.1
>>
>> Since we are using iproute2, so we will have dedicate routing table
>> per interface
>>
>> # ip route show table SCTP1
>> default via 110.1.1.254 dev eth1
>>
>> # ip route show table SCTP2
>> default via 120.1.1.254 dev eth2
>>
>> # ip rule ls
>> 0: from all lookup local
>> 101: from 110.1.1.1 lookup SCTP1
>> 102: from 120.1.1.1 lookup SCTP2
>> 32766: from all lookup main
>> 32767: from all lookup default
>>
>> How LKSCTP select source address to reply? If we know how it works,
>> then we may know what is going wrong.
>
> LKSCTP will prefer the address returned from the routing table as long
> as it is one of the addresses that is bound by the socket and are usable
> by the association.
>
> If the address returned from the route lookup is not part of the
> association, then lksctp attempts to lookup routes using one of the
> source addresses it has available. Usually the first lookup succeeds
> due to the host-model implementation in linux.
>
> You may want to change your rule set to be destination based. Then
> in the table associated with the rule, specify the source address
> you want to be used.
>
> -vlad
I have had similar qualms myself about this behavior, and I honestly
do
