Hello, No, I gave up in the end. Fortunately I was able to do what I needed with policy-based routing, which is not the same as VRF, but it did the trick in my scenario...
BR, George On 14 October 2017 at 01:22, Marrold <kamai...@marrold.co.uk> wrote: > Hi George, > > Did you have any luck getting VRFs working with Kamailio? > > Thanks > > Matthew > > On Thu, Oct 12, 2017 at 8:06 PM, Mark Boyce <m...@darkorigins.com> wrote: > >> Hi >> >> I *think* I may have found it … although still not sure what’s doing the >> filtering. >> >> If I force the sending IP to be the the VPN interface on the sender by >> adding; >> >> modparam("siptrace", "force_send_sock", "sip:10.0.0.2:5060”) >> >> So packets now arrive at the receiver from the 10. IP rather than from >> the public IP of the sender >> >> … and magically it starts working. >> >> So something in Ubuntu 16.04 / zerotier is “filtering” packets which >> arrive on the VPN interface but with a no VPN IP >> >> Cheers >> >> Mark >> >> >> On 12 Oct 2017, at 19:59, Sergey Safarov <s.safa...@gmail.com> wrote: >> >> This may be NAT/iptables issue if used some type of virtualization >> Check UNDEPLIED connection using conntrack >> >> Sergey >> >> чт, 12 окт. 2017 г. в 20:27, Mark Boyce <m...@darkorigins.com>: >> >>> Hi >>> >>> Now using listen=udp:0.0.0.0:9060 >>> >>> net stat agrees; >>> >>> netstat -plunt | grep kamailio >>> udp 0 0 0.0.0.0:9060 0.0.0.0:* >>> 20486/kamailio >>> >>> >>> TCPDump of Sending / Receiving to zero tier port >>> >>> 17:06:19.617467 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP >>> 0x0000: 4510 04f1 a504 0000 4011 904e 894a abd8 E.......@ >>> ..N.J.. >>> 0x0010: 0a05 0172 13c4 2364 04dd 261b 0110 >>> 0211 ...r..#d..&..... >>> 0x0020: 13c4 13d8 894a abd8 894a abd8 494e >>> 5649 .....J...J..INVI >>> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip: >>> 2@sbc-1.w >>> >>> Nothing in Kamailio Logs >>> >>> Swap to using the real IP and I’m seeing the log line from the top of >>> the main route; >>> >>> Oct 12 19:09:32 vps465590 homer[20491]: ERROR: <script>: route entered >>> >>> and the matching tcpdump from the public interface; >>> >>> 17:10:20.333000 IP sbc-1.white-label.com.sip > homer-test.9060: SIP >>> 0x0000: 4510 03d8 863c 0000 3911 6109 894a >>> abd8 E....<..9.a..J.. >>> 0x0010: 33ff 2d9e 13c4 2364 03c4 f5a8 0110 >>> 0216 3.-...#d........ >>> 0x0020: 07de 13c5 5c17 3523 894a abd8 494e >>> 5649 ....\.5#.J..INVI >>> 0x0030: 5445 2073 6970 3a32 4073 6263 2d31 2e77 TE.sip: >>> 2@sbc-1.w >>> >>> >>> Loading the dumped packets in to wireshark I can’t see any obvious >>> difference. >>> >>> >>> Will start up a new thread … thanks for the loan of yours :-) >>> >>> Mark >>> >>> >>> On 12 Oct 2017, at 17:21, George Diamantopoulos <georged...@gmail.com> >>> wrote: >>> >>> Hmm... Everything looks fine indeed, I can't see any obvious reason why >>> this isn't working... >>> >>> Just to be clear, have you tried listening on all interfaces: >>> listen=udp:0.0.0.0:9060 >>> >>> And if yes, can you capture traffic arriving on the ZeroTier interface >>> in this case? If it works, is there any reason why listening on all >>> interfaces is not acceptable in your use case? >>> >>> Regardless, it might be a good idea to start a new thread about this, >>> maybe it's related to the inner workings of the ZeroTier device and how it >>> interacts with the kernel... >>> >>> >>> On 12 October 2017 at 18:26, Mark Boyce <m...@darkorigins.com> wrote: >>> >>>> Hi George >>>> >>>> (IP / MACs below anonymised a bit) >>>> >>>> >>>> I’m working on a https://www.zerotier.com VPN endpoint which presents >>>> as a new interface; >>>> >>>> >>>> Real Interface >>>> >>>> ens3 Link encap:Ethernet HWaddr xx:xx:xx:xx:xx:xx >>>> inet addr: 1.2.3.4 Bcast: 1.2.3.4 Mask:255.255.255.255 >>>> inet6 addr: aaaaa::bbbb:cccc:dddd:eeee/64 Scope:Link >>>> UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1 >>>> RX packets:12978431 errors:0 dropped:0 overruns:0 frame:0 >>>> TX packets:11331437 errors:0 dropped:0 overruns:0 carrier:0 >>>> collisions:0 txqueuelen:1000 >>>> RX bytes:2508723344 <(250)%20872-3344> (2.5 GB) TX >>>> bytes:1149701606 (1.1 GB) >>>> >>>> loopback >>>> >>>> lo Link encap:Local Loopback >>>> inet addr:127.0.0.1 Mask:255.0.0.0 >>>> inet6 addr: ::1/128 Scope:Host >>>> UP LOOPBACK RUNNING MTU:65536 Metric:1 >>>> RX packets:58936 errors:0 dropped:0 overruns:0 frame:0 >>>> TX packets:58936 errors:0 dropped:0 overruns:0 carrier:0 >>>> collisions:0 txqueuelen:1 >>>> RX bytes:12470726 (12.4 MB) TX bytes:12470726 (12.4 MB) >>>> >>>> ZeroTier >>>> >>>> zt0 Link encap:Ethernet HWaddr yy:yy:yy:yy:yy:yy >>>> inet addr:10.0.0.1 Bcast:10.0.0.255 Mask:255.255.255.0 >>>> UP BROADCAST RUNNING MULTICAST MTU:2800 Metric:1 >>>> RX packets:11148 errors:0 dropped:0 overruns:0 frame:0 >>>> TX packets:1113 errors:0 dropped:0 overruns:0 carrier:0 >>>> collisions:0 txqueuelen:1000 >>>> RX bytes:3760555 (3.7 MB) TX bytes:584324 (584.3 KB) >>>> >>>> >>>> This is actually a SIPCapture/Homer node so the config is; >>>> >>>> modparam("sipcapture", "capture_on", 1) >>>> modparam("sipcapture", "hep_capture_on", 1) >>>> listen=udp:10.0.0.1:9060 >>>> >>>> The port bind looks ok; >>>> >>>> # netstat -plunt | grep kamailio >>>> udp 0 0 10.0.0.1:9060 0.0.0.0:* >>>> 19592/kamailio >>>> >>>> >>>> I’m tracking packets at the top of "route {“ >>>> >>>> route { >>>> xlog("route entered\r\n”); >>>> >>>> >>>> >>>> If I tcpdump the ZeroTier Interface ; >>>> >>>> tcpdump -i zt0 -p -X port 9060 >>>> >>>> 15:08:38.556323 IP sbc-1.white-label.com.sip > 10.0.0.1.9060: SIP >>>> 0x0000: 4510 0181 7883 0000 4011 c03f 894a abd8 E...x...@ >>>> ..?.J.. >>>> 0x0010: 0a05 0172 13c4 2364 016d e2e2 0110 >>>> 0216 ...r..#d.m...... >>>> 0x0020: 07a8 13c5 5c17 3523 894a abd8 4143 >>>> 4b20 ....\.5#.J..ACK. >>>> >>>> >>>> … but nothing seen in kamailio. >>>> >>>> If I swap over to sending to / listening on the public IP everything >>>> works as expected. >>>> >>>> >>>> >>>> Cheers >>>> Mark >>>> >>>> >>>> On 12 Oct 2017, at 15:56, George Diamantopoulos <georged...@gmail.com> >>>> wrote: >>>> >>>> Hello, >>>> >>>> That seems strange... What kind of device is that VPN device? Like a >>>> ppp0 interface? What is the output of "netstat -plunt | grep kamailio"? Are >>>> you using a "private" IP for the VPN interface and what's your main routing >>>> table like? >>>> >>>> Also what if you omit the "listen" directive in kamailio configuration? >>>> Does it listen on the VPN interface then? Kamailio will bind by default to >>>> all interfaces if no "listen" directive is used... >>>> >>>> On 12 October 2017 at 00:26, Mark Boyce <m...@darkorigins.com> wrote: >>>> >>>>> Hi All >>>>> >>>>> Just to dive in here with something which may be related. I was about >>>>> to do a post with a much simpler problem where I have one real NIC and one >>>>> VPN created NIC. If I listen to the public IP all’s well. If I listen to >>>>> the IP of the VPN NIC Kamailio doesn’t see any traffic. I can tcpdump on >>>>> either NIC and see the traffic arriving. >>>>> >>>>> Anyone seen / solved this before? >>>>> >>>>> Cheers >>>>> Mark >>>>> >>>>> On 11 Oct 2017, at 22:07, George Diamantopoulos <georged...@gmail.com> >>>>> wrote: >>>>> >>>>> Hello Daniel, >>>>> >>>>> I'm including the original sketch for clarity: >>>>> >>>>> +----------+ +-------------------+ >>>>> | eth0 | | vrf-green | >>>>> | 1.1.1.1 | | 127.0.0.1 | >>>>> +----------+ +-------------------+ >>>>> | >>>>> +----------+ >>>>> | eth1 | >>>>> | 2.2.2.2 | >>>>> +----------+ >>>>> >>>>> Thanks for the reply. Indeed, using the advertise option for the >>>>> "listen" directive will result in kamaiio no longer failing to start when >>>>> using force_send_socket(2.2.2.2:5060) with 2.2.2.2 being used in the >>>>> "advertise" option. >>>>> >>>>> However, this doesn't make much of a difference in that kamailio still >>>>> won't process IP packets arriving on eth1 at IP 2.2.2.2... I can see >>>>> OPTIONS coming with sngrep, but nothing in the logs, no replies, only >>>>> retransmissions from the originator. >>>>> In addition, while without VRF kamailio will select the right >>>>> interface with mhomed=1, this is no longer the case. >>>>> Also, If I try to send something to a host with force_send_socket, the >>>>> transaction fails with 477 Unfortunately error on sending to next hop >>>>> occurred. In the log, kamailio prints something along the lines of: >>>>> udp_send(): sendto(sock,0x7f5c4f937d08,1888,0,9.9.9.9:5060,16): >>>>> Invalid argument(22) >>>>> udp_send(): invalid sendtoparameters >>>>> msg_send_buffer(): udp_send failed >>>>> where 9.9.9.9 is the $dd and presumably force_send_socket(2.2.2.2:5060) >>>>> has been called before t_relay() >>>>> >>>>> Also, I additionally found out that if the networking configuration is >>>>> as follows: >>>>> >>>>> +----------+ +-------------------+ +-------------------+ >>>>> | eth0 | | vrf-green | | vrf-red | >>>>> | 1.1.1.1 | | 127.0.0.1 | | 127.0.0.1 | >>>>> +----------+ +-------------------+ +-------------------+ >>>>> | | >>>>> +----------+ +----------+ >>>>> | eth1 | | eth2 | >>>>> | 2.2.2.2 | | 3.3.3.3 | >>>>> +----------+ +----------+ >>>>> >>>>> Then this won't work for vrf-red: >>>>> listen=vrf-green:5060 advertise 2.2.2.2:5060 >>>>> listen=vrf-red:5060 advertise 3.3.3.3:5060 >>>>> One needs to use different IPs for the MASTER devices (say 127.0.0.1 >>>>> and 127.0.0.2) for kamailio to startup with the above listen directives. >>>>> >>>>> It's a bit surprising it doesn't work. According to a user on cumulus >>>>> slack channel (cumulus contributed the VRF code to the linux kernel): "any >>>>> app that has a command line or option switch to call SO_BINDTODEVICE on >>>>> the >>>>> sockets can be configured to use any VRF". >>>>> >>>>> I guess the problem lies in that kamailio binds to several sockets and >>>>> there are multiple routing tables to consult, so mhomed doesn't work as >>>>> expected (if every table has a default route and nothing more specific >>>>> than >>>>> that). But still, this doesn't really explain why binding to the VRF >>>>> master >>>>> net devices doesn't work (I mean, "ping -I vrf-green" works just fine for >>>>> ping)... >>>>> >>>>> I don't think VRF support is very useful for most users though, I just >>>>> had one corner case I needed to tackle and VRF seemed like an easy fix. >>>>> I'll think of an alternative at some point, I guess. >>>>> >>>>> On 26 September 2017 at 09:52, Daniel-Constantin Mierla < >>>>> mico...@gmail.com> wrote: >>>>> > >>>>> > Hello, >>>>> > >>>>> > I haven't worked with vrf here, so no first hand experience ... >>>>> > >>>>> > What happens if you try with next option? >>>>> > >>>>> > listen=vrf-green:5060 advertise 2.2.2.2:5060 >>>>> > >>>>> > Cheers, >>>>> > Daniel >>>>> > >>>>> > >>>>> > On 25.09.17 19:10, George Diamantopoulos wrote: >>>>> > >>>>> > Sorry to bump this, but it would be very useful if I knew whether >>>>> there's any point in pursuing this or not. Any hints? >>>>> > >>>>> > On 21 September 2017 at 14:06, George Diamantopoulos < >>>>> georged...@gmail.com> wrote: >>>>> >> >>>>> >> Hello, >>>>> >> >>>>> >> I have a use case where I need to have kamailio bind to a VRF >>>>> device. The configuration in question is similar to the example below, >>>>> where eth1 is a slave to the VRF-lite device: >>>>> >> >>>>> >> +----------+ +-------------------+ >>>>> >> | eth0 | | vrf-green | >>>>> >> | 1.1.1.1 | | 127.0.0.1 | >>>>> >> +----------+ +-------------------+ >>>>> >> | >>>>> >> +----------+ >>>>> >> | eth1 | >>>>> >> | 2.2.2.2 | >>>>> >> +----------+ >>>>> >> >>>>> >> Both the main routing table and "vrf-green" routing table have a >>>>> default route. >>>>> >> >>>>> >> What I need to be able to do is have kamailio bind to both >>>>> interfaces: >>>>> >> >>>>> >> listen=eth0:5060 >>>>> >> listen=vrf-green:5060 >>>>> >> >>>>> >> And additionally be able to use force_send_socket to select an >>>>> interface, for example: >>>>> >> >>>>> >> force_send_socket(udp:2.2.2.2:5060); >>>>> >> >>>>> >> However, I can't get this to work. The above configuration fails >>>>> because there is no listen directive for 2.2.2.2. Also, kamailio doesn't >>>>> process packets received on the VRF with the above listen directives, it >>>>> behaves as if it doesn't listen on 2.2.2.2 indeed. >>>>> >> >>>>> >> In addition using either of the below: >>>>> >> >>>>> >> listen=udp:2.2.2.2:5060 >>>>> >> or >>>>> >> listen=eth1:5060 >>>>> >> >>>>> >> fails with an error upon starting kamailio. >>>>> >> >>>>> >> According to the kernel documentation: >>>>> >> >>>>> >> Applications that are to work within a VRF need to bind their >>>>> socket to the >>>>> >> VRF device: >>>>> >> >>>>> >> setsockopt(sd, SOL_SOCKET, SO_BINDTODEVICE, dev, strlen(dev)+1); >>>>> >> >>>>> >> or to specify the output device using cmsg and IP_PKTINFO. >>>>> >> >>>>> >> The question is, is VRF useable with kamailio right now? Or is >>>>> development needed? Thanks! >>>>> >> >>>>> >> BR, >>>>> >> >>>>> >> George >>>>> > >>>>> > >>>>> > >>>>> > >>>>> > _______________________________________________ >>>>> > Kamailio (SER) - Users Mailing List >>>>> > sr-users@lists.kamailio.org >>>>> > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>>>> > >>>>> > >>>>> > -- >>>>> > Daniel-Constantin Mierla >>>>> > www.twitter.com/miconda -- www.linkedin.com/in/miconda >>>>> > Kamailio Advanced Training - www.asipto.com >>>>> > Kamailio World Conference - www.kamailioworld.com >>>>> >>>>> _______________________________________________ >>>>> Kamailio (SER) - Users Mailing List >>>>> sr-users@lists.kamailio.org >>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>>>> >>>>> >>>>> >>>>> >>>>> _______________________________________________ >>>>> Kamailio (SER) - Users Mailing List >>>>> sr-users@lists.kamailio.org >>>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>>>> >>>>> >>>> _______________________________________________ >>>> Kamailio (SER) - Users Mailing List >>>> sr-users@lists.kamailio.org >>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>>> >>>> >>>> >>>> _______________________________________________ >>>> Kamailio (SER) - Users Mailing List >>>> sr-users@lists.kamailio.org >>>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>>> >>>> >>> _______________________________________________ >>> Kamailio (SER) - Users Mailing List >>> sr-users@lists.kamailio.org >>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>> >>> >>> _______________________________________________ >>> Kamailio (SER) - Users Mailing List >>> sr-users@lists.kamailio.org >>> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >>> >> _______________________________________________ >> Kamailio (SER) - Users Mailing List >> sr-users@lists.kamailio.org >> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >> >> >> >> _______________________________________________ >> Kamailio (SER) - Users Mailing List >> sr-users@lists.kamailio.org >> https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users >> >> > > _______________________________________________ > Kamailio (SER) - Users Mailing List > sr-users@lists.kamailio.org > https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users > >
_______________________________________________ Kamailio (SER) - Users Mailing List sr-users@lists.kamailio.org https://lists.kamailio.org/cgi-bin/mailman/listinfo/sr-users