[vpp-dev] Steering multicast packets into SRv6 policy
Hi - Is it possible to steer L3 multicast packets (either IPv4 or IPv6) into an SRv6 policy? I tried using "sr steer l3" command but it seems only to create a routing entry in the unicast FIB, which isn't invoked since multicast packets are handled by "ip4-mfib-forward-lookup". This is what I tried: sr policy add bsid c0::999:2 next C1::999:2 encap sr steer l3 225.5.0.0/16 via bsid c0::999:2 This creates an entry in ip fib: 225.5.0.0/16 unicast-ip4-chain [@0]: dpo-load-balance: [proto:ip4 index:40 buckets:1 uRPF:43 to:[0:0]] [0] [@12]: dpo-load-balance: [proto:ip4 index:38 buckets:1 uRPF:-1 to:[0:0] via:[139:11676]] [0] [@13]: SR: Segment List index:[0] Segments:< c1::999:2 > - Weight: 1 But nothing created in mfib. When I ping 225.5.5.5 from the host attached, vpp trace shows: --- Start of thread 0 vpp_main --- Packet 1 06:57:49:822991: af-packet-input af_packet: hw_if_index 1 next-index 4 tpacket2_hdr: status 0x2081 len 98 snaplen 98 mac 66 net 80 sec 0x5ef667a3 nsec 0x2651795f vlan 0 vlan_tpid 0 06:57:49:823009: ethernet-input IP4: 02:42:ac:18:00:02 -> 02:fe:b6:09:64:2b 06:57:49:823012: ip4-input ICMP: 172.24.0.2 -> 225.5.5.5 tos 0x00, ttl 64, length 84, checksum 0xa884 dscp CS0 ecn NON_ECN fragment id 0x, flags DONT_FRAGMENT ICMP echo_request checksum 0x7c4f 06:57:49:823015: ip4-mfib-forward-lookup fib 0 entry 0 06:57:49:823018: ip4-mfib-forward-rpf entry 0 itf -1 flags 06:57:49:823020: ip4-drop ICMP: 172.24.0.2 -> 225.5.5.5 tos 0x00, ttl 64, length 84, checksum 0xa884 dscp CS0 ecn NON_ECN fragment id 0x, flags DONT_FRAGMENT ICMP echo_request checksum 0x7c4f 06:57:49:823021: error-drop rx:host-eth1 06:57:49:823023: drop ip4-input: Multicast RPF check failed Would appreciate any help or information - thanks Fang -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16835): https://lists.fd.io/g/vpp-dev/message/16835 Mute This Topic: https://lists.fd.io/mt/75142919/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: Private: Re: [vpp-dev] VPP C API application compilation issue on using S and W functions
If you want to pinch code from vpp_api_test, please read and understand it:
/* W: wait for results, with timeout */
#define W(ret) \
do {\
f64 timeout = vat_time_now (vam) + 1.0; \
socket_client_main_t *scm = vam->socket_client_main; \
ret = -99; \
\
if (scm && scm->socket_enable) \
vl_socket_client_read (5); \
while (vat_time_now (vam) < timeout) { \
if (vam->result_ready == 1) { \
ret = vam->retval; \
break; \
} \
vat_suspend (vam->vlib_main, 1e-5); \
} \
} while(0);
So what does the “W” macro do? It causes the vpp_api_test main thread to block
until vpp answers and the vpp_api_test rx pthread processes the reply, or a
timeout occurs.
You’ll need to declare and initialize a clib_time_t, a vat_main_t, then supply
the missing routines:
f64
vat_time_now (vat_main_t * vam)
{
return clib_time_now (>clib_time);
}
void
vat_suspend (vlib_main_t * vm, f64 interval)
{
/* do nothing to busy-wait, or call usleep, or call CLIB_PAUSE */
}
Given that you’re not using vpp_api_test, you might decide to use
vat_helper_macros.h as a guide and fabricate something which seems more at home
in your application.
HTH... Dave
From: Chinmaya Aggarwal
Sent: Friday, June 26, 2020 7:59 AM
To: Dave Barach (dbarach)
Subject: Private: Re: [vpp-dev] VPP C API application compilation issue on
using S and W functions
Hi Dave,
We have our application (using VPP C API) running outside of VPP. We linked
vpp libs in our application (vatplugin, vppinfra, vlibmemoryclient, svm,
vppapiclient). Attaching file vpp_connect_test.c that creates connection with
VPP and calls VPP C API to modify a sr policy. We have used function S to send
the message and W to wait for its reply. On compiling this C file we are
getting the following error : -
root@ggnlabvm-hnsnfvsdn03:~/vpp_api_test# make
gcc -Wall -g-c -o vpp_connect_test.o vpp_connect_test.c
gcc -Wall -g -I/usr/include/vpp_plugins -I/usr/include/ -o api_exec
vpp_connect_test.o -Wl,--start-group -lvatplugin -lvppinfra -lvlibmemoryclient
-lsvm -lvppapiclient -Wl,--end-group -lpthread -lm -lrt -ldl
vpp_connect_test.o: In function `del_sl_pol2_index6':
/root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_time_now'
/root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_suspend'
/root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_time_now'
collect2: error: ld returned 1 exit status
Makefile:35: recipe for target 'api_exec' failed
make: *** [api_exec] Error 1
Is there any library that needs to be linked and we have missed it? If not, can
we use S and W function in an external C application, in a way similar to how
VPP uses it?
Thanks and Regards,
Chinmaya Agarwal.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16834): https://lists.fd.io/g/vpp-dev/message/16834
Mute This Topic: https://lists.fd.io/mt/75126256/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] RDMA problem in master and stable/2005, started with commit introducing direct verb for Cx4/5 tx #mellanox #rdma
Hi Ben, Thanks, I tested that now but it did not help, it behaves the same also with "MLX5_SHUT_UP_BF=1" set. / Elias > Can you try to export "MLX5_SHUT_UP_BF=1" in your environment before > starting VPP (ie, VPP environment must contain this)? This should > disable the "BlueFlame" mechanism in Mellanox NIC. Otherwise I'll > need to take a deeper look. > > Best > ben -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16833): https://lists.fd.io/g/vpp-dev/message/16833 Mute This Topic: https://lists.fd.io/mt/75120690/21656 Mute #mellanox: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/mellanox Mute #rdma: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/rdma Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] RDMA problem in master and stable/2005, started with commit introducing direct verb for Cx4/5 tx #mellanox #rdma
> Thanks, now I tried it (the Patchset 2 variant) but it seems to behave > like before, the delay is sitll happening. Hmm thanks ☹ Can you try to export "MLX5_SHUT_UP_BF=1" in your environment before starting VPP (ie, VPP environment must contain this)? This should disable the "BlueFlame" mechanism in Mellanox NIC. Otherwise I'll need to take a deeper look. Best ben -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16832): https://lists.fd.io/g/vpp-dev/message/16832 Mute This Topic: https://lists.fd.io/mt/75120690/21656 Mute #mellanox: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/mellanox Mute #rdma: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/rdma Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] RDMA problem in master and stable/2005, started with commit introducing direct verb for Cx4/5 tx #mellanox #rdma
Hi Ben, Thanks, now I tried it (the Patchset 2 variant) but it seems to behave like before, the delay is sitll happening. Let me know if you have something more I could try. / Elias On Fri, 2020-06-26 at 12:04 +, Benoit Ganne (bganne) via lists.fd.io wrote: > Hi Elias, > > Thanks for the detailed report. I suspect you are correct, it seems > to be related to the doorbell update to notify the NIC there are some > work to do. > Could you check https://gerrit.fd.io/r/c/vpp/+/27708 and report > whether it fixes the issue? > > Best > ben -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16831): https://lists.fd.io/g/vpp-dev/message/16831 Mute This Topic: https://lists.fd.io/mt/75120690/21656 Mute #mellanox: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/mellanox Mute #rdma: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/rdma Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP C API application compilation issue on using S and W functions
Chinmaya, I don't think linking in vatplugin is a "supported" interface. Meaning it can change or go away at no notice. If you build a C application you'd likely be happier with using VAPI. Cheers, Ole > On 26 Jun 2020, at 13:58, Chinmaya Aggarwal wrote: > > Hi Dave, > We have our application (using VPP C API) running outside of VPP. We linked > vpp libs in our application (vatplugin, vppinfra, vlibmemoryclient, svm, > vppapiclient). Attaching file vpp_connect_test.c that creates connection with > VPP and calls VPP C API to modify a sr policy. We have used function S to > send the message and W to wait for its reply. On compiling this C file we are > getting the following error : - > root@ggnlabvm-hnsnfvsdn03:~/vpp_api_test# make > gcc -Wall -g-c -o vpp_connect_test.o vpp_connect_test.c > gcc -Wall -g -I/usr/include/vpp_plugins -I/usr/include/ -o api_exec > vpp_connect_test.o -Wl,--start-group -lvatplugin -lvppinfra > -lvlibmemoryclient -lsvm -lvppapiclient -Wl,--end-group -lpthread -lm -lrt > -ldl > vpp_connect_test.o: In function `del_sl_pol2_index6': > /root/vpp_api_test/vpp_connect_test.c:97: undefined reference to > `vat_time_now' > /root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_suspend' > /root/vpp_api_test/vpp_connect_test.c:97: undefined reference to > `vat_time_now' > collect2: error: ld returned 1 exit status > Makefile:35: recipe for target 'api_exec' failed > make: *** [api_exec] Error 1 > > Is there any library that needs to be linked and we have missed it? If not, > can we use S and W function in an external C application, in a way similar to > how VPP uses it? > > Thanks and Regards, > Chinmaya Agarwal. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16830): https://lists.fd.io/g/vpp-dev/message/16830 Mute This Topic: https://lists.fd.io/mt/75100285/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Fetching SR Policy data using VPP C API
Chinmaya, > To reiterate what we have understood, if we downgrade the api and declare it > experimental (by setting "option version = 0.0.1"), we need not add "option > status="in_progress" for the typedefs and messages we are modifying and we > can simply use the original typedefs and messages and do the desired > modifications in them. Is that correct? > Or is it necessary to define V2 functions and mark the older ones as > deprecated even after downgrading the API version. > > One more thing, if we move ahead with this api change process in an > experimental version as of now, will it be available in next official VPP > release, for our production use? It will be available. You are responsible for the contract between yourself and your consumers. An experimental API means just that. And if anyone wanted to base product on that, I would imagine they would at some point push back on the experimental status. Cheers, Ole-=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16829): https://lists.fd.io/g/vpp-dev/message/16829 Mute This Topic: https://lists.fd.io/mt/74894689/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] RDMA problem in master and stable/2005, started with commit introducing direct verb for Cx4/5 tx #mellanox #rdma
Hi Elias, Thanks for the detailed report. I suspect you are correct, it seems to be related to the doorbell update to notify the NIC there are some work to do. Could you check https://gerrit.fd.io/r/c/vpp/+/27708 and report whether it fixes the issue? Best ben > -Original Message- > From: vpp-dev@lists.fd.io On Behalf Of Elias Rudberg > Sent: vendredi 26 juin 2020 11:13 > To: vpp-dev@lists.fd.io > Subject: [vpp-dev] RDMA problem in master and stable/2005, started with > commit introducing direct verb for Cx4/5 tx #mellanox #rdma > > Hello VPP experts, > > There seems to be a problem with the RDMA driver in VPP when using > Mellanox ConnectX5 network interfaces. This problem appears for the > master branch and for the stable/2005 branch, while stable/2001 does > not have this problem. > > The problem is that when a frame with 2 packets is to be sent, only the > first packets is sent directly while the second packet gets delayed. It > seems like the second packet is only sent later, when some other frame > with other packets is to be sent then the delayed earlier packet is > also sent. > > Perhaps this can go undetected if there is lots of traffic all the > time, if there is always new traffic to flush out any delayed packets > from earlier. So to reproduce it, it seems best to have a testing setup > with very little traffic such that there are several seconds without > any traffic, then it seems like packets can get delayed for several > seconds. Note that the delay is not seen inside VPP where packet traces > look like the packets are sent directly, VPP thinks they are sent but > it seems some packets are held in the NIC and only sent later on. > Monitoring traffic arriving at the other end shows that there was a > delay. > > The behavior seems reproducible, except when there is other traffic > being sent soon after since that causes the delayed packets to be sent. > > The specific case when this came up for us was when using VPP for NAT > with ipfix logging turned on, and doing some ping tests. Then when a > single ping echo request packet is to be NATed, that usually works fine > but sometimes there is also a ipfix logging packet to be sent, that > ends up in the same frame so that the frame has 2 packets. Then the > ipfix logging packet gets sent directly while the ICMP packet is > delayed, sometimes so much that the ping failed, it timed out. I don't > think the problem has anything to do with NAT or ipfix logging, it > seems like a more general problem with the rdma plugin. > > Testing previous commits indicates that the problem started with this > commit: > > dc812d9a7 (rdma: introduce direct verb for Cx4/5 tx, 2019-12-16) > > That commit exists in master and in stable/2005 but not in stable/2001 > which fits with that this problem is seen for master and stable/2005 > but not for stable/2001. > > Tried updating to the latest Mellanox driver (v5.0-2.1.8) but that did > not help. > > In the code in src/plugins/rdma/output.c it seems like the function > rdma_device_output_tx_mlx5() is handling the packets, but I was not > able to fully understand how it works. There is a concept of a > "doorbell" function call there, apparently the idea is that when > packets are to be sent, info about the packets is prepared and then the > "doorbell" is used to alert the NIC that there are things to send. From > my limited understanding, it seems like the doorbell currently results > in only the first packet is really being physically sent by the NIC > directly, while remaining packets are somehow stored and sent later. So > far I don't understand exactly why that happens or how to fix it. > > As a workaround, it seems to work to simply revert the entire rdma > plugin to the way it looks in the stable/2001 branch, then the problem > seems to disappear. But that probably means we lose performance gains > and other improvements in the newer code. > > Can someone with insight in the rdma plugin please help try to fix > this? > > Best regards, > Elias -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16828): https://lists.fd.io/g/vpp-dev/message/16828 Mute This Topic: https://lists.fd.io/mt/75120690/21656 Mute #mellanox: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/mellanox Mute #rdma: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/rdma Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Fetching SR Policy data using VPP C API
Hi, To reiterate what we have understood, if we downgrade the api and declare it experimental (by setting "option version = 0.0.1"), we need not add "option status="in_progress" for the typedefs and messages we are modifying and we can simply use the original typedefs and messages and do the desired modifications in them. Is that correct? Or is it necessary to define V2 functions and mark the older ones as deprecated even after downgrading the API version. One more thing, if we move ahead with this api change process in an experimental version as of now, will it be available in next official VPP release, for our production use? Thanks and Regards, Chinmaya Agarwal. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16827): https://lists.fd.io/g/vpp-dev/message/16827 Mute This Topic: https://lists.fd.io/mt/74894689/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] VPP C API application compilation issue on using S and W functions
Hi Dave, We have our application (using VPP C API) running outside of VPP. We linked vpp libs in our application ( vatplugin, vppinfra, vlibmemoryclient, svm, vppapiclient). Attaching file vpp_connect_test.c that creates connection with VPP and calls VPP C API to modify a sr policy. We have used function S to send the message and W to wait for its reply. On compiling this C file we are getting the following error : - root@ggnlabvm-hnsnfvsdn03:~/vpp_api_test# make gcc -Wall -g -c -o vpp_connect_test.o vpp_connect_test.c gcc -Wall -g -I/usr/include/vpp_plugins -I/usr/include/ -o api_exec vpp_connect_test.o -Wl,--start-group -lvatplugin -lvppinfra -lvlibmemoryclient -lsvm -lvppapiclient -Wl,--end-group -lpthread -lm -lrt -ldl vpp_connect_test.o: In function `del_sl_pol2_index6': /root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_time_now' /root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_suspend' /root/vpp_api_test/vpp_connect_test.c:97: undefined reference to `vat_time_now' collect2: error: ld returned 1 exit status Makefile:35: recipe for target 'api_exec' failed make: *** [api_exec] Error 1 Is there any library that needs to be linked and we have missed it? If not, can we use S and W function in an external C application, in a way similar to how VPP uses it? Thanks and Regards, Chinmaya Agarwal. -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16826): https://lists.fd.io/g/vpp-dev/message/16826 Mute This Topic: https://lists.fd.io/mt/75100285/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Fetching SR Policy data using VPP C API
Hi Chinmaya,
Yeah - the only thing typedef can not be marked as in-progress (Do you think it
is useful to add such a function?)
The other approach is if the community using those APIs thinks the SR-related
messages still need some quicker iteration, an option is to declare them as
“in-progress” (as in transition to the new peocess) - then the api change
process will not apply and you can make changes directly, and *then* remove the
in-progress status later on, when you are happy about them.
These are the two options - one is focused on keeping the existing setups
maximally compatible, the other on minimizing the overhead at the expense of
compatibility, where the compatibility is not needed.
If you know well who the users of the API are, you can decide what is the best
option.
--a
> On 26 Jun 2020, at 11:08, Chinmaya Aggarwal wrote:
>
>
> Hi Andrew,
> We have gone through the api change process link. As per our understanding
> from the link, we have to create a new in-progress message for srv6_sid_list
> i.e
>
> typedef srv6_sid_list_v2
> {
> u8 num_sids;
> u32 weight;
> u32 sl_index;
> vl_api_ip6_address_t sids[16];
> option status="in_progress";
> };
>
> Pertaining to this change we have identified further changes that are
> required : -
>
> 1. A new message for sr_policy_add has to be created i.e.
>
> autoreply define sr_policy_add_v2
> {
> u32 client_index;
> u32 context;
> vl_api_ip6_address_t bsid_addr;
> u32 weight;
> bool is_encap;
> bool is_spray;
> u32 fib_table;
> vl_api_srv6_sid_list_v2_t sids;
> option status="in_progress";
> };
>
>
> 2. A new message for sr_policy_mod has to be created i.e.
>
> autoreply define sr_policy_mod_v2
> {
> u32 client_index;
> u32 context;
> vl_api_ip6_address_t bsid_addr;
> u32 sr_policy_index;
> u32 fib_table;
> vl_api_sr_policy_op_t operation;
> u32 sl_index;
> u32 weight;
> vl_api_srv6_sid_list_v2_t sids;
> option status="in_progress";
> };
>
> 3. A new message for sr_policies_details has to be created i.e.
>
> define sr_policies_details_v2
> {
> u32 context;
> vl_api_ip6_address_t bsid;
> bool is_spray;
> bool is_encap;
> u32 fib_table;
> u8 num_sid_lists;
> vl_api_srv6_sid_list_v2_t sid_lists[num_sid_lists];
> option status="in_progress";
> };
>
> Following changes are required to de done in sr_api.c :-
>
> 4. Since we have created new message sr_policy_add_v2, we need to create
> functions similar to vl_api_sr_policy_add_t_handler() and sr_policy_add() and
> modify them as per the new message.
>
> 5. Since we have created new message sr_policy_mod_v2, we need to create
> functions similar to vl_api_sr_policy_mod_t_handler () and sr_policy_mod ()
> and modify them as per the new message.
>
> 6. Since we have created new message sr_policies_details_v2, we need to
> create functions similar to vl_api_sr_policies_dump_t_handler () and
> send_sr_policies_details () and modify them as per the new message.
>
> 7. After we are done with changes 4, 5 and 6; we need to modify
> foreach_vpe_api_msg macro to register the appropriate handler functions.
>
> Is this what we are suppose to do or is there some other way to cater the new
> in-progress message?
>
>
> Thanks and Regards,
> Chinmaya Agarwal.
> From: Andrew Yourtchenko
> Sent: Thursday, June 25, 2020 10:56 PM
> To: Chinmaya Aggarwal
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Fetching SR Policy data using VPP C API
>
> CAUTION:This email originated from an external organization
>
> Hi Chinmaya,
>
> This fix causes an incompatible change to three messages because they are all
> using the compound type being modified... it *also* uncovered an at least a
> year-long bug that would cause these changes slip under the radar - so thanks
> for helping catch it ! :-)
>
> You can rebase your patch for this change on top of any the versions in
> https://gerrit.fd.io/r/c/vpp/+/27677 and run “make checkstyle-api” to see
> what I am talking about
>
> I am working with Ole on how to address the issue, meantime if you can take a
> look at https://wiki.fd.io/view/VPP/ApiChangeProcess and consider which
> action is the most appropriate for this scenario.
>
> Thanks !
>
> --a
>
>>> On 25 Jun 2020, at 11:39, Chinmaya Aggarwal
>>> wrote:
>>>
>> Fix for this issue submitted for gerrit review (Change Id : 27668)
> DISCLAIMER: This electronic message and all of its contents, contains
> information which is privileged, confidential or otherwise protected from
> disclosure. The information contained in this electronic mail transmission is
> intended for use only by the individual or entity to which it is addressed.
> If you are not the intended recipient or may have received this electronic
> mail transmission in error, please notify the sender immediately and delete /
> destroy all copies of this electronic mail transmission without disclosing,
> copying, distributing, forwarding,
Re: [vpp-dev] ipsec interface revisted.
> On Jun 26, 2020, at 4:22 AM, Neale Ranns (nranns) wrote:
>
> Hi Chris,
>
> As far as I'm concerned, it's your plugin, you can add whatever functionality
> you need. If you separate the new interface type out into another plugin, so
> it can be used without your feature, then the community will benefit twice.
> Let's just make sure we document the whys and hows of each model.
>
> My preferred outcome though would be to try and find a way for your feature
> to work with the existing model. If you'd like to explore those possibilities
> perhaps we could discuss code specifics.
Hi Neale,
I can try some code specifics.
A major part of IPTFS is the constructing of the inner ESP payload, and
emitting this with specific timing disassociated with the carried user traffic
as described previously (and documented in
https://tools.ietf.org/html/draft-ietf-ipsecme-iptfs-01).
I'm going to leave out optimizations that may re-use buffers etc.. to keep
things simple..
This IPTFS machinery receives and terminates the user (protected) packet buffer
through the graph. IPTFS encapsulates this user packet into 1 or more IPTFS
payloads along with other user traffic to the same SA. It then paces the output
of these IPTFS payloads on a disconnected output worker. These IPTFS payloads
are passed on to the esp_encrypt for adding ESP and the outer tunnel encap.
So: e.g.,
No IPTFS, SA tunnel mode.
[IP-A] [IP-A]
[IPsec-IP|ESP|IP-A]
worker A: [IP-A] -> ipsec-output -> esp-encrypt [outer-encap added] ->
ip-lookup ... -> "if-tx"
[IP-B] [IP-B]
[IPsec-IP|ESP|IP-B]
worker A: [IP-A] -> ipsec-output -> esp-encrypt [outer-encap added] ->
ip-lookup ... -> "if-tx"
[IP-C] [IP-C]
[IPsec-IP|ESP|IP-C]
worker A: [IP-A] -> ipsec-output -> esp-encrypt [outer-encap added] ->
ip-lookup ... -> "if-tx"
IPTFS:
[IP-A] [IP-A] [TFS|IP-A]
worker A: ipsec-output -> iptfs-encap-enq -> encapsualte with other user
traffic construct in PACEQ
[IP-B] [IP-B] [TFS|IP-A|IP-B]
worker A: ipsec-output -> iptfs-encap-enq -> encapsualte with other user
traffic construct in PACEQ
[IP-C] [IP-C]
[TFS|IP-A|IP-B|IP-C]
worker A: ipsec-output -> iptfs-encap-enq -> encapsualte with other user
traffic construct in PACEQ
[TFS|IP-A|IP-B|IP-C]
worker A: PACEQ -> iptfs-pacer (timed) enqueue to OUTQ
[TFS|IP-A|IP-B|IP-C] [TFS|IP-A|IP-B|IP-C]
[IPsec-IP|ESP|TFS|IP-A|IP-B|IP-C]
worker B: OUTQ -> iptfs_output (timed) -> esp-encrypt [outer-encap added] ->
ip-lookup ... -> "if-tx"
Code Details (where does IPTFS hook):
SA Policy Directed
in ipsec_output_inline
if (p0->policy == IPSEC_POLICY_ACTION_PROTECT)
{ ...
if (sa->protocol == IPSEC_PROTOCOL_ESP)
{
if (ipsec_sa_is_IPTFS (sa))
next_node_index = im->tfs_encap_node_index;
else if (is_ipv6)
next_node_index = im->esp6_encrypt_node_index;
else
next_node_index = im->esp4_encrypt_node_index;
Interface Directed
[iptfs init code]
ipsec_add_feature ("ip4-output", "iptfs-encap4-tun",
>encap4_tun_feature_index);
[tunnel create code]
In ipsec_tunnel_feature_set
Instead of:
arc = vnet_get_feature_arc_index ("ip4-output");
vnet_feature_enable_disable_with_index (arc, esp4_feature_index,
t->sw_if_index, enable,
>output_sa_index,
sizeof (t->output_sa_index));
A callback is made to the TFS code which does:
arc = vnet_get_feature_arc_index ("ip4-output");
vnet_feature_enable_disable_with_index (
arc, tfsm->encap4_tun_feature_index, t->sw_if_index, enable,
>output_sa_index, sizeof (t->output_sa_index));
This latter Interface Directed code is what has been removed from VPP. The
packets I receive on this path were not not (and should not) be already
pre-encapsulated with any outer tunnel IP header, or I'm going to have to
immediately remove this encap before placing the user traffic in the the TFS
payload (see above). Having the overhead of adding an IP header, then
immediately removing it simply to get traffic directed to my IPTFS encap
routine is not a reasonable replacement.
So the "new model" in VPP is that I have to add a non-encapsulating interface
to direct the user traffic to me for TFS processing (i.e., unencapsulated).
This interface is going to bear a striking resemblance to the one that was just
removed from 19.08 VPP. :)
Thanks,
Re: [vpp-dev] Fetching SR Policy data using VPP C API
Chinmaya, You have a couple of options. You can downgrade the whole API and declare it experimental. That's done by setting the semantic version to less than 1 in the .api file. e.g. option version="0.0.1"; You can leave the API in production status, but mark those messages you believe will change in the near future as "option status="in_progress";". Or create a new set of in production messages as _2 and mark the _1 messages as deprecated (option deprecated="true";. [...] > 7. After we are done with changes 4, 5 and 6; we need to modify > foreach_vpe_api_msg macro to register the appropriate handler functions. Please use the updated auto-generated code instead of doing this manually. > Is this what we are suppose to do or is there some other way to cater the new > in-progress message? Only use in-progress, if that's really the status of the message. I.e. it's unstable. Best regards, Ole -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16823): https://lists.fd.io/g/vpp-dev/message/16823 Mute This Topic: https://lists.fd.io/mt/74894689/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
[vpp-dev] RDMA problem in master and stable/2005, started with commit introducing direct verb for Cx4/5 tx #mellanox #rdma
Hello VPP experts, There seems to be a problem with the RDMA driver in VPP when using Mellanox ConnectX5 network interfaces. This problem appears for the master branch and for the stable/2005 branch, while stable/2001 does not have this problem. The problem is that when a frame with 2 packets is to be sent, only the first packets is sent directly while the second packet gets delayed. It seems like the second packet is only sent later, when some other frame with other packets is to be sent then the delayed earlier packet is also sent. Perhaps this can go undetected if there is lots of traffic all the time, if there is always new traffic to flush out any delayed packets from earlier. So to reproduce it, it seems best to have a testing setup with very little traffic such that there are several seconds without any traffic, then it seems like packets can get delayed for several seconds. Note that the delay is not seen inside VPP where packet traces look like the packets are sent directly, VPP thinks they are sent but it seems some packets are held in the NIC and only sent later on. Monitoring traffic arriving at the other end shows that there was a delay. The behavior seems reproducible, except when there is other traffic being sent soon after since that causes the delayed packets to be sent. The specific case when this came up for us was when using VPP for NAT with ipfix logging turned on, and doing some ping tests. Then when a single ping echo request packet is to be NATed, that usually works fine but sometimes there is also a ipfix logging packet to be sent, that ends up in the same frame so that the frame has 2 packets. Then the ipfix logging packet gets sent directly while the ICMP packet is delayed, sometimes so much that the ping failed, it timed out. I don't think the problem has anything to do with NAT or ipfix logging, it seems like a more general problem with the rdma plugin. Testing previous commits indicates that the problem started with this commit: dc812d9a7 (rdma: introduce direct verb for Cx4/5 tx, 2019-12-16) That commit exists in master and in stable/2005 but not in stable/2001 which fits with that this problem is seen for master and stable/2005 but not for stable/2001. Tried updating to the latest Mellanox driver (v5.0-2.1.8) but that did not help. In the code in src/plugins/rdma/output.c it seems like the function rdma_device_output_tx_mlx5() is handling the packets, but I was not able to fully understand how it works. There is a concept of a "doorbell" function call there, apparently the idea is that when packets are to be sent, info about the packets is prepared and then the "doorbell" is used to alert the NIC that there are things to send. From my limited understanding, it seems like the doorbell currently results in only the first packet is really being physically sent by the NIC directly, while remaining packets are somehow stored and sent later. So far I don't understand exactly why that happens or how to fix it. As a workaround, it seems to work to simply revert the entire rdma plugin to the way it looks in the stable/2001 branch, then the problem seems to disappear. But that probably means we lose performance gains and other improvements in the newer code. Can someone with insight in the rdma plugin please help try to fix this? Best regards, Elias -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16822): https://lists.fd.io/g/vpp-dev/message/16822 Mute This Topic: https://lists.fd.io/mt/75120690/21656 Mute #mellanox: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/mellanox Mute #rdma: https://lists.fd.io/g/fdio+vpp-dev/mutehashtag/rdma Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] Fetching SR Policy data using VPP C API
Hi Andrew,
We have gone through the api change process link. As per our understanding from
the link, we have to create a new in-progress message for srv6_sid_list i.e
typedef srv6_sid_list_v2
{
u8 num_sids;
u32 weight;
u32 sl_index;
vl_api_ip6_address_t sids[16];
option status="in_progress";
};
Pertaining to this change we have identified further changes that are required
: -
1. A new message for sr_policy_add has to be created i.e.
autoreply define sr_policy_add_v2
{
u32 client_index;
u32 context;
vl_api_ip6_address_t bsid_addr;
u32 weight;
bool is_encap;
bool is_spray;
u32 fib_table;
vl_api_srv6_sid_list_v2_t sids;
option status="in_progress";
};
2. A new message for sr_policy_mod has to be created i.e.
autoreply define sr_policy_mod_v2
{
u32 client_index;
u32 context;
vl_api_ip6_address_t bsid_addr;
u32 sr_policy_index;
u32 fib_table;
vl_api_sr_policy_op_t operation;
u32 sl_index;
u32 weight;
vl_api_srv6_sid_list_v2_t sids;
option status="in_progress";
};
3. A new message for sr_policies_details has to be created i.e.
define sr_policies_details_v2
{
u32 context;
vl_api_ip6_address_t bsid;
bool is_spray;
bool is_encap;
u32 fib_table;
u8 num_sid_lists;
vl_api_srv6_sid_list_v2_t sid_lists[num_sid_lists];
option status="in_progress";
};
Following changes are required to de done in sr_api.c :-
4. Since we have created new message sr_policy_add_v2, we need to create
functions similar to vl_api_sr_policy_add_t_handler() and sr_policy_add() and
modify them as per the new message.
5. Since we have created new message sr_policy_mod_v2, we need to create
functions similar to vl_api_sr_policy_mod_t_handler () and sr_policy_mod () and
modify them as per the new message.
6. Since we have created new message sr_policies_details_v2, we need to create
functions similar to vl_api_sr_policies_dump_t_handler () and
send_sr_policies_details () and modify them as per the new message.
7. After we are done with changes 4, 5 and 6; we need to modify
foreach_vpe_api_msg macro to register the appropriate handler functions.
Is this what we are suppose to do or is there some other way to cater the new
in-progress message?
Thanks and Regards,
Chinmaya Agarwal.
From: Andrew Yourtchenko
Sent: Thursday, June 25, 2020 10:56 PM
To: Chinmaya Aggarwal
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Fetching SR Policy data using VPP C API
CAUTION:This email originated from an external organization
Hi Chinmaya,
This fix causes an incompatible change to three messages because they are all
using the compound type being modified... it *also* uncovered an at least a
year-long bug that would cause these changes slip under the radar - so thanks
for helping catch it ! :-)
You can rebase your patch for this change on top of any the versions in
https://gerrit.fd.io/r/c/vpp/+/27677
and run “make checkstyle-api” to see what I am talking about
I am working with Ole on how to address the issue, meantime if you can take a
look at
https://wiki.fd.io/view/VPP/ApiChangeProcess
and consider which action is the most appropriate for this scenario.
Thanks !
--a
On 25 Jun 2020, at 11:39, Chinmaya Aggarwal wrote:
Fix for this issue submitted for gerrit review (Change Id : 27668)
DISCLAIMER: This electronic message and all of its contents, contains
information which is privileged, confidential or otherwise protected from
disclosure. The information contained in this electronic mail transmission is
intended for use only by the individual or entity to which it is addressed. If
you are not the intended recipient or may have received this electronic mail
transmission in error, please notify the sender immediately and delete /
destroy all copies of this electronic mail transmission without disclosing,
copying, distributing, forwarding, printing or retaining any part of it. Hughes
Systique accepts no responsibility for loss or damage arising from the use of
the information transmitted by this email including damage from virus.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#16821): https://lists.fd.io/g/vpp-dev/message/16821
Mute This Topic: https://lists.fd.io/mt/74894689/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-
Re: [vpp-dev] ipsec interface revisted.
Hi Chris, As far as I'm concerned, it's your plugin, you can add whatever functionality you need. If you separate the new interface type out into another plugin, so it can be used without your feature, then the community will benefit twice. Let's just make sure we document the whys and hows of each model. My preferred outcome though would be to try and find a way for your feature to work with the existing model. If you'd like to explore those possibilities perhaps we could discuss code specifics. /neale tpyed by my fat tumhbs -=-=-=-=-=-=-=-=-=-=-=- Links: You receive all messages sent to this group. View/Reply Online (#16820): https://lists.fd.io/g/vpp-dev/message/16820 Mute This Topic: https://lists.fd.io/mt/74962223/21656 Group Owner: vpp-dev+ow...@lists.fd.io Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com] -=-=-=-=-=-=-=-=-=-=-=-
