Re: [vpp-dev] NAT44 UDP sessions are not clearing

[carlito nueno]

Testing with 30 ip addresses (users) opening around 300 sessions each.

When using vpp-20.01 + fixes by you and Filip (before the port overloading
patches), total sessions and total transitory sessions were much smaller
(around 15062).

on vpp-20.05 with port overloading

NAT44 pool addresses:
130.44.9.8
  tenant VRF independent
  0 busy other ports
  32 busy udp ports
  *63071* busy tcp ports
  1 busy icmp ports
NAT44 twice-nat pool addresses:
max translations: 400
max translations per user: 500
established tcp LRU min session timeout 7792 (now 352)
transitory tcp LRU min session timeout 294 (now 352)
udp LRU min session timeout 312 (now 352)
total timed out sessions: 119312
total sessions: *128639*
total tcp sessions: 128607
total tcp established sessions: 9300
total tcp transitory sessions: *119307*
total tcp transitory (WAIT-CLOSED) sessions: 0
total tcp transitory (CLOSED) sessions: 0
total udp sessions: 32
total icmp sessions: 0

On Tue, Jun 2, 2020 at 8:42 PM carlito nueno via lists.fd.io  wrote:

> Hi Klement,
>
> Got it.
>
> Sorry one more question :)
>
> I did another test and I noticed that tcp transitory sessions increase
> rapidly when I create new sessions from new internal ip address really fast
> (without delay). for example:
> tcp sessions are never stopped, so tcp transitory sessions should be 0 at
> all times.
>
> from ip address 192.168.1.2
>
> NAT44 pool addresses:
> 130.44.9.8
>   tenant VRF independent
>   0 busy other ports
>   36 busy udp ports
>   7694 busy tcp ports
>   0 busy icmp ports
> NAT44 twice-nat pool addresses:
> max translations: 400
> max translations per user: 500
> established tcp LRU min session timeout 7842 (now 402)
> udp LRU min session timeout 670 (now 402)
> total timed out sessions: 0
> total sessions: 1203
> total tcp sessions: 1200
> total tcp established sessions: 1200
> total tcp transitory sessions: 0
> total tcp transitory (WAIT-CLOSED) sessions: 0
> total tcp transitory (CLOSED) sessions: 0
> total udp sessions: 3
> total icmp sessions: 0
>
> *added 600 sessions *from ip address 192.168.1.3
>
> NAT44 pool addresses:
> 130.44.9.8
>   tenant VRF independent
>   0 busy other ports
>   36 busy udp ports
>   9395 busy tcp ports
>   0 busy icmp ports
> NAT44 twice-nat pool addresses:
> max translations: 400
> max translations per user: 500
> established tcp LRU min session timeout 7845 (now 405)
> transitory tcp LRU min session timeout 644 (now 405)
> udp LRU min session timeout 670 (now 405)
> total timed out sessions: 0
> total sessions: 2904
> total tcp sessions: 2901
> total tcp established sessions: 1800
> total tcp transitory sessions: *1101*
> total tcp transitory (WAIT-CLOSED) sessions: 0
> total tcp transitory (CLOSED) sessions: 0
> total udp sessions: 3
> total icmp sessions: 0
>
> Thanks!
>
> On Tue, Jun 2, 2020 at 11:47 AM Klement Sekera -X (ksekera - PANTHEON TECH
> SRO at Cisco)  wrote:
>
>> Hi Carlito,
>>
>> For ED NAT it doesn’t, as ED NAT no longer has any “user” concept. The
>> code for different flavours of NAT needs to be split and polished anyway.
>> Idea is to have data/code/APIs separate where appropriate.
>>
>> Thanks,
>> Klement
>>
>> > On 2 Jun 2020, at 20:31, Carlito Nueno  wrote:
>> >
>> > Hi Klement,
>> >
>> > Really appreciate the detailed explanation! That makes sense and I
>> could see that behavior from my tests.
>> >
>> > Last question: does "max translations per user" matter any more because
>> the concept of user doesn't exist with new NAT?
>> > max translations: 400
>> > max translations per user: 500
>> >
>> > From my tests, each ip address can form as many sessions as needed as
>> long as the overall/total sessions stay under "max translations".
>> >
>> > Thanks!
>> >
>> > On Mon, Jun 1, 2020 at 12:47 AM Klement Sekera -X (ksekera - PANTHEON
>> TECH SRO at Cisco)  wrote:
>> > Hi,
>> >
>> > as you can see almost all of NAT sessions are timed out. NAT will
>> automatically free and reuse them when needed again.
>> >
>> > this line:
>> > > udp LRU min session timeout 5175 (now 161589)
>> > hints whether immediate reuse is possible. Minimum session timeout in
>> the LRU list for UDP sessions is 5175, while current vpp internal time is
>> 161589. This means the first element in LRU list for UDP session is ready
>> to be reaped.
>> >
>> > To avoid fluctuations in performance due to running periodic cleanup
>> processes, NAT instead attempts to free one session anytime there is a
>> request to create a new session. This means that at low steady rate,
>> maximum number of sessions will peak at some point. E.g. with UDP timeout
>> of 30 seconds and 100 sessions/second, after 30 seconds there will be
>> around 3000 sessions and new sessions will also start to force cleanups.
>> This will then cause the total sessions to remain at around 3000. If you
>> stop creating new traffic, all of these eventually time out (without
>> spending any CPU on these timeouts). If again after some time you start
>> traffic, sessions

Re: [vpp-dev] SEGMENTATION FAULT in load_balance_get()

[Rajith PR via lists.fd.io]

Hi Dave/Neal,

The adj_poison seems to be a filling pattern - - 0xfefe. Am I looking into
the right code or I have interpreted it incorrectly?

Thanks,
Rajith

On Tue, Jun 2, 2020 at 7:44 PM Dave Barach (dbarach) 
wrote:

> The code manages to access a poisoned adjacency – 0x131313 fill pattern –
> copying Neale for an opinion.
>
>
>
> D.
>
>
>
> *From:* vpp-dev@lists.fd.io  *On Behalf Of *Rajith
> PR via lists.fd.io
> *Sent:* Tuesday, June 2, 2020 10:00 AM
> *To:* vpp-dev 
> *Subject:* [vpp-dev] SEGMENTATION FAULT in load_balance_get()
>
>
>
> Hello All,
>
>
>
> In *19.08 VPP version* we are seeing a crash while accessing the
> *load_balance_pool*  in *load_balanc_get*() function. This is happening
> after *enabling worker threads*.
>
> As such the FIB programming is happening in the main thread and in one of
> the worker threads we see this crash.
>
> Also, this is seen when we *scale to 300K+ ipv4 routes.*
>
>
>
> Here is the complete stack,
>
>
>
> Thread 10 "vpp_wk_0" received signal SIGSEGV, Segmentation fault.
>
> [Switching to Thread 0x7fbe4aa8e700 (LWP 333)]
> 0x7fbef10636f8 in clib_bitmap_get (ai=0x1313131313131313, i=61) at
> /home/ubuntu/Scale/libvpp/src/vppinfra/bitmap.h:201
> 201  return i0 < vec_len (ai) && 0 != ((ai[i0] >> i1) & 1);
>
>
>
> Thread 10 (Thread 0x7fbe4aa8e700 (LWP 333)):
> #0  0x7fbef10636f8 in clib_bitmap_get (ai=0x1313131313131313, i=61) at
> /home/ubuntu/Scale/libvpp/src/vppinfra/bitmap.h:201
> #1  0x7fbef10676a8 in load_balance_get (lbi=61) at
> /home/ubuntu/Scale/libvpp/src/vnet/dpo/load_balance.h:222
> #2  0x7fbef106890c in ip4_lookup_inline (vm=0x7fbe8a5aa080,
> node=0x7fbe8b3fd380, frame=0x7fbe8a5edb40) at
> /home/ubuntu/Scale/libvpp/src/vnet/ip/ip4_forward.h:369
> #3  0x7fbef1068ead in ip4_lookup_node_fn_avx2 (vm=0x7fbe8a5aa080,
> node=0x7fbe8b3fd380, frame=0x7fbe8a5edb40)
> at /home/ubuntu/Scale/libvpp/src/vnet/ip/ip4_forward.c:95
> #4  0x7fbef0c6afec in dispatch_node (vm=0x7fbe8a5aa080,
> node=0x7fbe8b3fd380, type=VLIB_NODE_TYPE_INTERNAL,
> dispatch_state=VLIB_NODE_STATE_POLLING,
> frame=0x7fbe8a5edb40, last_time_stamp=381215594286358) at
> /home/ubuntu/Scale/libvpp/src/vlib/main.c:1207
> #5  0x7fbef0c6b7ad in dispatch_pending_node (vm=0x7fbe8a5aa080,
> pending_frame_index=2, last_time_stamp=381215594286358)
> at /home/ubuntu/Scale/libvpp/src/vlib/main.c:1375
> #6  0x7fbef0c6d3f0 in vlib_main_or_worker_loop (vm=0x7fbe8a5aa080,
> is_main=0) at /home/ubuntu/Scale/libvpp/src/vlib/main.c:1826
> #7  0x7fbef0c6dc73 in vlib_worker_loop (vm=0x7fbe8a5aa080) at
> /home/ubuntu/Scale/libvpp/src/vlib/main.c:1934
> #8  0x7fbef0cac791 in vlib_worker_thread_fn (arg=0x7fbe8de2a340) at
> /home/ubuntu/Scale/libvpp/src/vlib/threads.c:1754
> #9  0x7fbef092da48 in clib_calljmp () from
> /home/ubuntu/Scale/libvpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.1.0.1
> #10 0x7fbe4aa8dec0 in ?? ()
> #11 0x7fbef0ca700c in vlib_worker_thread_bootstrap_fn
> (arg=0x7fbe8de2a340) at /home/ubuntu/Scale/libvpp/src/vlib/threads.c:573
>
> Thanks in Advance,
>
> Rajith
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16626): https://lists.fd.io/g/vpp-dev/message/16626
Mute This Topic: https://lists.fd.io/mt/74627827/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] NAT44 UDP sessions are not clearing

[carlito nueno]

Hi Klement,

Got it.

Sorry one more question :)

I did another test and I noticed that tcp transitory sessions increase
rapidly when I create new sessions from new internal ip address really fast
(without delay). for example:
tcp sessions are never stopped, so tcp transitory sessions should be 0 at
all times.

from ip address 192.168.1.2

NAT44 pool addresses:
130.44.9.8
  tenant VRF independent
  0 busy other ports
  36 busy udp ports
  7694 busy tcp ports
  0 busy icmp ports
NAT44 twice-nat pool addresses:
max translations: 400
max translations per user: 500
established tcp LRU min session timeout 7842 (now 402)
udp LRU min session timeout 670 (now 402)
total timed out sessions: 0
total sessions: 1203
total tcp sessions: 1200
total tcp established sessions: 1200
total tcp transitory sessions: 0
total tcp transitory (WAIT-CLOSED) sessions: 0
total tcp transitory (CLOSED) sessions: 0
total udp sessions: 3
total icmp sessions: 0

*added 600 sessions *from ip address 192.168.1.3

NAT44 pool addresses:
130.44.9.8
  tenant VRF independent
  0 busy other ports
  36 busy udp ports
  9395 busy tcp ports
  0 busy icmp ports
NAT44 twice-nat pool addresses:
max translations: 400
max translations per user: 500
established tcp LRU min session timeout 7845 (now 405)
transitory tcp LRU min session timeout 644 (now 405)
udp LRU min session timeout 670 (now 405)
total timed out sessions: 0
total sessions: 2904
total tcp sessions: 2901
total tcp established sessions: 1800
total tcp transitory sessions: *1101*
total tcp transitory (WAIT-CLOSED) sessions: 0
total tcp transitory (CLOSED) sessions: 0
total udp sessions: 3
total icmp sessions: 0

Thanks!

On Tue, Jun 2, 2020 at 11:47 AM Klement Sekera -X (ksekera - PANTHEON TECH
SRO at Cisco)  wrote:

> Hi Carlito,
>
> For ED NAT it doesn’t, as ED NAT no longer has any “user” concept. The
> code for different flavours of NAT needs to be split and polished anyway.
> Idea is to have data/code/APIs separate where appropriate.
>
> Thanks,
> Klement
>
> > On 2 Jun 2020, at 20:31, Carlito Nueno  wrote:
> >
> > Hi Klement,
> >
> > Really appreciate the detailed explanation! That makes sense and I could
> see that behavior from my tests.
> >
> > Last question: does "max translations per user" matter any more because
> the concept of user doesn't exist with new NAT?
> > max translations: 400
> > max translations per user: 500
> >
> > From my tests, each ip address can form as many sessions as needed as
> long as the overall/total sessions stay under "max translations".
> >
> > Thanks!
> >
> > On Mon, Jun 1, 2020 at 12:47 AM Klement Sekera -X (ksekera - PANTHEON
> TECH SRO at Cisco)  wrote:
> > Hi,
> >
> > as you can see almost all of NAT sessions are timed out. NAT will
> automatically free and reuse them when needed again.
> >
> > this line:
> > > udp LRU min session timeout 5175 (now 161589)
> > hints whether immediate reuse is possible. Minimum session timeout in
> the LRU list for UDP sessions is 5175, while current vpp internal time is
> 161589. This means the first element in LRU list for UDP session is ready
> to be reaped.
> >
> > To avoid fluctuations in performance due to running periodic cleanup
> processes, NAT instead attempts to free one session anytime there is a
> request to create a new session. This means that at low steady rate,
> maximum number of sessions will peak at some point. E.g. with UDP timeout
> of 30 seconds and 100 sessions/second, after 30 seconds there will be
> around 3000 sessions and new sessions will also start to force cleanups.
> This will then cause the total sessions to remain at around 3000. If you
> stop creating new traffic, all of these eventually time out (without
> spending any CPU on these timeouts). If again after some time you start
> traffic, sessions will be freed and reused as required.
> >
> > Regards,
> > Klement
> >
> > > On 31 May 2020, at 22:07, carlito nueno 
> wrote:
> > >
> > > Hi all,
> > >
> > > I am using vpp v20.05 and running NAT44 in end-point dependent mode.
> > >
> > > To test NAT, I created 50k tcp and udp sessions and ran packets for 5
> mins. Then I stopped the test.
> > >
> > > As soon as the test is stopped, tcp established sessions is 0, tcp
> transitory sessions increase and all of the tcp sessions become 0 after
> about 7440 seconds.
> > > But UDP sessions are still "open", as the count is still high even
> after 24 hours. As you can see below, udp LRU session timeout is around
> 161589 and total udp sessions is around 29k
> > >
> > > Any advice? Let me know if I am missing anything.
> > >
> > > NAT44 pool addresses:
> > > 130.44.9.8
> > >   tenant VRF independent
> > >   0 busy other ports
> > >   29058 busy udp ports
> > >   0 busy tcp ports
> > >   0 busy icmp ports
> > > NAT44 twice-nat pool addresses:
> > > max translations: 400
> > > max translations per user: 1000
> > > udp LRU min session timeout 5175 (now 161589)
> > > total timed out sessions: 29025
> > > total sessions: 29058
> > > t

[vpp-dev] VPP forwarding packets not destined to it #vpp

[Nagaraju Vemuri]

Hi,

We are using linux bridge to connect different interfaces owned by different 
VPP instances.
When the bridge has no binding info about MAC-to-port, bridge is flooding 
packets to all interfaces.
Hence VPP receives some packets whose MAC address is owned by some other VPP 
instance.
We want to drop such packets. By default VPP is forwarding these packets.

We tried using " set interface l2 forward  disable ", but this did 
not help.

Please suggest what we can do.

Thanks,
Nagaraju
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16624): https://lists.fd.io/g/vpp-dev/message/16624
Mute This Topic: https://lists.fd.io/mt/74640593/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp&subid=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] NAT44 UDP sessions are not clearing

[Klement Sekera via lists.fd.io]

Hi Carlito,

For ED NAT it doesn’t, as ED NAT no longer has any “user” concept. The code for 
different flavours of NAT needs to be split and polished anyway. Idea is to 
have data/code/APIs separate where appropriate.

Thanks,
Klement

> On 2 Jun 2020, at 20:31, Carlito Nueno  wrote:
> 
> Hi Klement,
> 
> Really appreciate the detailed explanation! That makes sense and I could see 
> that behavior from my tests.
> 
> Last question: does "max translations per user" matter any more because the 
> concept of user doesn't exist with new NAT?
> max translations: 400
> max translations per user: 500
> 
> From my tests, each ip address can form as many sessions as needed as long as 
> the overall/total sessions stay under "max translations".
> 
> Thanks!
> 
> On Mon, Jun 1, 2020 at 12:47 AM Klement Sekera -X (ksekera - PANTHEON TECH 
> SRO at Cisco)  wrote:
> Hi,
> 
> as you can see almost all of NAT sessions are timed out. NAT will 
> automatically free and reuse them when needed again.
> 
> this line:
> > udp LRU min session timeout 5175 (now 161589)
> hints whether immediate reuse is possible. Minimum session timeout in the LRU 
> list for UDP sessions is 5175, while current vpp internal time is 161589. 
> This means the first element in LRU list for UDP session is ready to be 
> reaped.
> 
> To avoid fluctuations in performance due to running periodic cleanup 
> processes, NAT instead attempts to free one session anytime there is a 
> request to create a new session. This means that at low steady rate, maximum 
> number of sessions will peak at some point. E.g. with UDP timeout of 30 
> seconds and 100 sessions/second, after 30 seconds there will be around 3000 
> sessions and new sessions will also start to force cleanups. This will then 
> cause the total sessions to remain at around 3000. If you stop creating new 
> traffic, all of these eventually time out (without spending any CPU on these 
> timeouts). If again after some time you start traffic, sessions will be freed 
> and reused as required.
> 
> Regards,
> Klement
> 
> > On 31 May 2020, at 22:07, carlito nueno  wrote:
> > 
> > Hi all,
> > 
> > I am using vpp v20.05 and running NAT44 in end-point dependent mode.
> > 
> > To test NAT, I created 50k tcp and udp sessions and ran packets for 5 mins. 
> > Then I stopped the test.
> > 
> > As soon as the test is stopped, tcp established sessions is 0, tcp 
> > transitory sessions increase and all of the tcp sessions become 0 after 
> > about 7440 seconds.
> > But UDP sessions are still "open", as the count is still high even after 24 
> > hours. As you can see below, udp LRU session timeout is around 161589 and 
> > total udp sessions is around 29k
> > 
> > Any advice? Let me know if I am missing anything.
> > 
> > NAT44 pool addresses:
> > 130.44.9.8
> >   tenant VRF independent
> >   0 busy other ports
> >   29058 busy udp ports
> >   0 busy tcp ports
> >   0 busy icmp ports
> > NAT44 twice-nat pool addresses:
> > max translations: 400
> > max translations per user: 1000
> > udp LRU min session timeout 5175 (now 161589)
> > total timed out sessions: 29025
> > total sessions: 29058
> > total tcp sessions: 0
> > total tcp established sessions: 0
> > total tcp transitory sessions: 0
> > total tcp transitory (WAIT-CLOSED) sessions: 0
> > total tcp transitory (CLOSED) sessions: 0
> > total udp sessions: 29058
> > total icmp sessions: 0
> > 
> > Thanks!
> > 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16623): https://lists.fd.io/g/vpp-dev/message/16623
Mute This Topic: https://lists.fd.io/mt/74589316/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] NAT44 UDP sessions are not clearing

[carlito nueno]

Hi Klement,

Really appreciate the detailed explanation! That makes sense and I could
see that behavior from my tests.

Last question: does "max translations per user" matter any more because the
concept of user doesn't exist with new NAT?
max translations: 400
max translations per user: 500

>From my tests, each ip address can form as many sessions as needed as long
as the overall/total sessions stay under "max translations".

Thanks!

On Mon, Jun 1, 2020 at 12:47 AM Klement Sekera -X (ksekera - PANTHEON TECH
SRO at Cisco)  wrote:

> Hi,
>
> as you can see almost all of NAT sessions are timed out. NAT will
> automatically free and reuse them when needed again.
>
> this line:
> > udp LRU min session timeout 5175 (now 161589)
> hints whether immediate reuse is possible. Minimum session timeout in the
> LRU list for UDP sessions is 5175, while current vpp internal time is
> 161589. This means the first element in LRU list for UDP session is ready
> to be reaped.
>
> To avoid fluctuations in performance due to running periodic cleanup
> processes, NAT instead attempts to free one session anytime there is a
> request to create a new session. This means that at low steady rate,
> maximum number of sessions will peak at some point. E.g. with UDP timeout
> of 30 seconds and 100 sessions/second, after 30 seconds there will be
> around 3000 sessions and new sessions will also start to force cleanups.
> This will then cause the total sessions to remain at around 3000. If you
> stop creating new traffic, all of these eventually time out (without
> spending any CPU on these timeouts). If again after some time you start
> traffic, sessions will be freed and reused as required.
>
> Regards,
> Klement
>
> > On 31 May 2020, at 22:07, carlito nueno  wrote:
> >
> > Hi all,
> >
> > I am using vpp v20.05 and running NAT44 in end-point dependent mode.
> >
> > To test NAT, I created 50k tcp and udp sessions and ran packets for 5
> mins. Then I stopped the test.
> >
> > As soon as the test is stopped, tcp established sessions is 0, tcp
> transitory sessions increase and all of the tcp sessions become 0 after
> about 7440 seconds.
> > But UDP sessions are still "open", as the count is still high even after
> 24 hours. As you can see below, udp LRU session timeout is around 161589
> and total udp sessions is around 29k
> >
> > Any advice? Let me know if I am missing anything.
> >
> > NAT44 pool addresses:
> > 130.44.9.8
> >   tenant VRF independent
> >   0 busy other ports
> >   29058 busy udp ports
> >   0 busy tcp ports
> >   0 busy icmp ports
> > NAT44 twice-nat pool addresses:
> > max translations: 400
> > max translations per user: 1000
> > udp LRU min session timeout 5175 (now 161589)
> > total timed out sessions: 29025
> > total sessions: 29058
> > total tcp sessions: 0
> > total tcp established sessions: 0
> > total tcp transitory sessions: 0
> > total tcp transitory (WAIT-CLOSED) sessions: 0
> > total tcp transitory (CLOSED) sessions: 0
> > total udp sessions: 29058
> > total icmp sessions: 0
> >
> > Thanks!
> > 
>
>
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16622): https://lists.fd.io/g/vpp-dev/message/16622
Mute This Topic: https://lists.fd.io/mt/74589316/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP fails to start - error message EAL: FATAL: Cannot get hugepage information.

[Manoj Iyer]

Ben,

I built the 20.5 VPP and latest stable DPDK and I dont have the issue I 
reported.

$ sudo systemctl status vpp.service
? vpp.service - vector packet processing engine
   Loaded: loaded (/lib/systemd/system/vpp.service; enabled; vendor preset: enab
led)
   Active: active (running) since Tue 2020-06-02 18:21:36 UTC; 5s ago

Thanks a ton for all the help. Really really appreciate it.

Cheers
Manoj Iyer

From: Benoit Ganne (bganne) 
Sent: Tuesday, June 2, 2020 4:21 AM
To: Manoj Iyer ; Damjan Marion 
Cc: vpp-dev@lists.fd.io ; Rodney Schmidt 
; Kshitij Sudan 
Subject: RE: [vpp-dev] VPP fails to start - error message EAL: FATAL: Cannot 
get hugepage information.

Manoj, can you try VPP-20.05 instead of VPP-19.04? I suspect the DPDK-19.02 
used with VPP-19.04 is too old for your platform and skip 2MB hugepages. DPDK 
19.02 seems to support up to 3 different hugepages size and you appear to have 
4.

ben

> -Original Message-
> From: Manoj Iyer 
> Sent: mardi 2 juin 2020 05:57
> To: Damjan Marion ; Benoit Ganne (bganne)
> 
> Cc: vpp-dev@lists.fd.io; Rodney Schmidt ; Kshitij
> Sudan 
> Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> Cannot get hugepage information.
>
> Ben,
>
> Sorry for the delay in posting.
>
> Here is the strace output: https://pastebin.ubuntu.com/p/NgrtzYDvK7/
>
> Thanks
> Manoj Iyer
> 
>
> From: vpp-dev@lists.fd.io  on behalf of Benoit Ganne
> (bganne) via lists.fd.io 
> Sent: Thursday, May 28, 2020 2:21 AM
> To: Manoj Iyer ; Damjan Marion 
> Cc: vpp-dev@lists.fd.io ; Rodney Schmidt
> ; Kshitij Sudan 
> Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> Cannot get hugepage information.
>
> Could you share the output of:
> ~# strace vpp -c /usr/share/vpp/vpp.conf
> With DPDK enabled (ie with your targeted config which is failing)?
>
> ben
>
> > -Original Message-
> > From: Manoj Iyer 
> > Sent: jeudi 28 mai 2020 01:11
> > To: Damjan Marion ; Manoj Iyer 
> > Cc: Benoit Ganne (bganne) ; vpp-dev@lists.fd.io;
> Rodney
> > Schmidt ; Kshitij Sudan 
> > Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> > Cannot get hugepage information.
> >
> > Any more thoughts on this failure ?
> >
> > Thanks
> > Manoj Iyer
> > 
> >
> > From: vpp-dev@lists.fd.io  on behalf of Manoj Iyer
> > via lists.fd.io 
> > Sent: Tuesday, May 26, 2020 6:51 PM
> > To: Damjan Marion 
> > Cc: bga...@cisco.com ; vpp-dev@lists.fd.io  > d...@lists.fd.io>; Rodney Schmidt ; Kshitij Sudan
> > 
> > Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> > Cannot get hugepage information.
> >
> > $ lscpu
> >
> > Architecture: aarch64
> >
> > Byte Order:   Little Endian
> >
> > CPU(s):   8
> >
> > On-line CPU(s) list:  0
> >
> > Off-line CPU(s) list: 1-7
> >
> > Thread(s) per core:   1
> >
> > Core(s) per socket:   1
> >
> > Socket(s):1
> >
> > NUMA node(s): 1
> >
> > Vendor ID:ARM
> >
> > Model:3
> >
> > Model name:   Cortex-A72
> >
> > Stepping: r0p3
> >
> > BogoMIPS: 250.00
> >
> > L1d cache:unknown size
> >
> > L1i cache:unknown size
> >
> > L2 cache: unknown size
> >
> > NUMA node0 CPU(s):0
> >
> > Flags:fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
> >
> >
> > $ grep .  /sys/kernel/mm/hugepages/hugepages-*/*
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages_mempolicy:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/surplus_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages:1024
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages:1024
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages_mempolicy:1024
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/surplus_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/free_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_hugepages_mempolicy:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/surplus_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/free_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/nr_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/nr_hugepages_mempolicy:0

Re: [vpp-dev] How to trigger the perf test?

[Pei, Yulong]

csit-2n-clx-perftest 
mrrAND1cAND64bANDnic_intel-xxv710ANDeth-l2bdbasemaclrnNOTdrv_avf
From: vpp-dev@lists.fd.io  On Behalf Of Dave Wallace
Sent: Tuesday, June 2, 2020 10:07 PM
To: Yang, Zhiyong ; vpp-dev@lists.fd.io; 
csit-...@lists.fd.io
Subject: Re: [vpp-dev] How to trigger the perf test?

+ csit-...@lists.fd.io

On 6/2/2020 6:10 AM, Zhiyong Yang wrote:
Hi vpp/csit experts,

I have submitted one patch that tries to optimize l2 input node.  
https://gerrit.fd.io/r/c/vpp/+/27191
Want to see if it can help enhance the perf or not. But don't how to trigger 
the L2 BD perf test case on CSIT.
Who can help me about it ? Thanks a lot in advancement.

BR
Zhiyong





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16620): https://lists.fd.io/g/vpp-dev/message/16620
Mute This Topic: https://lists.fd.io/mt/74624445/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] SEGMENTATION FAULT in load_balance_get()

[Dave Barach via lists.fd.io]

The code manages to access a poisoned adjacency – 0x131313 fill pattern – 
copying Neale for an opinion.

D.

From: vpp-dev@lists.fd.io  On Behalf Of Rajith PR via 
lists.fd.io
Sent: Tuesday, June 2, 2020 10:00 AM
To: vpp-dev 
Subject: [vpp-dev] SEGMENTATION FAULT in load_balance_get()

Hello All,

In 19.08 VPP version we are seeing a crash while accessing the 
load_balance_pool  in load_balanc_get() function. This is happening after 
enabling worker threads.
As such the FIB programming is happening in the main thread and in one of the 
worker threads we see this crash.
Also, this is seen when we scale to 300K+ ipv4 routes.

Here is the complete stack,

Thread 10 "vpp_wk_0" received signal SIGSEGV, Segmentation fault.
[Switching to Thread 0x7fbe4aa8e700 (LWP 333)]
0x7fbef10636f8 in clib_bitmap_get (ai=0x1313131313131313, i=61) at 
/home/ubuntu/Scale/libvpp/src/vppinfra/bitmap.h:201
201  return i0 < vec_len (ai) && 0 != ((ai[i0] >> i1) & 1);

Thread 10 (Thread 0x7fbe4aa8e700 (LWP 333)):
#0  0x7fbef10636f8 in clib_bitmap_get (ai=0x1313131313131313, i=61) at 
/home/ubuntu/Scale/libvpp/src/vppinfra/bitmap.h:201
#1  0x7fbef10676a8 in load_balance_get (lbi=61) at 
/home/ubuntu/Scale/libvpp/src/vnet/dpo/load_balance.h:222
#2  0x7fbef106890c in ip4_lookup_inline (vm=0x7fbe8a5aa080, 
node=0x7fbe8b3fd380, frame=0x7fbe8a5edb40) at 
/home/ubuntu/Scale/libvpp/src/vnet/ip/ip4_forward.h:369
#3  0x7fbef1068ead in ip4_lookup_node_fn_avx2 (vm=0x7fbe8a5aa080, 
node=0x7fbe8b3fd380, frame=0x7fbe8a5edb40)
at /home/ubuntu/Scale/libvpp/src/vnet/ip/ip4_forward.c:95
#4  0x7fbef0c6afec in dispatch_node (vm=0x7fbe8a5aa080, 
node=0x7fbe8b3fd380, type=VLIB_NODE_TYPE_INTERNAL, 
dispatch_state=VLIB_NODE_STATE_POLLING,
frame=0x7fbe8a5edb40, last_time_stamp=381215594286358) at 
/home/ubuntu/Scale/libvpp/src/vlib/main.c:1207
#5  0x7fbef0c6b7ad in dispatch_pending_node (vm=0x7fbe8a5aa080, 
pending_frame_index=2, last_time_stamp=381215594286358)
at /home/ubuntu/Scale/libvpp/src/vlib/main.c:1375
#6  0x7fbef0c6d3f0 in vlib_main_or_worker_loop (vm=0x7fbe8a5aa080, 
is_main=0) at /home/ubuntu/Scale/libvpp/src/vlib/main.c:1826
#7  0x7fbef0c6dc73 in vlib_worker_loop (vm=0x7fbe8a5aa080) at 
/home/ubuntu/Scale/libvpp/src/vlib/main.c:1934
#8  0x7fbef0cac791 in vlib_worker_thread_fn (arg=0x7fbe8de2a340) at 
/home/ubuntu/Scale/libvpp/src/vlib/threads.c:1754
#9  0x7fbef092da48 in clib_calljmp () from 
/home/ubuntu/Scale/libvpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.1.0.1
#10 0x7fbe4aa8dec0 in ?? ()
#11 0x7fbef0ca700c in vlib_worker_thread_bootstrap_fn (arg=0x7fbe8de2a340) 
at /home/ubuntu/Scale/libvpp/src/vlib/threads.c:573
Thanks in Advance,
Rajith
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16619): https://lists.fd.io/g/vpp-dev/message/16619
Mute This Topic: https://lists.fd.io/mt/74627827/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] How to trigger the perf test?

[Dave Wallace]

+ csit-...@lists.fd.io

On 6/2/2020 6:10 AM, Zhiyong Yang wrote:


Hi vpp/csit experts,

I have submitted one patch that tries to optimize l2 input node. 
https://gerrit.fd.io/r/c/vpp/+/27191


Want to see if it can help enhance the perf or not. But don’t how to 
trigger the L2 BD perf test case on CSIT.


Who can help me about it ? Thanks a lot in advancement.

BR

Zhiyong





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16618): https://lists.fd.io/g/vpp-dev/message/16618
Mute This Topic: https://lists.fd.io/mt/74624445/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] SEGMENTATION FAULT in load_balance_get()

[Rajith PR via lists.fd.io]

Hello All,

In *19.08** VPP version* we are seeing a crash while accessing the
*load_balance_pool*  in *load_balanc_get*() function. This is happening
after *enabling worker threads*.
As such the FIB programming is happening in the main thread and in one of
the worker threads we see this crash.
Also, this is seen when we *scale to 300K+ ipv4 routes.*

Here is the complete stack,

Thread 10 "vpp_wk_0" received signal SIGSEGV, Segmentation fault.

[Switching to Thread 0x7fbe4aa8e700 (LWP 333)]
0x7fbef10636f8 in clib_bitmap_get (ai=0x1313131313131313, i=61) at
/home/ubuntu/Scale/libvpp/src/vppinfra/bitmap.h:201
201  return i0 < vec_len (ai) && 0 != ((ai[i0] >> i1) & 1);

Thread 10 (Thread 0x7fbe4aa8e700 (LWP 333)):
#0  0x7fbef10636f8 in clib_bitmap_get (ai=0x1313131313131313, i=61) at
/home/ubuntu/Scale/libvpp/src/vppinfra/bitmap.h:201
#1  0x7fbef10676a8 in load_balance_get (lbi=61) at
/home/ubuntu/Scale/libvpp/src/vnet/dpo/load_balance.h:222
#2  0x7fbef106890c in ip4_lookup_inline (vm=0x7fbe8a5aa080,
node=0x7fbe8b3fd380, frame=0x7fbe8a5edb40) at
/home/ubuntu/Scale/libvpp/src/vnet/ip/ip4_forward.h:369
#3  0x7fbef1068ead in ip4_lookup_node_fn_avx2 (vm=0x7fbe8a5aa080,
node=0x7fbe8b3fd380, frame=0x7fbe8a5edb40)
at /home/ubuntu/Scale/libvpp/src/vnet/ip/ip4_forward.c:95
#4  0x7fbef0c6afec in dispatch_node (vm=0x7fbe8a5aa080,
node=0x7fbe8b3fd380, type=VLIB_NODE_TYPE_INTERNAL,
dispatch_state=VLIB_NODE_STATE_POLLING,
frame=0x7fbe8a5edb40, last_time_stamp=381215594286358) at
/home/ubuntu/Scale/libvpp/src/vlib/main.c:1207
#5  0x7fbef0c6b7ad in dispatch_pending_node (vm=0x7fbe8a5aa080,
pending_frame_index=2, last_time_stamp=381215594286358)
at /home/ubuntu/Scale/libvpp/src/vlib/main.c:1375
#6  0x7fbef0c6d3f0 in vlib_main_or_worker_loop (vm=0x7fbe8a5aa080,
is_main=0) at /home/ubuntu/Scale/libvpp/src/vlib/main.c:1826
#7  0x7fbef0c6dc73 in vlib_worker_loop (vm=0x7fbe8a5aa080) at
/home/ubuntu/Scale/libvpp/src/vlib/main.c:1934
#8  0x7fbef0cac791 in vlib_worker_thread_fn (arg=0x7fbe8de2a340) at
/home/ubuntu/Scale/libvpp/src/vlib/threads.c:1754
#9  0x7fbef092da48 in clib_calljmp () from
/home/ubuntu/Scale/libvpp/build-root/install-vpp_debug-native/vpp/lib/libvppinfra.so.1.0.1
#10 0x7fbe4aa8dec0 in ?? ()
#11 0x7fbef0ca700c in vlib_worker_thread_bootstrap_fn
(arg=0x7fbe8de2a340) at /home/ubuntu/Scale/libvpp/src/vlib/threads.c:573

Thanks in Advance,
Rajith
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16617): https://lists.fd.io/g/vpp-dev/message/16617
Mute This Topic: https://lists.fd.io/mt/74627827/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ixge and rdma drivers

[Christian Hopps]

Ok. Thanks for the info.

FWIW, apparently x520-2 is still somewhat common b/c that's what I've got from 
new dell servers from mid-last year. Will try and get the newer cards in the 
future.

Thanks,
Chris.

> On Jun 2, 2020, at 7:59 AM, Damjan Marion via lists.fd.io 
>  wrote:
> 
> 
> Hi Christian,
> 
> ixgbe driver is deprecated, that code is very old, from days when VPP was not 
> open source.
> That driver was used for Intel Niantic family (82599, x5x0) of NICs which are 
> those days replaced by  Intel Fortville (x710, xl710, xvv710, x722).
> For fortville and soon for columbiaville NICs (e810), you can use native AVF 
> driver….
> 
> You need to create virtual function first, as AVF driver works only with VFs. 
> There is script which does the job in extras/scripts/vfctl
> 
> 
> i.e.
> 
> $ extras/scripts/vfctl create :61:00.1 1
> 
> Virtual Functions:
> ID PCI Addr PCI IDDriver   MAC Addr  Config
> 0  :61:06.0 8086:37cd vfio-pci  spoof checking off, link-state auto, 
> trust on
> 
> And the in in VPP:
> 
> vpp# create interface avf :61:06.0 name eth0 
> 
> 
> 
>> On 2 Jun 2020, at 09:40, Christian Hopps  wrote:
>> 
>> Hi vpp-dev,
>> 
>> I've been contemplating trying to use native drivers in place of DPDK with 
>> the understanding that I may be paying a ~20% penalty by using DPDK. So I 
>> went to try things out, but had some trouble. The systems in paticular I'm 
>> interested in have 10GE intel NICs in them which I believe would be 
>> supported by the ixge driver. I noticed that this driver has been marked 
>> deprecated in VPP though. Is there a replacement or is DPDK required for 
>> this NIC?
>> 
>> I also have systems that have mlx5 (and eventually will have connectx-6 
>> cards). These cards appear to be supported by the rdma native driver. I was 
>> able to create the interfaces and saw TX packets but no RX.  Is this driver 
>> considered stable and usable in 19.08 (and if not which release would it be 
>> consider so)?
>> 
>> Thanks,
>> Chris.
> 
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16616): https://lists.fd.io/g/vpp-dev/message/16616
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Crash in vlib_add_trace with multi worker mode

[Dave Barach via lists.fd.io]

Unless you fully communicate your configuration, you’ll have to debug the issue 
yourself. Are you using the standard handoff mechanism, or a mechanism of your 
own design?

The handoff demo plugin seems to work fine... See 
../src/examples/handoffdemo/{README.md, node.c} etc.

DBGvpp# sh trace

--- Start of thread 0 vpp_main ---
No packets in trace buffer
--- Start of thread 1 vpp_wk_0 ---
Packet 1

00:00:19:259770: pg-input
  stream x, 128 bytes, sw_if_index 0
  current data 0, length 128, buffer-pool 0, ref-count 1, trace handle 0x100
  : 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d
  0020: 
  0040: 
  0060: 
00:00:19:259851: handoffdemo-1
  HANDOFFDEMO: current thread 1

Packet 2

00:00:19:259770: pg-input
  stream x, 128 bytes, sw_if_index 0
  current data 0, length 128, buffer-pool 0, ref-count 1, trace handle 0x101
  : 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d
  0020: 
  0040: 
  0060: 
00:00:19:259851: handoffdemo-1
  HANDOFFDEMO: current thread 1

Packet 3

00:00:19:259770: pg-input
  stream x, 128 bytes, sw_if_index 0
  current data 0, length 128, buffer-pool 0, ref-count 1, trace handle 0x102
  : 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d
  0020: 
  0040: 
  0060: 
00:00:19:259851: handoffdemo-1
  HANDOFFDEMO: current thread 1

Packet 4

00:00:19:259770: pg-input
  stream x, 128 bytes, sw_if_index 0
  current data 0, length 128, buffer-pool 0, ref-count 1, trace handle 0x103
  : 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d
  0020: 
  0040: 
  0060: 
00:00:19:259851: handoffdemo-1
  HANDOFFDEMO: current thread 1

Packet 5

00:00:19:259770: pg-input
  stream x, 128 bytes, sw_if_index 0
  current data 0, length 128, buffer-pool 0, ref-count 1, trace handle 0x104
  : 000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d
  0020: 
  0040: 
  0060: 
00:00:19:259851: handoffdemo-1
  HANDOFFDEMO: current thread 1

--- Start of thread 2 vpp_wk_1 ---
Packet 1

00:00:19:259879: handoff_trace
  HANDED-OFF: from thread 1 trace index 0
00:00:19:259879: handoffdemo-2
  HANDOFFDEMO: current thread 2
00:00:19:259930: error-drop
  rx:local0
00:00:19:259967: drop
  handoffdemo-2: completed packets

Packet 2

00:00:19:259879: handoff_trace
  HANDED-OFF: from thread 1 trace index 1
00:00:19:259879: handoffdemo-2
  HANDOFFDEMO: current thread 2
00:00:19:259930: error-drop
  rx:local0
00:00:19:259967: drop
  handoffdemo-2: completed packets

Packet 3

00:00:19:259879: handoff_trace
  HANDED-OFF: from thread 1 trace index 2
00:00:19:259879: handoffdemo-2
  HANDOFFDEMO: current thread 2
00:00:19:259930: error-drop
  rx:local0
00:00:19:259967: drop
  handoffdemo-2: completed packets

Packet 4

00:00:19:259879: handoff_trace
  HANDED-OFF: from thread 1 trace index 3
00:00:19:259879: handoffdemo-2
  HANDOFFDEMO: current thread 2
00:00:19:259930: error-drop
  rx:local0
00:00:19:259967: drop
  handoffdemo-2: completed packets

Packet 5

00:00:19:259879: handoff_trace
  HANDED-OFF: from thread 1 trace index 4
00:00:19:259879: handoffdemo-2
  HANDOFFDEMO: current thread 2
00:00:19:259930: error-drop
  rx:local0
00:00:19:259967: drop
  handoffdemo-2: completed packets

DBGvpp#

From: vpp-dev@lists.fd.io  On Behalf Of Satya Murthy
Sent: Tuesday, June 2, 2020 7:11 AM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Crash in vlib_add_trace with multi worker mode

Hi ,

We are seeing a crash while doing add_trace for a vlib_buffer in our graph node.

#0 0x74ee0feb in raise () from /lib64/libc.so.6
#1 0x74ecb5c1 in abort () from /lib64/libc.so.6
#2 0x0040831c in os_panic () at 
/fdio/src/fdio.1810/src/vpp/vnet/main.c:368
#3 0x75f28f2f in debugger () at 
/fdio/src/fdio.1810/src/vppinfra/error.c:84
#4 0x75f2936a in _clib_error (how_to_die=2, function_name

Re: [vpp-dev] ixge and rdma drivers

[Damjan Marion via lists.fd.io]

Hi Christian,

ixgbe driver is deprecated, that code is very old, from days when VPP was not 
open source.
That driver was used for Intel Niantic family (82599, x5x0) of NICs which are 
those days replaced by  Intel Fortville (x710, xl710, xvv710, x722).
For fortville and soon for columbiaville NICs (e810), you can use native AVF 
driver….

You need to create virtual function first, as AVF driver works only with VFs. 
There is script which does the job in extras/scripts/vfctl


i.e.

$ extras/scripts/vfctl create :61:00.1 1

Virtual Functions:
ID PCI Addr PCI IDDriver   MAC Addr  Config
0  :61:06.0 8086:37cd vfio-pci  spoof checking off, link-state auto, 
trust on

And the in in VPP:

vpp# create interface avf :61:06.0 name eth0 



> On 2 Jun 2020, at 09:40, Christian Hopps  wrote:
> 
> Hi vpp-dev,
> 
> I've been contemplating trying to use native drivers in place of DPDK with 
> the understanding that I may be paying a ~20% penalty by using DPDK. So I 
> went to try things out, but had some trouble. The systems in paticular I'm 
> interested in have 10GE intel NICs in them which I believe would be supported 
> by the ixge driver. I noticed that this driver has been marked deprecated in 
> VPP though. Is there a replacement or is DPDK required for this NIC?
> 
> I also have systems that have mlx5 (and eventually will have connectx-6 
> cards). These cards appear to be supported by the rdma native driver. I was 
> able to create the interfaces and saw TX packets but no RX.  Is this driver 
> considered stable and usable in 19.08 (and if not which release would it be 
> consider so)?
> 
> Thanks,
> Chris.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16614): https://lists.fd.io/g/vpp-dev/message/16614
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Crash in vlib_add_trace with multi worker mode

[Satya Murthy]

Hi ,

We are seeing a crash while doing add_trace for a vlib_buffer in our graph node.

#0 0x74ee0feb in raise () from /lib64/libc.so.6
#1 0x74ecb5c1 in abort () from /lib64/libc.so.6
#2 0x0040831c in os_panic () at 
/fdio/src/fdio.1810/src/vpp/vnet/main.c:368
#3 0x75f28f2f in debugger () at 
/fdio/src/fdio.1810/src/vppinfra/error.c:84
#4 0x75f2936a in _clib_error (how_to_die=2, function_name=0x0, 
line_number=0, fmt=0x7fffb2083920 "%s:%d (%s) assertion `%s' fails")
at /fdio/src/fdio.1810/src/vppinfra/error.c:143
#5 0x7fffb2035d79 in vlib_validate_trace (tm=0x7fffbaccfd58, 
b=0x2aaab447d940)
at /fdio/src/fdio.1810/src/vlib/trace_funcs.h:53
#6 0x7fffb2035ec8 in vlib_add_trace (vm=0x7fffbaccfb40, r=0x7fffbb430f40, 
b=0x2aaab447d940, n_data_bytes=36)

The assert is pointing to following line in the code.
ASSERT (!pool_is_free_index (tm->trace_buffer_pool,
vlib_buffer_get_trace_index (b)));

One specific point with this crash is:
This crash is happening only when we have multiple workers in VPP. Following is 
the scenario this crash is happening.
1. Packet comes to worker-1 from nic card
2. graph node on worker-1 hands off the packet to worker-2
3. In worker-2, while processing the packet, we are trying to add the trace 
using vlib_add_trace and this crash occurs.

The trace buffer within vlib_buffer_t, is it specific to worker ?
If so, what happens, when the buffer gets hand off to another worker ?
Can this cause the above crash ?

--
Thanks & Regards,
Murthy
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16613): https://lists.fd.io/g/vpp-dev/message/16613
Mute This Topic: https://lists.fd.io/mt/74625018/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ixge and rdma drivers

[Elias Rudberg]

Hi Ben,

> > (I think there may be a problem with the rdma plugin for larger MTU
> > values but for MTU < 2000 or so, everything works fine.)
>  
> It should work, jumbo support was added in the last months. Or do you
> refer to something else?

I think I mean something else, a problem that I noticed a few weeks ago
but never had time to report it then. Now I tried again and it can
still be reproduced with the current master branch.

The setup is that I have one server running VPP doing NAT44 and then I
have two other servers on inside and outside. This works fine when the
MTU is 1500. Then I set the MTU to 3000 on all involved interfaces and
restart VPP. Now it works as longas only small packets are used, but as
soon as a packet larger than ~2048 bytes appears, VPP stops working.
(Doing e.g. ping -s 2100 is enough to trigger it.) After that VPP is
stuck in some kind of error state from which it does not recover, even
small packets are not forwarded after that.

I tried to investigate further and then it seemed like that what
happens is that the RDMA_DEVICE_F_ERROR flag is set in
src/plugins/rdma/input.c which causes the rdma plugin code to get
stuck, the error flag is never cleared it seems.

The reason why the larger packet size caused an error seems to be that
the log2_cq_size value used in src/plugins/rdma/input.c is
log2_cq_size = 11 which corresponds to 2^11 = 2048 bytes which is
roughly the packet size where the problem appears.

So I got the impression that the rdma plugin is limited to 2^11 = 2048
bytes MTU due to the log2_cq_size = 11 value. Maybe that can be
configured somehow? In any case, it seems bad that VPP gets stuck after
one such error appears, it would be better if it just increased an
error counter and dropped the packet.

Best regards,
Elias

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16612): https://lists.fd.io/g/vpp-dev/message/16612
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] How to trigger the perf test?

[Zhiyong Yang]

Hi vpp/csit experts,

I have submitted one patch that tries to optimize l2 input node.  
https://gerrit.fd.io/r/c/vpp/+/27191
Want to see if it can help enhance the perf or not. But don't how to trigger 
the L2 BD perf test case on CSIT.
Who can help me about it ? Thanks a lot in advancement.

BR
Zhiyong
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16611): https://lists.fd.io/g/vpp-dev/message/16611
Mute This Topic: https://lists.fd.io/mt/74624445/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP fails to start - error message EAL: FATAL: Cannot get hugepage information.

[Benoit Ganne (bganne) via lists.fd.io]

Manoj, can you try VPP-20.05 instead of VPP-19.04? I suspect the DPDK-19.02 
used with VPP-19.04 is too old for your platform and skip 2MB hugepages. DPDK 
19.02 seems to support up to 3 different hugepages size and you appear to have 
4.

ben

> -Original Message-
> From: Manoj Iyer 
> Sent: mardi 2 juin 2020 05:57
> To: Damjan Marion ; Benoit Ganne (bganne)
> 
> Cc: vpp-dev@lists.fd.io; Rodney Schmidt ; Kshitij
> Sudan 
> Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> Cannot get hugepage information.
> 
> Ben,
> 
> Sorry for the delay in posting.
> 
> Here is the strace output: https://pastebin.ubuntu.com/p/NgrtzYDvK7/
> 
> Thanks
> Manoj Iyer
> 
> 
> From: vpp-dev@lists.fd.io  on behalf of Benoit Ganne
> (bganne) via lists.fd.io 
> Sent: Thursday, May 28, 2020 2:21 AM
> To: Manoj Iyer ; Damjan Marion 
> Cc: vpp-dev@lists.fd.io ; Rodney Schmidt
> ; Kshitij Sudan 
> Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> Cannot get hugepage information.
> 
> Could you share the output of:
> ~# strace vpp -c /usr/share/vpp/vpp.conf
> With DPDK enabled (ie with your targeted config which is failing)?
> 
> ben
> 
> > -Original Message-
> > From: Manoj Iyer 
> > Sent: jeudi 28 mai 2020 01:11
> > To: Damjan Marion ; Manoj Iyer 
> > Cc: Benoit Ganne (bganne) ; vpp-dev@lists.fd.io;
> Rodney
> > Schmidt ; Kshitij Sudan 
> > Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> > Cannot get hugepage information.
> >
> > Any more thoughts on this failure ?
> >
> > Thanks
> > Manoj Iyer
> > 
> >
> > From: vpp-dev@lists.fd.io  on behalf of Manoj Iyer
> > via lists.fd.io 
> > Sent: Tuesday, May 26, 2020 6:51 PM
> > To: Damjan Marion 
> > Cc: bga...@cisco.com ; vpp-dev@lists.fd.io  > d...@lists.fd.io>; Rodney Schmidt ; Kshitij Sudan
> > 
> > Subject: Re: [vpp-dev] VPP fails to start - error message EAL: FATAL:
> > Cannot get hugepage information.
> >
> > $ lscpu
> >
> > Architecture: aarch64
> >
> > Byte Order:   Little Endian
> >
> > CPU(s):   8
> >
> > On-line CPU(s) list:  0
> >
> > Off-line CPU(s) list: 1-7
> >
> > Thread(s) per core:   1
> >
> > Core(s) per socket:   1
> >
> > Socket(s):1
> >
> > NUMA node(s): 1
> >
> > Vendor ID:ARM
> >
> > Model:3
> >
> > Model name:   Cortex-A72
> >
> > Stepping: r0p3
> >
> > BogoMIPS: 250.00
> >
> > L1d cache:unknown size
> >
> > L1i cache:unknown size
> >
> > L2 cache: unknown size
> >
> > NUMA node0 CPU(s):0
> >
> > Flags:fp asimd evtstrm aes pmull sha1 sha2 crc32 cpuid
> >
> >
> > $ grep .  /sys/kernel/mm/hugepages/hugepages-*/*
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/free_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_hugepages_mempolicy:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-1048576kB/surplus_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/free_hugepages:1024
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages:1024
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages_mempolicy:1024
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-2048kB/surplus_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/free_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_hugepages_mempolicy:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-32768kB/surplus_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/free_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/nr_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/nr_hugepages_mempolicy:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/nr_overcommit_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/resv_hugepages:0
> >
> > /sys/kernel/mm/hugepages/hugepages-64kB/surplus_hugepages:0
> >
> >
> >
> > $ grep . /sys/devices/system/node/node*/hugepages/hugepages-*/*
> >
> > /sys/devices/system/node/node0/hugepages/hugepages-
> > 1048576kB/free_hugepages:0
> >
> > /sys/devices/system/node/node0/hugepages/hugepages-
> > 1048576kB/nr_hugepages:0
> >
> > /sys/devices/system/node/node0/hugepages/hugepages-
> > 1048576kB/surplus_hugepages:
> >
> > /sys/devices/system/node/node0/hugepages/hugepages-
> > 2048kB/free_hugepages:1024
> >
> > /sys/devices/system/node/node0/hugepa

Re: [vpp-dev] ixge and rdma drivers

[Benoit Ganne (bganne) via lists.fd.io]

Hi Chris, Elias,



>> From: Christian Hopps:

>> I also have systems that have mlx5 (and eventually will have

>> connectx-6 cards). These cards appear to be supported by the rdma

>> native driver. I was able to create the interfaces and saw TX packets

>> but no RX.  Is this driver considered stable and usable in 19.08 (and

>> if not which release would it be consider so)?



It is considered stable and we are supporting it 😊

Re RX, did you sent packets to the VPP interface MAC? The interfaces are not in 
promiscuous mode by default, and VPP does not use the default interface MAC by 
default to not "steel" it from Linux, it behaves a bit like a VF.



> From: Elias Rudberg:

> (I think there may be a problem with the rdma plugin for larger MTU

> values but for MTU < 2000 or so, everything works fine.)



It should work, jumbo support was added in the last months. Or do you refer to 
something else?



ben


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16609): https://lists.fd.io/g/vpp-dev/message/16609
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Building with fresh Ubuntu Focal

[Ole Troan]

Hi Paul,

> Yes.  The build assumes that vppapigen runs in the global python environment 
> such as if in a fresh container.  
> 
> It is the same issue we have with generating vpp papi packages.  It installs 
> into the global environment and people have no option to pip install it 
> without downloading the sources and doing a pip install -e .  There is 
> a vpp_papi package on pypi, but it hasn't been updated in almost 2 years.
> 
> To fix the issue properly, vppapigen should become a python package 
> encapsulating its dependency on ply, so that it works independent of the 
> python version/location.
> 
> The same holds true of the test framework.

Your preferred approach would be that all Python scripts as part of the build / 
test process would run in Python virtualenv?
So first thing cmake does it create the venv and then all python runs from 
there?

Cheers,
Ole-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16608): https://lists.fd.io/g/vpp-dev/message/16608
Mute This Topic: https://lists.fd.io/mt/74604485/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ixge and rdma drivers

[Elias Rudberg]

Hi Chris,

About mlx5, we are using mlx5 cards with the VPP rdma plugin and it is
working fine for us, for VPP 19.08 and newer.

(I think there may be a problem with the rdma plugin for larger MTU
values but for MTU < 2000 or so, everything works fine.)

/ Elias


On Tue, 2020-06-02 at 03:40 -0400, Christian Hopps wrote:
> Hi vpp-dev,
> 
> I've been contemplating trying to use native drivers in place of DPDK
> with the understanding that I may be paying a ~20% penalty by using
> DPDK. So I went to try things out, but had some trouble. The systems
> in paticular I'm interested in have 10GE intel NICs in them which I
> believe would be supported by the ixge driver. I noticed that this
> driver has been marked deprecated in VPP though. Is there a
> replacement or is DPDK required for this NIC?
> 
> I also have systems that have mlx5 (and eventually will have
> connectx-6 cards). These cards appear to be supported by the rdma
> native driver. I was able to create the interfaces and saw TX packets
> but no RX.  Is this driver considered stable and usable in 19.08 (and
> if not which release would it be consider so)?
> 
> Thanks,
> Chris.
> 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16607): https://lists.fd.io/g/vpp-dev/message/16607
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] ixge and rdma drivers

[Christian Hopps]

Hi vpp-dev,

I've been contemplating trying to use native drivers in place of DPDK with the 
understanding that I may be paying a ~20% penalty by using DPDK. So I went to 
try things out, but had some trouble. The systems in paticular I'm interested 
in have 10GE intel NICs in them which I believe would be supported by the ixge 
driver. I noticed that this driver has been marked deprecated in VPP though. Is 
there a replacement or is DPDK required for this NIC?

I also have systems that have mlx5 (and eventually will have connectx-6 cards). 
These cards appear to be supported by the rdma native driver. I was able to 
create the interfaces and saw TX packets but no RX.  Is this driver considered 
stable and usable in 19.08 (and if not which release would it be consider so)?

Thanks,
Chris.-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16606): https://lists.fd.io/g/vpp-dev/message/16606
Mute This Topic: https://lists.fd.io/mt/74623336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] questions on IKEv2

[Mahdi Varasteh]

Hi Filip,

Thanks for your answer. I'm glad to hear them.
I understand the difficulties with ikev2_initiate_sa_init return value. And i 
don't think there is a feasible solution for it because of dependencies on an 
outside VPP source. Maybe events are the best choice.

Regards,
Mahdi
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16605): https://lists.fd.io/g/vpp-dev/message/16605
Mute This Topic: https://lists.fd.io/mt/74577612/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Welcome to VSAP project

[Jerome Tollet via lists.fd.io]

Hello Ping & VSAP folks,
Quite impressive speed up for Nginx!
Jerome

De :  au nom de "Yu, Ping" 
Date : lundi 1 juin 2020 à 11:10
À : "vpp-dev@lists.fd.io" , "csit-...@lists.fd.io" 

Cc : "Yu, Ping" , "Yu, Ping" 
Objet : [vpp-dev] Welcome to VSAP project

Hello, all

Glad to announce that the project VSAP(VPP Stack Acceleration Project) is 
online and this project aims to establish an industry user space application 
ecosystem based on VPP host stack. VSAP has done much work to optimize VPP host 
stack for Nginx, and we can see some exciting numbers comparing with kernel 
space stack.

The detailed project is below:
https://wiki.fd.io/view/VSAP

Welcome to join our mailing-list from below page.
https://lists.fd.io/g/vsap-dev


The VSAP project meeting time will be scheduled in as bi-weekly meeting in 
Friday 9:00 am PRC time( PT:  and next incoming meeting is Jun 12. Please join 
in this meeting via webex: https://intel.webex.com/meet/pyu4


Look forward to meeting with you.

Ping

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16604): https://lists.fd.io/g/vpp-dev/message/16604
Mute This Topic: https://lists.fd.io/mt/74599949/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-