Re: [vpp-dev] VXLAN Tunnel IF Names

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

There are even more filtering options, allowing you to specify a class or a 
single test function within a test case if needed.

Please consult `make test-help`.

Regards,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of John Lo (loj)
> Sent: Friday, February 2, 2018 10:40 PM
> To: Jon Loeliger 
> Cc: vpp-dev 
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
> Hi Jon,
> 
> 
> 
> You can easily run just vxlan tests by “make test TEST=vxlan”, or better yet
> “make test-debug TEST=vxlan”, to run just test_vxlan.py script in VPP normal
> or debug mode. Similarly for other test cases.
> 
> 
> 
> Regards,
> 
> John
> 
> 
> 
> From: Jon Loeliger [mailto:j...@netgate.com]
> Sent: Friday, February 02, 2018 4:18 PM
> To: John Lo (loj) 
> Cc: vpp-dev 
> Subject: Re: [vpp-dev] VXLAN Tunnel IF Names
> 
> 
> 
> On Fri, Feb 2, 2018 at 3:08 PM, Jon Loeliger   > wrote:
> 
> 
> 
>   I have submitted a patch to Gerrit.  When this passes "verify", I'll add
> you to be
> 
>   a Reviewer!
> 
> 
> 
> John,
> 
> 
> 
> How do I run just these tests locally?  I need to determine if
> 
> these failures are due to my patch or not.
> 
> 
> 
> Specifically, I am not able to succesfully "make test" on a CentOS system
> 
> due to Python having a memory corruption failure and dumping core on me.
> 
> As many tests until that point were passing though.
> 
> 
> 
> I'll add you as a patch reviewer anyway -- maybe you can spot something
> 
> brain-dead that I did in the patch...?
> 
> 
> 
> Thanks,
> 
> jdl
> 
> 
> 
> 
> 
> 20:46:28 20:33:41
> ===
> ===
> ==
> 20:46:28 20:33:41 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func ::
> *Ipv6 Multipath routing test cases*
> 20:46:28 20:33:41
> ===
> ===
> ==
> 20:46:28 20:34:06 TC01: IPv6 Equal-cost multipath routing :: [Top] TG=DUT
> | PASS |
> 20:46:28 20:34:06 
> ---
> -
> 20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ecmp-Func ::
> *Ipv6 Multipath routing test cases* | 
> PASS |
> 20:46:28 20:34:06 1 critical test, 1 passed, 0 failed
> 20:46:28 20:34:06 1 test total, 1 passed, 0 failed
> 20:46:28 20:34:06
> ===
> ===
> ==
> 20:46:28 20:34:06 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func ::
> *IPv6 Router Advertisement test cases*
> 20:46:28 20:34:06
> ===
> ===
> ==
> 20:46:28 20:34:20 TC01: DUT transmits RA on IPv6 enabled interface :: [Top]
> TG-DUT1-DUT2-TG.  | PASS |
> 20:46:28 20:34:20 
> ---
> -
> 20:46:28 20:34:36 TC02: DUT retransmits RA on IPv6 enabled interface after a
> set interval :: [Top] TG-DUT1-DUT2-TG.   | PASS |
> 20:46:28 20:34:36 
> ---
> -
> 20:46:28 20:34:49 TC03: DUT responds to Router Solicitation request :: [Top]
> TG-DUT1-DUT2-TG. | FAIL |
> 20:46:28 20:34:49 Traffic script execution failed
> 20:46:28 20:34:49 
> ---
> -
> 20:46:28 20:35:02 TC04: DUT responds to Router Solicitation request sent
> from link local address :: [Top] TG-DUT1-DUT2-TG.| 
> FAIL |
> 20:46:28 20:35:02 Traffic script execution failed
> 20:46:28 20:35:02 
> ---
> -
> 20:46:28 20:35:02 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ip6Ra-Func ::
> *IPv6 Router Advertisement test cases*| 
> PASS |
> 20:46:28 20:35:02 0 critical tests, 0 passed, 0 failed
> 20:46:28 20:35:02 4 tests total, 2 passed, 2 failed
> 20:46:28 20:35:02
> ===
> ===
> ==
> 
> 

Re: [vpp-dev] Why is IP reassemble not supported in VPP?

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Lollita Liu,

I am not in position of decision-making and I do not know the project 
priorities...

I'll have to leave this question open for others to answer.

Thanks,
Klement

> -Original Message-
> From: Lollita Liu [mailto:lollita@ericsson.com]
> Sent: Monday, January 22, 2018 3:25 AM
> To: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> <ksek...@cisco.com>; vpp-dev@lists.fd.io
> Cc: Kingwel Xie <kingwel@ericsson.com>; Terry Zhang Z
> <terry.z.zh...@ericsson.com>; Jordy You <jordy@ericsson.com>
> Subject: RE: Why is IP reassemble not supported in VPP?
> 
> Hi, Klement.
>   Thank you for your response. We will check the patch carefully. But I
> still have question about why IP reassemble is not there or be treated with
> lower priority. I think IP reassemble is mandatory feature as tunnel
> endpoint Please share with me about your thought. Thank you.
> 
> 
> BR/Lollita Liu
> 
> -Original Message-
> From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> [mailto:ksek...@cisco.com]
> Sent: Friday, January 19, 2018 8:26 PM
> To: Lollita Liu <lollita@ericsson.com>; vpp-dev@lists.fd.io
> Cc: Kingwel Xie <kingwel@ericsson.com>; Terry Zhang Z
> <terry.z.zh...@ericsson.com>; Jordy You <jordy@ericsson.com>
> Subject: RE: Why is IP reassemble not supported in VPP?
> 
> Hi Lollita Liu,
> 
> There is a pending patch in gerrit, which adds the support. So far, it wasn't
> reviewed nor merged. I don't have an ETA on that...
> 
> https://gerrit.fd.io/r/#/c/9532/
> 
> Thanks,
> Klement
> 
> > -Original Message-
> > From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io]
> > On Behalf Of Lollita Liu
> > Sent: Friday, January 19, 2018 5:07 AM
> > To: vpp-dev@lists.fd.io
> > Cc: Kingwel Xie <kingwel@ericsson.com>; Terry Zhang Z
> > <terry.z.zh...@ericsson.com>; Jordy You <jordy@ericsson.com>
> > Subject: [vpp-dev] Why is IP reassemble not supported in VPP?
> >
> > Hi,
> >
> > We are do investigation in the VPP source code now.
> > After checking the source code and doing testing, looks VPP is not able
> handle IP fragment.
> >
> >
> >
> > In source code, in function ip4_local_inline, looks
> > fragment will be treat as error packet finally because of
> IP4_ERROR_UNKNOWN_PROTOCOL.
> >
> >   /* Treat IP frag packets as "experimental" protocol
> > for now
> >
> >  until support of IP frag reassembly is
> > implemented */
> >
> >   proto0 = ip4_is_fragment (ip0) ? 0xfe :
> > ip0->protocol;
> >
> >   proto1 = ip4_is_fragment (ip1) ? 0xfe :
> > ip1->protocol;
> >
> > ...
> >
> > next0 = lm->local_next_by_ip_protocol[proto0];
> >
> > next1 = lm->local_next_by_ip_protocol[proto1];
> >
> > ...
> >
> > next0 =
> >
> > error0 != IP4_ERROR_UNKNOWN_PROTOCOL ?
> > IP_LOCAL_NEXT_DROP : next0;
> >
> >   next1 =
> >
> > error1 != IP4_ERROR_UNKNOWN_PROTOCOL ?
> > IP_LOCAL_NEXT_DROP : next1;
> >
> >
> >
> > The version is:
> >
> > DBGvpp# show version
> >
> > vpp v18.04-rc0~46-gc5239ad built by root on k8s1-node1 at Mon Jan 15
> > 06:05:03 UTC 2018
> >
> >
> >
> > My question is why IP reassemble is not supported in VPP? It is
> > understandable that IP reassemble is not required for pure packet
> forwarding.
> > But as a router platform, there are also plenty of control plane
> > packets should be handled, for example BGP packet, IKE packet, that's
> > the reason why there is local IP stack on VPP, and IP reassemble is a
> > basic requirement of local IP stack. How to handle the case if the BGP
> > peer send BGP message in several IP fragment to VPP? One BGP message
> > could be quite large depending on route number, and even BGP message
> > fragment can be avoid by MSS since it is based on TCP. How about the
> > case of IKE peer sending IKE message as IP fragments? The IKE message also
> could be quite large with certificate...
> >
> >
> >
> > BR/Lollita Liu

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Why is IP reassemble not supported in VPP?

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Lollita Liu,

There is a pending patch in gerrit, which adds the support. So far, it wasn't 
reviewed nor merged. I don't have an ETA on that...

https://gerrit.fd.io/r/#/c/9532/

Thanks,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of Lollita Liu
> Sent: Friday, January 19, 2018 5:07 AM
> To: vpp-dev@lists.fd.io
> Cc: Kingwel Xie ; Terry Zhang Z
> ; Jordy You 
> Subject: [vpp-dev] Why is IP reassemble not supported in VPP?
> 
> Hi,
> 
> We are do investigation in the VPP source code now. After 
> checking
> the source code and doing testing, looks VPP is not able handle IP fragment.
> 
> 
> 
> In source code, in function ip4_local_inline, looks fragment 
> will be
> treat as error packet finally because of IP4_ERROR_UNKNOWN_PROTOCOL.
> 
>   /* Treat IP frag packets as "experimental" protocol for now
> 
>  until support of IP frag reassembly is implemented */
> 
>   proto0 = ip4_is_fragment (ip0) ? 0xfe : ip0->protocol;
> 
>   proto1 = ip4_is_fragment (ip1) ? 0xfe : ip1->protocol;
> 
> ...
> 
> next0 = lm->local_next_by_ip_protocol[proto0];
> 
> next1 = lm->local_next_by_ip_protocol[proto1];
> 
> ...
> 
> next0 =
> 
> error0 != IP4_ERROR_UNKNOWN_PROTOCOL ?
> IP_LOCAL_NEXT_DROP : next0;
> 
>   next1 =
> 
> error1 != IP4_ERROR_UNKNOWN_PROTOCOL ?
> IP_LOCAL_NEXT_DROP : next1;
> 
> 
> 
> The version is:
> 
> DBGvpp# show version
> 
> vpp v18.04-rc0~46-gc5239ad built by root on k8s1-node1 at Mon Jan 15
> 06:05:03 UTC 2018
> 
> 
> 
> My question is why IP reassemble is not supported in VPP? It is
> understandable that IP reassemble is not required for pure packet forwarding.
> But as a router platform, there are also plenty of control plane packets 
> should
> be handled, for example BGP packet, IKE packet, that's the reason why there is
> local IP stack on VPP, and IP reassemble is a basic requirement of local IP
> stack. How to handle the case if the BGP peer send BGP message in several IP
> fragment to VPP? One BGP message could be quite large depending on route
> number, and even BGP message fragment can be avoid by MSS since it is
> based on TCP. How about the case of IKE peer sending IKE message as IP
> fragments? The IKE message also could be quite large with certificate...
> 
> 
> 
> BR/Lollita Liu

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Test scenario for VRF

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

I think these tests should touch vrfs...

~/v/test> ls *vrf*
test_ip4_vrf_multi_instance.py  test_ip4_vrf_multi_instance.pyc  
test_ip6_vrf_multi_instance.py  test_ip6_vrf_multi_instance.pyc

Regards,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of mirzaei.reza
> Sent: Tuesday, January 16, 2018 11:30 AM
> To: Vpp Dev 
> Subject: [vpp-dev] Test scenario for VRF
> 
> Hi
> 
> I want to know is there any test scenario for VRF using vpp?
> 
> Best regards
> 
> Reza

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] vpp api

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi, it's in

src/vpp-api/client

Regards,
Kement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of Holoo Gulakh
> Sent: Tuesday, December 12, 2017 10:49 AM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] vpp api
> 
> Hi,
> 
> In order to use VPP API in python, I use this command
> 
> "sudo –E LD_LIBRARY_PATH=/vpp/build-root/install-vpp_debug-
> native/vpp/lib64/libvppapiclient.so python testAPI.py"
> 
> As you can see in the above command, I should link libvppapiclient.so to my
> code
> 
> *** My question is that "where is the source code for libvppapiclient.so??"
> I want to inspect source code to see what and how it works ***
> 
> thanks in advance

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] FW: how to redirect specific UDP port data to a new node?

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Are you implementing a new graph node, which consumes UDP traffic? If so, then 
the BFD example is a way to redirect interesting traffic to your new node...

> -Original Message-
> From: lin huang [mailto:mit...@outlook.com]
> Sent: Monday, December 4, 2017 9:16 AM
> To: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> <ksek...@cisco.com>; vpp-dev@lists.fd.io
> Subject: RE: [vpp-dev] FW: how to redirect specific UDP port data to a new
> node?
> 
> Hi Klement,
> 
>   I want to redirect data to a new udp port. Do I need to modify VPP
> code?
> Adding a new UDP_DST_PORT_ enum types??
> 
> Thanks a lot!!!
> 
> >>-----Original Message-----
> >>From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> >>[mailto:ksek...@cisco.com]
> >>Sent: Monday, December 04, 2017 4:12 PM
> >>To: lin huang <mit...@outlook.com>; vpp-dev@lists.fd.io
> >>Subject: RE: [vpp-dev] FW: how to redirect specific UDP port data to a
> >>new node?
> >>
> >>Hi Lin Huang,
> >>
> >>You can take a look at bfd_udp_init() in bfd_udp.c to see how the BFD
> >>feature registers to listen on BFD ports using udp_register_dst_port().
> >>
> >>Thanks,
> >>Klement
> >>
> >>> -Original Message-
> >>> From: vpp-dev-boun...@lists.fd.io
> >>> [mailto:vpp-dev-boun...@lists.fd.io]
> >>> On Behalf Of lin huang
> >>> Sent: Monday, December 4, 2017 9:03 AM
> >>> To: vpp-dev@lists.fd.io
> >>> Subject: [vpp-dev] FW: how to redirect specific UDP port data to a new
> node?
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> From: lin huang
> >>> Sent: Monday, December 04, 2017 2:53 PM
> >>> To: vpp-dev-boun...@lists.fd.io
> >>> Subject: how to redirect specific UDP port data to a new node?
> >>>
> >>>
> >>>
> >>> Hi, all.
> >>>
> >>>  I want to know how to redirect specific UDP port data to a
> >>> new
> >>node?
> >>>
> >>> Dose udp_register_dst_port make effecet? Or using the arc with feature?
> >>>
> >>>
> >>>
> >>> Thanks!!!

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] FW: how to redirect specific UDP port data to a new node?

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Lin Huang,

You can take a look at bfd_udp_init() in bfd_udp.c to see how the BFD feature 
registers to listen on BFD ports using udp_register_dst_port().

Thanks,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of lin huang
> Sent: Monday, December 4, 2017 9:03 AM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] FW: how to redirect specific UDP port data to a new node?
> 
> 
> 
> 
> 
> From: lin huang
> Sent: Monday, December 04, 2017 2:53 PM
> To: vpp-dev-boun...@lists.fd.io
> Subject: how to redirect specific UDP port data to a new node?
> 
> 
> 
> Hi, all.
> 
>  I want to know how to redirect specific UDP port data to a new node?
> 
> Dose udp_register_dst_port make effecet? Or using the arc with feature?
> 
> 
> 
> Thanks!!!

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Question on node type "VLIB_NODE_TYPE_PROCESS"

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Also to for 'show trace' to show anything, you need to enable tracing ...

e.g. 

trace add  50

to trace next 50 packets

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of Dave Barach (dbarach)
> Sent: Friday, December 1, 2017 12:32 AM
> To: Yeddula, Avinash ; vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Question on node type "VLIB_NODE_TYPE_PROCESS"
> 
> At least for now, process nodes run on the main thread. See line 1587 of
> .../src/vlib/main.c.
> 
> 
> 
> The lldp-process is not super-complicated. Set a gdb breakpoint on line 157
> [switch(event_type)], cause it to do something, and you can walk through it, 
> etc.
> 
> 
> 
> HTH... Dave
> 
> 
> 
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of Yeddula, Avinash
> Sent: Thursday, November 30, 2017 5:49 PM
> To: vpp-dev@lists.fd.io
> Subject: [vpp-dev] Question on node type "VLIB_NODE_TYPE_PROCESS"
> 
> 
> 
> Hello,
> 
> 
> 
> I have a setup with, 1 worker thread (Core 8) and 1 main thread (Core 0).
> 
> 
> 
> As I read about the node type VLIB_NODE_TYPE_PROCESS, it says
> 
> "The graph node scheduler invokes these processes in much the same way as
> traditional vector-processing run-to-completion graph  nodes".
> 
> 
> 
> For eg..
> 
> A node like "lldp_process_node", as I see whenever a timeout occurs or an
> event has been generated, a frame has been sent out of an interface. The
> questions I have are..
> 
> 
> 
> 1.The part I'm not able to figure out yet is, where is (on which
> thread/core) this "lldp_process_node" running in the back ground ?  I'm
> assuming it cannot be worker thread.
> 
> 
> 
> 2.Would you please point me the piece of code in vpp infra, that
> schedules all nodes of type "VLIB_NODE_TYPE_PROCESS".
> 
> 
> 
> 3.I tried to turn on few debugs like this
> "VLIB_BUFFER_TRACE_TRAJECTORY" and few other ones. None of them seems
> to generate any traces/logs (show trace - doesn't give me any info). Any
> pointers on how to enable relevant logs for this activity.
> 
> 
> 
> Thanks
> 
> -Avinash
> 
> 

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Can't run tests (ImportError: No module named bier)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Sure, that's a good idea.
I haven't noticed this issue really since I wipe via git clean ...

Thanks,
Klement

> -Original Message-
> From: Marco Varlese [mailto:mvarl...@suse.de]
> Sent: Wednesday, November 22, 2017 4:43 PM
> To: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> <ksek...@cisco.com>; vpp-dev <vpp-dev@lists.fd.io>
> Subject: Re: [vpp-dev] Can't run tests (ImportError: No module named bier)
> 
> On Wed, 2017-11-22 at 15:31 +, Klement Sekera -X (ksekera - PANTHEON
> TECHNOLOGIES at Cisco) wrote:
> > Hi,
> >
> > `make test-wipe` should help
> Interesting... I ran "make wipe" before.
> May I suggest to have test-wipe run as part of the more generic wipe ???
> I'll submit a patch...
> >
> > Regards,
> > Klement
> Cheers,
> Marco
> >
> > > -Original Message-
> > > From: vpp-dev-boun...@lists.fd.io
> > > [mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Marco Varlese
> > > Sent: Wednesday, November 22, 2017 4:30 PM
> > > To: vpp-dev <vpp-dev@lists.fd.io>
> > > Subject: [vpp-dev] Can't run tests (ImportError: No module named
> > > bier)
> > >
> > > Hi,
> > >
> > > I just took latest master and cannot run tests anymore...
> > >
> > > Adding tests from directory tree /home/mvarlese/repo/vpp/test
> > > Traceback (most recent call last):
> > >   File "run_tests.py", line 146, in 
> > > discover_tests(d, cb)
> > >   File "/home/mvarlese/repo/vpp/test/discover_tests.py", line 27, in
> > > discover_tests
> > > module = importlib.import_module(name)
> > >   File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in
> > > import_module
> > > __import__(name)
> > >   File "/home/mvarlese/repo/vpp/test/test_bier.py", line 17, in 
> > > from scapy.contrib.bier import *
> > > ImportError: No module named bier
> > > Killing possible remaining process IDs:  9912 9914 No symlinks to
> > > failed tests'
> > > temporary directories found in /tmp/vpp-failed- unittests/.
> > > make[1]: *** [Makefile:124: test] Error 1
> > > make[1]: Leaving directory '/home/mvarlese/repo/vpp/test'
> > > make: *** [Makefile:362: test] Error 2
> > >
> > > Am I missing something? :(
> > >
> > >
> > > Cheers,
> > > Marco
> > >
> > > ___
> > > vpp-dev mailing list
> > > vpp-dev@lists.fd.io
> > > https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Can't run tests (ImportError: No module named bier)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

`make test-wipe` should help

Regards,
Klement

> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On
> Behalf Of Marco Varlese
> Sent: Wednesday, November 22, 2017 4:30 PM
> To: vpp-dev 
> Subject: [vpp-dev] Can't run tests (ImportError: No module named bier)
> 
> Hi,
> 
> I just took latest master and cannot run tests anymore...
> 
> Adding tests from directory tree /home/mvarlese/repo/vpp/test Traceback
> (most recent call last):
>   File "run_tests.py", line 146, in 
> discover_tests(d, cb)
>   File "/home/mvarlese/repo/vpp/test/discover_tests.py", line 27, in
> discover_tests
> module = importlib.import_module(name)
>   File "/usr/lib64/python2.7/importlib/__init__.py", line 37, in import_module
> __import__(name)
>   File "/home/mvarlese/repo/vpp/test/test_bier.py", line 17, in 
> from scapy.contrib.bier import *
> ImportError: No module named bier
> Killing possible remaining process IDs:  9912 9914 No symlinks to failed 
> tests'
> temporary directories found in /tmp/vpp-failed- unittests/.
> make[1]: *** [Makefile:124: test] Error 1
> make[1]: Leaving directory '/home/mvarlese/repo/vpp/test'
> make: *** [Makefile:362: test] Error 2
> 
> Am I missing something? :(
> 
> 
> Cheers,
> Marco
> 
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] make test-all

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

That's a good catch, Gabriel...

I tried checking out the version which introduced this test as part of:

commit 8a0a0ae60b8dd9da7cf53c895e85dc6daf67143d
Author: Hongjun Ni <hongjun...@intel.com>
Date:   Sat May 27 20:23:09 2017 +0800

Rework vxlan-gpe to support FIB 2.0 and bypass mode

Change-Id: I0324f945bdb4dd3b19151be6f3ce24a47a000104
Signed-off-by: Hongjun Ni <hongjun...@intel.com>

and sadly, it doesn't pass...

Hongjun, did this test pass for you? Could you please correct it so that
it passes when invoked with 'make test-all TEST=vxlan_gpe' (or other
filter which causes it to run)?

Thanks,
Klement

Quoting Gabriel Ganne (2017-11-14 17:08:20)
>Aside from this p2p_ethernet load test, the only failing extended test is
>the whole TestVxlanGpe class.
> 
>The reason is that scapy 2.3.3 does not know of layer VXLAN, and there is
>no scapy patch within vpp to address this.
> 
>How do you want to handle this ?
> 
>--
> 
>Gabriel Ganne
> 
>══
> 
>From: vpp-dev-boun...@lists.fd.io <vpp-dev-boun...@lists.fd.io> on behalf
>of Brian Brooks <brian.bro...@arm.com>
>    Sent: Tuesday, November 14, 2017 1:27:47 PM
>To: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco); Ole
>Troan
>Cc: vpp-dev@lists.fd.io
>Subject: Re: [vpp-dev] make test-all
> 
>It does not complete within 20 minutes on other machines.
> 
>> I don't think we should do scale tests as part of verification tests.
> 
>    Agree
> 
>-Original Message-
>From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
>[[1]mailto:ksek...@cisco.com]
>Sent: Tuesday, November 14, 2017 2:47 AM
>To: Ole Troan <otr...@employees.org>
>Cc: Dave Barach (dbarach) <dbar...@cisco.com>; John Lo (loj)
><l...@cisco.com>; Pavel Kotucek -X (pkotucek - PANTHEON TECHNOLOGIES at
>Cisco) <pkotu...@cisco.com>; vpp-dev@lists.fd.io; Brian Brooks
><brian.bro...@arm.com>
>Subject: Re: [vpp-dev] make test-all
> 
>It is ...
> 
>it takes ~10 seconds to create 1000 subifs on a beefy UCS and the case
>tries to create 10, so that would indicate a runtime of ~16minutes...
> 
>Quoting Ole Troan (2017-11-14 03:53:28)
>> Klement,
>>
>> > I don't know what to tell you ... I was never a fan of getting the
>> > API trace post test run and putting it into test log (which is the
>> > cause of 25MB allocation in this case - it's the CLI output) - from
>> > my POV this is duplicate information as every API call is already
>> > logged in-place when it's executed... BUT I didn't saw any harm in
>> > doing so (besides clutter) and since the "create bazillion
>> > subinterfaces" test went in without me being part of review process, I
>wasn't aware of it...
>>
>> If this is in test_p2p_ethernet, let me take that out.
>> I don't think we should do scale tests as part of verification tests.
>> I do not like the path we're on with regards to test run time.
>>
>> Cheers,
>> Ole
>>
>> > Quoting Dave Barach (dbarach) (2017-11-13 13:33:41)
>> >> Try increasing the size of the shared-memory API segment. An
>allocation of 25mb is failing. You might ask yourself how sane it is to
>generate that much output.
>> >>
>> >> Thanks… Dave
>> >>
>> >> -Original Message-
>> >> From: vpp-dev-boun...@lists.fd.io
>> >> [[2]mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Klement Sekera
>-X
>> >> (ksekera - PANTHEON TECHNOLOGIES at Cisco)
>> >> Sent: Monday, November 13, 2017 5:27 AM
>> >> To: John Lo (loj) <l...@cisco.com>; Pavel Kotucek -X (pkotucek -
>> >> PANTHEON TECHNOLOGIES at Cisco) <pkotu...@cisco.com>;
>> >> vpp-dev@lists.fd.io; Brian Brooks <brian.bro...@arm.com>
>> >> Subject: Re: [vpp-dev] make test-all
>> >>
>> >> So it seems that vpp coredumps while dumping the API trace after
>> >> creating all the interfaces...
>> >>
>> >> (gdb) bt
>> >> #0  0x7f14f4b1e428 in __GI_raise (sig=sig@entry=6) at
>> >> ../sysdeps/unix/sysv/linux/raise.c:54
>> >> #1  0x7f14f4b2002a in __GI_abort () at abort.c:89
>> >> #2  0x00405d83 in os_panic () at
>> >> /home/kseke

Re: [vpp-dev] make test-all

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

I don't know what to tell you ... I was never a fan of getting the API
trace post test run and putting it into test log (which is the cause
of 25MB allocation in this case - it's the CLI output) - from my POV
this is duplicate information as every API call is already logged
in-place when it's executed... BUT I didn't saw any harm in doing so
(besides clutter) and since the "create bazillion subinterfaces" test
went in without me being part of review process, I wasn't aware of it...

Cheers,
Klement

Quoting Dave Barach (dbarach) (2017-11-13 13:33:41)
> Try increasing the size of the shared-memory API segment. An allocation of 
> 25mb is failing. You might ask yourself how sane it is to generate that much 
> output. 
> 
> Thanks… Dave
> 
> -Original Message-
> From: vpp-dev-boun...@lists.fd.io [mailto:vpp-dev-boun...@lists.fd.io] On 
> Behalf Of Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> Sent: Monday, November 13, 2017 5:27 AM
> To: John Lo (loj) <l...@cisco.com>; Pavel Kotucek -X (pkotucek - PANTHEON 
> TECHNOLOGIES at Cisco) <pkotu...@cisco.com>; vpp-dev@lists.fd.io; Brian 
> Brooks <brian.bro...@arm.com>
> Subject: Re: [vpp-dev] make test-all
> 
> So it seems that vpp coredumps while dumping the API trace after
> creating all the interfaces...
> 
> (gdb) bt
> #0  0x7f14f4b1e428 in __GI_raise (sig=sig@entry=6) at 
> ../sysdeps/unix/sysv/linux/raise.c:54
> #1  0x7f14f4b2002a in __GI_abort () at abort.c:89
> #2  0x00405d83 in os_panic () at 
> /home/ksekera/vpp/build-data/../src/vpp/vnet/main.c:268
> #3  0x7f14f5fe5f86 in clib_mem_alloc_aligned_at_offset 
> (os_out_of_memory_on_failure=1, align_offset=0, align=1, size=25282098)
> at /home/ksekera/vpp/build-data/../src/vppinfra/mem.h:105
> #4  clib_mem_alloc (size=25282098) at 
> /home/ksekera/vpp/build-data/../src/vppinfra/mem.h:114
> #5  vl_msg_api_alloc_internal (may_return_null=0, pool=, 
> nbytes=25282098)
> at /home/ksekera/vpp/build-data/../src/vlibmemory/memory_shared.c:176
> #6  vl_msg_api_alloc (nbytes=nbytes@entry=25282082) at 
> /home/ksekera/vpp/build-data/../src/vlibmemory/memory_shared.c:207
> #7  0x00411392 in vl_api_cli_inband_t_handler (mp=0x300e2a0c) at 
> /home/ksekera/vpp/build-data/../src/vpp/api/api.c:223
> #8  0x7f14f5fdfa23 in vl_msg_api_handler_with_vm_node 
> (am=am@entry=0x7f14f620d460 , the_msg=the_msg@entry=0x300e2a0c,
> vm=vm@entry=0x7f14f5fd6260 , 
> node=node@entry=0x7f14b410e000) at 
> /home/ksekera/vpp/build-data/../src/vlibapi/api_shared.c:508
> #9  0x7f14f5fef35f in memclnt_process (vm=0x7f14f5fd6260 
> , node=0x7f14b410e000, f=)
> at /home/ksekera/vpp/build-data/../src/vlibmemory/memory_vlib.c:970
> 
> (gdb) p input
> $5 = {buffer = 0x7f14b56f6558 "dump 
> /tmp/vpp-unittest-P2PEthernetAPI-qRwMY6/vpp_api_trace.test_p2p_subif_creation_10k.log\n",
>   index = 18446744073709551615, buffer_marks = 0x7f14b592a240, fill_buffer = 
> 0x0, fill_buffer_arg = 0x0}
> 
> I'm pretty sure that the history of this mess was:
> 
> 1.) the test was added first as enhanced
> 2.) automatic dump of api trace was added, but only tested against 'make 
> test', not 'make test-all'
> 
> Thanks,
> Klement
> 
> Quoting Klement Sekera (2017-11-11 22:12:52)
> > Hi Brian,
> > 
> > it should. Though I just tried running it on latest master and got a
> > timeout in test_p2p_ethernet, which shouldn't happen. I see the test was
> > trying to create tens of thousands of interfaces... maybe something is
> > slower than usual?
> > 
> > Thanks,
> > Klement
> > 
> > Quoting Brian Brooks (2017-11-11 01:11:47)
> > >Should “make test-all” pass?
> > > 
> > > 
> > > 
> > >Thanks,
> > > 
> > >Brian
> > > 
> > > 
> > > 
> > >IMPORTANT NOTICE: The contents of this email and any attachments are
> > >confidential and may also be privileged. If you are not the intended
> > >recipient, please notify the sender immediately and do not disclose the
> > >contents to any other person, use it for any purpose, or store or copy 
> > > the
> > >information in any medium. Thank you.
> > ___
> > vpp-dev mailing list
> > vpp-dev@lists.fd.io
> > https://lists.fd.io/mailman/listinfo/vpp-dev
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] make test-all

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Gabriel,

no, I don't they are... the argument being that it takes too long IIRC.

Thanks,
Klement

Quoting Gabriel Ganne (2017-11-13 10:36:25)
>Hi Klement,
> 
>Are those extended tests run on a regular basis anywhere ?
>I do not see them called within any of the the continuous integration
>tests.
> 
>Regards,
> 
>--
> 
>Gabriel Ganne
> 
>══
> 
>From: vpp-dev-boun...@lists.fd.io <vpp-dev-boun...@lists.fd.io> on behalf
>    of Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
><ksek...@cisco.com>
>Sent: Saturday, November 11, 2017 10:12:55 PM
>To: vpp-dev@lists.fd.io; Brian Brooks; Pavel Kotucek -X (pkotucek -
>PANTHEON TECHNOLOGIES at Cisco); John Lo (loj)
>Subject: Re: [vpp-dev] make test-all
> 
>Hi Brian,
> 
>it should. Though I just tried running it on latest master and got a
>timeout in test_p2p_ethernet, which shouldn't happen. I see the test was
>trying to create tens of thousands of interfaces... maybe something is
>slower than usual?
> 
>Thanks,
>Klement
> 
>Quoting Brian Brooks (2017-11-11 01:11:47)
>>    Should “make test-all” pass?
>>
>>     
>>
>>    Thanks,
>>
>>    Brian
>>
>>     
>>
>>    IMPORTANT NOTICE: The contents of this email and any attachments are
>>    confidential and may also be privileged. If you are not the intended
>>    recipient, please notify the sender immediately and do not disclose
>the
>>    contents to any other person, use it for any purpose, or store or
>copy the
>>    information in any medium. Thank you.
>___
>vpp-dev mailing list
>vpp-dev@lists.fd.io
>[1]https://lists.fd.io/mailman/listinfo/vpp-dev
> 
> References
> 
>Visible links
>1. https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] make test-all

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Brian,

it should. Though I just tried running it on latest master and got a
timeout in test_p2p_ethernet, which shouldn't happen. I see the test was
trying to create tens of thousands of interfaces... maybe something is
slower than usual?

Thanks,
Klement

Quoting Brian Brooks (2017-11-11 01:11:47)
>Should “make test-all” pass?
> 
> 
> 
>Thanks,
> 
>Brian
> 
> 
> 
>IMPORTANT NOTICE: The contents of this email and any attachments are
>confidential and may also be privileged. If you are not the intended
>recipient, please notify the sender immediately and do not disclose the
>contents to any other person, use it for any purpose, or store or copy the
>information in any medium. Thank you.
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] ACL Build/Test Issues

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Quoting Jon Loeliger (2017-11-10 23:11:36)

>First, this is draconian for no really good reason.  Second, it should be
>fixed.  Third, I would do that except I am stupid and need a clue where
>or how to fix this situation so the tests are less draconian.  (Can we
>get a "less than 0" test instead of "equal to -1"?)

When writing the logic, it was sufficient (so far) for everybody to just
know what is the correct value to expect (0, -1, ..). If there really is
a situation when an API call can have a random (from the client's view)
return value, I can think of two approaches to deal with this...

1.) introduce a special case "expected_retval=None" which just skips
verifying the return value (in framework) leaving this up to the caller
2.) allow passing a functor instead of the expected value, which
fulfills the role of validator and have the call it, supplying the actual
"retval" from API response message as a parameter to the functor to figure
out whether it's one of the expected ones or nor ...

Thanks,
Klement
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Neighbor DUMP Takes Forever

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Jon,

the usual way to deal with possible-empty result sets is to wrap the
call with a control-ping.

1.) send control-ping
2.) send e.g. sw_interface_dump
3.) start collecting sw_interface_details responses (if any)
4.) once control-ping reply is received, you know that there will be no more
responses to the sw_interface_dump call.

This approach is used by all the existing API bindings, python, C VAPI,
C++ VAPI...

There have been proposals to have this logic built-in, but it doesn't
seem to have happened so far..

HTH,
Klement

Quoting Jon Loeliger (2017-11-08 19:25:03)
>Hi Folks,
>OK, maybe the Neighbor DUMP doesn't literally take forever.
>But it sure does take too long!  Here is the problem:
>When one goes to DUMP the Neighbor tables, one has no idea which interface
>might have ARP/NDP entries on them.  So one determines the total number of
>interfaces, and then one iterates over them one at a time asking for an 
>IP_NEIGHBOR_DUMP of each sw_if_index and AF combination.
>That's all fine.  The problem is that the DUMP response DETAILS never come
>for
>interfaces that have NO entries.  So the message handling times-out
>instead.  With
>a one-second timeout on each (interface, AF) pair, it takes 2 seconds per
>IF.
>It makes for a very slow process to determine the ARP/Neighbor tables.
>This used to be a problem in the NAT DUMP as well, but that was fixed.  I
>don't
>recall how off-hand, but at least one solution would be to always reply,
>and if a
>bogus entry is needed, do that.  Use a detail with a sw_if_index = ~0 if
>necessary.
>(Yeah, the ip_neighbor_t detail should be returning the sw_if_index too
>anyway.)
>Maybe something like that?
>Oh, the reason that you never see this problem with vppctl is that it
>doesn't
>use the API. It goes straight to internal data structures instead.
>Thanks,
>jdl
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] make test

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Holoo,

running 'make test-help' should explain all the possibilities, including
the one which you're asking about.

Cheers,
Klement

Quoting Holoo Gulakh (2017-11-03 13:01:12)
>Hi
>I want to know is there any make <...> command to make just one special
>test (for example test bmpls) instead of using make test, because this
>command makes all tests.
> 
>Best regards
>Holoo
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VPP_Python_API tutorial

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Reza,

what's unclear about the Python API bindings usage?

Could you be looking for a list of API calls & their parameters? If so,
then that is a different question and there might not be an answer
available in form of a reference guide...
The usual way is to take a look at the various .api files which describe APIs.
C/Python/JAVA bindings are auto-generated based on those .api files
(or .json, which themselves are generated from .api).

Thanks,
Klement

Quoting mirzaei.reza (2017-11-03 11:51:33)
>Hi
> 
>I'm looking for a complete VPP_Python_API tutorial, but i just
>find [1]this url.
> 
>Is there any good tutorial on how to use this APIs?
> 
>Best regards
> 
>Reza
> 
> References
> 
>Visible links
>1. https://wiki.fd.io/view/VPP/Python_API
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] gerrit 8872 centos validation failure (stable/1710)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

I think I saw an email a few days ago that this is resolved by rebasing
the patch.

HTH,
Klement

Quoting Dave Barach (dbarach) (2017-10-18 15:56:53)
>Please see [1]https://gerrit.fd.io/r/#/c/8872 and
>[2]https://jenkins.fd.io/job/vpp-verify-1710-centos7/53. I’ve already
>pressed the “recheck” button. The validation failure appears unrelated to
>the patch.
> 
> 
> 
>Thanks... Dave
> 
> 
> 
>12:50:49 Wrote:
>
> /w/workspace/vpp-verify-1710-centos7/dpdk/rpm/RPMS/x86_64/vpp-dpdk-devel-17.08-vpp1.x86_64.rpm
> 
>12:50:49 Executing(%clean): /bin/sh -e /var/tmp/rpm-tmp.cojge5
> 
>12:50:49 + umask 022
> 
>12:50:49 + cd /w/workspace/vpp-verify-1710-centos7/dpdk/rpm/BUILD
> 
>12:50:49 + /usr/bin/rm -rf
>
> /w/workspace/vpp-verify-1710-centos7/dpdk/rpm/BUILDROOT/vpp-dpdk-17.08-vpp1.x86_64
> 
>12:50:49 + exit 0
> 
>12:50:49 mv rpm/RPMS/x86_64/*.rpm .
> 
>12:50:49 git clean -fdx rpm
> 
>12:50:49 Removing rpm/BUILD/
> 
>12:50:49 Removing rpm/BUILDROOT/
> 
>12:50:49 Removing rpm/RPMS/
> 
>12:50:49 Removing rpm/SOURCES/
> 
>12:50:49 Removing rpm/SPECS/
> 
>12:50:49 Removing rpm/SRPMS/
> 
>12:50:49 Removing rpm/tmp/
> 
>12:50:49 make[2]: Leaving directory
>`/w/workspace/vpp-verify-1710-centos7/dpdk'
> 
>12:50:49 sudo rpm -Uih vpp-dpdk-devel-17.08-vpp1.x86_64.rpm
> 
>12:50:49 
> 
>12:50:49   package vpp-dpdk-devel-17.08-vpp2.x86_64 (which is newer than
>vpp-dpdk-devel-17.08-vpp1.x86_64) is already installed
> 
>12:50:49 make[1]: *** [install-rpm] Error 2
> 
>12:50:49 make[1]: Leaving directory
>`/w/workspace/vpp-verify-1710-centos7/dpdk'
> 
>12:50:49 make: *** [dpdk-install-dev] Error 2
> 
>12:50:49 Build step 'Execute shell' marked build as failure
> 
>12:50:49 $ ssh-agent -k
> 
>12:50:49 unset SSH_AUTH_SOCK;
> 
>12:50:49 unset SSH_AGENT_PID;
> 
>12:50:49 echo Agent pid 9677 killed;
> 
>12:50:50 [ssh-agent] Stopped.
> 
>12:50:50 Skipped archiving because build is not successful
> 
>12:50:50 [PostBuildScript] - Execution post build scripts.
> 
> 
> 
>Thanks… Dave
> 
> 
> 
> References
> 
>Visible links
>1. https://gerrit.fd.io/r/#/c/8872
>2. https://jenkins.fd.io/job/vpp-verify-1710-centos7/53
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] missing vpp_api module when I execute "make test"

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Burt,

looking at vpp_papi (which is Ole's deed), I see

vpp_api = ffi.dlopen('libvppapiclient.so')

so it looks like it couldn't load said .so. It surely should've been
built as part of 'make test'. Does it exist in your WS?

Ole, thoughts?

Regards,
Klement

Quoting Burt Silverman (2017-10-11 23:30:38)
>Hi Klement,
>Any idea what I am doing wrong? Is master the wrong place to try "make
>test"? I suppose it should be OK. I tried make test-wipe, but that does
>not help. I didn't see anything posted recently on lists or wiki. Thanks,
>Burt
>...
>Using
>/home/burts/vpp3/build-root/python/virtualenv/lib/python2.7/site-packages
>Finished processing dependencies for vpp-papi==1.4
>make -C ext
>make[2]: Entering directory '/home/burts/vpp3/test/ext'
>mkdir -p /home/burts/vpp3/build-root/vapi_test/
>/home/burts/vpp3/src/vpp-api/vapi/vapi_c_gen.py --prefix
>/home/burts/vpp3/build-root/vapi_test/ fake.api.json
>cc -o /home/burts/vpp3/build-root/vapi_test/vapi_c_test -std=gnu99 -g
>-Wall -pthread -I/home/burts/vpp3/src
>-I/home/burts/vpp3/build-root/install-vpp-native//vpp/include
>-I/home/burts/vpp3/build-root/vapi_test/ vapi_c_test.c
>-L/home/burts/vpp3/build-root/build-vpp-native/vpp/.libs/
>-L/home/burts/vpp3/build-root/build-vpp-native/vpp/vpp-api/vapi/.libs/
>-lvppinfra -lvlibmemoryclient -lsvm -lpthread -lcheck -lsubunit -lrt -lm
>-lvapiclient
>/home/burts/vpp3/src/vpp-api/vapi/vapi_cpp_gen.py --prefix
>/home/burts/vpp3/build-root/vapi_test/ fake.api.json
>g++ -o /home/burts/vpp3/build-root/vapi_test/vapi_cpp_test -std=c++11 -g
>-Wall -pthread -I/home/burts/vpp3/src
>-I/home/burts/vpp3/build-root/install-vpp-native//vpp/include
>-I/home/burts/vpp3/build-root/vapi_test/ vapi_cpp_test.cpp
>-L/home/burts/vpp3/build-root/build-vpp-native/vpp/.libs/
>-L/home/burts/vpp3/build-root/build-vpp-native/vpp/vpp-api/vapi/.libs/
>-lvppinfra -lvlibmemoryclient -lsvm -lpthread -lcheck -lsubunit -lrt -lm
>-lvapiclient
>make[2]: Leaving directory '/home/burts/vpp3/test/ext'
>Traceback (most recent call last):
>  File "sanity_import_vpp_papi.py", line 4, in 
>    import vpp_papi
>  File "build/bdist.linux-x86_64/egg/vpp_papi/__init__.py", line 2, in
>
>  File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 22, in
>
>ImportError: No module named vpp_api
>***
>* Sanity check failed, cannot import vpp_papi
>* to debug: 
>* 1. enter test shell:   make test-shell
>* 2. execute debugger:   gdb python -ex 'run sanity_import_vpp_papi.py'
>***
>Makefile:95: recipe for target 'sanity' failed
>make[1]: *** [sanity] Error 1
>make[1]: Leaving directory '/home/burts/vpp3/test'
>Makefile:353: recipe for target 'test' failed
>make: *** [test] Error 2
>burts@burtvb:~/vpp3$ 
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Test failing

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

I'm can't tell ... I just noticed that this seems to be the real
cause for failure but missed you mentioning it earlier..

Sorry for confusion :)

Klement

Quoting Marco Varlese (2017-10-11 13:22:57)
> Klement,
> 
> I did notice it and I pointed that out in my previous email too.
> Do you think that's caused by my patch?
> 
> 
> Cheers,
> Marco
> 
> On Wed, 2017-10-11 at 10:38 +0000, Klement Sekera -X (ksekera - PANTHEON
> TECHNOLOGIES at Cisco) wrote:
> > Did you notice this issue?
> > 
> > 08:54:52 Removing rpm/SRPMS/
> > 08:54:52 Removing rpm/tmp/
> > 08:54:52 make[2]: Leaving directory
> > `/w/workspace/vpp-verify-master-centos7/dpdk'
> > 08:54:52 sudo rpm -Uih vpp-dpdk-devel-17.08-vpp1.x86_64.rpm
> > 08:54:53 
> > 08:54:53  package vpp-dpdk-devel-17.08-vpp2.x86_64 (which is newer
> > than vpp-dpdk-devel-17.08-vpp1.x86_64) is already installed
> > 08:54:53 make[1]: *** [install-rpm] Error 2
> > 08:54:53 make[1]: Leaving directory
> > `/w/workspace/vpp-verify-master-centos7/dpdk'
> > 08:54:53 make: *** [dpdk-install-dev] Error 2
> > 08:54:53 Build step 'Execute shell' marked build as failure
> > 
> > Thanks,
> > Klement
> > 
> > Quoting Marco Varlese (2017-10-11 12:28:48)
> > > Hi Jan,
> > > 
> > > Thanks for pointing to the correct TEST failure.
> > > However, what I'd like to mention is that it's literally impossible to 
> > > get a
> > > build/run through for the past couple of days. 
> > > I am not sure if it's just me being unlucky but none of the "-1 Verified" 
> > > I
> > > get
> > > are due to the patch I submit... :(
> > > Please, if you have 2 spare mins, take a look at this patch 
> > > https://gerrit.f
> > > d.io
> > > /r/#/c/8733/ and see how many "recheck" I have and all failed.
> > > 
> > > 
> > > Cheers,
> > > Marco
> > > 
> > > On Wed, 2017-10-11 at 10:15 +, Jan Gelety -X (jgelety - PANTHEON
> > > TECHNOLOGIES at Cisco) wrote:
> > > > Hello Marco,
> > > > 
> > > > Regarding failure of vpp-csit-verify-virl-master job - the issue is not
> > > > the TC
> > > > mentioned by you below. "TC02: DUT reports packet flow for traffic with
> > > > local
> > > > destination address" is also marked as non-critical as it is expected to
> > > > fail
> > > > now. You can see it in the console log too:
> > > > 
> > > > 08:00:53 TC02: DUT reports packet flow for traffic with local 
> > > > destination
> > > > address :: [Top] TG-DUT1-DUT2-TG. [Cfg] On DUT1 configure I... | FAIL |
> > > > 08:00:53 Traffic script execution failed
> > > > 08:00:53 ---
> > > > 
> > > > -
> > > > 08:01:22 TC03: DUT reports packet flow for traffic with remote 
> > > > destination
> > > > address :: [Top] TG-DUT1-DUT2-TG. [Cfg] On DUT1 configure ... | PASS |
> > > > 08:01:22 ---
> > > > 
> > > > -
> > > > 08:01:22 Tests.Vpp.Func.Telemetry.Eth2P-Ethip6-Ip6Base-Ip6Ipfixbase-Func
> > > > ::
> > > > *IPFIX ipv6 test cases*  | PASS |
> > > > 08:01:22 0 critical tests, 0 passed, 0 failed
> > > > 08:01:22 3 tests total, 2 passed, 1 failed
> > > > 
> > > > => there is no critical test failure (only critical tests can block the
> > > > verification)
> > > > 
> > > > 
> > > > The failed test is " Eth2P-Ethip6-Ip6Base-Ipolicemarkbase-Func .TC01: 
> > > > VPP
> > > > policer 2R3C Color-aware marks packet":
> > > > 
> > > > 08:07:42 TC01: VPP policer 2R3C Color-aware marks packet :: [Top]
> > > > TG=DUT1. 
> > > > [ WARN ] Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ipolicemarkbase-Func -
> > > > TC01:
> > > > VPP policer 2R3C Color-aware marks packet 
> > > > 08:07:42 The VPP PIDs are not equal!
> > > > 08:07:42 Test Setup VPP PIDs: {'10.30.52.214': 9591, '10.30.52.215': 
> > > > 6792}
> > > > 08:07:42 Test Teardown VPP P

Re: [vpp-dev] Test failing

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Did you notice this issue?

08:54:52 Removing rpm/SRPMS/
08:54:52 Removing rpm/tmp/
08:54:52 make[2]: Leaving directory
`/w/workspace/vpp-verify-master-centos7/dpdk'
08:54:52 sudo rpm -Uih vpp-dpdk-devel-17.08-vpp1.x86_64.rpm
08:54:53 
08:54:53package vpp-dpdk-devel-17.08-vpp2.x86_64 (which is newer
than vpp-dpdk-devel-17.08-vpp1.x86_64) is already installed
08:54:53 make[1]: *** [install-rpm] Error 2
08:54:53 make[1]: Leaving directory
`/w/workspace/vpp-verify-master-centos7/dpdk'
08:54:53 make: *** [dpdk-install-dev] Error 2
08:54:53 Build step 'Execute shell' marked build as failure

Thanks,
Klement

Quoting Marco Varlese (2017-10-11 12:28:48)
> Hi Jan,
> 
> Thanks for pointing to the correct TEST failure.
> However, what I'd like to mention is that it's literally impossible to get a
> build/run through for the past couple of days. 
> I am not sure if it's just me being unlucky but none of the "-1 Verified" I 
> get
> are due to the patch I submit... :(
> Please, if you have 2 spare mins, take a look at this patch 
> https://gerrit.fd.io
> /r/#/c/8733/ and see how many "recheck" I have and all failed.
> 
> 
> Cheers,
> Marco
> 
> On Wed, 2017-10-11 at 10:15 +, Jan Gelety -X (jgelety - PANTHEON
> TECHNOLOGIES at Cisco) wrote:
> > Hello Marco,
> > 
> > Regarding failure of vpp-csit-verify-virl-master job - the issue is not the 
> > TC
> > mentioned by you below. "TC02: DUT reports packet flow for traffic with 
> > local
> > destination address" is also marked as non-critical as it is expected to 
> > fail
> > now. You can see it in the console log too:
> > 
> > 08:00:53 TC02: DUT reports packet flow for traffic with local destination
> > address :: [Top] TG-DUT1-DUT2-TG. [Cfg] On DUT1 configure I... | FAIL |
> > 08:00:53 Traffic script execution failed
> > 08:00:53 ---
> > -
> > 08:01:22 TC03: DUT reports packet flow for traffic with remote destination
> > address :: [Top] TG-DUT1-DUT2-TG. [Cfg] On DUT1 configure ... | PASS |
> > 08:01:22 ---
> > -
> > 08:01:22 Tests.Vpp.Func.Telemetry.Eth2P-Ethip6-Ip6Base-Ip6Ipfixbase-Func ::
> > *IPFIX ipv6 test cases*  | PASS |
> > 08:01:22 0 critical tests, 0 passed, 0 failed
> > 08:01:22 3 tests total, 2 passed, 1 failed
> > 
> > => there is no critical test failure (only critical tests can block the
> > verification)
> > 
> > 
> > The failed test is " Eth2P-Ethip6-Ip6Base-Ipolicemarkbase-Func .TC01: VPP
> > policer 2R3C Color-aware marks packet":
> > 
> > 08:07:42 TC01: VPP policer 2R3C Color-aware marks packet :: [Top]
> > TG=DUT1. 
> > [ WARN ] Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ipolicemarkbase-Func - 
> > TC01:
> > VPP policer 2R3C Color-aware marks packet 
> > 08:07:42 The VPP PIDs are not equal!
> > 08:07:42 Test Setup VPP PIDs: {'10.30.52.214': 9591, '10.30.52.215': 6792}
> > 08:07:42 Test Teardown VPP PIDs: {'10.30.52.214': 9863, '10.30.52.215': 
> > 6792}
> > 08:07:42 Tests.Vpp.Func.Ip6.Eth2P-Ethip6-Ip6Base-Ipolicemarkbase-Func - 
> > TC01:
> > VPP policer 2R3C Color-aware marks packet 
> > 08:07:42 The VPP PIDs are not equal!
> > 08:07:42 Test Setup VPP PIDs: {'10.30.52.214': 9591, '10.30.52.215': 6792}
> > 08:07:42 Test Teardown VPP PIDs: {'10.30.52.214': 9863, '10.30.52.215': 
> > 6792}
> > 08:07:42 | FAIL |
> > 08:07:42 Teardown failed:
> > 08:07:42 SSHTimeout: Timeout exception.
> > 08:07:42 Current contents of stdout buffer: 
> > 08:07:42 Current contents of stderr buffer:
> > 
> > It seems that there was VPP restart during execution of this test.
> > 
> > Regards,
> > Jan
> > 
> > -Original Message-
> > From: Marco Varlese [mailto:marco.varl...@suse.com] 
> > Sent: Wednesday, October 11, 2017 10:49
> > To: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) 
> >  > m>; Ed Kern (ejk) 
> > Cc: vpp-dev@lists.fd.io; csit-...@lists.fd.io
> > Subject: Re: [vpp-dev] Test failing
> > 
> > Today I can't get a build +1 Verified.
> > 
> > Different sort of errors and completely random...
> > 
> > https://jenkins.fd.io/job/vpp-verify-master-centos7/7520/console
> > 07:11:39 make[2]: Leaving directory `/w/workspace/vpp-verify-master-
> > centos7/dpdk'
> > 07:11:39 sudo rpm -Uih vpp-dpdk-devel-17.08-vpp1.x86_64.rpm
> > 07:11:39 
> > 07:11:39  package vpp-dpdk-devel-17.08-vpp2.x86_64 (which is newer than
> > vpp-dpdk-devel-17.08-vpp1.x86_64) is already installed
> > 07:11:39 make[1]: *** [install-rpm] Error 2
> > 07:11:39 make[1]: Leaving directory `/w/workspace/vpp-verify-master-
> > centos7/dpdk'
> > 07:11:39 make: *** [dpdk-install-dev] Error 2
> > 07:11:39 Build 

Re: [vpp-dev] Are VAPI ERROR messages normal during build?

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Ole,

there are quite a few more of these errors/warnings. Are you going to
fix them all ?:)

Klement

Quoting Ole Troan (2017-10-11 11:01:37)
> autoreply define pot_profile_add {
>   u32 client_index;
>   u32 context;
>   u8 id;
>   u8 validator;
>   u64 secret_key;
>   u64 secret_share;
>   u64 prime;
>   u8  max_bits;
>   u64 lpc;
>   u64 polynomial_public;
>   u8 list_name_len;
>   u8 list_name[0];
> };
> 
> The last field should have been:
> 
> u8 list_name[list_name_len];
> 
> I believe the Java binding isn't backwards compatible with the [0] either.
> Let me do a round of fixes to these. (Unless any of the IOAM people beat me 
> to it).
> 
> Best regards,
> Ole
> 
> > On 11 Oct 2017, at 08:41, Klement Sekera -X (ksekera - PANTHEON 
> > TECHNOLOGIES at Cisco) <ksek...@cisco.com> wrote:
> > 
> > Hi Burt,
> > 
> > these are existing issues in API files. E.g.
> > 
> >>   ERROR:VAPI C GEN:While parsing message `pot_profile_add': variable length
> >>   array `list_name' doesn't have reference to member containing the actual
> >>   length
> > 
> > means the python script generating new C/C++ API bindings (vapi) was unable
> > to figure out where is the length of `list_name' stored. From the point
> > of view of the python script, this is an error, because a VLA should
> > have a member containing it's length and such messages are ignored.
> > From the point of view of the whole build system, this is not a show
> > stopper because such messages are simply ignored and not included in the
> > the generated C/C++ API bindings.
> > 
> > Maybe changing it to warnings would be better?
> > 
> > Regards,
> > Klement
> > 
> > Quoting Burt Silverman (2017-10-10 20:19:59)
> >>   The build completes without error, but I see a lot of these ERROR:VAPI
> >>   messages, so I thought I'd ask if it is normal. Thanks.
> >>   Burt
> >>   make[5]: Entering directory
> >>   '/home/burts/vpp3/build-root/build-vpp-native/vpp/vpp-api/vapi'
> >> VAPI C GEN tcp.api.json  vapi/tcp.api.vapi.h
> >> VAPI C GEN span.api.json  vapi/span.api.vapi.h
> >> VAPI C GEN l2.api.json  vapi/l2.api.vapi.h
> >> VAPI C GEN cop.api.json  vapi/cop.api.vapi.h
> >> VAPI C GEN pot.api.json  vapi/pot.api.vapi.h
> >> VAPI C GEN ipsec.api.json  vapi/ipsec.api.vapi.h
> >> VAPI C GEN flow.api.json  vapi/flow.api.vapi.h
> >> VAPI C GEN vpe.api.json  vapi/vpe.api.vapi.h
> >> VAPI C GEN nat.api.json  vapi/nat.api.vapi.h
> >> VAPI C GEN mpls.api.json  vapi/mpls.api.vapi.h
> >> VAPI C GEN ip.api.json  vapi/ip.api.vapi.h
> >>   ERROR:VAPI C GEN:While parsing message `pot_profile_add': variable length
> >>   array `list_name' doesn't have reference to member containing the actual
> >>   length
> >>   ERROR:VAPI C GEN:While parsing message `pot_profile_activate': variable
> >>   length array `list_name' doesn't have reference to member containing the
> >>   actual length
> >>   ERROR:VAPI C GEN:While parsing message `pot_profile_del': variable length
> >>   array `list_name' doesn't have reference to member containing the actual
> >>   length
> >> VAPI C GEN memclnt.api.json  vapi/memclnt.api.vapi.h
> > ___
> > vpp-dev mailing list
> > vpp-dev@lists.fd.io
> > https://lists.fd.io/mailman/listinfo/vpp-dev
> 
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Dependency on libsubunit ? (and failing to run tests)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Marco,

as you can see in your review build failure, removing -lsubunit breaks
existing stuff...

Thanks,
Klement

Quoting Marco Varlese (2017-10-10 16:46:13)
> Hi Klement,
> 
> Yes, once check-devel is in place, then we can remove -lsubunit from
> test/ext/Makefile. Things would still work (at least on openSUSE).
> 
> I submitted a patch for review:
> https://gerrit.fd.io/r/#/c/8736/
> 
> 
> Cheers,
> Marco
> 
> On Tue, 2017-10-10 at 14:03 +, Klement Sekera -X (ksekera - PANTHEON
> TECHNOLOGIES at Cisco) wrote:
> > Hi Marco,
> > 
> > this issue is already being investigated by Tom Herbert. Strange thing
> > is that check requires subunit. I wonder if it works if you install
> > check only. Could you try removing -lsubunit from test/ext/Makefile?
> > 
> > Thanks,
> > Klement
> > 
> > Quoting Marco Varlese (2017-10-10 16:00:00)
> > > Hi all,
> > > 
> > > As of last week I could happily run tests (e.g. make test) on my
> > > distribution
> > > (openSUSE).
> > > 
> > > When I tried again today - with latest master - I couldn't anymore:
> > > 
> > > make[2]: Entering directory '/home/mvarlese/repo/vpp/test/ext'
> > > cc -o /home/mvarlese/repo/vpp/build-root/vapi_test/vapi_c_test -std=gnu99 
> > > -g
> > > -Wall -pthread -I/home/mvarlese/repo/vpp/src
> > > -I/home/mvarlese/repo/vpp/build-
> > > root/install-vpp-native//vpp/include -I/home/mvarlese/repo/vpp/build-
> > > root/vapi_test/ vapi_c_test.c -L/home/mvarlese/repo/vpp/build-root/build-
> > > vpp-
> > > native/vpp/.libs/ -L/home/mvarlese/repo/vpp/build-root/build-vpp-
> > > native/vpp/vpp-
> > > api/vapi/.libs/ -lvppinfra -lvlibmemoryclient -lsvm -lpthread -lcheck
> > > -lsubunit
> > > -lrt -lm -lvapiclient
> > > /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld:
> > > cannot
> > > find -lsubunit
> > > collect2: error: ld returned 1 exit status
> > > 
> > > I tried to find the newly introduced dependency on libsubunit but wasn't
> > > successful. 
> > > 
> > > Can anybody shed some light?
> > > Separate, we do not have that package on our distro...
> > > 
> > > 
> > > Cheers,
> > > Marco
> > > 
> > > ___
> > > vpp-dev mailing list
> > > vpp-dev@lists.fd.io
> > > https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Dependency on libsubunit ? (and failing to run tests)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Marco,

this issue is already being investigated by Tom Herbert. Strange thing
is that check requires subunit. I wonder if it works if you install
check only. Could you try removing -lsubunit from test/ext/Makefile?

Thanks,
Klement

Quoting Marco Varlese (2017-10-10 16:00:00)
> Hi all,
> 
> As of last week I could happily run tests (e.g. make test) on my distribution
> (openSUSE).
> 
> When I tried again today - with latest master - I couldn't anymore:
> 
> make[2]: Entering directory '/home/mvarlese/repo/vpp/test/ext'
> cc -o /home/mvarlese/repo/vpp/build-root/vapi_test/vapi_c_test -std=gnu99 -g
> -Wall -pthread -I/home/mvarlese/repo/vpp/src -I/home/mvarlese/repo/vpp/build-
> root/install-vpp-native//vpp/include -I/home/mvarlese/repo/vpp/build-
> root/vapi_test/ vapi_c_test.c -L/home/mvarlese/repo/vpp/build-root/build-vpp-
> native/vpp/.libs/ 
> -L/home/mvarlese/repo/vpp/build-root/build-vpp-native/vpp/vpp-
> api/vapi/.libs/ -lvppinfra -lvlibmemoryclient -lsvm -lpthread -lcheck 
> -lsubunit
> -lrt -lm -lvapiclient
> /usr/lib64/gcc/x86_64-suse-linux/7/../../../../x86_64-suse-linux/bin/ld: 
> cannot
> find -lsubunit
> collect2: error: ld returned 1 exit status
> 
> I tried to find the newly introduced dependency on libsubunit but wasn't
> successful. 
> 
> Can anybody shed some light?
> Separate, we do not have that package on our distro...
> 
> 
> Cheers,
> Marco
> 
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Problem with new c api patch commit 8f2a4ea merged on September 19

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Oh shoot, I never really learned the process..
Looking at the other email, I've already screwed this up on multiple
levels as I did not file a JIRA and committed to master instead of
release throttle.
How do we proceed in these cases?

Quoting Billy McFall (2017-10-05 17:08:50)
>Klement,
>Thank you for the path "drop python3 dependency" on master
>([1]https://gerrit.fd.io/r/#/c/8584/), it will make ours lives downstream
>much easier. I don't see the patch on stable/1710. Is there a plan to
>cherry pick to 1710? (Sorry if it already there, just didn't see it.)
>Thanks,
>Billy McFall
>    On Mon, Sep 25, 2017 at 8:03 AM, Klement Sekera -X (ksekera - PANTHEON
>TECHNOLOGIES at Cisco) <[2]ksek...@cisco.com> wrote:
> 
>  Quoting Thomas F Herbert (2017-09-25 13:31:55)
>  >    On 09/25/2017 05:02 AM, Marco Varlese wrote:
>  >
>  >  Thanks for the thorough explanation Klement!Based on that, I think
>  (2) is still the better option for the current situation...
>  >
>  >  Tom, how would that sound to you?
>  >
>  >
>  >  "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)"
>  [1]<[3]ksek...@cisco.com> 09/25/17 10:40 AM >>>
>  >
>      >  Quoting Marco Varlese (2017-09-25 10:26:50)
>  >
>  >  Hi Klement,
>  >
>  >
>  >  "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)"
>  [2]<[4]ksek...@cisco.com> 09/25/17 9:33 AM >>>
>  >
>  >  At the time of creating this patch, epel was part of Makefile and
>  >  python34 was installed as dependency from that repo.
>  >  (see [3][5]https://gerrit.fd.io/r/#/c/6983/53/Makefile)
>  >  At later time, the epel stuff disappeared and with it also
>  >  the possibility to add python34 as a centos dependency - commit
>  >  bd8e242024fcc2daffa77bdd6e2da1296ace5c69. I remember pointing this
>  out
>  >  in discussion with Neale, but I didn't get a chance to test whether
>  >  centos works or not before it was merged.
>  >
>  >    That's OK. You would have had to test with a system without EPEL
>  Repo
>  >    enabled to discover this problem.
>  >
>  >    I should have paid closer attention when this patch was first
>  discussed in
>  >    May. Please add me as a reviewer for patches that have implications
>  for
>  >    RPM packaging and requirements.
>  >
>  >
>  >  I think it would be nice though to update other scripts too so to
>  have one single python version used across the board.
>  >  Currently, to build VPP on our distribution I need to require both
>  python and python3 packages since some python scripts use one rather
>  than the other.
>  >  Aligning python versions would make downstream consumption better I
>  believe.
>  >  Is this something which will/could be done?
>  >
>  >  See below.
>  >
>  >
>  >
>  >  As for the solution, I can think of 3 options:
>  >
>  >  1.) require python3 (which has been around for some ~9 years now)
>  >  2.) disable generation of the C (and C++) API if python3 is not
>  detected
>  >
>  >  I think this would be a fair compromise for distros not supporting
>  (yet) python3. However, I am not sure how this would result in the VPP
>  CI... wouldn't this break all tests running over those API?
>  >
>  >  Tests are python2.7 because scapy wasn't python3 capable when we
>  >  designed the test framework.
>  >
>  >  These are language bindings, not different API calls. Only test_vapi,
>  >  which tests that the language bindings of different types (simple
>  >  request, dump, event, etc.) work would have to be disabled.
>  >
>  >    I think this is a good interim work-around until the distros have
>  Python3.
>  >    I can submit another patch to allow this the inclusion would be
>  enabled as
>  >    a default but could be disabled only in downstream builds in RHEL
>  and
>  >    Centos 7.
>  >
>  >    When will we have tests in CSIT that use the C/C++ APIs in 17.10?
>  >
>  >    Neale, are we planning to cherry-pick this patch into 17.10?
>  >
>  >    Do we have yet tests in CSIT that use the C/C++ APIs in 17.10?
> 
>  Let me rephrase my words to clarify possible confusion which I see. New
>  C/C++ API bin

Re: [vpp-dev] 回复： 回复： undefined reference to `vapi_register_msg'

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

You can, assuming that your .api file is named test.api:

1.) first you need to invoke vppapigen to generate JSON from your .api
file. For in-tree files, rules in src/suffix-rules.mk apply, so you can
mimick them by doing e.g.

gcc -E -P -C -x c test.api | ./build-root/tools/bin/vppapigen --input -
--json test.json

2.) now you need to generate vapi bindings based on the json:

src/vpp-api/vapi/vapi_c_gen.py test.json

This will then generate test.vapi.h, which you can #include in your
source code. The header file will also contain a macro named
DEFINE_VAPI_MSG_IDS_TEST_JSON

 this is based on the name of the API

You need to specify this macro *once* in one of your source files (e.g.
the test.c you mention in your email)

Regards,
Klement

Quoting 重新开始 (2017-10-03 11:42:05)
>Thank you! Klement.
>Yes, I am not good at makeflie. After init study, i make success. But i
>have a problem. I hope that you can answer me. If i write a plugin for vpp
>and define some apis. The question si : could I call the apis through vapi
>method?
>-- 原始邮件 --
>发件人: "Klement Sekera -X (ksekera - PA";;
>发送时间: 2017年10月3日(星期二) 下午5:01
>收件人: "重新开始"<15803846...@qq.com>;"vpp-dev";
>主题: Re: 回复： undefined reference to `vapi_register_msg'
>I don't see them in the compile command
> 
>>   cc -std=gnu99 -g -Wall -pthread -I/home/vpp/vpp/src
>>  -Ihome//vpp/vpp/build-root/install-vpp-native/vpp/include
>>  -I/home/test
>> test.c   -o test
> 
>I strongly suggest you learn how gcc & make works...
> 
>Quoting 重新开始 (2017-10-03 10:38:19)
>>Hi,
>>link libraries are same with test/ext/Makefile binaries. any other
>>libraries needed?
>>-- 原始邮件 --
>>发件人: "Klement Sekera -X (ksekera - PA";;
>>发送时间: 2017年10月3日(星期二) 下午4:30
>>收件人: "重新开
>始"<15803846...@qq.com>;"vpp-dev";
>>主题: Re: undefined reference to `vapi_register_msg'
>>Hi,
>>
>>link against the same libraries as test/ext/Makefile binaries do:
>>
>>-lvppinfra -lvlibmemoryclient -lsvm -lvapiclient
>>
>>Regards,
>>Klement
>>
>>Quoting 重新开始 (2017-10-03 10:17:24)
>>>HI, i make the vapi test program, and make output the flowing
>errors.
>>>Maybe, any *.so should i add?
>>>vpp@vpp-VirtualBox:~/test$ make
>>>cc -std=gnu99 -g -Wall -pthread -I/home/vpp/vpp/src
>>>-Ihome//vpp/vpp/build-root/install-vpp-native/vpp/include
>>-I/home/test
>>>test.c   -o test
>>>/tmp/ccq6LqEu.o: In function
>`__vapi_constructor_delete_subif_reply':
>>>/usr/include/vapi/vpe.api.vapi.h:6548: undefined reference to
>>>`vapi_register_msg'
>>>/tmp/ccq6LqEu.o: In function `__vapi_constructor_ip6_nd_event':
>>>/usr/include/vapi/vpe.api.vapi.h:6570: undefined reference to
>>>`vapi_register_msg'
>>>/tmp/ccq6LqEu.o: In function
>>>`__vapi_constructor_pg_create_interface_reply':
>>>/usr/include/vapi/vpe.api.vapi.h:6592: undefined reference to
>>>`vapi_register_msg'
>>>/tmp/ccq6LqEu.o: In function
>`__vapi_constructor_pg_capture_reply':
>>>/usr/include/vapi/vpe.api.vapi.h:6614: undefined reference to
>>>`vapi_register_msg'
>>>/tmp/ccq6LqEu.o: In function
>>>`__vapi_constructor_sw_interface_set_vpath_reply':
>>>/usr/include/vapi/vpe.api.vapi.h:6636: undefined reference to
>>>`vapi_register_msg'
>>>/tmp/ccq6LqEu.o:/usr/include/vapi/vpe.api.vapi.h:6658: more
>undefined
>>>references to `vapi_register_msg' follow
>>>/tmp/ccq6LqEu.o: In function `test_connect':
>>>/home/vpp/test/test.c:46: undefined reference to
>`vapi_ctx_alloc'
>>>/home/vpp/test/test.c:48: undefined reference to `vapi_connect'
>>>/home/vpp/test/test.c:51: undefined reference to
>`vapi_disconnect'
>>>/home/vpp/test/test.c:53: undefined reference to `vapi_ctx_free'
>>>collect2: error: ld returned 1 exit status
>>>: recipe for target 'test' failed
>>>make: *** [test] Error 1
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] 回复： undefined reference to `vapi_register_msg'

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

I don't see them in the compile command

>   cc -std=gnu99 -g -Wall -pthread -I/home/vpp/vpp/src
>  -Ihome//vpp/vpp/build-root/install-vpp-native/vpp/include
>  -I/home/test
> test.c   -o test

I strongly suggest you learn how gcc & make works...

Quoting 重新开始 (2017-10-03 10:38:19)
>Hi,
>link libraries are same with test/ext/Makefile binaries. any other
>libraries needed?
>-- 原始邮件 --
>发件人: "Klement Sekera -X (ksekera - PA";;
>发送时间: 2017年10月3日(星期二) 下午4:30
>收件人: "重新开始"<15803846...@qq.com>;"vpp-dev";
>主题: Re: undefined reference to `vapi_register_msg'
>Hi,
> 
>link against the same libraries as test/ext/Makefile binaries do:
> 
>-lvppinfra -lvlibmemoryclient -lsvm -lvapiclient
> 
>Regards,
>Klement
> 
>Quoting 重新开始 (2017-10-03 10:17:24)
>>HI, i make the vapi test program, and make output the flowing errors.
>>Maybe, any *.so should i add?
>>vpp@vpp-VirtualBox:~/test$ make
>>cc -std=gnu99 -g -Wall -pthread -I/home/vpp/vpp/src
>>-Ihome//vpp/vpp/build-root/install-vpp-native/vpp/include
>-I/home/test
>>test.c   -o test
>>/tmp/ccq6LqEu.o: In function `__vapi_constructor_delete_subif_reply':
>>/usr/include/vapi/vpe.api.vapi.h:6548: undefined reference to
>>`vapi_register_msg'
>>/tmp/ccq6LqEu.o: In function `__vapi_constructor_ip6_nd_event':
>>/usr/include/vapi/vpe.api.vapi.h:6570: undefined reference to
>>`vapi_register_msg'
>>/tmp/ccq6LqEu.o: In function
>>`__vapi_constructor_pg_create_interface_reply':
>>/usr/include/vapi/vpe.api.vapi.h:6592: undefined reference to
>>`vapi_register_msg'
>>/tmp/ccq6LqEu.o: In function `__vapi_constructor_pg_capture_reply':
>>/usr/include/vapi/vpe.api.vapi.h:6614: undefined reference to
>>`vapi_register_msg'
>>/tmp/ccq6LqEu.o: In function
>>`__vapi_constructor_sw_interface_set_vpath_reply':
>>/usr/include/vapi/vpe.api.vapi.h:6636: undefined reference to
>>`vapi_register_msg'
>>/tmp/ccq6LqEu.o:/usr/include/vapi/vpe.api.vapi.h:6658: more undefined
>>references to `vapi_register_msg' follow
>>/tmp/ccq6LqEu.o: In function `test_connect':
>>/home/vpp/test/test.c:46: undefined reference to `vapi_ctx_alloc'
>>/home/vpp/test/test.c:48: undefined reference to `vapi_connect'
>>/home/vpp/test/test.c:51: undefined reference to `vapi_disconnect'
>>/home/vpp/test/test.c:53: undefined reference to `vapi_ctx_free'
>>collect2: error: ld returned 1 exit status
>>: recipe for target 'test' failed
>>make: *** [test] Error 1
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] undefined reference to `vapi_register_msg'

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

link against the same libraries as test/ext/Makefile binaries do:

-lvppinfra -lvlibmemoryclient -lsvm -lvapiclient

Regards,
Klement

Quoting 重新开始 (2017-10-03 10:17:24)
>HI, i make the vapi test program, and make output the flowing errors.
>Maybe, any *.so should i add?
>vpp@vpp-VirtualBox:~/test$ make
>cc -std=gnu99 -g -Wall -pthread -I/home/vpp/vpp/src
>-Ihome//vpp/vpp/build-root/install-vpp-native/vpp/include -I/home/test
>test.c   -o test
>/tmp/ccq6LqEu.o: In function `__vapi_constructor_delete_subif_reply':
>/usr/include/vapi/vpe.api.vapi.h:6548: undefined reference to
>`vapi_register_msg'
>/tmp/ccq6LqEu.o: In function `__vapi_constructor_ip6_nd_event':
>/usr/include/vapi/vpe.api.vapi.h:6570: undefined reference to
>`vapi_register_msg'
>/tmp/ccq6LqEu.o: In function
>`__vapi_constructor_pg_create_interface_reply':
>/usr/include/vapi/vpe.api.vapi.h:6592: undefined reference to
>`vapi_register_msg'
>/tmp/ccq6LqEu.o: In function `__vapi_constructor_pg_capture_reply':
>/usr/include/vapi/vpe.api.vapi.h:6614: undefined reference to
>`vapi_register_msg'
>/tmp/ccq6LqEu.o: In function
>`__vapi_constructor_sw_interface_set_vpath_reply':
>/usr/include/vapi/vpe.api.vapi.h:6636: undefined reference to
>`vapi_register_msg'
>/tmp/ccq6LqEu.o:/usr/include/vapi/vpe.api.vapi.h:6658: more undefined
>references to `vapi_register_msg' follow
>/tmp/ccq6LqEu.o: In function `test_connect':
>/home/vpp/test/test.c:46: undefined reference to `vapi_ctx_alloc'
>/home/vpp/test/test.c:48: undefined reference to `vapi_connect'
>/home/vpp/test/test.c:51: undefined reference to `vapi_disconnect'
>/home/vpp/test/test.c:53: undefined reference to `vapi_ctx_free'
>collect2: error: ld returned 1 exit status
>: recipe for target 'test' failed
>make: *** [test] Error 1
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] unit test framework timeout

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

No need for such crude approach ;-)

ksekera@zglab-host-4 ~/vpp> make test-shell
...
(virtualenv) ksekera@zglab-host-4:~/vpp/test$ ./discover_tests.py
...
test_jvpp.py.TestJVpp.test_vpp_snat_callback_api
test_jvpp.py.TestJVpp.test_vpp_snat_future_api
test_dhcp.py.TestDHCP.test_dhcp6_proxy
test_dhcp.py.TestDHCP.test_dhcp_client
test_dhcp.py.TestDHCP.test_dhcp_proxy
(virtualenv) ksekera@zglab-host-4:~/vpp/test$

by changing "print_callback" in discover_tests.py, one can customize the
format or the action to be done for each test case (default being printing 
the test as show above).

Regards,
Klement

Quoting Dave Wallace (2017-10-02 17:31:28)
>Brian,
> 
>A brute-force means of achieving your goal would be to use the "TEST="
>option to run each test individually:
> 
> TEST=    - filter the set of tests:
>    by file-name  - only run tests from specified file, e.g.
>TEST=test_bfd selects all tests from test_bfd.py
>    by file-suffix    - same as file-name, but 'test_' is omitted e.g.
>TEST=bfd selects all tests from test_bfd.py
>    by wildcard   - wildcard filter is ..,
>each can be replaced by '*'
>    e.g. TEST='test_bfd.*.*' is equivalent to above
>example of filter by file-name
> TEST='bfd.*.*' is equivalent to above example
>of filter by file-suffix
> TEST='bfd.BFDAPITestCase.*' selects all tests
>from test_bfd.py which are part of BFDAPITestCase class
> TEST='bfd.BFDAPITestCase.test_add_bfd'
>selects a single test named test_add_bfd from test_bfd.py/BFDAPITestCase
> TEST='*.*.test_add_bfd' selects all test
>functions named test_add_bfd from all files/classes
> 
>With a bit of bash scripting, this turns out to be quite easy.  Here's a
>one-liner which executes all of the l2xc tests in
>.../vpp/test/test_l2xc.py:
> 
>for test in $(grep "def test_" test/test_l2xc.py | awk -e '{print $2}' |
>    cut -d'(' -f1) ; do make TEST="*.*.$test" test ; done
> 
>Thanks,
>-daw-
>On 10/02/2017 03:44 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES
>at Cisco) wrote:
> 
>  Hi Brian,
> 
>  FAILFAST=0 is the default, so there is no need to specify it.
> 
>  Usually, when a test case 'runs forever', it means that the VPP crashed
>  while processing an API call (did you see a message about a core found
>  in temporary directory?), in which case python will be stuck waiting for
>  a lock. Now, the problem is that there is no way to cancel that stuck
>  thread in Python (which is for most cases the main thread).
> 
>  So to work around this issue, the following approach is taken: First
>  thing the test framework does is fork - child continues running
>  the tests while parent monitors the child. There is a pipe open,
>  through which child sends keep-alive objects containing the name of
>  the test case, its temporary directory and some other info whenever
>  setUp() runs. That way parent can tell that the child is progressing
>  through the tests and also can say which test caused timeout.
> 
>  I spent quite some time trying to make it work so that the tests can
>  continue, but unless we refactor the code so that each test runs in its
>  own forked process, we won't be able to achieve your goal. So far, 100%
>  of these timeouts were caused by VPP dumping a core and it seemed to me
>  like a waste of time spending extra time to make the tests finish.
> 
>  Thanks,
>  Klement
> 
>  Quoting Brian Brooks (2017-09-22 19:07:53)
> 
>  Is there a way to make:
> 
>FAILFAST=0 TIMEOUT=t make test
> 
>  actually continue on to the next test if a test has reached timeout t?
> 
>  The motivation is that I see at least one test case running for forever,
>  and I want to be able to see how many test cases are failing in total.
> 
>  I didn't see that this was possible to do out of the box, so I tried one
>  approach: launch a thread that waits until the timeout has passed and then
>  send a term signal to the vpp process. If the test case completes before
>  timeout, the test case's "tearDown" routine will cancel the timeout thread.
>  The timeout and killing part works, except the test framework still does
>  not continue on to the next test case.
> 
>  Thoughts?
> 
>  Thanks,
>  Brian
>  ___
>  vpp-dev mailing list
>  [1]vpp-dev@lists.fd.io
>  [2]https://lists.fd.io/mailman/listinfo/vpp-dev
> 
>  ___
>  vpp-dev mailing list
>  [3]vpp-dev@lists.fd.io
>  [4]htt

Re: [vpp-dev] 回复： vapi test problem

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

The message produced by make does not correspond to the Makefile you attached.
I would suggest learning how `make' works first...

Quoting 重新开始 (2017-10-02 10:46:30)
>I have copy the .so files to /usr/lib/vpp. And in the makefile, i modified
>the path. THe output is same:
>make: *** No rule to make target>
>'/vpp/vpp-api/vapi/.libs/libvapiclient.so', needed by '/vapi_test/test'.
>-- 原始邮件 --
>发件人: "Klement Sekera -X (ksekera - PA";;
>发送时间: 2017年10月2日(星期一) 下午3:57
>收件人: "重新开始"<15803846...@qq.com>;"vpp-dev";
>主题: Re: [vpp-dev] vapi test problem
>Hi 重新开始,
> 
>taking Makefile out of vpp source tree and using it on its own is
>almost certainly not going to work.
> 
>You need to install the libvapiclient.so (and others) library
>and/or adapt linker flags to point inside your source tree so that the
>linker can find it.
> 
>This is not a VPP question, rather a compiler/linker/Makefile issue.
> 
>Regards,
>Klement
> 
>Quoting 重新开始 (2017-10-01 11:49:39)
>>Hi,
>>   I install vpp 18.01 on ubuntu 16.04, and test vapi. But make
>output:
>>make: *** No rule to make target
>>'/vpp/vpp-api/vapi/.libs/libvapiclient.so', needed by
>'/vapi_test/test'.
>>Stop.
>>Anyone met this problem?
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] vapi test problem

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi 重新开始,

taking Makefile out of vpp source tree and using it on its own is
almost certainly not going to work.

You need to install the libvapiclient.so (and others) library
and/or adapt linker flags to point inside your source tree so that the
linker can find it.

This is not a VPP question, rather a compiler/linker/Makefile issue.

Regards,
Klement

Quoting 重新开始 (2017-10-01 11:49:39)
>Hi,
>   I install vpp 18.01 on ubuntu 16.04, and test vapi. But make output:
>make: *** No rule to make target
>'/vpp/vpp-api/vapi/.libs/libvapiclient.so', needed by '/vapi_test/test'.
>Stop.
>Anyone met this problem?
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] unit test framework timeout

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Brian,

FAILFAST=0 is the default, so there is no need to specify it.

Usually, when a test case 'runs forever', it means that the VPP crashed
while processing an API call (did you see a message about a core found
in temporary directory?), in which case python will be stuck waiting for
a lock. Now, the problem is that there is no way to cancel that stuck
thread in Python (which is for most cases the main thread).

So to work around this issue, the following approach is taken: First
thing the test framework does is fork - child continues running
the tests while parent monitors the child. There is a pipe open,
through which child sends keep-alive objects containing the name of
the test case, its temporary directory and some other info whenever
setUp() runs. That way parent can tell that the child is progressing
through the tests and also can say which test caused timeout.

I spent quite some time trying to make it work so that the tests can
continue, but unless we refactor the code so that each test runs in its
own forked process, we won't be able to achieve your goal. So far, 100%
of these timeouts were caused by VPP dumping a core and it seemed to me
like a waste of time spending extra time to make the tests finish.

Thanks,
Klement

Quoting Brian Brooks (2017-09-22 19:07:53)
> Is there a way to make:
> 
>   FAILFAST=0 TIMEOUT=t make test
> 
> actually continue on to the next test if a test has reached timeout t?
> 
> The motivation is that I see at least one test case running for forever,
> and I want to be able to see how many test cases are failing in total.
> 
> I didn't see that this was possible to do out of the box, so I tried one
> approach: launch a thread that waits until the timeout has passed and then
> send a term signal to the vpp process. If the test case completes before
> timeout, the test case's "tearDown" routine will cancel the timeout thread.
> The timeout and killing part works, except the test framework still does
> not continue on to the next test case.
> 
> Thoughts?
> 
> Thanks,
> Brian
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] 回复： 回复： How to get interface stats using C api?

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

There are new C and C++ APIs added recently to vpp source code along
with test programs. You can take a look at test/ext/vapi_c_test.c and
vapi_cpp_test.cpp. Functions receiving stats are named test_stats_XXX().
These are there to verify that receiving events works. Functionally,
these are the same, they just test different ways of doing stuff.

At shared memory level it looks like this:

1.) client -> want_stats -> vpp
2.) vpp -> want_stats_reply -> client (confirms subscription)
3.) vpp ->  -> client (when timer fires)

in step 3.), client needs to read the shared memory queue in the
vpp->client direction (or use the appropriate XXX_read() function). Note
that stats take a few seconds to arrive (the test programs require
10-15 seconds of runtime to get some stats).

HTH,
Klement

Quoting 重新开始 (2017-09-27 04:55:29)
>That is ok！
> Now I am research calling the plugin apis by C. Especially how to call
>the event apis. I would appreciate it if you can write a brief client.
>-- 原始邮件 --
>发件人: "Keith Burns";;
>发送时间: 2017年9月27日(星期三) 上午10:47
>收件人: "重新开始"<15803846...@qq.com>;
>抄送: "vpp-dev";
>主题: Re: 回复： [vpp-dev] How to get interface stats using C api?
>Apologies,
>It's a callback.
>We probably need some decent literature written for how to consume the
>various C, C++, Python and Lua APIs
>If you have a preference for any of those languages I could write a brief
>client program for you and start work on documenting this stuff.
>On Sep 26, 2017 6:04 PM, "重新开始" <[1]15803846...@qq.com> wrote:
> 
>  Hi，
>Thank you for your detail information. After reading this, i can
>  understand. But after calling  "want_per_interface_simple_stats" with
>  enable_disable=1, num=1 and sw_ifs[4], i do not know how to receive this
>  stats. calling other function or variable? Can you tell me how to
>  receive stats in my c program?
> Thanks！
>  -- 原始邮件 --
>  发件人: "Keith Burns";<[2]alaga...@gmail.com>;
>  发送时间: 2017年9月27日(星期三) 上午6:39
>  收件人: "重新开
>  始"<[3]15803846...@qq.com>;"vpp-dev"<[4]vpp-dev@lists.fd.io>;
>  主题: Re: [vpp-dev] How to get interface stats using C api?
>  Hi there,
>  As of 17.10 you can now register for statistics on a per interface
>  level.
>  You need the sw_if_index of the interface you want stats for, but the
>  calls are in
>  [5]https://git.fd.io/vpp/tree/src/vpp/stats/stats.api
>  but as an example:
>  Register:
> 
>  autoreply define want_per_interface_simple_stats
>  {
>u32 client_index;
>u32 context;
>u32 enable_disable;
>u32 pid;
>u32 num;
>u32 sw_ifs[num];
> 
>  };
> 
>  So calling "want_per_interface_simple_stats" with enable_disable=1, num=1 
> and sw_ifs[4] will give you
> 
>  interface_simple_stats every 10sec for sw_if_index=4.
> 
>  The content of "interface_simple_stats" looks like:
> 
>  /** \brief Simple per interface stats counters structure
>  @param count - number of elements in message
>  @param timestamp - u32 vlib timestamp for control plane
>  @param data[count] - vl_api_vnet_simple_counter_t
> 
>  */
>  manual_print manual_endian define vnet_per_interface_simple_counters
>  {
>u32 count;
>u32 timestamp;
>vl_api_vnet_simple_counter_t data[count];
>  };
> 
>  Where each "vnet_simple_counter_t" looks like:
> 
>  [6]https://git.fd.io/vpp/tree/src/vnet/interface.api
> 
>  /** \brief Simple interface counter data type for 
> vnet_interface_simple_counters
>  @param sw_if_index - interface indexes for counters
>  @param drop - RX or TX drops due to buffer starvation
>  @param punt - used with VNET "punt" disposition
>  @param rx_ip4 - received IP4 packets
>  @param rx_ip6 - received IP6 packets
>  @param rx_no_buffer - no RX buffers available
>  @param rx_miss - receive misses
>  @param rx_error - receive errors
>  @param tx_error - transmit errors
>  @param rx_mpls - received MPLS packet
> 
>  */
>  typeonly manual_print manual_endian define vnet_simple_counter
>  {
>u32 sw_if_index;
>u64 drop;
>u64 punt;
>u64 rx_ip4;
>u64 rx_ip6;
>u64 rx_no_buffer;
>u64 rx_miss;
>u64 rx_error;
>u64 tx_error;
>u64 rx_mpls;
>  };
> 
>  So you will get back an array of counter structs for each sw_if_index you 
> registered for.
> 
>  At the moment the impl sends on sw_if_index per reply but that is going to 
> change so that for each client we send as much in one message as possible.
> 
>  On Tue, Sep 26, 2017 at 4:50 AM 重新开始 <[7]15803846...@qq.com> wrote:
> 
>  Hi,  everyone!
>  How to get interface stats using C api?
>  For example, I want to know "GigabitEthernete/0/1" rx packets , what should 
> I do?
> 
>  Thanks
> 
>___
>vpp-dev mailing list
>

Re: [vpp-dev] Problem with new c api patch commit 8f2a4ea merged on September 19

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Quoting Thomas F Herbert (2017-09-25 13:31:55)
>On 09/25/2017 05:02 AM, Marco Varlese wrote:
> 
>  Thanks for the thorough explanation Klement!Based on that, I think (2) is 
> still the better option for the current situation...
> 
>  Tom, how would that sound to you?
> 
> 
>  "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)" 
> [1]<ksek...@cisco.com> 09/25/17 10:40 AM >>>
> 
>  Quoting Marco Varlese (2017-09-25 10:26:50)
> 
>  Hi Klement,
> 
> 
>  "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)" 
> [2]<ksek...@cisco.com> 09/25/17 9:33 AM >>>
> 
>  At the time of creating this patch, epel was part of Makefile and
>  python34 was installed as dependency from that repo.
>  (see [3]https://gerrit.fd.io/r/#/c/6983/53/Makefile)
>  At later time, the epel stuff disappeared and with it also
>  the possibility to add python34 as a centos dependency - commit
>  bd8e242024fcc2daffa77bdd6e2da1296ace5c69. I remember pointing this out
>  in discussion with Neale, but I didn't get a chance to test whether
>  centos works or not before it was merged.
> 
>That's OK. You would have had to test with a system without EPEL Repo
>enabled to discover this problem.
> 
>I should have paid closer attention when this patch was first discussed in
>May. Please add me as a reviewer for patches that have implications for
>RPM packaging and requirements.
> 
> 
>  I think it would be nice though to update other scripts too so to have one 
> single python version used across the board.
>  Currently, to build VPP on our distribution I need to require both python 
> and python3 packages since some python scripts use one rather than the other.
>  Aligning python versions would make downstream consumption better I believe.
>  Is this something which will/could be done?
> 
>  See below.
> 
> 
> 
>  As for the solution, I can think of 3 options:
> 
>  1.) require python3 (which has been around for some ~9 years now)
>  2.) disable generation of the C (and C++) API if python3 is not detected
> 
>  I think this would be a fair compromise for distros not supporting (yet) 
> python3. However, I am not sure how this would result in the VPP CI... 
> wouldn't this break all tests running over those API?
> 
>  Tests are python2.7 because scapy wasn't python3 capable when we
>  designed the test framework.
> 
>  These are language bindings, not different API calls. Only test_vapi,
>  which tests that the language bindings of different types (simple
>  request, dump, event, etc.) work would have to be disabled.
> 
>I think this is a good interim work-around until the distros have Python3.
>I can submit another patch to allow this the inclusion would be enabled as
>a default but could be disabled only in downstream builds in RHEL and
>Centos 7.
> 
>When will we have tests in CSIT that use the C/C++ APIs in 17.10?
> 
>Neale, are we planning to cherry-pick this patch into 17.10?
> 
>Do we have yet tests in CSIT that use the C/C++ APIs in 17.10?

Let me rephrase my words to clarify possible confusion which I see. New
C/C++ API bindings are just a way of calling old VPP APIs in a more
user-friendly way. They still call e.g. sw_interface_dump - there is no
change to APIs themselves.

So it's important to distinguish between API (e.g. sw_interface_dump)
and API binding (which is a language-specific way of constructing
sw_interface_dump message and sending it over shared memory to VPP).

I don't know about CSIT, but "make test" tests use python API bindings,
which internally uses old C API bindings (vapiclient). Unless we want
to remove old C API, there is little motivation to rework python API
binding as it contains more or less the same logic as the new C/C++ API
bindings.

There is one test though which doesn't - test_vapi.py, which builds its
own test client binary for both new C and new C++ API to make sure that
it builds, links and executes as intended. It doesn't call python API
bindings at all, it only sets up the infra for running a test and then
spawns a binary, which does the actual testing (vs running native python
code as is the case for most of the other tests).

Now to answer your questions:

1.) there is no CSIT test which uses new C/C++ API
2.) there is only one "make test" test which uses new C/C++ API
(test_vapi), which is part of this change.

To disable this partially on centos, we need to exclude

a.) vpp-api/vapi/Makefile.am (don't build)
b.) test/ext/Makefile (don't build)
c.) test/test_vapi.py (don't run) - we can add code to skip this easily
if we can detect centos e.g. via some kind of environment variable from
within python..

> 
> 
> 
>  3.) convert the scrip

Re: [vpp-dev] Problem with new c api patch commit 8f2a4ea merged on September 19

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Quoting Marco Varlese (2017-09-25 10:26:50)
> Hi Klement,
> 
> >>> "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)" 
> >>> <ksek...@cisco.com> 09/25/17 9:33 AM >>>
> > At the time of creating this patch, epel was part of Makefile and
> > python34 was installed as dependency from that repo.
> > (see https://gerrit.fd.io/r/#/c/6983/53/Makefile)
> > At later time, the epel stuff disappeared and with it also
> > the possibility to add python34 as a centos dependency - commit
> > bd8e242024fcc2daffa77bdd6e2da1296ace5c69. I remember pointing this out
> > in discussion with Neale, but I didn't get a chance to test whether
> > centos works or not before it was merged.
> 
> I think it would be nice though to update other scripts too so to have one 
> single python version used across the board. 
> Currently, to build VPP on our distribution I need to require both python and 
> python3 packages since some python scripts use one rather than the other.
> Aligning python versions would make downstream consumption better I believe.
> Is this something which will/could be done?

See below.

> 
> > As for the solution, I can think of 3 options:
> > 
> > 1.) require python3 (which has been around for some ~9 years now)
> > 2.) disable generation of the C (and C++) API if python3 is not detected
> 
> I think this would be a fair compromise for distros not supporting (yet) 
> python3. However, I am not sure how this would result in the VPP CI... 
> wouldn't this break all tests running over those API?

Tests are python2.7 because scapy wasn't python3 capable when we
designed the test framework.

These are language bindings, not different API calls. Only test_vapi,
which tests that the language bindings of different types (simple
request, dump, event, etc.) work would have to be disabled.

> > 3.) convert the script to python2.7 (which is in the opposite direction of
> > where we would want to go wrt python version)
> > 
> > Thanks,
> > Klement
> 
> Cheers,
> Marco
> 
> Quoting Thomas F Herbert (2017-09-23 15:55:10)
> >All:
> > 
> >Commit 8f2a4ea, Gerrit,  [1]https://gerrit.fd.io/r/#/c/6983/ "Add new C
> >API"
> > 
> >introduces a dependency on Python 3 and breaks downstream builds for
> >Centos.
> > 
> >Unfortunately, neither RHEL nor Centos currently support Python 3.
> > 
> >Most VPPers are probably building with EPEL repo so this problem didn't
> >show up until now but actually there is no dependency on EPEL in the
> >Makefile or spec file.
> > 
> >If anybody can suggest a solution short of pushing Python 3 into the
> >downstream repos, I am open to suggestions.
> > 
> >--Tom
> > 
> >--
> >Thomas F Herbert
> >NFV and Fast Data Planes
> >Office of Technology
> >Red Hat
> > 
> > References
> > 
> >Visible links
> >1. https://gerrit.fd.io/r/#/c/6983/
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
> 
> 
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Problem with new c api patch commit 8f2a4ea merged on September 19

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

At the time of creating this patch, epel was part of Makefile and
python34 was installed as dependency from that repo.
(see https://gerrit.fd.io/r/#/c/6983/53/Makefile)
At later time, the epel stuff disappeared and with it also
the possibility to add python34 as a centos dependency - commit
bd8e242024fcc2daffa77bdd6e2da1296ace5c69. I remember pointing this out
in discussion with Neale, but I didn't get a chance to test whether
centos works or not before it was merged.

As for the solution, I can think of 3 options:

1.) require python3 (which has been around for some ~9 years now)
2.) disable generation of the C (and C++) API if python3 is not detected
3.) convert the script to python2.7 (which is in the opposite direction of
where we would want to go wrt python version)

Thanks,
Klement

Quoting Thomas F Herbert (2017-09-23 15:55:10)
>All:
> 
>Commit 8f2a4ea, Gerrit,  [1]https://gerrit.fd.io/r/#/c/6983/ "Add new C
>API"
> 
>introduces a dependency on Python 3 and breaks downstream builds for
>Centos.
> 
>Unfortunately, neither RHEL nor Centos currently support Python 3.
> 
>Most VPPers are probably building with EPEL repo so this problem didn't
>show up until now but actually there is no dependency on EPEL in the
>Makefile or spec file.
> 
>If anybody can suggest a solution short of pushing Python 3 into the
>downstream repos, I am open to suggestions.
> 
>--Tom
> 
>--
>Thomas F Herbert
>NFV and Fast Data Planes
>Office of Technology
>Red Hat
> 
> References
> 
>Visible links
>1. https://gerrit.fd.io/r/#/c/6983/
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] make test python segfault in ubuntu 16.04

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Dave,

the tests are not run in parallel. For each testcase  *class* a new vpp is
started and all the tests which are members of that class are run
against that vpp. Then the vpp is killed and another class is picked. We
use a helper thread to "pump" stdout & stderr from vpp to the logger,
motivation being - have it time-synchronized as much as possible with
the other logger messages. Afaik there is nothing which blocks the python
interpreter from reaping the thread.

Regarding timeouts - the first thing the framework does is setup a few
communication pipes and fork. Child sends periodic keepalives
by writing a tuple describing test class running, it's temporary
directory etc. to the pipe. If sufficient time passes without
activity on the pipe, the child is considered stuck, killed and the
messages which you describe appear. This usually happens if vpp
coredumps mid-api call, in which case the wait for shared memory
condition will never finish (as there is no vpp to signal that condition).

Regards,
Klement

Quoting Dave Wallace (2017-08-25 19:56:13)
>vpp-dev, Florin,
> 
>Below is an analysis of the all of the failures that this patch
>encountered before finally passing. None of the failures were related in
>any way to the code changes in the patch.
> 
>In summary, there appear to be a number of different factors involved with
>these failures.
> 
>  • Two failures appear to be caused by the run-time environment.
>  • An intermittent bug appears to exist in `L2BD Multi-instance test 5 -
>delete 5 BDs'
>  • The segfault shows lots of threads being run.  Are tests being
>executed in parallel?  If so, it would be interesting to serialize the
>tests to see if that fixes any of these issues.
> 
>I'm also seeing a variation in the order that the "make tests" are run (or
>at least in the order of the status reports).  My understanding of the
>'make test' python infrastructure is insufficient to make an intelligent
>guess as to whether this has any bearing on any of these failures.
> 
>I get more predictable result output when running make test locally on my
>own server, but the order of test output is different than in the CI test
>runs.  Locally, the order of tests appears to be the same between
>different runs of 'make test'.  I have also not seen any of these errors
>on my server which is running Ubuntu 17.04, although I have not done an
>endurance test either.
> 
>My recommendation based on this analysis is as follows:
>  1. The L2BD unit test issue be investigated by the appropriate 'make
>test' experts
>  2. vpp-verify-master-centos7, vpp-verify-master-ubuntu1604, and
>vpp-test-debug-master-ubuntu1604 jobs should be run operationally in the
>Container PoC environment with the rest of the jjb jobs run in the cloud
>infra.
> 
>Thanks,
>-daw-
> 
> %< 
>[ From [1]https://gerrit.fd.io/r/#/c/8133 ]
> 
>=> Container PoC Aug 24 8:36 PM  Patch Set 9:  Build Successful
>[2]http://jenkins.ejkern.net:8080/job/vpp-docs-verify-master/1515/ :
>SUCCESS
>
> [3]http://jenkins.ejkern.net:8080/job/vpp-make-test-docs-verify-master/1512/
>: SUCCESS
>[4]http://jenkins.ejkern.net:8080/job/vpp-verify-master-centos7/1983/ :
>SUCCESS
>
> [5]http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1301/
>: SUCCESS
>[6]http://jenkins.ejkern.net:8080/job/vpp-verify-master-ubuntu1604/2022/ :
>SUCCESS
>[7]http://jenkins.ejkern.net:8080/job/vpp-fake-csit-verify-master/1695/ :
>SUCCESS
> 
>=> fd.io JJB  Aug 24 9:19 PM  Patch Set 9:  Verified-1  Build Failed
>[8]https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/6775/ : FAILURE
>Logs:
>
> [9]https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1604/6775
> 
>  Failure Signature:
>    01:08:59  verify templates on IP6 datapath  Fatal Python error:
>  Segmentation fault
> 
>  Comment:
>    Python bug or resource starvation?  Lots of threads running...
>    Possibly due to bad environment/sick minion.
> 
>[10]https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/3098/ :
>SUCCESS
>[11]https://jenkins.fd.io/job/vpp-verify-master-centos7/6770/ : SUCCESS
>[12]https://jenkins.fd.io/job/vpp-csit-verify-virl-master/6781/ : SUCCESS
>[13]https://jenkins.fd.io/job/vpp-docs-verify-master/5370/ : SUCCESS
> 
>=> Container PoC  Aug 24 10:54 PM  Patch Set 9:  Build Successful
>[14]http://jenkins.ejkern.net:8080/job/vpp-docs-verify-master/1519/ :
>SUCCESS
>
> [15]http://jenkins.ejkern.net:8080/job/vpp-make-test-docs-verify-master/1516/
>: SUCCESS
>[16]http://jenkins.ejkern.net:8080/job/vpp-verify-master-centos7/1987/ :
>SUCCESS
>
> [17]http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1305/
>: SUCCESS
>

[vpp-dev] heads-up: failures while running tests against vpp with multiple workers

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi all,

TLDR: when running tests vs vpp with multiple workers, roughly 25% of
tests fail or crash vpp. It looks like buffer management is still not
completely thread safe.

I've pushed work-in-progress make test modification which runs the test
against both single-thread and multiple-worker vpp. There are quite a
few failures and/or coredumps while running against multiple-worker vpp.

These test cases are failing at this time:

ACLPluginConnTestCase
BFD4TestCase
BFDFIBTestCase
TestDHCP
Datapath
DisableFP
DisableIPFIX
Flowprobe
ReenableFP
ReenableIPFIX
TestGRE
TestIPv4FibCrud
TestIp4VrfMultiInst
TestIP6VrfMultiInst
TestL2fib
TestL2bdArpTerm
TestL2bdMultiInst
TestLB
TestNAT64
TestSNAT
TestSpan
TestVxlanGpe

it seems that there are still some thread safety issues with the buffer
management based on the TestSpan crash:

#2  0x00406d1e in os_exit (code=code@entry=1) at 
/home/ksekera/vpp/build-data/../src/vpp/vnet/main.c:287
#3  0x7f139af0c2fa in unix_signal_handler (signum=, 
si=, uc=)
at /home/ksekera/vpp/build-data/../src/vlib/unix/main.c:118
#4  
#5  mheap_put (v=0x7f1356bdf000, uoffset=18446744073709549696) at 
/home/ksekera/vpp/build-data/../src/vppinfra/mheap.c:797
#6  0x7f139aeb6574 in vlib_buffer_add_to_free_list (do_init=1 '\001', 
buffer_index=, f=0x7f1359d0f780,
vm=0x7f139b1252e0 ) at 
/home/ksekera/vpp/build-data/../src/vlib/buffer_funcs.h:861
#7  vlib_buffer_free_inline (follow_buffer_next=1, n_buffers=256, 
buffers=, vm=0x7f139b1252e0 )
at /home/ksekera/vpp/build-data/../src/vlib/buffer.c:705
#8  vlib_buffer_free_internal (vm=0x7f139b1252e0 , 
buffers=0x7f135b504110, n_buffers=)
at /home/ksekera/vpp/build-data/../src/vlib/buffer.c:730
#9  0x7f139aaba427 in vlib_buffer_free (n_buffers=256, buffers=, vm=0x7f139b1252e0 )
at /home/ksekera/vpp/build-data/../src/vlib/buffer_funcs.h:327
#10 pg_output (vm=0x7f139b1252e0 , node=, 
frame=)
at /home/ksekera/vpp/build-data/../src/vnet/pg/output.c:83

(gdb)
#5  mheap_put (v=0x7f1356bdf000, uoffset=18446744073709549696) at 
/home/ksekera/vpp/build-data/../src/vppinfra/mheap.c:797
797   if (e->n_user_data != n->prev_n_user_data)
(gdb) p *n
Cannot access memory at address 0x7f1556bde87c
(gdb)

here is the patch set if you want to try it out...

https://gerrit.fd.io/r/#/c/8090

it's still a bit clunky, as the testing is done in two phases - first
the full suite is run vs single-thread vpp, then vs multiple-worker vpp.
It's not straightforward to do this in one run (so that instead of
running A, B, C vs single and A, B, C vs multi we run A (vs single), A
(vs multi), B (vs single), B (vs multi), C (vs single), C (vs multi)) so
that's why for now it's implemented this way.

If you want to skip the single-thread tests to speed up your own
testing, run it like this:

env VPP_TEST_SKIP_SINGLE_THREAD=y make test

Currently, the number of worker threads is set as the core count minus
two, with a cap of 8. Higher number causes the ACL plugin to freak out
(memory allocation failure) and the VPP refuses to start, ruining the
day for everybody.

Regards,
Klement
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] jenkins build timeout

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi helpd...@fd.io,

I just noticed a 360 minute build timeout in jenkins..

https://jenkins.fd.io/job/vpp-verify-master-centos7/6639/console

timestamps for the build commands look unreal...

09:21:11   CC   svm/svm.lo
09:21:11   CC   svm/ssvm.lo
09:21:11   CC   svm/svm_fifo.lo
09:21:11   CC   svm/svm_fifo_segment.lo
09:21:12   CC   svm/svmdb.lo
09:21:16   CC   vlib/buffer.lo
09:21:54   CC   vlib/buffer_serialize.lo
...
11:34:28   CC   vnet/tcp/builtin_client.lo
11:35:22   CC   vnet/tcp/builtin_server.lo
11:36:41   CC   vnet/tcp/builtin_http_server.lo
11:36:51   CC   vnet/tcp/builtin_proxy.lo
11:38:17   CC   vnet/tcp/tcp_test.lo
11:38:19   CC   vnet/tcp/tcp.lo
11:40:07   CC   vnet/udp/udp.lo
...
14:15:29 copying selected object files to avoid basename conflicts...
14:15:32   CCLD test_svm_fifo1
14:15:38   CCLD libsvmdb.la
14:15:51   CCLD libvlibmemory.la
14:15:54   CCLD libvlibmemoryclient.la
14:16:11   CCLD vlib_unix
14:16:16   CCLD libvppapiclient.la
14:16:25   CCLD svmdbtool
14:16:50   CCLD bin/vpp_get_metrics
14:16:55   CCLD vpp_api_test
14:17:05   CCLD vpp_restart
14:17:20   CCLD bin/summary_stats_client
14:17:36   CCLD uri_udp_test
14:17:39   CCLD uri_tcp_test
14:17:55   CCLD libvlibsocket.la
14:18:10   CCLD libvppcom.la
14:19:12   CCLD vcl_test_server
14:19:12   CCLD vcl_test_client
14:49:28 Build timed out (after 360 minutes). Marking the build as
failed.

Any idea what causes this huge slowdowns?

Thanks,
Klement
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] about UT test framework for VPP

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hmm, I noticed that your system-wide scapy is 2.3.1, while we
specifically use version 2.3.3 in the tests. Also I wonder how it's
possible that the tests pass on centos in gerrit..

Please note that the system-wide or 'pip install'-wide scapy doesn't
matter as make test sets up its own virtualenv (think of it as its own
chroot with all the (python) stuff it needs)...

Quoting Billy McFall (2017-08-15 16:25:19)
>FYI - I was able to run "make test" on my Fedora laptop. I noticed the
>scapy wasn't installed, so I ran "pip uninstall scapy" on the CentOS
>server that is having the issue. Still has the problem.
>Here is the output:
>(virtualenv)[bmcfall@d2fxl02 test]$ scapy 
>INFO: Can't import matplotlib. Won't be able to plot.
>INFO: Can't import PyX. Won't be able to use psdump() or pdfdump().
>Traceback (most recent call last):
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/bin/scapy",
>line 25, in 
>    interact()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/main.py",
>line 300, in interact
>    scapy_builtins = __import__("all",globals(),locals(),".").__dict__
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/all.py",
>line 28, in 
>    from scapy.route6 import *
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/route6.py",
>line 273, in 
>    conf.route6 = Route6()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/route6.py",
>line 31, in __init__
>    self.resync()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/route6.py",
>line 44, in resync
>    self.routes = read_routes6()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/arch/linux.py",
>line 283, in read_routes6
>    cset = scapy.utils6.construct_source_candidate_set(d, dp, devaddrs,
>LOOPBACK_NAME)
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/utils6.py",
>line 50, in construct_source_candidate_set
>    if in6_isgladdr(addr) or in6_isuladdr(addr):
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/utils6.py",
>line 708, in in6_isgladdr
>    return in6_isincluded(str, '2000::', 3)
>      File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/utils6.py",
>line 651, in in6_isincluded
>    temp = inet_pton(socket.AF_INET6, addr)
>Billy McFall
>On Tue, Aug 15, 2017 at 10:02 AM, Klement Sekera -X (ksekera - PANTHEON
>TECHNOLOGIES at Cisco) <[1]ksek...@cisco.com> wrote:
> 
>  We do not use the system-wide scapy, instead we install a specific
>  version of scapy which we patch with our own stuff.
> 
>  Could you try "make test-shell" and run scapy from the spawned shell?
> 
>  Thanks,
>  Klement
> 
>  Quoting Billy McFall (2017-08-15 14:26:33)
>  >    Thanks Klement,
>  >    Details:
>  >      OS: CentOS Linux release 7.3.1611
>  >      Kernel: 3.10.0-514.21.1.el7.x86_64
>  >      sudo scapy
>      >      INFO: Can't import python gnuplot wrapper . Won't be able to
>  plot.
>  >      INFO: Can't import PyX. Won't be able to use psdump() or
>  pdfdump().
>  >      Welcome to Scapy (2.3.1)
>  >      >>>
>  >    Let me know if there is something else that would help.
>  >    Billy
>  >    On Tue, Aug 15, 2017 at 4:02 AM, Klement Sekera -X (ksekera -
>  PANTHEON
>  >    TECHNOLOGIES at Cisco) <[1][2]ksek...@cisco.com> wrote:
>  >
>  >      Hi Billy,
>  >
>  >      I haven't seen this issue yet, but it looks like this is a scapy
>  issue
>  >      on your box. Scapy is a 3rd party library which we use in the
>  test
>  >      framework. What is the exact version of your OS etc?
>  >
>  >      Thanks,
>  >      Klement
>  >
>  >   

Re: [vpp-dev] [csit-dev] about UT test framework for VPP

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

We do not use the system-wide scapy, instead we install a specific
version of scapy which we patch with our own stuff.

Could you try "make test-shell" and run scapy from the spawned shell?

Thanks,
Klement

Quoting Billy McFall (2017-08-15 14:26:33)
>Thanks Klement,
>Details:
>  OS: CentOS Linux release 7.3.1611
>  Kernel: 3.10.0-514.21.1.el7.x86_64
>  sudo scapy
>  INFO: Can't import python gnuplot wrapper . Won't be able to plot.
>  INFO: Can't import PyX. Won't be able to use psdump() or pdfdump().
>  Welcome to Scapy (2.3.1)
>  >>>
>Let me know if there is something else that would help.
>    Billy
>    On Tue, Aug 15, 2017 at 4:02 AM, Klement Sekera -X (ksekera - PANTHEON
>TECHNOLOGIES at Cisco) <[1]ksek...@cisco.com> wrote:
> 
>  Hi Billy,
> 
>  I haven't seen this issue yet, but it looks like this is a scapy issue
>  on your box. Scapy is a 3rd party library which we use in the test
>  framework. What is the exact version of your OS etc?
> 
>  Thanks,
>  Klement
> 
>  Quoting Billy McFall (2017-08-14 21:30:35)
>  >    I am trying to run "make test" on a CentOS bare metal server. I'm
>  getting
>  >    "Exception: Illegal syntax for IP address". What do I need to
>  >    setup/configure on my server before running "make test"?
>  >    Thanks,
>  >    Billy McFall
>  >    $ make test V=2
>  >    make -C /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root
>  >    PLATFORM=vpp TAG=vpp vpp-install
>  >    make[1]: Entering directory
>  >    `/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root'
>  >     Arch for platform 'vpp' is native 
>  >     Finding source for dpdk 
>  >     Makefile fragment found in
>  >   
>  
> /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-data/packages/[1][2]dpdk.mk
>  >    
>  >     Source found in
>  /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/dpdk 
>  >     Arch for platform 'vpp' is native 
>  >     Finding source for vpp 
>  >     Makefile fragment found in
>  >   
>  
> /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-data/packages/[2][3]vpp.mk
>  >    
>  >     Source found in
>  /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/src 
>  >     Configuring dpdk: nothing to do 
>  >     Building dpdk: nothing to do 
>  >     Installing dpdk: nothing to do 
>  >     Configuring vpp: nothing to do 
>  >     Building vpp: nothing to do 
>  >     Installing vpp: nothing to do 
>  >    make[1]: Leaving directory
>  >    `/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root'
>  >    make -C test
>  TEST_DIR=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test
>  >   
>  
> VPP_TEST_BUILD_DIR=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/build-vpp-native
>  >    VPP_TES
>  >   
>  
> T_BIN=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/bin/vpp
>  >   
>  
> VPP_TEST_PLUGIN_PATH=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/i
>  >   
>  
> nstall-vpp-native/vpp/lib/vpp_plugins:/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins
>  >    VPP_TEST_INSTALL_PATH=/home/bmcfall
>  >    /dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/
>  >   
>  
> LD_LIBRARY_PATH=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/lib/:/home/bm
>  >   
>  
> cfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/lib64/
>  >    EXTENDED_TESTS= PYTHON= test
>  >    make[1]: Entering directory
>  >    `/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test'
>  >    Traceback (most recent call last):
>  >      File "sanity_run_vpp.py", line 7, in 
>  >        from framework import VppTestCase, KeepAliveReporter
>  >      File
>  "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test/framework.py",
>  >    line 20, in 
>  >        from vpp_pg_interface import VppPGInterface
>  >      File
>  >   
>  "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test/vpp_pg_interface.py",
>  >    line 8, in 
>  >        from vpp_interface import VppInterface
>  >      File
>  >   
> 

Re: [vpp-dev] [csit-dev] about UT test framework for VPP

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Billy,

I haven't seen this issue yet, but it looks like this is a scapy issue
on your box. Scapy is a 3rd party library which we use in the test
framework. What is the exact version of your OS etc?

Thanks,
Klement

Quoting Billy McFall (2017-08-14 21:30:35)
>I am trying to run "make test" on a CentOS bare metal server. I'm getting
>"Exception: Illegal syntax for IP address". What do I need to
>setup/configure on my server before running "make test"?
>Thanks,
>Billy McFall
>$ make test V=2
>make -C /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root
>PLATFORM=vpp TAG=vpp vpp-install
>make[1]: Entering directory
>`/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root'
> Arch for platform 'vpp' is native 
> Finding source for dpdk 
> Makefile fragment found in
>/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-data/packages/[1]dpdk.mk
>
> Source found in /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/dpdk 
> Arch for platform 'vpp' is native 
> Finding source for vpp 
> Makefile fragment found in
>/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-data/packages/[2]vpp.mk
>
> Source found in /home/bmcfall/dev/2017_08_14_VPP_Master/vpp/src 
> Configuring dpdk: nothing to do 
> Building dpdk: nothing to do 
> Installing dpdk: nothing to do 
> Configuring vpp: nothing to do 
> Building vpp: nothing to do 
> Installing vpp: nothing to do 
>make[1]: Leaving directory
>`/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root'
>make -C test TEST_DIR=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test
>
> VPP_TEST_BUILD_DIR=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/build-vpp-native
>VPP_TES
>
> T_BIN=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/bin/vpp
>
> VPP_TEST_PLUGIN_PATH=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/i
>
> nstall-vpp-native/vpp/lib/vpp_plugins:/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/lib64/vpp_plugins
>VPP_TEST_INSTALL_PATH=/home/bmcfall
>/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/
>
> LD_LIBRARY_PATH=/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/lib/:/home/bm
>
> cfall/dev/2017_08_14_VPP_Master/vpp/build-root/install-vpp-native/vpp/lib64/
>EXTENDED_TESTS= PYTHON= test
>make[1]: Entering directory
>`/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test'
>Traceback (most recent call last):
>  File "sanity_run_vpp.py", line 7, in 
>    from framework import VppTestCase, KeepAliveReporter
>  File "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test/framework.py",
>line 20, in 
>    from vpp_pg_interface import VppPGInterface
>  File
>"/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test/vpp_pg_interface.py",
>line 8, in 
>    from vpp_interface import VppInterface
>  File
>"/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test/vpp_interface.py", line
>4, in 
>    from util import Host, mk_ll_addr
>  File "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/test/util.py", line 7,
>in 
>    from scapy.layers.inet6 import in6_mactoifaceid
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/layers/inet6.py",
>line 56, in 
>    import scapy.route6
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/route6.py",
>line 273, in 
>    conf.route6 = Route6()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/route6.py",
>line 31, in __init__
>    self.resync()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/route6.py",
>line 44, in resync
>    self.routes = read_routes6()
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/arch/linux.py",
>line 283, in read_routes6
>    cset = scapy.utils6.construct_source_candidate_set(d, dp, devaddrs,
>LOOPBACK_NAME)
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/utils6.py",
>line 50, in construct_source_candidate_set
>    if in6_isgladdr(addr) or in6_isuladdr(addr):
>  File
>
> "/home/bmcfall/dev/2017_08_14_VPP_Master/vpp/build-root/python/virtualenv/lib/python2.7/site-packages/scapy/utils6.py",
>line 708, in in6_isgladdr
>    return in6_isincluded(str, '2000::', 3)
>  File
>
> 

Re: [vpp-dev] VPP compile error building vpp ipsec on Fed 26

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Same happens on arch. At some point of time, openssl dropped the
definitions of some structures which vpp uses directly and made them
opaque to user, so instead of embedding them, application has to
hold pointers and use the suplied XXX_Init-like functions...

Regards,
Klement

Quoting Thomas F Herbert (2017-08-13 23:45:05)
>I am getting a compile error when building on Fedora 26.
> 
>I am building master commit 3f6ff19a30e9fbe5befb4cc3521d1812e5612197
> 
>With openssl-devel-1.1.0f-7.fc26.x86_64 installed.
> 
>  CC   vnet/ipsec/ipsec.lo
>In file included from
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/ipsec.c:25:0:
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:63:18:
>error: field ‘encrypt_ctx’ has incomplete type
>   EVP_CIPHER_CTX encrypt_ctx;
>  ^~~
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:65:18:
>error: field ‘decrypt_ctx’ has incomplete type
>   EVP_CIPHER_CTX decrypt_ctx;
>  ^~~
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:67:12:
>error: field ‘hmac_ctx’ has incomplete type
>   HMAC_CTX hmac_ctx;
>    ^~~~
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:
>In function ‘esp_init’:
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:274:7:
>error: implicit declaration of function ‘HMAC_CTX_init’; did you mean
>‘HMAC_CTX_new’? [-Werror=implicit-function-declaration]
>   HMAC_CTX_init (&(em->per_thread_data[thread_id].hmac_ctx));
>   ^
>   HMAC_CTX_new
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:
>In function ‘hmac_calc’:
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:301:3:
>error: ‘HMAC_Init’ is deprecated [-Werror=deprecated-declarations]
>   HMAC_Init (ctx, key, key_len, md);
>   ^
>In file included from /usr/include/openssl/opensslconf.h:42:0,
> from /usr/include/openssl/bn.h:31,
> from /usr/include/openssl/asn1.h:24,
> from /usr/include/openssl/objects.h:916,
> from /usr/include/openssl/evp.h:27,
> from /usr/include/openssl/hmac.h:15,
> from
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/esp.h:18,
> from
>
> /home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-data/../src/vnet/ipsec/ipsec.c:25:
>/usr/include/openssl/hmac.h:28:1: note: declared here
> DEPRECATEDIN_1_1_0(__owur int HMAC_Init(HMAC_CTX *ctx, const void *key,
>int len,
> ^
>cc1: all warnings being treated as errors
>make[6]: *** [Makefile:6098: vnet/ipsec/ipsec.lo] Error 1
>make[6]: *** Waiting for unfinished jobs
>make[6]: Leaving directory
>
> '/home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-root/build-vpp-native/vpp'
>make[5]: *** [Makefile:6979: all-recursive] Error 1
>make[5]: Leaving directory
>
> '/home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-root/build-vpp-native/vpp'
>make[4]: *** [Makefile:3562: all] Error 2
>make[4]: Leaving directory
>
> '/home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-root/build-vpp-native/vpp'
>make[3]: *** [Makefile:697: vpp-build] Error 2
>make[3]: Leaving directory
>'/home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-root'
>make[2]: *** [Makefile:933: install-packages] Error 1
>make[2]: Leaving directory
>'/home/therbert/fd.io/vpp/extras/rpm/vpp-17.10/build-root'
>error: Bad exit status from /var/tmp/rpm-tmp.kz3qOs (%build)
> 
>...
> 
>--
>Thomas F Herbert
>NFV and Fast Data Planes
>Office of Technology
>Red Hat
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Spurious make test failure (container POC)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

The 2 minute timeout is the result of my recent change. The framework
now forks and runs the test in a child process, and if the child process
fails to send a keep-alive (sent when a test case starts), then it's
killed. Otherwise there'd be no way to recover from stuck mutex or
deadlock..

Are you running the extended tests or the stock verify?

Quoting Ed Kern (ejk) (2017-08-10 00:08:19)
>klement,
>ok…ill think about how to do that without too much trouble in its current
>state..
>in the meantime…blowing out the cpu and memory a bit changed the error……
> 
>  21:49:42 create 1k of p2p subifs 
>  OK
>  21:49:42 
> ==
>  21:51:52 21:53:13,610 Timeout while waiting for child test runner process 
> (last test running was `drop rx packet not matching p2p subinterface' in 
> `/tmp/vpp-unittest-P2PEthernetIPV6-GDHSDK')!
>  21:51:52 Killing possible remaining process IDs:  19954 19962 19964
> 
>  21:45:05 PPPoE Test Case
>  21:45:05 ===21:48:13,778 Timeout while 
> waiting for child test runner process (last test running was `drop rx packet 
> not matching p2p subinterface' in `/tmp/vpp-unittest-P2PEthernetIPV6-I0REOQ')!
>  21:47:45 Killing possible remaining process IDs:  20017 20025 20027
> 
>  20:48:46 PPPoE Test Case
>  20:48:46 ===20:51:34,082 Timeout while 
> waiting for child test runner process (last test running was `drop rx packet 
> not matching p2p subinterface' in `/tmp/vpp-unittest-P2PEthernetIPV6-tQ5sP0')!
>  20:51:05 Killing possible remaining process IDs:  19919 19927 19929
> 
>anything new/different/exciting in here?
>Also the memory/cpu expansion (by roughly a third) these failures happen
>in the order of 2/3 minutes as opposed to a 90 leading to timeout failure.
>Since the verifies are still happily chugging along I ASSuME that this
>drop packet check isn’t happening in that suite?
>Ed
> 
>  On Aug 9, 2017, at 1:04 PM, Klement Sekera -X (ksekera - PANTHEON
>  TECHNOLOGIES at Cisco) <[1]ksek...@cisco.com> wrote:
>  Ed,
> 
>  it'd help if you could collect log.txt from a failed run so we could
>  peek under the hood... please see my other email in this thread...
> 
>  Thanks,
>  Klement
> 
>  Quoting Ed Kern (ejk) (2017-08-09 20:48:46)
> 
>  this is not you…or this patch…
>  the make test-debug has had a 90+% failure rate (read not 100%) for
>at
>  least the last 100 builds
>  (far back as my current logs go but will probably blow that out a
>bit now)
>  you hit the one that is seen most often… on that create 1k of p2p
>subifs 
>  the other much less frequent is 
> 
>13:40:24 CGNAT TCP session close initiated from outside network
>  OK
>13:40:24 =Build timed
>out (after 120 minutes). Marking the build as failed.
> 
>  so currently I’m allocating 1 MHz in cpu and 8G in memory for
>verify
>  and also for test-debug runs…
>  Im not obviously getting (as you can see) errors about it running
>out of
>  memory but I wonder if thats possibly whats happening..
>  its easy enough to blow my allocations out a bit and see if that
>makes a
>  difference..
>  If anyone has other ideas to try and happy to give them a shot..
>  appreciate the heads up
>  Ed
> 
>On Aug 9, 2017, at 12:07 PM, Dave Barach (dbarach)
><[1][2]dbar...@cisco.com> wrote:
>Please see [2][3]https://gerrit.fd.io/r/#/c/7927, and 
> 
>
> [3][4]http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1056/console
> 
>The patch in question is highly unlikely to cause this failure...
> 
> 
>14:37:11
>
> ==
>14:37:11 P2P Ethernet tests
>14:37:11
>
> ==
>14:37:11 delete/create p2p
>subif  OK
>14:37:11 create 100k of p2p
>subifs    SKIP
>14:37:11 create 1k of p2p
>subifs  Build
>timed out
&

Re: [vpp-dev] Spurious make test failure (container POC)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Ed,

it'd help if you could collect log.txt from a failed run so we could
peek under the hood... please see my other email in this thread...

Thanks,
Klement

Quoting Ed Kern (ejk) (2017-08-09 20:48:46)
>this is not you…or this patch…
>the make test-debug has had a 90+% failure rate (read not 100%) for at
>least the last 100 builds
>(far back as my current logs go but will probably blow that out a bit now)
>you hit the one that is seen most often… on that create 1k of p2p subifs 
>the other much less frequent is 
> 
>  13:40:24 CGNAT TCP session close initiated from outside network  
>  OK
>  13:40:24 =Build timed out 
> (after 120 minutes). Marking the build as failed.
> 
>so currently I’m allocating 1 MHz in cpu and 8G in memory for verify
>and also for test-debug runs…
>Im not obviously getting (as you can see) errors about it running out of
>memory but I wonder if thats possibly whats happening..
>its easy enough to blow my allocations out a bit and see if that makes a
>difference..
>If anyone has other ideas to try and happy to give them a shot..
>appreciate the heads up
>Ed
> 
>  On Aug 9, 2017, at 12:07 PM, Dave Barach (dbarach)
>  <[1]dbar...@cisco.com> wrote:
>  Please see [2]https://gerrit.fd.io/r/#/c/7927, and 
>   
>  
> [3]http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1056/console
>   
>  The patch in question is highly unlikely to cause this failure...
>   
>   
>  14:37:11
>  
> ==
>  14:37:11 P2P Ethernet tests
>  14:37:11
>  
> ==
>  14:37:11 delete/create p2p
>  subif  OK
>  14:37:11 create 100k of p2p
>  subifs    SKIP
>  14:37:11 create 1k of p2p
>  subifs  Build timed out
>  (after 120 minutes). Marking the build as failed.
>  16:24:49 $ ssh-agent -k
>  16:24:54 unset SSH_AUTH_SOCK;
>  16:24:54 unset SSH_AGENT_PID;
>  16:24:54 echo Agent pid 84 killed;
>  16:25:07 [ssh-agent] Stopped.
>  16:25:07 Build was aborted
>  16:25:09 [WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
>  16:25:11 Finished: FAILURE
>   
>  Thanks… Dave
> 
> References
> 
>Visible links
>1. mailto:dbar...@cisco.com
>2. https://gerrit.fd.io/r/#/c/7927
>3. 
> http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1056/console
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Spurious make test failure (container POC)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Dave,

this looks like something got stuck - and the only thing I know
which can get the framework stuck like this is if vpp coredumps mid-API,
in which case, python gets stuck doing unix_shared_memory_queue_sub(). I
actually pushed a patch today, which makes the test framework to fork at
start and the tests then run in child process, periodically sending
keep-alives via pipe to parent. There is a default timeout of 120
seconds for each test case, which, if not met will cause the parent to
kill the child and give up. So this will at least make this visible
sooner.

Anyhow, what we really need is to configure the jenkins to save the
/tmp/vpp-unittest-* stuff (or at least /tmp/vpp-unittest-*/log.txt)
so we can see what actually happened.

Alternatively, we could add V=2 to make test in the verify job, but that
would add at least 50MB of data to the console output, which would make
it cumbersome..

Any jenkins wizards around?

Thanks,
Klement

Quoting Dave Barach (dbarach) (2017-08-09 20:07:47)
>Please see [1]https://gerrit.fd.io/r/#/c/7927, and
> 
> 
> 
>
> [2]http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1056/console
> 
> 
> 
>The patch in question is highly unlikely to cause this failure...
> 
> 
> 
> 
> 
>14:37:11
>
> ==
> 
>14:37:11 P2P Ethernet tests
> 
>14:37:11
>
> ==
> 
>14:37:11 delete/create p2p
>subif  OK
> 
>14:37:11 create 100k of p2p
>subifs    SKIP
> 
>14:37:11 create 1k of p2p
>subifs  Build timed out
>(after 120 minutes). Marking the build as failed.
> 
>16:24:49 $ ssh-agent -k
> 
>16:24:54 unset SSH_AUTH_SOCK;
> 
>16:24:54 unset SSH_AGENT_PID;
> 
>16:24:54 echo Agent pid 84 killed;
> 
>16:25:07 [ssh-agent] Stopped.
> 
>16:25:07 Build was aborted
> 
>16:25:09 [WS-CLEANUP] Deleting project workspace...[WS-CLEANUP] done
> 
>16:25:11 Finished: FAILURE
> 
> 
> 
>Thanks… Dave
> 
> 
> 
> References
> 
>Visible links
>1. https://gerrit.fd.io/r/#/c/7927
>2. 
> http://jenkins.ejkern.net:8080/job/vpp-test-debug-master-ubuntu1604/1056/console
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Are you suggesting we put all the dependent python packages as part of
the vpp source code tree then?
Or is this request aimed at the test box maintainers?

Thanks,
Klement

Quoting Luke, Chris (2017-08-08 17:04:57)
> Sure, I know what pip is.
> 
> My contention is that virtualenv should not be going to the network; in the 
> old days it would dip into the system packages for anything the base venv 
> needed. It's a really poor and insecure assumption that unfettered internet 
> access is universally available. In this case, I suspect wherever the job 
> runs needs an HTTP proxy to fit that assumption.
> 
> So either pre-stage the packages you need, and tell pip where to find them, 
> or provide a proxy. :)
> 
> Chris.
> 
> 
> > -----Original Message-
> > From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > [mailto:ksek...@cisco.com]
> > Sent: Tuesday, August 08, 2017 10:51 AM
> > To: Dave Barach (dbarach) <dbar...@cisco.com>; Luke, Chris
> > <chris_l...@cable.comcast.com>; csit-...@lists.fd.io
> > Cc: vpp-dev@lists.fd.io
> > Subject: RE: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> > failure
> > 
> > Think of pip (used by virtualenv) like apt-get.  Python maintainers are
> > innocent in this case. When adding the virtualenv feature to test framework,
> > it never occured to me that somebody might be doing vpp development on
> > an offline machine. Laptop which was online and later went offline doesn't
> > count, because virtualenv uses cached versions for subsequent installations
> > afaik.
> > 
> > If this is a requirement, we'll need to think of a way to ship the python
> > packages which virtualenv requires - e.g. setuptools - ourselves.
> > But if we do that, we also need to have all the other packages which the 
> > test
> > framework uses (e.g. scapy) somewhere in-tree(?).
> > 
> > The packages currently used by test framework (apart from the implicit
> > requirements) are:
> > 
> > scapy==2.3.3 pexpect subprocess32 cffi
> > git+https://github.com/klement/py-lispnetworking@setup
> > 
> > the last one is a fork of py-lispnetworking, which is used by some LISP 
> > tests. I
> > forked it and added an installation script, which was missing in the 
> > original
> > repo so that we can install it in the same way as other packages.
> > 
> > I have a strong feeling that this isn't something which we want to do...
> > 
> > Klement
> > 
> > Quoting Luke, Chris (2017-08-08 16:31:31)
> > > I've seen precisely this issue with Python's virtualenv whenever the host
> > doesn't have internet access while the venv is being created; I work around 
> > it
> > using an HTTP proxy, though requiring internet access just to create a venv
> > seems like a particularly braindead decision by the Python maintainers.
> > >
> > > Chris.
> > >
> > > > -Original Message-
> > > > From: csit-dev-boun...@lists.fd.io
> > > > [mailto:csit-dev-boun...@lists.fd.io] On Behalf Of Klement Sekera -X
> > > > (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > > > Sent: Tuesday, August 08, 2017 8:23 AM
> > > > To: Dave Barach (dbarach) <dbar...@cisco.com>; csit-...@lists.fd.io
> > > > Cc: vpp-dev@lists.fd.io
> > > > Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master"
> > > > job failure
> > > >
> > > > Looks to me like a bug in the urllib3 python library while reporting
> > > > some kind of connection failure.. never seen it before..
> > > >
> > > > Klement
> > > >
> > > > Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> > > > >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> > > > >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> > > > master/2814/console.
> > > > >
> > > > >
> > > > >
> > > > >Any idea what this is all about?
> > > > >
> > > > >
> > > > >
> > > > >Thanks… Dave
> > > > >
> > > > >
> > > > >
> > > > > References
> > > > >
> > > > >Visible links
> > > > >1. https://gerrit.fd.io/r/#/c/7885
> > > > >2.
> > > > > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/2814/co
> > > > > nsol
> > > > > e
> > > > ___
> > > > csit-dev mailing list
> > > > csit-...@lists.fd.io
> > > > https://lists.fd.io/mailman/listinfo/csit-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] "vpp-make-test-docs-verify-master" job failure

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Think of pip (used by virtualenv) like apt-get.  Python maintainers
are innocent in this case. When adding the virtualenv feature to test
framework, it never occured to me that somebody might be doing vpp development
on an offline machine. Laptop which was online and later went offline doesn't
count, because virtualenv uses cached versions for subsequent installations
afaik.

If this is a requirement, we'll need to think of a way to ship the python
packages which virtualenv requires - e.g. setuptools - ourselves.
But if we do that, we also need to have all the other packages which the
test framework uses (e.g. scapy) somewhere in-tree(?).

The packages currently used by test framework (apart from the implicit
requirements) are:

scapy==2.3.3 pexpect subprocess32 cffi
git+https://github.com/klement/py-lispnetworking@setup

the last one is a fork of py-lispnetworking, which is used by some LISP
tests. I forked it and added an installation script, which was missing
in the original repo so that we can install it in the same way as other
packages.

I have a strong feeling that this isn't something which we want to do...

Klement

Quoting Luke, Chris (2017-08-08 16:31:31)
> I've seen precisely this issue with Python's virtualenv whenever the host 
> doesn't have internet access while the venv is being created; I work around 
> it using an HTTP proxy, though requiring internet access just to create a 
> venv seems like a particularly braindead decision by the Python maintainers.
> 
> Chris.
> 
> > -Original Message-
> > From: csit-dev-boun...@lists.fd.io [mailto:csit-dev-boun...@lists.fd.io] On
> > Behalf Of Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> > Sent: Tuesday, August 08, 2017 8:23 AM
> > To: Dave Barach (dbarach) <dbar...@cisco.com>; csit-...@lists.fd.io
> > Cc: vpp-dev@lists.fd.io
> > Subject: Re: [csit-dev] [vpp-dev] "vpp-make-test-docs-verify-master" job
> > failure
> > 
> > Looks to me like a bug in the urllib3 python library while reporting some 
> > kind
> > of connection failure.. never seen it before..
> > 
> > Klement
> > 
> > Quoting Dave Barach (dbarach) (2017-08-08 14:05:16)
> > >Please see [1]https://gerrit.fd.io/r/#/c/7885,
> > >[2]https://jenkins.fd.io/job/vpp-make-test-docs-verify-
> > master/2814/console.
> > >
> > >
> > >
> > >Any idea what this is all about?
> > >
> > >
> > >
> > >Thanks… Dave
> > >
> > >
> > >
> > > References
> > >
> > >Visible links
> > >1. https://gerrit.fd.io/r/#/c/7885
> > >2.
> > > https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/2814/consol
> > > e
> > ___
> > csit-dev mailing list
> > csit-...@lists.fd.io
> > https://lists.fd.io/mailman/listinfo/csit-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Building on Fedora 24

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Tomas,

could you please take a look at the main Makefile:

57 ifeq ($(OS_ID)-$(OS_VERSION_ID),fedora-25)
58 RPM_DEPENDS += python2-virtualenv
59 RPM_DEPENDS_GROUPS = 'C Development Tools and Libraries'
60 else
61 RPM_DEPENDS += python-virtualenv
62 RPM_DEPENDS_GROUPS = 'Development Tools'
63 endif

see how the fedora-25 is the version which uses python2-virtualenv
while all others use python-virtualenv? Could you please change fedora-25
to fedora-24 on line 57 and let us know if this smooths things out?
Maybe the fix is a simple one-liner...

Thanks,
Klement

Quoting Tomas Brännström (2017-06-27 16:08:31)
>Hello
>I'm having some troubles building VPP (latest master) from source on
>Fedora 24.
>At first when doing `make bootstrap' it complained about not finding
>python-virtualenv. I could get around this by changing changing the
>Makefile to look for "python2-virtualenv" which was the version that got
>installed.
>But when doing `make pgk-rpm' I get the following errors:
>make[2]: Entering directory '/home/fedora/git/vpp/extras/rpm/vpp-17.10'
>Please install missing RPMs: \npackage python-virtualenv is not
>installed\n
>by executing "make install-dep"\n
>Makefile:175: recipe for target
>'/home/fedora/git/vpp/extras/rpm/vpp-17.10/build-root/.bootstrap.ok'
>failed
>make[2]: ***
>[/home/fedora/git/vpp/extras/rpm/vpp-17.10/build-root/.bootstrap.ok] Error
>1
>make[2]: Leaving directory '/home/fedora/git/vpp/extras/rpm/vpp-17.10'
>error: Bad exit status from /var/tmp/rpm-tmp.qSFuzD (%build)
>RPM build errors:
>    Macro %python2_minor_version defined but not used within scope
>    Bad exit status from /var/tmp/rpm-tmp.qSFuzD (%build)
>Makefile:22: recipe for target 'all' failed
>make[1]: *** [all] Error 1
>make[1]: Leaving directory '/home/fedora/git/vpp/extras/rpm'
>Makefile:397: recipe for target 'pkg-rpm' failed
>make: *** [pkg-rpm] Error 2
>I tried changing to python2-virtualenv in that Makefile as well but it
>seems to change back into python-virtualenv, and besides, there seems to
> be other problems here as well.
>Is there a workaround for this or is  Fedora 24 simply not supported?
>/Tomas
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] VPP: Answer UDP Packets

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Couple of potential issues which I see:

1.) if this buffer is generated by vpp, it should be flagged as such
(b->flags |= VNET_BUFFER_LOCALLY_ORIGINATED;)

2.) I don't see it passed to another node - you need to create a frame
and associate the buffer with it, then pass the frame to the next node,
which would be either ip4-arp or ip4-rewrite for ipv4 case depending on
whether the arp resolution has taken place or not...

take a look at functions

bfd_udp_init() - where the next nodes are resolved
bfd_transport_udp4() - where the next node is decided and frame created

You can take a look at show error and show trace to see what happens to
the packet..

HTH,
Klement


Quoting Alessio Silvestro (2017-06-16 17:20:30)
>Dear Klement,
> 
>thanks for the hint. I had a look at the code bfd_main.c and in particular
>at the function bfd_send_periodic().
> 
>I wrote the function send_udp_packet()  -- see below -- in order to send
>out UDP packets.
> 
>The function is executed (I can see from the VPP terminal) and no
>exception is raised, however I do not see any packet out.
> 
>1 -- is the function correct or I am missing something?
> 
>2 -- if it is not correct, how I can see where the problem is?
> 
>Thanks,
> 
>Alessio
> 
>static void send_udp_packet(vlib_main_t * vm, vlib_node_runtime_t * rt){
>    u32 bi;
>    vlib_buffer_alloc (vm, , 1)
>    vlib_buffer_t *b = vlib_get_buffer (vm, bi);
>    ASSERT (b->current_data == 0);
>    memset (vnet_buffer (b), 0, sizeof (*vnet_buffer (b)));
>    VLIB_BUFFER_TRACE_TRAJECTORY_INIT (b);
>    add_udp4_transport (vm,b);
>}
> 
>int add_udp4_transport (vlib_main_t * vm, vlib_buffer_t * b)
>{  
>    vnet_buffer (b)->ip.adj_index[VLIB_RX] =  ~0;
>    vnet_buffer (b)->ip.adj_index[VLIB_TX]=  ~0;
>    vnet_buffer (b)->sw_if_index[VLIB_RX] = 0;
>    vnet_buffer (b)->sw_if_index[VLIB_TX] = ~0;
>    u8 *data;
>    udp_header_t *udp;
>    ip4_header_t *ip;
>    data = vlib_buffer_get_current (b);
>    udp = (udp_header_t *) (data - sizeof (udp_header_t));
>    ip = (ip4_header_t *) ((u8 *) udp - sizeof (ip4_header_t));
>    // Build packet header
>    u32 src_address = 167772161; //[1]10.0.0.1  u32 version
>    u32 dst_address = 167772162; // 10.0.0.2 u32 version
>    ip->src_address.as_u32 = src_address;
>    ip->dst_address.as_u32 = dst_address;
>    ip->ip_version_and_header_length = 0x45;
>    ip->ttl = 254;
>    ip->protocol = IP_PROTOCOL_UDP; //0x45 == IP_PROTOCOL_UDP
>    ip->length = clib_host_to_net_u16 (b->current_length + sizeof (*udp));
>    ip->checksum = ip4_header_checksum (ip);
>    udp->src_port = DNS_SERVER_PORT;
>    udp->dst_port = DNS_DST_PORT;
>    udp->length = clib_host_to_net_u16 (b->current_length);
>    udp->checksum = 0;
>    b->current_length = sizeof (*ip) + sizeof (*udp);
>    return 1;
>}
> 
>On Wed, Jun 14, 2017 at 6:56 PM, Klement Sekera -X (ksekera - PANTHEON
>TECHNOLOGIES at Cisco) <[2]ksek...@cisco.com> wrote:
> 
>  Hi Alessio,
> 
>  you can take a look at BFD code which
> 
>  a.) creates and sends its own UDP packets - bfd_main.c -
>  bfd_send_periodic() creates, encapsulates (UDP) and sends a packet out
>  b.) loops back packets received - bfd_udp.c - bfd_udp_echo_input()
> 
>  I'm not sure what's your use case, whether you are trying to reuse
>  existing buffer, but one of these should fit it.
> 
>  Regards,
>  Klement
> 
>  Quoting Alessio Silvestro (2017-06-14 17:22:14)
>  >    Dear all,
>  >    I implemented a new VPP node that receives UDP traffic using the
>  following
>  >    function:
>  >
>  >    udp_register_dst_port (vm, PORT, my_node.index , 1 /* is_ip4 */);
>  >
>  >    I am able to parse the packet and I would like to be able to send
>  back an
>  >    UDP packet.
>  >
>  >    Looking at the source code, the only function that seems fit my
>  scope is
>  >    the following (in ~/vpp/src/vnet/udp/udp.h) 
>  >
>  >    ip_udp_encap_one (vlib_main_t * vm, vlib_buffer_t * b0, u8 * ec0,
>  word
>  >    ec_len,
>  >
>  >              u8 is_ip4)
>  >
>  >    Is that correct or there is another function for this purpose?
>  >
>  >    Thanks in advance for any help.
>  >
>  >    Best regards,
>  >
>  >    Alessio
> 
> References
> 
>Visible links
>1. http://10.0.0.1/
>2. mailto:ksek...@cisco.com
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] shadow build system change adding test-debug job

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Ed, these tests don't rely on keeping an interactive sesion up, like BFD tests
do. Fast box is required only to pass e.g. the BFD tests reliably enough.
These are only part of test-all and test-debug-all targets because the
jenkins infrastructure wasn't able to pass them reliably enough..
I'm sorry if I didn't explain this deeply enough and caused confusion on
your side..

Thanks,
Klement

Quoting Ed Kern (ejk) (2017-06-14 22:07:18)
>alright klement/dave
>im a bit stuck again…
>i get about an 60+% failure rate out of test-debug even with higher than
>normal cpu settings (higher than just what i use for build verify)
>always right here
> 
>  19:43:38 
> ==
>  19:43:38 ERROR: L2 FIB test 7 - flush bd_id
>  19:43:38 
> --
>  19:43:38 Traceback (most recent call last):
>  19:43:38   File 
> "/workspace/vpp-test-debug-master-ubuntu1604/test/test_l2_fib.py", line 508, 
> in test_l2_fib_07
>  19:43:38 self.run_verify_negat_test(bd_id=1, dst_hosts=flushed)
>  19:43:38   File 
> "/workspace/vpp-test-debug-master-ubuntu1604/test/test_l2_fib.py", line 418, 
> in run_verify_negat_test
>  19:43:38 i.get_capture(0, timeout=timeout)
>  19:43:38   File 
> "/workspace/vpp-test-debug-master-ubuntu1604/test/vpp_pg_interface.py", line 
> 240, in get_capture
>  19:43:38 (len(capture.res), expected_count, name))
>  19:43:38 Exception: Captured packets mismatch, captured 9 packets, expected 
> 0 packets on pg0
>  19:43:38
>  19:43:38 
> ==
>  19:43:38 ERROR: L2 FIB test 8 - flush all
>  19:43:38 
> --
>  19:43:38 Traceback (most recent call last):
>  19:43:38   File 
> "/workspace/vpp-test-debug-master-ubuntu1604/test/test_l2_fib.py", line 522, 
> in test_l2_fib_08
>  19:43:38 self.run_verify_negat_test(bd_id=1, dst_hosts=flushed)
>  19:43:38   File 
> "/workspace/vpp-test-debug-master-ubuntu1604/test/test_l2_fib.py", line 418, 
> in run_verify_negat_test
>  19:43:38 i.get_capture(0, timeout=timeout)
>  19:43:38   File 
> "/workspace/vpp-test-debug-master-ubuntu1604/test/vpp_pg_interface.py", line 
> 240, in get_capture
>  19:43:38 (len(capture.res), expected_count, name))
>  19:43:38 Exception: Captured packets mismatch, captured 9 packets, expected 
> 0 packets on pg0
>  19:43:38
> 
>when it fails its always the same two tests…always the same exception
>    (captured 9, expected 0)
>its so consistent in its ‘death’ but so intermittent in frequency its
>freaking me out a bit…
>any thoughts?
>Ed
> 
>  On May 24, 2017, at 8:42 AM, Klement Sekera -X (ksekera - PANTHEON
>  TECHNOLOGIES at Cisco) <[1]ksek...@cisco.com> wrote:
>  I know that the functional BFD tests passed so unless there is a bug in
>  the tests, the failures are pretty much timing issues. From my
>  experience the load is the culprit as the BFD tests test interactive
>  sessions, which need to be kept alive. The timings currently are set at
>  300ms and for most tests two keep-alives can be missed before the
>  session
>  goes down on vpp side and asserts start failing. While this might seem
>  like ample time, especially on loaded systems there is a high chance
>  that at least one test will derp ...
> 
>  I've also seen derps even on idle systems, where a select() call (used
>  by python in its own sleep() implementation) with timeout of 100ms
>  returns
>  after 1-3 seconds.
> 
>  Try running the bfd tests only (make test-all TEST=bfd) while no other
>  tasks
>  are running - I think they should pass on your box just fine.
> 
>  Thanks,
>  Klement
> 
>  Quoting Ed Kern (ejk) (2017-05-24 16:27:10)
> 
>  right now its a VERY intentional mix…but depending on loading I
>could
>  easily see this coming up if those timings are strict.  
>  To not dodge your question max loading on my slowest node would be 3
>  concurrent builds on an Xeon™ E3-1240 v3 (4 cores @ 3.4Ghz)
>    yeah yeah stop laughing…..Do you have suggested or even
>guesstimate
>  minimums in this regard…I could pretty trivially route them towards
>      the larger set that I have right now if you think magic will result
>:)
>  Ed
>  PS thanks though..for whatever reason the type of errors I was
>getting
>  

Re: [vpp-dev] VPP: Answer UDP Packets

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Alessio,

you can take a look at BFD code which

a.) creates and sends its own UDP packets - bfd_main.c -
bfd_send_periodic() creates, encapsulates (UDP) and sends a packet out
b.) loops back packets received - bfd_udp.c - bfd_udp_echo_input()

I'm not sure what's your use case, whether you are trying to reuse
existing buffer, but one of these should fit it.

Regards,
Klement

Quoting Alessio Silvestro (2017-06-14 17:22:14)
>Dear all,
>I implemented a new VPP node that receives UDP traffic using the following
>function:
> 
>udp_register_dst_port (vm, PORT, my_node.index , 1 /* is_ip4 */);
> 
>I am able to parse the packet and I would like to be able to send back an
>UDP packet.
> 
>Looking at the source code, the only function that seems fit my scope is
>the following (in ~/vpp/src/vnet/udp/udp.h) 
> 
>ip_udp_encap_one (vlib_main_t * vm, vlib_buffer_t * b0, u8 * ec0, word
>ec_len,
> 
>          u8 is_ip4)
> 
>Is that correct or there is another function for this purpose?
> 
>Thanks in advance for any help.
> 
>Best regards,
> 
>Alessio
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] control_ping messages in acl plugin

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Khers,

why not use the already existing control_ping(_reply) messages?

Thanks,
Klement

Quoting khers (2017-06-07 15:17:36)
>Hi
> 
>I've need acl_control_ping and acl_control_ping_reply message to use dump
>api in acl plugin, so I pushed a commit to gerrit.
>Please review this [1]commit,
>I'm a new commiter, so I don't know what to do when my code is not
>reviewed.
> 
>Regards,
>khers
> 
> References
> 
>Visible links
>1. https://gerrit.fd.io/r/#/c/6919/
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] shadow build system change adding test-debug job

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

I know that the functional BFD tests passed so unless there is a bug in
the tests, the failures are pretty much timing issues. From my
experience the load is the culprit as the BFD tests test interactive
sessions, which need to be kept alive. The timings currently are set at
300ms and for most tests two keep-alives can be missed before the session
goes down on vpp side and asserts start failing. While this might seem
like ample time, especially on loaded systems there is a high chance
that at least one test will derp ...

I've also seen derps even on idle systems, where a select() call (used
by python in its own sleep() implementation) with timeout of 100ms returns
after 1-3 seconds.

Try running the bfd tests only (make test-all TEST=bfd) while no other tasks
are running - I think they should pass on your box just fine.

Thanks,
Klement

Quoting Ed Kern (ejk) (2017-05-24 16:27:10)
>right now its a VERY intentional mix…but depending on loading I could
>easily see this coming up if those timings are strict.  
>To not dodge your question max loading on my slowest node would be 3
>concurrent builds on an Xeon™ E3-1240 v3 (4 cores @ 3.4Ghz)
>  yeah yeah stop laughing…..Do you have suggested or even guesstimate
>minimums in this regard…I could pretty trivially route them towards
>the larger set that I have right now if you think magic will result :)
>Ed
>PS thanks though..for whatever reason the type of errors I was getting
>didn’t naturally steer my mind towards cpu/io binding.
> 
>  On May 24, 2017, at 12:57 AM, Klement Sekera -X (ksekera - PANTHEON
>  TECHNOLOGIES at Cisco) <[1]ksek...@cisco.com> wrote:
>  Hi Ed,
> 
>  how fast are your boxes? And how many cores? The BFD tests struggle to
>  meet
>  the aggresive timings on slower boxes...
> 
>  Thanks,
>  Klement
> 
>  Quoting Ed Kern (ejk) (2017-05-23 20:43:55)
> 
>  No problem.
>  If anyone is curious in rubbernecking the accident that is the
>current
>  test-all (at least for my build system)
>  adding a comment of
>  testall
>  SHOULD trigger and fire it off on my end.
>  make it all pass and you win a beer (or beverage of your choice)  
>  Ed
> 
>On May 23, 2017, at 11:34 AM, Dave Wallace
><[1][2]dwallac...@gmail.com>
>wrote:
>Ed,
> 
>Thanks for adding this to the shadow build system.  Real data on
>the
>cost and effectiveness of this will be most useful.
> 
>-daw-
>On 5/23/2017 1:30 PM, Ed Kern (ejk) wrote:
> 
>In the vpp-dev call a couple hours ago there was a discussion of
>running test-debug on a regular/default? basis.
>As a trial I’ve added a new job to the shadow build system:
> 
>vpp-test-debug-master-ubuntu1604
> 
>Will do a make test-debug,  as part of verify set, as an ADDITIONAL
>job.
> 
>I gave a couple passes with test-all but can’t ever get a clean run
>with test-all (errors in test_bfd and test_ip6 ).
>I don’t think this is unusual or unexpected.  Ill leave it to someone
>else to say that ‘all’ throwing failures is a good thing.
>I’m happy to add another job for/with test-all if someone wants to
>actually debug those errors.
> 
>flames, comments,concerns welcome..
> 
>Ed
> 
>PS Please note/remember that all these tests are non-voting regardless
>of success or failure.
>___
>vpp-dev mailing list
>[2][3]vpp-dev@lists.fd.io
>[3][4]https://lists.fd.io/mailman/listinfo/vpp-dev
> 
>References
> 
>  Visible links
>  1. [5]mailto:dwallac...@gmail.com
>  2. [6]mailto:vpp-dev@lists.fd.io
>  3. [7]https://lists.fd.io/mailman/listinfo/vpp-dev
> 
> References
> 
>Visible links
>1. mailto:ksek...@cisco.com
>2. mailto:dwallac...@gmail.com
>3. mailto:vpp-dev@lists.fd.io
>4. https://lists.fd.io/mailman/listinfo/vpp-dev
>5. mailto:dwallac...@gmail.com
>6. mailto:vpp-dev@lists.fd.io
>7. https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] shadow build system change adding test-debug job

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Ed,

how fast are your boxes? And how many cores? The BFD tests struggle to meet
the aggresive timings on slower boxes...

Thanks,
Klement

Quoting Ed Kern (ejk) (2017-05-23 20:43:55)
>No problem.
>If anyone is curious in rubbernecking the accident that is the current
>test-all (at least for my build system)
>adding a comment of
>testall
>SHOULD trigger and fire it off on my end.
>make it all pass and you win a beer (or beverage of your choice)  
>Ed
> 
>  On May 23, 2017, at 11:34 AM, Dave Wallace <[1]dwallac...@gmail.com>
>  wrote:
>  Ed,
> 
>  Thanks for adding this to the shadow build system.  Real data on the
>  cost and effectiveness of this will be most useful.
> 
>  -daw-
>  On 5/23/2017 1:30 PM, Ed Kern (ejk) wrote:
> 
>  In the vpp-dev call a couple hours ago there was a discussion of running 
> test-debug on a regular/default? basis.
>  As a trial I’ve added a new job to the shadow build system:
> 
>  vpp-test-debug-master-ubuntu1604
> 
>  Will do a make test-debug,  as part of verify set, as an ADDITIONAL job.
> 
> 
>  I gave a couple passes with test-all but can’t ever get a clean run with 
> test-all (errors in test_bfd and test_ip6 ).
>  I don’t think this is unusual or unexpected.  Ill leave it to someone else 
> to say that ‘all’ throwing failures is a good thing.
>  I’m happy to add another job for/with test-all if someone wants to actually 
> debug those errors.
> 
>  flames, comments,concerns welcome..
> 
>  Ed
> 
>  PS Please note/remember that all these tests are non-voting regardless of 
> success or failure.
>  ___
>  vpp-dev mailing list
>  [2]vpp-dev@lists.fd.io
>  [3]https://lists.fd.io/mailman/listinfo/vpp-dev
> 
> References
> 
>Visible links
>1. mailto:dwallac...@gmail.com
>2. mailto:vpp-dev@lists.fd.io
>3. https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Cannot build v17.04 nor v17.07-rc0 due to warnings treated as errors: self-comparison always evaluates to true [-Werror=tautological-compare]

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Jean-Christophe,

I've pushed a patch to disable these warnings on gcc6. You can either wait
until it's merged or you can grab it from gerrit so that you can move
forward...: 

https://gerrit.fd.io/r/#/c/6370/

Regards,
Klement

Quoting jean-christophe Manciot (2017-04-23 13:48:54)
>Ubuntu 16.10 Linux 4.8
>VPP git sources: [1]https://gerrit.fd.io/r/vpp
>VPP branches: stable/1704 & master
>VPP tags: v17.04 & v17.07-rc0
>Building with the following workflow:
>        echo 
>        echo Cleaning
>        echo 
>        cd git-vpp
>        sudo -u actionmystique -H git-reset-clean-pull-checkout.sh $branch
>$tag
>        echo ---
>        echo Configuring
>        echo ---
>        cd build-root
>        sudo -u actionmystique -H ./bootstrap.sh
>        echo ---
>        echo "Trying to not treat warnings as errors"
>        echo ---
>        # Array which will contain all found file names
>        files_list=()
>        while IFS= read -d $'\0' -r file ; do
>                files_list=("${files_list[@]}" "$file")
>        done < <(find . -name "Makefile" -print0)
>        while IFS= read -d $'\0' -r file ; do
>                files_list=("${files_list[@]}" "$file")
>        done < <(find . -name "*.mk" -print0)
>        for file in "${files_list[@]}"
>        do
>                sed -i 's|-Werror||g' "$file"
>        done
>        echo -
>        echo Compiling
>        echo -
>        sudo -u actionmystique -H make V=0 PLATFORM=vpp TAG=vpp
>install-deb
>leads to the following log for v17.04 & v17.07-rc0, despite an attempt to
>workaround the issue above in "Trying to not treat warnings as errors":
>
> ---
>...
>  CC       vnet/ipsec/ipsec_if.lo
>/home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:
>In function ‘bfd_cli_udp_session_add’:
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:428:17:
>error: self-comparison always evaluates to true
>[-Werror=tautological-compare]
>   foreach_bfd_cli_udp_session_add_cli_param (CHECK_MANDATORY);
>                 ^~
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:428:199:
>error: self-comparison always evaluates to true
>[-Werror=tautological-compare]
>   foreach_bfd_cli_udp_session_add_cli_param (CHECK_MANDATORY);
>                                                                         
>                                                                         
>                                                   ^ 
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:428:381:
>error: self-comparison always evaluates to true
>[-Werror=tautological-compare]
>   foreach_bfd_cli_udp_session_add_cli_param (CHECK_MANDATORY);
>                                                                         
>                                                                         
>                               ^ 
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:428:561:
>error: self-comparison always evaluates to true
>[-Werror=tautological-compare]
>   foreach_bfd_cli_udp_session_add_cli_param (CHECK_MANDATORY);
>                                                                         
>                                                                         
>         ^ 
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:428:751:
>error: self-comparison always evaluates to true
>[-Werror=tautological-compare]
>   foreach_bfd_cli_udp_session_add_cli_param (CHECK_MANDATORY);
>                                                                         
>                                                                         
>                                                     
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:428:943:
>error: self-comparison always evaluates to true
>[-Werror=tautological-compare]
>   foreach_bfd_cli_udp_session_add_cli_param (CHECK_MANDATORY);
>                                                                         
>                                                             ^ 
>/home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:
>In function ‘bfd_cli_udp_session_mod’:
>
> /home/actionmystique/src/VPP/git-vpp/build-data/../src/vnet/bfd/bfd_cli.c:519:17:
>error: self-comparison always evaluates to true
>

[vpp-dev] Fwd: Change in vpp[master]: make test: python interpreter customization

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

does anybody know how to make virl pass? This is a make test related
change only and so far I've seen 3 virl failures. Interestingly, a
dependent change on this one passed on the first try.

Thanks,
Klement

Forwarded message from fd.io JJB (Code Review) (2017-04-12 18:10:53):
> fd.io JJB has posted comments on this change. ( https://gerrit.fd.io/r/6163 )
> 
> Change subject: make test: python interpreter customization
> ..
> 
> 
> Patch Set 1: Verified-1
> 
> Build Failed 
> 
> https://jenkins.fd.io/job/vpp-csit-verify-virl-master/4899/ : FAILURE
> 
> Logs: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-virl-master/4899
> 
> https://jenkins.fd.io/job/vpp-verify-master-ubuntu1404/4901/ : SUCCESS
> 
> Logs: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1404/4901
> 
> https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/4900/ : SUCCESS
> 
> Logs: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-ubuntu1604/4900
> 
> https://jenkins.fd.io/job/vpp-verify-master-centos7/4895/ : SUCCESS
> 
> Logs: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-verify-master-centos7/4895
> 
> https://jenkins.fd.io/job/vpp-docs-verify-master/3495/ : SUCCESS
> 
> Logs: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-docs-verify-master/3495
> 
> https://jenkins.fd.io/job/vpp-make-test-docs-verify-master/1223/ : SUCCESS
> 
> Logs: 
> https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-make-test-docs-verify-master/1223
> 
> -- 
> To view, visit https://gerrit.fd.io/r/6163
> To unsubscribe, visit https://gerrit.fd.io/r/settings
> 
> Gerrit-MessageType: comment
> Gerrit-Change-Id: I67a658fc927303468cc67f0ac192317ca2907625
> Gerrit-PatchSet: 1
> Gerrit-Project: vpp
> Gerrit-Branch: master
> Gerrit-Owner: Klement Sekera 
> Gerrit-Reviewer: Klement Sekera 
> Gerrit-Reviewer: fd.io JJB 
> Gerrit-HasComments: No
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] vpp make test is not working

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Jan,

1. Filip is already working on a fix.
2. make test was accidentally removed from the make verify, Filip will
add it back with his fix.

Regards,
Klement

Quoting Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco) (2017-02-22 
18:30:55)
>Hello,
> 
> 
> 
>Currently all make test are failing with the same error (run locally on my
>development VM system – ubuntu16.04):
> 
> 
> 
>
> ==
> 
>ERROR: setUpClass (test_ip6.TestIPv6)
> 
>
> --
> 
>Traceback (most recent call last):
> 
>  File "/home/vpp/Documents/vpp/test/test_ip6.py", line 31, in setUpClass
> 
>    super(TestIPv6, cls).setUpClass()
> 
>  File "/home/vpp/Documents/vpp/test/framework.py", line 242, in
>setUpClass
> 
>    cls.vapi = VppPapiProvider(cls.shm_prefix, cls.shm_prefix, cls)
> 
>  File "/home/vpp/Documents/vpp/test/vpp_papi_provider.py", line 54, in
>__init__
> 
>    self.papi = VPP(jsonfiles)
> 
>  File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 78, in
>__init__
> 
>    self.add_message(m[0], m[1:])
> 
>  File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 286, in
>add_message
> 
>    args[field_name] = self.__struct(*f)
> 
>  File "build/bdist.linux-x86_64/egg/vpp_papi/vpp_papi.py", line 151, in
>__struct
> 
>    raise ValueError(1, 'Invalid message type: ' + t)
> 
>ValueError: (1, u'Invalid message type: vl_api_local_locator_t')
> 
> 
> 
> 
> 
>It seems that issue has been introduced by following commit:
> 
> 
> 
>vpp@vpp-VirtualBox:~/Documents/vpp$ git bisect good
> 
>694396dc589b4fe75b1fad02fde1d3c3cdaeef04 is the first bad commit
> 
>commit 694396dc589b4fe75b1fad02fde1d3c3cdaeef04
> 
>Author: Filip Tehlar 
> 
>Date:   Fri Feb 17 14:29:11 2017 +0100
> 
> 
> 
>    Add Overlay Network Engine API
> 
>   
> 
>Change-Id: I6b5984df176688f0722a2888e73f05d8ed8b9310
> 
>    Signed-off-by: Filip Tehlar 
> 
> 
> 
>:04 04 b3fb6d7c953b74ba85e0345f2987452f643b28dc
>a73d9ebb51dcda1fb4795961af1ce5fce418009b M src
> 
> 
> 
>So there are two questions:
> 
> 
> 
>1.  Could somebody have a look on this issue, please?
> 
>2.  Are these (make test) tests really part of vpp-verify-master-{os}
>jobs (i.e. part of make verify)? I didn’t find any information about
>result of these tests in console output of the test
>([1]https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/3963/console).
> 
> 
> 
>Thanks,
> 
>Jan
> 
> References
> 
>Visible links
>1. https://jenkins.fd.io/job/vpp-verify-master-ubuntu1604/3963/console
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] avoiding spoof protection for BFD ECHO packets

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi guys,

BFD echo function allows testing datapaths only and thus using more
aggresive rates and faster detection by using packets, which are
processed only by the sender and simply looped back by the receiver.
Each peer declares the willingness/rate at which it will loop back
echo packets and each side decides to use the feature or not locally.

For the BFD over UDP, the echo packets are recognized by having
destination port 3785.

To implement this in VPP, we need to

1.) loop back echo packets from remote side - this is easy, already done
2.) be able to send the packets out and receive them - this hits the
current spoofing protection, when a packet with destination set to our
own IP address gets dropped like this:

...
00:00:00:708351: ip4-local
UDP: 172.16.2.1 -> 172.16.1.1
  tos 0x00, ttl 255, length 52, checksum 0x6096
  fragment id 0x
UDP: 49152 -> 3785
  length 32, checksum 0x
00:00:00:708351: error-drop
  ip4-input: ip4 spoofed local-address packet drops

in this example 172.16.1.1 is the address on the interface receiving the
packet.

Discussion with Neale yielded a few possible solutions, none of which is
great:

1.) add input feature to siphon BFD packets instead of going to
ip4-local node
2a.) skip checks in ip4-local node based on BFD ports
2b.) skip checks in ip4-local node based on UDP port registration (via
udp_register_dst_port())
3.) add information for prefix/address to FIB to skip checks for this
entry

based on discussion, here are the downsides of each:

1.) taxes all input packets
2.) layering violation, caches misses in b.) case
3.) exposes VPP to spoofed packets for non-BFD traffic

based on these, 2.) seems to hurt the least.. with 2a.) being the
easiest to implement to move forward..

I would appreciate thoughts/ideas from more experienced people..

Thanks,
Klement
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] make test: heads up - handling expected API failure

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

when doing API tests, sometimes it's expected that API fails. For
example, one might test adding the same configuration twice, or removing
non-existent configuration, expecting failure.

With my recent change https://gerrit.fd.io/r/#/c/4875/ merged, it's possible
to easily write these scenarios using a simple:

def test_api_fail(self):
...
with self.vapi.expect_negative_api_retval():
self.vapi.api_call()  # expected to fail

self.vapi.other_api_call()  # expected to pass

the expect_negative_api_retval() changes internal state to expect
failures, which is reverted back once the 'with' context is left. There
is also expect_zero_api_retval() mostly for sake of completness, so one
might write the above code also this way:

def test_api_fail(self):
   ...
   self.vapi.expect_negative_api_retval()
   self.vapi.api_call()  # expected to fail
   self.vapi.expect_zero_api_retval()
   self.vapi.other_api_call()  # expected to pass

I think the first one is simpler, more readable and less error prone
(because of the guaranteed expectation rollback).

Thanks,
Klement
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [FD.io Helpdesk #35687] Re: Verify job failure(s)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

+1 on this in general

comment regarding "make test": for required python packages, (e.g.
scapy), we install them in "virtualenv", which is python way of having
your own controlled playground. Thus, whatever (python) system packages
are there, "make test" ignores them. This allows us to pick the exact
version in test/Makefile (as we do for scapy, because we apply our
patches every time we install it, so we don't want the upstream
to break our test), so at least for python stuff in "make test", this
can be more or less ignored.

Thanks,
Klement

Quoting Dave Wallace (2017-01-20 04:38:33)
>Ed, Thanh, Vanessa,
> 
>IMHO, updating the ubuntu packages every time a VM is spun up is a bug
>wrt. being able to reproduce some (hopefully rare) build/test issues. 
>Since every VM is potentially running with different versions of OS
>components, when a failure occurs (e.g. in "make test"), then it may be
>necessary to recreate the exact run-time environment in order to reproduce
>the failure. Unless the complete package list is being archived for every
>VM instance that is spun up, this may not be possible. 
> 
>My experience is that those rare cases where a tool or environment issue
>causes a failure, the cost to find the issue is extraordinarily high if
>you do not have the ability to recreate the EXACT build/run-time
>environment.  This is why CSIT does not update OS components in the VM
>initialization scripts and the VM images are built from a specific package
>list instead of pulling the latest versions from the apt repositories.
> 
>My recommendation is that the VM images be updated periodically (weekly or
>whenever a new security update is released) and the package lists archived
>for each VM image version.  Each VM image should also be verified against
>a known good VPP commit version as is done with CSIT branches.  Ideally we
>should build a fully automated continuous deployment model to reduce the
>amount of work to update the VM images to running a Jenkins job to
>build/test/deploy a new VM image from the latest packages versions.
> 
>With that automation in place, this mechanism could be extended for use by
>CSIT as well as "make test", thus ensuring that all of our testing was
>done with the same OS component version.  Ideally, all projects should be
>using the same OS components to ensure that everything is tested in the
>same run-time environment.
> 
>Thanks,
>-daw-
>On 1/19/2017 8:31 PM, Thanh Ha via RT wrote:
> 
>  The issue with the 16.04 Ubuntu image is fixed now (but we may require some 
> additional actions which I'll send to Vanessa to in case this issue comes up 
> again). We fixed this issue tonight by rebuilding ubuntu1604 and deploying 
> the new image.
> 
>  I'm going to close this ticket as resolved and we'll take the additional 
> task to find a way to ensure this doesn't appear again off of this ticket.
> 
>  If you're not interested in the detailed analysis you can stop reading now.
> 
>  For those interested I suspect that the lock issue will appear again 
> (although I could be wrong). The reason I believe so is that our vm init 
> script runs "apt-get update" as an initialization step when the VM boots up 
> at creation time via this script [0]. Ed mentioned that we didn't see this in 
> the past and it only started appear again recently as we deployed another 
> patch to disable Ubuntu's unattended updates.
> 
>  I believe a possible reason we will see this issue appear again due to [0] 
> is because of we switched from using JClouds to OpenStack Jenkins plugins for 
> node spinnup and there's difference in how the init-script is executed 
> depending on which plugin is being used.
> 
>  JClouds Plugin:
> 
>  1) boot vm
>  2) wait for ssh access
>  3) copies init-script into vm via ssh
>  4) executes init-script, and doesn't continue processing until script is 
> complete
>  5) once init-script is complete, passes vm over to job and job starts
> 
>  OpenStack Plugin:
> 
>  1) boot vm and passes init-script in as User Data
>  2) init-script runs inside vm without Jenkins intervention, thus is a 
> non-blocking function
>  3) in parallel jenkins waits for ssh access to vm
>  4) ssh's into vm and passes vm over to job and job starts running
> 
>  In the OpenStack plugin case step 4 can execute while step 2 is still 
> running apt-get update in the background because it was a non-blocking 
> function.
> 
>  A few ideas I have to get around this.
> 
>  a) Allow init-script to continue running apt-get update however have a shell 
> script at the start of Ubuntu jobs that waits for the lock to get released 
> before allowing the job to start
> 
>  b) Remove apt-get update from init-script and make the job run apt-get 
> update at the beginning of it's execution
> 
>  c) Regularly update VMs to ensure that apt-get update always runs quickly
> 
>   Regards,
>  Thanh
> 
>  [0] 
> 

Re: [vpp-dev] [csit-dev] vpp make test for verify - are we there yet ? [awoken]

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi,

the modification timestamp of .py is stored inside .pyc... According to
what I found, the .pyc begins with 4 magic bytes, followed by 4
timestamp bytes (source file modification timestamp) and the rest
is the marshalled code object.

So doing a touch a.py, touch a.pyc won't fool it, since it doesn't care
for the .pyc modification time.

Could you verify the unix timestamps from a failed run?

Thanks,
Klement

Quoting Neale Ranns (nranns) (2017-01-05 15:13:14)
> Hi Klement,
> 
> We’re patching mpls.py – in include nsh.py for reference.
> 
> This run worked:
> 
> -rw-r--r-- 1 nranns nranns918 2017-01-05 08:15:39.151230150 -0500 mpls.py
> -rw-r--r-- 1 nranns nranns   1499 2017-01-05 08:15:39.655234966 -0500 mpls.pyc
> -rw-r--r-- 1 nranns nranns   2737 2017-01-05 08:15:38.631225000 -0500 nsh.py
> -rw-r--r-- 1 nranns nranns   3745 2017-01-05 08:15:38.895227000 -0500 nsh.pyc
> 
> this run failed:
> 
> -rw-r--r-- 1 nranns nranns918 2017-01-05 08:53:46.697139915 -0500 mpls.py
> -rw-r--r-- 1 nranns nranns   1286 2017-01-05 08:53:46.325136000 -0500 mpls.pyc
> -rw-r--r-- 1 nranns nranns   2737 2017-01-05 08:53:46.057133000 -0500 nsh.py
> -rw-r--r-- 1 nranns nranns   3745 2017-01-05 08:53:46.325136000 -0500 nsh.pyc
> 
> Is the time-stamp accuracy only to the nearest second?
> 
> Thanks,
> neale
> 
> 
> On 05/01/2017, 13:04, "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at 
> Cisco)" <ksek...@cisco.com> wrote:
> 
> Neale,
> 
> the byte code should be always updated if out of date on import, if
> it's not so, then that would be a python bug, which seems improbable.
> 
> If you are able to repro this, could you check the timestamps for the
> correspoding .py/.pyc files? Maybe somehow the .py file's timestamp is
> in the past (from .pyc POV)?
> 
> Thanks,
> Klement
> 
> Quoting Neale Ranns (nranns) (2017-01-05 13:39:44)
> >Hi Dave,
> > 
> > 
> > 
> >After a little more experimentation I think this error is related to 
> the
> >way we patch scapy post install. The scapy installation comes with 
> byte
> >code, after we apply the patch this byte code may or may not be 
> updated.
> >If I force the generation of the byte code by either ‘touch
> >/mpls.py’ or ‘find  -name *.pyc –exec rm \;’ then all 
> is
> >well. The find and rm command I can add to the makefile. But there 
> must be
> >a better way.
> > 
> >Ideas? My python noob skills are stretched as it is J
> > 
> > 
> > 
> >Thanks,
> > 
> >neale
> > 
>     >     
> > 
> > 
> > 
> >  From: Dave Wallace <dwallac...@gmail.com>
> >  Date: Wednesday, 4 January 2017 at 18:19
> >  To: "Neale Ranns (nranns)" <nra...@cisco.com>, "Klement Sekera -X
> >  (ksekera - PANTHEON TECHNOLOGIES at Cisco)" <ksek...@cisco.com>, 
> "Maciek
> >  Konstantynowicz (mkonstan)" <mkons...@cisco.com>
> >  Cc: "csit-...@lists.fd.io" <csit-...@lists.fd.io>, 
> "vpp-dev@lists.fd.io"
> >  <vpp-dev@lists.fd.io>
> >  Subject: Re: [csit-dev] [vpp-dev] vpp make test for verify - are we
> >  there yet ? [awoken]
> > 
> >   
> > 
> >  Neale,
> > 
> >  I could not find scapy installed in the normal path (i.e. "which
> >  scapy"), but I'm not sure if it is directly executable.  You can 
> find
> >  the "V=1 TEST=mpls make test" output on
> >  vagrant+virtualbox:puppetlabs/ubuntu-16.04.64-nocm in this pastebin
> >  doc:  [1]http://pastebin.com/uGAUREhK
> > 
> >  Thanks,
> >  -daw-
> > 
> >  On 1/4/2017 10:59 AM, Neale Ranns (nranns) wrote:
> > 
> > 
> > 
> >Hi,
> > 
> > 
> > 
> >The reason the MPLS tests are failing is because scapy cannot 
> decode
> >an MPLS label stack (output from some .show() instrumentation);
> > 
> > 
> > 
> >
> > 
> >###[ Ethernet ]###
> > 
> >  dst   = 02:01:00:00:ff:02
> > 
> >  src   = 02:fe:e5:05:0e:13
> > 
> >  type  = 0x8847
> > 

Re: [vpp-dev] vpp make test for verify - are we there yet ? [awoken]

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Yep, seems to be it.
Not sure how to handle this forward. Testing with python is not ideal
for timing checks...
A better test would be to isolate the BFD node from VPP, mock the VPP
infrastructure (node scheduling) and verify that it is sending the frames
out in a timely fashion (all this in C, emulating CPU clock).
But the effort to do that is magnitudes higher as compared to writing
the python test..

Maciek, Damjan, thoughts?

Thanks,
Klement

Quoting Andrew  Yourtchenko (2017-01-05 14:48:03)
> For me on a not very fast xhyve VM with ubuntu 16.04 (my macbook pro)
> on master the BFD tests appear racy, see below.
> 
> Is that supposed to happen ?
> 
> My results of "make test" on master:
> 
> 
> ==
> ERROR: verify session goes down after inactivity
> --
> Traceback (most recent call last):
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 258, in test_conn_down
> self.bfd_session_up()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 199, in bfd_session_up
> p, ttp = self.wait_for_bfd_packet()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 184, in wait_for_bfd_packet
> p = self.pg0.wait_for_packet(timeout=timeout)
>   File "/home/ubuntu/vpp/test/vpp_pg_interface.py", line 271, in 
> wait_for_packet
> "timeout" % self.out_path)
> Exception: Capture file
> /tmp/vpp-unittest-BFDTestCase-9GSnEV/pg0_out.pcap did not appear
> within timeout
> 
> ==
> ERROR: hold BFD session up
> --
> Traceback (most recent call last):
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 251, in test_hold_up
> self.bfd_session_up()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 199, in bfd_session_up
> p, ttp = self.wait_for_bfd_packet()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 184, in wait_for_bfd_packet
> p = self.pg0.wait_for_packet(timeout=timeout)
>   File "/home/ubuntu/vpp/test/vpp_pg_interface.py", line 271, in 
> wait_for_packet
> "timeout" % self.out_path)
> Exception: Capture file
> /tmp/vpp-unittest-BFDTestCase-9GSnEV/pg0_out.pcap did not appear
> within timeout
> 
> ==
> ERROR: immediately honor remote min rx reduction
> --
> Traceback (most recent call last):
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 295, in
> test_immediate_remote_min_rx_reduce
> self.wait_for_bfd_packet()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 184, in wait_for_bfd_packet
> p = self.pg0.wait_for_packet(timeout=timeout)
>   File "/home/ubuntu/vpp/test/vpp_pg_interface.py", line 297, in 
> wait_for_packet
> raise Exception("Packet didn't arrive within timeout")
> Exception: Packet didn't arrive within timeout
> 
> ==
> ERROR: bring BFD session up
> --
> Traceback (most recent call last):
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 213, in test_session_up
> self.bfd_session_up()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 199, in bfd_session_up
> p, ttp = self.wait_for_bfd_packet()
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 184, in wait_for_bfd_packet
> p = self.pg0.wait_for_packet(timeout=timeout)
>   File "/home/ubuntu/vpp/test/vpp_pg_interface.py", line 271, in 
> wait_for_packet
> "timeout" % self.out_path)
> Exception: Capture file
> /tmp/vpp-unittest-BFDTestCase-9GSnEV/pg0_out.pcap did not appear
> within timeout
> 
> ==
> FAIL: verify slow periodic control frames while session down
> --
> Traceback (most recent call last):
>   File "/home/ubuntu/vpp/test/test_bfd.py", line 228, in test_slow_timer
> after - before, 0.70, 1.05, "time between slow packets")
>   File "/home/ubuntu/vpp/test/framework.py", line 542, in assert_in_range
> self.assertTrue(expected_min <= real_value <= expected_max, msg)
> AssertionError: Invalid time between slow packets: 1.12259793282 out
> of range <0.7,1.05>
> 
> ==
> FAIL: Load Balancer IP4 GRE4
> --
> Traceback (most recent call last):
>   File "/home/ubuntu/vpp/test/test_lb.py", line 159, in test_lb_ip4_gre4
> self.checkCapture(gre4=True, isv4=True)
>   File "/home/ubuntu/vpp/test/test_lb.py", line 98, in checkCapture
> self.pg0.assert_nothing_captured()
>   File "/home/ubuntu/vpp/test/vpp_pg_interface.py", line 228, in
> assert_nothing_captured
> self.name)
> 

Re: [vpp-dev] [csit-dev] vpp make test for verify - are we there yet ? [awoken]

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Neale,

the byte code should be always updated if out of date on import, if
it's not so, then that would be a python bug, which seems improbable.

If you are able to repro this, could you check the timestamps for the
correspoding .py/.pyc files? Maybe somehow the .py file's timestamp is
in the past (from .pyc POV)?

Thanks,
Klement

Quoting Neale Ranns (nranns) (2017-01-05 13:39:44)
>Hi Dave,
> 
> 
> 
>After a little more experimentation I think this error is related to the
>way we patch scapy post install. The scapy installation comes with byte
>code, after we apply the patch this byte code may or may not be updated.
>If I force the generation of the byte code by either ‘touch
>/mpls.py’ or ‘find  -name *.pyc –exec rm \;’ then all is
>well. The find and rm command I can add to the makefile. But there must be
>a better way.
> 
>Ideas? My python noob skills are stretched as it is J
> 
> 
> 
>Thanks,
> 
>neale
> 
> 
> 
> 
> 
>  From: Dave Wallace <dwallac...@gmail.com>
>  Date: Wednesday, 4 January 2017 at 18:19
>      To: "Neale Ranns (nranns)" <nra...@cisco.com>, "Klement Sekera -X
>  (ksekera - PANTHEON TECHNOLOGIES at Cisco)" <ksek...@cisco.com>, "Maciek
>  Konstantynowicz (mkonstan)" <mkons...@cisco.com>
>  Cc: "csit-...@lists.fd.io" <csit-...@lists.fd.io>, "vpp-dev@lists.fd.io"
>  <vpp-dev@lists.fd.io>
>  Subject: Re: [csit-dev] [vpp-dev] vpp make test for verify - are we
>  there yet ? [awoken]
> 
>   
> 
>  Neale,
> 
>  I could not find scapy installed in the normal path (i.e. "which
>  scapy"), but I'm not sure if it is directly executable.  You can find
>  the "V=1 TEST=mpls make test" output on
>  vagrant+virtualbox:puppetlabs/ubuntu-16.04.64-nocm in this pastebin
>  doc:  [1]http://pastebin.com/uGAUREhK
> 
>  Thanks,
>  -daw-
> 
>  On 1/4/2017 10:59 AM, Neale Ranns (nranns) wrote:
> 
> 
> 
>Hi,
> 
> 
> 
>The reason the MPLS tests are failing is because scapy cannot decode
>an MPLS label stack (output from some .show() instrumentation);
> 
> 
> 
>
> 
>###[ Ethernet ]###
> 
>  dst   = 02:01:00:00:ff:02
> 
>  src   = 02:fe:e5:05:0e:13
> 
>  type  = 0x8847
> 
>###[ MPLS ]###
> 
> label = 33L
> 
> cos   = 0L
> 
> s = 0L   <<<<<< NON End-of-stack
> 
> ttl   = 254
> 
>###[ Padding ]###
> 
>load  = '\x00\x061\xffE\x00\x00#\x00\x01\x00\x00@\x11
>\xa6\xac\x10\x01\x01\xac\x10\x01\x02\x04\xd2\x04\xd2\x00\x0f\xd0\x92257
>1 1'
> 
> 
> 
>we patch scapy for this purpose, see;
> 
>  /test/patches/scapy-2.3.3/mpls.py.patch
> 
> 
> 
>on my vagrant 14.04 there is no other scapy installation, this patch
>applies fine and all MPLS tests pass.
> 
>On a UCS 14.04, with another scapy installation:
> 
>  /usr/local/lib/python2.7/dist-packages/scapy/contrib/mpls.py
> 
>the MPLS test fail.
> 
>    On a UCS 16.04 no scpay installation, tests pass.
> 
> 
> 
>Do you have scapy installed in the machines on which the tests
>fail/pass? Or am I barking up the wrong virtualenv tree J
> 
> 
> 
>Thanks,
> 
>neale
> 
> 
> 
> 
> 
> 
> 
>  From: Dave Wallace [2]<dwallac...@gmail.com>
>  Date: Wednesday, 4 January 2017 at 15:38
>  To: "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)"
>  [3]<ksek...@cisco.com>, "Maciek Konstantynowicz (mkonstan)"
>  [4]<mkons...@cisco.com>, "Neale Ranns (nranns)"
>  [5]<nra...@cisco.com>
>  Cc: [6]"csit-...@lists.fd.io" [7]<csit-...@lists.fd.io>,
>  [8]"vpp-dev@lists.fd.io" [9]<vpp-dev@lists.fd.io>
>  Subject: Re: [csit-dev] [vpp-dev] vpp make test for verify - are we
>  there yet ? [awoken]
> 
>           
> 
>  Klement,
> 
>      I'm in the process of modifying the base Vagrantfile/build scripts
>  to replace building the VPP native package and installing it in the
>  VM to simply running "make test" which takes much less time.  All of
>  the test ru

Re: [vpp-dev] [csit-dev] vpp make test for verify - are we there yet ? [awoken]

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Dave, there is no sign of test framework installing/patching scapy in
the log you provided.

Could you please do in the same workspace

make test-wipe

and then run the make test again? I don't think it'll actually help us,
because there should be no way for a python inside virtualenv to get its
hands on outside libs (unless there is a bug of course), but let's see
what happens...

Thanks,
Klement

Quoting Dave Wallace (2017-01-04 19:19:04)
>Neale,
> 
>I could not find scapy installed in the normal path (i.e. "which scapy"),
>but I'm not sure if it is directly executable.  You can find the "V=1
>TEST=mpls make test" output on
>vagrant+virtualbox:puppetlabs/ubuntu-16.04.64-nocm in this pastebin doc: 
>[1]http://pastebin.com/uGAUREhK
> 
>Thanks,
>-daw-
> 
>On 1/4/2017 10:59 AM, Neale Ranns (nranns) wrote:
> 
>   
> 
>  Hi,
> 
>   
> 
>  The reason the MPLS tests are failing is because scapy cannot decode an
>  MPLS label stack (output from some .show() instrumentation);
> 
>   
> 
>  
> 
>  ###[ Ethernet ]###
> 
>    dst   = 02:01:00:00:ff:02
> 
>    src   = 02:fe:e5:05:0e:13
> 
>    type  = 0x8847
> 
>  ###[ MPLS ]###
> 
>   label = 33L
> 
>   cos   = 0L
> 
>   s = 0L   <<<<<< NON End-of-stack
> 
>   ttl   = 254
> 
>  ###[ Padding ]###
> 
>  load  = '\x00\x061\xffE\x00\x00#\x00\x01\x00\x00@\x11
>  \xa6\xac\x10\x01\x01\xac\x10\x01\x02\x04\xd2\x04\xd2\x00\x0f\xd0\x92257
>  1 1'
> 
>   
> 
>  we patch scapy for this purpose, see;
> 
>    /test/patches/scapy-2.3.3/mpls.py.patch
> 
>   
> 
>  on my vagrant 14.04 there is no other scapy installation, this patch
>  applies fine and all MPLS tests pass.
> 
>  On a UCS 14.04, with another scapy installation:
> 
>    /usr/local/lib/python2.7/dist-packages/scapy/contrib/mpls.py
> 
>  the MPLS test fail.
> 
>  On a UCS 16.04 no scpay installation, tests pass.
> 
>   
> 
>  Do you have scapy installed in the machines on which the tests
>  fail/pass? Or am I barking up the wrong virtualenv tree J
> 
>   
> 
>  Thanks,
> 
>  neale
> 
>   
> 
>   
> 
>   
> 
>From: Dave Wallace [2]<dwallac...@gmail.com>
>Date: Wednesday, 4 January 2017 at 15:38
>To: "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)"
>[3]<ksek...@cisco.com>, "Maciek Konstantynowicz (mkonstan)"
>[4]<mkons...@cisco.com>, "Neale Ranns (nranns)" [5]<nra...@cisco.com>
>Cc: [6]"csit-...@lists.fd.io" [7]<csit-...@lists.fd.io>,
>[8]"vpp-dev@lists.fd.io" [9]<vpp-dev@lists.fd.io>
>Subject: Re: [csit-dev] [vpp-dev] vpp make test for verify - are we
>there yet ? [awoken]
> 
> 
> 
>Klement,
> 
>I'm in the process of modifying the base Vagrantfile/build scripts to
>replace building the VPP native package and installing it in the VM to
>simply running "make test" which takes much less time.  All of the
>test runs that I have performed are clean builds on a virgin git clone
>repo.
> 
>That being said, I'll give the "git clean -dfX */" a try and see if I
>get different results on the 2nd pass and report back.
> 
>Thanks,
>-daw-
> 
>On 1/4/17 10:23 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES
>at Cisco) wrote:
> 
>  Hi Dave,
> 
>   
> 
>  could you please try running git clean -dfX */ in a workspace, where you see
> 
>  the failures, and then givet it another try? For me this seems to clear 
> issues
> 
>  up. Damjans recent changes might have caused (quoting Damjan) "major
> 
>  disturbances in the force", leaving build artifacts behind.
> 
>   
> 
>  Just be sure to stash/commit any changes you have in the workspace since
> 
>  the git clean nukes everything unknown from the ws.
> 
>   
> 
>  Thanks,
> 
>  Klement
> 
>   
> 
>  Quoting Dave Wallace (2017-01-04 16:12:38)
> 
>     As I mentioned on the VPP call yesterday, I'm seeing what appears to be
> 
>     the same GRE test failures on Ubuntu 16.10 (bare metal).  I'm also seeing
> 
>     the same failures on vagrant+virtualbox with
> 
>     puppetlabs/ubuntu-16.04.64-nocm. I also am getting failures on 3 out of 4
> 
> 

Re: [vpp-dev] [csit-dev] vpp make test for verify - are we there yet ? [awoken]

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

gt;   File "/vpp/test/test_bfd.py", line 187, in test_slow_timer
> self.wait_for_bfd_packet()
>   File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet
> p = self.pg0.wait_for_packet(timeout=timeout)
>   File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet
> self.wait_for_capture_file(timeout)
>   File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file
> raise Exception("Capture file did not appear within timeout")
>  Exception: Capture file did not appear within timeout
> 
>  ==
>  ERROR: no packets when zero BFD RemoteMinRxInterval
>  --
>  Traceback (most recent call last):
>   File "/vpp/test/test_bfd.py", line 201, in test_zero_remote_min_rx
> p = self.wait_for_bfd_packet()
>   File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet
>     p = self.pg0.wait_for_packet(timeout=timeout)
>   File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet
> self.wait_for_capture_file(timeout)
>   File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file
> raise Exception("Capture file did not appear within timeout")
>  Exception: Capture file did not appear within timeout
> 
>  Ran 64 tests in 84.161s
> 
> 
> 
> 
> 
> 
>  On 04/01/2017, 09:45, [2]"csit-dev-boun...@lists.fd.io on behalf of Klement 
> Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)" 
> [3]<csit-dev-boun...@lists.fd.io on behalf of ksek...@cisco.com> wrote:
> 
> I tried doing a git checkout origin/stable/1701 - is that the correct
> branch? On that one, the GRE indeed doesn't work, but I don't think it's
> 'make test' issue, here's a sample from packet trace:
> 
> --- Start of thread 0 vpp_main ---
> Packet 1
> 
> 00:00:07:119382: pg-input
>   stream pcap1, 88 bytes
>   current data 0, length 88, free-list 6, trace 0x0
>   IP4: 02:01:00:00:ff:02 -> 02:fe:39:17:74:65
>   GRE: 2.2.2.2 -> 172.16.1.1
> tos 0x00, ttl 64, length 74, checksum 0xc96f
> fragment id 0x0001
>   GRE 0x0001
> 00:00:07:119802: ethernet-input
>   IP4: 02:01:00:00:ff:02 -> 02:fe:39:17:74:65
> 00:00:07:120048: ip4-input
>   GRE: 2.2.2.2 -> 172.16.1.1
> tos 0x00, ttl 64, length 74, checksum 0xc96f
> fragment id 0x0001
>   GRE 0x0001
> 00:00:07:120113: ip4-lookup
>   fib 0 dpo-idx 5 flow hash: 0x
>   GRE: 2.2.2.2 -> 172.16.1.1
> tos 0x00, ttl 64, length 74, checksum 0xc96f
> fragment id 0x0001
>   GRE 0x0001
> 00:00:07:120404: ip4-local
> GRE: 2.2.2.2 -> 172.16.1.1
>   tos 0x00, ttl 64, length 74, checksum 0xc96f
>   fragment id 0x0001
> GRE 0x0001
> 00:00:07:120495: gre-input
>   GRE: tunnel 0 len 74 src 2.2.2.2 dst 172.16.1.1
> 00:00:07:120543: error-drop
>   gre-input: unknown protocol
> 
> all 50 packets printed in the packet trace share the same fate...
> 
> Thanks,
> Klement
> 
> Quoting Maciek Konstantynowicz (mkonstan) (2017-01-03 18:46:19)
> 
>// Typed this email before the live discussion on vpp call just now -
>still sending it out :)
>Hello, After the previous ver of this thread went into a frenzy of emails
>and fixes in vpp make test code,
>I wanted to re-check the situation.
>And as it is a New Year, I’m not lazy anymore, and did run make test in
>both vpp branches :)
>vpp master branch works, stable/1701 does not..
>1. master
>Ran 65 tests in 90.838s
>OK (skipped=5)
>make[1]: Leaving directory `/home/maciek/src/vpp2/test'
>2. stable/1701
>  ==
>  ERROR: GRE tunnel Tests
>  Exception: Capture file did not appear within timeout
>  ==
>  ERROR: GRE tunnel L2 Tests
>  Exception: Capture file did not appear within timeout
>  ==
>  ERROR: MPLS Local Label Binding test
>  AttributeError: label
>  ==
>  ERROR: MPLS label imposition test
>  IndexError: Layer [IP] not found
>  ==
>  ERROR: MPLS label swap tests
>

Re: [vpp-dev] Regarding S-BFD and testing of BFD in VPP

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Loval,

I skimmed through the RFC 7882 and it shouldn't be too much trouble adding
that functionality to VPP. The existing BFD code is still a work in
progress and the only reason it's been pushed to the git was the API
freeze, so please do expect more patches to be pushed and things could
change.

Currently, there is no CLI, only API access, which is partially
implemented. The global configuration APIs are not implemented yet,
only per-session ones: bfd_udp_add, bfd_udp_del, bfd_udp_session_dump.

I suggest you take a look at test/test_bfd.py, which contains the unit
tests for the BFD feature. Again, this is not complete and more tests
will follow.

To run the tests, you can do e.g.

make test-debug TEST=bfd V=2

(adjust test verbosity V= to your liking, 2 is most verbose, 0 least)

The VPP BFD code is roughly split into BFDs-specific functionality and
transport-specific functionality - that being UDP only at this time,
while thinking about e.g. MPLS in the future.

Thanks,
Klement

Quoting Lovaljeet Kaur (2016-12-15 08:23:32)
>Adding  vpp-dev@lists.fd.io dev mailing list
> 
>-Lovaljeet Kaur/DEL/TCS wrote: -
>To: ksek...@cisco.com
>From: Lovaljeet Kaur/DEL/TCS
>Date: 12/15/2016 12:22PM
>Cc: Deepankar Gupta/DEL/TCS@TCS
>Subject: Regarding S-BFD and testing of BFD in VPP
> 
>Hi Klement,
> 
>I want to implement Seamless BFD (S-BFD) in VPP over the BFD code that you
>have submitted.
>So i would want to understand and test the existing BFD code.
>Can you please guide me with the testing scenarios on how we can test BFD
>or is there any Debug CLI in VPP through which we can test.
> 
>Regards,
>Loval
> 
>=-=-=
>Notice: The information contained in this e-mail
>message and/or attachments to it may contain
>confidential or privileged information. If you are
>not the intended recipient, any dissemination, use,
>review, distribution, printing or copying of the
>information contained in this e-mail message
>and/or attachments to it are strictly prohibited. If
>you have received this communication in error,
>please notify us by reply e-mail or telephone and
>immediately and permanently delete the message
>and any attachments. Thank you
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Regarding multiple Instances of VVP

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Yes, you can, using shared memory prefix.

https://wiki.fd.io/view/VPP/Command-line_Arguments#.22api-segment.22_parameters

Also the tests invoked using "make test" are using this feature.
(tests/framework.py: setUpConstants(): vpp_cmdline =...)

Regards,
Klement

On Tue, Nov 22, 2016 at 03:41:19PM +0530, sudhanshu wrote:
> Hello,
> 
> Can we run multiple VPP instances on one machine without using Virtual
> Machine ?
> 
> 
> Thanks & Best Regards
> 
> Sudhanshu Kumar Srivastav

> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Creating vpp-test-verify job ... - it's time I think !

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi Pierre,

I've enabled the tests and remove the checkes from the ipv6 branch in
checkCapture as suggested.

Now I still see failures, even a vpp crash.

Please see the attached diff and let me know if I need to do something
else.

Attached also the output of the test case..

Thanks,
Klement

On Tue, Nov 08, 2016 at 04:16:18PM +, Pierre Pfister (ppfister) wrote:
> Right.
> 
> Klement Sekera was kindly trying to make testing with IPv6 and GRE6 work (It 
> was partially tested, but there are issues with scapy).
> Tests are failing because he enabled those tests.
> I will suggest on the gerrit patch that he rolls-back to the previous test 
> situation.
> 
> - Pierre
> 
> 
> > Le 8 nov. 2016 à 16:37, Maciek Konstantynowicz (mkonstan) 
> >  a écrit :
> > 
> > Oups… Agree. I know someone was working on it.. but it’s still failing.. 
> > And I don’t know how much work to make it stop doing so..
> > If you could help that would be great.
> > 
> > $ git blame test/test_lb.py
> > 
> > -Maciek
> > 
> >> On 8 Nov 2016, at 15:23, Pierre Pfister (ppfister)  
> >> wrote:
> >> 
> >> Hello Maciek,
> >> 
> >> Do you know what is going wrong with the LB test case ?
> >> Nobody bothered to tell me the test was failing.
> >> It would be cool to have a chance to fix it rather than just disabling it.
> >> 
> >> Thanks,
> >> 
> >> - Pierre
> >> 
> >> 
> >> 
> >>> Le 8 nov. 2016 à 15:57, Maciek Konstantynowicz (mkonstan) 
> >>>  a écrit :
> >>> 
> >>> Hi,
> >>> 
> >>> Following from recent discussions on weekly VPP call, IMV the test 
> >>> framework and 
> >>> tests in master branch are ready to be used regularly on a per patch 
> >>> basis.
> >>> 
> >>> Hence I propose someone to create a vpp-test-verify* job(s) to do exactly 
> >>> that.
> >>> Ed, could you do this or assist some other volunteer to do this?
> >>> For ubuntu1404, ubuntu1604, centos7, etc.
> >>> 
> >>> The only failing test due to unfinished port to somewhat refactored test 
> >>> framework is test_lb.py,
> >>> and Klement just pushed a patch that will skip this test: 
> >>> https://gerrit.fd.io/r/#/c/3726/
> >>> 
> >>> Thoughts?
> >>> 
> >>> -Maciek
> >>> 
> >>> P.S. Current tests in vpp master branch are: 
> >>> $ ls test
> >>> doclog.py scapy_handlers   test_ip6.pyc   
> >>> test_l2xc.py   test_mpls.pyc   vpp_interface.py   vpp_pg_interface.pyc
> >>> framework.py   log.pyctemplate_bd.py   test_ip.py 
> >>> test_l2xc.pyc  test_vxlan.py   vpp_interface.pyc  vpp_sub_interface.py
> >>> framework.pyc  Makefile   template_bd.pyc  test_ip.pyctest_lb.py  
> >>>test_vxlan.pyc  vpp_papi_provider.py   vpp_sub_interface.pyc
> >>> hook.pyrun_tests.py   test_infra.mdtest_l2bd.py   test_lb.pyc 
> >>>util.py vpp_papi_provider.pyc
> >>> hook.pyc   run_tests.pyc  test_ip6.py  test_l2bd.pyc  
> >>> test_mpls.py   util.pycvpp_pg_interface.py
> >>> 
> >>> $ make test
> >>> 
> >>> ==
> >>> IPv6 Test Case
> >>> ==
> >>> IPv6 FIB test   OK
> >>> ==
> >>> MPLS Test Case
> >>> ==
> >>> MPLS V4 Explicit NULL test  OK
> >>> MPLS V6 Explicit NULL test  OK
> >>> ==
> >>> IPv4 Test Case
> >>> ==
> >>> IPv4 FIB test   OK
> >>> ==
> >>> L2XC Test Case
> >>> ==
> >>> L2XC test   OK
> >>> ==
> >>> VXLAN Test Case
> >>> ==
> >>> Decapsulation test  OK
> >>> Encapsulation test  OK
> >>> ==
> >>> L2BD Test Case
> >>> ==
> >>> L2BD MAC learning test  OK
> >>> ___
> >>> vpp-dev mailing list
> >>> vpp-dev@lists.fd.io
> >>> https://lists.fd.io/mailman/listinfo/vpp-dev
> > 
> 
> ___
> vpp-dev mailing list
> vpp-dev@lists.fd.io
> https://lists.fd.io/mailman/listinfo/vpp-dev
diff --git a/test/test_lb.py b/test/test_lb.py
index 9f39361..1268543 

Re: [vpp-dev] L2BD test failing during MAC learning - VPP-518

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi all,

regarding the discussed changes, here's a patch:

https://gerrit.fd.io/r/#/c/3641/

please do check out:

make test-help

to get a list of knobs to turn with a brief explanation. I got rid of
all the different one-letter switches (kept only the familiar
V=) plus added a stepping feature requested by Neale.

Regards,
Klement

On Fri, Oct 28, 2016 at 08:24:45PM +, Klement Sekera -X (ksekera - PANTHEON 
TECHNOLOGIES at Cisco) wrote:
> Hi Dave,
> 
> I completely agree with you regarding the gdb server pains. I remember
> the horrible IOS-XR gdb server experience, like 'step' taking 10
> seconds, breakpoints not working randomly and so.
> 
> But - on x86[_64] I think it's usable. And I think that this is the main
> selling point. The 'make test' is meant mainly as a developer tool. Make
> writing unit tests easy and fun and developers will write them even if
> they don't have to because it saves them time and having to repeat mundane 
> tasks.
> E.g. now that I'm working on the BFD - it's actually much more faster
> to use the python with scapy for repeated test runs as compared to say
> setting up a nexus to be the bfd peer and (re)spawning vpp after each crash,
> configuring it again etc.  This way I just punch a magic button and voila,
> the test case does this all for me. Added bonus - we get unit tests.
> 
> There is no need to run the UT framework as root. Actually, that was one
> of the first things I did (remove the sudo mess). But on default ubuntu,
> this means that you either need to make some system changes or you need
> to run gdb as root.
> 
> Regarding dependencies - you can live in your own sandbox. Think
> --prefix or so. That's how I actually got a usable vim (on aurora).
> And clang - after building my own libc and gcc of course. Etc.
> It's pain, but one-time pain and it can be done if you have the willpower.
> Or if you don't have any other way :-)
> 
> So - let's keep both options - it's not that much code and lets everybody
> use what they like.
> 
> Thanks,
> Klement
> 
> On Fri, Oct 28, 2016 at 09:34:02PM +0200, Dave Barach (dbarach) wrote:
> >Dear Klement,
> > 
> > 
> > 
> >The G=1 story works for me, that’d be fine.
> > 
> > 
> > 
> >I hate gdbserver based on years’ worth of experience. Here’s problem 1:
> >run gdb + gdbserver on ppc64-be, aarch64-le and so forth. If it's your
> >lucky day, it will just about work on one CPU architecture other than
> >x86_84. Mix in a half-baked experimental vendor tool chain, and I can all
> >but guarantee that your mileage will vary.
> > 
> > 
> > 
> >Problem 2: performance, especially when using stream sockets [TCP] 
> > instead
> >of Unix domain sockets.  
> > 
> > 
> > 
> >Why is it necessary for the UT framework to run vpp-lite as root? I run
> >vpp-lite images as uid=1000 all the time. Let’s see if we can figure out
> >how to avoid running as root...
> > 
> > 
> > 
> >Segway into developers using systems without root access: how could that
> >work? Someone stuck in that situation wouldn’t be able to install build
> >dependencies and build an image; forget all about the unit test 
> > framework.
> > 
> >     
> > 
> >Aurora systems [a Cisco IT internal PAAS offering] are more or less
> >unusable for any kind of vpp work. Cisco IT is not going to let you
> >install anything.   
> > 
> > 
> > 
> >Thanks… Dave
> > 
> > 
> > 
> >-Original Message-
> >From: Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
> >Sent: Friday, October 28, 2016 1:59 PM
> >To: Dave Barach (dbarach) <dbar...@cisco.com>
> >Cc: Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco)
> ><jgel...@cisco.com>; vpp-dev@lists.fd.io; csit-...@lists.fd.io
> >Subject: Re: [vpp-dev] L2BD test failing during MAC learning - VPP-518
> > 
> > 
> > 
> >Hi Dave,
> > 
> > 
> > 
> >the reason why I went the gdbserver way is this:
> > 
> > 
> > 
> >Attaching to program:
> > 
> >/home/ksekera/vpp/build-root/install-vpp_lite_debug-native/vpp/bin/vpp,
> > 
> >process 7101
> > 
> >Could not attach to process.  If your uid matches the uid of the target
> > 
> >process, check the setting of /proc/sys/kernel/yama/ptrace_scope, or try
> > 
> >again as the root user.  For more details, see
> > 
> >/etc/sysctl.d/10-ptrace.conf
>

[vpp-dev] duplicate vnet api errno

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

Hi all,

is this kosher? If yes how does the caller find out which return value
is applicable for his API?

ksekera@zglab-host-2 ~/v/v/vnet> grep '\-63' api_errno.h
_(ADDRESS_MATCHES_INTERFACE_ADDRESS, -63, "Address matches interface
address") \
_(NO_SUCH_NODE, -63, "No such graph node")
\

Thanks,
Klement
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev