[vpp-dev] dpdk plugin: RSS keys for ICE

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Executive summary:
Jira ticket VPP-2072 [4] describes how random RSS keys
make some performance test results too random to be useful.
This vpp-dev thread can be used for discussing ways
how to make CSIT results deterministic.

Longer version:
As far as I can tell, dpdk plugin + ICE driver
is the only combination of VPP plugin and NIC driver
(of those currently tested in CSIT) that uses random RSS keys.
As the goal of CSIT performance testing is to detect performance changes,
we want to avoid any avoidable randomness.

Previously, AVF plugin had the same issue,
and it was changed [5] to use a static RSS key.
Additionally, it was the same key as dpdk plugin with i40e driver uses,
which means the performance comparison of the two plugins is now fairer.
(The distribution of traffic flows over VPP workers is now the same.)

We (CSIT) want to make dpdk plugin use a static RSS key also with ICE driver,
but it is not clear how we (VPP) should implement it.
Some options:
1. Add a patch file (to build/external/patches/).
2. Leverage a more official DPDK API (if available).
3. Wait for next DPDK version with a better support.

If we want to offer users the choice (random, static, or even user provided RSS 
key),
the question is how users would specify it (startup.conf, binary API, …).

I imagine other apps (not just VPP dpdk plugin) may also want deterministic RSS,
so a long-term solution will be in DPDK code
(that is why I added bunch of DPDK people to To and Cc).
In the meantime, VPP can apply any medium-term workaround.

Note that VPP 22.03 is already released,
and the primary CSIT testbed affected (3n-snr) has hyperthreading off,
so 1-core tests are not affected.
Other testbeds usually do not test with small number of flows
or at least offer a different NIC.
All that means this issue is not a high priority.

Vratko.

[4] https://jira.fd.io/browse/VPP-2072
[5] https://gerrit.fd.io/r/c/vpp/+/33705/2/src/plugins/avf/device.c

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#22663): https://lists.fd.io/g/vpp-dev/message/22663
Mute This Topic: https://lists.fd.io/mt/97361460/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] request-response between vlib processes

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> why we ended up needing such a complicated procedure to apply what looked 
> like simple updates.

All I can tell is that avf_process was preferring to suspend itself
(rather than busy wait) already in the first AVF commit [13],
and later the wait times got even longer [14].

> I’m pretty sure we could mark things as down and program an asyc cleanup from 
> within the avf layer.

Probably yes, but the avf layer should keep track of the removal,
so it knows to be careful on subsequent add.
I guess the current way is safer, especially if we want to panic early on any 
error.

> it would be good if we could avoid suspending the current process in avf
> because it can’t know all the ways in which the calling process could be 
> signaled.

Yes, requiring each process to exhibit only one agent behavior would fix the 
issue,
avf_send_to_pf could call vlib_time_now in a loop (instead of suspending 
avf_process).

But I think VPP could easily offer some support for multiple agent behaviors,
for example by allowing processes to list which event types should be able to 
wake them
(so vl_api_clnt_process in avf_process_request will not wake up upon 
SOCKET_READ_EVENT).

Vratko.

[13] https://gerrit.fd.io/r/c/vpp/+/10457/41/src/plugins/avf/device.c#377
[14] https://gerrit.fd.io/r/c/vpp/+/21831/3/src/plugins/avf/device.c#415

From: vpp-dev@lists.fd.io  On Behalf Of Florin Coras
Sent: Tuesday, 2022-September-13 19:54
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] request-response between vlib processes

Hi Vratko,


On Sep 13, 2022, at 5:03 AM, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES 
at Cisco) via lists.fd.io<http://lists.fd.io> 
mailto:vrpolak=cisco@lists.fd.io>> wrote:

In general, most of “communication” between VPP components
is done by directly calling C functions,
so it makes sense avf_flag_change is being called within vl_api_clnt_process 
process.
It is avf_process_request (called directly by avf_flag_change)
that decides to hand-off the request to avf_process process for async handling,
so it should make sure to resume the API process correctly upon the response.

> just to set a mac address?

In my particular test the async operation switches promiscuous mode on an 
interface,
but I guess it does not really matter what a particular operation does.
What matters is there is a synchronous API call (l2_patch_add_del in my test)
which only indirectly causes an asynchronous operation (as the interface uses 
AVF driver).


Didn’t have an issue with how the api ends up calling avf_process_request. I 
was just wondering why we ended up needing such a complicated procedure to 
apply what looked like simple updates.


> Do we really need to block the binary api

The l2_patch_add_del does block.
Especially in the “del” case, the subsequent API calls
need to know whether the interface is gone yet or not.

I’m pretty sure we could mark things as down and program an asyc cleanup from 
within the avf layer. That is, if async is necessary, for deletes we should be 
able to provide a return code as soon as we find that the device/state exists 
and program the removal.

But for adds, it would be good if we could avoid suspending the current process 
in avf because it can’t know all the ways in which the calling process could be 
signaled.


> pass opaques in requests

As usual, there are several ways to make it work,
we just need to pick one (and put an example usage into the docs).

And I believe that’s what we’re discussing here :-)

Florin



Vratko.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Florin Coras
Sent: Monday, 2022-September-12 23:11
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] request-response between vlib processes

Hi Vratko,

Do we really need to block the binary api waiting for a reply from another vpp 
process just to set a mac address?

If setting up the mac (or similar) cannot be done synchronously, probably api 
handlers should hand over all those requests to another vpp process, 
vl_api_async_req_process, that takes care of async execution and generation of 
api replies. You could also pass opaques in requests and maybe expect backends, 
like avf_process, to bounce that opaques back for demuxing.

Regards,
Florin



On Sep 12, 2022, at 4:49 AM, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES 
at Cisco) vialists.fd.io<http://lists.fd.io/> 
mailto:vrpolak=cisco@lists.fd.io>> wrote:

[resending to the correct vpp-dev e-mail address]

Short version:
Vratko would appreciate something like 
vlib_current_process_wait_for_one_time_event_or_clock.

Medium version:
One instance of request-response interaction between vlib processes had a bug 
[11].
Vratko contributed a fix [9] for the immediate issue,
but the proper fix was left hinted in TODOs (and discussed in the long version).

Long version:

Vlib supports processes and signals, see cor

Re: [vpp-dev] benchmark for a patch

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> a specific patch which is not merged?

It depends whether the unmerged patch is for VPP or CSIT (or you have two).

1. If VPP code is merged, various csit-verify-perf-* jobs can be triggered
using csit-{node}-{arch}-perftest trigger word in CSIT Gerrit comment.
Example: [0].
The VPP version used is specified in the CSIT code in 
VPP_STABLE_VER_UBUNTU_JAMMY,
and it must be available in packagecloud.

2. If CSIT code is merged (needs to be in the latest oper branch),
vpp-csit-perf-* jobs can be triggered using perftest-{node}-{arch} word
in VPP Gerrit comment. Example: [1]. More detailed documentation: [2].

3. If both VPP and CSIT code is unmerged, vpp-csit-verify-perf-* job
can be started manually in Jenkins WebUI, but only if you have access rights 
for that.
Try if this [3] link lets you (after logging into Jenkins) specify CSIT_REF to 
be used.
Not sure how to get access rights nowadays, I assume it can be requested 
somewhere here [4].

Vratko.

[0] 
https://gerrit.fd.io/r/c/csit/+/36787/1#message-ead2f6d770c8d741e5bf4933e5bf215702550467
[1] 
https://gerrit.fd.io/r/c/vpp/+/36707/6#message-b71fe7ef4d8f54a7c478f90aa5b50a9d6fd6718f
[2] 
https://github.com/FDio/csit/blob/8d85a976dc8bd2c960044b32e830eb97f63f5ffe/docs/report/introduction/methodology_rca/methodology_perpatch_performance_tests.rst
[3] 
https://jenkins.fd.io/view/vpp/job/vpp-csit-verify-perf-master-ubuntu2204-x86_64-2n-icx/build?delay=0sec
[4] http://support.linuxfoundation.org/

From: vpp-dev@lists.fd.io  On Behalf Of Stanislav Zaikin
Sent: Thursday, 2022-September-01 17:19
To: vpp-dev 
Subject: [vpp-dev] benchmark for a patch

Hello folks,

Is it possible to trigger a benchmark (csit, only 1 test case) for a specific 
patch which is not merged?

--
Best regards
Stanislav Zaikin

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21874): https://lists.fd.io/g/vpp-dev/message/21874
Mute This Topic: https://lists.fd.io/mt/93397909/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] request-response between vlib processes

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

In general, most of “communication” between VPP components
is done by directly calling C functions,
so it makes sense avf_flag_change is being called within vl_api_clnt_process 
process.
It is avf_process_request (called directly by avf_flag_change)
that decides to hand-off the request to avf_process process for async handling,
so it should make sure to resume the API process correctly upon the response.

> just to set a mac address?

In my particular test the async operation switches promiscuous mode on an 
interface,
but I guess it does not really matter what a particular operation does.
What matters is there is a synchronous API call (l2_patch_add_del in my test)
which only indirectly causes an asynchronous operation (as the interface uses 
AVF driver).

> Do we really need to block the binary api

The l2_patch_add_del does block.
Especially in the “del” case, the subsequent API calls
need to know whether the interface is gone yet or not.

> pass opaques in requests

As usual, there are several ways to make it work,
we just need to pick one (and put an example usage into the docs).

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Florin Coras
Sent: Monday, 2022-September-12 23:11
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] request-response between vlib processes

Hi Vratko,

Do we really need to block the binary api waiting for a reply from another vpp 
process just to set a mac address?

If setting up the mac (or similar) cannot be done synchronously, probably api 
handlers should hand over all those requests to another vpp process, 
vl_api_async_req_process, that takes care of async execution and generation of 
api replies. You could also pass opaques in requests and maybe expect backends, 
like avf_process, to bounce that opaques back for demuxing.

Regards,
Florin


On Sep 12, 2022, at 4:49 AM, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES 
at Cisco) via lists.fd.io<http://lists.fd.io> 
mailto:vrpolak=cisco@lists.fd.io>> wrote:

[resending to the correct vpp-dev e-mail address]

Short version:
Vratko would appreciate something like 
vlib_current_process_wait_for_one_time_event_or_clock.

Medium version:
One instance of request-response interaction between vlib processes had a bug 
[11].
Vratko contributed a fix [9] for the immediate issue,
but the proper fix was left hinted in TODOs (and discussed in the long version).

Long version:

Vlib supports processes and signals, see corresponding sections in the docs [7].
Using the actor model vocabulary, a (vlib) process is an actor,
and (vlib) signaling a (vlib) event means sending a message between actors.
There is no vlib name for actor behavior [10].

The typical use of event signaling in VPP is “fire and forget”,
meaning a “request” without any need to respond.
That means a typical process has just one behavior;
the side effects of a process are given by event type (and data),
not directly by the sequence of previous events received.

But there is an exception (and in future there may be more).
The process avf_process, when handling AVF_PROCESS_EVENT_REQ
and detecting that was signaled by some other process,
it signals back a “response” event.
The main reason is that some operations may take unreasonably long time,
and we prefer VPP to crash there (instead of getting stuck)
so we can see the backtrace.

A typical process that signaled AVF_PROCESS_EVENT_REQ is vl_api_clnt_process,
whose loop usually handles SOCKET_READ_EVENT events.
I mean, this socket API handling process has no idea about AVF plugin specific 
needs,
but it can call avf_process_request function which (upon detecting it is not 
called
from avf_process process) does the signaling and waiting.

But this means vl_api_clnt_process is the first process (that I know of) with 
two behaviors.
The first one focuses on handling new API messages,
the second one focuses on handling the AVF response (especially the lack 
thereof in time).
As clib_panic is called when the response does not arrive,
(and I hope there are never two requests at the same time)
the first behavior never encounters the AVF response.
But the second behavior can encounter SOCKET_READ_EVENT.
The VPP-2033 [11] bug is what happens in that case.

A minor issue is that the “response” event is defined just by
event type being zero, which would not work in (hypothetical future) scenarios
when a single process waits for two different responses.

Reading through node_funcs.h I found 
vlib_current_process_wait_for_one_time_event [12],
which looks suited for waiting for “single response” events,
but it lacks the time awareness of vlib_process_wait_for_event_or_clock.
If we had something like vlib_current_process_wait_for_one_time_event_or_clock
(and its example usage in the docs), handling the response would become easier.

Vratko.

[7] 
https://github.com/FDio/vpp/blob/9ad39c026c8a3c945a7003c4aa4f5cb1d4c80160/docs/developer/corearchitecture/vlib.rst
[9] https://gerrit.fd.io/r/c/vpp/+/37083
[10] https://

[vpp-dev] request-response between vlib processes

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

[resending to the correct vpp-dev e-mail address]

Short version:
Vratko would appreciate something like 
vlib_current_process_wait_for_one_time_event_or_clock.

Medium version:
One instance of request-response interaction between vlib processes had a bug 
[11].
Vratko contributed a fix [9] for the immediate issue,
but the proper fix was left hinted in TODOs (and discussed in the long version).

Long version:

Vlib supports processes and signals, see corresponding sections in the docs [7].
Using the actor model vocabulary, a (vlib) process is an actor,
and (vlib) signaling a (vlib) event means sending a message between actors.
There is no vlib name for actor behavior [10].

The typical use of event signaling in VPP is “fire and forget”,
meaning a “request” without any need to respond.
That means a typical process has just one behavior;
the side effects of a process are given by event type (and data),
not directly by the sequence of previous events received.

But there is an exception (and in future there may be more).
The process avf_process, when handling AVF_PROCESS_EVENT_REQ
and detecting that was signaled by some other process,
it signals back a “response” event.
The main reason is that some operations may take unreasonably long time,
and we prefer VPP to crash there (instead of getting stuck)
so we can see the backtrace.

A typical process that signaled AVF_PROCESS_EVENT_REQ is vl_api_clnt_process,
whose loop usually handles SOCKET_READ_EVENT events.
I mean, this socket API handling process has no idea about AVF plugin specific 
needs,
but it can call avf_process_request function which (upon detecting it is not 
called
from avf_process process) does the signaling and waiting.

But this means vl_api_clnt_process is the first process (that I know of) with 
two behaviors.
The first one focuses on handling new API messages,
the second one focuses on handling the AVF response (especially the lack 
thereof in time).
As clib_panic is called when the response does not arrive,
(and I hope there are never two requests at the same time)
the first behavior never encounters the AVF response.
But the second behavior can encounter SOCKET_READ_EVENT.
The VPP-2033 [11] bug is what happens in that case.

A minor issue is that the “response” event is defined just by
event type being zero, which would not work in (hypothetical future) scenarios
when a single process waits for two different responses.

Reading through node_funcs.h I found 
vlib_current_process_wait_for_one_time_event [12],
which looks suited for waiting for “single response” events,
but it lacks the time awareness of vlib_process_wait_for_event_or_clock.
If we had something like vlib_current_process_wait_for_one_time_event_or_clock
(and its example usage in the docs), handling the response would become easier.

Vratko.

[7] 
https://github.com/FDio/vpp/blob/9ad39c026c8a3c945a7003c4aa4f5cb1d4c80160/docs/developer/corearchitecture/vlib.rst
[9] https://gerrit.fd.io/r/c/vpp/+/37083
[10] https://en.wikipedia.org/wiki/Actor_model#Behaviors
[11] https://jira.fd.io/browse/VPP-2033
[12] 
https://github.com/FDio/vpp/blob/16052480c377127f9cb7facbab53f46e595b27cf/src/vlib/node_funcs.h#L1186

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21871): https://lists.fd.io/g/vpp-dev/message/21871
Mute This Topic: https://lists.fd.io/mt/93630182/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Crash when creating AVF interface

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

VPP should not crash without a good reason,
and possible issues with PAPI are not good enough a reason.

So I opened a ticket [4] once I saw the backtrace.
I will add more details there when I have any
(e.g. what exactly is different from usual CSIT daily runs).

Vratko.

[4] https://jira.fd.io/browse/VPP-2033

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21786): https://lists.fd.io/g/vpp-dev/message/21786
Mute This Topic: https://lists.fd.io/mt/92938076/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Community meeting today

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> The closest thing for strongswan to issue – at least in Linux – is the ip 
> xfrm command

That is why you need to create a new plugin for SSWAN (in SSWAN repo).

> To do so we only have to refer a few header files in our code

Are those GPLv2+ header files?

> Also it will be unsafe to do so

When VPP needs to do something safely (e.g. when creating memif)
it uses Unix Domain Socket (UDS).

> All ideas welcome.

I guess creating a quick and dirty "protocol" would not be difficult,
for example using gRPC.

The hard part is choosing transport that makes sense from both
security and performance point of view,
and defining message format that can be evolved (as in VPP API process;
but maybe also from interoperability point of view, think protocol version 
negotiation).

I bet SSWAN developers would have even better ideas
on how to integrate their GPLv2+ core with third party (Apache-2.0) dataplanes.

> translating from that standard communication channel to VPP calls

The other extreme is to consider VPP binary API over UDS "standard enough".
In that case, no translation is needed on VPP side, just a SSWAN plugin.

I mean, VPP Ipsec API messages are Production and versioned,
so no VPP version related surprises should happen.

Not sure SSWAN developers would want to maintain such a specific plugin.
Also not sure SSWAN CI is flexible enough for system tests with VPP in the loop,
but the VPP side can be mocked when not testing with real traffic.

As you described, that plugin can sit in a separate (GPLv2+) repository,
but building and testing it will be even trickier than in SSWAN repo.

And finally, that "separate repository" can be just extras subdirectory of VPP 
repo,
which is basically your proposal (except I propose API oved UDS to avoid 
license issues).

System testing (with both SSWAN, VPP, and the new plugin code in the loop, and 
real traffic)
will not be easy, but doable in fd.io CSIT (assuming the new plugin has 
packages uploaded
to fd.io packagecloud, or some other standard artifact repository CSIT can 
download from).

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Fan Zhang
Sent: Wednesday, 2022-June-15 15:18
To: vpp-dev@lists.fd.io
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Oginski, GabrielX 
Subject: Re: [vpp-dev] Community meeting today

Hi Vratko,

It is a great idea as it is non-intrusive and make both applications not 
dependent to each other.
However I cannot find a way (yet!) to make it work as linux-nl
The closest thing for strongswan to issue – at least in Linux – is the ip xfrm 
command, which I believe having no hooks to capture.
Also it will be unsafe to do so – imagine a malware process was hiding 
somewhere to capture all SAs.

What we are proposing is

  *   SSWAN supports externally linking a plugin, which is how it work on 
different OS.
  *   To do so we only have to refer a few header files in our code, and 
generate a .so file in VPP scope.
  *   In our code we will translate a SSWAN that translate SA 
management/routing cmds to VPP CAPI calls.
  *   When the .so file is compiled – we may provide a script, or user manually 
copy the .so file to the location SSWAN stores plugins.
  *   Rebooting SSWAN next time the plugin should be enabled and active.

However I cannot think of a way to change make test for this work:

  *   We need a strongswan application up and running and capable of adding 
plugin.
  *   I wouldn’ say we cannot, but will be hard to use scapy’s ikev2 (as we are 
adding support to strongswan and we targets data path validation only).

To test the implementation we can use similar way as existing 
vpp/extras/strongswan/run.sh, but this time we focus on data path checking.
All ideas welcome.

Regards,
Fan

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Tuesday, June 14, 2022 5:57 PM
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Community meeting today

> a way to make VPP IPsec work better w/ StrongSwan

As Damjan said, it would be a control plane adapter.
I guess it is also a management plane adapter
(translating higher level commands into VPP API commands).

From existing management plane adapters, it reminds me of linux_nl.
I read it listens to netlink messages.
I like the approach of listening to more standardized messages
coming over more standardized communication channel
(as opposed to creating a plugin for Linux kernel to call VPP binary API 
directly).

I read StrongSwan can work on Windows (not just Linux),
so it has to have some abstraction of dataplane management.
What you can do is to create a "remote/userspace dataplane" plugin
(contribute to StrongSwan repo, using their GPLv2+)
that translates from StrongSwan internals into a standard communication channel
(no idea which one, maybe 

Re: [vpp-dev] Configuring linux kernel bond interface in VPP

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> VPP 22.06 documentation which is the equivalent

Correctly not mentioning the DPDK configuration way,
as it was removed due to https://gerrit.fd.io/r/c/vpp/+/19867

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Dave Wallace
Sent: Wednesday, 2022-June-22 18:21
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Configuring linux kernel bond interface in VPP

Folks,

Unfortunately when performing an internet search for VPP's documentation, the 
first link found is commonly a very old copy on Read-The-Docs and often times a 
cloned RTD project.  Equally sad is the fact that the early doc sets did not 
include release labels, so it is not easy to discover the documentation is 
dated.

Here is a link to the VPP 22.06 documentation which is the equivalent circa VPP 
18.07 doc section that Pim references below: 
https://s3-docs.fd.io/vpp/22.06/configuration/config_getting_started.html#configuration-file-startup-conf

Thanks,
-daw-
On 6/18/22 1:36 PM, Pim van Pelt wrote:
Hoi,


On Fri, Jun 17, 2022 at 8:41 AM Chinmaya Aggarwal 
mailto:chinmaya.agar...@hsc.com>> wrote:
Hi,

We configure linux kernel physical interface on VPP using below steps:-

1. Whitelisting the physical interface in /etc/vpp/startup.conf.
2. Bringing the kernel interface down.
3. Restarting vpp service.

Is there a similar way by which we can configure linux bond interface in VPP? I 
don't see pci address for bond interface in linux.
No, you cannot create a bond in Linux and make use of that in VPP. Bonding is a 
kernel driver that makes use of physical network card (and their driver), so 
you will not see a PCI address for a bonded ethernet device in Linux.

However, you can do it the other way around -- create the bond in VPP and then 
expose it in Linux using the Linux Control Plane.
See an example of this in https://ipng.ch/s/articles/2021/08/25/vpp-4.html

There's another way of creating the bond in startup.conf -- search for the 
string eth_bond0 in the documentation here:
https://my-vpp-docs.readthedocs.io/en/latest/gettingstarted/users/configuring/startup.html


groet,
Pim
--
Pim van Pelt mailto:p...@ipng.nl>>
PBVP1-RIPE - http://www.ipng.nl/










-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21574): https://lists.fd.io/g/vpp-dev/message/21574
Mute This Topic: https://lists.fd.io/mt/91816581/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] wait CLI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> each worker gets only a shallow copy [4] of the process list.

Now I think that is not the cause.

> the verify job run [3] showed no effect.

As the tests use cli_inband API to execute CLI commands,
I removed another cause of worker barrier,
but patch set #4 of [2] still shows too few calls.
Additionally, marking cli_inband as thread safe
somehow reduced the dataplane performance (after the wait).

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Monday, 2022-June-20 19:26
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] wait CLI

A CLI command "wait" was added in [0],
and a related function vlib_process_suspend
was there in the first open commit [1].
I guess both are intended to be used during initialization,
forcing an exec script to slow down,
e.g. for a DHCP response to arrive.

But when I tried (in [2])
to teach "wait" it does not need to stop workers,
the verify job run [3] showed no effect.
If I am looking correctly, that is because
each worker gets only a shallow copy [4] of the process list.

Is this behavior intended?
Is there a way to pause only CLI processing
without also affecting worker threads?

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/23004/10/src/vlib/unix/cli.c
[1] https://gerrit.fd.io/r/c/vpp/+/23/3/vlib/vlib/node_funcs.h#326
[2] https://gerrit.fd.io/r/c/vpp/+/36444
[3] https://jenkins.fd.io/job/vpp-verify-master-ubuntu2004-x86_64/9215/
[4] https://gerrit.fd.io/r/c/vpp/+/23/3/vlib/vlib/threads.c#665

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21571): https://lists.fd.io/g/vpp-dev/message/21571
Mute This Topic: https://lists.fd.io/mt/91882341/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] wait CLI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

A CLI command "wait" was added in [0],
and a related function vlib_process_suspend
was there in the first open commit [1].
I guess both are intended to be used during initialization,
forcing an exec script to slow down,
e.g. for a DHCP response to arrive.

But when I tried (in [2])
to teach "wait" it does not need to stop workers,
the verify job run [3] showed no effect.
If I am looking correctly, that is because
each worker gets only a shallow copy [4] of the process list.

Is this behavior intended?
Is there a way to pause only CLI processing
without also affecting worker threads?

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/23004/10/src/vlib/unix/cli.c
[1] https://gerrit.fd.io/r/c/vpp/+/23/3/vlib/vlib/node_funcs.h#326
[2] https://gerrit.fd.io/r/c/vpp/+/36444
[3] https://jenkins.fd.io/job/vpp-verify-master-ubuntu2004-x86_64/9215/
[4] https://gerrit.fd.io/r/c/vpp/+/23/3/vlib/vlib/threads.c#665

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21556): https://lists.fd.io/g/vpp-dev/message/21556
Mute This Topic: https://lists.fd.io/mt/91882341/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Community meeting today

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> a way to make VPP IPsec work better w/ StrongSwan

As Damjan said, it would be a control plane adapter.
I guess it is also a management plane adapter
(translating higher level commands into VPP API commands).

From existing management plane adapters, it reminds me of linux_nl.
I read it listens to netlink messages.
I like the approach of listening to more standardized messages
coming over more standardized communication channel
(as opposed to creating a plugin for Linux kernel to call VPP binary API 
directly).

I read StrongSwan can work on Windows (not just Linux),
so it has to have some abstraction of dataplane management.
What you can do is to create a "remote/userspace dataplane" plugin
(contribute to StrongSwan repo, using their GPLv2+)
that translates from StrongSwan internals into a standard communication channel
(no idea which one, maybe some user-defined netlink protocol),
and then on the other side you will have a plugin similar to linux_nl
(VPP repo, Apache2 license) translating from that standard communication channel
to VPP calls (direct C calls, or binary API).

This way you can test the VPP plugin in make test
(assuming the standard communication channel is not too exotic for Python to 
handle)
and also StrongSwan plugin in their CI.

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Fan Zhang
Sent: Tuesday, 2022-June-14 10:39
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Community meeting today

Hi,

I was wondering if we can squeeze in a quick topic to discuss in today’s 
community call?
We are working on a way to make VPP IPsec work better w/ StrongSwan and want to 
discuss the upstream plan of it.

Regards,
Fan

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21541): https://lists.fd.io/g/vpp-dev/message/21541
Mute This Topic: https://lists.fd.io/mt/91745080/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/leave/1480452/21656/631435203/xyzzy 
[arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] hw_if_index API

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

TL;DR: Is there an API way to get hw_if_index value?

CLI "show hardware-interfaces" works for humans,
but some parsing is needed for machines to extract the index value.

For software interfaces, there is sw_interface_dump,
but I do not see any API way to get hw_if_index.

API users need that value for calls related to hardware capabilities:
flow_enable, flow_disable, vxlan_offload_rx and gtpu_offload_rx
(implementations call vnet_get_hw_interface).

Some of them are used in CSIT, which happens to work [0]
just because that suite does not create any non-hardware interfaces
(before the target hardware interface is created).

Vratko.

[0] 
https://gerrit.fd.io/r/c/csit/+/36120/4/resources/libraries/python/InterfaceUtil.py#1127

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21416): https://lists.fd.io/g/vpp-dev/message/21416
Mute This Topic: https://lists.fd.io/mt/91183386/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Gerrit reviews

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

According to wiki [0], I am writing here
to request Gerrit reviews on my recent contributions.

35697 [1] is the usual API endianness stuff.
send_interface_tx_placement_details code contains a FIXME,
but until that is done, my workaround is needed.

35838 [2] is also a common issue, a CLI command that
consumes all the multiline input (and not just its own line),
so it does not work when executed using "unix exec" part of startup.conf.

The following is a piped command that can detect similarly defective
CLI implementations:
$ fgrep -rn UNFORMAT_END_OF_INPUT | fgrep while | fgrep cli | fgrep -v 
line_input

Not sure if the detected code pattern can be legitimate in some circumstances.
(If the "fgrep cli" is skipped, I see it also selects lines
where the code is supposed to parse the whole multiline input.)
If not, we can teach the checkstyle script to reject it
(but that is probably a discussion for another thread).

Vratko.

[0] 
https://wiki.fd.io/view/VPP/Pulling,_Building,_Running,_Hacking_and_Pushing_VPP_Code#How_Do_I_Get_My_Patch_Reviewed_.3F
[1] https://gerrit.fd.io/r/c/vpp/+/35697
[2] https://gerrit.fd.io/r/c/vpp/+/35838

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#21196): https://lists.fd.io/g/vpp-dev/message/21196
Mute This Topic: https://lists.fd.io/mt/90240072/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] memory layout effects on VPP performance?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

When I was running tests for this [0] revert,
I saw different performance consequences than expected
(not reversal of the original [1] regression).
After rebasing, I am seeing yet another set of unexpected performance 
differences.

This has reminded me of Stabilizer [2] utility,
I learned about it by watching this [3] video.

What do you think, is it possible that also for VPP,
some regressions are caused just by compiler changing the memory layout
(without any "real" change in the program logic)?
If yes, should we try to apply Stabilizer to VPP?

(Re [0], I am leaning towards giving up on it.)

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/35164
[1] 
https://gerrit.fd.io/r/c/vpp/+/34568/2#message-63e1cdafce46e7d389ffb45d79df4fb622681888
[2] https://github.com/ccurtsinger/stabilizer
[3] https://www.youtube.com/watch?v=r-TLSBdHe1A

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20889): https://lists.fd.io/g/vpp-dev/message/20889
Mute This Topic: https://lists.fd.io/mt/89321880/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] issues found in RC2 testing

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Sorry for late notice, discoveries were coming faster than I was able to 
investigate.

Not listing:
- Regressions and failures with cause in CSIT code or drivers or firmware or 
hardware.
- Regressions and failures identified but also already fixed on stable/2202.

New in 22.02 (not present in 21.10)::

Regressions:

- Regression across all dpdk_plugin tests:
  - Cause is somewhere in DPDK upgrade, MTU rework or other related change.
  - In effect it disables the workaround for VPP-1876 tests are attempting to 
apply (setting high MTU).
  - The single VPP change where regression happens: 
https://gerrit.fd.io/r/c/vpp/+/34839
  - More details: 
https://gerrit.fd.io/r/c/vpp/+/34839/4#message-d5d0c80dc32b1aa29a142fb48b6c132ca04ff332
  - No easy fix in sight.

- Regression in tests with l2-input graph node:
  - Cache miss introduced in https://gerrit.fd.io/r/c/vpp/+/34568
  - More details: 
https://gerrit.fd.io/r/c/vpp/+/34568/2#message-63e1cdafce46e7d389ffb45d79df4fb622681888
  - Revert ready: https://gerrit.fd.io/r/c/vpp/+/35164
  - Due to misunderstanding, testing the revert begun late.
  - Some trouble verifying it restores L2 performance, maybe it introduces IP6 
regression?

- Regression in NAT44ED CSP tests:
  - Caused by a refactor/fix: https://gerrit.fd.io/r/c/vpp/+/34077
  - Not much details beyond consistent performance difference.
  - No idea about fix.

Failures:

- Jumbo tests with higher encapsulation overhead:
  - Related to MTU, new default value is not high enough compared to what CSIT 
is attempting to set.
  - Also started by https://gerrit.fd.io/r/c/vpp/+/34839
  - Also no easy fix in sight.


Older issues not discovered before::

Failures:

- Memif tests with jumbo frames:
  - Not clear yet if the cause is in VPP or in CSIT.

- IMIX tests on 3n-icx fail for hmac512sha:
  - The 570 byte component of IMIX is dropped.
  - Does not happen on 3n-skx.
  - Does not happen for aes256gcm.
  - Also present with VPP 21.10 release code (we did no have Icelake testbeds 
when doing CSIT 2110 release).
  - Cause not identified yet.

Other:

- DPDK testpmd has performance spread higher than expected.
  - Sometimes there is multi-band structure visible. 
https://jira.fd.io/browse/CSIT-1798
  - Affects also VPP vhost tests.

- Latency spikes:
  - Not deterministic, frequency is log in general.
  - ARM and AMD seems to be affected more than Intel.

Vratko.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20888): https://lists.fd.io/g/vpp-dev/message/20888
Mute This Topic: https://lists.fd.io/mt/89319622/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] older issues still affecting CSIT

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Hello, this is part two of what I had prepared
for VPP monthly meeting.

Older VPP Jira tickets:

- https://jira.fd.io/browse/VPP-1972 (Regression in NAT44ed performance)
  There might be multiple additional regressions,
  either in performance or in memory usage, affecting NAT tests.
  Example trending page: 
https://s3-docs.fd.io/csit/master/trending/trending/nat44-ed-udp-cps-2n-skx-xxv710.html
  We stopped running the maximum scale test (those with
  h262144-p63-s16515072 in name) to avoid failures and long test durations.

- https://jira.fd.io/browse/VPP-1947 (Simpler processing is less efficient with 
more workers)
  Affects "Speedup Multi-Core" report graphs (see the link in the first Jira 
comment).

CSIT Jira tickets (probably driver issues) VPP developers may help fix:

- https://jira.fd.io/browse/CSIT-1791 (Performance regression in RDMA tests)
  Two symptoms, present since we upgraded CSIT machines to Ubuntu 20.04.

- https://jira.fd.io/browse/CSIT-1782 (4c AVF tests are failing when trying to 
create interface)
  Occasionally present despite a CSIT workaround.

Vratko.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20464): https://lists.fd.io/g/vpp-dev/message/20464
Mute This Topic: https://lists.fd.io/mt/86935620/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] non-production APIs used by CSIT

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Hello, this is part one of what I had prepared
for VPP monthly meeting.

Non-production (deprecated or in-progress) VPP API messages
used by CSIT currently are these ones:
adl_allowlist_enable_disable, adl_interface_enable_disable,
af_xdp_create, crypto_sw_scheduler_set_worker,
det44_interface_add_del_feature, det44_plugin_enable_disable,
flow_add, flow_del, flow_disable, flow_enable,
nat44_plugin_enable_disable, nat44_show_running_config
(plus their *_reply partners).

If there is a production API to be used instead,
please reply here.
If not, consider making the message production
(e.g. by bumping the version in the .api file to not start with 0.).

Context: https://wiki.fd.io/view/VPP/ApiChangeProcess
CSIT should ideally use production API messages only,
so their structure is guarded by CRC.

Vratko.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#20463): https://lists.fd.io/g/vpp-dev/message/20463
Mute This Topic: https://lists.fd.io/mt/86935350/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] vpp got stucked after bridge and loop interfaces is created and snat is configured #nat44

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Sounds like the issue fixed in [0] (so after 21.06).

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/33018

From: vpp-dev@lists.fd.io  On Behalf Of g.good...@gmail.com
Sent: Thursday, 2021-August-19 14:58
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] vpp got stucked after bridge and loop interfaces is created 
and snat is configured #nat44


vpp version: 21.06
vpp main core will be stucked after bridge and loop interfaces and snat is 
configured, here is my topology.

/--\/--\/--\

|  ||  ||  |

|client  enp0s8  GE0/2/0   vpp  GE0/5/0  enp0s10  server   |

|  ||  ||  |

\--/\--/\--/



   192.0.2.0/24192.168.3.0/24
and here is my configuration:
nat44 enable
nat44 forwarding enable
nat44 add int address GigabitEthernet5/0/0
set int nat44 in GigabitEthernet2/0/0 out GigabitEthernet5/0/0 output-feature
create tap id 0
set interface state tap0 up
set int l2 bridge GigabitEthernet2/0/0 1
set int l2 bridge tap0 1
create loopback interface
set int l2 bridge loop0 1 bvi
set int ip addr loop0 192.0.2.11/24
set int state loop0 up

vpp will be stucked after a few ping from client to server, here is backtrace 
info in gdb:
#0  0x7f980557f0d1 in internal_mallinfo (m=0x7f97bb18b040) at 
/usr/src/debug/vpp-0.1/src/vppinfra/dlmalloc.c:2099
#1  0x7f98055707d7 in mspace_mallinfo (msp=) at 
/usr/src/debug/vpp-0.1/src/vppinfra/dlmalloc.c:4803
#2  clib_mem_get_heap_usage (heap=, 
usage=usage@entry=0x7f97bb05df40) at 
/usr/src/debug/vpp-0.1/src/vppinfra/mem_dlmalloc.c:475
#3  0x55c1903304fa in do_stat_segment_updates (sm=0x55c1903c7ac0 
) at /usr/src/debug/vpp-0.1/src/vpp/stats/stat_segment.c:661
#4  stat_segment_collector_process (vm=0x7f98056b2680 , 
rt=, f=) at 
/usr/src/debug/vpp-0.1/src/vpp/stats/stat_segment.c:761
#5  0x7f9805648897 in vlib_process_bootstrap (_a=) at 
/usr/src/debug/vpp-0.1/src/vlib/main.c:1477
#6  0x7f9805587d80 in clib_calljmp () from /lib64/libvppinfra.so.0.1
#7  0x7f97bd38add0 in ?? ()

after debugs i found the reason, vpp counts the packets through snat, it store 
the result in nm->counters.fastpath.in2out.icmp, which is a vector struct, the 
size of the vector is based on interfaces index, based on my configures above, 
the size is 3, but after i configured loop and bridge interfaces, both the new 
interface index is bigger than 3. when packets pass through snat, it thought 
packet is received from loop interface, and then got out of bonds when writing 
vector.
and my question is:
1.based on my configuration above, does packets counts saved to loop interface 
is correct?
2.Besides avoiding misconfiguration, how to fix it?
thanks a lot.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19986): https://lists.fd.io/g/vpp-dev/message/19986
Mute This Topic: https://lists.fd.io/mt/84995915/21656
Mute #nat44:https://lists.fd.io/g/vpp-dev/mutehashtag/nat44
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] sw_interface_dump currently cann't dump interface which sw_if_index == 0 #vapi

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> I would argue that this case can be classified as "bugfix"

Yes, [0] was an ugly workaround, we can call it a bug.
But the important thing is there are users depending on the buggy behavior.
Also, the "buggy" behavior was explained in message documentation,
so if we "fix" the bug, we will need to make incompatible edit
in the documentation.

Here is the longer version of the story.
Once upon a time, VPP had no strict process for handling API changes,
and sw_interface_dump message had no sw_if_index param.
Paul added [4] that param with the usual ~0 standing for "all".
But one of "downstream" users was VAT, and Paul had not added
anything related to the new param before message is sent [5].
In those times, we had no support for non-zero default values,
so VPP saw zero as the value for sw_if_index and returned
just details for the local interface, no other.
CSIT was using the VAT call, so suddenly all tests started failing
due to missing details for other interfaces.
There was a discussion on vpp-dev, in 4 threads: [6] [7] [8] [9].
I was not able to figure out a quick enough fix for VAT,
but my workaround for API has been merged.
Of course, after [2] nobody recalled
that the workaround is no longer needed.

> the reason for it is minimize the amount of forced work downstream,
> caused by the API changes, and minimize the element of surprise.

Ok, so there are actually three solutions (in my decreasing preference).

One is to add sw_interface_dump_v2 without deprecating sw_interface_dump.
That way, old users are not surprised, and no work is required,
but the new users may be confused on why there are two very similar messages.
(it counts as kind of "read and think to realize no real coding is needed" 
work).
An example for multiple similar messages is
how we added create_loopback_instance without removing create_loopback.

The second solution is to add _v2 but also deprecate the old message,
to keep the API simpler (the release after next)
at the cost of bothering those old users who relied on the "buggy" behavior.

The third solution is to keep the message name,
but change description and behavior (and bump semver),
and rely on old users (starting with VAT) to update their usage
(after recovering from the initial surprise).

> and giving the folks a few weeks (namely, until after RC1), to adapt.

That would be fine for people consuming master HEAD regularly.
I imagine there are people who upgrade VPP once a release (relying on API 
stability)
without reading vpp-dev mailing list much.
Those are going to be surprised by the third solution.

Vratko.

[4] https://gerrit.fd.io/r/c/vpp/+/18693
[5] 
https://github.com/FDio/vpp/blob/6407ba56a392f37322001d0ffdca002223b095c0/src/vat/api_format.c#L5978
[6] https://lists.fd.io/g/vpp-dev/topic/30423722#12521
[7] https://lists.fd.io/g/vpp-dev/topic/30426855#12548
[8] https://lists.fd.io/g/vpp-dev/topic/31234917#12817
[9] https://lists.fd.io/g/vpp-dev/topic/31307751#12840

-Original Message-
From: Andrew 👽 Yourtchenko  
Sent: Monday, 2021-May-03 11:41
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: jiangxiaom...@outlook.com; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] sw_interface_dump currently cann't dump interface which 
sw_if_index == 0 #vapi

I would argue that this case can be classified as "bugfix" - there was
no good reason to use the 0 as a wildcard value in the first place,
since it is a valid sw_if_index, and there is a perfectly good
"wildcard value" of ~0 that already works, right ?

So I would say this discussion should serve as the announcement (Or
there can be another separate thread with the explicit subject of it),
and post-branch of stable/2106, we can apply the fix to consider the 0
to be a valid interface ID.

And of course bump the semver for those who look at it.

It's always useful to keep in mind "why" the process is in place, when
evaluating how to apply it:

the reason for it is minimize the amount of forced work downstream,
caused by the API changes, and minimize the element of surprise.

In this case, having the "buggy" clients use ~0 in place of 0 (if they
ever used that) is strictly less work than having *all* clients to
switch to using a new message name even if they used the ~0 to begin
with.

We can take care of minimizing the "element of surprise" by this
discussion, or maybe a separate mail - and giving the folks a few
weeks (namely, until after RC1), to adapt.

This way the spirit of why the process there in the first place will
be fulfilled, without incurring the unnecessary effort for everyone.

Does this make sense ?

--a

On 5/3/21, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
via lists.fd.io  wrote:
>> Is there any plan for support selecting only index==0 ?
>
> Good news first.
> I added the TODO here [0], but since then
> CSIT sto

Re: [vpp-dev] sw_interface_dump currently cann't dump interface which sw_if_index == 0 #vapi

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> Is there any plan for support selecting only index==0 ?

Good news first.
I added the TODO here [0], but since then
CSIT stopped using the VAT command in [1],
and other uses based on PAPI should be ready since [2].

The bad news is VPP now has more strict process [3]
regarding changes to API.
This particular case is debatable, as the change
will not affect the message CRC value,
only the behavior for a particular argument value.
But I think it would be an API change overall,
so not directly allowed (as sw_interface_dump is a production API).

The indirect fix is to add a new API message
(sw_interface_dump_v2) and mark the old one as deprecated.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/19106
[1] https://gerrit.fd.io/r/c/csit/+/21149
[2] https://gerrit.fd.io/r/c/vpp/+/23530
[3] https://wiki.fd.io/view/VPP/ApiChangeProcess

From: vpp-dev@lists.fd.io  On Behalf Of 
jiangxiaom...@outlook.com
Sent: Thursday, 2021-April-29 13:41
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] sw_interface_dump currently cann't dump interface which 
sw_if_index == 0 #vapi

Hi  Vratko Polak,

currently sw_interface_dump will dump all interfaces if sw_if_index == 0. In my 
case, i need get interface information by id without by dumping all, but there 
is no way for this.
I notice there is a TODO comment for sw_interface_dump in interface.api file.
/** \brief Request all or filtered subset of sw_interface_details
@param client_index - opaque cookie to identify the sender
@param context - sender context, to match reply w/ request
@param sw_if_index - index of the interface to dump info on, 0 or ~0 if on 
all
  TODO: Support selecting only index==0 when CSIT is ready.
@param name_filter_valid - 1 if requesting a filtered subset of records 
else 0
  if name filter is set as valid, sw_if_index value is ignored and all 
interfaces are examined
@param name_filter - interface name substring filter. Eg. loop1 returns 
[loop1, loop10]
*/
define sw_interface_dump
Is there any plan for support selecting only index==0 ?

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19314): https://lists.fd.io/g/vpp-dev/message/19314
Mute This Topic: https://lists.fd.io/mt/82453036/21656
Mute #vapi:https://lists.fd.io/g/vpp-dev/mutehashtag/vapi
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP behavior stability document

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Hello.

I am working on a document, discussing improvements
to the current API process that go beyond just API.

It is currently a draft (of limited readability),
but several people are giving me feedback already,
so maybe some of you want to read and comment too.

Here [0] is the Gerrit link.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/31786

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19271): https://lists.fd.io/g/vpp-dev/message/19271
Mute This Topic: https://lists.fd.io/mt/82266112/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] Faster PAPI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

 putting CSIT utilities on VPP machine,
but sometimes it is the least evil (e.g. we do that for reading stats segment).

1a) with VAT1 is a fourth bottleneck solution,
that is why we still use it.
6a) is also a fourth bottleneck solution, but with stable API.

26019 contains solutions for first, second and third bottleneck.
31920 is a solution for second bottleneck, assuming first bottleneck
is solved as in 26019 and for third bottleneck we use 26019 or something else.

1b) solves (avoids) first and second bottleneck,
but it has its own third bottleneck.
Luckily, many binary data serialization solutions
could be adapted for json data serialization.

Vratko.

-Original Message-
From: otr...@employees.org  
Sent: Thursday, 2021-February-04 08:47
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: Paul Vinciguerra ; Peter Mikus -X (pmikus - 
PANTHEON TECH SRO at Cisco) ; csit-...@lists.fd.io; 
vpp-dev@lists.fd.io
Subject: Re: [csit-dev] Faster PAPI

> 1. Keep the status quo.
> That means using VAT for some operations (e.g. adding multiple routes [5]),
> and creating "exec scripts" [6] for operations without VAT one-liner.
> Pros: No work needed, good speed, old VPP versions are supported.
> Cons: Relying on VAT functionality (outside API compatibility rules).

1b) Pre-generate a JSON file with all commands and load that into VPP with VAT2.
1c) Use a debug CLI script file and "exec"

> 2. Support "vector operations" in VPP via binary API.
> This will probably need a new VPP plugin to host the implementations.
> Pros: Fast speed, small CSIT work, guarded by API compatibility rules.
> Cons: New VPP plugin of questionable usefulness outside CSIT,
> plugin maintainer needed, old VPP versions not supported.

Could you ellaborate on this idea?

> 3. VPP PAPI improvements only.
> No changes to VPP API, just changes to PAPI to allow better speed for socket 
> interactions.
> CSIT would need a fast way to synthetize binary messages.
> Pros: Small VPP work, good speed, only "official" VPP API is used.
> Cons: Brittle CSIT message handling, old VPP versions not supported.

What would this be?
E.g. we could do serializations in C with a small Python wrapper.

> 4. CSIT hacks only (Gerrit 26019).
> No changes to VPP API nor PAPI. CSIT code messes with PAPI internals.
> CSIT needs a fast way to synthetize binary messages.
> Pros: Code is ready, good speed, old VPP versions are supported.
> Cons: Brittle CSIT message handling, risky with respect to VPP PAPI changes.

This is the idea explained below, where you serialize message once and replays 
right?
In addition to tweaking the reader thread etc?

5) Paul's proposal. I don't know if he has measured performance impact on that.

6) The add 4 million routes that CSIT uses, generates routes according to a set 
of parameters.
   We could add a small plugin on the VPP side that exposes an API of the sort
   "create  routes from ".

Is it the serialisation, or the message RTT or both?
If serialisation, doing it in C should help.
If it's the RTT, larger messages is one option.
Uploading all messages to VPP and then ask VPP to process them, receiving a 
single reply is also an option.
Is this your "vector" idea?

> The open questions:
> Do you see any other options?
> Did I miss some important pros or cons?
> Which option do you prefer?

The lowest hanging fruit is likely 6. But longer term I'd prefer a more generic 
solution.

Best regards,
Ole

> [2] https://lists.fd.io/g/vpp-dev/topic/78362835#18092
> [3] https://gerrit.fd.io/r/c/csit/+/26019/140
> [4] 
> https://gerrit.fd.io/r/c/csit/+/26019/140#message-314d168d8951b539e588e644a875624f5ca3fb77
> [5] 
> https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/templates/vat/vpp_route_add.vat
> [6] 
> https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/libraries/python/TestConfig.py#L121-L150
> 
> From: vpp-dev@lists.fd.io  On Behalf Of Vratko Polak -X 
> (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
> Sent: Thursday, 2020-May-07 18:35
> To: vpp-dev@lists.fd.io
> Cc: csit-...@lists.fd.io
> Subject: [vpp-dev] Faster PAPI
> 
> Hello people interested in PAPI (VPP's Python API client library).
> 
> In CSIT, our tests are using PAPI to interact with VPP.
> We are using socket transport (instead of shared memory transport),
> as VPP is running on machines separate from machines running the tests.
> We use SSH to forward the socket between the machines.
> 
> Some of our scale tests need to send high number of commands towards VPP.
> The largest test sends 4 million commands (ip_route_add_del with ip6 
> addresses).
> You can imagine that can take a while.
> Even using PAPI in asynchronous mode, it takes 

Re: [vpp-dev] Faster PAPI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

plugin maintainer needed, old VPP versions not supported.

3. VPP PAPI improvements only.
No changes to VPP API, just changes to PAPI to allow better speed for socket 
interactions.
CSIT would need a fast way to synthetize binary messages.
Pros: Small VPP work, good speed, only "official" VPP API is used.
Cons: Brittle CSIT message handling, old VPP versions not supported.

4. CSIT hacks only (Gerrit 26019).
No changes to VPP API nor PAPI. CSIT code messes with PAPI internals.
CSIT needs a fast way to synthetize binary messages.
Pros: Code is ready, good speed, old VPP versions are supported.
Cons: Brittle CSIT message handling, risky with respect to VPP PAPI changes.

The open questions:
Do you see any other options?
Did I miss some important pros or cons?
Which option do you prefer?

Vratko.

[2] https://lists.fd.io/g/vpp-dev/topic/78362835#18092
[3] https://gerrit.fd.io/r/c/csit/+/26019/140
[4] 
https://gerrit.fd.io/r/c/csit/+/26019/140#message-314d168d8951b539e588e644a875624f5ca3fb77
[5] 
https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/templates/vat/vpp_route_add.vat
[6] 
https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/libraries/python/TestConfig.py#L121-L150

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io<http://lists.fd.io>
Sent: Thursday, 2020-May-07 18:35
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Cc: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>
Subject: [vpp-dev] Faster PAPI

Hello people interested in PAPI (VPP's Python API client library).

In CSIT, our tests are using PAPI to interact with VPP.
We are using socket transport (instead of shared memory transport),
as VPP is running on machines separate from machines running the tests.
We use SSH to forward the socket between the machines.

Some of our scale tests need to send high number of commands towards VPP.
The largest test sends 4 million commands (ip_route_add_del with ip6 addresses).
You can imagine that can take a while.
Even using PAPI in asynchronous mode, it takes tens of minutes per million 
commands.

I was able to speed that up considerably, just by changing code on CSIT side.
The exact code change is [0], but that may be hard to review.
Gerrit does not even recognize the new PapiSocketExecutor.py
to be an edited copy of the old PapiExecutor.py file.

That code relies on the fact that Python is quite permissive language,
not really distinguishing private fields and methods from public ones.
So the current code is vulnerable to refactors of VPP PAPI code.
Also, pylint (static code analysis tool CSIT uses) is complaining.

The proper fix is to change the VPP PAPI code,
so that it exposes the inner parts the new CSIT code needs to access
(or some abstractions of them).

For that I have created [1], which shows the changed VPP PAPI code.
Commit message contains a simplified example of how the new features can be 
used.

The changed VPP code allows three performance improvements.

1. Capturing raw bytes sent.
For complicated commands, many CPU cycles are spent serializing
command arguments (basically nested python dicts) into bytes.
If user (CSIT code) has access to the message as serialized by PAPI (VPP code),
the user can choose a faster method to create subsequent serialized data.
Implementing this on CSIT side improved the speed, but not greatly enough.
(See bytes_template.py in [0] for the faster data generator.)
The VPP code [1] introduces fields remember_sent and last_sent.

2. Reading replies without de-serializing them.
This was already possible by calling transport.q.get(),
but had next to no effect on PAPI speed.
Replies are usually short, so deserialization does not take too many cycles.

3. Disabling the background reader thread.
By default, socket transport creates (upon connect) a background thread,
which select()s on the socket, reads any messages,
and put()s them to transport.q (multiprocessing.Queue).
I am not sure whether it is the multithreading (waiting for Python interpreter
to switch between threads), or Queue (locks, its own reader thread),
but overall this was the remaining bottleneck.
The VPP code exposes public methods for stopping and starting the thread.

Back to point 2:
With the reading thread stopped, transport.q is not filled,
so another way to read the reply is needed.
The VPP code contained transport._read(),
the underscore hinting this is an internal method
(leading to the abovementioned pylint complaints).
The VPP change [1] renames that method to read_message(),
adding a docstring explaining it has to be used
when the reading thread is stopped.

Finally, with all 3 improvements, CSIT will be able
to execute million PAPI commands in around 15 seconds.

Even if something like [1] is merged to VPP,
CSIT will still use [0] for some time,
so we are able 

Re: [vpp-dev] Cisco VIC 1495 performance with DPDK and VPP 21.01

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> For VPP release 20.01, I have found few [0] results [1]

I meant VPP 21.01; links are pointing there, it is just a typo.
Vratko.

-Original Message-
From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Sent: Tuesday, 2021-April-06 20:59
To: Manoj Kumar (manojku5) 
Cc: vpp-dev@lists.fd.io
Subject: RE: Cisco VIC 1495 performance with DPDK and VPP 21.01

In CSIT, we have one testbed type (3-node Haswell)
with older cards (Cisco VIC 1385 and even Cisco VIC 1227),
but we are running tests on those very rarely
(those NICs are not very popular, and we prefer to run
larger test selection on fewer NICs).

For VPP release 20.01, I have found few [0] results [1]
for Cisco VIC 1227, not sure what happened to Cisco VIC 1385 tests.

For Cisco VIC 1495/97 specifically, we do not have them in the lab,
and we do not have many slots available either.
The good news is the tests do not need any special configuration
for Cisco NICs. If DPDK supports them, they run equally well as Intel NICs.

Vratko.

[0] 
https://docs.fd.io/csit/rls2101/report/detailed_test_results/vpp_performance_results_3n_hsw/ip4_vic1227.html
[1] 
https://docs.fd.io/csit/rls2101/report/detailed_test_results/vpp_performance_results_3n_hsw/l2_vic1227.html

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Manoj Kumar 
(manojku5) via lists.fd.io
Sent: Friday, 2021-March-26 09:22
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Cisco VIC 1495 performance with DPDK and VPP 21.01

Hi Team,

I am new to VPP developer's forum and wanted to understand the Cisco VIC 
1495/97 performance with DPDK and VPP 21.01.

Could someone please guide me to some links containing the performance details 
for different NICs e.g. Cisco VIC 1495 vs Mellanox vs Intel XL710 .

Regards
-Manoj


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19117): https://lists.fd.io/g/vpp-dev/message/19117
Mute This Topic: https://lists.fd.io/mt/81623267/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Cisco VIC 1495 performance with DPDK and VPP 21.01

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

In CSIT, we have one testbed type (3-node Haswell)
with older cards (Cisco VIC 1385 and even Cisco VIC 1227),
but we are running tests on those very rarely
(those NICs are not very popular, and we prefer to run
larger test selection on fewer NICs).

For VPP release 20.01, I have found few [0] results [1]
for Cisco VIC 1227, not sure what happened to Cisco VIC 1385 tests.

For Cisco VIC 1495/97 specifically, we do not have them in the lab,
and we do not have many slots available either.
The good news is the tests do not need any special configuration
for Cisco NICs. If DPDK supports them, they run equally well as Intel NICs.

Vratko.

[0] 
https://docs.fd.io/csit/rls2101/report/detailed_test_results/vpp_performance_results_3n_hsw/ip4_vic1227.html
[1] 
https://docs.fd.io/csit/rls2101/report/detailed_test_results/vpp_performance_results_3n_hsw/l2_vic1227.html

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Manoj Kumar 
(manojku5) via lists.fd.io
Sent: Friday, 2021-March-26 09:22
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Cisco VIC 1495 performance with DPDK and VPP 21.01

Hi Team,

I am new to VPP developer's forum and wanted to understand the Cisco VIC 
1495/97 performance with DPDK and VPP 21.01.

Could someone please guide me to some links containing the performance details 
for different NICs e.g. Cisco VIC 1495 vs Mellanox vs Intel XL710 .

Regards
-Manoj


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#19116): https://lists.fd.io/g/vpp-dev/message/19116
Mute This Topic: https://lists.fd.io/mt/81623267/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Bug in vl_socket_read_ready() function

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> I see that this change is done as part of following commit.

I only edited variable names and such.
The previous code contained the same logic [1],
and it seems to come from the initial code [2].

> we are deleting the socket_main.input_buffer

No, we are deleting just some (perhaps all)
elements (bytes) from the vector (buffer).

> it detected use-after-poison

Looking at [3], only the area after the remaining values is poisoned.
But macros that can enlarge the vector should unpoison that area,
so I am not sure what use does address sanitizer see.

Vratko.

[1] https://gerrit.fd.io/r/c/vpp/+/22547/4/src/vlibmemory/socket_api.c#b304
[2] https://gerrit.fd.io/r/c/vpp/+/23/3/vlib-api/vlibsocket/socksvr_vlib.c#314
[3] https://gerrit.fd.io/r/c/vpp/+/21937/10/src/vppinfra/vec.h#798

From: vpp-dev@lists.fd.io  On Behalf Of Satya Murthy
Sent: Wednesday, 2021-February-10 10:42
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Bug in vl_socket_read_ready() function

Hi ,

While using AddressSanitizer with VPP, it detected use-after-poison in 
vl_socket_read_ready() function.

Closely looking at the code, it seems that this is happening due to the 
following code.
 if (vec_len (msg_buffer) > msgbuf_len)
  /* There are some fragments left. Shrink the msg_buffer to simplify 
logic. */
  vec_delete (msg_buffer, msgbuf_len, 0);   < we are deleting 
the socket_main.input_buffer, which is initialized in api_init, but not again 
after this delete.

So, when we try to use this buffer again, its detected by libASAN.

I see that this change is done as part of following commit.

--
commit 6a6af6ea1a77b5818e717047b5d01251ef6d024a
Author: Vratko Polak mailto:vrpo...@cisco.com>>
Date:   Mon Oct 7 14:52:53 2019 +0200

api: comment, simplify and fix api socket read

The function vl_socket_read_ready did contain some comments already,
but as they stated, the logic has to be tricky to cover multiple cases.
-

Can someone please comment, if I am missing anything here ?
I see that this code is present even in the latest stable code base.

--
Thanks & Regards,
Murthy

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18721): https://lists.fd.io/g/vpp-dev/message/18721
Mute This Topic: https://lists.fd.io/mt/80528036/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Scapy license in VPP

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> we use scapy to create a separate pcap file

We also use Scapy data structures
to parse packets received from VPP
to verify they contain the correct values.
Example: [6].

> we need to use GPLv3 (which is compatible with both).

Only for programs such as VPP test framework
which link with both GPL (Scapy) and Apache (VPP PAPI).
But that includes also programs that depend
on Apache-licensed library only by mistake [7].

Vratko.

[6] 
https://github.com/FDio/vpp/blob/e988726cbfb1b1f618c4034aa16e41364f9c48a2/test/test_ip4.py#L160-L182
[7] 
https://github.com/FDio/csit/blob/624e580159dc056035d7e962582e86a000f8c3d6/GPL/traffic_scripts/send_ip_check_headers.py#L35

From: Paul Vinciguerra 
Sent: Thursday, 2021-February-04 17:05
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: t...@lists.fd.io; Kinsella, Ray ; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Scapy license in VPP

This is good information.

Can we take a step back and approach this from a different perspective?  Why do 
we actually need scapy?
The general process is that we use scapy to create a separate pcap file that is 
fed to the vpp packet generator.
If we instead create named pcap files that we pre-package with the tests, we 
would 1) resolve this issue 2) speed up the actual execution of the tests.

The only usage of scapy outside of the unit tests proper that comes to mind is 
within test/utils.py

Paul

On Thu, Feb 4, 2021 at 9:54 AM Vratko Polak -X (vrpolak - PANTHEON TECH SRO at 
Cisco) mailto:vrpo...@cisco.com>> wrote:
Few quick comments and links.

Turns out GPLv2-or-later is not by itself compatible with Apache [2],
we need to use GPLv3 (which is compatible with both).

The questions on dynamic linking are still open [3] [4] [5].

We probably need to distinguish several Programs
available from VPP repository.
VPP PAPI (library) is a separate distribution
from VPP test framework.

When a user starts VPP test framework,
linking with VPP PAPI in runtime,
therefore creating a derived work.
GPLv3 would apply to that Program,
but it does not mean we need to mention GPLv3
in VPP PAPI license headers.

This only works because VPP PAPI does not share
any code nor data structures specific to VPP test framework.
Modules which link with Scapy do share data structures,
so the "derived work" nature is apparent already in repo.

Vratko.

[2] https://www.gnu.org/licenses/license-list.html#apache2
[3] 
https://tech.popdata.org/the-gpl-license-and-linking-still-unclear-after-30-years/
[4] https://lwn.net/Articles/548216/
[5] 
https://www.kilpatricktownsend.com/-/media/2019/IP-Desk-Reference-2015/A-Practical-Guide-to-Open-Source-Software.ashx?la=en&hash=895293E77B462CF85E4D3E80AF8DC83143D0C165

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io<http://lists.fd.io>
Sent: Friday, 2021-January-29 18:10
To: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Cc: t...@lists.fd.io<mailto:t...@lists.fd.io>; Kinsella, Ray 
mailto:ray.kinse...@intel.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Scapy license in VPP

> Why do you say that vpp_papi need to be dual licensed?

I think e-mail reply would be long,
and I will need to address comments anyway,
so I respond via a Gerrit change [1].

Vratko.

[1] https://gerrit.fd.io/r/c/vpp/+/31025

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Friday, 2021-January-29 15:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
mailto:vrpo...@cisco.com>>
Cc: t...@lists.fd.io<mailto:t...@lists.fd.io>; Kinsella, Ray 
mailto:ray.kinse...@intel.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Scapy license in VPP

Why do you say that vpp_papi need to be dual licensed?

On Thu, Jan 28, 2021 at 12:43 PM Vratko Polak -X (vrpolak - PANTHEON 
TECHNOLOGIES at Cisco) via lists.fd.io<http://lists.fd.io> 
mailto:cisco@lists.fd.io>> wrote:
[cid:image001.gif@01D6FB23.7B58E9A0]
First draft created [0] for the change that will switch
licenses for Python files used together with Scapy.

For some files, I was not sure whether they are used together with Scapy.
One big detail is that vpp_papi needs to have dual license,
as test framework integrates with it (and with scapy).
If I understand the licensing logic correctly,
CSIT tests can still choose to use vpp_papi under Apache license option.
But we may need to discuss that with lawyers.

Ray, you may need to upgrade your contributor-finding shell pipeline
to cover all files I added the new license into.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/30998


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18675): https://lists.fd.io/g/vpp-dev/message/18675
Mute This Topic: https://lists.fd.io/mt/80189290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Scapy license in VPP

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Few quick comments and links.

Turns out GPLv2-or-later is not by itself compatible with Apache [2],
we need to use GPLv3 (which is compatible with both).

The questions on dynamic linking are still open [3] [4] [5].

We probably need to distinguish several Programs
available from VPP repository.
VPP PAPI (library) is a separate distribution
from VPP test framework.

When a user starts VPP test framework,
linking with VPP PAPI in runtime,
therefore creating a derived work.
GPLv3 would apply to that Program,
but it does not mean we need to mention GPLv3
in VPP PAPI license headers.

This only works because VPP PAPI does not share
any code nor data structures specific to VPP test framework.
Modules which link with Scapy do share data structures,
so the "derived work" nature is apparent already in repo.

Vratko.

[2] https://www.gnu.org/licenses/license-list.html#apache2
[3] 
https://tech.popdata.org/the-gpl-license-and-linking-still-unclear-after-30-years/
[4] https://lwn.net/Articles/548216/
[5] 
https://www.kilpatricktownsend.com/-/media/2019/IP-Desk-Reference-2015/A-Practical-Guide-to-Open-Source-Software.ashx?la=en&hash=895293E77B462CF85E4D3E80AF8DC83143D0C165

From: vpp-dev@lists.fd.io  On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Friday, 2021-January-29 18:10
To: Paul Vinciguerra 
Cc: t...@lists.fd.io; Kinsella, Ray ; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Scapy license in VPP

> Why do you say that vpp_papi need to be dual licensed?

I think e-mail reply would be long,
and I will need to address comments anyway,
so I respond via a Gerrit change [1].

Vratko.

[1] https://gerrit.fd.io/r/c/vpp/+/31025

From: Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>>
Sent: Friday, 2021-January-29 15:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
mailto:vrpo...@cisco.com>>
Cc: t...@lists.fd.io<mailto:t...@lists.fd.io>; Kinsella, Ray 
mailto:ray.kinse...@intel.com>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Scapy license in VPP

Why do you say that vpp_papi need to be dual licensed?

On Thu, Jan 28, 2021 at 12:43 PM Vratko Polak -X (vrpolak - PANTHEON 
TECHNOLOGIES at Cisco) via lists.fd.io<http://lists.fd.io> 
mailto:cisco@lists.fd.io>> wrote:
[cid:image001.gif@01D6FB0A.76F1EF60]
First draft created [0] for the change that will switch
licenses for Python files used together with Scapy.

For some files, I was not sure whether they are used together with Scapy.
One big detail is that vpp_papi needs to have dual license,
as test framework integrates with it (and with scapy).
If I understand the licensing logic correctly,
CSIT tests can still choose to use vpp_papi under Apache license option.
But we may need to discuss that with lawyers.

Ray, you may need to upgrade your contributor-finding shell pipeline
to cover all files I added the new license into.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/30998


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18671): https://lists.fd.io/g/vpp-dev/message/18671
Mute This Topic: https://lists.fd.io/mt/80189290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> The rules are managed via [1].

Does that mean the rules are guaranteed to survive
any sudden Jenkins upgrade (needed for security reasons)?

I will feel safer if we had a copy of the database in ci-management.
That way, if I contribute a new rule,
you can manually apply it when merging the change
(maybe even apply and test on Sandbox before merging).

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Dave Wallace
Sent: Wednesday, 2021-February-03 17:32
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: vpp-dev 
Subject: Re: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications

Vratko,

The rules are managed via [1].  Vanessa gave me permission to create/edit these 
rules which appear to be applied universally to all jenkins.fd.io projects.  
Let me know if there are any rules you would like me to add.

I agree that JCASC would be the ideal means of managing the rules, but as you 
noted is not currently available.

Thanks,
-daw-
On 2/3/2021 5:06 AM, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
wrote:

Nice.



new rules to the VPP Failure Cause Management database [0]



How do we manage the rules?

I think the database is different from usual plugin configuration,

so "Jenkins configuration as code" plugin will not help directly [5].



But I think we can manage the exported database manually in ci-management.



Vratko.



[5] https://issues.jenkins.io/browse/JENKINS-63773



-Original Message-

From: vpp-dev@lists.fd.io 
 On Behalf Of Benoit Ganne 
(bganne) via lists.fd.io

Sent: Wednesday, 2021-February-03 09:21

To: Dave Wallace ; vpp-dev 


Subject: Re: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications



Very nice!

Do you also get statistics per failure cause?



Best

ben



-Original Message-

From: vpp-dev@lists.fd.io 
 On Behalf Of Dave Wallace

Sent: mercredi 3 février 2021 02:28

To: vpp-dev 

Subject: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications



Folks,



Per a request by Dave Barach in the VPP Monthly Community Meeting, I

have added new rules to the VPP Failure Cause Management database [0] of

the Jenkins Build Analyzer plugin [1] to provide notifications in

gerrit/jenkins of build failure indications.  The rules are applied to

the log of each Jenkins job which fails to identify the failure cause in

the build log.



You can preview the notifications in the gerrit change [2] which I have

been using to verify that the rules work correctly.  You will see the

description of the build failure cause listed on the Gerrit notification

status line.  The jenkins job status page will contain links in the

"Identified problems" section of the page for each rule.



Please note that in there is an issue with the reporting of multiple

instances of same failure contained in a build log (or maybe that is a

non-feature of the plugin).  Presently only the 1st failure is listed in

the indications list on the jenkins build status page.  In particular,

in the case of multiple 'make test' testcase failures [3], you will need

to search the log for any additional failures beyond one linked to in

the 'Indication 1' link [4].



I will continue to monitor job failures for viable failure signatures

that I can add to the .  If you find a failure that you think is a good

candidate, then please directly email me the URL to the jenkins job

build status page along with the string to search for in the log and I

will add a new rule for it.



Thanks,

-daw-



[0] https://plugins.jenkins.io/build-failure-analyzer/

[1] https://jenkins.fd.io/view/vpp/failure-cause-management/

[2] https://gerrit.fd.io/r/c/vpp/+/31092

[3] https://jenkins.fd.io/job/vpp-verify-master-ubuntu1804-x86_64/2050/

[4]

https://jenkins.fd.io/job/vpp-verify-master-ubuntu1804-

x86_64/2050/consoleFull#-14603433817be1b593-44ad-45a7-8ca6-701783403621


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18661): https://lists.fd.io/g/vpp-dev/message/18661
Mute This Topic: https://lists.fd.io/mt/80343945/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Faster PAPI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> Hello people interested in PAPI (VPP's Python API client library).

Hello again.
This is an update e-mail, adding some information,
while still asking basically the same questions.

Since my first e-mail, there was some private communication,
mostly related to reasons the vanilla performance is not good,
and how improvements to VAT [2] can help.

> The exact code change is [0], but that may be hard to review.

The current patch set [3] is a little better.

> For that I have created [1], which shows the changed VPP PAPI code.

Still mostly unfinished, I need to familiarize better with shmem transport.

The main inputs came from Peter, who expressed
dislike [4] on how brittle the fast binary message generation is,
and he prefers "we will call a vector operation and PAPI just executes it".

Let me summarize the current options as I see them.

1. Keep the status quo.
That means using VAT for some operations (e.g. adding multiple routes [5]),
and creating "exec scripts" [6] for operations without VAT one-liner.
Pros: No work needed, good speed, old VPP versions are supported.
Cons: Relying on VAT functionality (outside API compatibility rules).

2. Support "vector operations" in VPP via binary API.
This will probably need a new VPP plugin to host the implementations.
Pros: Fast speed, small CSIT work, guarded by API compatibility rules.
Cons: New VPP plugin of questionable usefulness outside CSIT,
plugin maintainer needed, old VPP versions not supported.

3. VPP PAPI improvements only.
No changes to VPP API, just changes to PAPI to allow better speed for socket 
interactions.
CSIT would need a fast way to synthetize binary messages.
Pros: Small VPP work, good speed, only "official" VPP API is used.
Cons: Brittle CSIT message handling, old VPP versions not supported.

4. CSIT hacks only (Gerrit 26019).
No changes to VPP API nor PAPI. CSIT code messes with PAPI internals.
CSIT needs a fast way to synthetize binary messages.
Pros: Code is ready, good speed, old VPP versions are supported.
Cons: Brittle CSIT message handling, risky with respect to VPP PAPI changes.

The open questions:
Do you see any other options?
Did I miss some important pros or cons?
Which option do you prefer?

Vratko.

[2] https://lists.fd.io/g/vpp-dev/topic/78362835#18092
[3] https://gerrit.fd.io/r/c/csit/+/26019/140
[4] 
https://gerrit.fd.io/r/c/csit/+/26019/140#message-314d168d8951b539e588e644a875624f5ca3fb77
[5] 
https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/templates/vat/vpp_route_add.vat
[6] 
https://github.com/FDio/csit/blob/b5073afc4a941ea33ce874e016fe86384ae7a60d/resources/libraries/python/TestConfig.py#L121-L150

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Thursday, 2020-May-07 18:35
To: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Cc: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>
Subject: [vpp-dev] Faster PAPI

Hello people interested in PAPI (VPP's Python API client library).

In CSIT, our tests are using PAPI to interact with VPP.
We are using socket transport (instead of shared memory transport),
as VPP is running on machines separate from machines running the tests.
We use SSH to forward the socket between the machines.

Some of our scale tests need to send high number of commands towards VPP.
The largest test sends 4 million commands (ip_route_add_del with ip6 addresses).
You can imagine that can take a while.
Even using PAPI in asynchronous mode, it takes tens of minutes per million 
commands.

I was able to speed that up considerably, just by changing code on CSIT side.
The exact code change is [0], but that may be hard to review.
Gerrit does not even recognize the new PapiSocketExecutor.py
to be an edited copy of the old PapiExecutor.py file.

That code relies on the fact that Python is quite permissive language,
not really distinguishing private fields and methods from public ones.
So the current code is vulnerable to refactors of VPP PAPI code.
Also, pylint (static code analysis tool CSIT uses) is complaining.

The proper fix is to change the VPP PAPI code,
so that it exposes the inner parts the new CSIT code needs to access
(or some abstractions of them).

For that I have created [1], which shows the changed VPP PAPI code.
Commit message contains a simplified example of how the new features can be 
used.

The changed VPP code allows three performance improvements.

1. Capturing raw bytes sent.
For complicated commands, many CPU cycles are spent serializing
command arguments (basically nested python dicts) into bytes.
If user (CSIT code) has access to the message as serialized by PAPI (VPP code),
the user can choose a faster method to create subsequent serialized data.
Implementing this on CSIT side improved the speed, but not greatly enough.
(See bytes_te

Re: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Nice.

> new rules to the VPP Failure Cause Management database [0]

How do we manage the rules?
I think the database is different from usual plugin configuration,
so "Jenkins configuration as code" plugin will not help directly [5].

But I think we can manage the exported database manually in ci-management.

Vratko.

[5] https://issues.jenkins.io/browse/JENKINS-63773

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Benoit Ganne 
(bganne) via lists.fd.io
Sent: Wednesday, 2021-February-03 09:21
To: Dave Wallace ; vpp-dev 
Subject: Re: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications

Very nice!
Do you also get statistics per failure cause?

Best
ben

> -Original Message-
> From: vpp-dev@lists.fd.io  On Behalf Of Dave Wallace
> Sent: mercredi 3 février 2021 02:28
> To: vpp-dev 
> Subject: [vpp-dev] Jenkins/Gerrit Failure Cause Notifications
> 
> Folks,
> 
> Per a request by Dave Barach in the VPP Monthly Community Meeting, I
> have added new rules to the VPP Failure Cause Management database [0] of
> the Jenkins Build Analyzer plugin [1] to provide notifications in
> gerrit/jenkins of build failure indications.  The rules are applied to
> the log of each Jenkins job which fails to identify the failure cause in
> the build log.
> 
> You can preview the notifications in the gerrit change [2] which I have
> been using to verify that the rules work correctly.  You will see the
> description of the build failure cause listed on the Gerrit notification
> status line.  The jenkins job status page will contain links in the
> "Identified problems" section of the page for each rule.
> 
> Please note that in there is an issue with the reporting of multiple
> instances of same failure contained in a build log (or maybe that is a
> non-feature of the plugin).  Presently only the 1st failure is listed in
> the indications list on the jenkins build status page.  In particular,
> in the case of multiple 'make test' testcase failures [3], you will need
> to search the log for any additional failures beyond one linked to in
> the 'Indication 1' link [4].
> 
> I will continue to monitor job failures for viable failure signatures
> that I can add to the .  If you find a failure that you think is a good
> candidate, then please directly email me the URL to the jenkins job
> build status page along with the string to search for in the log and I
> will add a new rule for it.
> 
> Thanks,
> -daw-
> 
> [0] https://plugins.jenkins.io/build-failure-analyzer/
> [1] https://jenkins.fd.io/view/vpp/failure-cause-management/
> [2] https://gerrit.fd.io/r/c/vpp/+/31092
> [3] https://jenkins.fd.io/job/vpp-verify-master-ubuntu1804-x86_64/2050/
> [4]
> https://jenkins.fd.io/job/vpp-verify-master-ubuntu1804-
> x86_64/2050/consoleFull#-14603433817be1b593-44ad-45a7-8ca6-701783403621

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18653): https://lists.fd.io/g/vpp-dev/message/18653
Mute This Topic: https://lists.fd.io/mt/80343945/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Scapy license in VPP

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> Why do you say that vpp_papi need to be dual licensed?

I think e-mail reply would be long,
and I will need to address comments anyway,
so I respond via a Gerrit change [1].

Vratko.

[1] https://gerrit.fd.io/r/c/vpp/+/31025

From: Paul Vinciguerra 
Sent: Friday, 2021-January-29 15:29
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: t...@lists.fd.io; Kinsella, Ray ; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Scapy license in VPP

Why do you say that vpp_papi need to be dual licensed?

On Thu, Jan 28, 2021 at 12:43 PM Vratko Polak -X (vrpolak - PANTHEON 
TECHNOLOGIES at Cisco) via lists.fd.io<http://lists.fd.io> 
mailto:cisco@lists.fd.io>> wrote:
[cid:image001.gif@01D6F669.E8857E30]
First draft created [0] for the change that will switch
licenses for Python files used together with Scapy.

For some files, I was not sure whether they are used together with Scapy.
One big detail is that vpp_papi needs to have dual license,
as test framework integrates with it (and with scapy).
If I understand the licensing logic correctly,
CSIT tests can still choose to use vpp_papi under Apache license option.
But we may need to discuss that with lawyers.

Ray, you may need to upgrade your contributor-finding shell pipeline
to cover all files I added the new license into.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/30998



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18632): https://lists.fd.io/g/vpp-dev/message/18632
Mute This Topic: https://lists.fd.io/mt/80189290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Scapy license in VPP

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

First draft created [0] for the change that will switch
licenses for Python files used together with Scapy.

For some files, I was not sure whether they are used together with Scapy.
One big detail is that vpp_papi needs to have dual license,
as test framework integrates with it (and with scapy).
If I understand the licensing logic correctly,
CSIT tests can still choose to use vpp_papi under Apache license option.
But we may need to discuss that with lawyers.

Ray, you may need to upgrade your contributor-finding shell pipeline
to cover all files I added the new license into.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/30998

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18622): https://lists.fd.io/g/vpp-dev/message/18622
Mute This Topic: https://lists.fd.io/mt/80189290/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] devicetest verify failures

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

When new CSIT oper branch got created today,
VPP verify jobs started to failing, even though each test passed.
That should be fixed [0] now [1].

We are working on verify job improvements on CSIT side
to prevent similar issues in the future.

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/30329
[1] https://gerrit.fd.io/r/c/csit/+/30299

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18267): https://lists.fd.io/g/vpp-dev/message/18267
Mute This Topic: https://lists.fd.io/mt/78781748/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] event-logger debug CLI cleanup patch failing validation: [vpp gerrit 30124]

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

In the meantime, [0] got merged everywhere important.
You can try to remove the old CLI to see if it passes now.

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach
Sent: Thursday, 2020-November-26 14:36
To: 'Andrew 👽 Yourtchenko' ; Vratko Polak -X (vrpolak - 
PANTHEON TECH SRO at Cisco) 
Cc: csit-...@lists.fd.io; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] [csit-dev] event-logger debug CLI cleanup patch failing 
validation: [vpp gerrit 30124]

Thanks for the pointer(s), guys. Fixed, new patch-set uploaded...

From: Andrew 👽 Yourtchenko mailto:ayour...@gmail.com>>
Sent: Thursday, November 26, 2020 8:18 AM
To: Dave Barach mailto:v...@barachs.net>>
Cc: vrpo...@cisco.com<mailto:vrpo...@cisco.com>; 
csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] [csit-dev] event-logger debug CLI cleanup patch failing 
validation: [vpp gerrit 30124]

I see the failure in the doc verify job:


22:49:10 Processing siphon for markdown from clicmd.siphon...

22:49:10 INFO:siphon.process.clicmd:Parsing items in file 
"build-root/docs/siphon_fragments/clicmd.siphon".

22:49:12 ERROR:siphon.process.clicmd:Exception parsing item: {

22:49:12 "block": "VLIB_CLI_COMMAND (elog_trace_command, static) =\n ;\n",

22:49:12 "directory": "src/vlib",

22:49:12 "file": "src/vlib/cli.c",

22:49:12 "line_end": 1592,

22:49:12 "line_start": 1591,

22:49:12 "siphon_block": ""

22:49:12 }

22:49:12 VLIB_CLI_COMMAND (elog_trace_command, static) =

22:49:12  ;

22:49:12

22:49:12 Traceback (most recent call last):

22:49:12   File 
"/w/workspace/vpp-docs-verify-master-ubuntu1804-x86_64/doxygen/siphon-process", 
line 72, in 

22:49:12 processor.load_json(args.input)

22:49:12   File 
"/w/workspace/vpp-docs-verify-master-ubuntu1804-x86_64/doxygen/siphon/process.py",
 line 203, in load_json

22:49:12



“make doxygen” locally shows the problem as well.



Removing the space in front of the open/close braces in the elog trace block 
makes it happy. 🤷‍♂️ FWIW.





--a

On 26 Nov 2020, at 13:36, Dave Barach 
mailto:v...@barachs.net>> wrote:

OK, thanks for [0]. Even with the old debug CLI supported on the vpp side, the 
patch continues to fail validation. I’m going to try again (“recheck”) and see 
what happens.

D.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io
Sent: Thursday, November 26, 2020 4:47 AM
To: Dave Barach mailto:v...@barachs.net>>
Cc: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] [csit-dev] event-logger debug CLI cleanup patch failing 
validation: [vpp gerrit 30124]

Compatibility enhancement on CSIT side: [0].

> easy enough to support both “elog trace” and “event-logger trace” on the vpp 
> side.

When [0] is merged (also to oper branch),
the support on VPP side can be dropped.

> add a binary API and use it in the csit robot framework

+1 to that, especially if the API is marked as Production,
so it is covered by [1].

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/30147
[1] https://wiki.fd.io/view/VPP/ApiChangeProcess

From: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io> 
mailto:csit-...@lists.fd.io>> On Behalf Of Dave Barach
Sent: Wednesday, 2020-November-25 22:42
To: csit-...@lists.fd.io<mailto:csit-...@lists.fd.io>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: [csit-dev] event-logger debug CLI cleanup patch failing validation: 
[vpp gerrit 30124]

/resource/libraries/python/VPPUtil.py appears responsible:

@staticmethod
def vpp_enable_elog_traces(node):
"""Enable API/CLI/Barrier traces on the specified topology node.

:param node: Topology node.
:type node: dict
"""
PapiSocketExecutor.run_cli_cmd_on_all_sockets(
node, u"elog trace api cli barrier")

For the moment, it’s easy enough to support both “elog trace” and “event-logger 
trace” on the vpp side.

The obvious medium-term answer would be to add a binary API and use it in the 
csit robot framework, instead of depending on debug CLI [which is never a Good 
Idea].

FWIW... Dave


-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18155): https://lists.fd.io/g/vpp-dev/message/18155
Mute This Topic: https://lists.fd.io/mt/78526554/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] event-logger debug CLI cleanup patch failing validation: [vpp gerrit 30124]

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Compatibility enhancement on CSIT side: [0].

> easy enough to support both “elog trace” and “event-logger trace” on the vpp 
> side.

When [0] is merged (also to oper branch),
the support on VPP side can be dropped.

> add a binary API and use it in the csit robot framework

+1 to that, especially if the API is marked as Production,
so it is covered by [1].

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/30147
[1] https://wiki.fd.io/view/VPP/ApiChangeProcess

From: csit-...@lists.fd.io  On Behalf Of Dave Barach
Sent: Wednesday, 2020-November-25 22:42
To: csit-...@lists.fd.io; vpp-dev@lists.fd.io
Subject: [csit-dev] event-logger debug CLI cleanup patch failing validation: 
[vpp gerrit 30124]

/resource/libraries/python/VPPUtil.py appears responsible:

@staticmethod
def vpp_enable_elog_traces(node):
"""Enable API/CLI/Barrier traces on the specified topology node.

:param node: Topology node.
:type node: dict
"""
PapiSocketExecutor.run_cli_cmd_on_all_sockets(
node, u"elog trace api cli barrier")

For the moment, it’s easy enough to support both “elog trace” and “event-logger 
trace” on the vpp side.

The obvious medium-term answer would be to add a binary API and use it in the 
csit robot framework, instead of depending on debug CLI [which is never a Good 
Idea].

FWIW... Dave

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18149): https://lists.fd.io/g/vpp-dev/message/18149
Mute This Topic: https://lists.fd.io/mt/78526554/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Python API modules

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> something has gone wrong here.

Yeah, sorry for the confusion.

I consult the maintainers file when double-checking who to add
as reviewers. But when I think the maintainers file is outdated,
I do not hesitate to add [0] or remove [1] anybody
based on my (limited) perception.

Usually I am too focused on fixing VPP
to properly think about the process of editing the maintainers file.
Do we have a document with guidelines related to that?

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/20366
[1] https://gerrit.fd.io/r/c/vpp/+/22672

From: vpp-dev@lists.fd.io  On Behalf Of Ole Troan
Sent: Monday, 2020-November-16 15:16
To: Paul Vinciguerra 
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Marcos - Mgiga ; vpp-dev 
Subject: Re: [vpp-dev] Python API modules

Hi Paul,

> He/Vratko removed me as one of the maintainers of papi.  
> https://gerrit.fd.io/r/c/vpp/+/22672.  I'm cool with not being a maintainer, 
> kinda funny that it was stuffed into another changeset.  I have been called 
> out repeatedly for submitting unrelated changes ;)

My apologies, something has gone wrong here. I wasn't even aware that you got 
removed.
I have reviewed the history and spoken to Vratko to understand what has 
happened here. Vratko was editing the MAINTAINERS file and made changes in good 
faith reflecting what he believed was the current state. Expecting reviewers to 
comment. I missed that in code review, and the patch went in. I do not believe 
that's a correct way of doing things, and since this was done in error, I'd 
like to add you back as maintainer for PAPI. That is revert to the state prior 
to that patch.

Apologies again, and let's fix that error.

Best regards,
Ole



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.
View/Reply Online (#18043): https://lists.fd.io/g/vpp-dev/message/18043
Mute This Topic: https://lists.fd.io/mt/78229638/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Increasing NAT worker handoff frame queue size NAT_FQ_NELTS to avoid congestion drops?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

>> Would you consider changing to a larger value in the official VPP code?

Maybe make it configurable?
I mean, after 28980 is merged and you still find tweaking the value helpful.

Vratko.

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Klement Sekera via 
lists.fd.io
Sent: Friday, 2020-November-13 16:15
To: Elias Rudberg 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Increasing NAT worker handoff frame queue size 
NAT_FQ_NELTS to avoid congestion drops?

Hi Elias,

I’ve already debugged this and came to the conclusion that it’s the infra which 
is the weak link. I was seeing congestion drops at mild load, but not at full 
load. Issue is that with handoff, there is uneven workload. For simplicity’s 
sake, just consider thread 1 handing off all the traffic to thread 2. What 
happens is that for thread 1, the job is much easier, it just does some ip4 
parsing and then hands packet to thread 2, which actually does the heavy 
lifting of hash inserts/lookups/translation etc. 64 element queue can hold 64 
frames, one extreme is 64 1-packet frames, totalling 64 packets, other extreme 
is 64 255-packet frames, totalling ~16k packets. What happens is this: thread 1 
is mostly idle and just picking a few packets from NIC and every one of these 
small frames creates an entry in the handoff queue. Now thread 2 picks one 
element from the handoff queue and deals with it before picking another one. If 
the queue has only 3-packet or 10-packet elements, then thread 2 can never 
really get into what VPP excels in - bulk processing.

Q: Why doesn’t it pick as many packets as possible from the handoff queue? 
A: It’s not implemented.

I already wrote a patch for it, which made all congestion drops which I saw (in 
above synthetic test case) disappear. Mentioned patch 
https://gerrit.fd.io/r/c/vpp/+/28980 is sitting in gerrit.

Would you like to give it a try and see if it helps your issue? We shouldn’t 
need big queues under mild loads anyway …

Regards,
Klement

> On 13 Nov 2020, at 16:03, Elias Rudberg  wrote:
> 
> Hello VPP experts,
> 
> We are using VPP for NAT44 and we get some "congestion drops", in a
> situation where we think VPP is far from overloaded in general. Then
> we started to investigate if it would help to use a larger handoff
> frame queue size. In theory at least, allowing a longer queue could
> help avoiding drops in case of short spikes of traffic, or if it
> happens that some worker thread is temporarily busy for whatever
> reason.
> 
> The NAT worker handoff frame queue size is hard-coded in the
> NAT_FQ_NELTS macro in src/plugins/nat/nat.h where the current value is
> 64. The idea is that putting a larger value there could help.
> 
> We have run some tests where we changed the NAT_FQ_NELTS value from 64
> to a range of other values, each time rebuilding VPP and running an
> identical test, a test case that is to some extent trying to mimic our
> real traffic, although of course it is simplified. The test runs many
> iperf3 tests simultaneously using TCP, combined with some UDP traffic
> chosen to trigger VPP to create more new sessions (to make the NAT
> "slowpath" happen more).
> 
> The following NAT_FQ_NELTS values were tested:
> 16
> 32
> 64  <-- current value
> 128
> 256
> 512
> 1024
> 2048  <-- best performance in our tests
> 4096
> 8192
> 16384
> 32768
> 65536
> 131072
> 
> In those tests, performance was very bad for the smallest NAT_FQ_NELTS
> values of 16 and 32, while values larger than 64 gave improved
> performance. The best results in terms of throughput were seen for
> NAT_FQ_NELTS=2048. For even larger values than that, we got reduced
> performance compared to the 2048 case.
> 
> The tests were done for VPP 20.05 running on a Ubuntu 18.04 server
> with a 12-core Intel Xeon CPU and two Mellanox mlx5 network cards. The
> number of NAT threads was 8 in some of the tests and 4 in some of the
> tests.
> 
> According to these tests, the effect of changing NAT_FQ_NELTS can be
> quite large. For example, for one test case chosen such that
> congestion drops were a significant problem, the throughput increased
> from about 43 to 90 Gbit/second with the amount of congestion drops
> per second reduced to about one third. In another kind of test,
> throughput increased by about 20% with congestion drops reduced to
> zero. Of course such results depend a lot on how the tests are
> constructed. But anyway, it seems clear that the choice of
> NAT_FQ_NELTS value can be important and that increasing it would be
> good, at least for the kind of usage we have tested now.
> 
> Based on the above, we are considering changing NAT_FQ_NELTS from 64
> to a larger value and start trying that in our production environment
> (so far we have only tried it in a test environment).
> 
> Were there specific reasons for setting NAT_FQ_NELTS to 64?
> 
> Are there some potential drawbacks or dangers of changing it to a
> larger value?
> 
> Would you consider changing to a larger value in the offici

Re: [vpp-dev] Issue faced while deleting segment list in sr policy

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> the value we get in *sl_index_iterate at line 870 is 0 rather than the 4.

The pointer sl_index_iterate points to what would be the next index in the list.

> why this is happening?

That is the normal thing for-loops [0] in C do, if break does not happen.
See [1].

A fix could be something simple like [2].

Vratko.

[0] 
https://github.com/FDio/vpp/blob/stable/2005/src/vppinfra/vec_bootstrap.h#L211
[1] https://stackoverflow.com/a/20302114
[2] https://gerrit.fd.io/r/c/vpp/+/27972

From: vpp-dev@lists.fd.io  On Behalf Of Chinmaya Aggarwal
Sent: Monday, 2020-July-20 14:43
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Issue faced while deleting segment list in sr policy

Hi,
We are testing VAPI for sr policy.
We added two sr policies :-

Policy 1
bsid : 3d97:5fd3:fd2b:e77:a0:d358:c873:9a98
SID List at index 0 : 2001:19:a1::1,2001:19:a2::1,2001:19:a3::1
SID List at index 1 : 2001:19:b1::1,2001:19:b2::1,2001:19:b3::1
SID List at index 2 : 2001:19:c1::1,2001:19:c2::1,2001:19:c3::1

Policy 2
bsid : 6d97:5fd3:fd2b:e77:a0:d358:c873:9a98
SID List at index 3 : 2001:19:d1::1,2001:19:d3::1
SID List at index 4 : 2001:19:e1::1,2001:19:e2::1,2001:19:e3::1

We are trying to delete the segment list at index 0 for policy 2 (ideally it 
should return an error). But on executing sr_policy_mod api, our application 
along with VPP gets hanged. On debugging the code we observed that in the below 
code snippet

File : /src/vnet/srv6/sr_policy_rewrite.c
Function : sr_policy_mod

865  /* Check that the SR list does exist and is assigned to the sr policy 
*/
866  vec_foreach (sl_index_iterate, sr_policy->segments_lists)
867  if (*sl_index_iterate == sl_index)
868break;
869
870  if (*sl_index_iterate != sl_index)

After all the indexes in sr_policy->segments_lists are iterated, the value we 
get in *sl_index_iterate at line 870 is 0 rather than the 4. Somehow, after the 
complete iteration, the index is reset to 0.

Can anyone please suggest why this is happening?


Thanks and Regards,
Chinmaya Agarwal.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#17004): https://lists.fd.io/g/vpp-dev/message/17004
Mute This Topic: https://lists.fd.io/mt/75680646/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [csit-dev] [vpp-dev] Vectors/node and packet size

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

>> on all the logic hidden behind rte_eth_rx_burst call,

> won’t rte_eth_rx_burst just end up using the pmd registered functions.

Yes. But I personally do not have visibility to DPDK software,
not to mention NIC firmware and hardware.

From the fact that small-ish traffic does not lead to 1 vector/node,
I conclude batching happens somewhere. And I do not see any batching
in VPP code (except the 32 packet threshold one).

> why would the rx threshold or the number of bits in the vector matter to us?

The threshold is there to increase efficiency.
If less then 32 packets are read, traffic is probably not very high,
and VPP chooses to reduce latency by processing packets immediately.
If 32 or more packets are read, traffic is probably high enough
for risk of packet loss becoming more important than latency.
In that case, VPP attempts immediate reads,
to increase vectors/node even more for the processing phase.

There is a possibility for experimentation, by re-compiling VPP
with a different threshold value, to see how does latency, throughput
and vectors/node change.

> there is no copying or extra storage involved
> Is there some other memory limitation?

Once again, I personally have no clear idea.
If I had to guess, I would say there is "copying" over PCI
(between NIC and CPU cache). Packet size matters there.

Vratko.

From: csit-...@lists.fd.io  On Behalf Of Jeremy Brown via 
lists.fd.io
Sent: Monday, 2020-July-06 16:56
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Dave Barach (dbarach) ; vpp-dev@lists.fd.io; Dany Gregoire 

Cc: csit-...@lists.fd.io
Subject: Re: [csit-dev] [vpp-dev] Vectors/node and packet size

Thanks for you reply.. I looked at your references, and I did notice in 
docs.fd.io they seem to see the same behavior, which is good to corroborate 
what were seeing… but I am still unclear as to why… I assumed that we are 
simply dealing with pointers to the pkt, and there is no copying or extra 
storage involved… why would the rx threshold or the number of bits in the 
vector matter to us? Is there some other memory limitation?

Also, won’t rte_eth_rx_burst just end up using the pmd registered functions. We 
noticed the same behavior in both i40e and mlx drivers. I’m not saying that 
both pmd drivers can’t have a similar problem, just a data point.

Sent from Mail for Windows 10

From: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at 
Cisco)
Sent: Monday, July 6, 2020 8:10 AM
Subject: RE: [vpp-dev] Vectors/node and packet size

> I was wondering if anyone had noticed some similar behavior.

Not until you mentioned it, but the pattern is visible in CSIT results.
See for example here [0] (the page is very long,
and you have to scroll over multicore results to compare packet sizes).

> dpdk-input

In your result, calls outnumber vectors 100:1. As the subsequent nodes
are only called if there was a non-zero number of vectors at input,
VPP mostly waits for DPDK to read new packets.

In the CSIT results, pps is higher,
and VPP spends more time actually processing the packets.

> Using a 64 byte packet, we see a vectors/node of ~80.

One packet takes (64 + 20) * 8 = 672 bits on wire,
so one node call handles ~53760 bits.

> Simply changing that packet size to 1400 we see the same vectors/node fall 
> down to ~2.

2 * (1400 + 20) * 8 = 22720 bits per node call.
That is ~42% of the previous value.

> there seems to be a non-linear decrease

In VPP code I see one non-linearity here [1].
The threshold value (32 packets per call)
is 40% of the observed ~80 value, so that might be it.
Assuming DPDK reads return with a fixed frequency.

But of course, the real behavior heavily depends
on all the logic hidden behind rte_eth_rx_burst call,
and we know [2] that can be complicated.

Vratko.

[0] 
https://docs.fd.io/csit/rls2001/report/test_operational_data/vpp_performance_operational_data_2n_clx/ip4_xxv710.html#n1l-25ge2p1xxv710-ethip4-ip4base-ndrpdr
[1] 
https://github.com/FDio/vpp/blob/dfb19cabe20ccf1cbd1aa714f493ccd322839b91/src/plugins/dpdk/device/node.c#L316-L317
[2] https://jira.fd.io/browse/VPP-1876

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Jeremy Brown via 
lists.fd.io
Sent: Wednesday, 2020-July-01 18:10
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>; 
vpp-dev@lists.fd.io; Dany Gregoire 
mailto:dany_grego...@affirmednetworks.com>>
Subject: Re: [vpp-dev] Vectors/node and packet size

VPP is restarted before each run, all test runs are the same duration. There is 
no dead air time.

Sent from Mail for Windows 10

From: Dave Barach (dbarach)
Sent: Wednesday, July 1, 2020 9:52 AM
To: bjerem...@yahoo.com; 
vpp-dev@lists.fd.io; Dany 
Gregoire
Subject: RE: [vpp-d

Re: [vpp-dev] Vectors/node and packet size

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> I was wondering if anyone had noticed some similar behavior.

Not until you mentioned it, but the pattern is visible in CSIT results.
See for example here [0] (the page is very long,
and you have to scroll over multicore results to compare packet sizes).

> dpdk-input

In your result, calls outnumber vectors 100:1. As the subsequent nodes
are only called if there was a non-zero number of vectors at input,
VPP mostly waits for DPDK to read new packets.

In the CSIT results, pps is higher,
and VPP spends more time actually processing the packets.

> Using a 64 byte packet, we see a vectors/node of ~80.

One packet takes (64 + 20) * 8 = 672 bits on wire,
so one node call handles ~53760 bits.

> Simply changing that packet size to 1400 we see the same vectors/node fall 
> down to ~2.

2 * (1400 + 20) * 8 = 22720 bits per node call.
That is ~42% of the previous value.

> there seems to be a non-linear decrease

In VPP code I see one non-linearity here [1].
The threshold value (32 packets per call)
is 40% of the observed ~80 value, so that might be it.
Assuming DPDK reads return with a fixed frequency.

But of course, the real behavior heavily depends
on all the logic hidden behind rte_eth_rx_burst call,
and we know [2] that can be complicated.

Vratko.

[0] 
https://docs.fd.io/csit/rls2001/report/test_operational_data/vpp_performance_operational_data_2n_clx/ip4_xxv710.html#n1l-25ge2p1xxv710-ethip4-ip4base-ndrpdr
[1] 
https://github.com/FDio/vpp/blob/dfb19cabe20ccf1cbd1aa714f493ccd322839b91/src/plugins/dpdk/device/node.c#L316-L317
[2] https://jira.fd.io/browse/VPP-1876

From: vpp-dev@lists.fd.io  On Behalf Of Jeremy Brown via 
lists.fd.io
Sent: Wednesday, 2020-July-01 18:10
To: Dave Barach (dbarach) ; vpp-dev@lists.fd.io; Dany 
Gregoire 
Subject: Re: [vpp-dev] Vectors/node and packet size

VPP is restarted before each run, all test runs are the same duration. There is 
no dead air time.

Sent from Mail for Windows 10

From: Dave Barach (dbarach)
Sent: Wednesday, July 1, 2020 9:52 AM
To: bjerem...@yahoo.com; 
vpp-dev@lists.fd.io; Dany 
Gregoire
Subject: RE: [vpp-dev] Vectors/node and packet size

In order for the statistics to be accurate, please be sure to do the following:

Start traffic... “clear run”... wait a while to accumulate data... “show run”

Otherwise, the statistics will probably include a huge amount of dead airtime, 
data from previous runs, etc.

HTH... Dave

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Jeremy Brown via 
lists.fd.io
Sent: Monday, June 29, 2020 12:22 PM
To: vpp-dev@lists.fd.io; Dany Gregoire 
mailto:dany_grego...@affirmednetworks.com>>
Subject: [vpp-dev] Vectors/node and packet size

Greetings,

This is my first post to the forum, so if this is not the right place for this 
post please let me know.

I had a question on VPP performance. We are running two testcases, we limit it 
to single threaded and just using one core in order to reduce as many variables 
as we can. In the two testcases, the only thing that changes, is the size 
incoming packet to VPP.

Using a 64 byte packet, we see a vectors/node of ~80. Simply changing that 
packet size to 1400 we see the same vectors/node fall down to ~2.

This is regardless of pps… there seems to be a non-linear decrease in 
vectors/node with increasing packet size. I was wondering if anyone had noticed 
some similar behavior.


64- byte packets

Thread 1 vpp_wk_0 (lcore 2)
Time 98.9, average vectors/node 80.35, last 128 main loops 0.00 per node 0.00
  vector rates in 1.2643e5, out 1.2643e5, drop 0.e0, punt 2.0228e-2
 Name State Calls  Vectors
Suspends Clocks   Vectors/Call
VirtualFuncEthernet88/10/4-out   active  90915 6249981  
 0  1.06e1   68.75
VirtualFuncEthernet88/10/4-txactive  90915 6249981  
 0  4.06e1   68.75
VirtualFuncEthernet88/11/5-out   active  73270 6249981  
 0  9.27e0   85.30
VirtualFuncEthernet88/11/5-txactive  73270 6249981  
 0  4.05e1   85.30
arp-inputactive  2   2  
 0  3.51e41.00
dpdk-input   polling116612933712499964  
 0  1.38e4 .01
error-punt   active  2   2  
 0  5.56e31.00
ethernet-input   active  2   2  
 0  1.47e41.00
gtpu4-encap  active  90914 6249980 

Re: [vpp-dev] ikev2 API & new API change process

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

I think there are two issues.

One issue is with APIs that were not intended to be Production yet.
For ikev2 that basically means the version has been set [5]
above 0.x.y only by mistake.
As downgrading versions is always risky,
marking messages as "In-Progress" instead is better.

The other issue is how to determine which APIs are really Production,
and which are In-Progress with wrong version value.
I believe every existing API should be reviewed,
before we claim the new process [3] is applied.
There are sub-issues, like who will decide, how much testing is needed,
and how widespread a usage implies an API is effectively Production already.

In contrast, the question of how to deal with Production APIs
(existence of which hinders the work on legitimate improvements)
is a non-issue.
The new process has good reasons to force the expensive way with deprecation.

Personally, I think it would be a good idea to mark ALL APIs as In-Progress,
as it matches the (lack of) guarantees in previous releases,
and let maintainers mark some messages as Production on their own pace.

Vratko.

[5] https://gerrit.fd.io/r/c/vpp/+/17803/3/src/plugins/ikev2/ikev2.api

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Benoit Ganne 
(bganne) via lists.fd.io
Sent: Tuesday, 2020-June-16 18:29
To: vpp-dev@lists.fd.io
Cc: Filip Tehlar -X (ftehlar - PANTHEON TECH SRO at Cisco) ; 
Andrew Yourtchenko (ayourtch) 
Subject: [vpp-dev] ikev2 API & new API change process

Hi all,

The ikev2 plugin is currently getting some much needed love [1] [2] but it also 
means that its API is starting to show its age.
We'd like to mark it as "In-Progress" under the new API change process [3] so 
that we can quickly improve it.
The rationale is:
 - the requirement for an API to transition to "In-Production" is for the API 
to be stable and *tested* - there are currently no ikev2 unit test at all, and 
in fact the 1st proposed API change was motivated by the introduction of the 
initial test cases [4]
 - we are not aware of any users of the ikev2 plugin API today apart from the 
Calico-VPP integration

Of course, a possibility would be to deprecate the existing one and to 
introduce a new one as "In-Progress" at the expense of more work.

What does the community think here?

Thanks
ben

[1] https://jira.fd.io/browse/VPP-1889
[2] https://jira.fd.io/browse/VPP-1904
[3] https://wiki.fd.io/view/VPP/ApiChangeProcess
[4] https://gerrit.fd.io/r/c/vpp/+/27536
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16743): https://lists.fd.io/g/vpp-dev/message/16743
Mute This Topic: https://lists.fd.io/mt/74919923/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] VPP-1876

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Just a notification for people not monitoring VPP Jira.
I have just opened https://jira.fd.io/browse/VPP-1876
"Worse performance with DPDK driver when MTU is set to 2022 or less"
I may update the value to 2023 depending on result of a scheduled run.
As explained in the Description, this is an old issue,
not reported before just because in CSIT it appears as a progression.

Vratko.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16470): https://lists.fd.io/g/vpp-dev/message/16470
Mute This Topic: https://lists.fd.io/mt/74376797/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Faster PAPI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Hello people interested in PAPI (VPP's Python API client library).

In CSIT, our tests are using PAPI to interact with VPP.
We are using socket transport (instead of shared memory transport),
as VPP is running on machines separate from machines running the tests.
We use SSH to forward the socket between the machines.

Some of our scale tests need to send high number of commands towards VPP.
The largest test sends 4 million commands (ip_route_add_del with ip6 addresses).
You can imagine that can take a while.
Even using PAPI in asynchronous mode, it takes tens of minutes per million 
commands.

I was able to speed that up considerably, just by changing code on CSIT side.
The exact code change is [0], but that may be hard to review.
Gerrit does not even recognize the new PapiSocketExecutor.py
to be an edited copy of the old PapiExecutor.py file.

That code relies on the fact that Python is quite permissive language,
not really distinguishing private fields and methods from public ones.
So the current code is vulnerable to refactors of VPP PAPI code.
Also, pylint (static code analysis tool CSIT uses) is complaining.

The proper fix is to change the VPP PAPI code,
so that it exposes the inner parts the new CSIT code needs to access
(or some abstractions of them).

For that I have created [1], which shows the changed VPP PAPI code.
Commit message contains a simplified example of how the new features can be 
used.

The changed VPP code allows three performance improvements.

1. Capturing raw bytes sent.
For complicated commands, many CPU cycles are spent serializing
command arguments (basically nested python dicts) into bytes.
If user (CSIT code) has access to the message as serialized by PAPI (VPP code),
the user can choose a faster method to create subsequent serialized data.
Implementing this on CSIT side improved the speed, but not greatly enough.
(See bytes_template.py in [0] for the faster data generator.)
The VPP code [1] introduces fields remember_sent and last_sent.

2. Reading replies without de-serializing them.
This was already possible by calling transport.q.get(),
but had next to no effect on PAPI speed.
Replies are usually short, so deserialization does not take too many cycles.

3. Disabling the background reader thread.
By default, socket transport creates (upon connect) a background thread,
which select()s on the socket, reads any messages,
and put()s them to transport.q (multiprocessing.Queue).
I am not sure whether it is the multithreading (waiting for Python interpreter
to switch between threads), or Queue (locks, its own reader thread),
but overall this was the remaining bottleneck.
The VPP code exposes public methods for stopping and starting the thread.

Back to point 2:
With the reading thread stopped, transport.q is not filled,
so another way to read the reply is needed.
The VPP code contained transport._read(),
the underscore hinting this is an internal method
(leading to the abovementioned pylint complaints).
The VPP change [1] renames that method to read_message(),
adding a docstring explaining it has to be used
when the reading thread is stopped.

Finally, with all 3 improvements, CSIT will be able
to execute million PAPI commands in around 15 seconds.

Even if something like [1] is merged to VPP,
CSIT will still use [0] for some time,
so we are able to test older VPP versions.

So, any comments on [1], or other ideas
on what changes are needed on VPP side
so users can achieve good PAPI speed using public PAPI methods?

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/26019/108
[1] https://gerrit.fd.io/r/c/vpp/+/26946/1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16274): https://lists.fd.io/g/vpp-dev/message/16274
Mute This Topic: https://lists.fd.io/mt/74054504/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> 3n-hsw

Around 2nd of April there happened a regression
in eth-l2bdscale1mmaclrn test, visible here [0].
Trending needed multiple runs to identify it is there,
and alerting is configured not to report "old" regressions
(so it did not report this one).

Anyway, the closer look shows previously the test had
larger stdev, and even closer look shows there was small
duration stretching going on.
The regression than happened when [1] fixed it by adding TRex workers.
Unintentionally, as it was fixing primarily Mellanox tests.
See [2] for more details on that.

Vratko.

[0] https://docs.fd.io/csit/master/trending/trending/l2-3n-hsw-xl710.html#t1c
[1] https://gerrit.fd.io/r/c/csit/+/26262
[2] https://lists.fd.io/g/csit-report/message/2406

-Original Message-
From: csit-rep...@lists.fd.io  On Behalf Of 
nore...@jenkins.fd.io
Sent: Friday, 2020-April-03 18:07
To: Fdio+Csit-Report via Email Integration 
Subject: [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email

Following regressions occured in the last trending job runs, listed per testbed 
type.



2n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/881, 
VPP version: 20.05-rc0~433-g0c7aa7ab5~b930

No regressions

3n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/862, 
VPP version: 20.05-rc0~433-g0c7aa7ab5~b930

tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-2t1c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-4t2c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-8t4c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.vm 
vhost.25ge2p1xxv710-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-8t4c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr


2n-clx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/282, 
VPP version: 20.05-rc0~456-g57a5a2df5~b953

tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-2t1c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-4t2c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-ethip4-ip4base-eth-2memif-1dcr-mrr.64b-8t4c-avf-ethip4-ip4base-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-100ge2p1cx556a-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-100ge2p1cx556a-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp

Re: [vpp-dev] [csit-report] Regressions as of 2020-04-18 14:00:15 UTC #email

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> -rnd-mrr

This is a consequence of [0],
fixing an old bug in CSIT code.
Previously, the traffic was not random enough.

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/26456

-Original Message-
From: csit-rep...@lists.fd.io  On Behalf Of 
nore...@jenkins.fd.io
Sent: Saturday, 2020-April-18 17:51
To: Fdio+Csit-Report via Email Integration 
Subject: [csit-report] Regressions as of 2020-04-18 14:00:15 UTC #email

Following regressions occured in the last trending job runs, listed per testbed 
type.



2n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/886, 
VPP version: 20.05-rc0~545-g8daeea9a5~b1042

tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale2m-rnd-mrr.64b-2t1c-avf-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale2m-rnd-mrr.64b-4t2c-avf-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale2m-rnd-mrr.64b-2t1c-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-avf-dot1q-ip6base-mrr.78b-2t1c-avf-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-dot1q-ip6base-mrr.78b-2t1c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-avf-dot1q-ip6base-mrr.78b-4t2c-avf-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-dot1q-ip6base-mrr.78b-4t2c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-avf-dot1q-ip6base-mrr.78b-8t4c-avf-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-dot1q-ip6base-mrr.78b-8t4c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-avf-dot1q-ip6base-mrr.78b-2t1c-avf-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-avf-dot1q-ip6base-mrr.78b-4t2c-avf-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-avf-dot1q-ip6base-mrr.78b-8t4c-avf-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-dot1q-ip6base-mrr.78b-2t1c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-dot1q-ip6base-mrr.78b-8t4c-dot1q-ip6base-mrr
tests.vpp.perf.l2.2n1l-10ge2p1x710-avf-eth-l2xcbase-mrr.64b-2t1c-avf-eth-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-dot1q-l2xcbase-mrr.64b-2t1c-avf-dot1q-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-dot1q-l2xcbase-mrr.64b-2t1c-dot1q-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-dot1q-l2xcbase-mrr.64b-4t2c-dot1q-l2xcbase-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc-mrr.64b-8t4c-avf-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc-mrr


2n-clx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/297, 
VPP version: 20.05-rc0~554-gce815deb7~b1051

tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale2m-rnd-mrr.64b-2t1c-avf-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale200k-rnd-mrr.64b-2t1c-avf-ethip4-ip4scale200k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale2m-rnd-mrr.64b-4t2c-avf-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale200k-rnd-mrr.64b-4t2c-avf-ethip4-ip4scale200k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale20k-rnd-mrr.64b-2t1c-ethip4-ip4scale20k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale200k-rnd-mrr.64b-2t1c-ethip4-ip4scale200k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale2m-rnd-mrr.64b-2t1c-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale20k-rnd-mrr.64b-4t2c-ethip4-ip4scale20k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale200k-rnd-mrr.64b-4t2c-ethip4-ip4scale200k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale2m-rnd-mrr.64b-4t2c-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip6.2n1l-100ge2p1cx556a-rdma-dot1q-ip6base-mrr.78b-2t1c-rdma-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-100ge2p1cx556a-rdma-dot1q-ip6base-mrr.78b-4t2c-rdma-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-100ge2p1cx556a-rdma-dot1q-ip6base-mrr.78b-8t4c-rdma-dot1q-ip6base-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-4t2c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc-mrr.64b-4t2c-avf-eth-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc-mrr


3n-hsw, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master/1148, VPP 
version: 20.05-rc0~554-gce815deb7~b1051

No regressions

3n-tsh, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-tsh/182, 
VPP version: 20.05-rc0~554-gce815deb7~b3316

No regressions

3n-dnv, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-dnv/389, 
VPP version: 20.05-rc0~554-gce815deb7~b1051

No regressions



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16118): https://lists.fd.io/g/vpp-dev/message/16118
Mute This Topic: https://lists.fd.io/mt/73147141/21656
Mute #email: https://lists.fd.io/mk?hashtag=email&subid=1480452
Group Owner: vpp-dev+ow...@lists.fd

Re: [vpp-dev] [csit-report] Progressions as of 2020-04-19 02:00:13 UTC #email

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> 3n-hsw

Progressions showing the fixes described in [4]
do restore the previous performance.

Vratko.

[4] https://lists.fd.io/g/csit-report/message/2488

-Original Message-
From: csit-rep...@lists.fd.io  On Behalf Of 
nore...@jenkins.fd.io
Sent: Sunday, 2020-April-19 06:07
To: Fdio+Csit-Report via Email Integration 
Subject: [csit-report] Progressions as of 2020-04-19 02:00:13 UTC #email

Following progressions occured in the last trending job runs, listed per 
testbed type.



2n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/886, 
VPP version: 20.05-rc0~545-g8daeea9a5~b1042

tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-avf-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-avf-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-ethip4-ip4base-eth-2memif-1dcr-mrr.64b-2t1c-ethip4-ip4base-eth-2memif-1dcr-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-dot1q-ip4base-mrr.64b-2t1c-avf-dot1q-ip4base-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-dot1q-ip4base-mrr.64b-8t4c-avf-dot1q-ip4base-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4base-mrr.64b-8t4c-avf-ethip4-ip4base-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale20k-mrr.64b-8t4c-avf-ethip4-ip4scale20k-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale200k-mrr.64b-8t4c-avf-ethip4-ip4scale200k-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale2m-mrr.64b-8t4c-avf-ethip4-ip4scale2m-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale20k-rnd-mrr.64b-8t4c-avf-ethip4-ip4scale20k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale200k-rnd-mrr.64b-8t4c-avf-ethip4-ip4scale200k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4-ip4scale2m-rnd-mrr.64b-8t4c-avf-ethip4-ip4scale2m-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-dot1q-ip4base-mrr.64b-2t1c-dot1q-ip4base-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4base-mrr.64b-8t4c-ethip4-ip4base-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale20k-mrr.64b-8t4c-ethip4-ip4scale20k-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale200k-mrr.64b-8t4c-ethip4-ip4scale200k-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale2m-mrr.64b-8t4c-ethip4-ip4scale2m-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale20k-rnd-mrr.64b-8t4c-ethip4-ip4scale20k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-ethip4-ip4scale200k-rnd-mrr.64b-8t4c-ethip4-ip4scale200k-rnd-mrr
tests.vpp.perf.ip4.2n1l-25ge2p1xxv710-avf-ethip4udp-ip4base-iacl50sf-10kflows-mrr.64b-8t4c-avf-ethip4udp-ip4base-iacl50sf-10kflows-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-eth-l2patch-mrr.64b-2t1c-avf-eth-l2patch-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-eth-l2patch-mrr.64b-4t2c-avf-eth-l2patch-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-mrr.64b-4t2c-avf-eth-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-mrr.64b-8t4c-avf-eth-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-eth-l2bdbasemaclrn-mrr.64b-8t4c-avf-eth-l2bdbasemaclrn-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-dot1q-l2bdbasemaclrn-mrr.64b-2t1c-dot1q-l2bdbasemaclrn-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2xcbase-mrr.64b-2t1c-eth-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-dot1q-l2bdbasemaclrn-mrr.64b-4t2c-dot1q-l2bdbasemaclrn-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2patch-mrr.64b-4t2c-eth-l2patch-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2xcbase-mrr.64b-4t2c-eth-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-dot1q-l2xcbase-mrr.64b-8t4c-dot1q-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2patch-mrr.64b-8t4c-eth-l2patch-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2xcbase-mrr.64b-8t4c-eth-l2xcbase-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2bdbasemaclrn-mrr.64b-8t4c-eth-l2bdbasemaclrn-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-avf-eth-l2bdscale10kmaclrn-mrr.64b-8t4c-avf-eth-l2bdscale10kmaclrn-mrr
tests.vpp.perf.l2.2n1l-25ge2p1xxv710-eth-l2bdscale10kmaclrn-mrr.64b-2t1c-eth-l2bdscale10kmaclrn-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-2t1c-avf-eth-l2xcbase-eth-2vhostvr1024-1vm-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-4t2c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr.64b-4t2c-avf-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-4t2c-avf-eth-l2xcbase-eth-2vhostvr1024-1vm-mrr
tests.vpp.perf.vm 
vhost.2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-eth-2vhostvr1024-1vm-vppl2xc-mrr.64b-2t1c-avf-d

Re: [vpp-dev] [csit-report] Regressions as of 2020-04-15 02:00:17 UTC #email

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> 3n-hsw

This turned out to be a bug on CSIT side,
breaking numa detection, affecting every NIC on numa 1.
The bug was introduced in [0] and fixed in [1] and [2].

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/25363
[1] https://gerrit.fd.io/r/c/csit/+/26569
[2] https://gerrit.fd.io/r/c/csit/+/26572

-Original Message-
From: csit-rep...@lists.fd.io  On Behalf Of 
nore...@jenkins.fd.io
Sent: Wednesday, 2020-April-15 06:15
To: Fdio+Csit-Report via Email Integration 
Subject: [csit-report] Regressions as of 2020-04-15 02:00:17 UTC #email

Following regressions occured in the last trending job runs, listed per testbed 
type.



2n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/885, 
VPP version: 20.05-rc0~528-g7357043d2~b1025

tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-avf-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.ip6.2n1l-10ge2p1x710-ethip6-ip6base-mrr.78b-2t1c-ethip6-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-avf-ethip6-ip6base-mrr.78b-4t2c-avf-ethip6-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-ethip6-ip6base-mrr.78b-2t1c-ethip6-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-dot1q-ip6base-mrr.78b-4t2c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.2n1l-25ge2p1xxv710-ethip6-ip6base-mrr.78b-4t2c-ethip6-ip6base-mrr


3n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/864, 
VPP version: 20.05-rc0~509-g9cbfb4c51~b1006

tests.vpp.perf.l2.25ge2p1xxv710-dot1q-l2xcbase-mrr.64b-4t2c-dot1q-l2xcbase-mrr


2n-clx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/293, 
VPP version: 20.05-rc0~532-g4fde4ae03~b1029

tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-avf-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr


3n-hsw, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master/1145, VPP 
version: 20.05-rc0~534-gd724e4f43~b1031

tests.vpp.perf.ip4.40ge2p1xl710-dot1q-ip4base-mrr.64b-1t1c-dot1q-ip4base-mrr
tests.vpp.perf.ip4.40ge2p1xl710-dot1q-ip4base-mrr.64b-2t2c-dot1q-ip4base-mrr
tests.vpp.perf.ip4 
tunnels.40ge2p1xl710-ethip4vxlan-l2xcbase-mrr.64b-1t1c-ethip4vxlan-l2xcbase-mrr
tests.vpp.perf.ip4 
tunnels.40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-mrr.64b-1t1c-ethip4vxlan-l2bdbasemaclrn-mrr
tests.vpp.perf.ip4 
tunnels.40ge2p1xl710-ethip4vxlan-l2xcbase-mrr.64b-2t2c-ethip4vxlan-l2xcbase-mrr
tests.vpp.perf.ip4 
tunnels.40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-mrr.64b-2t2c-ethip4vxlan-l2bdbasemaclrn-mrr
tests.vpp.perf.ip4 
tunnels.40ge2p1xl710-ethip4vxlan-l2xcbase-mrr.64b-4t4c-ethip4vxlan-l2xcbase-mrr
tests.vpp.perf.ip4 
tunnels.40ge2p1xl710-ethip4vxlan-l2bdbasemaclrn-mrr.64b-4t4c-ethip4vxlan-l2bdbasemaclrn-mrr
tests.vpp.perf.ip6.40ge2p1xl710-dot1q-ip6base-mrr.78b-1t1c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.40ge2p1xl710-ethip6-ip6base-mrr.78b-1t1c-ethip6-ip6base-mrr
tests.vpp.perf.ip6.40ge2p1xl710-dot1q-ip6base-mrr.78b-2t2c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.40ge2p1xl710-ethip6-ip6base-mrr.78b-2t2c-ethip6-ip6base-mrr
tests.vpp.perf.ip6.40ge2p1xl710-dot1q-ip6base-mrr.78b-4t4c-dot1q-ip6base-mrr
tests.vpp.perf.ip6.40ge2p1xl710-ethip6-ip6base-mrr.78b-4t4c-ethip6-ip6base-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr.imix-1t1c-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr.imix-1t1c-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1000tnlsw-ip4base-int-aes256gcm-mrr.imix-1t1c-ethip4ipsec1000tnlsw-ip4base-int-aes256gcm-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1000tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr.imix-1t1c-ethip4ipsec1000tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1tnlsw-ip4base-int-aes256gcm-mrr.imix-1t1c-ethip4ipsec1tnlsw-ip4base-int-aes256gcm-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr.imix-1t1c-ethip4ipsec1tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr.imix-2t2c-ethip4ipsec4tnlsw-ip4base-int-aes256gcm-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr.imix-2t2c-ethip4ipsec4tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1000tnlsw-ip4base-int-aes256gcm-mrr.imix-2t2c-ethip4ipsec1000tnlsw-ip4base-int-aes256gcm-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1000tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr.imix-2t2c-ethip4ipsec1000tnlsw-ip4base-int-aes128cbc-hmac512sha-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1tnlsw-ip4base-int-aes256gcm-mrr.imix-2t2c-ethip4ipsec1tnlsw-ip4base-int-aes256gcm-mrr
tests.vpp.perf.crypto.40ge2p1xl710-ethip4ipsec1tnlsw-ip4base-int-aes128cbc-hmac512sha

[vpp-dev] faster processing slower rate

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

Hello VPP experts.

I have encountered an unexpected pattern in performance results.
So I wonder, is there a bug somewhere (VPP or CSIT),
or is there a subtle reason why the performance pattern should be expected?

Usually, the more processing VPP in a particular test has to do,
the smaller forwarding rate it achieves.
That is why l2patch tests are usually the fastest.
(I am talking about MRR here; so maximal offered load,
and we measure the rate of packets making it through back to Traffic Generator).

But I noticed this stops being true in some cases.
Specifically on Cascadelake testbeds, when VPP uses 4 physical cores
(HT on, so 8 VPP workers), l2patch no longer has the best MRR.
(Also seen on Skylake.)

After some examination, I selected two tests to compare.
Bidirectional traffic, single Intel-xxv710 NIC, two ports (one per direction),
4 receive queues per port (9 transmit queues as automatically selected by VPP),
AVF driver, and both tests use l2 cross-connect.
The difference being one test handles dot1q on top of that.

Even though the dot1q test has larger [1] vectors/call than the other [2]
(expected, as the loop with dot1q has more work to do),
and dot1q shows small number of rx discards [3] (the other shows none [4]),
while neither showing anything bad in "show error",
still the dot1q forwards above 35 Mpps (see Message here [5] for the 10 trial 
results),
compared to the other test's under 31 Mpps [6].

The same pattern can be seen in other tests, although there usually are
details differing between the dot1q and plain tests.
For example with DPDK driver, there is not-that-small amount of "rx missed",
the dot1q already showing smaller number [7] than the other [8].
(I have even seen tx error in l2patch tests, but that could be a separate 
issue.)

So, have you seen this behavior? Can you explain it?
The only guess I have is that the faster test is polling rx queues more 
frequently,
and that somehow slows down the NICs ability to actually receive packets fast 
enough?
I give that less than 1% probability of explaining the difference.
The workers are loaded in a fairly uniform way, so that is not an issue.
Perhaps something in dot1q handling makes it inherently cheaper for NIC to 
process?
Even then, I do not think that explains why dot1q-l2xc becomes faster than 
l2patch.

Vratko.

[1] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k10-k1-k1-k4-k1
[2] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s2-t1-k2-k9-k1-k1-k4-k1
[3] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k10-k1-k8-k1
[4] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s2-t1-k2-k9-k1-k8-k1
[5] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s1-t1
[6] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s2-t1
[7] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s3-t1-k2-k10-k1-k8-k1
[8] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/153/archives/log.html.gz#s1-s1-s1-s1-s4-t1-k2-k9-k1-k8-k1
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#16041): https://lists.fd.io/g/vpp-dev/message/16041
Mute This Topic: https://lists.fd.io/mt/72901093/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via lists.fd.io]

> 100ge2p1cx556a-rdma

No VPP code change.
Up to now, RDMA tests were suffering from "duration stretching".
That is a phenomenon where TRex takes longer to send
the required number of packets, so device under tests
experiences smaller offered load and MRR value is inflated.
We have suppressed the phenomenon
by tweaking [0] core usage and max rate (pps) limit.
The new numbers should be reliable now.

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/26262

-Original Message-
From: csit-rep...@lists.fd.io  On Behalf Of 
nore...@jenkins.fd.io
Sent: Friday, 2020-April-03 18:07
To: Fdio+Csit-Report via Email Integration 
Subject: [csit-report] Regressions as of 2020-04-03 14:00:14 UTC #email

Following regressions occured in the last trending job runs, listed per testbed 
type.



2n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/881, 
VPP version: 20.05-rc0~433-g0c7aa7ab5~b930

No regressions

3n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/862, 
VPP version: 20.05-rc0~433-g0c7aa7ab5~b930

tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-2t1c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-2t1c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-4t2c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-4t2c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-8t4c-avf-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.25ge2p1xxv710-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-8t4c-avf-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.vm 
vhost.25ge2p1xxv710-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr.64b-8t4c-avf-dot1q-l2xcbase-eth-2vhostvr1024-1vm-mrr


2n-clx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/282, 
VPP version: 20.05-rc0~456-g57a5a2df5~b953

tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-2t1c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-4t2c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-avf-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-avf-ethip4-ip4base-eth-2memif-1dcr-mrr.64b-8t4c-avf-ethip4-ip4base-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-25ge2p1xxv710-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-8t4c-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-100ge2p1cx556a-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-100ge2p1cx556a-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-2t1c-rdma-eth-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-100ge2p1cx556a-rdma-eth-l2xcbase-eth-2memif-1dcr-mrr.64b-2t1c-rdma-eth-l2xcbase-eth-2memif-1dcr-mrr
tests.vpp.perf.container 
memif.2n1l-100ge2p1cx556a-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr.64b-4t2c-rdma-dot1q-l2bdbasemaclrn-eth-2memif-1dcr-mrr
tests.v

[vpp-dev] Regression in SRv6

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Trending has detected a performance regression in our SRv6 tests.
I summarized my current findings here [5].

Vratko

[5] https://gerrit.fd.io/r/c/vpp/+/25647#message-7d415999_6a0ba69f

-Original Message-
From: csit-rep...@lists.fd.io  On Behalf Of 
nore...@jenkins.fd.io
Sent: Wednesday, 2020-March-25 17:34
To: Fdio+Csit-Report via Email Integration 
Subject: [csit-report] Regressions as of 2020-03-25 14:00:16 UTC #email

Following regressions occured in the last trending job runs, listed per testbed 
type.



2n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-skx/879, 
VPP version: 20.05-rc0~389-g547a61654~b884

No regressions

3n-skx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-skx/860, 
VPP version: 20.05-rc0~389-g547a61654~b884

No regressions

2n-clx, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-clx/273, 
VPP version: 20.05-rc0~402-g2a6642e95~b899

No regressions

3n-hsw, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master/1124, VPP 
version: 20.05-rc0~398-gdb88ffba2~b895

tests.vpp.perf.srv6.40ge2p1xl710-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-1t1c-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-1t1c-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-1t1c-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-1t1c-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-1t1c-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-1t1c-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-2t2c-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-2t2c-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-2t2c-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-2t2c-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-2t2c-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-2t2c-ethip6srhip6-ip6base-srv6proxy-stat-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6ip6-ip6base-srv6enc1sid-mrr.78b-4t4c-ethip6ip6-ip6base-srv6enc1sid-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6enc2sids-mrr.78b-4t4c-ethip6srhip6-ip6base-srv6enc2sids-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr.78b-4t4c-ethip6srhip6-ip6base-srv6enc2sids-nodecaps-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-dyn-mrr.78b-4t4c-ethip6srhip6-ip6base-srv6proxy-dyn-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-masq-mrr.78b-4t4c-ethip6srhip6-ip6base-srv6proxy-masq-mrr
tests.vpp.perf.srv6.40ge2p1xl710-ethip6srhip6-ip6base-srv6proxy-stat-mrr.78b-4t4c-ethip6srhip6-ip6base-srv6proxy-stat-mrr


3n-tsh, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-tsh/158, 
VPP version: 20.05-rc0~398-gdb88ffba2~b3160

No regressions

2n-dnv, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-2n-dnv/371, 
VPP version: 20.05-rc0~398-gdb88ffba2~b895

No regressions

3n-dnv, CSIT build: 
https://jenkins.fd.io/view/csit/job/csit-vpp-perf-mrr-daily-master-3n-dnv/365, 
VPP version: 20.05-rc0~398-gdb88ffba2~b895

tests.vpp.perf.ip6.10ge2p1x553-ethip6-ip6scale2m-mrr.78b-4t4c-ethip6-ip6scale2m-mrr



-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15888): https://lists.fd.io/g/vpp-dev/message/15888
Mute This Topic: https://lists.fd.io/mt/72569683/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Increase MTU from ethernet packets

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> 65535 bytes with ETHERNET_MAX_PACKET_BYTES

I take it as a query on how easy it is for VPP
to support super jumbo frames [0].

> most networks use approximately 9000

In theory, VPP could allow higher values to be set on demand,
while making sure the default MTU values remain small enough.

Even if networking hardware lacks the support,
users can see benefits in memif and similar "virtual" devices.

The real question is whether VPP will work when recompiled
with the higher limit, or whether some pieces of VPP code will blow up
(because they assume frame size is never much larger than ~9000 bytes).

Vratko.

[0] https://en.wikipedia.org/wiki/Jumbo_frame#Super_jumbo_frames

From: vpp-dev@lists.fd.io  On Behalf Of  ??
Sent: Saturday, 2020-March-21 16:19
To: Алексей ; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Increase MTU from ethernet packets

Hello,
I think you got very strange requirement, jumbo frame can be up to 16000 but 
most networks use approximatly 9000.
VPP has standart burned in value.

От: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> от имени Алексей 
mailto:ne-vleza...@yandex.ru>>
Отправлено: 21 марта 2020 г. 1:44:37
Кому: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>>
Тема: [vpp-dev] Increase MTU from ethernet packets

Hello, VPP support maximum mtu 9216 bytes. Else increase mtu from 65535
bytes with ETHERNET_MAX_PACKET_BYTES environment and recompile VPP?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15855): https://lists.fd.io/g/vpp-dev/message/15855
Mute This Topic: https://lists.fd.io/mt/72446469/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release branches?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

>> Does that mean we want an api-crc job also for 1908 stream?

After some backporting, CSIT is ready for that job now.
All it takes is a VPP developer to vote +1 (or -1)
this [1] ci-management change.

Vratko.

[1] https://gerrit.fd.io/r/c/ci-management/+/25457

From: vpp-dev@lists.fd.io  On Behalf Of Vratko Polak -X 
(vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io
Sent: Thursday, 2020-February-27 14:56
To: Andrew Yourtchenko 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

> Adding a new API into an existing file will still change the CRC on the 
> plugin/module

Looking into a .api.json file, I see "crc" only for messages,
but the whole file also ends with a "vl_api_version" value.

> if we are aiming for binary compatibility

Crc values guard against backward-incompatible edits.
Vl_api_version changes even for backward compatible edits
(within the same .api file), so perhaps we can tolerate such change.

Vratko.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Andrew 
Yourtchenko
Sent: Wednesday, February 26, 2020 6:19 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
mailto:vrpo...@cisco.com>>
Cc: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

Hmm so that’s an interesting question. Adding a new API into an existing file 
will still change the CRC on the plugin/module - which means that if we are 
aiming for binary compatibility (which is I think what we are doing) - it means 
we can only allow the one-shot addition of new .api files, right ?

--a

On 26 Feb 2020, at 17:26, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at 
Cisco) mailto:vrpo...@cisco.com>> wrote:

> as soon as the CRC of any existing messages does not change, a patch is okay 
> to include into 19.08

Does that mean we want an api-crc job also for 1908 stream?
We currently have api-crc job only for master,
because other branches were not expected to edit .api files.

Vratko.

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Andrew 
Yourtchenko
Sent: Tuesday, February 25, 2020 6:49 PM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: Christian Hopps mailto:cho...@chopps.org>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>; 
vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io>
Subject: Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

With my 19.08 release manager hat on,
For the current state of the matters my view is “as soon as the CRC of any 
existing messages does not change, a patch is okay to include into 19.08”.

This applies recursively, meaning that if one add a new message to 1908, it 
comes in as “frozen” with no further changes. Why ? Because if it is not 
perfect API wise it’s not stable, it should first settle down on master.

That’s the logic I had been using so far.

I am currently working on an 19.08 api translation layer  experiment that in 
principle should allow more freedom... but till that is successful that is my 
point of view. Unless the community decides otherwise, of course.

--a


On 25 Feb 2020, at 16:56, Dave Barach via Lists.Fd.Io 
mailto:dbarach=cisco@lists.fd.io>> wrote:

Dear Chris,

Adding missing flags to ipsec_sad_flags shouldn’t break much of anything. Never 
say never, but for me, I wouldn’t hesitate to merge such a patch.

Adding an entirely new message will renumber subsequent core engine messages, 
and implies client recompilation. My $0.02: again, not such a big deal.

What do other people think?

Dave

P.S. in master/latest, enum ipsec_sad_flags has moved to 
.../src/vnet/ipsec/ipsec_types.api...


From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Christian Hopps
Sent: Tuesday, February 25, 2020 10:25 AM
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Christian Hopps mailto:cho...@chopps.org>>
Subject: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

I've got a couple of changes to the ipsec API that I'd like to upstream to 
match the vpp "kernel" code I'm going try and upstream to strongswan.

1) Add: Add an ip_route_lookup/reply pair (semver minor++)
2) Fix: Add IS_INBOUND flag (value 0x40) to ipsec_sad_flags (semver patch++)
optional) Fix: possibly add the other missing flags to ipsec_sad_flags so they 
can be properly returned on queries.

I think submitting these for release branches is OK after reading 
https://wiki.fd.io/view/VPP/API_Versioning

I'm coding to 19.08 right now, if I'd like to have those changes in that branch 
I would imagine I'd need to also submit changes for 20.01 and master?

I admit to being confused about the CRC st

Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release branches?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> Adding a new API into an existing file will still change the CRC on the 
> plugin/module

Looking into a .api.json file, I see "crc" only for messages,
but the whole file also ends with a "vl_api_version" value.

> if we are aiming for binary compatibility

Crc values guard against backward-incompatible edits.
Vl_api_version changes even for backward compatible edits
(within the same .api file), so perhaps we can tolerate such change.

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Andrew Yourtchenko
Sent: Wednesday, February 26, 2020 6:19 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

Hmm so that’s an interesting question. Adding a new API into an existing file 
will still change the CRC on the plugin/module - which means that if we are 
aiming for binary compatibility (which is I think what we are doing) - it means 
we can only allow the one-shot addition of new .api files, right ?

--a


On 26 Feb 2020, at 17:26, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at 
Cisco) mailto:vrpo...@cisco.com>> wrote:

> as soon as the CRC of any existing messages does not change, a patch is okay 
> to include into 19.08

Does that mean we want an api-crc job also for 1908 stream?
We currently have api-crc job only for master,
because other branches were not expected to edit .api files.

Vratko.

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Andrew 
Yourtchenko
Sent: Tuesday, February 25, 2020 6:49 PM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: Christian Hopps mailto:cho...@chopps.org>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

With my 19.08 release manager hat on,
For the current state of the matters my view is “as soon as the CRC of any 
existing messages does not change, a patch is okay to include into 19.08”.

This applies recursively, meaning that if one add a new message to 1908, it 
comes in as “frozen” with no further changes. Why ? Because if it is not 
perfect API wise it’s not stable, it should first settle down on master.

That’s the logic I had been using so far.

I am currently working on an 19.08 api translation layer  experiment that in 
principle should allow more freedom... but till that is successful that is my 
point of view. Unless the community decides otherwise, of course.

--a



On 25 Feb 2020, at 16:56, Dave Barach via Lists.Fd.Io 
mailto:dbarach=cisco@lists.fd.io>> wrote:

Dear Chris,

Adding missing flags to ipsec_sad_flags shouldn’t break much of anything. Never 
say never, but for me, I wouldn’t hesitate to merge such a patch.

Adding an entirely new message will renumber subsequent core engine messages, 
and implies client recompilation. My $0.02: again, not such a big deal.

What do other people think?

Dave

P.S. in master/latest, enum ipsec_sad_flags has moved to 
.../src/vnet/ipsec/ipsec_types.api...


From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Christian Hopps
Sent: Tuesday, February 25, 2020 10:25 AM
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Christian Hopps mailto:cho...@chopps.org>>
Subject: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

I've got a couple of changes to the ipsec API that I'd like to upstream to 
match the vpp "kernel" code I'm going try and upstream to strongswan.

1) Add: Add an ip_route_lookup/reply pair (semver minor++)
2) Fix: Add IS_INBOUND flag (value 0x40) to ipsec_sad_flags (semver patch++)
optional) Fix: possibly add the other missing flags to ipsec_sad_flags so they 
can be properly returned on queries.

I think submitting these for release branches is OK after reading 
https://wiki.fd.io/view/VPP/API_Versioning

I'm coding to 19.08 right now, if I'd like to have those changes in that branch 
I would imagine I'd need to also submit changes for 20.01 and master?

I admit to being confused about the CRC stuff, and the warnings in the 19.08.1 
release notes and what those warnings imply. Is it safe to assume the CRC stuff 
can be ignored and external clients will still work (given no semver major 
change) even if a CRC chagnes?

Side Note: from the API link: "If a new message type is added, the old message 
type must be maintained for at least one release. The change must be included 
in the release notes, and it would be helpful if the message could be tagged as 
deprecated in the API language and a logging message given to the user."

Given there are 3 releases per year, only maintaining an old compatible 
function for 1 release seems rather aggressive. It does say "at least" though. 
:)

Thanks,
Chris.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15586): https://lists.fd.io/g/vpp-de

Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> How long do we have to wait?
> I'd much rather revert 22698.

Go ahead, it looks it will take a day or two on our end.

Vratko.

-Original Message-
From: otr...@employees.org  
Sent: Wednesday, February 26, 2020 5:19 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: Dave Barach (dbarach) ; Maciek Konstantynowicz 
(mkonstan) ; vpp-dev ; Jakub Grajciar 
-X (jgrajcia - PANTHEON TECH SRO at Cisco) 
Subject: Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...



> On 26 Feb 2020, at 16:36, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
> Cisco) via Lists.Fd.Io  wrote:
> 
> This [0] change got merged. It got wrong Verified+1
> due to naginator retrying one job
> (and Gerrit forgetting about other ones).
> Basically the same as these [1] two [2] older failures.
>  
> Not sure whether is better to revert the change
> or wait for CSIT to catch up.

How long do we have to wait?
If very long at all, I'd much rather revert 22698.

Cheers,
Ole


>  
> Vratko.
>  
> [0] https://gerrit.fd.io/r/c/vpp/+/22698
> [1] https://lists.fd.io/g/vpp-api-dev/message/32
> [2] https://lists.fd.io/g/vpp-dev/message/11754
>  
> From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach via 
> Lists.Fd.Io
> Sent: Wednesday, February 26, 2020 4:10 PM
> To: Maciek Konstantynowicz (mkonstan) 
> Cc: vpp-dev@lists.fd.io
> Subject: Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious 
> reason...
>  
> Thanks... Dave
>  
> From: Maciek Konstantynowicz (mkonstan)  
> Sent: Wednesday, February 26, 2020 10:09 AM
> To: Dave Barach (dbarach) 
> Cc: csit-...@lists.fd.io; Ed Kern (ejk) ; Dave Wallace 
> (dwallace) ; vpp-dev 
> Subject: Re: Gerrit 25463 fails API CRC check for no obvious reason...
>  
> Hi Dave, thanks for notifying, we have just discussed this on the CSIT 
> project call, Vratko is looking into this.
> Here are the incompatible API CRCs that are tripping the job. Will notify on 
> this thread once issue is nailed down.
>  
> Cheers,
> -Maciek
>  
> 13:24:04 Incompatible API CRCs found in .api.json files:
> 13:24:04 {
> 13:24:04  "ipsec_interface_add_del_spd":"0x80f80cbb",
> 13:24:04  "ipsec_sad_entry_add_del":"0x57cc13fa",
> 13:24:04  "ipsec_select_backend":"0x5bcfd3b7",
> 13:24:04  "ipsec_spd_add_del":"0x20e89a95",
> 13:24:04  "ipsec_spd_entry_add_del":"0x9f384b8d",
> 13:24:04  "ipsec_tunnel_if_add_del":"0x2b135e68",
> 13:24:04  "ipsec_tunnel_if_add_del_reply":"0x5383d31f"
> 13:24:04 }
> 13:24:04 RuntimeError('Incompatible API CRCs found in .api.json files:\n{\n 
> "ipsec_interface_add_del_spd":"0x80f80cbb",\n 
> "ipsec_sad_entry_add_del":"0x57cc13fa",\n 
> "ipsec_select_backend":"0x5bcfd3b7",\n "ipsec_spd_add_del":"0x20e89a95",\n 
> "ipsec_spd_entry_add_del":"0x9f384b8d",\n 
> "ipsec_tunnel_if_add_del":"0x2b135e68",\n 
> "ipsec_tunnel_if_add_del_reply":"0x5383d31f"\n}',)
>  
>  
> 
> On 26 Feb 2020, at 14:44, Dave Barach (dbarach)  wrote:
>  
> Other patches failing as well. Please investigate AYEC.
>  
> Thanks... Dave
>  
> P.S. See https://gerrit.fd.io/r/c/vpp/+/25463. No .api files involved.
>  
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15556): https://lists.fd.io/g/vpp-dev/message/15556
Mute This Topic: https://lists.fd.io/mt/71566402/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release branches?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> as soon as the CRC of any existing messages does not change, a patch is okay 
> to include into 19.08

Does that mean we want an api-crc job also for 1908 stream?
We currently have api-crc job only for master,
because other branches were not expected to edit .api files.

Vratko.

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Andrew 
Yourtchenko
Sent: Tuesday, February 25, 2020 6:49 PM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: Christian Hopps mailto:cho...@chopps.org>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>; 
vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

With my 19.08 release manager hat on,
For the current state of the matters my view is “as soon as the CRC of any 
existing messages does not change, a patch is okay to include into 19.08”.

This applies recursively, meaning that if one add a new message to 1908, it 
comes in as “frozen” with no further changes. Why ? Because if it is not 
perfect API wise it’s not stable, it should first settle down on master.

That’s the logic I had been using so far.

I am currently working on an 19.08 api translation layer  experiment that in 
principle should allow more freedom... but till that is successful that is my 
point of view. Unless the community decides otherwise, of course.

--a


On 25 Feb 2020, at 16:56, Dave Barach via Lists.Fd.Io 
mailto:dbarach=cisco@lists.fd.io>> wrote:

Dear Chris,

Adding missing flags to ipsec_sad_flags shouldn’t break much of anything. Never 
say never, but for me, I wouldn’t hesitate to merge such a patch.

Adding an entirely new message will renumber subsequent core engine messages, 
and implies client recompilation. My $0.02: again, not such a big deal.

What do other people think?

Dave

P.S. in master/latest, enum ipsec_sad_flags has moved to 
.../src/vnet/ipsec/ipsec_types.api...


From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Christian Hopps
Sent: Tuesday, February 25, 2020 10:25 AM
To: vpp-dev mailto:vpp-dev@lists.fd.io>>
Cc: Christian Hopps mailto:cho...@chopps.org>>
Subject: [vpp-dev] Can I submit some API changes for 19.08, et al. release 
branches?

I've got a couple of changes to the ipsec API that I'd like to upstream to 
match the vpp "kernel" code I'm going try and upstream to strongswan.

1) Add: Add an ip_route_lookup/reply pair (semver minor++)
2) Fix: Add IS_INBOUND flag (value 0x40) to ipsec_sad_flags (semver patch++)
optional) Fix: possibly add the other missing flags to ipsec_sad_flags so they 
can be properly returned on queries.

I think submitting these for release branches is OK after reading 
https://wiki.fd.io/view/VPP/API_Versioning

I'm coding to 19.08 right now, if I'd like to have those changes in that branch 
I would imagine I'd need to also submit changes for 20.01 and master?

I admit to being confused about the CRC stuff, and the warnings in the 19.08.1 
release notes and what those warnings imply. Is it safe to assume the CRC stuff 
can be ignored and external clients will still work (given no semver major 
change) even if a CRC chagnes?

Side Note: from the API link: "If a new message type is added, the old message 
type must be maintained for at least one release. The change must be included 
in the release notes, and it would be helpful if the message could be tagged as 
deprecated in the API language and a logging message given to the user."

Given there are 3 releases per year, only maintaining an old compatible 
function for 1 release seems rather aggressive. It does say "at least" though. 
:)

Thanks,
Chris.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#1): https://lists.fd.io/g/vpp-dev/message/1
Mute This Topic: https://lists.fd.io/mt/71535081/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

This [0] change got merged. It got wrong Verified+1
due to naginator retrying one job
(and Gerrit forgetting about other ones).
Basically the same as these [1] two [2] older failures.

Not sure whether is better to revert the change
or wait for CSIT to catch up.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/22698
[1] https://lists.fd.io/g/vpp-api-dev/message/32
[2] https://lists.fd.io/g/vpp-dev/message/11754

From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach via 
Lists.Fd.Io
Sent: Wednesday, February 26, 2020 4:10 PM
To: Maciek Konstantynowicz (mkonstan) 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...

Thanks... Dave

From: Maciek Konstantynowicz (mkonstan) 
mailto:mkons...@cisco.com>>
Sent: Wednesday, February 26, 2020 10:09 AM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: csit-...@lists.fd.io; Ed Kern (ejk) 
mailto:e...@cisco.com>>; Dave Wallace (dwallace) 
mailto:dwall...@cisco.com>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Subject: Re: Gerrit 25463 fails API CRC check for no obvious reason...

Hi Dave, thanks for notifying, we have just discussed this on the CSIT project 
call, Vratko is looking into this.
Here are the incompatible API CRCs that are tripping the job. Will notify on 
this thread once issue is nailed down.

Cheers,
-Maciek


13:24:04 Incompatible API CRCs found in .api.json files:

13:24:04 {

13:24:04  "ipsec_interface_add_del_spd":"0x80f80cbb",

13:24:04  "ipsec_sad_entry_add_del":"0x57cc13fa",

13:24:04  "ipsec_select_backend":"0x5bcfd3b7",

13:24:04  "ipsec_spd_add_del":"0x20e89a95",

13:24:04  "ipsec_spd_entry_add_del":"0x9f384b8d",

13:24:04  "ipsec_tunnel_if_add_del":"0x2b135e68",

13:24:04  "ipsec_tunnel_if_add_del_reply":"0x5383d31f"

13:24:04 }

13:24:04 RuntimeError('Incompatible API CRCs found in .api.json files:\n{\n 
"ipsec_interface_add_del_spd":"0x80f80cbb",\n 
"ipsec_sad_entry_add_del":"0x57cc13fa",\n 
"ipsec_select_backend":"0x5bcfd3b7",\n "ipsec_spd_add_del":"0x20e89a95",\n 
"ipsec_spd_entry_add_del":"0x9f384b8d",\n 
"ipsec_tunnel_if_add_del":"0x2b135e68",\n 
"ipsec_tunnel_if_add_del_reply":"0x5383d31f"\n}',)


On 26 Feb 2020, at 14:44, Dave Barach (dbarach) 
mailto:dbar...@cisco.com>> wrote:

Other patches failing as well. Please investigate AYEC.

Thanks... Dave

P.S. See https://gerrit.fd.io/r/c/vpp/+/25463. No .api files involved.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15550): https://lists.fd.io/g/vpp-dev/message/15550
Mute This Topic: https://lists.fd.io/mt/71566402/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

From: vpp-dev@lists.fd.io  On Behalf Of Dave Barach via 
Lists.Fd.Io
Sent: Wednesday, February 26, 2020 4:10 PM
To: Maciek Konstantynowicz (mkonstan) 
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Gerrit 25463 fails API CRC check for no obvious reason...

Thanks... Dave

From: Maciek Konstantynowicz (mkonstan) 
mailto:mkons...@cisco.com>>
Sent: Wednesday, February 26, 2020 10:09 AM
To: Dave Barach (dbarach) mailto:dbar...@cisco.com>>
Cc: csit-...@lists.fd.io; Ed Kern (ejk) 
mailto:e...@cisco.com>>; Dave Wallace (dwallace) 
mailto:dwall...@cisco.com>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Subject: Re: Gerrit 25463 fails API CRC check for no obvious reason...

Hi Dave, thanks for notifying, we have just discussed this on the CSIT project 
call, Vratko is looking into this.
Here are the incompatible API CRCs that are tripping the job. Will notify on 
this thread once issue is nailed down.

Cheers,
-Maciek


13:24:04 Incompatible API CRCs found in .api.json files:

13:24:04 {

13:24:04  "ipsec_interface_add_del_spd":"0x80f80cbb",

13:24:04  "ipsec_sad_entry_add_del":"0x57cc13fa",

13:24:04  "ipsec_select_backend":"0x5bcfd3b7",

13:24:04  "ipsec_spd_add_del":"0x20e89a95",

13:24:04  "ipsec_spd_entry_add_del":"0x9f384b8d",

13:24:04  "ipsec_tunnel_if_add_del":"0x2b135e68",

13:24:04  "ipsec_tunnel_if_add_del_reply":"0x5383d31f"

13:24:04 }

13:24:04 RuntimeError('Incompatible API CRCs found in .api.json files:\n{\n 
"ipsec_interface_add_del_spd":"0x80f80cbb",\n 
"ipsec_sad_entry_add_del":"0x57cc13fa",\n 
"ipsec_select_backend":"0x5bcfd3b7",\n "ipsec_spd_add_del":"0x20e89a95",\n 
"ipsec_spd_entry_add_del":"0x9f384b8d",\n 
"ipsec_tunnel_if_add_del":"0x2b135e68",\n 
"ipsec_tunnel_if_add_del_reply":"0x5383d31f"\n}',)


On 26 Feb 2020, at 14:44, Dave Barach (dbarach) 
mailto:dbar...@cisco.com>> wrote:

Other patches failing as well. Please investigate AYEC.

Thanks... Dave

P.S. See https://gerrit.fd.io/r/c/vpp/+/25463. No .api files involved.

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15548): https://lists.fd.io/g/vpp-dev/message/15548
Mute This Topic: https://lists.fd.io/mt/71566402/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Arm verify job broken

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> accidentally separated two scripts that had to stay together.

My fault, I had no idea this kind of coupling existed.
Adding comments [1] to prevent similar errors in future.

Vratko.

[1] https://gerrit.fd.io/r/c/ci-management/+/24348

From: vpp-dev@lists.fd.io  On Behalf Of Christian Hopps
Sent: Wednesday, January 15, 2020 1:10 AM
To: Ed Kern (ejk) 
Cc: Christian Hopps ; Andrew 👽 Yourtchenko 
; Florin Coras ; vpp-dev 

Subject: Re: [vpp-dev] Arm verify job broken



> On Jan 14, 2020, at 6:27 PM, Ed Kern via Lists.Fd.Io 
> mailto:ejk=cisco@lists.fd.io>> wrote:
>
> short answer: fixed..
>
>
> longer answer:  a ci-management patch which did a whole bunch of cleanup and 
> readability improvements accidentally
> separated two scripts that had to stay together.  Having the script apart 
> meant arm was using only 1 core instead
> of the correct number of 16 cores.
>
> This lead to all arm jobs answering the question ‘what does a yellow light 
> mean’  (apologies to those that miss the taxi reference)

Now that's a blast from the past... :)

Thanks,
Chris.


>
> ci-man patch being merged now…all subsequent jobs should be back to normal 
> timing.
>
> Ed
>
>
>
>> On Jan 14, 2020, at 1:54 PM, Andrew 👽 Yourtchenko 
>> mailto:ayour...@gmail.com>> wrote:
>>
>> better to debug some before changing anything.
>>
>> With my RM hat on I would rather understand what is going on, if it is 24 
>> hours before the RC1 milestone ;-)
>>
>> --a
>>
>>> On 14 Jan 2020, at 21:28, Ed Kern via Lists.Fd.Io 
>>> mailto:ejk=cisco@lists.fd.io>> wrote:
>>>
>>>  ok FOR THE MOST PART…  build times haven’t changed  (@20 min)
>>>
>>> make test portion has gone off the chain and is often hitting the 120 min 
>>> build timeout threshold.
>>>
>>> Ive currently got several jobs running on sandbox testing
>>>
>>> a. without any timeouts (just to see if we are still passing on master)
>>> b. three different memory and disk profiles (just in case someone/thing got 
>>> messy and we have had another memory spike)
>>>
>>> the bad news is that it will be at least a couple of hours more than likely 
>>> before I get any results that I feel are actionable.
>>>
>>>
>>> Having said that im happy to shoot the build timeout for the time being if 
>>> committers feel that is warranted to ‘keep the train rolling’
>>>
>>> Ed
>>>
>>>
>>>
>>>
 On Jan 14, 2020, at 1:07 PM, Florin Coras 
 mailto:fcoras.li...@gmail.com>> wrote:

 Thanks, Ed!

 Florin

> On Jan 14, 2020, at 11:53 AM, Ed Kern (ejk) 
> mailto:e...@cisco.com>> wrote:
>
> looking into it now…..
>
> its a strange one ill tell you that up front…failures are all over the 
> place inside the build and even some hitting the 120 min timeout….
>
> more as i dig hopefully
>
> thanks for the ping
>
> Ed
>
>
>
>> On Jan 14, 2020, at 11:40 AM, Florin Coras 
>> mailto:fcoras.li...@gmail.com>> wrote:
>>
>> Hi,
>>
>> Jobs have been failing since yesterday. Did anybody try to look into it?
>>
>> Regards,
>> Florin
>>
>

>>>
>>>
>
>

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15181): https://lists.fd.io/g/vpp-dev/message/15181
Mute This Topic: https://lists.fd.io/mt/69698855/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] python api over tcp?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> This should be done as an intermediate agent.

I have a hypothetical question.
(I am not trying to be cheeky.
I feel there may be a difference,
I just do not see it.)

Imagine a world where VPP supports
(CLI via vppctl and) binary API only over shared memory.
No support for binary API over Unix Domain Socket.
(Sockets are only used for getting file descriptors,
so interaction via shared memory can start.)

And imagine there is a discussion on how to make
binary API available for remote users.
Somebody suggests adding UDS as the new way
to exchange binary API messages (in network byte order).

Would that suggestion also result in
"This should be done as an intermediate agent." and
"I don't think this should go into the main VPP executable."?

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Ole Troan
Sent: Tuesday, January 14, 2020 12:29 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: Paul Vinciguerra ; Christian Hopps 
; vpp-dev 
Subject: Re: [vpp-dev] python api over tcp?

Hi Vratko,


> In my eyes, this e-mail thread contains two largely independent questions.
>
> The first question is how we can make stats available over a "wire".
> Remote user asks over the wire, the request is seen on VPP side of the wire
> as a byte sequence.
> Some (currently missing) VPP code parses that request,
> looks into the shared memory, and serializes the requested stats
> into a response (sequence of bytes), put to the wire for user to read from.
>
> Ssh + vpp_get_stats acts as a wire, but the format is human (not machine) 
> friendly.
> Similarly to how CLI is less machine friendly than binary API.
>
> I expect the first implementation of machine friendly "stats over wire"
> to use Unix Domain Socket as the wire, just because VPP already has
> a dispatcher for handling binary messages of prescribed type there.
> Also, UDS has smaller overhead than other wires,
> especially if VPP is in a container.
>
> A sub-question is whether to support explicit polling,
> or subscribing for notifications (or both).
> In CSIT we want explicit polling.
>
> The second question is what wire is the best for stats transport.
> (And whether we should add support for more wire types.)
> > There's the (naive) prometheus example in the repo, vpp_get_stats,
> > there is a Telegraf plugin, a simple gNMI/gRPC plugin.
>
> I suspect the Prometheus example is for publish/subscribe,
> and vpp_get_stats is for explicit polling.
> Not sure about the others.

This should be done as an intermediate agent.

E.g. like:
https://github.com/vpp-telemetry-pfe/gnmi-grpc

Or the example Prometheus server.

I'd imagine both push and pull would have to be supported. But I don't think 
this should go into the main VPP executable.


Best regards,
Ole

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15159): https://lists.fd.io/g/vpp-dev/message/15159
Mute This Topic: https://lists.fd.io/mt/69538850/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] python api over tcp?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

In my eyes, this e-mail thread contains two largely independent questions.

The first question is how we can make stats available over a "wire".
Remote user asks over the wire, the request is seen on VPP side of the wire
as a byte sequence.
Some (currently missing) VPP code parses that request,
looks into the shared memory, and serializes the requested stats
into a response (sequence of bytes), put to the wire for user to read from.

Ssh + vpp_get_stats acts as a wire, but the format is human (not machine) 
friendly.
Similarly to how CLI is less machine friendly than binary API.

I expect the first implementation of machine friendly "stats over wire"
to use Unix Domain Socket as the wire, just because VPP already has
a dispatcher for handling binary messages of prescribed type there.
Also, UDS has smaller overhead than other wires,
especially if VPP is in a container.

A sub-question is whether to support explicit polling,
or subscribing for notifications (or both).
In CSIT we want explicit polling.

The second question is what wire is the best for stats transport.
(And whether we should add support for more wire types.)
> There's the (naive) prometheus example in the repo, vpp_get_stats,
> there is a Telegraf plugin, a simple gNMI/gRPC plugin.

I suspect the Prometheus example is for publish/subscribe,
and vpp_get_stats is for explicit polling.
Not sure about the others.

Vratko.

From: Paul Vinciguerra 
Sent: Thursday, January 9, 2020 11:39 PM
To: Ole Troan 
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Christian Hopps ; vpp-dev 
Subject: Re: [vpp-dev] python api over tcp?

My thought was to add a daemon that listened to tls and wrapped the shm 
transport/libapiclient, that would stay coresident with vpp then connect with 
the papi client from a remote system via tls.


On Jan 9, 2020, at 1:38 PM, Ole Troan 
mailto:otr...@employees.org>> wrote:

Hi Paul,


On 9 Jan 2020, at 19:10, Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>> wrote:

Sounds like a little scope creep going on there.
If you provide protobuf3 encoded api messages to/from vpp, I'll add a grpc 
listener option to vpp_papi, in the interim, I'd be glad to add a tls wrapped 
listener if there is interest.

Were you thinking of adding a simple tls/tcp to uds proxy? Must be something 
off the shelf that can be used. Netcat? Then you want to amend vpp_papi to be 
able to sit at the other end of that? What did you plan to do with security?

The grpc idea isn’t too dissimilar. Just pass vpp api messages encapsulated in 
grpc. One request, one reply and an event message.

Cheers
Ole




On Thu, Jan 9, 2020 at 12:00 PM Ole Troan 
mailto:otr...@employees.org>> wrote:



On 9 Jan 2020, at 16:50, Paul Vinciguerra 
mailto:pvi...@vinciconsulting.com>> wrote:

Is there any objection to adding a tls listener and an instance to the stats 
client to vpp_papi?

Use grpc as transport?

Cheers,
Ole




On Jan 9, 2020, at 6:45 AM, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
Cisco) via Lists.Fd.Io<http://Lists.Fd.Io> 
mailto:cisco@lists.fd.io>> wrote:

CSIT uses VPP API via socket (tunneled over SSH) for most interactions.

We also read stats for just one (I think) purpose,
reading runtime stats (/sys/node).
The way we do that is historical and convoluted,
for the result see INFO line at [1].

Looking at the result, the appropriate API way
would be to use send some _dump message
and process the _details responses, one per node name.

Vratko.

[1] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/58/archives/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k9-k1-k1-k4-k1

From: vpp-dev@lists.fd.io<mailto:vpp-dev@lists.fd.io> 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Christian Hopps
Sent: Thursday, January 9, 2020 12:05 PM
To: Ole Troan mailto:otr...@employees.org>>
Cc: Christian Hopps mailto:cho...@chopps.org>>; vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] python api over tcp?



> On Jan 9, 2020, at 5:44 AM, Ole Troan 
> mailto:otr...@employees.org>> wrote:
>
> Christian,
>
>>> For exporting data out of the stats segment, I believe there is already 
>>> quite a few solutions.
>>> There's the (naive) prometheus example in the repo, vpp_get_stats, there is 
>>> a Telegraf plugin, a simple gNMI/gRPC plugin.
>>
>> Right I've used vpp_get_stats and may run that with ssh and awk. I guess it 
>> just seems odd on first encountering this that the CLI provided the data, 
>> but the binary API didn't. I suppose the view is that exposing the stats 
>> segment in shared memory *is* the binary API. :)
>
> What certainly would make sense to do, is to put a wrapper on top of 
> vpp_stats.py that gives you a higher level API of accessing the stats.
> E.g. a get_interface_counters(

Re: [vpp-dev] python api over tcp?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

CSIT uses VPP API via socket (tunneled over SSH) for most interactions.

We also read stats for just one (I think) purpose,
reading runtime stats (/sys/node).
The way we do that is historical and convoluted,
for the result see INFO line at [1].

Looking at the result, the appropriate API way
would be to use send some _dump message
and process the _details responses, one per node name.

Vratko.

[1] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-perf-verify-master-2n-clx/58/archives/log.html.gz#s1-s1-s1-s1-s1-t1-k2-k9-k1-k1-k4-k1

From: vpp-dev@lists.fd.io  On Behalf Of Christian Hopps
Sent: Thursday, January 9, 2020 12:05 PM
To: Ole Troan 
Cc: Christian Hopps ; vpp-dev 
Subject: Re: [vpp-dev] python api over tcp?



> On Jan 9, 2020, at 5:44 AM, Ole Troan 
> mailto:otr...@employees.org>> wrote:
>
> Christian,
>
>>> For exporting data out of the stats segment, I believe there is already 
>>> quite a few solutions.
>>> There's the (naive) prometheus example in the repo, vpp_get_stats, there is 
>>> a Telegraf plugin, a simple gNMI/gRPC plugin.
>>
>> Right I've used vpp_get_stats and may run that with ssh and awk. I guess it 
>> just seems odd on first encountering this that the CLI provided the data, 
>> but the binary API didn't. I suppose the view is that exposing the stats 
>> segment in shared memory *is* the binary API. :)
>
> What certainly would make sense to do, is to put a wrapper on top of 
> vpp_stats.py that gives you a higher level API of accessing the stats.
> E.g. a get_interface_counters(). The stat segment also contains the name to 
> interface index mapping (/if/names).
> Want to have a go?

I'm actually going to use vpp_get_stats (run remotely using ssh) for now. I'm 
using vpp_papi on a single testing server (so it connects to each of the VPP 
/run/vpp/api.sock over ssh forwarded sockets), so it doesn't have access to 
their shared memory segments.

>
> I am also exploring putting much more information into the stat segment, 
> essentially making it into an operational data store (RFC8342). Don't hold 
> your breath. But any help appreciated.

I will be looking at doing some YANG models later this year, so if the timing 
aligns.. :)

Thanks,
Chris.


> Cheers,
> Ole

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#15109): https://lists.fd.io/g/vpp-dev/message/15109
Mute This Topic: https://lists.fd.io/mt/69538850/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] merge jobs failing..

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> No its not clear to me why it verified…

Because the verification run happened before [2] was merged
(in other words, 23913 should have been rebased after that).

I think this kind of merge error still happens infrequently enough,
and any "remove +1 from affected open changes" ideas
would be voted down as too much hassle.

Vratko.

[2] https://gerrit.fd.io/r/c/vpp/+/23999

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Wednesday, December 18, 2019 1:03 AM
To: Ed Kern (ejk) 
Cc: Benoit Ganne (bganne) ; vpp-dev 
Subject: Re: [vpp-dev] merge jobs failing..

Fixed: https://gerrit.fd.io/r/c/vpp/+/24041

On Tue, Dec 17, 2019 at 7:00 PM Ed Kern via Lists.Fd.Io 
mailto:cisco@lists.fd.io>> wrote:

Hey benoit,

Happened to notice (because I was watching the queue anyway since santa damjan 
did a bunch of merges) that ubuntu test/merges started failing
after
https://gerrit.fd.io/r/c/vpp/+/23913

No its not clear to me why it verified…
it looks like a simple typo where you meant to do
from scapy.contrib.geneve import GENEVE
but did
from scapy.layers.geneve import GENEVE

in test/test_trace_filter.py

anyway..all ubuntu merge jobs are failing after that point so if this ISNT the 
error feel
free to flame me and/or the real root cause.

Ed
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14915): https://lists.fd.io/g/vpp-dev/message/14915
Mute This Topic: https://lists.fd.io/mt/68776336/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14929): https://lists.fd.io/g/vpp-dev/message/14929
Mute This Topic: https://lists.fd.io/mt/68776336/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [csit-dev] [vpp-dev] Spurious API CRC failures

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> automating_vpp_api_flag_day.rst

Reading that, I see various improvements I wanted to add,
but never found time to do it.
So, here [5] is a quick edit,
with more details on the CSIT side of the process,
and with links to existing changes as examples.

Vratko.

[5] https://gerrit.fd.io/r/c/csit/+/23966

From: csit-...@lists.fd.io  On Behalf Of Dave Wallace
Sent: Thursday, December 12, 2019 12:58 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Dave Barach (dbarach) ; csit-...@lists.fd.io; Maciek 
Konstantynowicz (mkonstan) ; Andrew Yourtchenko 
; Jan Gelety -X (jgelety - PANTHEON TECH SRO at Cisco) 

Cc: vpp-dev@lists.fd.io
Subject: Re: [csit-dev] [vpp-dev] Spurious API CRC failures

Vratko,

Thanks for the info.

Can you please add a section on the proper way to add/manage the collections in 
csit/resources/api/vpp/supported_crcs.yaml to 
csit/docs/automating_vpp_api_flag_day.rst?

-daw-
On 12/11/2019 11:01 AM, Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
wrote:
> Unfortunately when I pushed the patch [2], it failed to pass the crc check

Added explanations to gerrit: [3], [4].

Vratko.

[3] 
https://gerrit.fd.io/r/c/csit/+/23926/1/resources/api/vpp/supported_crcs.yaml#34
[4] 
https://gerrit.fd.io/r/c/csit/+/23926/1/resources/api/vpp/supported_crcs.yaml#273

From: csit-...@lists.fd.io 
 On Behalf Of Dave Wallace
Sent: Wednesday, December 11, 2019 4:40 AM
To: Dave Barach (dbarach) ; 
csit-...@lists.fd.io; Maciek Konstantynowicz 
(mkonstan) ; Andrew Yourtchenko 
; Vratko Polak -X (vrpolak - 
PANTHEON TECH SRO at Cisco) ; Jan 
Gelety -X (jgelety - PANTHEON TECH SRO at Cisco) 

Cc: vpp-dev@lists.fd.io
Subject: Re: [csit-dev] [vpp-dev] Spurious API CRC failures

Correction inline...

On 12/10/2019 10:37 PM, Dave Wallace via Lists.Fd.Io wrote:
Jan/Vratko,

I spent the past several hours attempting to debug this issue.  When testing 
locally, using vpp master HEAD and csit oper-191209, I was able to reproduce 
the problem when running csit/resources/tools/integrated/check_crc.py

After attempting several iterations of reverting [0] and/or [1], I found that 
adding the crc's that were changed in [0] [1] to the main collection in 
csit/resources/api/vpp/supported_crcs.yaml would pass locally.  Unfortunately 
when I pushed the patch [2], it failed to pass the crc check in the 
csit-vpp-device-master-ubuntu1804-1n-skx verify job [2] (which I subsequently 
abandoned).

The changes in [1], don't look correct to me, since they are not related to 
gerrit 21706/17. I would think they either belong in their own collection or 
the main collection should be modified with the new crcs and not the ones in 
the "21706/17" collection.  I tried the latter experiment but the verify check 
failed locally.

At this point, it seems to me that there is a bug in the VppApiChecker but it 
is not clear to me where this root cause is when looking at the code. Or 
perhaps my local runtime environment is not correct.  I'll let you investigate 
further.

Hopefully this will help you resolve the issue quicker.

Thanks,
-daw-
[0] https://gerrit.fd.io/r/c/csit/+/23914
[1] https://gerrit.fd.io/r/c/csit/+/23921
[2] https://gerrit.fd.io/r/c/csit/+/23926

On 12/10/2019 9:03 AM, Dave Barach via Lists.Fd.Io wrote:
Folks,

This patch (among others) https://gerrit.fd.io/r/c/vpp/+/23625 changes zero 
APIs, but fails API CRC validation.

Please fix AYEC. We’re dead in the water.

Thanks... Dave

04:37:59 
/w/workspace/vpp-csit-verify-api-crc-master/src/tools/vppapigen/generate_json.py
04:38:10 Searching '/w/workspace/vpp-csit-verify-api-crc-master/src' for .api 
files.
04:38:10 json files written to: 
/w/workspace/vpp-csit-verify-api-crc-master/build-root/install-vpp-native/vpp/share/vpp/api/.
04:38:10 +++ python3 csit/resources/tools/integrated/check_crc.py
04:38:11 RuntimeError:
04:38:11 Incompatible API CRCs found in .api.json files:
04:38:11 {
04:38:11  "ip_address_details":"0xb1199745",
04:38:11  "ip_address_dump":"0x2d033de4",
04:38:11  "ip_neighbor_add_del":"0x105518b6",
04:38:11  "ip_route_add_del":"0xc1ff832d",
04:38:11  "ip_table_add_del":"0x0ffdaec0",
04:38:11  "sw_interface_ip6nd_ra_config":"0x3eb00b1c"
04:38:11 }
04:38:11 RuntimeError('Incompatible API CRCs found in .api.json files:\n{\n 
"ip_address_details":"0xb1199745",\n "ip_address_dump":"0x2d033de4",\n 
"ip_neighbor_add_del":"0x105518b6",\n "ip_route_add_del":"0xc1ff832d",\n 
"ip_table_add_del":"0x0ffdaec0",\n 
"sw_interface_ip6nd_ra_config":"0x3eb00b1c"\n}',)
04:38:11
04:38:11 @@@
04:38:11
04:38:11 VPP CSIT API CHECK FAIL!
04:38:11
04:38:11 This means the patch under test has missing messages,
04:38:11 or messages with unexpected CRCs compared to what

Re: [csit-dev] [vpp-dev] Spurious API CRC failures

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> Unfortunately when I pushed the patch [2], it failed to pass the crc check

Added explanations to gerrit: [3], [4].

Vratko.

[3] 
https://gerrit.fd.io/r/c/csit/+/23926/1/resources/api/vpp/supported_crcs.yaml#34
[4] 
https://gerrit.fd.io/r/c/csit/+/23926/1/resources/api/vpp/supported_crcs.yaml#273

From: csit-...@lists.fd.io  On Behalf Of Dave Wallace
Sent: Wednesday, December 11, 2019 4:40 AM
To: Dave Barach (dbarach) ; csit-...@lists.fd.io; Maciek 
Konstantynowicz (mkonstan) ; Andrew Yourtchenko 
; Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
; Jan Gelety -X (jgelety - PANTHEON TECH SRO at Cisco) 

Cc: vpp-dev@lists.fd.io
Subject: Re: [csit-dev] [vpp-dev] Spurious API CRC failures

Correction inline...

On 12/10/2019 10:37 PM, Dave Wallace via Lists.Fd.Io wrote:
Jan/Vratko,

I spent the past several hours attempting to debug this issue.  When testing 
locally, using vpp master HEAD and csit oper-191209, I was able to reproduce 
the problem when running csit/resources/tools/integrated/check_crc.py

After attempting several iterations of reverting [0] and/or [1], I found that 
adding the crc's that were changed in [0] [1] to the main collection in 
csit/resources/api/vpp/supported_crcs.yaml would pass locally.  Unfortunately 
when I pushed the patch [2], it failed to pass the crc check in the 
csit-vpp-device-master-ubuntu1804-1n-skx verify job [2] (which I subsequently 
abandoned).

The changes in [1], don't look correct to me, since they are not related to 
gerrit 21706/17. I would think they either belong in their own collection or 
the main collection should be modified with the new crcs and not the ones in 
the "21706/17" collection.  I tried the latter experiment but the verify check 
failed locally.

At this point, it seems to me that there is a bug in the VppApiChecker but it 
is not clear to me where this root cause is when looking at the code. Or 
perhaps my local runtime environment is not correct.  I'll let you investigate 
further.

Hopefully this will help you resolve the issue quicker.

Thanks,
-daw-
[0] https://gerrit.fd.io/r/c/csit/+/23914
[1] https://gerrit.fd.io/r/c/csit/+/23921
[2] https://gerrit.fd.io/r/c/csit/+/23926

On 12/10/2019 9:03 AM, Dave Barach via Lists.Fd.Io wrote:
Folks,

This patch (among others) https://gerrit.fd.io/r/c/vpp/+/23625 changes zero 
APIs, but fails API CRC validation.

Please fix AYEC. We’re dead in the water.

Thanks... Dave

04:37:59 
/w/workspace/vpp-csit-verify-api-crc-master/src/tools/vppapigen/generate_json.py
04:38:10 Searching '/w/workspace/vpp-csit-verify-api-crc-master/src' for .api 
files.
04:38:10 json files written to: 
/w/workspace/vpp-csit-verify-api-crc-master/build-root/install-vpp-native/vpp/share/vpp/api/.
04:38:10 +++ python3 csit/resources/tools/integrated/check_crc.py
04:38:11 RuntimeError:
04:38:11 Incompatible API CRCs found in .api.json files:
04:38:11 {
04:38:11  "ip_address_details":"0xb1199745",
04:38:11  "ip_address_dump":"0x2d033de4",
04:38:11  "ip_neighbor_add_del":"0x105518b6",
04:38:11  "ip_route_add_del":"0xc1ff832d",
04:38:11  "ip_table_add_del":"0x0ffdaec0",
04:38:11  "sw_interface_ip6nd_ra_config":"0x3eb00b1c"
04:38:11 }
04:38:11 RuntimeError('Incompatible API CRCs found in .api.json files:\n{\n 
"ip_address_details":"0xb1199745",\n "ip_address_dump":"0x2d033de4",\n 
"ip_neighbor_add_del":"0x105518b6",\n "ip_route_add_del":"0xc1ff832d",\n 
"ip_table_add_del":"0x0ffdaec0",\n 
"sw_interface_ip6nd_ra_config":"0x3eb00b1c"\n}',)
04:38:11
04:38:11 @@@
04:38:11
04:38:11 VPP CSIT API CHECK FAIL!
04:38:11
04:38:11 This means the patch under test has missing messages,
04:38:11 or messages with unexpected CRCs compared to what CSIT needs.
04:38:11 Either this Change and/or its ancestors were editing .api files,
04:38:11 or your chain is not rebased upon the recent enough VPP codebase.
04:38:11
04:38:11 Please rebase the patch to see if that fixes the problem.
04:38:11 If that fails email csit-...@lists.fd.io 
for a new
04:38:11 operational branch supporting the api changes.
04:38:11
04:38:11 @@@
04:38:11 Build step 'Execute shell' marked build as failure




-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.



View/Reply Online (#14852): https://lists.fd.io/g/vpp-dev/message/14852

Mute This Topic: https://lists.fd.io/mt/67971752/675079

Group Owner: vpp-dev+ow...@lists.fd.io

Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dwallac...@gmail.com]

-=-=-=-=-=-=-=-=-=-=-=-




-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.



View/Reply Online (#14858): https://lists.fd.io/g/vpp-dev/message/14858

Mute This Topic: https://lists.fd.io/mt/67971752/675079

Group Owner: vpp-dev+ow...@lists.fd.io

Unsubscribe: https://l

Re: [vpp-dev] CSIT - performance tests failing on Taishan

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

>> the attached patch

Converted to Gerrit: [1].

Vratko.

[1] https://gerrit.fd.io/r/c/vpp/+/23849

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Juraj Linkeš
Sent: Thursday, December 5, 2019 11:11 AM
To: Lijian Zhang (Arm Technology China) ; Peter Mikus -X 
(pmikus - PANTHEON TECH SRO at Cisco) ; Benoit Ganne (bganne) 
; Maciek Konstantynowicz (mkonstan) ; 
vpp-dev ; csit-...@lists.fd.io
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Honnappa Nagarahalli 
Subject: Re: [vpp-dev] CSIT - performance tests failing on Taishan

Hi Lijian,

The patch helped, I can't reproduce the issue now.

Thanks,
Juraj

-Original Message-
From: Lijian Zhang (Arm Technology China) 
Sent: Thursday, December 5, 2019 7:16 AM
To: Juraj Linkeš ; Peter Mikus -X (pmikus - 
PANTHEON TECH SRO at Cisco) ; Benoit Ganne (bganne) 
; Maciek Konstantynowicz (mkonstan) ; 
vpp-dev ; csit-...@lists.fd.io
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Honnappa Nagarahalli 
Subject: RE: CSIT - performance tests failing on Taishan

Hi Juraj,
Could you please try the attached patch?
Thanks.
-Original Message-
From: Juraj Linkeš 
Sent: 2019年12月4日 18:12
To: Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) ; 
Benoit Ganne (bganne) ; Maciek Konstantynowicz (mkonstan) 
; vpp-dev ; csit-...@lists.fd.io
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
Lijian Zhang (Arm Technology China) ; Honnappa 
Nagarahalli 
Subject: RE: CSIT - performance tests failing on Taishan

Hi Ben, Lijian, Honnappa,

The issue is reproducible after the second invocation of show pci:
DBGvpp# show pci
Address  Sock VID:PID Link Speed   Driver  Product Name 
   Vital Product Data
:11:00.0   2  8086:10fb   5.0 GT/s x8  ixgbe
:11:00.1   2  8086:10fb   5.0 GT/s x8  ixgbe
0002:f9:00.0   0  15b3:1015   8.0 GT/s x8  mlx5_core   CX4121A - ConnectX-4 
LX SFP28   PN: MCX4121A-ACAT_C12

   EC: A1

   SN: MT1745K13032

   V0: 0x 50 43 49 65 47 65 6e 33 ...

   RV: 0x ba
0002:f9:00.1   0  15b3:1015   8.0 GT/s x8  mlx5_core   CX4121A - ConnectX-4 
LX SFP28   PN: MCX4121A-ACAT_C12

   EC: A1

   SN: MT1745K13032

   V0: 0x 50 43 49 65 47 65 6e 33 ...

   RV: 0x ba DBGvpp# show pci
Address  Sock VID:PID Link Speed   Driver  Product Name 
   Vital Product Data
:11:00.0   2  8086:10fb   5.0 GT/s x8  ixgbe
:11:00.1   2  8086:10fb   5.0 GT/s x8  ixgbe
Aborted
Makefile:546: recipe for target 'run' failed
make: *** [run] Error 134

I've tried to do some debugging with a debug build:
(gdb) bt
...
#5  0xbe775000 in format_vlib_pci_vpd (s=0x7efa9e80 "0002:f9:00.0   
0  15b3:1015   8.0 GT/s x8  mlx5_core   CX4121A - ConnectX-4 LX SFP28", 
args=0x7ef729b0) at /home/testuser/vpp/src/vlib/pci/pci.c:230
...
(gdb) frame 5
#5  0xbe775000 in format_vlib_pci_vpd (s=0x7efa9e80 "0002:f9:00.0   
0  15b3:1015   8.0 GT/s x8  mlx5_core   CX4121A - ConnectX-4 LX SFP28", 
args=0x7ef729b0) at /home/testuser/vpp/src/vlib/pci/pci.c:230
230   else if (*(u16 *) & data[p] == *(u16 *) id)
(gdb) info locals
data = 0x7efa9cd0 "PN\025MCX4121A-ACAT_C12EC\002A1SN\030MT1745K13032", 
' ' , "V0\023PCIeGen3 x8RV\001\272"
id = 0xaaa8  indent = 91 string_types = {0xbe7b7950 "PN", 
0xbe7b7958 "EC", 0xbe7b7960 "SN", 0xbe7b7968 "MN", 0x0} p = 0 
first_line = 1

Looks like something went wrong with the 'id' variable. More is attached.

As a temporary workaround (until we fix this), we're going to replace show pci 
with something else in CSIT: https://gerrit.fd.io/r/c/csit/+/23785

Juraj

-Original Message-
From: Peter Mikus -X (pmikus - PANTHEON TECH SRO at Cisco) 
Sent: Tuesday, December 3, 2019 3:58 PM
To: Juraj Linkeš ; Benoit Ganne (bganne) 
; Maciek Konstantynowicz (mkonstan) ; 
vpp-dev ; csit-...@lists.fd.io
Cc: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) ; 
lijian.zh...@arm.com; Honnappa Nagarahalli 
Subject: RE: CSIT - performance tests failing on Taishan

Latest update is that Benoit has no access over VPN so he did try to replicate 
in local lab (assuming x86).
I will do quick fix in CSIT. I will disable MLX driver on Taishan.

Peter Mikus
Engineer - Software
Cisc

Re: [vpp-dev] vpp19.08 ipsec vpp_papi

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> This situation does not happen when I use CLI like this

I think the difference is that CLI is restricted,
it can only accept printable characters on input.
Therefore it assumes it gets "hexlified" value,
and applies "unhexlify" on its input.

Contrary to that, PAPI (hopefully) can handle
arbitrary u8 arrays, so it does not unhexlify.

> local_crypto_key = "2b7e151628aed2a6abf7158809cf4f3d",

You can try to use the unhexlified (binary) string:
  local_crypto_key = 
b"\x2b\x7e\x15\x16\x28\xae\xd2\xa6\xab\xf7\x15\x88\x09\xcf\x4f\x3d",

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Sunday, November 24, 2019 4:31 PM
To: Terry 
Cc: vpp-dev 
Subject: Re: [vpp-dev] vpp19.08 ipsec vpp_papi

That output is not random.  It is the hex of your string.
2b7e -> 32 62 37 65

On Sun, Nov 24, 2019 at 8:06 AM Terry 
mailto:zenghao...@163.com>> wrote:
Dear VPP experts,

I'm trying to configure ipsec with python API in vpp19.08.
My configurations are as follows:

reply = vpp.api.ipsec_tunnel_if_add_del(is_add = 1,
local_ip = "192.168.1.1",
remote_ip = "192.168.2.2",
local_spi = 1031,
remote_spi = 1030,
crypto_alg = 7,
local_crypto_key_len = 16,
local_crypto_key = "2b7e151628aed2a6abf7158809cf4f3d",
remote_crypto_key_len = 16,
remote_crypto_key = "2b7e151628aed2a6abf7158809cf4f3d",
integ_alg = 2,
local_integ_key_len = 16,
local_integ_key = "4339314b55523947594d6d3547666b45",
remote_integ_key_len = 16,
remote_integ_key = "4339314b55523947594d6d3547666b45",
renumber = 1,
show_instance = 1)
But the output SA information is as follows:
vpp# show ipsec sa 0
[0] sa 2147483648 (0x8000) spi 1030 (0x0406) protocol:esp flags:[tunnel 
inbound aead ]
   locks 1
   salt 0x0
   seq 0 seq-hi 0
   last-seq 0 last-seq-hi 0 window 

   crypto alg aes-gcm-128 key 32623765313531363238616564326136
   integrity alg sha1-96 key 3439333134623535353233393437
   packets 0 bytes 0
   table-ID 0 tunnel src 192.168.2.2 dst 192.168.1.1

The crypto_key I configured is '2b7e151628aed2a6abf7158809cf4f3d', but the 
output key is '32623765313531363238616564326136'.
The output crypto key looks like a random number.
This situation does not happen when I use CLI like this:
'create ipsec tunnel local-ip 192.168.1.1 remote-ip 192.168.2.2 local-spi 1031 
remote-spi 1030 local-crypto-key 2b7e151628aed2a6abf7158809cf4f3d 
remote-crypto-key 2b7e151628aed2a6abf7158809cf4f3d crypto-alg aes-gcm-128'

Could you please give me some help?

Best regards,
Arvin




-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14676): https://lists.fd.io/g/vpp-dev/message/14676
Mute This Topic: https://lists.fd.io/mt/61874477/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14689): https://lists.fd.io/g/vpp-dev/message/14689
Mute This Topic: https://lists.fd.io/mt/61874477/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Change to Gerrit

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> implementing something like [0]

+1, but...

> skip the jobs and set verify label after the codestyle checks if no files 
> were changed

... how do you imagine this being implemented in ci-management?

Currently, the other jobs are triggered by checkstyle_success
Gerrit comment, but how would the new checkstyle check know
when to skip it?

I can imagine having two checkstyle jobs
(one skippable and commenting, one unskippable and not commenting),
but +1 from the latter job would not remove the previous -1 from the former.

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Wednesday, November 20, 2019 7:33 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] Change to Gerrit

How would the group feel about implementing something like [0], so that changes 
to the commit message don't trigger rebuilds?

To enforce the commit message structure, we could skip the jobs and set verify 
label after the codestyle checks if no files were changed.
Maybe others don't care, but I don't like wasting cpu cycles/developer's time, 
and I weigh that before clarifying a commit message.

[0] 
https://gerrit-review.googlesource.com/Documentation/config-labels.html#label_copyAllScoresIfNoCodeChange
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14660): https://lists.fd.io/g/vpp-dev/message/14660
Mute This Topic: https://lists.fd.io/mt/60969892/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Requirements for DPDK pmd/feature testing in CSIT vpp-device-test jobs

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> a +2 triggers a pre-commit gate where VOM and the dist builds and the 
> extended tests are run.
> If everything passes, the change is merged, if not, the +2 is removed.

+1 to that, but I highly doubt Gerrit supports such a workflow.

Additional features we would like to have:
+ Manual trigger to run the extended tests without also merging,
  (for debugging purposes after previous extended tests failed).
+ The gate job rebases on HEAD and only one run is allowed at a time
  (to avoid merge errors).
+ The gate job squashes multiple pending changes before testing
  (to keep up with the high frequency of +2 without introducing long queues).

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Tuesday, November 19, 2019 12:10 PM
To: Paul Vinciguerra 
Cc: Dave Wallace ; vpp-dev 
Subject: Re: [vpp-dev] Requirements for DPDK pmd/feature testing in CSIT 
vpp-device-test jobs

Can we expand this discussion to discuss the VPP CI workflow?  I would like to 
see a decoupling of development and integration.
As I mentioned the other day,  It would be great if we could rebuild the 
containers whenever a commit updated the Makefile or the requirements.txt files.

I'd also like to throw out the idea of breaking up the verify job.  I think 
that if we were to remove VOM and the dist builds from verify and change the 
workflow so that a +2 triggers a pre-commit gate where VOM and the dist builds 
and the extended tests are run.  If everything passes, the change is merged, if 
not, the +2 is removed.  The existing csit job could be non-voting (so the csit 
folks could have a heads up) in the first phase, and voting in pre-commit-phase.

Paul

On Mon, Nov 18, 2019 at 12:18 PM Paul Vinciguerra via 
Lists.Fd.Io 
mailto:vinciconsulting@lists.fd.io>>
 wrote:
Hi Dave.

Where does the vpp-device-test live?

On Mon, Nov 18, 2019 at 11:13 AM Dave Wallace 
mailto:dwallac...@gmail.com>> wrote:
Folks,

Per the topic in last week's monthly VPP community meeting, the topic of DPDK 
pmd/feature testing in the CSIT devicetest job was discussed in the most recent 
CSIT community meeting (Wed 11/13).

In the beginning of the VPP project, DPDK pmd/feature testing was performed in 
the VIRL based CSIT test suites. DPDK was moved from the VPP core feature set 
into a plugin in VPP 17.04 and in later releases native device drivers were 
implemented.  Subsequently, DPDK testing was removed from the CSIT VIRL tests.  
Also the CSIT team put a plan put in place for all of the VIRL tests to be 
moved and the VIRL servers re-purposed.  In addition, the CSIT vpp-device-test 
job was created to provide test coverage of device level VPP features that 
cannot be tested in VPP's 'make test' framework. The plan for re-purposing the 
VIRL servers is complete and the vpp-device-test job is slated to become voting 
once it is stable enough for continuous-integration testing.

The CSIT team would like input from the VPP community on exactly what DPDK 
PMD's and/or features are required to be added to the CSIT vpp-device-test jobs.

Thanks,
-daw-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14617): https://lists.fd.io/g/vpp-dev/message/14617
Mute This Topic: https://lists.fd.io/mt/60208819/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14618): https://lists.fd.io/g/vpp-dev/message/14618
Mute This Topic: https://lists.fd.io/mt/60208819/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14655): https://lists.fd.io/g/vpp-dev/message/14655
Mute This Topic: https://lists.fd.io/mt/60208819/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Multiple patch validation failures: yamllint AWOL

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> Makefile by testing for JENKINS_URL?

Inputs to the program make should be understandable by human.
When I want to predict locally what would the checkstyle job vote,
I prefer something like:
  make checkstyle-all
rather than:
  JENKINS_URL=jenkins.fd.io make checkstyle

Or did you mean a different variable?
Here [3] is a typical list.

Vratko.

[3] 
https://jenkins.fd.io/view/vpp/job/vpp-checkstyle-verify-master/12455/injectedEnvVars/

From: vpp-dev@lists.fd.io  On Behalf Of Paul Vinciguerra
Sent: Wednesday, November 13, 2019 4:28 PM
To: Dave Wallace 
Cc: vpp-dev 
Subject: Re: [vpp-dev] Multiple patch validation failures: yamllint AWOL

How do people feel about managing the CI where possible via the Makefile by 
testing for JENKINS_URL?

On Wed, Nov 13, 2019 at 10:11 AM Dave Wallace 
mailto:dwallac...@gmail.com>> wrote:
Status update:

Florin fixed 'make checkstyle' with [1]  (Thanks Florin! :)

Ed Kern has added 'make install-dep' to the checkstyle verify build executors.

The revert of my ci-management patch [2] will be abandoned.

Florin's patch will be reverted once the new build executors are available and 
verified that yamllint is available.

Sorry for the bog-up.

Thanks,
-daw-

[1] https://gerrit.fd.io/r/c/vpp/+/23391
[2] https://gerrit.fd.io/r/c/ci-management/+/23410

On 11/13/2019 8:58 AM, Dave Wallace via Lists.Fd.Io wrote:
Dave,

My bad.  I had Vanessa merge a patch to ci-management [0] which I tested 
locally, but forgot that the build executors don't run 'make install-dep'.

Unfortunately the ci-management verify jobs don't actually run any of the CI 
jobs to ensure that the system still works with the patch being submitted...

I'm in the process of reverting the patch until the build executors can be 
upgraded.

Thanks,
-daw-
[0] https://gerrit.fd.io/r/c/ci-management/+/23364
On 11/13/2019 7:35 AM, Dave Barach via Lists.Fd.Io wrote:
I wonder where “yamllint” went? Multiple patches affected...

07:28:23 clang-format version 6.0.0-1ubuntu2 (tags/RELEASE_600/final)
07:28:24 HEAD detached at dce8b380a
07:28:24 nothing to commit, working tree clean
07:28:24 ***
07:28:24 * VPP CHECKSTYLE SUCCESSFULLY COMPLETED
07:28:24 ***
07:28:24 yamllint /w/workspace/vpp-checkstyle-verify-master/src
07:28:24 make: yamllint: Command not found
07:28:24 Makefile:561: recipe for target 'checkstyle' failed
07:28:24 make: *** [checkstyle] Error 127
07:28:24 Build step 'Execute shell' marked build as failure

FWIW... Dave


-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.



View/Reply Online (#14579): https://lists.fd.io/g/vpp-dev/message/14579

Mute This Topic: https://lists.fd.io/mt/55791974/675079

Group Owner: vpp-dev+ow...@lists.fd.io

Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dwallac...@gmail.com]

-=-=-=-=-=-=-=-=-=-=-=-


-=-=-=-=-=-=-=-=-=-=-=-

Links: You receive all messages sent to this group.



View/Reply Online (#14583): https://lists.fd.io/g/vpp-dev/message/14583

Mute This Topic: https://lists.fd.io/mt/55791974/675079

Group Owner: vpp-dev+ow...@lists.fd.io

Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[dwallac...@gmail.com]

-=-=-=-=-=-=-=-=-=-=-=-

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14589): https://lists.fd.io/g/vpp-dev/message/14589
Mute This Topic: https://lists.fd.io/mt/55791974/1594641
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  
[pvi...@vinciconsulting.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14596): https://lists.fd.io/g/vpp-dev/message/14596
Mute This Topic: https://lists.fd.io/mt/55791974/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] API and plugins

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> plugins are not disabled by default

So? Default is not minimal, we are explicitly disabling default plugins:
plugins { plugin default { disable } plugin dpdk_plugin.so { enable } }

Vratko.

From: vpp-dev@lists.fd.io  On Behalf Of Andrew Yourtchenko
Sent: Friday, November 8, 2019 4:17 PM
To: Vratko Polak -X (vrpolak - PANTHEON TECH SRO at Cisco) 
Cc: vpp-api-...@lists.fd.io; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] API and plugins

These “New” plugins are not disabled by default, how comes they are not loaded 
in your scenario if you say you don’t know about them?

What does the config that experiences problems look like ?

--a


On 8 Nov 2019, at 14:47, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
Cisco) via Lists.Fd.Io 
mailto:vrpolak=cisco@lists.fd.io>> wrote:


Questions: If an API message handler is moved into a different plugin,
does it constitute a (backward incompatible) API change?
If yes, should we use similar checks and processes as for CRC changes?

Example (not merged yet): sw_interface_ip6nd_ra_config
is being moved from vnet [0] to plugins/ip6-nd [1].

Discussion: While section 7 of RFC2544 [2] requires the tests
to be run with all plugins enabled, CSIT tests are also aimed
at detecting performance regressions in specific functions,
which is less noisy when enabling minimal set of plugins needed.
In order to keep equivalence between configurations
of performance and functional tests,
CSIT uses minimal plugin configurations when verifying API functionality.
As ip6-nd is a new plugin introduced in 22819,
CSIT functional tests are not enabling it,
which results in the API message to become unrecognized [3] by the VPP process.
The current CRC checks aggregate all messages (with CRCs)
into a flat list, so the information on which plugin handles a message is lost.
Should we (CSIT) change the checks to track also the implementing plugin
(making 22819 fail the CRC job and triggering API upgrade process [4])?

Note: The issue of changing plugin requirements
was first encountered in a Change [5],
which does not touch .api files,
so even the proposed update of CRC job would not have detected it.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/22819/10/src/vnet/ip/ip.api#b253
[1] https://gerrit.fd.io/r/c/vpp/+/22819/10/src/plugins/ip6-nd/ip6_nd.api#93
[2] https://tools.ietf.org/html/rfc2544
[3] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-device-master-1n-skx/4129/archives/log.html.gz#s1-s1-s1-s6-s1-t1-k2-k8-k13
[4] 
https://github.com/FDio/csit/blob/f7d43390a6ce60284f54cad4e66b66d1ecd4a166/docs/automating_vpp_api_flag_day.rst
[5] https://gerrit.fd.io/r/c/vpp/+/22980

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14546): https://lists.fd.io/g/vpp-dev/message/14546
Mute This Topic: https://lists.fd.io/mt/47762399/675608
Group Owner: vpp-dev+ow...@lists.fd.io<mailto:vpp-dev+ow...@lists.fd.io>
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [ayour...@gmail.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14548): https://lists.fd.io/g/vpp-dev/message/14548
Mute This Topic: https://lists.fd.io/mt/47762399/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] API and plugins

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Questions: If an API message handler is moved into a different plugin,
does it constitute a (backward incompatible) API change?
If yes, should we use similar checks and processes as for CRC changes?

Example (not merged yet): sw_interface_ip6nd_ra_config
is being moved from vnet [0] to plugins/ip6-nd [1].

Discussion: While section 7 of RFC2544 [2] requires the tests
to be run with all plugins enabled, CSIT tests are also aimed
at detecting performance regressions in specific functions,
which is less noisy when enabling minimal set of plugins needed.
In order to keep equivalence between configurations
of performance and functional tests,
CSIT uses minimal plugin configurations when verifying API functionality.
As ip6-nd is a new plugin introduced in 22819,
CSIT functional tests are not enabling it,
which results in the API message to become unrecognized [3] by the VPP process.
The current CRC checks aggregate all messages (with CRCs)
into a flat list, so the information on which plugin handles a message is lost.
Should we (CSIT) change the checks to track also the implementing plugin
(making 22819 fail the CRC job and triggering API upgrade process [4])?

Note: The issue of changing plugin requirements
was first encountered in a Change [5],
which does not touch .api files,
so even the proposed update of CRC job would not have detected it.

Vratko.

[0] https://gerrit.fd.io/r/c/vpp/+/22819/10/src/vnet/ip/ip.api#b253
[1] https://gerrit.fd.io/r/c/vpp/+/22819/10/src/plugins/ip6-nd/ip6_nd.api#93
[2] https://tools.ietf.org/html/rfc2544
[3] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-device-master-1n-skx/4129/archives/log.html.gz#s1-s1-s1-s6-s1-t1-k2-k8-k13
[4] 
https://github.com/FDio/csit/blob/f7d43390a6ce60284f54cad4e66b66d1ecd4a166/docs/automating_vpp_api_flag_day.rst
[5] https://gerrit.fd.io/r/c/vpp/+/22980

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14546): https://lists.fd.io/g/vpp-dev/message/14546
Mute This Topic: https://lists.fd.io/mt/47762399/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] surprising performance on denverton

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Hello everyone.

I am working on automating "git bisect" process
for locating mainly performance regressions and progressions,
(also usable for locating breakages and fixes).

Of course, the process works correctly
only if the performance results are stable enough.
And we know from the per-patch perf VPP verify job
that many testcases are not reliable enough.
Instead of code quality only, something else
is also influencing the results, and repeated trials
do not seem independent enough for usual statistical methods to work.

While testing the "bisection script" prototype,
I have noticed that the results from Denverton platform
show smaller spread in trial results,
enabling the script to locate smaller regressions reliably.

But when I tried ip4base testcase (2node testbed, meaning one
traffic generator and one VPP on physical machine),
I found very surprising results.
Full log is [0], but here are results (MRR,
number of 64B packets received in 1 second under line rate load)
of 60 trials run back-to-back on the same VPP build and testbed:

[6058929.0, 6044436.0, 6051129.0, 6070631.0, 6056215.0, 6061268.0,
 6057762.0, 6059699.0, 6063921.0, 6066904.0, 6051627.0, 6055370.0,
 6047920.0, 6069624.0, 6054088.0, 6055737.0, 6047438.0, 6047390.0,
 6060160.0, 6052960.0, 6056360.0, 6055028.0, 6045457.0, 6060301.0,
 6058869.0, 6059033.0, 6059880.0, 9712980.0, 8810073.0, 6050160.0,
 6063784.0, 6057699.0, 6061905.0, 6059174.0, 6061494.0, 6057585.0,
 6043699.0, 6045381.0, 6048290.0, 6051779.0, 9009111.0, 8817494.0,
 8847234.0, 7014022.0, 7385958.0, 10867843.0, 10991701.0, 10926844.0,
 10971236.0, 6056055.0, 6048881.0, 6059600.0, 6037948.0, 6047664.0,
 6057797.0, 6053424.0, 6057050.0, 6044720.0, 6042256.0, 6054110.0]

The ~6.05 Mpps values are consistent and usually seen on each VPP build,
but the red values can reach almost double of that.
Such values are quite rare.
The VPP build which got the red values
has come from a change which only adds a "make test" test;
so I believe earlier builds can also do that,
60 was just not enough tries to make the red values happen.

Has anybody seen similar results?
Has anybody got an idea of what could be happening inside VPP?
Can we fix VPP to be more consistent (ideally on the higher performance)?

Vratko.

[0] 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-csit-verify-perf-master-2n-dnv/6/console.log.gz
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14543): https://lists.fd.io/g/vpp-dev/message/14543
Mute This Topic: https://lists.fd.io/mt/47503893/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VAT can support define a macro? #vapi

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Macros are currently not supported in .api files.

> define MAX_NUM   (12)

The "define" keyword defines API messages, not macros.

The C preprocessor is one possibility to add support for macros,
but not sure whether it would play well with .api syntax.

You can try to add that (or a similar feature) by yourself,
mainly to this [0] directory.

Vratko.

[0] https://github.com/FDio/vpp/tree/master/src/tools/vppapigen

From: vpp-dev@lists.fd.io  On Behalf Of wei_sky2...@163.com
Sent: Tuesday, September 24, 2019 1:14 PM
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] VAT can support define a macro? #vapi


[Edited Message Follows]
in api file
I want to define a macro such as
define MAX_NUM   (12)

define my_structure
{
   u32 a;
   u8[MAX_NUM];
}
"define MAX_NUM   (12)" is can't be recognized，how can i do to define this？
Thanks
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14047): https://lists.fd.io/g/vpp-dev/message/14047
Mute This Topic: https://lists.fd.io/mt/34274817/21656
Mute #vapi: https://lists.fd.io/mk?hashtag=vapi&subid=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [csit-dev] API check failure

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Fixed.
Short report: https://lists.fd.io/g/vpp-api-dev/message/34
Longer discussion on how to prevent similar things
from happening in future: #csit-dev channel on FD.io slack.
We will update the flag day document when we conclude the discussion.

Vratko.

-Original Message-
From: Dave Barach (dbarach)  
Sent: Friday, September 20, 2019 1:41 PM
To: Benoit Ganne (bganne) ; Maciek Konstantynowicz (mkonstan) 
; Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
Cisco) ; Andrew Yourtchenko (ayourtch) ; 
Ed Kern (ejk) ; Dave Wallace -X (dwallace - NETPACE INC at 
Cisco) 
Cc: csit-...@lists.fd.io; vpp-dev@lists.fd.io
Subject: RE: [csit-dev] API check failure
Importance: High

Yes. Since folks have been unable to debug the problem so far, we need to
disable voting on the API CRC checker job. 

Ed Kern [or anyone with appropriate access rights!]: please do that
immediately, and send email to vpp-dev@lists.fd.io so folks can "recheck"
patches which failed for no good reason.

We've built up a backlog of patches to recheck, and it's reached the point
where we need to get things moving again.

Thanks... Dave

-Original Message-
From: Benoit Ganne (bganne)  
Sent: Friday, September 20, 2019 5:58 AM
To: Maciek Konstantynowicz (mkonstan) ; Dave Barach
(dbarach) ; Vratko Polak -X (vrpolak - PANTHEON
TECHNOLOGIES at Cisco) 
Cc: csit-...@lists.fd.io
Subject: RE: [csit-dev] API check failure

I am facing the same issue: https://gerrit.fd.io/r/c/vpp/+/22103
I rebased on top of https://gerrit.fd.io/r/c/vpp/+/21458/7 but it is
complaining about this:

Dir check found incompatible API CRCs:
{
 "cop_interface_enable_disable":"0x5501adee",
 "cop_whitelist_enable_disable":"0xdebe13ea"
}
RuntimeError('Dir check found incompatible API CRCs:\n{\n
"cop_interface_enable_disable":"0x5501adee",\n
"cop_whitelist_enable_disable":"0xdebe13ea"\n}',)

Is there anything else to be done?

ben

> -Original Message-
> From: csit-...@lists.fd.io  On Behalf Of Maciek 
> Konstantynowicz (mkonstan) via Lists.Fd.Io
> Sent: jeudi 19 septembre 2019 21:59
> To: Dave Barach (dbarach) ; Vratko Polak -X 
> (vrpolak - PANTHEON TECHNOLOGIES at Cisco) 
> Cc: csit-...@lists.fd.io
> Subject: Re: [csit-dev] API check failure
> 
> + Vratko
> 
> 
> 
>   On 19 Sep 2019, at 18:50, Dave Barach (dbarach)   > wrote:
> 
>   https://gerrit.fd.io/r/c/vpp/+/22150/2/src/vnet/interface_cli.c
> 
>   is a one-line fix which changes
> 
>   if (pp->pcap_enable == 0)
> 
>   to
> 
>   if (pp->pcap_enable)
> 
>   in one place. Clearly no API change, but unfortunately resulted in 
> the CSIT API check failure shown below. The patch appears not to need 
> a rebase.
> 
>   Mind taking a look?
> 
>   Thanks... Dave
> 
> 
>   11:42:05 RuntimeError:
>   11:42:05 Dir check found incompatible API CRCs:
>   11:42:05 {
>   11:42:05  "cop_interface_enable_disable":"0x69d24598",
>   11:42:05  "cop_whitelist_enable_disable":"0x8bb8f6dc"
>   11:42:05 }
>   11:42:05 RuntimeError('Dir check found incompatible API CRCs:\n{\n 
> "cop_interface_enable_disable":"0x69d24598",\n
> "cop_whitelist_enable_disable":"0x8bb8f6dc"\n}',)
>   11:42:05
>   11:42:05
> 
>   11:42:05
>   11:42:05 VPP CSIT API CHECK FAIL!
>   11:42:05
>   11:42:05 This means the patch under test has missing messages,
>   11:42:05 or messages with unexpected CRCs compared to what CSIT 
> needs.
>   11:42:05 Either this Change and/or its ancestors were editing .api 
> files,
>   11:42:05 or your chain is not rebased upon the recent enough VPP 
> codebase.
>   11:42:05
>   11:42:05 Please rebase the patch to see if that fixes the problem.
>   11:42:05 If that fails email csit-...@lists.fd.io  d...@lists.fd.io>  for a new
>   11:42:05 operational branch supporting the api changes.
>   11:42:05
>   11:42:05
> 
>   11:42:05 Build step 'Execute shell' marked build as failure
>   11:42:05 $ ssh-agent -k
>   11:42:05 unset SSH_AUTH_SOCK;
>   11:42:05 unset SSH_AGENT_PID;
>   11:42:05 echo Agent pid 67 killed;
>   11:42:05 [ssh-agent] Stopped.
>   11:42:07 [PostBuildScript] - Executing post build scripts.
>   11:42:07 [vpp-csit-verify-api-crc-master] $ /bin/bash 
> /tmp/jenkins13211504889301879538.sh
>   11:42:07 Build logs: https://logs.fd.io/production/vex-yul-
> rot-jenkins-1/vpp-csit-verify-api-crc-
> master/755">https://logs.fd.io/production/vex-yul-rot-jenkins-1/vpp-cs
> it-
> verify-api-crc-master/755
> 

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14033): https://lists.fd.io/g/vpp-dev/message/14033
Mute This Topic: https://lists.fd.io/mt/34229401/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vp

[vpp-dev] CRC job reliable again

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Yesterday, we did a risky thing of merging
two [0] VPP [1] changes affecting APIs.
Without verifying CSIT really is ready
for the end result. Fixed [2] only now.

From the process point of view,
we have some possible improvements planned,
but until we are able to implement them,
we will be sticking to one API change at a time.

Vratko.

P.S.: Trending jobs also failed,
due to missing cherry-pick. Also fixed [3].

[0] https://gerrit.fd.io/r/c/vpp/+/21997
[1] https://gerrit.fd.io/r/c/vpp/+/21458
[2] https://gerrit.fd.io/r/c/csit/+/22177
[3] https://gerrit.fd.io/r/c/csit/+/22164

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#14030): https://lists.fd.io/g/vpp-dev/message/14030
Mute This Topic: https://lists.fd.io/mt/34229140/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VAT to PAPI issue with policer

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

VAT classify_add_del_session contains "skip_n 1",
but PAPI classify_add_del_session does not contain
"skip_n_vectors" argument, (so the default 0 is used).

Not sure whether that is causing the different behavior,
but there may be other differences.

Vratko.

From: csit-...@lists.fd.io  On Behalf Of Jan Gelety via 
Lists.Fd.Io
Sent: Wednesday, September 18, 2019 10:30 AM
To: vpp-dev@lists.fd.io
Cc: csit-...@lists.fd.io
Subject: [csit-dev] VAT to PAPI issue with policer

Hello vpp-dev,

We are converting CSIT Policer.py library to use PAPI instead of VAT commands. 
Unfortunately policer functional
test is failing with „Invalid DSCP: 0, should be 20 “ in case of configuration 
via PAPI.

The root cause seems to be that policer_index  is not set for policer-classify 
as reported in packet trace:

00:00:17:623180: ip4-policer-classify
  POLICER_CLASSIFY: sw_if_index 1 next 1 table 0 offset -1 policer_index -1
00:00:17:623185: ip4-lookup
  fib 0 dpo-idx 1 flow hash: 0x
  TCP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 64, length 40, checksum 0x3eae
fragment id 0x0001
  TCP: 20 -> 80
seq. 0x ack 0x
flags 0x02 SYN, tcp header: 20 bytes
window 8192, checksum 0x535d
00:00:17:623194: ip4-rewrite
  tx_sw_if_index 2 dpo-idx 1 : ipv4 via 20.20.20.2 
VirtualFunctionEthernet3b/a/3: mtu:9200 face0203face06030800 flow hash: 
0x
  : face0203face06030800452800013f063fae0a0a0a021414
  0020: 14020014005050022000535d
00:00:17:623205: VirtualFunctionEthernet3b/a/3-output

While in case of configuration via VAT policer_index is set correctly:

00:00:16:770119: ip4-input-no-checksum
  TCP: 10.10.10.2 -> 20.20.20.2
tos 0x00, ttl 64, length 40, checksum 0x3eae
fragment id 0x0001
  TCP: 20 -> 80
seq. 0x ack 0x
flags 0x02 SYN, tcp header: 20 bytes
window 8192, checksum 0x535d
00:00:16:770128: ip4-policer-classify
  POLICER_CLASSIFY: sw_if_index 1 next 1 table 0 offset 1200 policer_index 0
00:00:16:770132: ip4-lookup
  fib 0 dpo-idx 1 flow hash: 0x
  TCP: 10.10.10.2 -> 20.20.20.2
tos 0x50, ttl 64, length 40, checksum 0x3e5e
fragment id 0x0001
  TCP: 20 -> 80
seq. 0x ack 0x
flags 0x02 SYN, tcp header: 20 bytes
window 8192, checksum 0x535d
00:00:16:770140: ip4-rewrite
  tx_sw_if_index 2 dpo-idx 1 : ipv4 via 20.20.20.2 
VirtualFunctionEthernet3b/a/3: mtu:9200 badc0ffe0203badc0ffe06030800 flow hash: 
0x
  : badc0ffe0203badc0ffe060308004550002800013f063f5e0a0a0a021414
  0020: 14020014005050022000535d
00:00:16:770146: VirtualFunctionEthernet3b/a/3-output


VAT configuration

policer_add_del name policer1  cir 100 eir 150 cb 64 eb 64 rate_type pps 
round_type closest type 2r3c-2698 conform_action transmit exceed_action 
mark-and-transmit AF22 violate_action transmit color-aware
classify_add_del_table mask l3 ip4 dst
classify_add_del_session policer-hit-next 0 exceed-color table-index 0 skip_n 1 
match_n 2 match l3 ip4 dst 20.20.20.2
policer_classify_set_interface sw_if_index 1 ip4-table 0

have been converted to PAPI configuration

policer_add_del(conform_action_type=1,eir=150,name='policer1',cb=64,is_add=1,rate_type=1,eb=64,cir=100,violate_action_type=1,exceed_dscp=20,violate_dscp=0,color_aware=1,conform_dscp=0,type=2,exceed_action_type=2,round_type=0)
classify_add_del_table(next_table_index=4294967295,miss_next_index=4294967295,mask_len=32,match_n_vectors=2,mask='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\xff\xff\xff\xff\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',current_data_offset=0,memory_size=2097152,table_index=4294967295,nbuckets=2,skip_n_vectors=0,is_add=1,current_data_flag=0)
classify_add_del_session(advance=0,match_len=32,is_add=1,opaque_index=1,hit_next_index=0,table_index=0,action=0,match='\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x14\x14\x14\x02\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00',metadata=0)
policer_classify_set_interface(ip6_table_index=4294967295,ip4_table_index=0,l2_table_index=4294967295,is_add=1,sw_if_index=1)

based on the corresponding VAT code in api_format.c file, that means


  *   policer_index returned by policer_add_del API function has been used for 
parameter hit_next_index of classify_add_del_session API function
  *   new_table_index returned by classify_add_del_table API function has been 
used for parameter table_index of classify_add_del_session API function
  *   value of opaque_index parameter of classify_add_del_session API function 
has been set based on required pre-color setting (exceed-color in this case)

Can somebody help us to fix the issue, please?

Test log: 
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-device-master-ubuntu1804-1n-skx/3683/archives/log.html.gz#s1-s1-s1-s4-s5-t1
Tested on Ubuntu bionic system with vpp build 20.01-rc0~174-g3d1ef873d~b3588.

Thanks,
Jan
-=-

[vpp-dev] Temporary CRC failures

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

[0] has been merged, [1] has been postponed.

The VPP API CRC jobs should be reliable from now on again.


Vratko.


[0] https://gerrit.fd.io/r/c/vpp/+/21508

[1] https://gerrit.fd.io/r/c/vpp/+/21706

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13959): https://lists.fd.io/g/vpp-dev/message/13959
Mute This Topic: https://lists.fd.io/mt/34105743/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP API sometimes hangs on ip_route_details response #vppcapi #vpp_stability #vpp #binapi

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Looks like VPP-1753 to me.


I do not have much to add,

heisenbugs are hard to figure out.


Vratko.



From: vpp-dev@lists.fd.io  on behalf of 
sylvain.cadil...@jaguar-network.com 
Sent: Wednesday, September 4, 2019 01:36
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] VPP API sometimes hangs on ip_route_details response 
#vppcapi #vpp_stability #vpp #binapi


Hi vpp-dev,

I'm running a C program which adds/removes interfaces, IPs, routes, etc. to VPP 
via the C API, along with a Python test suite which checks the results of the 
first program multiple times by querying VPP using the Python bindings.

I'm seeing a fairly reproductible issue (when repeating the test suite a number 
of times), where the VPP API thread remains blocked, always after the 
clib_mem_free of a ip_route_details reply message (response to Python).

Traces:

  *   This is VPP 19.08 with slight plugin changes; I can provide the packages 
if needed.
  *   The backtrace is here: 
https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-backtrace
  *   The startup config file is here: 
https://gist.github.com/SCadilhac/bb9ca9600d757b13726e05ae34923c1a#file-startup-conf
  *   I don't know how to extract the API dump, as the CLI gets unresponsive, 
and the process /tmp/api_post_mortem file remains unwritten.
  *   The gzipped core dump is here: 
http://www.netfishers.onl/downloads/core.24542.gz

Any help is welcome :-). Is there anything I'm obviously doing wrong?
I'm happy to open a Jira issue if this can help.

Thanks,

Sylvain
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13905): https://lists.fd.io/g/vpp-dev/message/13905
Mute This Topic: https://lists.fd.io/mt/33132760/21656
Mute #vpp: https://lists.fd.io/mk?hashtag=vpp&subid=1480452
Mute #binapi: https://lists.fd.io/mk?hashtag=binapi&subid=1480452
Mute #vpp_stability: https://lists.fd.io/mk?hashtag=vpp_stability&subid=1480452
Mute #vppcapi: https://lists.fd.io/mk?hashtag=vppcapi&subid=1480452
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] how VAT work with Socket IP Address

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

>> See ssh -R / -L for example.

> Is this way not feasible?


It is feasible. In CSIT we use this (ssh -L) way,

although it took me some time to tune the ssh command

(so the tunnel is closed either on-demand or by timeout).


Here [0] is a part of the code composing the command,

in case you are interested in (complicated but working) example.


Also, some systems have fairly old ssh client,

according to [1] you need at least OpenSSH 6.7

(just on the app side, vpp side can have older ssh server).


Vratko.


[0] 
https://github.com/FDio/csit/blob/545216fdee77b0b9ddc8d5e8c0f2e5662cacea76/resources/libraries/python/PapiExecutor.py#L249-L255

[1] https://lwn.net/Articles/609321/



From: vpp-dev@lists.fd.io  on behalf of 
wei_sky2...@163.com 
Sent: Thursday, August 29, 2019 13:17
To: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] how VAT work with Socket IP Address

Thank you for your answer!
I want to develop an app on an PC,and VPP run on another PC,my app use VPP API 
to config VPP?Is this way not feasible?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13882): https://lists.fd.io/g/vpp-dev/message/13882
Mute This Topic: https://lists.fd.io/mt/33064939/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] ValueError: Fieldname 'async' is a python keyword and is not accessible via the python API

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

If next version of Python adds a new reserved keyword,

it will "break" clients using PAPI in that version

(for calls using that keyword as argument name) anyway.


The main downside is that our current checks

only look at locally installed Python.

So [3] got +1 just because verify jobs use Python 2 and not 3.


Vratko.


[3] https://gerrit.fd.io/r/c/vpp/+/21371



From: vpp-dev@lists.fd.io  on behalf of Florin Coras 

Sent: Wednesday, August 21, 2019 17:47
To: Damjan Marion
Cc: vpp-dev; Yu, Ping; Ole Troan (otroan)
Subject: Re: [vpp-dev] ValueError: Fieldname 'async' is a python keyword and is 
not accessible via the python API


On Aug 21, 2019, at 8:32 AM, Damjan Marion 
mailto:dmar...@me.com>> wrote:


On 21 Aug 2019, at 17:15, Florin Coras 
mailto:fcoras.li...@gmail.com>> wrote:

Interesting. Want to push a patch to change async_enable or something like that 
here [1]? Or I can do that.

Florin

[1] https://git.fd.io/vpp/diff/src/plugins/tlsopenssl/tls_openssl.api?id=be4d1aa

On Aug 21, 2019, at 7:41 AM, Damjan Marion via Lists.Fd.Io 
mailto:dmarion=me@lists.fd.io>> wrote:


May be due to the fact that I use newer ubuntu version, but i'm not able to 
build VPP on master.

This is error message I get:

[1283/1678] Generating API header 
/home/damarion/cisco/vpp8/bui...build-vpp_debug-native/vpp/plugins/tlsopenssl/tls_openssl.api.h
FAILED: plugins/tlsopenssl/tls_openssl.api.h
cd 
/home/damarion/cisco/vpp8/build-root/build-vpp_debug-native/vpp/plugins/tlsopenssl
 && mkdir -p 
/home/damarion/cisco/vpp8/build-root/build-vpp_debug-native/vpp/plugins/tlsopenssl
 && /home/damarion/cisco/vpp8/src/tools/vppapigen/vppapigen --includedir 
/home/damarion/cisco/vpp8/src --input 
/home/damarion/cisco/vpp8/src/plugins/tlsopenssl/tls_openssl.api --output 
/home/damarion/cisco/vpp8/build-root/build-vpp_debug-native/vpp/plugins/tlsopenssl/tls_openssl.api.h
ValueError: Fieldname 'async' is a python keyword and is not accessible via the 
python API.


And looks like reverting [1] helps

[1] https://git.fd.io/vpp/commit/?id=be4d1aa


It sounds to me like right solution to the problem is to avoid python 
keyword-related constrains in API msg fields.

Next time they may add new keyword and break existing API definitions

Agreed. I suspect the issue is that we pass those fields as named params in 
python apis, but Ole should know the actual answer.

Florin
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13802): https://lists.fd.io/g/vpp-dev/message/13802
Mute This Topic: https://lists.fd.io/mt/32978936/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] API change on master

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

VPP API change [0] has been merged.

For open VPP changes that means

the next rechecks or uploads (without rebasing)

will fail the CRC check job.


Vratko.


[0] https://gerrit.fd.io/r/c/vpp/+/18361
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13795): https://lists.fd.io/g/vpp-dev/message/13795
Mute This Topic: https://lists.fd.io/mt/32965901/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [tsc] [vpp-dev] Project Proposal for uDPI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> UDPI will be easily misunderstood as UDP+I.

> uniDPI is ok?


+1 to uniDPI.


Vratko.



From: 李幸福(Xingfu Li) 
Sent: Tuesday, August 13, 2019 12:45
To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco); Ni, Hongjun; 
Alexey; t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W; Hong, Yang A; Chang, Harry; 
gu.ji...@zte.com.cn; Shan Jianghua; Zhang Yang; Wu Shuai; Xia Yuying; Fan 
Chenggang; ?? , "Liu Zhong" , 
"Zhao Yong" , "Chen Haiquan" 
Subject: Re: Re: [tsc] [vpp-dev] Project Proposal for uDPI

uniDPI is ok?
UDPI will be easily misunderstood as UDP+I.


李幸福/Xingfu Li
山东华辰泰尔信息科技股份有限公司/Shandong Huachentel Information Technology Co., Ltd.
山东省济南市高新区舜泰广场8号楼西区17层 250101/West of F17, Building 8 of Shuntai Square, 
High-tech Zone, Jinan, China 250101
电话/Tel：86-531-62325307 86-531-88877658-8019
网址/Web：www.huachentel.com

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
Cisco)
Date: 2019-08-13 18:34
To: Ni, Hongjun; 
Alexey; 
t...@lists.fd.io
CC: vpp-dev@lists.fd.io; Wang, Xiang 
W; Hong, Yang A; 
Chang, Harry; 
gu.ji...@zte.com.cn; Shan 
Jianghua; Zhang 
Yang; Li 
Xingfu; Wu Shuai; 
Xia Yuying; Fan 
Chenggang; Gao 
Feng; Liu Zhong; 
Zhao Yong; Chen Haiquan
Subject: Re: [tsc] [vpp-dev] Project Proposal for uDPI

>> uDPI

> microDPI


Universal Deep Packet Inspection is the new project here.

Would it make sense to use UDPI as its abbreviation

(e.g. capital U, to distinguish from "micro")?


Vratko.



From: t...@lists.fd.io  on behalf of Ni, Hongjun 

Sent: Tuesday, August 13, 2019 03:47
To: Ni, Hongjun; Alexey; t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W; Hong, Yang A; Chang, Harry; 
gu.ji...@zte.com.cn; Shan Jianghua; Zhang Yang; Li Xingfu; Wu Shuai; Xia 
Yuying; Fan Chenggang; Gao Feng; Liu Zhong; Zhao Yong; Chen Haiquan
Subject: Re: [tsc] [vpp-dev] Project Proposal for uDPI

Hi Alexey and all,

Alexey and I had a discussion offline.
It seems that we mentioned two different projects except the same project name.

The project Alexey mentioned is “microDPI packet inspection for network 
analytics”.
It is developed based on Machine Learning and Python language, using GPL3.0 
License.
Last activity for this project is 10.04.2017, which is 2 years ago. That is why 
Alexey said it is died.
It supports six specific protocols/applications.

The project we proposed here is “Universal Deep Packet Inspection”.
It is developed based on industry regex matching library, VPP, and C language, 
using Apache 2.0 License.
We plan to support most industry network protocols/applications.

Both have different scopes, features, implementations and Licenses.

Thanks,
Hongjun

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Ni, Hongjun
Sent: Friday, August 9, 2019 10:03 PM
To: Alexey ; t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W ; Hong, Yang A 
; Chang, Harry ; 
gu.ji...@zte.com.cn; Shan Jianghua ; Zhang Yang 
; Li Xingfu ; Wu Shuai 
; Xia Yuying ; Fan Chenggang 
; Gao Feng ; Liu Zhong 
; Zhao Yong ; Chen Haiquan 

Subject: Re: [vpp-dev] Project Proposal for uDPI

Hi Alexey,

Could you give some thoughts on this?

Thanks,
Hongjun

From: vpp-dev@lists.fd.io 
[mailto:vpp-dev@lists.fd.io] On Behalf Of Alexey
Sent: Friday, August 9, 2019 6:54 PM
To: Ni, Hongjun mailto:hongjun...@intel.com>>; 
t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W 
mailto:xiang.w.w...@intel.com>>; Hong, Yang A 
mailto:yang.a.h...@intel.com>>; Chang, Harry 
mailto:harry.ch...@intel.com>>; 
gu.ji...@zte.com.cn; Shan Jianghua 
mailto:shanjia...@chinatelecom.cn>>; Zhang Yang 
mailto:zhangy@chinatelecom.cn>>; Li Xingfu 
mailto:lixin...@huachentel.com>>; Wu Shuai 
mailto:wush...@inspur.com>>; Xia Yuying 
mailto:yuying...@yxlink.com>>; Fan Chenggang 
mailto:fanchengg...@sunyainfo.com>>; Gao Feng 
mailto:davidf...@tencent.com>>; Liu Zhong 
mailto:liuzho...@chinaunicom.cn>>; Zhao Yong 
mailto:zhaoyon...@huawei.com>>; Chen Haiquan 
mailto:o...@yunify.com>>
Subject: Re: [vpp-dev] Project Proposal for uDPI

udpi is died


09.08.2019, 04:11, "Ni, Hongjun" 
mailto:hongjun...@intel.com>>:

Hello FD.io TSCs



Please accept this project proposal for uDPI for consideration.

https://wiki.fd.io/view/Proje

Re: [tsc] [vpp-dev] Project Proposal for uDPI

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

>> uDPI

> microDPI


Universal Deep Packet Inspection is the new project here.

Would it make sense to use UDPI as its abbreviation

(e.g. capital U, to distinguish from "micro")?


Vratko.



From: t...@lists.fd.io  on behalf of Ni, Hongjun 

Sent: Tuesday, August 13, 2019 03:47
To: Ni, Hongjun; Alexey; t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W; Hong, Yang A; Chang, Harry; 
gu.ji...@zte.com.cn; Shan Jianghua; Zhang Yang; Li Xingfu; Wu Shuai; Xia 
Yuying; Fan Chenggang; Gao Feng; Liu Zhong; Zhao Yong; Chen Haiquan
Subject: Re: [tsc] [vpp-dev] Project Proposal for uDPI

Hi Alexey and all,

Alexey and I had a discussion offline.
It seems that we mentioned two different projects except the same project name.

The project Alexey mentioned is "microDPI packet inspection for network 
analytics".
It is developed based on Machine Learning and Python language, using GPL3.0 
License.
Last activity for this project is 10.04.2017, which is 2 years ago. That is why 
Alexey said it is died.
It supports six specific protocols/applications.

The project we proposed here is "Universal Deep Packet Inspection".
It is developed based on industry regex matching library, VPP, and C language, 
using Apache 2.0 License.
We plan to support most industry network protocols/applications.

Both have different scopes, features, implementations and Licenses.

Thanks,
Hongjun

From: vpp-dev@lists.fd.io [mailto:vpp-dev@lists.fd.io] On Behalf Of Ni, Hongjun
Sent: Friday, August 9, 2019 10:03 PM
To: Alexey ; t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W ; Hong, Yang A 
; Chang, Harry ; 
gu.ji...@zte.com.cn; Shan Jianghua ; Zhang Yang 
; Li Xingfu ; Wu Shuai 
; Xia Yuying ; Fan Chenggang 
; Gao Feng ; Liu Zhong 
; Zhao Yong ; Chen Haiquan 

Subject: Re: [vpp-dev] Project Proposal for uDPI

Hi Alexey,

Could you give some thoughts on this?

Thanks,
Hongjun

From: vpp-dev@lists.fd.io 
[mailto:vpp-dev@lists.fd.io] On Behalf Of Alexey
Sent: Friday, August 9, 2019 6:54 PM
To: Ni, Hongjun mailto:hongjun...@intel.com>>; 
t...@lists.fd.io
Cc: vpp-dev@lists.fd.io; Wang, Xiang W 
mailto:xiang.w.w...@intel.com>>; Hong, Yang A 
mailto:yang.a.h...@intel.com>>; Chang, Harry 
mailto:harry.ch...@intel.com>>; 
gu.ji...@zte.com.cn; Shan Jianghua 
mailto:shanjia...@chinatelecom.cn>>; Zhang Yang 
mailto:zhangy@chinatelecom.cn>>; Li Xingfu 
mailto:lixin...@huachentel.com>>; Wu Shuai 
mailto:wush...@inspur.com>>; Xia Yuying 
mailto:yuying...@yxlink.com>>; Fan Chenggang 
mailto:fanchengg...@sunyainfo.com>>; Gao Feng 
mailto:davidf...@tencent.com>>; Liu Zhong 
mailto:liuzho...@chinaunicom.cn>>; Zhao Yong 
mailto:zhaoyon...@huawei.com>>; Chen Haiquan 
mailto:o...@yunify.com>>
Subject: Re: [vpp-dev] Project Proposal for uDPI

udpi is died


09.08.2019, 04:11, "Ni, Hongjun" 
mailto:hongjun...@intel.com>>:

Hello FD.io TSCs



Please accept this project proposal for uDPI for consideration.

https://wiki.fd.io/view/Project_Proposals/uDPI



So far, this project has 11 founders and 15 initial committers from

Intel, ZTE, China Telecom, HuachenTel, Inspur, Yxlink, Sunyainfo, Tencent, 
China Unicom, Huawei, QingCloud.



If possible, I would like to present this on TSC meeting.



Thanks,

Hongjun
,

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13684): https://lists.fd.io/g/vpp-dev/message/13684
Mute This Topic: https://lists.fd.io/mt/32805807/675634
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub 
[devel-net-ne-vleza...@yandex.ru]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13719): https://lists.fd.io/g/vpp-dev/message/13719
Mute This Topic: https://lists.fd.io/mt/32851161/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Reminder: 19.08 RC1 is TODAY starting 18:00 UTC

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

The job [1] is non-voting, so if needed the queued runs
can be cancelled, which unblocks gerrit without affecting the overall vote.
Several people (including me) have Jenkins rights to do that.

It looks like the required executor stopped being available.
I hear the appropriate people have been notified already.

Vratko.

[1] https://jenkins.fd.io/job/vpp-csit-verify-device-master-1n-tx2/

From: vpp-dev@lists.fd.io  on behalf of Andrew Yourtchenko 

Sent: Wednesday, August 7, 2019 10:37
To: Benoit Ganne (bganne)
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Reminder: 19.08 RC1 is TODAY starting 18:00 UTC

Yeah, the timing is :) I will keep an eye on it so it goes in, if anyone 
else had other patches that they want in before rc1, please shout.

--a

> On 7 Aug 2019, at 10:32, Benoit Ganne (bganne)  wrote:
>
> Hi all,
>
> It would be good to have this patch https://gerrit.fd.io/r/c/vpp/+/21089 in 
> rc1, however, Jenkins seems to be in a weird state with the csit-verify on 
> ARM stuck...
>
> Best
> ben
>
>> -Original Message-
>> From: vpp-dev@lists.fd.io  On Behalf Of Andrew
>> Yourtchenko
>> Sent: mercredi 7 août 2019 08:20
>> To: vpp-dev 
>> Subject: [vpp-dev] Reminder: 19.08 RC1 is TODAY starting 18:00 UTC
>>
>> Hi all,
>>
>> just a reminder that today 18:00 UTC is the RC1 milestone.
>>
>> --a
>> 
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13678): https://lists.fd.io/g/vpp-dev/message/13678
Mute This Topic: https://lists.fd.io/mt/32779849/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] flag day to enable API flag day process

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Yesterday I forgot to notify

that the job [8] is fixed [9] now.


It will be kept non-voting for a week or so

to make sure no CRC surprises are about to get merged

just before the voting is enabled again.


Vratko.


[8] https://jenkins.fd.io/job/vpp-csit-verify-api-crc-master/

[9] https://gerrit.fd.io/r/c/csit/+/21060



From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Thursday, August 1, 2019 16:35
To: vpp-api-...@lists.fd.io; vpp-dev@lists.fd.io; csit-dev
Cc: Dave Wallace (dwallac...@gmail.com); Ed Kern (ejk)
Subject: RE: flag day to enable API flag day process


> job is created

... but it quickly turned to non-voting,
because a recent merge [4] changed CRCs.
It has been reverted [5] since then,
but verifying everything is fixed [6] on CSIT side
is delayed [7].

Vratko.

[4] https://gerrit.fd.io/r/c/vpp/+/20946
[5] https://gerrit.fd.io/r/c/vpp/+/20997
[6] https://gerrit.fd.io/r/c/csit/+/21002
[7] https://lists.fd.io/g/vpp-dev/message/13642

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Wednesday, July 31, 2019 6:38 PM
To: 'vpp-api-...@lists.fd.io' ; 'vpp-dev@lists.fd.io' 
; 'csit-dev' 
Cc: 'Dave Wallace (dwallac...@gmail.com)' ; Ed Kern (ejk) 

Subject: RE: flag day to enable API flag day process

[1] is now merged, job is created, and I see [3] it passing.
Vratko.
[3] https://jenkins.fd.io/job/vpp-csit-verify-api-crc-master/2/console

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Wednesday, July 31, 2019 10:41 AM
To: 'vpp-api-...@lists.fd.io' 
mailto:vpp-api-...@lists.fd.io>>; 
'vpp-dev@lists.fd.io' mailto:vpp-dev@lists.fd.io>>; 
'csit-dev' mailto:csit-...@lists.fd.io>>
Cc: 'Dave Wallace (dwallac...@gmail.com)' 
mailto:dwallac...@gmail.com>>; Ed Kern (ejk) 
mailto:e...@cisco.com>>
Subject: RE: flag day to enable API flag day process

[0] is merged, but [1] is stuck due to [2].
Vratko.
[2] https://lists.fd.io/g/vpp-dev/message/13627

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Monday, July 22, 2019 7:14 PM
To: vpp-api-...@lists.fd.io; 
vpp-dev@lists.fd.io; csit-dev 
mailto:csit-...@lists.fd.io>>
Cc: Dave Wallace (dwallac...@gmail.com) 
mailto:dwallac...@gmail.com>>; Ed Kern (ejk) 
mailto:e...@cisco.com>>
Subject: flag day to enable API flag day process

The main API flag document: [0].

I have most of implementation ready,
see [1] and comments there.

The missing parts is e-mail alerting
and postponing main vpp verify jobs
until API check passes (older stable/ VPP branches
are tricky to support in that way).

Please review, I would like to merge very soon.

Keeping CSIT list updated without automated
verify jobs takes some effort,
and it is pointless to add CRC checking only into CSIT
if VPP is not gated against API changes.
That is why both CSIT and ci-management parts
have to be merged together.

I will add the missing features
once the new verification is enabled.

Vratko.

[0] https://gerrit.fd.io/r/18356
[1] https://gerrit.fd.io/r/20708
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13667): https://lists.fd.io/g/vpp-dev/message/13667
Mute This Topic: https://lists.fd.io/mt/32561359/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] 19.08 RC1 Milestone (stable branch pull) is next Wed 7 Aug 2019!

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> Commit first to the release throttle, then "git cherry-pick" into master

Next release, next reminder why [1] I think the order should be reversed.

Vratko.

[1] https://lists.fd.io/g/vpp-dev/message/12786

-Original Message-
From: vpp-dev@lists.fd.io  On Behalf Of Andrew Yourtchenko
Sent: Wednesday, July 31, 2019 10:26 PM
To: vpp-dev 
Subject: [vpp-dev] 19.08 RC1 Milestone (stable branch pull) is next Wed 7 Aug 
2019!

Hi all,

F0 milestone is complete.

RC1 milestone is scheduled to for next Wednesday 7 August 2019, commencing at 
18:00 UTC.

Please get all of the low-risk patches required for VPP Release 19.08 merged 
into master branch before then.

On Wednesday at 18:00 UTC the branch 'master' will be closed for the period of 
time that it takes to create 'stable/1908' with the associated Jenkins jobs 
operational.

A refresher about the life after RC1:

Once the stable branch is pulled, only bug fixes are allowed to be merged 
there, and all bug fixes should conform to the following rules:

  - All bug fixes must be double-committed to the release throttle as well as 
to the master branch
  - Commit first to the release throttle, then "git cherry-pick" into master
  - Manual merges may be required, depending on the degree of divergence 
between throttle and master
  - All bug fixes need to have a Jira ticket
  - Please put Jira IDs into the commit messages.
  - Please use the same Jira ID for commits into the stable branch and master.

--a

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13645): https://lists.fd.io/g/vpp-dev/message/13645
Mute This Topic: https://lists.fd.io/mt/32671722/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] flag day to enable API flag day process

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> job is created

… but it quickly turned to non-voting,
because a recent merge [4] changed CRCs.
It has been reverted [5] since then,
but verifying everything is fixed [6] on CSIT side
is delayed [7].

Vratko.

[4] https://gerrit.fd.io/r/c/vpp/+/20946
[5] https://gerrit.fd.io/r/c/vpp/+/20997
[6] https://gerrit.fd.io/r/c/csit/+/21002
[7] https://lists.fd.io/g/vpp-dev/message/13642

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Wednesday, July 31, 2019 6:38 PM
To: 'vpp-api-...@lists.fd.io' ; 'vpp-dev@lists.fd.io' 
; 'csit-dev' 
Cc: 'Dave Wallace (dwallac...@gmail.com)' ; Ed Kern (ejk) 

Subject: RE: flag day to enable API flag day process

[1] is now merged, job is created, and I see [3] it passing.
Vratko.
[3] https://jenkins.fd.io/job/vpp-csit-verify-api-crc-master/2/console

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Wednesday, July 31, 2019 10:41 AM
To: 'vpp-api-...@lists.fd.io' 
mailto:vpp-api-...@lists.fd.io>>; 
'vpp-dev@lists.fd.io' mailto:vpp-dev@lists.fd.io>>; 
'csit-dev' mailto:csit-...@lists.fd.io>>
Cc: 'Dave Wallace (dwallac...@gmail.com)' 
mailto:dwallac...@gmail.com>>; Ed Kern (ejk) 
mailto:e...@cisco.com>>
Subject: RE: flag day to enable API flag day process

[0] is merged, but [1] is stuck due to [2].
Vratko.
[2] https://lists.fd.io/g/vpp-dev/message/13627

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Monday, July 22, 2019 7:14 PM
To: vpp-api-...@lists.fd.io; 
vpp-dev@lists.fd.io; csit-dev 
mailto:csit-...@lists.fd.io>>
Cc: Dave Wallace (dwallac...@gmail.com) 
mailto:dwallac...@gmail.com>>; Ed Kern (ejk) 
mailto:e...@cisco.com>>
Subject: flag day to enable API flag day process

The main API flag document: [0].

I have most of implementation ready,
see [1] and comments there.

The missing parts is e-mail alerting
and postponing main vpp verify jobs
until API check passes (older stable/ VPP branches
are tricky to support in that way).

Please review, I would like to merge very soon.

Keeping CSIT list updated without automated
verify jobs takes some effort,
and it is pointless to add CRC checking only into CSIT
if VPP is not gated against API changes.
That is why both CSIT and ci-management parts
have to be merged together.

I will add the missing features
once the new verification is enabled.

Vratko.

[0] https://gerrit.fd.io/r/18356
[1] https://gerrit.fd.io/r/20708
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13643): https://lists.fd.io/g/vpp-dev/message/13643
Mute This Topic: https://lists.fd.io/mt/32561359/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] troubles with centos7 packages

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Continuing from #vpp-dev slack channel:

How can I figure out the correct package names for centos? The previous way 
does not work for recent builds.
> 13:01:27  No package vpp-19.08-rc0~794_g7520e17~b7546 available.
https://logs.fd.io/production/vex-yul-rot-jenkins-1/csit-vpp-functional-master-centos7-virl/8564/console-timestamp.log.gz
16:10
This comment looks relevant, but I do not see a clear connection.
https://gerrit.fd.io/r/c/vpp/+/20756/12/src/cmake/pack.cmake#59
16:20
I thought it is .x86_64 at the end 
https://packagecloud.io/fdio/master/packages/el/7/vpp-19.08-rc0~794_g7520e17~b7546.x86_64.rpm
but it is not:
> 16:18:08 No package vpp-19.08-rc0~794_g7520e17~b7546.x86_64 available.
https://jenkins.fd.io/job/csit-vpp-functional-master-centos7-virl/8566/console

VPP developers, any hints?
CSIT committers, can we make centos jobs non-voting for now?

Vratko.
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13642): https://lists.fd.io/g/vpp-dev/message/13642
Mute This Topic: https://lists.fd.io/mt/32679631/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] flag day to enable API flag day process

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

[1] is now merged, job is created, and I see [3] it passing.
Vratko.
[3] https://jenkins.fd.io/job/vpp-csit-verify-api-crc-master/2/console

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Wednesday, July 31, 2019 10:41 AM
To: 'vpp-api-...@lists.fd.io' ; 'vpp-dev@lists.fd.io' 
; 'csit-dev' 
Cc: 'Dave Wallace (dwallac...@gmail.com)' ; Ed Kern (ejk) 

Subject: RE: flag day to enable API flag day process

[0] is merged, but [1] is stuck due to [2].
Vratko.
[2] https://lists.fd.io/g/vpp-dev/message/13627

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Monday, July 22, 2019 7:14 PM
To: vpp-api-...@lists.fd.io; 
vpp-dev@lists.fd.io; csit-dev 
mailto:csit-...@lists.fd.io>>
Cc: Dave Wallace (dwallac...@gmail.com) 
mailto:dwallac...@gmail.com>>; Ed Kern (ejk) 
mailto:e...@cisco.com>>
Subject: flag day to enable API flag day process

The main API flag document: [0].

I have most of implementation ready,
see [1] and comments there.

The missing parts is e-mail alerting
and postponing main vpp verify jobs
until API check passes (older stable/ VPP branches
are tricky to support in that way).

Please review, I would like to merge very soon.

Keeping CSIT list updated without automated
verify jobs takes some effort,
and it is pointless to add CRC checking only into CSIT
if VPP is not gated against API changes.
That is why both CSIT and ci-management parts
have to be merged together.

I will add the missing features
once the new verification is enabled.

Vratko.

[0] https://gerrit.fd.io/r/18356
[1] https://gerrit.fd.io/r/20708
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13633): https://lists.fd.io/g/vpp-dev/message/13633
Mute This Topic: https://lists.fd.io/mt/32561359/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] FD.io - Gerrit 2.16 Changes

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Hi Klement.

Using %private avoids triggering verify jobs,
and you can use UI to set the desired state after that.

Based on my testing [0], "Unmark private" action triggers
verify jobs if and only if "fd.io JJB" is in reviewers list
(same as with "recheck" comment).

From what I can see, Jenkins allows some tweaking
using "Exclude WIP Changes" flag in trigger configuration,
but I do not see any support for that flag in Jenkins Job Builder.

Vratko.

[0] https://gerrit.fd.io/r/c/csit/+/20983

From: vpp-dev@lists.fd.io  On Behalf Of Klement Sekera via 
Lists.Fd.Io
Sent: Wednesday, July 31, 2019 4:29 PM
To: Vanessa Valderrama ; 
ci-management-...@lists.fd.io; cicn-...@lists.fd.io; csit-...@lists.fd.io; 
deb-d...@lists.fd.io; disc...@lists.fd.io; dmm-...@lists.fd.io; 
govpp-...@lists.fd.io; hc2...@lists.fd.io; honeycomb-...@lists.fd.io; 
infra-contai...@lists.fd.io; infra-steer...@lists.fd.io; 
nsh_sfc-...@lists.fd.io; odp4vpp-...@lists.fd.io; one-...@lists.fd.io; 
p4vpp-...@lists.fd.io; pma-tools-...@lists.fd.io; puppet-f...@lists.fd.io; 
rpm_d...@lists.fd.io; sweetcomb-...@lists.fd.io; tldk-...@lists.fd.io; 
trex-...@lists.fd.io; t...@lists.fd.io; vpp-dev@lists.fd.io; 
jvpp-...@lists.fd.io; hicn-...@lists.fd.io
Cc: vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] FD.io - Gerrit 2.16 Changes

Hi Vanessa,

I pushed a couple of changes via your suggested

git push origin HEAD:refs/for/master%wip

and I noticed that on all of these the verify jobs were (attempted to) run. One 
of the advantages of draft changes is that it doesn’t eat resources for 
pointless verify jobs.

Thanks,
Klement

From: vpp-dev@lists.fd.io 
mailto:vpp-dev@lists.fd.io>> On Behalf Of Vanessa 
Valderrama
Sent: Tuesday, July 30, 2019 10:21 PM
To: ci-management-...@lists.fd.io; 
cicn-...@lists.fd.io; 
csit-...@lists.fd.io; 
deb-d...@lists.fd.io; 
disc...@lists.fd.io; 
dmm-...@lists.fd.io; 
govpp-...@lists.fd.io; 
hc2...@lists.fd.io; 
honeycomb-...@lists.fd.io; 
infra-contai...@lists.fd.io; 
infra-steer...@lists.fd.io; 
nsh_sfc-...@lists.fd.io; 
odp4vpp-...@lists.fd.io; 
one-...@lists.fd.io; 
p4vpp-...@lists.fd.io; 
pma-tools-...@lists.fd.io; 
puppet-f...@lists.fd.io; 
rpm_d...@lists.fd.io; 
sweetcomb-...@lists.fd.io; 
tldk-...@lists.fd.io; 
trex-...@lists.fd.io; 
t...@lists.fd.io; 
vpp-dev@lists.fd.io; 
jvpp-...@lists.fd.io; 
hicn-...@lists.fd.io
Subject: [vpp-dev] FD.io - Gerrit 2.16 Changes


Changes that will happen with Gerrit:

1) The 'New UI' for Gerrit will become the default UI

2) The Draft work flow is removed and replaced with 'Work in Progress'
aka 'WIP' and 'Private' workflows. Unfortunately git-review does not
support either of these workflows directly. Utilizing them means either
pushing your changes the manual way for either system or pushing them up
with git-review and then marking the change via the UI into either of
the workflows.
To push a private change you may do so as follows:
git push origin HEAD:refs/for/master%private

To pull it out of private you may do so as follows:
git push origin HEAD:refs/for/master%remove-private

To push a WIP you may do so as follows:
git push origin HEAD:refs/for/master%wip

To mark it ready for review you may do so as follows:
git push origin HEAD:refs/for/master%ready

Once a change is in either private or WIP state it does not switch the
change to a ready state until the current state has been removed.

In both cases, the state can be set via the UI by selecting the tripple
dot menu option on the change and selecting the appropriate option.

To remove WIP state press the 'START REVIEW' button. To remove the
private state you must do so via the menu.

NOTE: We are not moving to Gerrit 3 at this time. That is on the road
map but we need to come to the latest 2.x as we have to do various
migrations that are only available at the 2.16 level before we can move
to Gerrit 3.

Thank you,
Vanessa
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13632): https://lists.fd.io/g/vpp-dev/message/13632
Mute This Topic: https://lists.fd.io/mt/32658320/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=

Re: [vpp-dev] flag day to enable API flag day process

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

[0] is merged, but [1] is stuck due to [2].
Vratko.
[2] https://lists.fd.io/g/vpp-dev/message/13627

From: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco)
Sent: Monday, July 22, 2019 7:14 PM
To: vpp-api-...@lists.fd.io; vpp-dev@lists.fd.io; csit-dev 

Cc: Dave Wallace (dwallac...@gmail.com) ; Ed Kern (ejk) 

Subject: flag day to enable API flag day process

The main API flag document: [0].

I have most of implementation ready,
see [1] and comments there.

The missing parts is e-mail alerting
and postponing main vpp verify jobs
until API check passes (older stable/ VPP branches
are tricky to support in that way).

Please review, I would like to merge very soon.

Keeping CSIT list updated without automated
verify jobs takes some effort,
and it is pointless to add CRC checking only into CSIT
if VPP is not gated against API changes.
That is why both CSIT and ci-management parts
have to be merged together.

I will add the missing features
once the new verification is enabled.

Vratko.

[0] https://gerrit.fd.io/r/18356
[1] https://gerrit.fd.io/r/20708
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13628): https://lists.fd.io/g/vpp-dev/message/13628
Mute This Topic: https://lists.fd.io/mt/32561359/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] [discuss] FD.io Maintenance: 2019-07-30 @ 1700 UTC (10:00am PDT) - 2100 UTC (2:00pm PDT)

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> If you experience any issues please open a ticket at 
> support.linuxfoundation.org.

Executors for ci-management jjb jobs are not available, opened [3].

[3] https://jira.linuxfoundation.org/servicedesk/customer/portal/2/IT-17012

From: disc...@lists.fd.io  On Behalf Of Vanessa Valderrama
Sent: Tuesday, July 30, 2019 8:58 PM
To: t...@lists.fd.io; disc...@lists.fd.io; infra-steer...@lists.fd.io; 
vpp-dev@lists.fd.io; csit-...@lists.fd.io; Ed Kern ; Anton 
Baranov ; Trishan de Lanerolle 

Subject: Re: [discuss] FD.io Maintenance: 2019-07-30 @ 1700 UTC (10:00am PDT) - 
2100 UTC (2:00pm PDT)


Maintenance is complete. All systems are now available. We'll be doing some 
additional testing and monitoring throughout the day. If you experience any 
issues please open a ticket at support.linuxfoundation.org.
Thank you,
Vanessa
On 07/30/2019 12:01 PM, Vanessa Valderrama wrote:

Starting maintenance

On 07/30/2019 11:02 AM, Vanessa Valderrama wrote:

Jenkins has been placed in shutdown mode for maintenance



On 07/29/2019 10:29 AM, Vanessa Valderrama wrote:

Maintenance reminder

On 07/25/2019 04:09 PM, Vanessa Valderrama wrote:

What:

FD.io system maintenance

  *   Jenkins

 *   OS updates
 *   Upgrade from 2.164.3 to 2.176.2

*   https://jenkins.io/changelog-stable/

 *   Update plug-ins
 *   Memory upgrade

  *   Nexus

 *   OS updates

  *   Gerrit

 *   OS updates
 *   Upgrade from 2.14.6 to 2.16

*   https://www.gerritcodereview.com/2.16.html

  *   JIRA

 *   OS updates

  *   Sonar

 *   OS updates

  *   OpenGrok OS updates

When:
2019-07-30 @ 1700 UTC (10:00am PDT) - 2100 UTC (2:00pm PDT)

Impact:

The maintenance will require a reboot of each FD.io system. Jenkins will be 
placed in shutdown mode at 1600 UTC.  Please let us know if there are any jobs 
that cannot be aborted.  The following systems will be impacted:

  *   Jenkins production
  *   Jenkins sandbox
  *   Nexus
  *   Gerrit
  *   JIRA
  *   Sonar
  *   OpenGrok





-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13627): https://lists.fd.io/g/vpp-dev/message/13627
Mute This Topic: https://lists.fd.io/mt/32664689/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] flag day to enable API flag day process

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

The main API flag document: [0].

I have most of implementation ready,
see [1] and comments there.

The missing parts is e-mail alerting
and postponing main vpp verify jobs
until API check passes (older stable/ VPP branches
are tricky to support in that way).

Please review, I would like to merge very soon.

Keeping CSIT list updated without automated
verify jobs takes some effort,
and it is pointless to add CRC checking only into CSIT
if VPP is not gated against API changes.
That is why both CSIT and ci-management parts
have to be merged together.

I will add the missing features
once the new verification is enabled.

Vratko.

[0] https://gerrit.fd.io/r/18356
[1] https://gerrit.fd.io/r/20708
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13538): https://lists.fd.io/g/vpp-dev/message/13538
Mute This Topic: https://lists.fd.io/mt/32561359/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] Ubuntu1604 merge job is broken

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Paul has already commented [0] on this,
hopefully my subject draws more attention.

Basically, packagecould does not like the look
of ubuntu1604 VOM package, leading not only
to missing VOM package, but also other packages
unlucky enough to come after VOM in upload order.

This includes packages such as vpp-plugin-core.
Funnily enough, CSIT tests using ubuntu1604
do not really miss that package.

So, who has the knowledge to fix the VOM package?

Vratko.

[0] https://gerrit.fd.io/r/c/20585/#message-1f1876f6_baf3a8c0

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13526): https://lists.fd.io/g/vpp-dev/message/13526
Mute This Topic: https://lists.fd.io/mt/32524787/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] VPP api python scripts in packages for out-of-tree VPP plugins.

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Adding vpp-dev, as I am not sure who reads vpp-api-dev.

Here [0] is my attempt at a simple solution.

Vratko.

[0] https://gerrit.fd.io/r/20595

From: vpp-api-...@lists.fd.io  On Behalf Of Luca 
Muscariello via Lists.Fd.Io
Sent: Tuesday, 2019-July-09 15:56
To: vpp-api-...@lists.fd.io; hicn-...@lists.fd.io
Subject: [vpp-api-dev] VPP api python scripts in packages for out-of-tree VPP 
plugins.

Hi,


I have a use case to share:

A VPP plugin that is developed out of tree and exposes a binary API,
needs a few python scripts to generate language bindings for C, C++ etc.

vpp/plain/src/vpp-api/vapi/
vapi_c_gen.py
vapi_json_parser.py
vapi_cpp_gen.py

These scripts are not available in any deb/rpm archive currently.

As a countermeasure we have to retrieve the scripts from the web repo.


https://github.com/FDio/hicn/blob/master/hicn-plugin/CMakeLists.txt#L210-L212


It makes the cmake file very ugly.
For out-of-tree plugins this is very common as they can be built using 
vpp-dev/lib only.

Is there any work we can help with to get these scripts into a package, e.g. 
vpp-dev?

Thanks
Luca


_._,_._,_

Links:

You receive all messages sent to this group.

View/Reply Online (#18) | Reply 
To 
Group
 | Reply To 
Sender
 | Mute This Topic | New 
Topic

Your Subscription | Contact 
Group Owner | 
Unsubscribe 
[vrpo...@cisco.com]
_._,_._,_
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13482): https://lists.fd.io/g/vpp-dev/message/13482
Mute This Topic: https://lists.fd.io/mt/32418626/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] new make target

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

I have opened a Jira task [0]
for a new improvement request.

Its description also hints at current plans
for implementation API CRC job, documented in [1].

Vratko.

[0] https://jira.fd.io/browse/VPP-1715
[1] https://gerrit.fd.io/r/18356

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13385): https://lists.fd.io/g/vpp-dev/message/13385
Mute This Topic: https://lists.fd.io/mt/32230319/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

[vpp-dev] FW: [tsc] RESPONSE NEEDED: LFN Staff Survey of the Community

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

I associate communities with mailing lists,
so forwarding (from TSC list) to the two lists
I engage with the most.

Vratko.

From: t...@lists.fd.io  On Behalf Of Brandon Wick
Sent: Wednesday, 2019-June-26 04:37
Subject: [tsc] RESPONSE NEEDED: LFN Staff Survey of the Community

LFN Community,

In order to get a better sense for the community and how we could do our jobs 
better, we've pulled together a short LFN Staff Survey of the Community. We 
believe that the TSCs for each Project are in the best position to determine 
which lists they feel are most appropriate for distribution; so if there are 
additional lists which you believe should be included for your Project, please 
let us know.

The anonymous survey is 21 questions and should take about 5-10 minutes to 
complete. Please complete the survey by July 10th. Thank you for helping us 
improve LFN.

Take Survey Here: https://www.surveymonkey.com/r/PT68D3Q

Best,

Brandon Wick
Senior Integrated Marketing Manager
The Linux Foundation
bw...@linuxfoundation.org
+1.917.282.0960

-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#1039): https://lists.fd.io/g/tsc/message/1039
Mute This Topic: https://lists.fd.io/mt/32211506/870305
Group Owner: tsc+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/tsc/unsub  [vrpo...@cisco.com]
-=-=-=-=-=-=-=-=-=-=-=-
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13384): https://lists.fd.io/g/vpp-dev/message/13384
Mute This Topic: https://lists.fd.io/mt/32229894/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] what is alternative for l2fib_add_fwd_entry in vpp stable/1810?

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

Seems to be removed here [0],
l2fib_add_entry being the replacement.
And of course, use stable/1904 or later.

Vratko.

[0] https://gerrit.fd.io/r/c/14699/4/src/vnet/l2/l2_fib.h

From: vpp-dev@lists.fd.io  On Behalf Of Mostafa Salari
Sent: Sunday, 2019-June-16 10:00
To: vpp-dev@lists.fd.io
Subject: [vpp-dev] what is alternative for l2fib_add_fwd_entry in vpp 
stable/1810?

I used  l2fib_add_fwd_entry  in my vpp stable/1807.
But that is removed in stable/1810!
What function can i use instead of it?
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13382): https://lists.fd.io/g/vpp-dev/message/13382
Mute This Topic: https://lists.fd.io/mt/32207120/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-

Re: [vpp-dev] Plugin Versions

[Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) via Lists.Fd.Io]

> We have plugins which doesn’t expose any APIs

Then why do we track version numbers for such plugins?

Vratko.

From: Damjan Marion 
Sent: Monday, 2019-May-27 13:56
To: Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) 

Cc: Paul Vinciguerra ; vpp-dev@lists.fd.io
Subject: Re: [vpp-dev] Plugin Versions




On 27 May 2019, at 13:54, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at 
Cisco) via Lists.Fd.Io 
mailto:vrpolak=cisco@lists.fd.io>> wrote:

SemVer says:
> Patch version Z (x.y.Z | x > 0) MUST be incremented if only backwards 
> compatible bug fixes are introduced.
> A bug fix is defined as an internal change that fixes incorrect behavior.

That means API version should be bumped even if the change
only affects the implementation (and not API definition itself).

In principle, if there is an internal change
which has not fixed any incorrect behavior,
SemVer says you are not required to bump the patch version.

> multiple version of plugins can provide exact same binary API

That could happen when the plugin undergoes such an internal change
that does not affect behavior at all (at least not positively).
Do you have a realistic example of that?
All I can think of is renaming a variable, or similar silly examples.

Either way, I think it is safe to assume that any plugin version bump
can have at least some behavior consequence visible to user,
so bumping API at the same time is simpler.

Alternatively, we can change the plugin versioning scheme
to acomodate "invisible" bumps.
X.Y.Z.W (Major.Minor.Patch.Internal).

BTW We have plugins which doesn’t expose any APIs, by design….
-=-=-=-=-=-=-=-=-=-=-=-
Links: You receive all messages sent to this group.

View/Reply Online (#13156): https://lists.fd.io/g/vpp-dev/message/13156
Mute This Topic: https://lists.fd.io/mt/31757481/21656
Group Owner: vpp-dev+ow...@lists.fd.io
Unsubscribe: https://lists.fd.io/g/vpp-dev/unsub  [arch...@mail-archive.com]
-=-=-=-=-=-=-=-=-=-=-=-