Re: [vpp-dev] [FD.io Helpdesk #35687] Re: Verify job failure(s)

[Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)]

+1 on this in general

comment regarding "make test": for required python packages, (e.g.
scapy), we install them in "virtualenv", which is python way of having
your own controlled playground. Thus, whatever (python) system packages
are there, "make test" ignores them. This allows us to pick the exact
version in test/Makefile (as we do for scapy, because we apply our
patches every time we install it, so we don't want the upstream
to break our test), so at least for python stuff in "make test", this
can be more or less ignored.

Thanks,
Klement

Quoting Dave Wallace (2017-01-20 04:38:33)
>Ed, Thanh, Vanessa,
> 
>IMHO, updating the ubuntu packages every time a VM is spun up is a bug
>wrt. being able to reproduce some (hopefully rare) build/test issues. 
>Since every VM is potentially running with different versions of OS
>components, when a failure occurs (e.g. in "make test"), then it may be
>necessary to recreate the exact run-time environment in order to reproduce
>the failure. Unless the complete package list is being archived for every
>VM instance that is spun up, this may not be possible. 
> 
>My experience is that those rare cases where a tool or environment issue
>causes a failure, the cost to find the issue is extraordinarily high if
>you do not have the ability to recreate the EXACT build/run-time
>environment.  This is why CSIT does not update OS components in the VM
>initialization scripts and the VM images are built from a specific package
>list instead of pulling the latest versions from the apt repositories.
> 
>My recommendation is that the VM images be updated periodically (weekly or
>whenever a new security update is released) and the package lists archived
>for each VM image version.  Each VM image should also be verified against
>a known good VPP commit version as is done with CSIT branches.  Ideally we
>should build a fully automated continuous deployment model to reduce the
>amount of work to update the VM images to running a Jenkins job to
>build/test/deploy a new VM image from the latest packages versions.
> 
>With that automation in place, this mechanism could be extended for use by
>CSIT as well as "make test", thus ensuring that all of our testing was
>done with the same OS component version.  Ideally, all projects should be
>using the same OS components to ensure that everything is tested in the
>same run-time environment.
> 
>Thanks,
>-daw-
>On 1/19/2017 8:31 PM, Thanh Ha via RT wrote:
> 
>  The issue with the 16.04 Ubuntu image is fixed now (but we may require some 
> additional actions which I'll send to Vanessa to in case this issue comes up 
> again). We fixed this issue tonight by rebuilding ubuntu1604 and deploying 
> the new image.
> 
>  I'm going to close this ticket as resolved and we'll take the additional 
> task to find a way to ensure this doesn't appear again off of this ticket.
> 
>  If you're not interested in the detailed analysis you can stop reading now.
> 
>  For those interested I suspect that the lock issue will appear again 
> (although I could be wrong). The reason I believe so is that our vm init 
> script runs "apt-get update" as an initialization step when the VM boots up 
> at creation time via this script [0]. Ed mentioned that we didn't see this in 
> the past and it only started appear again recently as we deployed another 
> patch to disable Ubuntu's unattended updates.
> 
>  I believe a possible reason we will see this issue appear again due to [0] 
> is because of we switched from using JClouds to OpenStack Jenkins plugins for 
> node spinnup and there's difference in how the init-script is executed 
> depending on which plugin is being used.
> 
>  JClouds Plugin:
> 
>  1) boot vm
>  2) wait for ssh access
>  3) copies init-script into vm via ssh
>  4) executes init-script, and doesn't continue processing until script is 
> complete
>  5) once init-script is complete, passes vm over to job and job starts
> 
>  OpenStack Plugin:
> 
>  1) boot vm and passes init-script in as User Data
>  2) init-script runs inside vm without Jenkins intervention, thus is a 
> non-blocking function
>  3) in parallel jenkins waits for ssh access to vm
>  4) ssh's into vm and passes vm over to job and job starts running
> 
>  In the OpenStack plugin case step 4 can execute while step 2 is still 
> running apt-get update in the background because it was a non-blocking 
> function.
> 
>  A few ideas I have to get around this.
> 
>  a) Allow init-script to continue running apt-get update however have a shell 
> script at the start of Ubuntu jobs that waits for the lock to get released 
> before allowing the job to start
> 
>  b) Remove apt-get update from init-script and make the job run apt-get 
> update at the beginning of it's execution
> 
>  c) Regularly update VMs to ensure that apt-get update always runs quickly
> 
>   Regards,
>  Thanh
> 
>  [0] 
> 

Re: [vpp-dev] [FD.io Helpdesk #35687] Re: Verify job failure(s)

[Thanh Ha]

FWIW in OpenDaylight we don't typically run yum update or apt-get update in
our init-scripts on VM spinup. At the job level we only install
dependencies needed by the build. I'm not sure why fd.io is running
upgrades but it was existing in the script when I looked at it. System
upgrades during VM spinup is not something the OpenDaylight project does at
least.

Regards,
Thanh

On Thu, Jan 19, 2017 at 10:38 PM, Dave Wallace  wrote:

> Ed, Thanh, Vanessa,
>
> IMHO, updating the ubuntu packages every time a VM is spun up is a bug
> wrt. being able to reproduce some (hopefully rare) build/test issues.
> Since every VM is potentially running with different versions of OS
> components, when a failure occurs (e.g. in "make test"), then it may be
> necessary to recreate the exact run-time environment in order to reproduce
> the failure. Unless the complete package list is being archived for every
> VM instance that is spun up, this may not be possible.
>
> My experience is that those rare cases where a tool or environment issue
> causes a failure, the cost to find the issue is extraordinarily high if you
> do not have the ability to recreate the EXACT build/run-time environment.  
> This
> is why CSIT does not update OS components in the VM initialization scripts
> and the VM images are built from a specific package list instead of pulling
> the latest versions from the apt repositories.
>
> My recommendation is that the VM images be updated periodically (weekly or
> whenever a new security update is released) and the package lists archived
> for each VM image version.  Each VM image should also be verified against a
> known good VPP commit version as is done with CSIT branches.  Ideally we
> should build a fully automated continuous deployment model to reduce the
> amount of work to update the VM images to running a Jenkins job to
> build/test/deploy a new VM image from the latest packages versions.
>
> With that automation in place, this mechanism could be extended for use by
> CSIT as well as "make test", thus ensuring that all of our testing was done
> with the same OS component version.  Ideally, all projects should be using
> the same OS components to ensure that everything is tested in the same
> run-time environment.
>
> Thanks,
> -daw-
>
> On 1/19/2017 8:31 PM, Thanh Ha via RT wrote:
>
> The issue with the 16.04 Ubuntu image is fixed now (but we may require some 
> additional actions which I'll send to Vanessa to in case this issue comes up 
> again). We fixed this issue tonight by rebuilding ubuntu1604 and deploying 
> the new image.
>
> I'm going to close this ticket as resolved and we'll take the additional task 
> to find a way to ensure this doesn't appear again off of this ticket.
>
> If you're not interested in the detailed analysis you can stop reading now.
>
> For those interested I suspect that the lock issue will appear again 
> (although I could be wrong). The reason I believe so is that our vm init 
> script runs "apt-get update" as an initialization step when the VM boots up 
> at creation time via this script [0]. Ed mentioned that we didn't see this in 
> the past and it only started appear again recently as we deployed another 
> patch to disable Ubuntu's unattended updates.
>
> I believe a possible reason we will see this issue appear again due to [0] is 
> because of we switched from using JClouds to OpenStack Jenkins plugins for 
> node spinnup and there's difference in how the init-script is executed 
> depending on which plugin is being used.
>
> JClouds Plugin:
>
> 1) boot vm
> 2) wait for ssh access
> 3) copies init-script into vm via ssh
> 4) executes init-script, and doesn't continue processing until script is 
> complete
> 5) once init-script is complete, passes vm over to job and job starts
>
> OpenStack Plugin:
>
> 1) boot vm and passes init-script in as User Data
> 2) init-script runs inside vm without Jenkins intervention, thus is a 
> non-blocking function
> 3) in parallel jenkins waits for ssh access to vm
> 4) ssh's into vm and passes vm over to job and job starts running
>
> In the OpenStack plugin case step 4 can execute while step 2 is still running 
> apt-get update in the background because it was a non-blocking function.
>
> A few ideas I have to get around this.
>
> a) Allow init-script to continue running apt-get update however have a shell 
> script at the start of Ubuntu jobs that waits for the lock to get released 
> before allowing the job to start
>
> b) Remove apt-get update from init-script and make the job run apt-get update 
> at the beginning of it's execution
>
> c) Regularly update VMs to ensure that apt-get update always runs quickly
>
>  Regards,
> Thanh
>
> [0] https://git.fd.io/ci-management/tree/jenkins-scripts/basic_settings.sh#n14
>
>
> On Thu Jan 19 19:23:59 2017, hagbard wrote:
>
> FYI... helpdesk is on it, and its being worked in #fdio-infra on IRC
>
> Ed
>
> On Thu, Jan 19, 2017 at 4:31 PM, Ed Warnicke  
>  wrote:
>
>
> Looping in help desk.

Re: [vpp-dev] [FD.io Helpdesk #35687] Re: Verify job failure(s)

[Thanh Ha]

If we have consensus that apt-get update during vm init is a bad idea then
this patch might be a good quick solution [0].

Regards,
Thanh

[0] https://gerrit.fd.io/r/4797

On Thu, Jan 19, 2017 at 10:47 PM, Ed Warnicke  wrote:

> Thanh,
>
> I'm not quite sure the logic of having it at that particular point
> either.  Something to investigate.
>
> Ed
>
> On Thu, Jan 19, 2017 at 8:44 PM, Thanh Ha 
> wrote:
>
>> FWIW in OpenDaylight we don't typically run yum update or apt-get update
>> in our init-scripts on VM spinup. At the job level we only install
>> dependencies needed by the build. I'm not sure why fd.io is running
>> upgrades but it was existing in the script when I looked at it. System
>> upgrades during VM spinup is not something the OpenDaylight project does at
>> least.
>>
>> Regards,
>> Thanh
>>
>>
>> On Thu, Jan 19, 2017 at 10:38 PM, Dave Wallace 
>> wrote:
>>
>>> Ed, Thanh, Vanessa,
>>>
>>> IMHO, updating the ubuntu packages every time a VM is spun up is a bug
>>> wrt. being able to reproduce some (hopefully rare) build/test issues.
>>> Since every VM is potentially running with different versions of OS
>>> components, when a failure occurs (e.g. in "make test"), then it may be
>>> necessary to recreate the exact run-time environment in order to reproduce
>>> the failure. Unless the complete package list is being archived for every
>>> VM instance that is spun up, this may not be possible.
>>>
>>> My experience is that those rare cases where a tool or environment issue
>>> causes a failure, the cost to find the issue is extraordinarily high if you
>>> do not have the ability to recreate the EXACT build/run-time environment.  
>>> This
>>> is why CSIT does not update OS components in the VM initialization scripts
>>> and the VM images are built from a specific package list instead of pulling
>>> the latest versions from the apt repositories.
>>>
>>> My recommendation is that the VM images be updated periodically (weekly
>>> or whenever a new security update is released) and the package lists
>>> archived for each VM image version.  Each VM image should also be verified
>>> against a known good VPP commit version as is done with CSIT branches.
>>> Ideally we should build a fully automated continuous deployment model to
>>> reduce the amount of work to update the VM images to running a Jenkins job
>>> to build/test/deploy a new VM image from the latest packages versions.
>>>
>>> With that automation in place, this mechanism could be extended for use
>>> by CSIT as well as "make test", thus ensuring that all of our testing was
>>> done with the same OS component version.  Ideally, all projects should be
>>> using the same OS components to ensure that everything is tested in the
>>> same run-time environment.
>>>
>>> Thanks,
>>> -daw-
>>>
>>> On 1/19/2017 8:31 PM, Thanh Ha via RT wrote:
>>>
>>> The issue with the 16.04 Ubuntu image is fixed now (but we may require some 
>>> additional actions which I'll send to Vanessa to in case this issue comes 
>>> up again). We fixed this issue tonight by rebuilding ubuntu1604 and 
>>> deploying the new image.
>>>
>>> I'm going to close this ticket as resolved and we'll take the additional 
>>> task to find a way to ensure this doesn't appear again off of this ticket.
>>>
>>> If you're not interested in the detailed analysis you can stop reading now.
>>>
>>> For those interested I suspect that the lock issue will appear again 
>>> (although I could be wrong). The reason I believe so is that our vm init 
>>> script runs "apt-get update" as an initialization step when the VM boots up 
>>> at creation time via this script [0]. Ed mentioned that we didn't see this 
>>> in the past and it only started appear again recently as we deployed 
>>> another patch to disable Ubuntu's unattended updates.
>>>
>>> I believe a possible reason we will see this issue appear again due to [0] 
>>> is because of we switched from using JClouds to OpenStack Jenkins plugins 
>>> for node spinnup and there's difference in how the init-script is executed 
>>> depending on which plugin is being used.
>>>
>>> JClouds Plugin:
>>>
>>> 1) boot vm
>>> 2) wait for ssh access
>>> 3) copies init-script into vm via ssh
>>> 4) executes init-script, and doesn't continue processing until script is 
>>> complete
>>> 5) once init-script is complete, passes vm over to job and job starts
>>>
>>> OpenStack Plugin:
>>>
>>> 1) boot vm and passes init-script in as User Data
>>> 2) init-script runs inside vm without Jenkins intervention, thus is a 
>>> non-blocking function
>>> 3) in parallel jenkins waits for ssh access to vm
>>> 4) ssh's into vm and passes vm over to job and job starts running
>>>
>>> In the OpenStack plugin case step 4 can execute while step 2 is still 
>>> running apt-get update in the background because it was a non-blocking 
>>> function.
>>>
>>> A few ideas I have to get around this.
>>>
>>> a) Allow init-script to continue running apt-get update however have a 
>>> shell script at the s

Re: [vpp-dev] [csit-dev] sporadically failing functional tests: L2BD and VXLANoIPv4oVLAN

[Matej Klotton -X (mklotton - PANTHEON TECHNOLOGIES at Cisco)]

I marked testcase as non-critical in patches.

https://gerrit.fd.io/r/#/c/4803/
https://gerrit.fd.io/r/#/c/4804/
https://gerrit.fd.io/r/#/c/4805/


From: csit-dev-boun...@lists.fd.io [mailto:csit-dev-boun...@lists.fd.io] On 
Behalf Of Jan Gelety -X (jgelety - PANTHEON TECHNOLOGIES at Cisco)
Sent: 17. januára 2017 16:27
To: Damjan Marion (damarion) 
Cc: csit-...@lists.fd.io; vpp-dev 
Subject: Re: [csit-dev] [vpp-dev] sporadically failing functional tests: L2BD 
and VXLANoIPv4oVLAN

Unfortunately it failed again... :( It seems that packet has been correctly 
processed in DUT1 but didn't reach DUT2:

Packet trace form DUT1:

Packet 1

00:00:13:305429: dpdk-input
  GigabitEthernet0/4/0 rx queue 0
  buffer 0x4e03: current data 0, length 60, free-list 0, totlen-nifb 0, trace 
0x0
  PKT MBUF: port 0, nb_segs 1, pkt_len 60
buf_len 2176, data_len 60, ol_flags 0x0, data_off 128, phys_addr 0x5ad33fc0
packet_type 0x0
  IP4: fa:16:3e:3d:5f:7c -> fa:16:3e:70:a8:34
  ICMP: 192.168.100.1 -> 192.168.100.2
tos 0x00, ttl 64, length 28, checksum 0x318c
fragment id 0x0001
  ICMP echo_request checksum 0xf7ff
00:00:13:305463: ethernet-input
  IP4: fa:16:3e:3d:5f:7c -> fa:16:3e:70:a8:34
00:00:13:305473: l2-input
  l2-input: sw_if_index 1 dst fa:16:3e:70:a8:34 src fa:16:3e:3d:5f:7c
00:00:13:305476: l2-learn
  l2-learn: sw_if_index 1 dst fa:16:3e:70:a8:34 src fa:16:3e:3d:5f:7c bd_index 1
00:00:13:305482: l2-fwd
  l2-fwd:   sw_if_index 1 dst fa:16:3e:70:a8:34 src fa:16:3e:3d:5f:7c bd_index 1
00:00:13:305485: l2-flood
  l2-flood: sw_if_index 1 dst fa:16:3e:70:a8:34 src fa:16:3e:3d:5f:7c bd_index 1
00:00:13:305486: l2-output
  l2-output: sw_if_index 6 dst fa:16:3e:70:a8:34 src fa:16:3e:3d:5f:7c
00:00:13:305508: vxlan4-encap
  VXLAN encap to vxlan_tunnel0 vni 23
00:00:13:305511: ip4-load-balance
  fib 6 dpo-idx 14 flow hash: 0x
  UDP: 172.16.0.1 -> 172.16.0.2
tos 0x00, ttl 254, length 96, checksum 0x6469
fragment id 0x
  UDP: 41828 -> 4789
length 76, checksum 0x
00:00:13:305516: ip4-rewrite
  tx_sw_if_index 5 dpo-idx 2 : ipv4 via 172.16.0.2 GigabitEthernet0/6/0.10: 
IP4: fa:16:3e:60:21:23 -> fa:16:3e:52:55:ca 802.1q vlan 10 flow hash: 0x
  IP4: fa:16:3e:60:21:23 -> fa:16:3e:52:55:ca 802.1q vlan 10
  UDP: 172.16.0.1 -> 172.16.0.2
tos 0x00, ttl 253, length 96, checksum 0x6569
fragment id 0x
  UDP: 41828 -> 4789
length 76, checksum 0x
00:00:13:305521: GigabitEthernet0/6/0-output
  GigabitEthernet0/6/0.10
  IP4: fa:16:3e:60:21:23 -> fa:16:3e:52:55:ca 802.1q vlan 10
  UDP: 172.16.0.1 -> 172.16.0.2
tos 0x00, ttl 253, length 96, checksum 0x6569
fragment id 0x
  UDP: 41828 -> 4789
length 76, checksum 0x
00:00:13:305524: GigabitEthernet0/6/0-tx
  GigabitEthernet0/6/0 tx queue 0
  buffer 0x4e03: current data -54, length 114, free-list 0, totlen-nifb 0, 
trace 0x0
  IP4: fa:16:3e:60:21:23 -> fa:16:3e:52:55:ca 802.1q vlan 10
  UDP: 172.16.0.1 -> 172.16.0.2
tos 0x00, ttl 253, length 96, checksum 0x6569
fragment id 0x
  UDP: 41828 -> 4789
length 76, checksum 0x

Packet trace form DUT2:

No packets in trace buffer


Anyway, it has passed with vpp build 17.04-rc0~64-gf952692~b1689_amd64 as well 
as vpp build 17.01-rc2~5-g257d5e2~b17_amd64.

I forced recheck with newer vpp build from the master: 
17.04-rc0~105-g5a3a6c0~b1730_amd64 for sure.

From: csit-dev-boun...@lists.fd.io 
[mailto:csit-dev-boun...@lists.fd.io] On Behalf Of Jan Gelety -X (jgelety - 
PANTHEON TECHNOLOGIES at Cisco)
Sent: Tuesday, January 17, 2017 15:16
To: Damjan Marion (damarion) mailto:damar...@cisco.com>>
Cc: csit-...@lists.fd.io; vpp-dev 
mailto:vpp-dev@lists.fd.io>>
Subject: Re: [csit-dev] [vpp-dev] sporadically failing functional tests: L2BD 
and VXLANoIPv4oVLAN

Hello Damjan,

Needed fix has been merged to master and cherry-picked to csit branch 
oper-170108 that is currently used to verify vpp patch.

I did recheck on your commit https://gerrit.fd.io/r/#/c/4613/14 - it should 
pass now. We will keep our eyes on it.

Regards,
Jan

From: vpp-dev-boun...@lists.fd.io 
[mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Matej Klotton -X (mklotton - 
PANTHEON TECHNOLOGIES at Cisco)
Sent: Tuesday, January 17, 2017 10:29
To: Damjan Marion (damarion) mailto:damar...@cisco.com>>; 
csit-...@lists.fd.io
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] [csit-dev] sporadically failing functional tests: L2BD 
and VXLANoIPv4oVLAN

I'll fix the test cases.

From: vpp-dev-boun...@lists.fd.io 
[mailto:vpp-dev-boun...@lists.fd.io] On Behalf Of Damjan Marion (damarion)
Sent: 16. januára 2017 22:39
To: csit-...@lists.fd.io
Cc: vpp-dev mailto:vpp-dev@lists.fd.io>>
Subject: Re: [vpp-dev] [csit-dev] sporadically failing functional tests: L2BD 
and VXLANoIPv4oVLAN


Can we please disabl

Re: [vpp-dev] [FD.io Helpdesk #35687] Re: Verify job failure(s)

[Dave Wallace]

Klement,

Good to know.  Virtualenv allows deterministic specification of the 
python infrastructure/tools, however, this does not cover the VPP 
build/run-time environment.


Thanks,
-daw-

On 1/20/2017 4:15 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES 
at Cisco) wrote:

+1 on this in general

comment regarding "make test": for required python packages, (e.g.
scapy), we install them in "virtualenv", which is python way of having
your own controlled playground. Thus, whatever (python) system packages
are there, "make test" ignores them. This allows us to pick the exact
version in test/Makefile (as we do for scapy, because we apply our
patches every time we install it, so we don't want the upstream
to break our test), so at least for python stuff in "make test", this
can be more or less ignored.

Thanks,
Klement

Quoting Dave Wallace (2017-01-20 04:38:33)

Ed, Thanh, Vanessa,

IMHO, updating the ubuntu packages every time a VM is spun up is a bug
wrt. being able to reproduce some (hopefully rare) build/test issues.
Since every VM is potentially running with different versions of OS
components, when a failure occurs (e.g. in "make test"), then it may be
necessary to recreate the exact run-time environment in order to reproduce
the failure. Unless the complete package list is being archived for every
VM instance that is spun up, this may not be possible.

My experience is that those rare cases where a tool or environment issue
causes a failure, the cost to find the issue is extraordinarily high if
you do not have the ability to recreate the EXACT build/run-time
environment.  This is why CSIT does not update OS components in the VM
initialization scripts and the VM images are built from a specific package
list instead of pulling the latest versions from the apt repositories.

My recommendation is that the VM images be updated periodically (weekly or
whenever a new security update is released) and the package lists archived
for each VM image version.  Each VM image should also be verified against
a known good VPP commit version as is done with CSIT branches.  Ideally we
should build a fully automated continuous deployment model to reduce the
amount of work to update the VM images to running a Jenkins job to
build/test/deploy a new VM image from the latest packages versions.

With that automation in place, this mechanism could be extended for use by
CSIT as well as "make test", thus ensuring that all of our testing was
done with the same OS component version.  Ideally, all projects should be
using the same OS components to ensure that everything is tested in the
same run-time environment.

Thanks,
-daw-
On 1/19/2017 8:31 PM, Thanh Ha via RT wrote:

  The issue with the 16.04 Ubuntu image is fixed now (but we may require some 
additional actions which I'll send to Vanessa to in case this issue comes up 
again). We fixed this issue tonight by rebuilding ubuntu1604 and deploying the 
new image.

  I'm going to close this ticket as resolved and we'll take the additional task 
to find a way to ensure this doesn't appear again off of this ticket.

  If you're not interested in the detailed analysis you can stop reading now.

  For those interested I suspect that the lock issue will appear again (although I could 
be wrong). The reason I believe so is that our vm init script runs "apt-get 
update" as an initialization step when the VM boots up at creation time via this 
script [0]. Ed mentioned that we didn't see this in the past and it only started appear 
again recently as we deployed another patch to disable Ubuntu's unattended updates.

  I believe a possible reason we will see this issue appear again due to [0] is 
because of we switched from using JClouds to OpenStack Jenkins plugins for node 
spinnup and there's difference in how the init-script is executed depending on 
which plugin is being used.

  JClouds Plugin:

  1) boot vm
  2) wait for ssh access
  3) copies init-script into vm via ssh
  4) executes init-script, and doesn't continue processing until script is 
complete
  5) once init-script is complete, passes vm over to job and job starts

  OpenStack Plugin:

  1) boot vm and passes init-script in as User Data
  2) init-script runs inside vm without Jenkins intervention, thus is a 
non-blocking function
  3) in parallel jenkins waits for ssh access to vm
  4) ssh's into vm and passes vm over to job and job starts running

  In the OpenStack plugin case step 4 can execute while step 2 is still running 
apt-get update in the background because it was a non-blocking function.

  A few ideas I have to get around this.

  a) Allow init-script to continue running apt-get update however have a shell 
script at the start of Ubuntu jobs that waits for the lock to get released 
before allowing the job to start

  b) Remove apt-get update from init-script and make the job run apt-get upd

[vpp-dev] Latest master building on Ubuntu 16.04

[Keith Burns]

Did a fresh clone and "make bootstrap" returns:

 Building dpdk in
/home/alagalah/git/work/fdio/vpp/build-root/build-vpp_debug-native/dpdk 
make[2]: Entering directory '/home/alagalah/git/work/fdio/vpp/dpdk'
/bin/bash: rpm: command not found
/bin/bash: rpm: command not found
/bin/bash: rpm: command not found
/bin/bash: rpm: command not found
==
Building DPDK from source. Consider installing development
package by invoking 'make dpdk-install-dev' from the
top level directory
==
make config
make[3]: Entering directory '/home/alagalah/git/work/fdio/vpp/dpdk'
make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make
rule.
/bin/bash: rpm: command not found
/bin/bash: rpm: command not found
/bin/bash: rpm: command not found
/bin/bash: rpm: command not found

Not sure why RPM is popping up in there
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] VPP 17.01 Released

[Damjan Marion (damarion)]

The VPP 17.01 release i up. Many thanks to all contributors and testers
and specially Ed for helping to roll this release over the finish line.

.rpm and .deb packages are uploaded to the nexus server.

New features in the VPP 17.01:

- Integrated November 2016 DPDK release
- Complete rework of Forwarding Information Base (FIB)
- Performance Improvements
  - Improvements in DPDK input and output nodes
  - Improvements in L2 path
  - Improvmeents in IPv4 lookup node
- Feature Arcs Improvements
  - Consolidation of the code
  - New feature arcs
- device-input
- interface-output
- DPDK Cryptodev Support
  - Software and Hardware Crypto Support
- DPDK HQoS support
- Simple Port Analyzer (SPAN)
- Bidirectional Forwarding Detection
  - Basic implementation
- IPFIX Improvements
- L2 GRE over IPSec tunnels
- Link Layer Discovery Protocol (LLDP)
- Vhost-user Improvements
  - Performance Improvements
  - Multi-queue
  - Reconnect
- LISP Enhancements
  - Source/Dest control plane support
  - L2 over LISP and GRE
  - Map-Register/Map-Notify/RLOC-probing support
  - L2 API improvements, overall code hardening
- Plugins:
  - New: ACL
  - New: Flow per Packet
  - Improved: SNAT
- Multi-threading
- Flow export
- Doxygen Enhancements
- Luajit API bindings
- API Refactoring
  - file split
  - message signatures
- Python and Scapy based unit testing infrastructure
  - Infrastructure
  - Various tests
- Packet Generator improvements
- TUN/TAP jumbo frames support
- Other various bug fixes and improvements

Thanks,

Damjan
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] Latest master building on Ubuntu 16.04

[Damjan Marion]

> On 20 Jan 2017, at 20:31, Keith Burns  wrote:
> 
> Did a fresh clone and "make bootstrap" returns:
> 
>  Building dpdk in 
> /home/alagalah/git/work/fdio/vpp/build-root/build-vpp_debug-native/dpdk 
> make[2]: Entering directory '/home/alagalah/git/work/fdio/vpp/dpdk'
> /bin/bash: rpm: command not found
> /bin/bash: rpm: command not found
> /bin/bash: rpm: command not found
> /bin/bash: rpm: command not found
> ==
> Building DPDK from source. Consider installing development
> package by invoking 'make dpdk-install-dev' from the
> top level directory
> ==
> make config
> make[3]: Entering directory '/home/alagalah/git/work/fdio/vpp/dpdk'
> make[3]: warning: jobserver unavailable: using -j1.  Add '+' to parent make 
> rule.
> /bin/bash: rpm: command not found
> /bin/bash: rpm: command not found
> /bin/bash: rpm: command not found
> /bin/bash: rpm: command not found
> 
> Not sure why RPM is popping up in there….

Mea culpa. It is cosmetic, I will fix it...

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [csit-dev] VPP 17.01 Released

[Florin Coras]

Awesome! 

Thank you for the hard work Damjan and congratulations to the entire community!

Florin

> On Jan 20, 2017, at 2:51 PM, Damjan Marion (damarion)  
> wrote:
> 
> 
> The VPP 17.01 release i up. Many thanks to all contributors and testers
> and specially Ed for helping to roll this release over the finish line.
> 
> .rpm and .deb packages are uploaded to the nexus server.
> 
> New features in the VPP 17.01:
> 
> - Integrated November 2016 DPDK release
> - Complete rework of Forwarding Information Base (FIB)
> - Performance Improvements
>  - Improvements in DPDK input and output nodes
>  - Improvements in L2 path
>  - Improvmeents in IPv4 lookup node
> - Feature Arcs Improvements
>  - Consolidation of the code
>  - New feature arcs
>- device-input
>- interface-output
> - DPDK Cryptodev Support
>  - Software and Hardware Crypto Support
> - DPDK HQoS support
> - Simple Port Analyzer (SPAN)
> - Bidirectional Forwarding Detection
>  - Basic implementation
> - IPFIX Improvements
> - L2 GRE over IPSec tunnels
> - Link Layer Discovery Protocol (LLDP)
> - Vhost-user Improvements
>  - Performance Improvements
>  - Multi-queue
>  - Reconnect
> - LISP Enhancements
>  - Source/Dest control plane support
>  - L2 over LISP and GRE
>  - Map-Register/Map-Notify/RLOC-probing support
>  - L2 API improvements, overall code hardening
> - Plugins:
>  - New: ACL
>  - New: Flow per Packet
>  - Improved: SNAT
>- Multi-threading
>- Flow export
> - Doxygen Enhancements
> - Luajit API bindings
> - API Refactoring
>  - file split
>  - message signatures
> - Python and Scapy based unit testing infrastructure
>  - Infrastructure
>  - Various tests
> - Packet Generator improvements
> - TUN/TAP jumbo frames support
> - Other various bug fixes and improvements
> 
> Thanks,
> 
> Damjan
> ___
> csit-dev mailing list
> csit-...@lists.fd.io
> https://lists.fd.io/mailman/listinfo/csit-dev

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [discuss] VPP 17.01 Released

[Dave Barach (dbarach)]

Hey Damjan,

Congrats to you and the entire community for all the hard work which went into 
the 17.01 release!

Dave

From: Damjan Marion (damarion)
Sent: Friday, January 20, 2017 6:38 PM
To: vpp-internal(mailer list) 
Subject: Fwd: [discuss] VPP 17.01 Released




Begin forwarded message:

From: "Damjan Marion (damarion)" mailto:damar...@cisco.com>>
Subject: [discuss] VPP 17.01 Released
Date: 20 January 2017 at 23:51:45 GMT+1
To: vpp-dev mailto:vpp-dev@lists.fd.io>>, 
"t...@lists.fd.io" 
mailto:t...@lists.fd.io>>, 
"disc...@lists.fd.io" 
mailto:disc...@lists.fd.io>>, 
"csit-...@lists.fd.io" 
mailto:csit-...@lists.fd.io>>, 
"honeycomb-...@lists.fd.io" 
mailto:honeycomb-...@lists.fd.io>>, 
"nsh_sfc-...@lists.fd.io" 
mailto:nsh_sfc-...@lists.fd.io>>


The VPP 17.01 release i up. Many thanks to all contributors and testers
and specially Ed for helping to roll this release over the finish line.

.rpm and .deb packages are uploaded to the nexus server.

New features in the VPP 17.01:

- Integrated November 2016 DPDK release
- Complete rework of Forwarding Information Base (FIB)
- Performance Improvements
 - Improvements in DPDK input and output nodes
 - Improvements in L2 path
 - Improvmeents in IPv4 lookup node
- Feature Arcs Improvements
 - Consolidation of the code
 - New feature arcs
   - device-input
   - interface-output
- DPDK Cryptodev Support
 - Software and Hardware Crypto Support
- DPDK HQoS support
- Simple Port Analyzer (SPAN)
- Bidirectional Forwarding Detection
 - Basic implementation
- IPFIX Improvements
- L2 GRE over IPSec tunnels
- Link Layer Discovery Protocol (LLDP)
- Vhost-user Improvements
 - Performance Improvements
 - Multi-queue
 - Reconnect
- LISP Enhancements
 - Source/Dest control plane support
 - L2 over LISP and GRE
 - Map-Register/Map-Notify/RLOC-probing support
 - L2 API improvements, overall code hardening
- Plugins:
 - New: ACL
 - New: Flow per Packet
 - Improved: SNAT
   - Multi-threading
   - Flow export
- Doxygen Enhancements
- Luajit API bindings
- API Refactoring
 - file split
 - message signatures
- Python and Scapy based unit testing infrastructure
 - Infrastructure
 - Various tests
- Packet Generator improvements
- TUN/TAP jumbo frames support
- Other various bug fixes and improvements

Thanks,

Damjan
___
discuss mailing list
disc...@lists.fd.io
https://lists.fd.io/mailman/listinfo/discuss

___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [discuss] VPP 17.01 Released

[Dave Wallace]

Kudos Damian and the entire fd.io community !

-daw-

On Jan 20, 2017 6:58 PM, "Dave Barach (dbarach)"  wrote:

Hey Damjan,



Congrats to you and the entire community for all the hard work which went
into the 17.01 release!



Dave



*From:* Damjan Marion (damarion)
*Sent:* Friday, January 20, 2017 6:38 PM
*To:* vpp-internal(mailer list) 
*Subject:* Fwd: [discuss] VPP 17.01 Released







Begin forwarded message:



*From: *"Damjan Marion (damarion)" 

*Subject: [discuss] VPP 17.01 Released*

*Date: *20 January 2017 at 23:51:45 GMT+1

*To: *vpp-dev , "t...@lists.fd.io" , "
disc...@lists.fd.io" , "csit-...@lists.fd.io" <
csit-...@lists.fd.io>, "honeycomb-...@lists.fd.io" <
honeycomb-...@lists.fd.io>, "nsh_sfc-...@lists.fd.io" <
nsh_sfc-...@lists.fd.io>




The VPP 17.01 release i up. Many thanks to all contributors and testers
and specially Ed for helping to roll this release over the finish line.

.rpm and .deb packages are uploaded to the nexus server.

New features in the VPP 17.01:

- Integrated November 2016 DPDK release
- Complete rework of Forwarding Information Base (FIB)
- Performance Improvements
 - Improvements in DPDK input and output nodes
 - Improvements in L2 path
 - Improvmeents in IPv4 lookup node
- Feature Arcs Improvements
 - Consolidation of the code
 - New feature arcs
   - device-input
   - interface-output
- DPDK Cryptodev Support
 - Software and Hardware Crypto Support
- DPDK HQoS support
- Simple Port Analyzer (SPAN)
- Bidirectional Forwarding Detection
 - Basic implementation
- IPFIX Improvements
- L2 GRE over IPSec tunnels
- Link Layer Discovery Protocol (LLDP)
- Vhost-user Improvements
 - Performance Improvements
 - Multi-queue
 - Reconnect
- LISP Enhancements
 - Source/Dest control plane support
 - L2 over LISP and GRE
 - Map-Register/Map-Notify/RLOC-probing support
 - L2 API improvements, overall code hardening
- Plugins:
 - New: ACL
 - New: Flow per Packet
 - Improved: SNAT
   - Multi-threading
   - Flow export
- Doxygen Enhancements
- Luajit API bindings
- API Refactoring
 - file split
 - message signatures
- Python and Scapy based unit testing infrastructure
 - Infrastructure
 - Various tests
- Packet Generator improvements
- TUN/TAP jumbo frames support
- Other various bug fixes and improvements

Thanks,

Damjan
___
discuss mailing list
disc...@lists.fd.io
https://lists.fd.io/mailman/listinfo/discuss



___
discuss mailing list
disc...@lists.fd.io
https://lists.fd.io/mailman/listinfo/discuss
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

[vpp-dev] Tuple Classify

[yug...@telincn.com]

Hi, all
Is there any ways to redirect specified five tuples packet to a certain node?

Regards,
Ewan



yug...@telincn.com
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev

Re: [vpp-dev] [discuss] VPP 17.01 Released

[yug...@telincn.com]

+1  



yug...@telincn.com
 
From: Dave Barach (dbarach)
Date: 2017-01-21 07:58
To: Damjan Marion (damarion); t...@lists.fd.io; csit-...@lists.fd.io; 
honeycomb-...@lists.fd.io; disc...@lists.fd.io; vpp-dev; nsh_sfc-...@lists.fd.io
Subject: Re: [vpp-dev] [discuss] VPP 17.01 Released
Hey Damjan,
 
Congrats to you and the entire community for all the hard work which went into 
the 17.01 release!  
 
Dave
 
From: Damjan Marion (damarion) 
Sent: Friday, January 20, 2017 6:38 PM
To: vpp-internal(mailer list) 
Subject: Fwd: [discuss] VPP 17.01 Released
 
 


Begin forwarded message:
 
From: "Damjan Marion (damarion)" 
Subject: [discuss] VPP 17.01 Released
Date: 20 January 2017 at 23:51:45 GMT+1
To: vpp-dev , "t...@lists.fd.io" , 
"disc...@lists.fd.io" , "csit-...@lists.fd.io" 
, "honeycomb-...@lists.fd.io" 
, "nsh_sfc-...@lists.fd.io" 
 

The VPP 17.01 release i up. Many thanks to all contributors and testers
and specially Ed for helping to roll this release over the finish line.

.rpm and .deb packages are uploaded to the nexus server.

New features in the VPP 17.01:

- Integrated November 2016 DPDK release
- Complete rework of Forwarding Information Base (FIB)
- Performance Improvements
 - Improvements in DPDK input and output nodes
 - Improvements in L2 path
 - Improvmeents in IPv4 lookup node
- Feature Arcs Improvements
 - Consolidation of the code
 - New feature arcs
   - device-input
   - interface-output
- DPDK Cryptodev Support
 - Software and Hardware Crypto Support
- DPDK HQoS support
- Simple Port Analyzer (SPAN)
- Bidirectional Forwarding Detection
 - Basic implementation
- IPFIX Improvements
- L2 GRE over IPSec tunnels
- Link Layer Discovery Protocol (LLDP)
- Vhost-user Improvements
 - Performance Improvements
 - Multi-queue
 - Reconnect
- LISP Enhancements
 - Source/Dest control plane support
 - L2 over LISP and GRE
 - Map-Register/Map-Notify/RLOC-probing support
 - L2 API improvements, overall code hardening
- Plugins:
 - New: ACL
 - New: Flow per Packet
 - Improved: SNAT
   - Multi-threading
   - Flow export
- Doxygen Enhancements
- Luajit API bindings
- API Refactoring
 - file split
 - message signatures
- Python and Scapy based unit testing infrastructure
 - Infrastructure
 - Various tests
- Packet Generator improvements
- TUN/TAP jumbo frames support
- Other various bug fixes and improvements

Thanks,

Damjan
___
discuss mailing list
disc...@lists.fd.io
https://lists.fd.io/mailman/listinfo/discuss
 
___
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev