The BFD failures are caused by timing issues, most of which were
resolved by patches which didn't make it to 17.01. I already worked with
Neal who transplanted the most significant patch and most of the issues
went away. So we would need to backport the test framework patches
for this to work on 17.01.
Also one issue wouldn't go away: There is a test where packets need to
arrive within 0.75-1sec window. An error margin of 0.05 sec is present,
but for Neale on his setup it wasn't enough. His packet was late by
14ms. I would assume that his mac running whatever he has running
(browser, email client etc.) plus a VM (vagrant) with vpp and unit test
framework could easily not meet these deadlines. I'm wondering how
fast/loaded are the jenkins machines and whether the reliability will be
good enough...
Thanks,
Klement
Quoting Maciek Konstantynowicz (mkonstan) (2017-01-04 15:47:25)
> baremetal trusty:
>
> $ lsb_release -a
> No LSB modules are available.
> Distributor ID: Ubuntu
> Description: Ubuntu 14.04.4 LTS
> Release: 14.04
> Codename: trusty
> $ uname -a
> Linux homes1 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 UTC
> 2016 x86_64 x86_64 x86_64 GNU/Linux
>
> -Maciek
>
> > On 4 Jan 2017, at 13:42, Neale Ranns (nranns) <nra...@cisco.com> wrote:
> >
> >
> > Hi Klement, Maciek,
> >
> > What environment are you running the tests in?
> > I get different results. Running in the default vagrant env provided in
> > 17.0. Everything passes apart from:
> >
> > ======================================================================
> > Bidirectional Forwarding Detection (BFD)
> > ======================================================================
> > verify session goes down after inactivity OK
> > hold BFD session up OK
> > large remote RequiredMinRxInterval ERROR [ temp
> > dir used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]
> > bring BFD session up ERROR [ temp
> > dir used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]
> > verify slow periodic control frames while session down ERROR [ temp
> > dir used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]
> > no packets when zero BFD RemoteMinRxInterval ERROR [ temp
> > dir used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]
> >
> > details below. Different runs give different combinations of failures from
> > this BFD list.
> >
> > /neale
> >
> > =====================================================================
> > ERROR: large remote RequiredMinRxInterval
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "/vpp/test/test_bfd.py", line 257, in test_large_required_min_rx
> > self.bfd_session_up()
> > File "/vpp/test/test_bfd.py", line 219, in bfd_session_up
> > p = self.wait_for_bfd_packet()
> > File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet
> > p = self.pg0.wait_for_packet(timeout=timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet
> > self.wait_for_capture_file(timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file
> > raise Exception("Capture file did not appear within timeout")
> > Exception: Capture file did not appear within timeout
> >
> > ======================================================================
> > ERROR: bring BFD session up
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "/vpp/test/test_bfd.py", line 234, in test_session_up
> > self.bfd_session_up()
> > File "/vpp/test/test_bfd.py", line 219, in bfd_session_up
> > p = self.wait_for_bfd_packet()
> > File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet
> > p = self.pg0.wait_for_packet(timeout=timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet
> > self.wait_for_capture_file(timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file
> > raise Exception("Capture file did not appear within timeout")
> > Exception: Capture file did not appear within timeout
> >
> > ======================================================================
> > ERROR: verify slow periodic control frames while session down
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "/vpp/test/test_bfd.py", line 187, in test_slow_timer
> > self.wait_for_bfd_packet()
> > File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet
> > p = self.pg0.wait_for_packet(timeout=timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet
> > self.wait_for_capture_file(timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file
> > raise Exception("Capture file did not appear within timeout")
> > Exception: Capture file did not appear within timeout
> >
> > ======================================================================
> > ERROR: no packets when zero BFD RemoteMinRxInterval
> > ----------------------------------------------------------------------
> > Traceback (most recent call last):
> > File "/vpp/test/test_bfd.py", line 201, in test_zero_remote_min_rx
> > p = self.wait_for_bfd_packet()
> > File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet
> > p = self.pg0.wait_for_packet(timeout=timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet
> > self.wait_for_capture_file(timeout)
> > File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file
> > raise Exception("Capture file did not appear within timeout")
> > Exception: Capture file did not appear within timeout
> >
> > Ran 64 tests in 84.161s
> >
> >
> >
> >
> >
> >
> > On 04/01/2017, 09:45, "csit-dev-boun...@lists.fd.io on behalf of Klement
> > Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)"
> > <csit-dev-boun...@lists.fd.io on behalf of ksek...@cisco.com> wrote:
> >
> > I tried doing a git checkout origin/stable/1701 - is that the correct
> > branch? On that one, the GRE indeed doesn't work, but I don't think it's
> > 'make test' issue, here's a sample from packet trace:
> >
> > ------------------- Start of thread 0 vpp_main -------------------
> > Packet 1
> >
> > 00:00:07:119382: pg-input
> > stream pcap1, 88 bytes
> > current data 0, length 88, free-list 6, trace 0x0
> > IP4: 02:01:00:00:ff:02 -> 02:fe:39:17:74:65
> > GRE: 2.2.2.2 -> 172.16.1.1
> > tos 0x00, ttl 64, length 74, checksum 0xc96f
> > fragment id 0x0001
> > GRE 0x0001
> > 00:00:07:119802: ethernet-input
> > IP4: 02:01:00:00:ff:02 -> 02:fe:39:17:74:65
> > 00:00:07:120048: ip4-input
> > GRE: 2.2.2.2 -> 172.16.1.1
> > tos 0x00, ttl 64, length 74, checksum 0xc96f
> > fragment id 0x0001
> > GRE 0x0001
> > 00:00:07:120113: ip4-lookup
> > fib 0 dpo-idx 5 flow hash: 0x00000000
> > GRE: 2.2.2.2 -> 172.16.1.1
> > tos 0x00, ttl 64, length 74, checksum 0xc96f
> > fragment id 0x0001
> > GRE 0x0001
> > 00:00:07:120404: ip4-local
> > GRE: 2.2.2.2 -> 172.16.1.1
> > tos 0x00, ttl 64, length 74, checksum 0xc96f
> > fragment id 0x0001
> > GRE 0x0001
> > 00:00:07:120495: gre-input
> > GRE: tunnel 0 len 74 src 2.2.2.2 dst 172.16.1.1
> > 00:00:07:120543: error-drop
> > gre-input: unknown protocol
> >
> > all 50 packets printed in the packet trace share the same fate...
> >
> > Thanks,
> > Klement
> >
> > Quoting Maciek Konstantynowicz (mkonstan) (2017-01-03 18:46:19)
> >> // Typed this email before the live discussion on vpp call just now -
> >> still sending it out :)
> >> Hello, After the previous ver of this thread went into a frenzy of emails
> >> and fixes in vpp make test code,
> >> I wanted to re-check the situation.
> >> And as it is a New Year, I’m not lazy anymore, and did run make test in
> >> both vpp branches :)
> >> vpp master branch works, stable/1701 does not..
> >> 1. master
> >> Ran 65 tests in 90.838s
> >> OK (skipped=5)
> >> make[1]: Leaving directory `/home/maciek/src/vpp2/test'
> >> 2. stable/1701
> >> ======================================================================
> >> ERROR: GRE tunnel Tests
> >> Exception: Capture file did not appear within timeout
> >> ======================================================================
> >> ERROR: GRE tunnel L2 Tests
> >> Exception: Capture file did not appear within timeout
> >> ======================================================================
> >> ERROR: MPLS Local Label Binding test
> >> AttributeError: label
> >> ======================================================================
> >> ERROR: MPLS label imposition test
> >> IndexError: Layer [IP] not found
> >> ======================================================================
> >> ERROR: MPLS label swap tests
> >> AttributeError: label
> >> ======================================================================
> >> ERROR: MPLS Tunnel Tests
> >> ----------------------------------------------------------------------
> >> IndexError: Layer [IP] not found
> >> Ran 64 tests in 59.980s
> >> FAILED (errors=6, skipped=5)
> >> make[1]: *** [test] Error 1
> >> make[1]: Leaving directory `/home/maciek/src/vpp2/test'
> >> make: *** [test] Error 2
> >> Fix?
> >> -Maciek
> >>
> >> On 12 Dec 2016, at 16:19, Maciek Konstantynowicz (mkonstan)
> >> <[1]mkons...@cisco.com> wrote:
> >> Hello, Does anyone know if vpp make test is back on track to be ready
> >> to
> >> be used for vpp make verify jobs on a per patch basis?
> >> Being lazy I know - cause I could run it myself :)
> >> -Maciek
> >>
> >> References
> >>
> >> Visible links
> >> 1. mailto:mkons...@cisco.com
> > _______________________________________________
> > csit-dev mailing list
> > csit-...@lists.fd.io
> > https://lists.fd.io/mailman/listinfo/csit-dev
> >
>
_______________________________________________
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
