Klement/Neale,

I finally got back to debugging this issue. After doing a make test-wipe, then make test. The patched python files and their corresponding .pyc files had the same timestamp with a 1-second resolution. I deleted all .pyc files and then all of the tests passed. After doing some research, I confirmed that python uses a 1-second resolution when checking whether to recompile the *.pyc file.

I have verified that the following patch resolves this issue both on Ubuntu 16.04 in Vagrant and Ubuntu 16.10 on bare metal:

https://gerrit.fd.io/r/#/c/4896/

Thanks,
-daw-

On 01/05/2017 03:54 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco) wrote:
Dave, there is no sign of test framework installing/patching scapy in
the log you provided.

Could you please do in the same workspace

make test-wipe

and then run the make test again? I don't think it'll actually help us,
because there should be no way for a python inside virtualenv to get its
hands on outside libs (unless there is a bug of course), but let's see
what happens...

Thanks,
Klement

Quoting Dave Wallace (2017-01-04 19:19:04)
    Neale,

    I could not find scapy installed in the normal path (i.e. "which scapy"),
    but I'm not sure if it is directly executable.  You can find the "V=1
    TEST=mpls make test" output on
    vagrant+virtualbox:puppetlabs/ubuntu-16.04.64-nocm in this pastebin doc:
    [1]http://pastebin.com/uGAUREhK

    Thanks,
    -daw-

    On 1/4/2017 10:59 AM, Neale Ranns (nranns) wrote:

      Hi,

      The reason the MPLS tests are failing is because scapy cannot decode an
      MPLS label stack (output from some .show() instrumentation);

      <class 'scapy.layers.l2.Ether'>

      ###[ Ethernet ]###

        dst       = 02:01:00:00:ff:02

        src       = 02:fe:e5:05:0e:13

        type      = 0x8847

      ###[ MPLS ]###

           label     = 33L

           cos       = 0L

           s         = 0L       <<<<<< NON End-of-stack

           ttl       = 254

      ###[ Padding ]###

              load      = '\x00\x061\xffE\x00\x00#\x00\x01\x00\x00@\x11
      \xa6\xac\x10\x01\x01\xac\x10\x01\x02\x04\xd2\x04\xd2\x00\x0f\xd0\x92257
      1 1'

      we patch scapy for this purpose, see;

        <ROOT>/test/patches/scapy-2.3.3/mpls.py.patch

      on my vagrant 14.04 there is no other scapy installation, this patch
      applies fine and all MPLS tests pass.

      On a UCS 14.04, with another scapy installation:

        /usr/local/lib/python2.7/dist-packages/scapy/contrib/mpls.py

      the MPLS test fail.

      On a UCS 16.04 no scpay installation, tests pass.

      Do you have scapy installed in the machines on which the tests
      fail/pass? Or am I barking up the wrong virtualenv tree J

      Thanks,

      neale

        From: Dave Wallace [2]<dwallac...@gmail.com>
        Date: Wednesday, 4 January 2017 at 15:38
        To: "Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)"
        [3]<ksek...@cisco.com>, "Maciek Konstantynowicz (mkonstan)"
        [4]<mkons...@cisco.com>, "Neale Ranns (nranns)" [5]<nra...@cisco.com>
        Cc: [6]"csit-...@lists.fd.io" [7]<csit-...@lists.fd.io>,
        [8]"vpp-dev@lists.fd.io" [9]<vpp-dev@lists.fd.io>
        Subject: Re: [csit-dev] [vpp-dev] vpp make test for verify - are we
        there yet ? [awoken]

        Klement,

        I'm in the process of modifying the base Vagrantfile/build scripts to
        replace building the VPP native package and installing it in the VM to
        simply running "make test" which takes much less time.  All of the
        test runs that I have performed are clean builds on a virgin git clone
        repo.

        That being said, I'll give the "git clean -dfX */" a try and see if I
        get different results on the 2nd pass and report back.

        Thanks,
        -daw-

        On 1/4/17 10:23 AM, Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES
        at Cisco) wrote:

  Hi Dave,

  could you please try running git clean -dfX */ in a workspace, where you see

  the failures, and then givet it another try? For me this seems to clear issues

  up. Damjans recent changes might have caused (quoting Damjan) "major

  disturbances in the force", leaving build artifacts behind.

  Just be sure to stash/commit any changes you have in the workspace since

  the git clean nukes everything unknown from the ws.

  Thanks,

  Klement

  Quoting Dave Wallace (2017-01-04 16:12:38)

     As I mentioned on the VPP call yesterday, I'm seeing what appears to be

     the same GRE test failures on Ubuntu 16.10 (bare metal).  I'm also seeing

     the same failures on vagrant+virtualbox with

     puppetlabs/ubuntu-16.04.64-nocm. I also am getting failures on 3 out of 4

     MPLS tests on both environments.

     However, all tests pass on vagrant+virtualbox with

     puppetlabs/ubuntu-14.04.64-nocm.

     Thanks,

     -daw-

     On 1/4/17 9:47 AM, Maciek Konstantynowicz (mkonstan) wrote:

   baremetal trusty:

   $ lsb_release -a

   No LSB modules are available.

   Distributor ID: Ubuntu

   Description:    Ubuntu 14.04.4 LTS

   Release:    14.04

   Codename:   trusty

   $ uname -a

   Linux homes1 3.16.0-77-generic #99~14.04.1-Ubuntu SMP Tue Jun 28 19:17:10 
UTC 2016 x86_64 x86_64 x86_64 GNU/Linux

   -Maciek

   On 4 Jan 2017, at 13:42, Neale Ranns (nranns) [1][10]<nra...@cisco.com> 
wrote:

   Hi Klement, Maciek,

   What environment are you running the tests in?

   I get different results. Running in the default vagrant env provided in 
17.0. Everything passes apart from:

   ======================================================================

   Bidirectional Forwarding Detection (BFD)

   ======================================================================

   verify session goes down after inactivity                   OK

   hold BFD session up                                         OK

   large remote RequiredMinRxInterval                          ERROR [ temp dir 
used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]

   bring BFD session up                                        ERROR [ temp dir 
used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]

   verify slow periodic control frames while session down      ERROR [ temp dir 
used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]

   no packets when zero BFD RemoteMinRxInterval                ERROR [ temp dir 
used by test case: /tmp/vpp-unittest-BFDTestCase-F7O6hZ ]

   details below. Different runs give different combinations of failures from 
this BFD list.

   /neale

   =====================================================================

   ERROR: large remote RequiredMinRxInterval

   ----------------------------------------------------------------------

   Traceback (most recent call last):

    File "/vpp/test/test_bfd.py", line 257, in test_large_required_min_rx

      self.bfd_session_up()

    File "/vpp/test/test_bfd.py", line 219, in bfd_session_up

      p = self.wait_for_bfd_packet()

    File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet

      p = self.pg0.wait_for_packet(timeout=timeout)

    File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet

      self.wait_for_capture_file(timeout)

    File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file

      raise Exception("Capture file did not appear within timeout")

   Exception: Capture file did not appear within timeout

   ======================================================================

   ERROR: bring BFD session up

   ----------------------------------------------------------------------

   Traceback (most recent call last):

    File "/vpp/test/test_bfd.py", line 234, in test_session_up

      self.bfd_session_up()

    File "/vpp/test/test_bfd.py", line 219, in bfd_session_up

      p = self.wait_for_bfd_packet()

    File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet

      p = self.pg0.wait_for_packet(timeout=timeout)

    File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet

      self.wait_for_capture_file(timeout)

    File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file

      raise Exception("Capture file did not appear within timeout")

   Exception: Capture file did not appear within timeout

   ======================================================================

   ERROR: verify slow periodic control frames while session down

   ----------------------------------------------------------------------

   Traceback (most recent call last):

    File "/vpp/test/test_bfd.py", line 187, in test_slow_timer

      self.wait_for_bfd_packet()

    File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet

      p = self.pg0.wait_for_packet(timeout=timeout)

    File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet

      self.wait_for_capture_file(timeout)

    File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file

      raise Exception("Capture file did not appear within timeout")

   Exception: Capture file did not appear within timeout

   ======================================================================

   ERROR: no packets when zero BFD RemoteMinRxInterval

   ----------------------------------------------------------------------

   Traceback (most recent call last):

    File "/vpp/test/test_bfd.py", line 201, in test_zero_remote_min_rx

      p = self.wait_for_bfd_packet()

    File "/vpp/test/test_bfd.py", line 171, in wait_for_bfd_packet

      p = self.pg0.wait_for_packet(timeout=timeout)

    File "/vpp/test/vpp_pg_interface.py", line 217, in wait_for_packet

      self.wait_for_capture_file(timeout)

    File "/vpp/test/vpp_pg_interface.py", line 204, in wait_for_capture_file

      raise Exception("Capture file did not appear within timeout")

   Exception: Capture file did not appear within timeout

   Ran 64 tests in 84.161s

   On 04/01/2017, 09:45, [2][11]"csit-dev-boun...@lists.fd.io on behalf of Klement Sekera 
-X (ksekera - PANTHEON TECHNOLOGIES at Cisco)" [3][12]<csit-dev-boun...@lists.fd.io on 
behalf of ksek...@cisco.com> wrote:

      I tried doing a git checkout origin/stable/1701 - is that the correct

      branch? On that one, the GRE indeed doesn't work, but I don't think it's

      'make test' issue, here's a sample from packet trace:

      ------------------- Start of thread 0 vpp_main -------------------

      Packet 1

      00:00:07:119382: pg-input

        stream pcap1, 88 bytes

        current data 0, length 88, free-list 6, trace 0x0

        IP4: 02:01:00:00:ff:02 -> 02:fe:39:17:74:65

        GRE: 2.2.2.2 -> 172.16.1.1

          tos 0x00, ttl 64, length 74, checksum 0xc96f

          fragment id 0x0001

        GRE 0x0001

      00:00:07:119802: ethernet-input

        IP4: 02:01:00:00:ff:02 -> 02:fe:39:17:74:65

      00:00:07:120048: ip4-input

        GRE: 2.2.2.2 -> 172.16.1.1

          tos 0x00, ttl 64, length 74, checksum 0xc96f

          fragment id 0x0001

        GRE 0x0001

      00:00:07:120113: ip4-lookup

        fib 0 dpo-idx 5 flow hash: 0x00000000

        GRE: 2.2.2.2 -> 172.16.1.1

          tos 0x00, ttl 64, length 74, checksum 0xc96f

          fragment id 0x0001

        GRE 0x0001

      00:00:07:120404: ip4-local

          GRE: 2.2.2.2 -> 172.16.1.1

            tos 0x00, ttl 64, length 74, checksum 0xc96f

            fragment id 0x0001

          GRE 0x0001

      00:00:07:120495: gre-input

        GRE: tunnel 0 len 74 src 2.2.2.2 dst 172.16.1.1

      00:00:07:120543: error-drop

        gre-input: unknown protocol

      all 50 packets printed in the packet trace share the same fate...

      Thanks,

      Klement

      Quoting Maciek Konstantynowicz (mkonstan) (2017-01-03 18:46:19)

     // Typed this email before the live discussion on vpp call just now -

     still sending it out :)

     Hello, After the previous ver of this thread went into a frenzy of emails

     and fixes in vpp make test code,

     I wanted to re-check the situation.

     And as it is a New Year, I’m not lazy anymore, and did run make test in

     both vpp branches :)

     vpp master branch works, stable/1701 does not..

     1. master

         Ran 65 tests in 90.838s

         OK (skipped=5)

         make[1]: Leaving directory `/home/maciek/src/vpp2/test'

     2. stable/1701

       ======================================================================

       ERROR: GRE tunnel Tests

       Exception: Capture file did not appear within timeout

       ======================================================================

       ERROR: GRE tunnel L2 Tests

       Exception: Capture file did not appear within timeout

       ======================================================================

       ERROR: MPLS Local Label Binding test

       AttributeError: label

       ======================================================================

       ERROR: MPLS label imposition test

       IndexError: Layer [IP] not found

       ======================================================================

       ERROR: MPLS label swap tests

       AttributeError: label

       ======================================================================

       ERROR: MPLS Tunnel Tests

       ----------------------------------------------------------------------

       IndexError: Layer [IP] not found

       Ran 64 tests in 59.980s

       FAILED (errors=6, skipped=5)

       make[1]: *** [test] Error 1

       make[1]: Leaving directory `/home/maciek/src/vpp2/test'

       make: *** [test] Error 2

     Fix?

     -Maciek

       On 12 Dec 2016, at 16:19, Maciek Konstantynowicz (mkonstan)

       [4][13]<[1]mkons...@cisco.com> wrote:

       Hello, Does anyone know if vpp make test is back on track to be ready to

       be used for vpp make verify jobs on a per patch basis?

       Being lazy I know - cause I could run it myself :)

       -Maciek

   References

     Visible links

     1. [5][14]mailto:mkons...@cisco.com

      _______________________________________________

      csit-dev mailing list

      [[15]6]csit-...@lists.fd.io

      [7][16]https://lists.fd.io/mailman/listinfo/csit-dev

   _______________________________________________

   csit-dev mailing list

   [[17]8]csit-...@lists.fd.io

   [9][18]https://lists.fd.io/mailman/listinfo/csit-dev

  References

     Visible links

     1. [19]mailto:nra...@cisco.com

     2. 
[20]mailto:csit-dev-boun...@lists.fd.ioonbehalfofklementsekera-x(ksekera-pantheontechnologiesatcisco)

     3. [21]mailto:csit-dev-boun...@lists.fd.ioonbehalfofksek...@cisco.com

     4. mailto:[[22]1]mkons...@cisco.com

     5. [23]mailto:mkons...@cisco.com

     6. [24]mailto:csit-...@lists.fd.io

     7. [25]https://lists.fd.io/mailman/listinfo/csit-dev

     8. [26]mailto:csit-...@lists.fd.io

     9. [27]https://lists.fd.io/mailman/listinfo/csit-dev

References

    Visible links
    1. http://pastebin.com/uGAUREhK
    2. mailto:dwallac...@gmail.com
    3. mailto:ksek...@cisco.com
    4. mailto:mkons...@cisco.com
    5. mailto:nra...@cisco.com
    6. mailto:csit-...@lists.fd.io
    7. mailto:csit-...@lists.fd.io
    8. mailto:vpp-dev@lists.fd.io
    9. mailto:vpp-dev@lists.fd.io
   10. mailto:nra...@cisco.com
   11. 
mailto:csit-dev-boun...@lists.fd.ioonbehalfofklementsekera-x%28ksekera-pantheontechnologiesatcisco%29
   12. mailto:csit-dev-boun...@lists.fd.ioonbehalfofksek...@cisco.com
   13. mailto:[1]mkons...@cisco.com
   14. mailto:mkons...@cisco.com
   15. mailto:6]csit-...@lists.fd.io
   16. https://lists.fd.io/mailman/listinfo/csit-dev
   17. mailto:8]csit-...@lists.fd.io
   18. https://lists.fd.io/mailman/listinfo/csit-dev
   19. mailto:nra...@cisco.com
   20. 
mailto:csit-dev-boun...@lists.fd.ioonbehalfofklementsekera-x%28ksekera-pantheontechnologiesatcisco%29
   21. mailto:csit-dev-boun...@lists.fd.ioonbehalfofksek...@cisco.com
   22. mailto:1]mkons...@cisco.com
   23. mailto:mkons...@cisco.com
   24. mailto:csit-...@lists.fd.io
   25. https://lists.fd.io/mailman/listinfo/csit-dev
   26. mailto:csit-...@lists.fd.io
   27. https://lists.fd.io/mailman/listinfo/csit-dev


_______________________________________________
vpp-dev mailing list
vpp-dev@lists.fd.io
https://lists.fd.io/mailman/listinfo/vpp-dev
  • Re: [vpp-dev]... Maciek Konstantynowicz (mkonstan)
    • Re: [vpp... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
      • Re: ... Neale Ranns (nranns)
        • ... Maciek Konstantynowicz (mkonstan)
          • ... Dave Wallace
            • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
              • ... Dave Wallace
              • ... Neale Ranns (nranns)
              • ... Dave Wallace
              • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
              • ... Dave Wallace
              • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
              • ... Neale Ranns (nranns)
              • ... Neale Ranns (nranns)
              • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
              • ... Neale Ranns (nranns)
              • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
          • ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)
    • Re: [vpp... Andrew 👽 Yourtchenko
      • Re: ... Klement Sekera -X (ksekera - PANTHEON TECHNOLOGIES at Cisco)

Reply via email to