Finally got around to tracing this bug.
As indicated by Ben I used git bisect to find the issue. Very neat feature of
git!
Copy paste of my findings:
<<
Attending to the instructions received in the mailing list, I have managed to
find the culprit via git bisect.
The bug I am experiencing seems to be fixed after removing the OVS GRE IPsec
tunnel functionality.
Here is the log of the process:
```
$ git bisect log
git bisect start
# bad: [c298ef781c2d35d939fe163cbc2f41ea7b1cb8d1] Set release date for 2.7.0.
git bisect bad c298ef781c2d35d939fe163cbc2f41ea7b1cb8d1
# good: [f4b0e64cffb4777ff03d48621c3eadcf1d8c19f3] Set release date for 2.6.1.
git bisect good f4b0e64cffb4777ff03d48621c3eadcf1d8c19f3
# good: [4c714486187dada924f1e1fc2fdbec99cddd777f] Prepare for 2.6.0.
git bisect good 4c714486187dada924f1e1fc2fdbec99cddd777f
# bad: [048318e0726c94876e504de7b578f929e9f509a0] netdev: Count ports within
mutex.
git bisect bad 048318e0726c94876e504de7b578f929e9f509a0
# bad: [b614c894ee55b07009f3e5b10ddec7556773fcc4] netdev-dpdk: Configure flow
control only when necessary.
git bisect bad b614c894ee55b07009f3e5b10ddec7556773fcc4
# good: [cc4583aadd2a55b340bc7917305b41bbba29c9f2] ovn-northd: Add
load-balancers to gateway routers.
git bisect good cc4583aadd2a55b340bc7917305b41bbba29c9f2
# good: [9ef3a410b2d93ae6890c6e831dd3ed84d3c374d9] datapath-windows: Add define
for last module number
git bisect good 9ef3a410b2d93ae6890c6e831dd3ed84d3c374d9
# good: [9e9d0384910e8d1e96463be19378de3fe9d64877] openvswitch: deprecates
support for IPsec tunnel port.
git bisect good 9e9d0384910e8d1e96463be19378de3fe9d64877
# bad: [dd0dc9eda0e0b4a6b2f8f4dee442be6865e60c89] revalidator: Reuse xlate_ukey
from deletion.
git bisect bad dd0dc9eda0e0b4a6b2f8f4dee442be6865e60c89
# bad: [9f02d70c11e14366f2136caaedcebd629551a131] ofp-actions: Style fixes.
git bisect bad 9f02d70c11e14366f2136caaedcebd629551a131
# good: [5e8bc3c549ca9bfa02c5525c02cb4ee12ef1f06e] ovsdb: Fix memory leak when
disposing 'replication_dbs'
git bisect good 5e8bc3c549ca9bfa02c5525c02cb4ee12ef1f06e
# bad: [9120cfc05cd49d4ba1a47eb97e6407e72a5d33f7] netdev-linux: Use ethtool
when miimon fails.
git bisect bad 9120cfc05cd49d4ba1a47eb97e6407e72a5d33f7
# bad: [2b02d770c4cb381ec32cd4b7b1e991c42b448884] openvswitch: Allow external
IPsec tunnel management.
git bisect bad 2b02d770c4cb381ec32cd4b7b1e991c42b448884
# first bad commit: [2b02d770c4cb381ec32cd4b7b1e991c42b448884] openvswitch:
Allow external IPsec tunnel management.
```
The details of the commit
```
2b02d770c4cb381ec32cd4b7b1e991c42b448884 is the first bad commit
commit 2b02d770c4cb381ec32cd4b7b1e991c42b448884
Author: Pravin B Shelar <pshe...@ovn.org>
Date: Sat Sep 24 11:44:53 2016 -0700
openvswitch: Allow external IPsec tunnel management.
OVS GRE IPsec tunnel support has multiple issues, Therefore
it was deprecated in OVS 2.6.
Following patch removes support for GRE IPsec and allows external
IPsec tunnel management for any type of tunnel not just GRE.
e.g. user can encrypt Geneve or VxLan traffic.
It can be done by using openflow pipeline to set skb-mark
and using IPsec keying daemons to implement IPsec tunnels.
This packet can be matched for the skb-mark to encrypt
selective tunnel traffic.
VMware-BZ: 1710701
Signed-off-by: Pravin B Shelar <pshe...@ovn.org>
Acked-by: Ansis Atteka <aatt...@ovn.org>
:100644 100644 6e284aac32557ba75408375d58546ac7745433dd
069ab424c644ef0ee992340f450e1dae5fc42454 M NEWS
:100644 100644 cf5343714937c28b90ca5f3457d23f0ed5df1ebf
53b0fafe3a56e604c2b62097c54ef6dcd135fe83 M README.md
:040000 040000 636109402f0a3a4798d68f9cf85399b7230d7819
ed14940340a22190843c9270740039e404521b59 M debian
:040000 040000 5444eb414d2383ec4340243cbfcfcdb0da80e144
2c5687e8813086c083ef2df7382a6fc17f782040 M lib
:040000 040000 d6cd119e1715c6716dd8fc509169d1499d3a2e31
310aa4ce502847c2824d4b04bd539e60b2100b26 M ofproto
:040000 040000 208fe072c0f3a3f8139f695019c81fcffcfbd373
6f73eb237766d6bab68fc36389c2ed5fd57e2240 M tests
:040000 040000 0c796059e07e889bded911f39324d6e9e3cb6771
e5ac1d4e6d492899d2194bf451d5393513e5fea1 M utilities
:040000 040000 5aa99398dbba68c93985189042fa126bf0c527ed
aa9d605b55ac9e79524b9151d4173eaccb8eb9d0 M vswitchd
```
Is it feasible to port this to next release of OvS 2.5.3 ?
>>
Best,
Jesus
-----Original Message-----
From: Ben Pfaff [mailto:b...@ovn.org]
Sent: 12 September 2017 01:01
To: Llorente Santos Jesus <jesus.llorente.san...@aalto.fi>
Cc: ovs-discuss@openvswitch.org
Subject: Re: [ovs-discuss] How to submit a bug report
I didn't realize that there was different information on the mailing list and
in ovs-issues. Thanks for adding the missing information.
"git bisect" would probably take 10 tries or less to find the commit, so I'd
still like to encourage you to try that approach.
On Mon, Sep 11, 2017 at 09:27:46PM +0000, Llorente Santos Jesus wrote:
> Hi Ben, thank you for getting back to me on this.
>
> I noticed that my previous email with the steps to reproduce was sent to
> ovs-dev mailing list. It somewhat felt like the appropriate place to send
> such detailed issue.
> In any case, I have added those same steps to the open issue in the
> ovs-issues repo. I hope that simplifies the follow up!
>
> I spent quite bit of time looking at the code but I couldn't really see
> anything odd. Comparing 2.5.3 with 2.7.0, showed a vast number of commits. I
> deepened into the tunnelling code for GRE, VXLAN and Geneve but couldn't stop
> the difference. It's quite odd, in my experience, marking packets does not
> invalidate in any case receiving them into the socket.
>
> The issue here is that 2.5.3 is the current LTS version, which in late June
> received a new update. The release note says "Bug fixes", so maybe there is
> still something we can fix there...
>
> Best,
> Jesus
>
>
> -----Original Message-----
> From: Ben Pfaff [mailto:b...@ovn.org]
> Sent: 12 September 2017 00:03
> To: Llorente Santos Jesus <jesus.llorente.san...@aalto.fi>
> Cc: ovs-discuss@openvswitch.org
> Subject: Re: [ovs-discuss] How to submit a bug report
>
> On Mon, Sep 11, 2017 at 07:27:56PM +0000, Llorente Santos Jesus wrote:
> > I have been seeing buggy behaviour with the tunnelling ports of OvS.
> > I would like to ask some help in how to properly submit a bug report I
> > created an issue in the git "ovs-issues"
> > (https://github.com/openvswitch/ovs-issues/issues/134) about 2 weeks ago,
> > but it seems no one is monitoring that.
> > I sent an email to b...@openvswitch.org following these
> > instructions
> > (http://docs.openvswitch.org/en/latest/internals/bugs/)
> > I have also sent an email to this mailing list, but no one has responded my
> > message.
> >
> > Can anyone help me? I would really like to get this annoying bug fixed!
>
> Your report has definitely been recorded. I can think of a few reasons why
> no one has acted on it, besides simply being busy. One is that it does not
> give readers a good way to reproduce the problem, even though it talks about
> a very particular scenario. Developers are more likely to try to reproduce
> and fix a bug when it doesn't require a lot of forethought on how to get
> started. Another is that it sounds like the problem has been fixed in a
> later version. It's not too rewarding to fix bugs that are already fixed.
>
> Given that it's been fixed, you could use "git bisect" to try to find out
> what commit fixed it. If you reported that, it could allow a developer to
> understand and fix the problem without much additional work.
_______________________________________________
discuss mailing list
disc...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
