Anil, openflowpugin-dev could you look at this bug and the questions Vivek has? This could be a blocker for carbon and nitrogen as it happens consistently for both versions.
Thanks, Sam ---------- Forwarded message ---------- From: N Vivekanandan <[email protected]> Date: Wed, Sep 13, 2017 at 7:46 AM Subject: Bug 7826 update To: Sam Hague <[email protected]> Cc: Swati Udhavrao Niture <[email protected]>, Kiran N Upadhyaya <[email protected]> Hi Sam, We analyzed the exception today here. The Openflow-inventory Operational Datastore is not managed by Genius / Netvirt. The management of data on that DS is driven by OpenFlowPlugin and OVSDB. The same inventory-node is present in config-DS equivalent openflow:inventory-nodes. However, that inventory-node is absent in Operational DS, which results in the group-id exception being thrown. I request that we get help from OpenFlowPlugin team to take a look on why there is a transaction initiated on an inventory-node that does not exist? We referred the following links in the bug: Config DS has the node correctly: https://logs.opendaylight.org/releng/jenkins092/netvirt- csit-1node-openstack-ocata-upstream-stateful-nitrogen/ 132/log.html.gz#s1-s2-s1-t1-k3-k1-k2-k1-k2-k33-k4 *Node: openflow:1041420389861* https://logs.opendaylight.org/releng/jenkins092/netvirt- csit-1node-openstack-ocata-upstream-stateful-nitrogen/132/odl1_karaf.log.gz 2017-08-11 16:58:50,470 | WARN | rd-dispatcher-91 | ShardDataTree | 211 - org.opendaylight.controller.sal-distributed-datastore - 1.6.0.SNAPSHOT | member-1-shard-inventory-operational: Store Tx member-1-datastore-operational-fe-0-chn-8-txn-4725-0: Data validation failed for path /(urn:opendaylight:inventory?revision=2013-08-19)nodes/ node/node[{(urn:opendaylight:inventory?revision=2013-08-19)id= *openflow:1041420389861*}]/AugmentationIdentifier{ childNames=[(urn:opendaylight:flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow:inventory?revision=2013-08-19)table, (urn:opendaylight:flow:inventory?revision=2013-08-19)group, (urn:opendaylight:flow:inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow:inventory?revision=2013-08-19)software, (urn:opendaylight:flow:inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow:inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow:inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow:inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow:inventory?revision=2013-08-19)description, (urn:opendaylight:flow:inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow:inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19) meter]}/(urn:opendaylight:flow:inventory?revision=2013- 08-19)group/group[{(urn:opendaylight:flow:inventory? revision=2013-08-19)group-id=210004}]. org.opendaylight.yangtools.yang.data.api.schema.tree. ModifiedNodeDoesNotExistException: Node /(urn:opendaylight:inventory? revision=2013-08-19)nodes/node/node[{(urn:opendaylight: inventory?revision=2013-08-19)id=*openflow:1041420389861*}]/ AugmentationIdentifier{childNames=[(urn:opendaylight: flow:inventory?revision=2013-08-19)port-number, (urn:opendaylight:flow: inventory?revision=2013-08-19)stale-group, (urn:opendaylight:flow: inventory?revision=2013-08-19)supported-match-types, (urn:opendaylight:flow: inventory?revision=2013-08-19)table, (urn:opendaylight:flow: inventory?revision=2013-08-19)group, (urn:opendaylight:flow: inventory?revision=2013-08-19)manufacturer, (urn:opendaylight:flow: inventory?revision=2013-08-19)software, (urn:opendaylight:flow: inventory?revision=2013-08-19)ip-address, (urn:opendaylight:flow: inventory?revision=2013-08-19)serial-number, (urn:opendaylight:flow: inventory?revision=2013-08-19)table-features, (urn:opendaylight:flow: inventory?revision=2013-08-19)supported-actions, (urn:opendaylight:flow: inventory?revision=2013-08-19)hardware, (urn:opendaylight:flow: inventory?revision=2013-08-19)description, (urn:opendaylight:flow: inventory?revision=2013-08-19)switch-features, (urn:opendaylight:flow: inventory?revision=2013-08-19)supported-instructions, (urn:opendaylight:flow:inventory?revision=2013-08-19)stale-meter, (urn:opendaylight:flow:inventory?revision=2013-08-19) meter]}/(urn:opendaylight:flow:inventory?revision=2013- 08-19)group/group[{(urn:opendaylight:flow:inventory? revision=2013-08-19)group-id=210004}] does not exist. Cannot apply modification to its children. -- Thanks, Vivek *From:* Swati Udhavrao Niture *Sent:* Tuesday, September 05, 2017 11:12 AM *To:* Sam Hague *Cc:* N Vivekanandan; Aswin Suryanarayanan; P Govinda Rajulu; Kency Kurian; Kiran N Upadhyaya; Vishal Thapar; Jamo Luhrsen *Subject:* RE: push to finish up the exceptions and warnings in csit Hi Sam, Will try to reproduce this in local set-up and confirm. Will get back to you. Thanks, Swati. *From:* Sam Hague [mailto:[email protected] <[email protected]>] *Sent:* Friday, September 01, 2017 10:35 PM *To:* Swati Udhavrao Niture *Cc:* N Vivekanandan; Aswin Suryanarayanan; P Govinda Rajulu; Kency Kurian; Kiran N Upadhyaya; Vishal Thapar; Jamo Luhrsen *Subject:* Re: push to finish up the exceptions and warnings in csit On Fri, Sep 1, 2017 at 2:56 AM, Swati Udhavrao Niture < [email protected]> wrote: Hi Vivek/Sam, Regarding Bug 7826 (https://bugs.opendaylight.org/show_bug.cgi?id=7826), ELAN and L3 groups, for which the exception of “group-id <> does not exist” is coming, those groups are present in config DS. We could see that the exceptions are for operational DS: transaction member-1-datastore-operational-fe-0-chn-11-txn-968-0 on backend. And as far as I know, the operational DS will not be touched by the application. Swati, I wasn't sure what the conclusion was. Are you saying netvirt and genius do not update operational so this is some other applications issue? The logs seem to show that there is an ordering issue where the group is being updated, but that node isn't there. do you think openflowplugin is in error here? I have cross-checked it from CSIT logs https://logs.opendaylight.org/ releng/jenkins092/netvirt-csit-1node-openstack-ocata- gate-stateful-oxygen/30/odl1_karaf.log.gz that I had triggered for https://git.opendaylight.org/gerrit/#/c/62184/ Thanks, Swati. *From:* Swati Udhavrao Niture *Sent:* Monday, August 28, 2017 8:48 PM *To:* N Vivekanandan; Sam Hague; Aswin Suryanarayanan; P Govinda Rajulu; Kency Kurian *Cc:* Vishal Thapar; Jamo Luhrsen *Subject:* RE: push to finish up the exceptions and warnings in csit Hi Vivek/Sam, The fixes for the 7826 bug were already raised (the changes, we thought, might work, but they didn’t). So need little more time to work on this one. Thanks, Swati. *From:* N Vivekanandan *Sent:* Monday, August 28, 2017 8:18 PM *To:* Sam Hague; Aswin Suryanarayanan; P Govinda Rajulu; Kency Kurian; Swati Udhavrao Niture *Cc:* Vishal Thapar; Jamo Luhrsen *Subject:* RE: push to finish up the exceptions and warnings in csit Hi Sam, 8953 has been fixed and is almost ready to merge (if CSIT passes) here: https://git.opendaylight.org/gerrit/#/c/62186/ For 8860, I will raise a fix shortly. I had 9013 and 7826 being looked by Govind, Kency and Swati. Hi Govind/Kency/Swati, Can you please let us know where we are with fixing bugs 9013 and 7826? -- Thanks, Vivek *From:* Sam Hague [mailto:[email protected] <[email protected]>] *Sent:* Monday, August 28, 2017 7:27 PM *To:* N Vivekanandan <[email protected]>; Aswin Suryanarayanan < [email protected]> *Cc:* Vishal Thapar <[email protected]>; Jamo Luhrsen < [email protected]> *Subject:* Re: push to finish up the exceptions and warnings in csit Vivek, following up on this thread. All these bugs are very consistent in the csit runs. Though 8860 I thought we fixed by removing the log - of which I can't seem to find the gerrit though. Thanks, Sam *ID▲ <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=bug_id%20DESC&query_based_on=>* *Product <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=product%2Cbug_id&query_based_on=>* *Comp <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=component%2Cbug_id&query_based_on=>* *Assignee <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=assigned_to%2Cbug_id&query_based_on=>* *Status <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=bug_status%2Cbug_id&query_based_on=>* *Resolution <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=resolution%2Cbug_id&query_based_on=>* *Summary <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=short_desc%2Cbug_id&query_based_on=>* *Changed <https://bugs.opendaylight.org/buglist.cgi?bug_status=UNCONFIRMED&bug_status=CONFIRMED&bug_status=IN_PROGRESS&bug_status=WAITING_FOR_REVIEW&field0-0-0=product&field0-0-1=component&field1-0-0=status_whiteboard&list_id=83867&query_format=advanced&type0-0-0=substring&type0-0-1=substring&type1-0-0=substring&value0-0-0=netvirt&value0-0-1=netvirt&value1-0-0=csit%3Aexception&order=changeddate%2Cbug_id&query_based_on=>* 7826 <https://bugs.opendaylight.org/show_bug.cgi?id=7826> netvirt General swati.udhavrao.niture@erics... IN_P --- Data validation failed for path /(nodes/node/node/../group-id <https://bugs.opendaylight.org/show_bug.cgi?id=7826> 2017-08-17 7830 <https://bugs.opendaylight.org/show_bug.cgi?id=7830> netvirt General [email protected] CONF --- Conflicting modification for path /nodes/node/node../flow/flow-id <https://bugs.opendaylight.org/show_bug.cgi?id=7830> 2017-08-15 8860 <https://bugs.opendaylight.org/show_bug.cgi?id=8860> netvirt General [email protected] CONF --- NPE at org.opendaylight.netvirt.vpnmanager.SubnetmapChangeListener. getElanTag <https://bugs.opendaylight.org/show_bug.cgi?id=8860> Wed 10:42 8953 <https://bugs.opendaylight.org/show_bug.cgi?id=8953> netvirt General [email protected] WAIT --- IllegalArgumentException: vrfEntry is missing mandatory descendant origin <https://bugs.opendaylight.org/show_bug.cgi?id=8953> Wed 08:25 9013 <https://bugs.opendaylight.org/show_bug.cgi?id=9013> netvirt General [email protected] UNCO --- ElanUtils: RPC Call to Get egress actions for interface, OptimisticLockFailedException <https://bugs.opendaylight.org/show_bug.cgi?id=9013> Tue 15:27 On Thu, Aug 17, 2017 at 11:27 AM, Sam Hague <[email protected]> wrote: Adding Aswin to thread as I think he is looking at some of the bugs also. On Thu, Aug 17, 2017 at 3:09 AM, N Vivekanandan <[email protected]> wrote: > Hi Sam, > > We must be able to get all these addressed by this week. > > More details below as [Vivek] > > -----Original Message----- > From: Sam Hague [mailto:[email protected]] > Sent: Monday, August 14, 2017 1:31 AM > To: N Vivekanandan <[email protected]>; Vishal Thapar < [email protected]>; Jamo Luhrsen <[email protected]> > Subject: push to finish up the exceptions and warnings in csit > > Vivek, > > can we start a more aggressive cleanup for the following bugs? These are the ones mainly repeating in the current csit runs. There look to only be 6 exceptions left right now. > > There are 8 exception bugs right now of which 6 are listed below. 2 I don't see in the last runs, but these other 6 are repeatable. I assigned a few but a couple are still not assigned. I would like to get a push to get these fixed this week if possible. > > Thanks, Sam > > ------------------------------------------------------------ ------------------------------------ > Error Exceptions: > [0.1] https://logs.opendaylight.org/releng/jenkins092/netvirt- csit-1node-openstack-ocata-upstream-stateful-nitrogen/ 134/odl1_exception.log.gz > > [1] https://bugs.opendaylight.org/show_bug.cgi?id=7826 > Data validation failed for path /(nodes/node/node/../group-id > - this is one of the nastiest. It popes up under a few tests but is pretty consistent in the tests it does hit. > [Vivek] Bulk of this exception is due to Remote BC ELAN groups not getting cleaned up correctly. > Swati is raising a fix for this. > > [2] https://bugs.opendaylight.org/show_bug.cgi?id=7830 > Conflicting modification for path /nodes/node/node../flow/flow-id > [Vivek] Harder nut to crack. Looking in ! > > > [3] https://bugs.opendaylight.org/show_bug.cgi?id=8945 > Missing key is getVrfId. Supplied key is VpnInstanceOpDataEntryKey [] > [Vivek] Merged Kirans' fix for this into master and nitrogen. > Carbon is closed for commits, so did not merge the carbon review here: > https://git.opendaylight.org/gerrit/#/c/61804/1 > > [4] https://bugs.opendaylight.org/show_bug.cgi?id=8982 > NPE: the Function passed to Optional.transform() must not return null. > [Vivek] I see that Stephen is handling this one. > > [5] https://bugs.opendaylight.org/show_bug.cgi?id=8953 > IllegalArgumentException: vrfEntry is missing mandatory descendant origin > [Vivek] Have root-caused this and fix is being raised by Kiran. > > [6] https://bugs.opendaylight.org/show_bug.cgi?id=8983 > Data validation failed for path /snatint-ip-port-map/intip-port-map. > [Vivek] Have requested NAT folks to take a look at this one. > > -- > Thanks, > > Vivek > > ------------------------------------------------------------ ------------------------------------ > Error Logs: > https://bugs.opendaylight.org/show_bug.cgi?id=8984 > Cannot get, portOp for port 44dff6ac-b787-477e-92c6-4e34a679668b is not available in datastore > - this happens a lot. I wonder if it really is not an error and should be lower level? > > Actually quite a few errors are dumped in the log. I think half are not really errors and are just logged at the wrong level. > https://logs.opendaylight.org/releng/jenkins092/netvirt- csit-1node-openstack-ocata-upstream-stateful-nitrogen/134/odl1_err.log.gz
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
