Hi Faseela, Things to check:
1. Which odl is master for this switch and is it master for any other switch? 2. Is the flow present in config DS? 3. What was showsvc status for this ODL node? Regards, Vishal. On Thu, Jul 26, 2018 at 10:40 PM, Faseela K <[email protected]> wrote: > Taking a look at the logs, i can see the below log line in > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ > builder-copy-sandbox-logs/260/faseela-l2-netvirt-csit-3node- > 0cmb-1ctl-2cmp-openstack-queens-upstream-stateful- > oxygen/18/odl_1/odl1_karaf.log.gz > > 10 mins before the test suite starts : > > > > 2018-07-25T23:00:41,688 | INFO | pool-241-thread-1 | > WeightedCentralizedSwitchScheduler | 361 - > org.opendaylight.netvirt.natservice-impl > - 0.6.3.SNAPSHOT | addSwitch: Adding 167740445697660 dpnId to > switchWeightsMap > > > > 2018-07-25T23:00:42,274 | DEBUG | epollEventLoopGroup-9-1 | > SalFlowServiceImpl | 385 - org.opendaylight.openflowplugin.impl - > 0.6.3.SNAPSHOT | Flow add with id=DHCPTableMissFlowForExternalTunnel > finished without error > > > > But the flow is not present on the switch when the suite starts. > > > > We have 3 OVS in the test, but i see only this flow-id log for 2 switches. > Anyways the compute in question, is 167740445697660, so let us assume that > the flow was getting pushed for the same, but doesn't show up on switch. > But there are different failures showing up in the logs including the below > one. Let me know if this could impact: > > > > > > 2018-07-25T23:01:09,328 | ERROR | > opendaylight-cluster-data-notification-dispatcher-97 > | DefaultConfigPusher | 378 - org.opendaylight. > openflowplugin.applications.of-switch-config-pusher - 0.6.3.SNAPSHOT | > Future (eventually) failed: addFlow org.opendaylight.controller. > md.sal.dom.api.DOMRpcImplementationNotAvailableException: No > implementation of RPC AbsoluteSchemaPath{path=[(urn: > opendaylight:module:config?revision=2014-10-15)set-config]} available at > org.opendaylight.controller.md.sal.dom.broker.impl. > RoutedDOMRpcRoutingTableEntry.invokeRpc(RoutedDOMRpcRoutingTableEntry.java:85) > [229:org.opendaylight.controller.sal-broker-impl:1.7.3.SNAPSHOT] at > org.opendaylight.controller.md.sal.dom.broker.impl. > DOMRpcRoutingTable.invokeRpc(DOMRpcRoutingTable.java:178) > [229:org.opendaylight.controller.sal-broker-impl:1.7.3.SNAPSHOT] at > org.opendaylight.controller.md.sal.dom.broker.impl.DOMRpcRouter.invokeRpc(DOMRpcRouter.java:102) > [229:org.opendaylight.controller.sal-broker-impl:1.7.3.SNAPSHOT] at > Proxybb8ecbf1_eb9d_4480_a18e_4a2007f00a3c.invokeRpc(Unknown Source) [?:?] > at Proxy40f45215_a726_4be9_ac6b_649862163662.invokeRpc(Unknown Source) > [?:?] at org.opendaylight.controller.md.sal.binding.impl. > RpcServiceAdapter.invoke0(RpcServiceAdapter.java:68) > [226:org.opendaylight.controller.sal-binding-broker-impl:1.7.3.SNAPSHOT] > at org.opendaylight.controller.md.sal.binding.impl. > RpcServiceAdapter.access$000(RpcServiceAdapter.java:46) > [226:org.opendaylight.controller.sal-binding-broker-impl:1.7.3.SNAPSHOT] > at org.opendaylight.controller.md.sal.binding.impl.RpcServiceAdapter$ > RpcInvocationStrategy.invoke(RpcServiceAdapter.java:165) > [226:org.opendaylight.controller.sal-binding-broker-impl:1.7.3.SNAPSHOT] > at org.opendaylight.controller.md.sal.binding.impl. > RpcServiceAdapter.invoke(RpcServiceAdapter.java:99) [226:org.opendaylight. > controller.sal-binding-broker-impl:1.7.3.SNAPSHOT] at > com.sun.proxy.$Proxy113.setConfig(Unknown Source) [388:org.opendaylight. > openflowplugin.model.flow-service:0.6.3.SNAPSHOT] at org.opendaylight. > openflowplugin.openflow.ofswitch.config.DefaultConfigPusher. > onDataTreeChanged(DefaultConfigPusher.java:84) [378:org.opendaylight. > openflowplugin.applications.of-switch-config-pusher:0.6.3.SNAPSHOT] at > org.opendaylight.controller.md.sal.binding.impl. > BindingDOMDataTreeChangeListenerAdapter.onDataTreeChanged( > BindingDOMDataTreeChangeListenerAdapter.java:41) [226:org.opendaylight. > controller.sal-binding-broker-impl:1.7.3.SNAPSHOT] at > org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor. > dataChanged(DataTreeChangeListenerActor.java:67) [239:org.opendaylight. > controller.sal-distributed-datastore:1.7.3.SNAPSHOT] at > org.opendaylight.controller.cluster.datastore.DataTreeChangeListenerActor. > handleReceive(DataTreeChangeListenerActor.java:41) [239:org.opendaylight. > controller.sal-distributed-datastore:1.7.3.SNAPSHOT] at > org.opendaylight.controller.cluster.common.actor.AbstractUntypedActor. > onReceive(AbstractUntypedActor.java:38) [232:org.opendaylight. > controller.sal-clustering-commons:1.7.3.SNAPSHOT] at > akka.actor.UntypedActor$$anonfun$receive$1.applyOrElse(UntypedActor.scala:167) > [42:com.typesafe.akka.actor:2.5.11] at > akka.actor.Actor.aroundReceive(Actor.scala:517) > [42:com.typesafe.akka.actor:2.5.11] at akka.actor.Actor. > aroundReceive$(Actor.scala:515) [42:com.typesafe.akka.actor:2.5.11] at > akka.actor.UntypedActor.aroundReceive(UntypedActor.scala:97) > [42:com.typesafe.akka.actor:2.5.11] at akka.actor.ActorCell. > receiveMessage(ActorCell.scala:590) [42:com.typesafe.akka.actor:2.5.11] > at akka.actor.ActorCell.invoke(ActorCell.scala:559) > [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.Mailbox. > processMailbox(Mailbox.scala:257) [42:com.typesafe.akka.actor:2.5.11] at > akka.dispatch.Mailbox.run(Mailbox.scala:224) > [42:com.typesafe.akka.actor:2.5.11] at > akka.dispatch.Mailbox.exec(Mailbox.scala:234) > [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin. > ForkJoinTask.doExec(ForkJoinTask.java:260) [42:com.typesafe.akka.actor:2.5.11] > at akka.dispatch.forkjoin.ForkJoinPool$WorkQueue. > runTask(ForkJoinPool.java:1339) [42:com.typesafe.akka.actor:2.5.11] at > akka.dispatch.forkjoin.ForkJoinPool.runWorker(ForkJoinPool.java:1979) > [42:com.typesafe.akka.actor:2.5.11] at akka.dispatch.forkjoin. > ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:107) > [42:com.typesafe.akka.actor:2.5.11] > > > > *From:* Faseela K > *Sent:* Thursday, July 26, 2018 7:50 AM > *To:* D Arunprakash <[email protected]>; 'Anil Vishnoi' < > [email protected]> > *Cc:* Gobinath . <[email protected]>; 'Sam Hague' <[email protected]>; > Dayavanti Gopal Kamath <[email protected]>; 'Jamo > Luhrsen' <[email protected]>; 'Vishal Thapar' <[email protected]>; ' > [email protected]' <openflowplugin-dev@lists. > opendaylight.org> > > *Subject:* RE: Netvirt Cluster CSIT : Table Miss Entry programming > failing randomly > > > > Arun, > > Here are the logs with DEBUG enabled. > > https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/ > builder-copy-sandbox-logs/260/faseela-l2-netvirt-csit-3node- > 0cmb-1ctl-2cmp-openstack-queens-upstream-stateful-oxygen/18/ > > This time, table 18 entry is missing. > > Thanks, > > Faseela > > > > *From:* Faseela K > *Sent:* Wednesday, July 25, 2018 4:34 PM > *To:* D Arunprakash <[email protected]>; Anil Vishnoi < > [email protected]> > *Cc:* Gobinath . <[email protected]>; Sam Hague <[email protected]>; > Dayavanti Gopal Kamath <[email protected]>; Jamo > Luhrsen <[email protected]>; Vishal Thapar <[email protected]>; > [email protected] > *Subject:* RE: Netvirt Cluster CSIT : Table Miss Entry programming > failing randomly > > > > Arun, > > I have enabled the requested TRACE logs and running the CSIT now. > > Ever since I enabled the TRACEs it has not failed J. > > Let us see if the issue gets reproduced in another 24 hours. > > Thanks, > > Faseela > > > > *From:* D Arunprakash > *Sent:* Wednesday, July 25, 2018 7:53 AM > *To:* Faseela K <[email protected]>; Anil Vishnoi < > [email protected]> > *Cc:* Gobinath . <[email protected]>; Sam Hague <[email protected]>; > Dayavanti Gopal Kamath <[email protected]>; Jamo > Luhrsen <[email protected]>; Vishal Thapar <[email protected]>; > [email protected] > *Subject:* RE: Netvirt Cluster CSIT : Table Miss Entry programming > failing randomly > > > > Faseela, > > Could you please enable the following package logging and provide the logs? > > > > log:set DEBUG org.opendaylight.openflowplugin.impl > > > > Regards, > > Arun > > > > *From:* Faseela K > *Sent:* Tuesday, July 24, 2018 11:44 PM > *To:* Anil Vishnoi <[email protected]> > *Cc:* D Arunprakash <[email protected]>; Gobinath . < > [email protected]>; Sam Hague <[email protected]>; Dayavanti Gopal > Kamath <[email protected]>; Jamo Luhrsen < > [email protected]>; Vishal Thapar <[email protected]>; > [email protected] > *Subject:* RE: Netvirt Cluster CSIT : Table Miss Entry programming > failing randomly > > > > Hi Arun, > > We are still hitting this issue, and this can be seen on stable/oxygen > as well. > > In some cases the flow will be programmed initially, but after some > time it disappears. > > I do see a lot of “Switch Idle state occurred, > node=/10.30.170.51:60858|auxId=0” messages in the logs. Is that anything to > worry about? > > If there is any specific TRACE logs to be enabled while running the CSIT, > could you please let me know? I can run the CSIT with the same and supply the > logs. > > Thanks, > > Faseela > > > > *From:* Faseela K > *Sent:* Tuesday, July 17, 2018 1:24 PM > *To:* Anil Vishnoi <[email protected]> > *Cc:* D Arunprakash <[email protected]>; Gobinath . < > [email protected]>; Sam Hague <[email protected]>; Dayavanti Gopal > Kamath <[email protected]>; Jamo Luhrsen < > [email protected]>; Vishal Thapar <[email protected]>; > [email protected] > *Subject:* Netvirt Cluster CSIT : Table Miss Entry programming failing > randomly > > > > Anil, > > > > I have raised the below JIRA, where I see Table Miss flow for Table 43 is > sometimes not getting programmed on one of the computes in netvirt 3 node > CSIT. > > https://jira.opendaylight.org/browse/OPNFLWPLUG-1028 > > I do see flow being pushed to config/inventory by netvirt, but it is > missing on switch. > > This makes the basic DHCP itself fail for netvirt as this is one of the > important flows needed for all broadcasts to work. > > Any pointers? > > > > Thanks, > > Faseela >
_______________________________________________ openflowplugin-dev mailing list [email protected] https://lists.opendaylight.org/mailman/listinfo/openflowplugin-dev
