thanks ________________________________ Od: Luis Gomez <ece...@gmail.com> Odoslané: piatok, 21. apríla 2017 2:12:12 Komu: Jozef Bacigál Kópia: Miroslav Macko; controller-dev; openflowplugin-dev; integration-...@lists.opendaylight.org Predmet: Re: [integration-dev] CSIT test bugs
I added cluster logs to the bug. On Apr 20, 2017, at 12:00 AM, Jozef Bacigál <jozef.baci...@pantheon.tech<mailto:jozef.baci...@pantheon.tech>> wrote: Luis can you send me the karaf logs you made with DEBUG ? Jozef ________________________________ Od: Luis Gomez <ece...@gmail.com<mailto:ece...@gmail.com>> Odoslané: štvrtok, 20. apríla 2017 3:01:56 Komu: Miroslav Macko; controller-dev Kópia: Jozef Bacigál; openflowplugin-dev; integration-...@lists.opendaylight.org<mailto:integration-...@lists.opendaylight.org> Predmet: Re: [integration-dev] CSIT test bugs CC-ing controller-dev anyway, These messages shown in one of the candidates (.101) after OF device owner (.103) is killed. RPC "add-flow" to the candidate (.101) which is not the new owner will fail until we see the message *Leader can perform its duties again* below after few mins. This behavior is not observed in Boron where RPC works very quickly after OF device owner is killed. Current bug: https://bugs.opendaylight.org/show_bug.cgi?id=8185 BR/Luis On Apr 19, 2017, at 5:28 PM, Luis Gomez <ece...@gmail.com<mailto:ece...@gmail.com>> wrote: I activated DEBUG for openflow and again I do not see anything conclusive, however I observed following cluster logging in the candidate instance I send the RPC and fails: 2017-04-19 23:45:48,003 | INFO | ult-dispatcher-4 | kka://opendaylight-cluster-data) | 179 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550] - Leader can currently not perform its duties, reachability status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (1), akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (4)], member status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.103:2550 Up seen=false] 2017-04-19 23:46:27,990 | INFO | ult-dispatcher-6 | kka://opendaylight-cluster-data) | 179 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550] - Leader can currently not perform its duties, reachability status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (1), akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (4)], member status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.103:2550 Up seen=false] 2017-04-19 23:47:28,000 | INFO | ult-dispatcher-3 | kka://opendaylight-cluster-data) | 179 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550] - Leader can currently not perform its duties, reachability status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (1), akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (4)], member status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.103:2550 Up seen=false] 2017-04-19 23:48:28,000 | INFO | ult-dispatcher-3 | kka://opendaylight-cluster-data) | 179 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550] - Leader can currently not perform its duties, reachability status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (1), akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 -> akka.tcp://opendaylight-cluster-data@192.168.0.103:2550: Unreachable [Unreachable] (4)], member status: [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.102:2550 Up seen=true, akka.tcp://opendaylight-cluster-data@192.168.0.103:2550 Up seen=false] 2017-04-19 23:48:31,999 | INFO | lt-dispatcher-17 | kka://opendaylight-cluster-data) | 179 - com.typesafe.akka.slf4j - 2.4.17 | Cluster Node [akka.tcp://opendaylight-cluster-data@192.168.0.101:2550] - Leader can perform its duties again Also the RPC seems to work after the last line which is at least 3 mins after the owner goes down. Any idea what is going on or should we involve cluster people here? BR/Luis On Apr 18, 2017, at 11:59 PM, Miroslav Macko <miroslav.ma...@pantheon.tech<mailto:miroslav.ma...@pantheon.tech>> wrote: Luis it would be great, if you can try with DEBUG on org.opendaylight.openflowplugin.impl. Thanks a lot, Miro ________________________________ Od: Luis Gomez <ece...@gmail.com<mailto:ece...@gmail.com>> Odoslané: streda, 19. apríla 2017 5:57:35 Komu: Jozef Bacigál Kópia: openflowplugin-dev; Miroslav Macko; integration-...@lists.opendaylight.org<mailto:integration-...@lists.opendaylight.org> Predmet: Re: [integration-dev] CSIT test bugs As far as I can tell the issue is not consistent, using local robot automation I can reproduce after a while trying. The log of the failing RPC instance does not show anything extraordinary, can you provide some DEBUG I can try? BR/Luis On Apr 18, 2017, at 9:55 AM, Luis Gomez <ece...@gmail.com<mailto:ece...@gmail.com>> wrote: OK, let me see if I find a way for easy reproduction. On Apr 18, 2017, at 3:52 AM, Jozef Bacigál <jozef.baci...@pantheon.tech<mailto:jozef.baci...@pantheon.tech>> wrote: Luis to the Cluster issue 2) it is failing always, because we were not able to reproduce it. Miroslav started 3 node cluster with 3 switches connected to all controllers. If send to any controler any RPC comming witout error. After killing master a new master is elected and if sended an RPC to any controller is still ok. RPC comming without error. Is there any other way to do it ? Tested on master actual master branch. Jozef ________________________________ Od: Luis Gomez <ece...@gmail.com<mailto:ece...@gmail.com>> Odoslané: piatok, 14. apríla 2017 21:30:41 Komu: openflowplugin-dev Kópia: integration-...@lists.opendaylight.org<mailto:integration-...@lists.opendaylight.org> Predmet: [integration-dev] CSIT test bugs It is been a while, lets review current CSIT test bugs after latest patches: Boron and Carbon bugs: 1) Switch scalability regression due to missing table miss flows. I think table miss application does not scale and that impacts scalability test. Bug: https://bugs.opendaylight.org/show_bug.cgi?id=8103 Tracked: https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-1node-periodic-sw-scalability-daily-only-boron/plot/Switch%20Scalability Carbon "cluster" bugs: 1) Switch is still seen after owner goes down in non OFHA scenario. Bug: https://bugs.opendaylight.org/show_bug.cgi?id=6459 Tracked: https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-periodic-bulkomatic-clustering-daily-only-carbon 2) Send RPC to non-owner after original owner is killed is not stable in Carbon. Bug: https://bugs.opendaylight.org/show_bug.cgi?id=8185 Tracked: https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-clustering-only-carbon 3) Cluster perf regression in Carbon: Bug: https://bugs.opendaylight.org/show_bug.cgi?id=6755 Tracked: https://jenkins.opendaylight.org/releng/view/openflowplugin/job/openflowplugin-csit-3node-periodic-bulkomatic-clustering-daily-only-carbon/ BR/Luis _______________________________________________ integration-dev mailing list integration-...@lists.opendaylight.org<mailto:integration-...@lists.opendaylight.org> https://lists.opendaylight.org/mailman/listinfo/integration-dev
_______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
