Also, I want to point out that our CSIT did it's job here. We are using the cluster deploy scripts in CSIT in order to catch and fix these things, and we are presumably fine and working going forward. without this in CSIT, we likely would have released with the issue.
JamO On 01/09/2017 11:24 AM, Vratko Polak -X (vrpolak - PANTHEON TECHNOLOGIES at Cisco) wrote: > When you have a tight coupling like this, > general terms such as "controller people" or "the community" > rarely achieve the desired focus. > > I prefer such notes to be as close to the code as possible. > https://git.opendaylight.org/gerrit/#/c/50140/1/opendaylight/md-sal/sal-clustering-config/src/main/resources/initial/akka.conf > > Vratko. > > -----Original Message----- > From: controller-dev-boun...@lists.opendaylight.org > [mailto:controller-dev-boun...@lists.opendaylight.org] On Behalf Of Luis Gomez > Sent: 9 January, 2017 18:13 > To: controller-dev@lists.opendaylight.org > Cc: netvirt-...@lists.opendaylight.org; genius-...@lists.opendaylight.org; > Kochba, Alon <alo...@hpe.com>; mdsal-...@lists.opendaylight.org; > integration-...@lists.opendaylight.org; Peretz, Ravit <ravit.per...@hpe.com>; > Aizer, Koby <koby.ai...@hpe.com> > Subject: Re: [controller-dev] [mdsal-dev] 3node cluster regression in Carbon > - since Jan 5th > > Patch looks good but in future I would ask controller people to inform > broadly the community when you do these changes specially if they impact the > cluster configuration. > > BR/Luis > > >> On Jan 9, 2017, at 5:28 AM, Peretz, Ravit <ravit.per...@hpe.com> wrote: >> >> CSIT uses configure_cluster.sh to configure 3node jobs which creates >> akka.conf with akka.tcp instead of akka. >> >> Please review my fix patch: >> https://git.opendaylight.org/gerrit/50129 >> >> Thanks, >> Ravit. >> >> -----Original Message----- >> From: Tomas Cere -X (tcere - PANTHEON TECHNOLOGIES at Cisco) >> [mailto:tc...@cisco.com] >> Sent: יום ב 09 ינואר 2017 15:00 >> To: Robert Varga <n...@hq.sk>; Peretz, Ravit <ravit.per...@hpe.com>; >> integration-...@lists.opendaylight.org; >> netvirt-...@lists.opendaylight.org; genius-...@lists.opendaylight.org; >> controller-dev@lists.opendaylight.org; >> mdsal-...@lists.opendaylight.org >> Cc: Aizer, Koby <koby.ai...@hpe.com>; Kochba, Alon <alo...@hpe.com> >> Subject: RE: [controller-dev] [mdsal-dev] 3node cluster regression in >> Carbon - since Jan 5th >> >> It's already tracked by : >> https://bugs.opendaylight.org/show_bug.cgi?id=7493 >> I suspect it's the tests providing their own akka.conf which does not have >> the artery setup included. >> >> Tomas >> >> -----Original Message----- >> From: controller-dev-boun...@lists.opendaylight.org >> [mailto:controller-dev-boun...@lists.opendaylight.org] On Behalf Of >> Robert Varga >> Sent: Monday, January 09, 2017 13:59 >> To: Peretz, Ravit <ravit.per...@hpe.com>; >> integration-...@lists.opendaylight.org; >> netvirt-...@lists.opendaylight.org; genius-...@lists.opendaylight.org; >> controller-dev@lists.opendaylight.org; >> mdsal-...@lists.opendaylight.org >> Cc: Aizer, Koby <koby.ai...@hpe.com>; Kochba, Alon <alo...@hpe.com> >> Subject: Re: [controller-dev] [mdsal-dev] 3node cluster regression in >> Carbon - since Jan 5th >> >> >> >> On 01/09/2017 01:53 PM, Peretz, Ravit wrote: >>> Hi all, >>> >>> >>> >>> It seems like there is a massive 3node cluster regression in carbon, >>> since approximately 21:0PM January 5^th . >>> >>> We can see that many 3node CSIT fails across projects. >>> >>> >>> >>> After a quick look Koby and I have found what we assume is the faulty >>> zip. We looked at the last successful openflowplugin-3 node run: >>> >>> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3 >>> n ode-clustering-only-carbon/379/console.log.gz >>> >>> which used distribution: 0.6.0-20170105.205720-2879.zip >>> >>> >>> >>> the next run failed, with the same error we are seeing now in all >>> runs (at least the few we checked): >>> >>> https://logs.opendaylight.org/releng/jenkins092/openflowplugin-csit-3 >>> n ode-clustering-only-carbon/380/console.log.gz >>> >>> which used distribution: 0.6.0-20170105.235121-2883.zip >>> >>> >>> >>> >>> >>> we were able to narrow it down to a single distribution that was the >>> first to fail: >>> >>> https://logs.opendaylight.org/releng/jenkins092/netvirt-legacy-csit-3 >>> n ode-clustering-only-carbon/144/console.log.gz >>> >>> 0.6.0-20170105.222635-2880.zip >>> >>> >>> >>> *We would appreciate your help with narrowing down the faulty >>> commit/s triggering the 0.6.0-20170105.222635-2880.zip distribution.* >>> >>> >>> >>> >>> >>> The first error we saw in the logs is : >>> 2017-01-06 05:25:30,158 | ERROR | lt-dispatcher-12 | >>> ClusterActorRefProvider | 199 - com.typesafe.akka.slf4j - >>> 2.4.16 | No root guardian at >>> [akka.tcp://opendaylight-cluster-data@10.29.13.119:2550] >>> >>> java.lang.IllegalArgumentException: Wrong protocol of >>> [akka.tcp://opendaylight-cluster-data@10.29.13.119:2550/], expected >>> [akka] >> >> Looks like mismatch in setup, this is related to akka artery -- the protocol >> should be akka, not akka.tcp. Where is this configuration coming from? >> >> Regards, >> Robert >> _______________________________________________ >> controller-dev mailing list >> controller-dev@lists.opendaylight.org >> https://lists.opendaylight.org/mailman/listinfo/controller-dev >> _______________________________________________ >> controller-dev mailing list >> controller-dev@lists.opendaylight.org >> https://lists.opendaylight.org/mailman/listinfo/controller-dev > > _______________________________________________ > controller-dev mailing list > controller-dev@lists.opendaylight.org > https://lists.opendaylight.org/mailman/listinfo/controller-dev > _______________________________________________ > controller-dev mailing list > controller-dev@lists.opendaylight.org > https://lists.opendaylight.org/mailman/listinfo/controller-dev > _______________________________________________ controller-dev mailing list controller-dev@lists.opendaylight.org https://lists.opendaylight.org/mailman/listinfo/controller-dev
