Re: [controller-dev] owner changed failure

[Anil Vishnoi]

On Thu, Sep 20, 2018 at 3:38 AM Vratko Polák 
wrote:

> > It really isn't necessary that the new owner stays the same
>
>
> For some applications it does not matter.
>
> For other applications (such as openflow)
>
> each owner change means the old owner has to disconnect
>
> from the device and the new owner has to connect.
>
That's not entirely correct. It won't disconnect/connect, but yes it pushed
master/slave role down to the switch and that is also a cost. So for OFP,
minimum flapping of ownership is the optimal solution.

> That can be expensive, and in worst case it can lead to data loss.
>
>
> > I'm not sure if it's guaranteed the owner won't change after
> > re-join
>
>
> I know this test happened to work when we developed it,
>
> but that behavior might depend on timing details
>
> of an underlying algorithm implementation.
>
>
> For curiosity, I looked at unit tests.
>
> I have not found anything conclusive,
>
> but I found one comment [0] related to timing.
>
>
> Vratko.
>
>
> [0]
> https://github.com/opendaylight/controller/blob/9bce68c4712d00951d121be68b09578bc6e09151/opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/entityownership/DistributedEntityOwnershipIntegrationTest.java#L224-L226
> --
> *From:* Jamo Luhrsen 
> *Sent:* Thursday, September 20, 2018 7:45:12 AM
> *To:* Tom Pantelis
> *Cc:* controller-dev; pgup...@cisco.com; Vratko Polák
> *Subject:* Re: owner changed failure
>
>
>
> On 09/19/2018 02:36 PM, Tom Pantelis wrote:
> >
> >
> > On Wed, Sep 19, 2018 at 4:46 PM Jamo Luhrsen  mailto:jluhr...@gmail.com >> wrote:
> >
> > \
> >  > Anyway we'll need to enable debug
> for org.opendaylight.controller.cluster.datastore.entityownership.  I would
> > suggest to
> >  > pull out that test on its own like you've done before (run it
> standalone in sandbox I guess). Also delete the log
> > files
> >  > in between each run. This will make debugging much easier.
> >
> > Is there not enough info in the existing logs?
> >
> >
> > No - not with the default INFO logging. In order to dig deeper we need
> to enable targeted debug, in this
> > case org.opendaylight.controller.cluster.datastore.entityownership.
>
> I don't think it worked. Here is what the logger config lines looked
> like:
>
> 22:38:53
> log4j2.logger.org_opendaylight_org_opendaylight_controller_cluster_datastore_entityownership.name
> =
>
> org.opendaylight.org.opendaylight.controller.cluster.datastore.entityownership
> 22:38:53
> log4j2.logger.org_opendaylight_org_opendaylight_controller_cluster_datastore_entityownership.level
> = DEBUG
>
> you can see the whole org.ops4j.pax.logging.cfg file here:
>
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/console-timestamp.log.gz
>
> but, I don't see any DEBUG messages showing up in these karaf logs:
>
>
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/odl_1/odl1_karaf.log.gz
>
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/odl_2/odl2_karaf.log.gz
>
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/odl_3/odl3_karaf.log.gz
>
>
> > You can easily trim the
> > karaf logs based on the test cases. We are logging the start of every
> > suite and test case.
> >
> >
> > I think it's  much easier and faster to debug a failing test if it's
> isolated. Of course the logs are much smaller and
> > don't require trimming.  Enabling debug can result in huge logs even
> just running one test, let alone the whole batch of
> > them. Also I assume this test fails sporadically which means it needs to
> be run over and over. Doing that with the
> > entire job will take a long time. But if it's too much of a pain to
> isolate the test, then OK.
>
> The suite takes aprox 90 from start to finish and it's just a matter of
> adding a single string to the parameters when we start the build. This
> failure has happened 2 of the 3 jobs we ran. That's easiest for now,
> and splitting up the karaf logs after the fact shouldn't be too bad.
>
> Thanks,
> JamO
> ___
> controller-dev mailing list
> controller-dev@lists.opendaylight.org
> https://lists.opendaylight.org/mailman/listinfo/controller-dev
>


-- 
Thanks
Anil
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease neon failed to build sal-inmemory-datastore from controller

[Thanh Ha]

On Thu, Sep 20, 2018 at 7:05 AM Robert Varga  wrote:

> On 20/09/2018 10:34, Ariel Adam wrote:
> > Tom, looks like a compilation failure:
> >
> > *00:28:35* [INFO]
> -
> > *00:28:35* [INFO]
> -
> > *00:28:35* [ERROR] COMPILATION ERROR : *00:28:35* [INFO]
> -
> > *00:28:35* [ERROR]
> >
> /w/workspace/autorelease-release-neon/controller/opendaylight/md-sal/sal-inmemory-datastore/src/main/java/org/opendaylight/controller/md/sal/dom/store/impl/InMemoryDOMDataStore.java:[38,20]
> > no suitable constructor found for
> >
> InMemoryDOMDataStore(java.lang.String,org.opendaylight.mdsal.common.api.LogicalDatastoreType,java.util.concurrent.ExecutorService,int,boolean)
> >
> >
> > What do you think?
>
> autorelease.git is not updated to pull correct revision of mdsal.git --
> trailing by two patches.
>

This patch should fix that https://git.opendaylight.org/gerrit/76310

That should not have slipped through, we have a daily task that checks for
submodule updates that's supposed to be ensuring we are not out of sync.
Likely the job is not working as we expect it to.

Regards,
Thanh
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease neon failed to build sal-inmemory-datastore from controller

[Robert Varga]

On 20/09/2018 10:34, Ariel Adam wrote:
> Tom, looks like a compilation failure:
> 
> *00:28:35* [INFO] 
> -
> *00:28:35* [INFO] 
> -
> *00:28:35* [ERROR] COMPILATION ERROR : *00:28:35* [INFO] 
> -
> *00:28:35* [ERROR]
> /w/workspace/autorelease-release-neon/controller/opendaylight/md-sal/sal-inmemory-datastore/src/main/java/org/opendaylight/controller/md/sal/dom/store/impl/InMemoryDOMDataStore.java:[38,20]
> no suitable constructor found for
> InMemoryDOMDataStore(java.lang.String,org.opendaylight.mdsal.common.api.LogicalDatastoreType,java.util.concurrent.ExecutorService,int,boolean)
> 
> 
> What do you think?

autorelease.git is not updated to pull correct revision of mdsal.git --
trailing by two patches.

Bye,
Robert



signature.asc
Description: OpenPGP digital signature
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] owner changed failure

[Vratko Polák]

> It really isn't necessary that the new owner stays the same


For some applications it does not matter.

For other applications (such as openflow)

each owner change means the old owner has to disconnect

from the device and the new owner has to connect.

That can be expensive, and in worst case it can lead to data loss.


> I'm not sure if it's guaranteed the owner won't change after
> re-join


I know this test happened to work when we developed it,

but that behavior might depend on timing details

of an underlying algorithm implementation.


For curiosity, I looked at unit tests.

I have not found anything conclusive,

but I found one comment [0] related to timing.


Vratko.


[0] 
https://github.com/opendaylight/controller/blob/9bce68c4712d00951d121be68b09578bc6e09151/opendaylight/md-sal/sal-distributed-datastore/src/test/java/org/opendaylight/controller/cluster/datastore/entityownership/DistributedEntityOwnershipIntegrationTest.java#L224-L226


From: Jamo Luhrsen 
Sent: Thursday, September 20, 2018 7:45:12 AM
To: Tom Pantelis
Cc: controller-dev; pgup...@cisco.com; Vratko Polák
Subject: Re: owner changed failure



On 09/19/2018 02:36 PM, Tom Pantelis wrote:
>
>
> On Wed, Sep 19, 2018 at 4:46 PM Jamo Luhrsen  > wrote:
>
> \
>  > Anyway we'll need to enable debug for 
> org.opendaylight.controller.cluster.datastore.entityownership.  I would
> suggest to
>  > pull out that test on its own like you've done before (run it 
> standalone in sandbox I guess). Also delete the log
> files
>  > in between each run. This will make debugging much easier.
>
> Is there not enough info in the existing logs?
>
>
> No - not with the default INFO logging. In order to dig deeper we need to 
> enable targeted debug, in this
> case org.opendaylight.controller.cluster.datastore.entityownership.

I don't think it worked. Here is what the logger config lines looked
like:

22:38:53  
log4j2.logger.org_opendaylight_org_opendaylight_controller_cluster_datastore_entityownership.name
 =
org.opendaylight.org.opendaylight.controller.cluster.datastore.entityownership
22:38:53  
log4j2.logger.org_opendaylight_org_opendaylight_controller_cluster_datastore_entityownership.level
 = DEBUG

you can see the whole org.ops4j.pax.logging.cfg file here:
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/console-timestamp.log.gz

but, I don't see any DEBUG messages showing up in these karaf logs:

https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/odl_1/odl1_karaf.log.gz
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/odl_2/odl2_karaf.log.gz
https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/controller-csit-3node-clustering-ask-all-oxygen/3/odl_3/odl3_karaf.log.gz


> You can easily trim the
> karaf logs based on the test cases. We are logging the start of every
> suite and test case.
>
>
> I think it's  much easier and faster to debug a failing test if it's 
> isolated. Of course the logs are much smaller and
> don't require trimming.  Enabling debug can result in huge logs even just 
> running one test, let alone the whole batch of
> them. Also I assume this test fails sporadically which means it needs to be 
> run over and over. Doing that with the
> entire job will take a long time. But if it's too much of a pain to isolate 
> the test, then OK.

The suite takes aprox 90 from start to finish and it's just a matter of
adding a single string to the parameters when we start the build. This
failure has happened 2 of the 3 jobs we ran. That's easiest for now,
and splitting up the karaf logs after the fact shouldn't be too bad.

Thanks,
JamO
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev

Re: [controller-dev] [release] Autorelease neon failed to build sal-inmemory-datastore from controller

[Ariel Adam]

Tom, looks like a compilation failure:

*00:28:35* [INFO]
-*00:28:35*
[INFO] -*00:28:35*
[ERROR] COMPILATION ERROR : *00:28:35* [INFO]
-*00:28:35*
[ERROR] 
/w/workspace/autorelease-release-neon/controller/opendaylight/md-sal/sal-inmemory-datastore/src/main/java/org/opendaylight/controller/md/sal/dom/store/impl/InMemoryDOMDataStore.java:[38,20]
no suitable constructor found for
InMemoryDOMDataStore(java.lang.String,org.opendaylight.mdsal.common.api.LogicalDatastoreType,java.util.concurrent.ExecutorService,int,boolean)


What do you think?

Thanks.

On Thu, Sep 20, 2018 at 3:28 AM Jenkins 
wrote:

> Attention controller-devs,
>
> Autorelease neon failed to build sal-inmemory-datastore from controller in
> build
> 44. Attached is a snippet of the error message related to the
> failure that we were able to automatically parse as well as console logs.
>
>
> Console Logs:
>
> https://logs.opendaylight.org/releng/vex-yul-odl-jenkins-1/autorelease-release-neon/44
>
> Jenkins Build:
> https://jenkins.opendaylight.org/releng/job/autorelease-release-neon/44/
>
> Please review and provide an ETA on when a fix will be available.
>
> Thanks,
> ODL releng/autorelease team
>
> ___
> release mailing list
> rele...@lists.opendaylight.org
> https://lists.opendaylight.org/mailman/listinfo/release
>
___
controller-dev mailing list
controller-dev@lists.opendaylight.org
https://lists.opendaylight.org/mailman/listinfo/controller-dev