Hi!
We've met two new problems:
1. On a test corrupted DB with two broken references checks reported only of
them.
There are two broken refs from two "virtual_machine_interface" to the same
(nonexistent) "routing_instance"
virtual_machine_interface:ee0e2cd7-dcbc-411b-81f3-9408b58bca7e --(ref)-->
routing_instance:c54f7813-b813-44f5-bad7-001679bc8783
virtual_machine_interface:453a323a-d069-4a44-b29d-539b77b16d6d --(ref)-->
routing_instance:c54f7813-b813-44f5-bad7-001679bc8783
Checks report:
broken references
virtual_machine_interface/453a323a-d069-4a44-b29d-539b77b16d6d ->
routing_instance/c54f7813-b813-44f5-bad7-001679bc8783
Broken ref from ee0e.. even not loaded into graph:
gremlin> g.V('ee0e2cd7-dcbc-411b-81f3-9408b58bca7e').outE()
==>e[832][ee0e2cd7-dcbc-411b-81f3-9408b58bca7e-parent->73e52d96-de4b-433a-bb2e-cf5abc7c0668]
==>e[556][ee0e2cd7-dcbc-411b-81f3-9408b58bca7e-ref->9a2dde17-52cf-4f0f-a8e0-ce981dc5d374]
==>e[1004][ee0e2cd7-dcbc-411b-81f3-9408b58bca7e-ref->7b87cd81-cd5c-4a93-aa14-fa09d643823b]
But ref it exists in Cassandra:
$ cqlsh -e "SELECT blobAsText(key), blobAsText(column1) FROM
config_db_uuid.obj_uuid_table WHERE
key=textAsBlob('ee0e2cd7-dcbc-411b-81f3-9408b58bca7e')" | grep routing_instance
ee0e2cd7-dcbc-411b-81f3-9408b58bca7e |
ref:routing_instance:c54f7813-b813-44f5-bad7-001679bc8783
2. When running checks on a DB with heavy (test) load (creating/removing
networks, interfaces, VMs) false positives are generated:
E.g., A --ref--> B.
1. check reads A from cassandra
2. A and B are removed
3. check does not find B and reports it as broken ref
Didn't you meet the same problem with changing DB?
09.11.2017, 13:46, "Jean-Philippe Braun" <[email protected]>:
> That's indeed quite strange! I will have a look to handle such case.
>
> Did the checks yield any results ?
>
> JP
>
> Le Friday 03 Nov 2017 à 12:20:30 (+0500), [email protected] a écrit :
>> Thanks for ideas!
>>
>> I've found that some backrefs in obj_uuid_table to 'instance_ip' are
>> duplicated with strange backrefs to 'instance-ip':
>>
>>
>> obj_uuid_table:d4f5b4c5-50ab-4989-a131-c6288125c616,backref:instance_ip:094be277-eac2-4e50-88ed-c689c6c39f73,{is_weakref:
>> false, attr: null}
>>
>> obj_uuid_table:d4f5b4c5-50ab-4989-a131-c6288125c616,backref:instance-ip:094be277-eac2-4e50-88ed-c689c6c39f73,{is_weakref:
>> false, attr: null}
>>
>> After removal of backrefs with 'instance-ip', the database has been
>> imported successfully.
>> Now trying to find where in the code such backrefs can appear.
>>
>> 27.10.2017, 20:42, "Jean-Philippe Braun" <[email protected]>:
>> > Thanks for trying out the tool,
>> >
>> > From my understanding the dump wasn't properly generated, if you have
>> > the possibility of sharing me the dump file I could have a look at it.
>> >
>> > Since your are running the docker image, you can find the dump.json file
>> > from the mountpoint in the host system, something like:
>> >
>> /var/lib/docker/devicemapper/mnt/8e791d0973b92ab926675831ef08eeb68ed3d8cb66741e5c28f668f38e1e0e3c/rootfs/srv/apache-tinkerpop-gremlin-console/
>> >
>> > If not, it means that two different edges with the same id were created,
>> > its something that shouldn't happen. You can open the dump with a text
>> > editor and look for that particular id and see to which resources theses
>> > edges refers to. It might reflect an issue in the contrail DB.
>> >
>> > I've never encounter this kind of issue on our platforms though.
>> >
>> > JP
>> >
>> > Le Friday 27 Oct 2017 à 20:03:44 (+0500), [email protected] a
>> écrit :
>> >> Hello!
>> >>
>> >> Thank you for the tools!
>> >>
>> >> Trying to run contrail-fsck on a test db, get an error:
>> >> "Error in checks.groovy at [8:
>> g.io(graphson()).readGraph(graphFilename);] -
>> >> Edge with id already exists: 1694"
>> >>
>> >> What further steps would you recommend for troubleshooting?
>> >>
>> >> 18.10.2017, 23:07, "Illia Polliul" <[email protected]>:
>> >>
>> >> That utility looks very helpful! I’ll definitely check it, it can
>> be a
>> >> life-saver at certain moments.
>> >> My appreciation for all contrail-api-cli stuff
>> >>
>> >> Best regards,
>> >> Illia Polliul
>> >> Sr. Deployment Engineer | Mirantis, Inc.
>> >> Skype: ilya_pollul
>> >>
>> >> On Oct 17, 2017, at 1:02 PM, Jean-Philippe Braun
>> <[email protected]>
>> >> wrote:
>> >>
>> >> Hi all,
>> >>
>> >> Since several months @cloudwatt we worked on some tools to be
>> able to
>> >> check the consistency of our contrail config DB on our different
>> >> platforms.
>> >>
>> >> As you know the contrail data model is a graph with resources
>> connected
>> >> to each others with parent or ref links so it felt natural to
>> use a
>> >> graph db to process the config DB. We choose the tinkerpop
>> stack[1]
>> >> that
>> >> provide a language to traverse graphs: gremlin.
>> >>
>> >> You will find the different tools we worked on the repo
>> >> https://github.com/eonpatapon/contrail-gremlin.
>> >>
>> >> We also provide a simple to use docker image to run the
>> consistency
>> >> checks[2] we have. It's a good entrypoint to discover the
>> project. More
>> >> info at:
>> https://github.com/eonpatapon/contrail-gremlin/tree/master/
>> >> docker.
>> >>
>> >> If you have any comments or feedback, don't hesitate
>> >>
>> >> Thanks
>> >>
>> >> Jean-Philippe
>> >>
>> >> [1] http://tinkerpop.apache.org/
>> >> [2] https://github.com/eonpatapon/contrail-gremlin/blob/master/
>> >> gremlin-checks/checks.groovy
>> >> _______________________________________________
>> >> Dev mailing list
>> >> [email protected]
>> >> http://lists.opencontrail.org/mailman/listinfo/
>> >> dev_lists.opencontrail.org
>> >>
>> >> ,
>> >>
>> >> _______________________________________________
>> >> Dev mailing list
>> >> [email protected]
>> >>
>> http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
_______________________________________________
Dev mailing list
[email protected]
http://lists.opencontrail.org/mailman/listinfo/dev_lists.opencontrail.org
