Hi Ilya,
could you please take a look on this?
Maybe you can advice any direction how to investigate this issue?
Thanks in advance.
Regards,
Vladislav Odintsov
> On 24 Nov 2022, at 21:10, Anton Vazhnetsov <dragen15...@gmail.com> wrote:
>
> Hi, Terry!
>
> In continuation to our yesterday’s conversation [0], we were able to
> reproduce the issue with KeyError. We found that the problem is not connected
> with ovsdb-server load but it appears when the ovsdb-server schema is
> converted online (it even doesn’t matter whether the real ovs schema is
> changed) while the active connection persists.
> Please use next commands to reproduce it:
>
> # in terminal 1
>
> ovsdb-tool create ./ovs.db /usr/share/ovn/ovn-nb.ovsschema
> ovsdb-server --remote punix://$(pwd)/ovs.sock $(pwd)/ovs.db -vconsole:dbg
>
>
> # in terminal 2. run python shell
> python3
> # setup connection
> import ovsdbapp.schema.ovn_northbound.impl_idl as nb_idl
> from ovsdbapp.backend.ovs_idl import connection
>
> remote = "unix:///<path to ovs.sock>"
>
> def get_idl():
> """Connection getter."""
>
> idl = connection.OvsdbIdl.from_server(remote, "OVN_Northbound",
> leader_only=False)
> return nb_idl.OvnNbApiIdlImpl(connection.Connection(idl, 100))
>
> idl = get_idl()
>
>
> # in terminal 1
> ovsdb-client convert unix:$(pwd)/ovs.sock /usr/share/ovn/ovn-nb.ovsschema
>
> # in terminal 2 python shell:
> idl.ls_add("test").execute()
>
>
> We get following traceback:
>
> Traceback (most recent call last):
> File
> "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/connection.py",
> line 131, in run
> txn.results.put(txn.do_commit())
> File
> "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
> line 143, in do_commit
> self.post_commit(txn)
> File
> "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/transaction.py",
> line 73, in post_commit
> command.post_commit(txn)
> File "/usr/lib/python3.6/site-packages/ovsdbapp/backend/ovs_idl/command.py",
> line 94, in post_commit
> row = self.api.tables[self.table_name].rows[real_uuid]
> File "/usr/lib64/python3.6/collections/__init__.py", line 991, in __getitem__
> raise KeyError(key)
> KeyError: UUID('0256afa4-6dd0-4c2c-b6a2-686a360ab331')
>
> In ovsdb-server debug logs we see that update2 or update3 messages are not
> sent from server in response to client’s transaction, just reply with result
> UUID:
> 2022-11-24T17:42:36Z|00306|poll_loop|DBG|wakeup due to [POLLIN] on fd 18
> (///root/ovsdb-problem/ovs.sock<->) at lib/stream-fd.c:157
> 2022-11-24T17:42:36Z|00307|jsonrpc|DBG|unix#5: received request,
> method="transact",
> params=["OVN_Northbound",{"uuid-name":"row03ef28d6_93f1_43bc_b07a_eae58d4bd1c5","table":"Logical_Switch","op":"insert","row":{"name”:"test"}}],
> id=5
> 2022-11-24T17:42:36Z|00308|jsonrpc|DBG|unix#5: send reply,
> result=[{"uuid":["uuid","4eb7c407-beec-46ca-b816-19f942e57721"]}], id=5
>
> We checked same with ovn-nbctl running in daemon mode and found that the
> problem is not reproduced (ovsdb-server after database conversion sends out
> update3 message to ovn-nbctl daemon process in response to transaction, for
> example ovs-appctl -t <nbctl daemon socket> run ls-add test-ls):
> 2022-11-24T17:54:51Z|00623|jsonrpc|DBG|unix#7: received request,
> method="transact",
> params=["OVN_Northbound",{"uuid-name":"rowcdb152ce_a9af_4761_b965_708ad300fcb7","table":"Logical_Switch","op":"insert","row":{"name":"test-ls"}},{"comment":"ovn-nbctl:
> run ls-add test-ls","op":"comment"}], id=5
> 2022-11-24T17:54:51Z|00624|jsonrpc|DBG|unix#7: send notification,
> method="update3",
> params=[["monid","OVN_Northbound"],"00000000-0000-0000-0000-000000000000",{"Logical_Switch":{"0b147f2c-248d-496a-b718-a5328d3c2995":{"insert":{"name":"test-ls"}}}}]
> 2022-11-24T17:54:51Z|00625|jsonrpc|DBG|unix#7: send reply,
> result=[{"uuid":["uuid","0b147f2c-248d-496a-b718-a5328d3c2995"]},{}], id=5
>
> So it seems that the problem is in python-ovs, not in ovsdb-server.
> We tested with ovsdb-server running version 2.17.3 and python-ovs 2.13.5 and
> also python-ovs 2.17.3, the behaviour is the same.
>
> Do you have any ideas what can be a reason for such behaviour?
>
> 0:
> https://review.opendev.org/c/openstack/ovsdbapp/+/865454/comments/674c57e6_3849591b
>
> Regards, Anton.
_______________________________________________
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
