Hi Numan, both 3187b9fef124e038e474270a2728fe94bdca8eef (ovn-northd: introduce new allow-stateless ACL verb) and 127bf166ccf4a2509f670c48a00b0340039f20d2 (northd: Support flow offloading for logical switches with no ACLs.) got merged in upstream master, and this combination broke the following tests:
774: ovn -- ACL allow-stateless omit conntrack - Port_Group -- ovn-northd-ddlog -- dp-groups=yes FAILED (ovn-northd.at:2752) 775: ovn -- ACL allow-stateless omit conntrack - Port_Group -- ovn-northd-ddlog FAILED (ovn-northd.at:2752) while the other scenarios are passing: 768: ovn -- ACL allow-stateless omit conntrack - Logical_Switch -- ovn-northd -- dp-groups=yes ok 769: ovn -- ACL allow-stateless omit conntrack - Logical_Switch -- ovn-northd ok 770: ovn -- ACL allow-stateless omit conntrack - Logical_Switch -- ovn-northd-ddlog -- dp-groups=yes ok 771: ovn -- ACL allow-stateless omit conntrack - Logical_Switch -- ovn-northd-ddlog ok 772: ovn -- ACL allow-stateless omit conntrack - Port_Group -- ovn-northd -- dp-groups=yes ok 773: ovn -- ACL allow-stateless omit conntrack - Port_Group -- ovn-northd ok These scenarios (both ok and FAILED) were added with allow-stateless patch. If I revert "northd: Support flow offloading for logical switches with no ACLs.", all tests pass. Two things to note: 1) only ddlog tests fail; 2) only port_group scenarios fail while logical_switch counterparts don't. Scenarios fail with the following message in testsuite.log: +2021-05-13T01:06:37Z|00001|ovntrace|WARN|lsp1: unknown logical port +2021-05-13T01:06:37Z|00002|ovntrace|WARN|microflow does not specify ingress port This is because SB database Port_Binding table is empty when ovn-trace is executed. In ddlog northd log, I see the following actions: 2021-05-13T01:06:37.902Z|00113|jsonrpc|DBG|unix:/home/ihrachys/dev/ovn/tests/testsuite.dir/774/ovn-nb/ovn-nb.sock: received notification, method="update", params=[["monid","OVN_Northbound"],{"Port_Group":{"9ad4d0eb-e643-43e3-aba9-bf80e5349622":{"new":{"ports":["set",[["uuid","04169cef-5ba2-46ee-803c-559e448f9e0e"],["uuid","edd3283e-a056-49d5-9412-6422f01c66df"]]],"name":"pg","external_ids":["map",[]],"acls":["set",[["uuid","02193db0-0b05-4078-a46f-905e12585a22"],["uuid","0277809c-c018-4a35-9100-6f24102ec204"],["uuid","1580f03b-0fb3-4a4b-866b-70c23f8029c5"],["uuid","2a62a6ad-f7f7-42e8-a757-0e5c5215cd0f"],["uuid","2be534fe-8646-4f17-9cec-9cdff8cd1d21"],["uuid","4a714f8c-b901-4ba6-99cd-6150c715f758"],["uuid","7783fd4d-d28f-4636-a191-99c8bd611761"],["uuid","fbfa736e-03d0-4924-9c14-dd3dbc9bb743"]]]},"old":{"acls":["set",[["uuid","02193db0-0b05-4078-a46f-905e12585a22"],["uuid","0277809c-c018-4a35-9100-6f24102ec204"],["uuid","1580f03b-0fb3-4a4b-866b-70c23f8029c5"],["uuid","2a62a6ad-f7f7-42e8-a757-0e5c5215cd0f"],["uuid","2be534fe-8646-4f17-9cec-9cdff8cd1d21"],["uuid","4a714f8c-b901-4ba6-99cd-6150c715f758"],["uuid","7783fd4d-d28f-4636-a191-99c8 bd611761"]]]}}},"ACL":{"fbfa736e-03d0-4924-9c14-dd3dbc9bb743":{"new":{"name":["set",[]],"priority":1,"log":false,"external_ids":["map",[]],"direction":"to-lport","meter":["set",[]],"action":"allow-stateless","match":"tcp","severity":["set",[]]}}}}] 2021-05-13T01:06:37.902Z|00114|jsonrpc|DBG|unix:/home/ihrachys/dev/ovn/tests/testsuite.dir/774/ovn-nb/ovn-nb.sock: received notification, method="update", params=[["monid","OVN_Northbound"],{"NB_Global":{"ae62228c-aa3d-479a-a251-e612e38e7fdc":{"new":{"name":"","sb_cfg_timestamp":1620867997820,"hv_cfg":1,"nb_cfg":2,"external_ids":["map",[]],"options":["map",[["mac_prefix","06:30:c8"],["max_tunid","16711680"],["northd_internal_version","21.03.90-20.17.0-56.0"],["svc_monitor_mac","c2:cb:ea:d4:18:86"],["use_logical_dp_groups","true"]]],"sb_cfg":1,"ssl":["set",[]],"ipsec":false,"hv_cfg_timestamp":0,"connections":["set",[]],"nb_cfg_timestamp":1620867997730},"old":{"nb_cfg":1}}}}] 2021-05-13T01:06:37.934Z|00115|jsonrpc|DBG|unix:/home/ihrachys/dev/ovn/tests/testsuite.dir/774/ovn-sb/ovn-sb.sock: received reply, result=[{"uuid":["uuid","1409c320-3e56-2ea6-9f96-ec17e491f2b2"]},{"uuid":["uuid","212b5074-33d6-6f20-d3d4-453b01bb7484"]},{},{}], id=23 2021-05-13T01:06:37.934Z|00116|jsonrpc|DBG|unix:/home/ihrachys/dev/ovn/tests/testsuite.dir/774/ovn-nb/ovn-nb.sock: send request, method="transact", params=["OVN_Northbound",{"where":[["_uuid","==",["uuid","ae62228c-aa3d-479a-a251-e612e38e7fdc"]]],"table":"NB_Global","op":"update","row":{"ipsec":false,"hv_cfg":2,"hv_cfg_timestamp":0,"sb_cfg":1,"options":["map",[["mac_prefix","06:30:c8"],["max_tunid","16711680"],["northd_internal_version","21.03.90-20.17.0-56.0"],["svc_monitor_mac","c2:cb:ea:d4:18:86"],["use_logical_dp_groups","true"]]],"nb_cfg_timestamp":1620867997903}},{"comment":"ovn-northd-ddlog","op":"comment"}], id=24 2021-05-13T01:06:37.934Z|00117|jsonrpc|DBG|unix:/home/ihrachys/dev/ovn/tests/testsuite.dir/774/ovn-sb/ovn-sb.sock: send request, method="transact", params=["OVN_Southbound",{"where":[["_uuid","==",["uuid","04169cef-5ba2-46ee-803c-559e448f9e0e"]]],"op":"delete","table":"Port_Binding"},{"where":[["_uuid","==",["uuid","edd3283e-a056-49d5-9412-6422f01c66df"]]],"op":"delete","table":"Port_Binding"},{"where":[["_uuid","==",["uuid","ae62228c-aa3d-479a-a251-e612e38e7fdc"]]],"table":"SB_Global","op":"update","row":{"ipsec":false,"nb_cfg":2,"options":["map",[["mac_prefix","06:30:c8"],["max_tunid","16711680"],["northd_internal_version","21.03.90-20.17.0-56.0"],["svc_monitor_mac","c2:cb:ea:d4:18:86"],["use_logical_dp_groups","true"]]]}},{"uuid":"00a9f1f3-9ea2-99ba-ba0e-fafe925c097e","table":"Logical_Flow","op":"insert","row":{"pipeline":"egress","priority":1001,"external_ids":["map",[["stage-hint","fbfa736e"],["stage-name","ls_out_pre_acl"]]],"actions":"next;","table_id":1,"logical_dp_group":["set",[]],"match":"tcp","logical_datapath":["set",[["uuid","4f0f8080-2932-4fb7-a078-35dcdc79f008"]]]}},{"where":[["_uuid","==",["uuid","0d2adc0c-4db9- d75b-5eb0-8010400304c7"]]],"op":"delete","table":"Logical_Flow"},{"uuid":"19887afd-d967-793f-e5ad-371d082ff81b","table":"Logical_Flow","op":"insert","row":{"pipeline":"egress","priority":1001,"external_ids":["map",[["stage-hint","fbfa736e"],["stage-name","ls_out_acl"]]],"actions":"next;","table_id":4,"logical_dp_group":["set",[]],"match":"tcp","logical_datapath":["set",[["uuid","4f0f8080-2932-4fb7-a078-35dcdc79f008"]]]}},{"where":[["_uuid","==",["uuid","56ede221-43b9-b92d-e14a-c45ea67ca519"]]],"op":"delete","table":"Logical_Flow"},{"where":[["_uuid","==",["uuid","621eae93-4679-97b2-8c1b-fe38d11fe6a6"]]],"op":"delete","table":"Logical_Flow"},{"where":[["_uuid","==",["uuid","80975751-13c8-3a5c-073b-7627fecf7050"]]],"op":"delete","table":"Logical_Flow"},{"comment":"ovn-northd-ddlog","op":"comment"},{"lock":"ovn_northd","op":"assert"}], id=25 Note the last action where all port bindings are dropped. I couldn't figure out what triggers it, so I went and reverted bits of your patch trying to understand what triggers that. And it seems like the change that breaks it is in lswitch.dl, where &Switch.has_acls is initialized through LogicalSwitchHasACLs(ls._uuid, has_acls). When I remove this line (and a bunch of others to make it compile), tests pass again. I tried to revert just northd.dl changes and tests still fail unless I revert the changes in lswitch.dl. I am ignorant to know what it really means. Perhaps you have some ideas? I also found that if I add the following lines in the test case, then it passes too: for direction in from to; do ovn-nbctl acl-del pg ${direction}-lport 3 tcp done This goes before ov-nbctl acl-add pg ${direction}-lport 1 tcp allow-stateless Since ddlog is declarative, it's hard to debug it. Ideas where to go from here? PS: while digging this issue, I also realized that when the new allow-stateless rules are mixed with allow-related, their priorities are not properly honoured (meaning, allow-stateless rules with a lower priority still beat allow-related rules with a higher priority). But this seems like a separate issue. Figured I better mention it regardless in case it's of relevance. Thanks for reading, Ihar _______________________________________________ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev