Hello!
I would like to see how OVN ACLs are being matched against traffic.
And at the first attempt i'm gonna use ACL logs. It looks fine but has
performance overhead - about 30% for 2 hw (550mbps vs 400mbps - no
logs vs with logs)
As I see it is possible to sample ACL traffic via IPFIX and psample.
Since psample works in kernel-space I expect much less overhead.
But I faced with strange (for me) behavior: number of samples packets
is not equal to (it is less than) logged.
How could I get the same behavior for ACL samples?
I need to get samples for each ACL match as it works with ACL logs.
Thank you!
The setup:
2 hw, each runs vm: vm1 on hw1 and vm2 on hw2, each hw has OVS and
ovn-controller
1 control-node witn OVN NB, SB and northd
2 logical switches, 2 logical switch ports (one per LS, one per vm -
each vm on its own hw), 1 logical router
each LSP connected to LR, so I can send and receive traffic from one
vm to another
LSP1: vm1.hw1: 192.168.10.11 66:e1:67:af:49:51
LRP1: router.port1 192.168.10.1 52:55:00:7b:32:91
LRP2: router.port2 192.168.20.1 52:55:00:25:9f:27
LSP2: vm2.hw2 192.168.20.12 76:0d:59:e4:9c:e1
2 ACLs with logging (one for egress and one for ingress):
_uuid : d8796705-7a73-42d3-bf08-9b7241a76bb7
action : allow-related
direction : from-lport
label : 0
log : true
match : "tcp && (tcp.dst == 9000)"
meter : []
name : allow-out
options : {}
priority : 32767
sample_est : []
sample_new : []
severity : debug
tier : 0
_uuid : 309dc28a-e6c0-4fab-a3b3-38baf5dc11e5
action : allow-related
direction : to-lport
label : 0
log : true
match : "tcp && (tcp.dst == 9000)"
meter : []
name : port-9000
options : {}
priority : 32766
sample_est : []
sample_new : []
severity : debug
tier : 0
Next I start server on vm2:
nc -lp 9000
and open the connection from vm1 to vm1:
nc -p 11112 192.168.20.12 9000
I see handshake connection from vm1.hw1:11112 to vm2.hw2:9000 in
ovn-controller logs on hw1
// allow-out: vm1.hw1 --SYN--> router.port1
2025-12-16T22:02:26.879Z|00294|acl_log(ovn_pinctrl0)|INFO|name="allow-out",
verdict=allow, severity=debug, direction=from-lport:
tcp,vlan_tci=0x0000,dl_src=66:e1:67:af:49:51,dl_dst=52:55:00:7b:32:91,nw_src=192.168.10.11,nw_dst=192.168.20.12,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,tp_src=11112,tp_dst=9000,tcp_flags=syn
// port-9000: vm1.hw1 --SYN--> router.port1
2025-12-16T22:02:26.880Z|00295|acl_log(ovn_pinctrl0)|INFO|name="port-9000",
verdict=allow, severity=debug, direction=to-lport:
tcp,vlan_tci=0x0000,dl_src=66:e1:67:af:49:51,dl_dst=52:55:00:7b:32:91,nw_src=192.168.10.11,nw_dst=192.168.20.12,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,tp_src=11112,tp_dst=9000,tcp_flags=syn
// allow-out: router.port2 --SYN--> vm2.hw2
2025-12-16T22:02:26.880Z|00296|acl_log(ovn_pinctrl0)|INFO|name="allow-out",
verdict=allow, severity=debug, direction=from-lport:
tcp,vlan_tci=0x0000,dl_src=52:55:00:25:9f:27,dl_dst=76:0d:59:e4:9c:e1,nw_src=192.168.10.11,nw_dst=192.168.20.12,nw_tos=0,nw_ecn=0,nw_ttl=63,nw_frag=no,tp_src=11112,tp_dst=9000,tcp_flags=syn
// ... should receive SYN-ACK here - no logs
// allow-out: vm1.hw1 --ACK--> router.port1
2025-12-16T22:02:26.881Z|00297|acl_log(ovn_pinctrl0)|INFO|name="allow-out",
verdict=allow, severity=debug, direction=from-lport:
tcp,vlan_tci=0x0000,dl_src=66:e1:67:af:49:51,dl_dst=52:55:00:7b:32:91,nw_src=192.168.10.11,nw_dst=192.168.20.12,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,tp_src=11112,tp_dst=9000,tcp_flags=ack
// port-9000: vm1.hw1 --ACK--> router.port1
2025-12-16T22:02:26.882Z|00298|acl_log(ovn_pinctrl0)|INFO|name="port-9000",
verdict=allow, severity=debug, direction=to-lport:
tcp,vlan_tci=0x0000,dl_src=66:e1:67:af:49:51,dl_dst=52:55:00:7b:32:91,nw_src=192.168.10.11,nw_dst=192.168.20.12,nw_tos=0,nw_ecn=0,nw_ttl=64,nw_frag=no,tp_src=11112,tp_dst=9000,tcp_flags=ack
// allow-out: router.port2 --ACK--> vm2.hw2
2025-12-16T22:02:26.882Z|00299|acl_log(ovn_pinctrl0)|INFO|name="allow-out",
verdict=allow, severity=debug, direction=from-lport:
tcp,vlan_tci=0x0000,dl_src=52:55:00:25:9f:27,dl_dst=76:0d:59:e4:9c:e1,nw_src=192.168.10.11,nw_dst=192.168.20.12,nw_tos=0,nw_ecn=0,nw_ttl=63,nw_frag=no,tp_src=11112,tp_dst=9000,tcp_flags=ack
Now I setup ACL samples and psample. Run psample agent on hw1 to catch
psample messages.
And here what I see.
1. If I setup only sample_new for both ACL, I see only next packets:
// sample_new allow-out: packet_crc=12fc vm1.hw1 --SYN--> router.port1
// sample_new port-9000: packet_crc=12fc vm1.hw1 --SYN--> router.port1
2. If I set up sample_new+sample_est for egress ACL (uuid=d8796705
name=allow-out), and sample_new for ingress ACL (uuid=309dc28a
name=port-9000) I see next packets:
// sample_new allow-out: packet_crc=4daf vm1.hw1 --SYN--> router.port1
// sample_new port-9000: packet_crc=4daf vm1.hw1 --SYN--> router.port1
// sample_est allow-out: packet_crc=32d4 vm1.hw1 <--SYN+ACK-- router.port1
// sample_est allow-out: packet_crc=0628 vm1.hw1 --ACK--> router.port1
// sample_new port-9000: packet_crc=0628 vm1.hw1 --ACK--> router.port1
3. If I set up sample_new for egress ACL (uuid=d8796705
name=allow-out), and sample_new+sample_est for ingress ACL
(uuid=309dc28a name=port-9000) I see next packets:
// sample_new allow-out: packet_crc=a1bf vm1.hw1 --SYN--> router.port1
// sample_new port-9000: packet_crc=a1bf vm1.hw1 --SYN--> router.port1
// sample_est port-9000: packet_crc=4896 vm1.hw1 --ACK--> router.port1
4. If I set up sample_new+sample_est for each ACL, I see next packets:
// sample_new allow-out: packet_crc=993d vm1.hw1 --SYN--> router.port1
// sample_new port-9000: packet_crc=993d vm1.hw1 --SYN--> router.port1
// sample_est port-9000: packet_crc=4c4e vm1.hw1 <--SYN+ACK-- router.port1
// sample_new port-9000: packet_crc=3cd1 vm1.hw1 --ACK--> router.port1
Now lets see what is going on on datapath.
Find logical flows for the ACL rules:
// for ingress ACL uuid=309dc28a name=port-9000
ovn-sbctl find Logical_Flow external_ids:stage-hint=309dc28a
_uuid : cff79706-0ec5-455e-9b2c-e2fc6622247f
actions : "log(name=\"port-9000\", severity=debug,
verdict=allow); reg8[16] = 1; reg0[1] = 1; reg0[13] = 1; reg3 = 1002;
reg9 = 1004; reg8[0..7] = 4; reg8[8..15] = 4; reg8[19..20] = 2; next;"
controller_meter : []
external_ids : {source="northd.c:7027", stage-hint="309dc28a",
stage-name=ls_out_acl_eval}
flow_desc : []
logical_datapath : []
logical_dp_group : 519b350b-5f83-4ffb-919f-d725100dabb6
match : "reg8[30..31] == 0 && reg0[7] == 1 && (tcp &&
(tcp.dst == 9000))"
pipeline : egress
priority : 33766
table_id : 6
tags : {}
hash : 0
_uuid : fd7f2a45-9622-40d1-b0ed-3bdf92a3530c
actions : "log(name=\"port-9000\", severity=debug,
verdict=allow); reg8[16] = 1; reg0[1] = 1; reg0[13] = 1; reg3 = 1002;
reg9 = 1004; reg8[0..7] = 4; reg8[8..15] = 4; reg8[19..20] = 2; next;"
controller_meter : []
external_ids : {source="northd.c:7051", stage-hint="309dc28a",
stage-name=ls_out_acl_eval}
flow_desc : []
logical_datapath : []
logical_dp_group : 519b350b-5f83-4ffb-919f-d725100dabb6
match : "reg8[30..31] == 0 && reg0[8] == 1 && (tcp &&
(tcp.dst == 9000))"
pipeline : egress
priority : 33766
table_id : 6
tags : {}
hash : 0
// for egress ACL uuid=d8796705 name=allow-out
ovn-sbctl find Logical_Flow external_ids:stage-hint=d8796705
_uuid : 41c54125-53cf-4ad2-b355-9446a1f22fb4
actions : "log(name=\"allow-out\", severity=debug,
verdict=allow); reg8[16] = 1; reg0[13] = 1; reg3 = 1001; reg9 = 0;
reg8[0..7] = 4; reg8[8..15] = 0; reg8[19..20] = 0; next;"
controller_meter : []
external_ids : {source="northd.c:7051", stage-hint=d8796705,
stage-name=ls_in_acl_eval}
flow_desc : []
logical_datapath : []
logical_dp_group : 519b350b-5f83-4ffb-919f-d725100dabb6
match : "reg8[30..31] == 0 && reg0[8] == 1 && (tcp &&
(tcp.dst == 9000))"
pipeline : ingress
priority : 33767
table_id : 8
tags : {}
hash : 0
_uuid : 9762c985-6f8e-4158-9552-f866455e5e52
actions : "log(name=\"allow-out\", severity=debug,
verdict=allow); reg8[16] = 1; reg0[1] = 1; reg0[13] = 1; reg3 = 1001;
reg9 = 0; reg8[0..7] = 4; reg8[8..15] = 0; reg8[19..20] = 0; next;"
controller_meter : []
external_ids : {source="northd.c:7027", stage-hint=d8796705,
stage-name=ls_in_acl_eval}
flow_desc : []
logical_datapath : []
logical_dp_group : 519b350b-5f83-4ffb-919f-d725100dabb6
match : "reg8[30..31] == 0 && reg0[7] == 1 && (tcp &&
(tcp.dst == 9000))"
pipeline : ingress
priority : 33767
table_id : 8
tags : {}
hash : 0
Next, dump corresponded flows on hw1
ovs-ofctl dump-flows br-int | grep -P
'41c54125|9762c985|cff79706|fd7f2a45' | grep -P 'n_packets=[^0]'
cookie=0x9762c985, duration=37342.273s, table=16, n_packets=3,
n_bytes=222, idle_age=36753,
priority=33767,tcp,reg0=0x80/0x80,reg8=0/0xc0000000,metadata=0x1,tp_dst=9000
actions=controller(userdata=00.00.00.07.00.00.00.00.00.07.61.6c.6c.6f.77.2d.6f.75.74),load:0x1->OXM_OF_PKT_REG4[48],load:0x1->NXM_NX_XXREG0[97],load:0x1->NXM_NX_XXREG0[109],load:0x3e9->NXM_NX_XXREG0[0..31],load:0->OXM_OF_PKT_REG4[0..31],load:0x4->OXM_OF_PKT_REG4[32..39],load:0->OXM_OF_PKT_REG4[40..47],load:0->OXM_OF_PKT_REG4[51..52],resubmit(,17)
cookie=0x41c54125, duration=37342.273s, table=16, n_packets=15,
n_bytes=1014, idle_age=36726,
priority=33767,tcp,reg0=0x100/0x100,reg8=0/0xc0000000,metadata=0x2,tp_dst=9000
actions=controller(userdata=00.00.00.07.00.00.00.00.00.07.61.6c.6c.6f.77.2d.6f.75.74),load:0x1->OXM_OF_PKT_REG4[48],load:0x1->NXM_NX_XXREG0[109],load:0x3e9->NXM_NX_XXREG0[0..31],load:0->OXM_OF_PKT_REG4[0..31],load:0x4->OXM_OF_PKT_REG4[32..39],load:0->OXM_OF_PKT_REG4[40..47],load:0->OXM_OF_PKT_REG4[51..52],resubmit(,17)
cookie=0x41c54125, duration=37342.273s, table=16, n_packets=12,
n_bytes=792, idle_age=36726,
priority=33767,tcp,reg0=0x100/0x100,reg8=0/0xc0000000,metadata=0x1,tp_dst=9000
actions=controller(userdata=00.00.00.07.00.00.00.00.00.07.61.6c.6c.6f.77.2d.6f.75.74),load:0x1->OXM_OF_PKT_REG4[48],load:0x1->NXM_NX_XXREG0[109],load:0x3e9->NXM_NX_XXREG0[0..31],load:0->OXM_OF_PKT_REG4[0..31],load:0x4->OXM_OF_PKT_REG4[32..39],load:0->OXM_OF_PKT_REG4[40..47],load:0->OXM_OF_PKT_REG4[51..52],resubmit(,17)
cookie=0xfd7f2a45, duration=37466.169s, table=51, n_packets=17,
n_bytes=1154, idle_age=36726,
priority=33766,tcp,reg0=0x100/0x100,reg8=0/0xc0000000,metadata=0x1,tp_dst=9000
actions=controller(userdata=00.00.00.07.00.00.00.00.00.07.70.6f.72.74.2d.39.30.30.30),load:0x1->OXM_OF_PKT_REG4[48],load:0x1->NXM_NX_XXREG0[97],load:0x1->NXM_NX_XXREG0[109],load:0x3ea->NXM_NX_XXREG0[0..31],load:0x3ec->OXM_OF_PKT_REG4[0..31],load:0x4->OXM_OF_PKT_REG4[32..39],load:0x4->OXM_OF_PKT_REG4[40..47],load:0x2->OXM_OF_PKT_REG4[51..52],resubmit(,52)
After opening connection from vm1 to vm2
nc -p 11112 192.168.20.12 9000
cookie=0x9762c985, duration=37358.186s, table=16, n_packets=4, ...
cookie=0x41c54125, duration=37358.186s, table=16, n_packets=17, ...
cookie=0x41c54125, duration=37358.186s, table=16, n_packets=13, ...
cookie=0xfd7f2a45, duration=37482.084s, table=51, n_packets=19, ...
So as you see n_packets is increased - total by 6 - and it is equal to
ACL log records.
--
//wbr, Dmitry L.
_______________________________________________
discuss mailing list
[email protected]
https://mail.openvswitch.org/mailman/listinfo/ovs-discuss
