[ovs-dev] [PATCH ovn] ovn-northd: bypass ct for allow ACLs
For allow ACLs, bypass connection tracking by avoiding setting ct
hints for matching traffic. Avoid sending all traffic to ct when a
stateful ACL is present. Before the patch, this unnecessarily hit
performance when mixed ACL action types were used for the same
datapath.
===
For performance measurements, ovn-fake-multinode environment and qperf
were used. Performance measured between two virtual nodes, two ports
that belong to different LSs connected via router. Using qperf,
performance was measured for UDP, TCP, SCTP protocols (using
_lat and _bw tests). The qperf version used:
0.4.9-16.fc31.x86_64. Each test scenario was executed five times and
averages compared.
Tests were executed with `allow` rules for the tested protocol and
`allow-related` for another protocol set for both ports, both
directions, e.g. for TCP scenario, the following ACLs were defined:
ovn-nbctl acl-add sw0 to-lport 100 tcp allow
ovn-nbctl acl-add sw0 from-lport 100 tcp allow
ovn-nbctl acl-add sw1 to-lport 100 tcp allow
ovn-nbctl acl-add sw1 from-lport 100 tcp allow
ovn-nbctl acl-add sw0 to-lport 100 sctp allow-related
ovn-nbctl acl-add sw0 from-lport 100 sctp allow-related
ovn-nbctl acl-add sw1 to-lport 100 sctp allow-related
ovn-nbctl acl-add sw1 from-lport 100 sctp allow-related
In this particular environment, improvement was seen in send_bw,
latency, and msg_rate measurements, where applicable, for all three
protocols under test.
for UDP, send_bw: 293.6 MB/sec => 313.2 MB/sec (+6.68%)
latency: 16 us => 14.08 us (-12%)
msg_rate: 62.56 K/sec => 71.06 K/sec (+13.59%)
for TCP, latency: 18.6 us => 14.88 us (-20%)
msg_rate: 53.8 K/sec => 67.28 K/sec (+25.06%)
for SCTP, latency: 21.98 us => 19.42 us (-11.65%)
msg_rate: 45.58 K/sec => 51.54 K/sec (+13.08%)
Interestingly, some performance improvement was also seen for the same
scenarios with no ACLs set at all, albeit significantly more
negligible.
for UDP, send_bw: 320.0 MB/sec => 338.6 MB/sec (+5.81%)
latency: 13.74 us => 12.88 us (-6.68%)
msg_rate: 73.02 K/sec => 77.84 K/sec (+6.6%)
for TCP, latency: 15.62 us => 14.26 us (-9.54%)
msg_rate: 64.02 K/sec => 70.26 K/sec (+9.75%)
for SCTP, latency: 19.56 us => 18.16 us (-7.16%)
msg_rate: 51.16 K/sec => 55.12 K/sec (+7.74%)
Comparable numbers can be captured with iperf. It may be useful to run
more tests in a more elaborate (bare metal) environment.
===
The patch takes inspiration from a now abandoned patch:
"ovn-northd: Support mixing stateless/stateful ACLs with
Stateless_Filter." by Dumitru Ceara.
Signed-off-by: Ihar Hrachyshka
---
v1: initial version.
v2: rebased after conflict.
---
NEWS| 1 +
northd/ovn-northd.8.xml | 9 +-
northd/ovn-northd.c | 166 +++
tests/ovn-northd.at | 287
4 files changed, 400 insertions(+), 63 deletions(-)
diff --git a/NEWS b/NEWS
index 530c5d42f..548a45fb7 100644
--- a/NEWS
+++ b/NEWS
@@ -7,6 +7,7 @@ Post-v21.03.0
(This may take testing and tuning to be effective.) This version of OVN
requires DDLog 0.36.
- Introduce ovn-controller incremetal processing engine statistics
+ - Bypass connection tracking for ACL "allow" action processing.
OVN v21.03.0 - 12 Mar 2021
-
diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml
index a62f5c057..f38d71682 100644
--- a/northd/ovn-northd.8.xml
+++ b/northd/ovn-northd.8.xml
@@ -419,7 +419,9 @@
before eventually advancing to ingress table ACLs. If
special ports such as route ports or localnet ports can't use ct(), a
priority-110 flow is added to skip over stateful ACLs. IPv6 Neighbor
- Discovery and MLD traffic also skips stateful ACLs.
+ Discovery and MLD traffic also skips stateful ACLs. For stateless "allow"
+ ACLs, a flow is added to bypass setting the hint for connection tracker
+ processing.
@@ -603,10 +605,7 @@
allow ACLs translate into logical flows with
-the next; action. If there are any stateful ACLs
-on this datapath, then allow ACLs translate to
-ct_commit; next; (which acts as a hint for the next tables
-to commit the connection to conntrack),
+the next; action.
allow-related ACLs translate into logical
diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c
index 4783e43d7..cd343a3e3 100644
--- a/northd/ovn-northd.c
+++ b/northd/ovn-northd.c
@@ -4943,7 +4943,58 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct
ovn_port *op,
}
static void
-build_pre_acls(struct ovn_datapath *od, struct hmap *lflows)
+build_stateless_filter(struct ovn_datapath *od,
+ const struct nbrec_acl *acl,
+ struct hmap *lflows)
+{
+/* Stateless filters must be applied in both directions so that reply
+ * traffic bypasses conntrack too.
+ */
[ovs-dev] [PATCH ovn] ovn-northd: bypass ct for allow ACLs
For allow ACLs, bypass connection tracking by avoiding setting ct
hints for matching traffic. Avoid sending all traffic to ct when a
stateful ACL is present. Before the patch, this unnecessarily hit
performance when mixed ACL action types were used for the same
datapath.
===
For performance measurements, ovn-fake-multinode environment and qperf
were used. Performance measured between two virtual nodes, two ports
that belong to different LSs connected via router. Using qperf,
performance was measured for UDP, TCP, SCTP protocols (using
_lat and _bw tests). The qperf version used:
0.4.9-16.fc31.x86_64. Each test scenario was executed five times and
averages compared.
Tests were executed with `allow` rules for the tested protocol and
`allow-related` for another protocol set for both ports, both
directions, e.g. for TCP scenario, the following ACLs were defined:
ovn-nbctl acl-add sw0 to-lport 100 tcp allow
ovn-nbctl acl-add sw0 from-lport 100 tcp allow
ovn-nbctl acl-add sw1 to-lport 100 tcp allow
ovn-nbctl acl-add sw1 from-lport 100 tcp allow
ovn-nbctl acl-add sw0 to-lport 100 sctp allow-related
ovn-nbctl acl-add sw0 from-lport 100 sctp allow-related
ovn-nbctl acl-add sw1 to-lport 100 sctp allow-related
ovn-nbctl acl-add sw1 from-lport 100 sctp allow-related
In this particular environment, improvement was seen in send_bw,
latency, and msg_rate measurements, where applicable, for all three
protocols under test.
for UDP, send_bw: 293.6 MB/sec => 313.2 MB/sec (+6.68%)
latency: 16 us => 14.08 us (-12%)
msg_rate: 62.56 K/sec => 71.06 K/sec (+13.59%)
for TCP, latency: 18.6 us => 14.88 us (-20%)
msg_rate: 53.8 K/sec => 67.28 K/sec (+25.06%)
for SCTP, latency: 21.98 us => 19.42 us (-11.65%)
msg_rate: 45.58 K/sec => 51.54 K/sec (+13.08%)
Interestingly, some performance improvement was also seen for the same
scenarios with no ACLs set at all, albeit significantly more
negligible.
for UDP, send_bw: 320.0 MB/sec => 338.6 MB/sec (+5.81%)
latency: 13.74 us => 12.88 us (-6.68%)
msg_rate: 73.02 K/sec => 77.84 K/sec (+6.6%)
for TCP, latency: 15.62 us => 14.26 us (-9.54%)
msg_rate: 64.02 K/sec => 70.26 K/sec (+9.75%)
for SCTP, latency: 19.56 us => 18.16 us (-7.16%)
msg_rate: 51.16 K/sec => 55.12 K/sec (+7.74%)
Comparable numbers can be captured with iperf. It may be useful to run
more tests in a more elaborate (bare metal) environment.
===
The patch takes inspiration from a now abandoned patch:
"ovn-northd: Support mixing stateless/stateful ACLs with
Stateless_Filter." by Dumitru Ceara.
Signed-off-by: Ihar Hrachyshka
---
NEWS| 1 +
northd/ovn-northd.8.xml | 9 +-
northd/ovn-northd.c | 166 +++
tests/ovn-northd.at | 287
4 files changed, 400 insertions(+), 63 deletions(-)
diff --git a/NEWS b/NEWS
index 3037a8e66..d25f30c3b 100644
--- a/NEWS
+++ b/NEWS
@@ -6,6 +6,7 @@ Post-v21.03.0
expected to scale better than the C implementation, for large deployments.
(This may take testing and tuning to be effective.) This version of OVN
requires DDLog 0.36.
+ - Bypass connection tracking for ACL "allow" action processing.
OVN v21.03.0 - 5 Mar 2020
-
diff --git a/northd/ovn-northd.8.xml b/northd/ovn-northd.8.xml
index a16937a21..55a3eec23 100644
--- a/northd/ovn-northd.8.xml
+++ b/northd/ovn-northd.8.xml
@@ -385,7 +385,9 @@
before eventually advancing to ingress table ACLs. If
special ports such as route ports or localnet ports can't use ct(), a
priority-110 flow is added to skip over stateful ACLs. IPv6 Neighbor
- Discovery and MLD traffic also skips stateful ACLs.
+ Discovery and MLD traffic also skips stateful ACLs. For stateless "allow"
+ ACLs, a flow is added to bypass setting the hint for connection tracker
+ processing.
@@ -569,10 +571,7 @@
allow ACLs translate into logical flows with
-the next; action. If there are any stateful ACLs
-on this datapath, then allow ACLs translate to
-ct_commit; next; (which acts as a hint for the next tables
-to commit the connection to conntrack),
+the next; action.
allow-related ACLs translate into logical
diff --git a/northd/ovn-northd.c b/northd/ovn-northd.c
index ac872aade..a2e1b9920 100644
--- a/northd/ovn-northd.c
+++ b/northd/ovn-northd.c
@@ -4943,7 +4943,58 @@ skip_port_from_conntrack(struct ovn_datapath *od, struct
ovn_port *op,
}
static void
-build_pre_acls(struct ovn_datapath *od, struct hmap *lflows)
+build_stateless_filter(struct ovn_datapath *od,
+ const struct nbrec_acl *acl,
+ struct hmap *lflows)
+{
+/* Stateless filters must be applied in both directions so that reply
+ * traffic bypasses conntrack too.
+ */
+ovn_lflow_add_with_hint(lflows, od, S_SWI
