Re: [ovs-dev] [PATCH] ofproto: Bridge del causing trafic drop for all VM.

[Adrián Moreno]

On Mon, Jan 20, 2025 at 01:28:42PM +0530, Vipul Ashri via dev wrote:
> RCA: This issue was reported in multi-bridge OVS environment, having
> dynamic bridges. Each VM is tied with one bridge via vhu and a VM
> add/del causes bridge add/del. We found individual bridge deletion,
> while deleting VM was impacting overall OVS traffic, causing temp
> traffic outage even for other unrelated VMs. The packets are dropped
> with datapath_drop_lock_error coverage counter. We found that when
> deleting a bridge, we disable upcall for some time and flush entire
> dpctl flows. As we donot have per bridge dpctl flow, so any packets
> coming during this period of time will miss dpctl flow and go for
> upcall, which being disabled now will be dropped.
>
> Fix is to  bypass "ofproto->ofproto_class->flush(ofproto)", once
> this callback holds the lock and dp flows flushed, leads to complete
> traffic outage. As per my analysis no need to do udpif_flush() to
> flush all datapath flows instead needed flows will be auto-deleted
> for required bridge with port delete or destroy APIs already called
> further in codeflow and bridge add/delete paths.
>
> Signed-off-by: Vipul Ashri 
> ---
>  ofproto/ofproto.c | 7 ++-
>  1 file changed, 6 insertions(+), 1 deletion(-)
>
> diff --git a/ofproto/ofproto.c b/ofproto/ofproto.c
> index 3df64efb9..0f0e20269 100644
> --- a/ofproto/ofproto.c
> +++ b/ofproto/ofproto.c
> @@ -1839,7 +1839,12 @@ ofproto_destroy(struct ofproto *p, bool del)
>  return;
>  }
>
> -ofproto_flush__(p, del);
> +/* Changed arg from "del" -> "false" to fix traffic outage during
> + * individual bridge deletion.

Hi Vipul.

We don't need this log-type comment. Git will take care of showing what
was the previous value.

> + * DP flows cleanup shall be taken care automatically during individual
> + * port delete/destruct while processing evicting flows.
> + */
> +ofproto_flush__(p, false);

I understand your use-case and the inconvenience of bridge deletion
causing a full datapath flush. In this case, it does make sense to just
remove the ofproto part of the bridge and let the revalidators take care
of the datapath flows.

However, I think there is a bit more to it.
To start with, this change will result is "ofproto_flush__" to always be
called with "false", which makes me wonder if we'd need an argument at
all.

Also, I think we should look carefully at the other case: when we're
tearing down the entire datapath to see if we could create a race or
some strange behavior by leaving the datapath flows while we tear
_everything_ down. I have not done enough tests (nor have seen them done
by someone else) be confident here.

Third, this patch tries to skip calling:
ofproto->ofproto_class->flush(ofproto);

This function seems only called here and its commend seems to me rather
vague:

/* Every "struct rule" in 'ofproto' is about to be deleted, one by one.
 * This function may prepare for that, for example by clearing state in
 * advance.  It should *not* actually delete any "struct rule"s from
 * 'ofproto', only prepare for it.
 *
 * This function is optional; it's really just for optimization in case
 * it's cheaper to delete all the flows from your hardware in a single pass
 * than to do it one by one. */

The ideal scenario would be for this function to actually remove the
flows that belong to this ofproto but it might not be very clean to do
this and, unless we add support of bulk flushing in the datapath,
would be done "one by one".

'Every "struct rule" in 'ofproto' is about to be deleted, one by one.'

This happens in ofproto_class->destruct. Which means we should be able
to safely assume this is called inmediately before destruct() which ends
up calling "close_dpif_backer" who, smartly, does not whipe out the
entire datapath if there is some other ofproto (i.e: bridge) using it:

static void
close_dpif_backer(struct dpif_backer *backer, bool del)
{
struct simap_node *node;

ovs_assert(backer->refcount > 0);

if (--backer->refcount) {
return;
}

An idea would be to do the same, i.e: only flush if "backer->refcount > 1".

Final comment: this patch should have a test.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v1] dpif-netdev: Modify code comments for function parse_tcp_flags.

[Adrián Moreno]

On Mon, Jan 13, 2025 at 03:52:03PM +0800, Allen Chen via dev wrote:
> Fixes: e7e9973b80d3 ("dpif-netdev: Forwarding optimization for flows with a 
> simple match")
>
> Signed-off-by: Allen Chen 

Thanks for the patch, LGTM.
Reviewed-by: Adrian Moreno 

> ---
>  lib/flow.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
>
> diff --git a/lib/flow.c b/lib/flow.c
> index 9be437524..0eb34892f 100644
> --- a/lib/flow.c
> +++ b/lib/flow.c
> @@ -1187,7 +1187,7 @@ parse_dl_type(const void **datap, size_t *sizep, 
> ovs_be16 *first_vlan_tci_p)
>   * If 'packet' is not an Ethernet packet embedding TCP, returns 0.
>   * 'dl_type_p' will be set only if the 'packet' is an Ethernet packet.
>   * 'nw_frag_p' will be set only if the 'packet' is an IP packet.
> - * 'first_vlan_tci' will be set only if the 'packet' contains vlan header.
> + * 'first_vlan_tci_p' will be set only if the 'packet' contains vlan header.
>   *
>   * The caller must ensure that 'packet' is at least ETH_HEADER_LEN bytes
>   * long.'*/
> --
> 2.33.0.windows.2
>
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v4] utilities: Add upcall statistics to the kernel_delay.py script.

[Adrián Moreno]

On Thu, Dec 19, 2024 at 03:29:43PM +0100, Eelco Chaudron wrote:
> This patch installs a kernel return probe on ovs_dp_upcall() to record
> all successful and failed calls per CPU. The statistics are reported
> when the script exits, providing insights into the upcall behavior.
>
> This is an example output:
>
>   # UPCALL STATISTICS (TOTAL [CPU_ID: N_UPCALL_PER_CPU, ...]):
> Total upcalls   : 183247 [0: 8937, 2: 14464, 4: 10372, 6: 4254, ...]
> Successfull upcalls : 120195 [0: 5945, 2: 6379, 4: 5966, 6: 4254, ...]
> Failed upcalls  : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
>11, EAGAIN   : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
>
> Signed-off-by: Eelco Chaudron 
>

Looks good to me, thanks.

Reviewed-by: Adrian Moreno 
Tested-by: Adrian Moreno 

> ---
>  v4: - Fixed no failures case. Updated header.
>  v3: - Fixed flake8 error.
>  v2: - Cleanup and use per-CPU hash vs per-CPU array for counters.
> ---
>  utilities/automake.mk   |1
>  utilities/usdt-scripts/kernel_delay.py  |  136 
> +++
>  utilities/usdt-scripts/kernel_delay.rst |9 ++
>  3 files changed, 145 insertions(+), 1 deletion(-)
>
> diff --git a/utilities/automake.mk b/utilities/automake.mk
> index acc1af4e0..22260b254 100644
> --- a/utilities/automake.mk
> +++ b/utilities/automake.mk
> @@ -149,6 +149,7 @@ FLAKE8_PYFILES += utilities/ovs-pcap.in \
>   utilities/ovs-pipegen.py \
>   utilities/usdt-scripts/dpif_op_nl_monitor.py \
>   utilities/usdt-scripts/flow_reval_monitor.py \
> + utilities/usdt-scripts/kernel_delay.py \
>   utilities/usdt-scripts/upcall_monitor.py \
>   utilities/usdt-scripts/upcall_cost.py
>
> diff --git a/utilities/usdt-scripts/kernel_delay.py 
> b/utilities/usdt-scripts/kernel_delay.py
> index de6b0c9de..367d27e34 100755
> --- a/utilities/usdt-scripts/kernel_delay.py
> +++ b/utilities/usdt-scripts/kernel_delay.py
> @@ -39,6 +39,7 @@
>  #
>  import argparse
>  import datetime
> +import errno
>  import os
>  import pytz
>  import psutil
> @@ -556,6 +557,36 @@ TRACEPOINT_PROBE(irq, softirq_exit)
>  data->start_ns = 0;
>  return 0;
>  }
> +
> +
> +/*
> + * For measuring upcall statistics (per CPU).
> + */
> +BPF_PERCPU_HASH(upcall_count);
> +
> +#if 
> +int kretprobe__ovs_dp_upcall(struct pt_regs *ctx)
> +{
> +int ret = PT_REGS_RC(ctx);
> +u64 zero = 0;
> +u64 *entry;
> +u64 key;
> +
> +if (!capture_enabled__())
> +return 0;
> +
> +if (ret >= 0)
> +key = 0;
> +else
> +key = -ret;
> +
> +entry = upcall_count.lookup_or_try_init(&key, &zero);
> +if (entry)
> +*entry += 1;
> +
> +return 0;
> +}
> +#endif /* For measuring upcall statistics/errors. */
>  """
>
>
> @@ -887,6 +918,7 @@ def reset_capture():
>  bpf["stack_traces"].clear()
>  bpf["stop_start"].clear()
>  bpf["stop_data"].clear()
> +bpf["upcall_count"].clear()
>
>
>  #
> @@ -900,6 +932,97 @@ def print_timestamp(msg):
>  print(time_string)
>
>
> +#
> +# Get errno short string
> +#
> +def get_errno_short(err):
> +try:
> +return errno.errorcode[err]
> +except KeyError:
> +return "_unknown_"
> +
> +
> +#
> +# Format a eBPF per-cpu hash entry (if the count is > 0)
> +#
> +def format_per_cpu_hash(cpu_hash, key=None, skip_key=None):
> +per_cpu = ""
> +
> +if key is not None:
> +total = cpu_hash.sum(key).value
> +if total > 0:
> +for cpu, value in enumerate(cpu_hash.getvalue(key)):
> +if value == 0:
> +continue
> +
> +per_cpu += " {}: {},".format(cpu, value)
> +else:
> +total = 0
> +total_cpu = None
> +
> +for key in cpu_hash.keys():
> +if skip_key is not None and skip_key.value == key.value:
> +continue
> +
> +if total_cpu is None:
> +total_cpu = [0] * len(cpu_hash.getvalue(key))
> +
> +for cpu, value in enumerate(cpu_hash.getvalue(key)):
> +total_cpu[cpu] += value
> +total += value
> +
> +if total >= 0 and total_cpu:
> +for cpu, value in enumerate(total_cpu):
> +if value == 0:
> +continue
> +
> +per_cpu += " {}: {},".format(cpu, value)
> +
> +return total, per_cpu.strip(", ")
> +
> +
> +#
> +# Display kernel upcall statistics
> +#
> +def display_upcall_results():
> +upcalls = bpf["upcall_count"]
> +have_upcalls = False
> +
> +for k in upcalls:
> +if upcalls.sum(k).value == 0:
> +continue
> +have_upcalls = True
> +break
> +
> +if not have_upcalls:
> +return
> +
> +print("\n\n# UPCALL STATISTICS (TOTAL [CPU_ID: N_UPCALLS_PER_CPU, 
> ...]):\n"
> +"  Total upcalls   : {} [{}]".format(
> +*format_per_cpu_hash(upcalls)))
> +
> +for k in sorted(upcalls, key=lambda x: i

Re: [ovs-dev] [PATCH v3] utilities: Add upcall statistics to the kernel_delay.py script.

[Adrián Moreno]

On Tue, Dec 17, 2024 at 04:21:52PM +0100, Eelco Chaudron wrote:
> This patch installs a kernel return probe on ovs_dp_upcall() to record
> all successful and failed calls per CPU. The statistics are reported
> when the script exits, providing insights into the upcall behavior.
>
> This is an example output:
>
>   # UPCALL STATISTICS (TOTAL [CPU_ID: UPCALL_PER_CPU, ...]):
> Total upcalls   : 183247 [0: 8937, 2: 14464, 4: 10372, 6: 4254, ...]
> Successfull upcalls : 120195 [0: 5945, 2: 6379, 4: 5966, 6: 4254, ...]
> Failed upcalls  : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
>11, EAGAIN   : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
>
> Signed-off-by: Eelco Chaudron 
>
> ---
>  v3: - Fixed flake8 error.
>  v2: - Cleanup and use per-CPU hash vs per-CPU array for counters.
> ---
>  utilities/automake.mk   |1
>  utilities/usdt-scripts/kernel_delay.py  |  132 
> +++
>  utilities/usdt-scripts/kernel_delay.rst |9 ++
>  3 files changed, 141 insertions(+), 1 deletion(-)
>
> diff --git a/utilities/automake.mk b/utilities/automake.mk
> index acc1af4e0..22260b254 100644
> --- a/utilities/automake.mk
> +++ b/utilities/automake.mk
> @@ -149,6 +149,7 @@ FLAKE8_PYFILES += utilities/ovs-pcap.in \
>   utilities/ovs-pipegen.py \
>   utilities/usdt-scripts/dpif_op_nl_monitor.py \
>   utilities/usdt-scripts/flow_reval_monitor.py \
> + utilities/usdt-scripts/kernel_delay.py \
>   utilities/usdt-scripts/upcall_monitor.py \
>   utilities/usdt-scripts/upcall_cost.py
>
> diff --git a/utilities/usdt-scripts/kernel_delay.py 
> b/utilities/usdt-scripts/kernel_delay.py
> index de6b0c9de..9caf4bd62 100755
> --- a/utilities/usdt-scripts/kernel_delay.py
> +++ b/utilities/usdt-scripts/kernel_delay.py
> @@ -39,6 +39,7 @@
>  #
>  import argparse
>  import datetime
> +import errno
>  import os
>  import pytz
>  import psutil
> @@ -556,6 +557,36 @@ TRACEPOINT_PROBE(irq, softirq_exit)
>  data->start_ns = 0;
>  return 0;
>  }
> +
> +
> +/*
> + * For measuring upcall statistics (per CPU).
> + */
> +BPF_PERCPU_HASH(upcall_count);
> +
> +#if 
> +int kretprobe__ovs_dp_upcall(struct pt_regs *ctx)
> +{
> +int ret = PT_REGS_RC(ctx);
> +u64 zero = 0;
> +u64 *entry;
> +u64 key;
> +
> +if (!capture_enabled__())
> +return 0;
> +
> +if (ret >= 0)
> +key = 0;
> +else
> +key = -ret;
> +
> +entry = upcall_count.lookup_or_try_init(&key, &zero);
> +if (entry)
> +*entry += 1;
> +
> +return 0;
> +}
> +#endif /* For measuring upcall statistics/errors. */
>  """
>
>
> @@ -887,6 +918,7 @@ def reset_capture():
>  bpf["stack_traces"].clear()
>  bpf["stop_start"].clear()
>  bpf["stop_data"].clear()
> +bpf["upcall_count"].clear()
>
>
>  #
> @@ -900,6 +932,93 @@ def print_timestamp(msg):
>  print(time_string)
>
>
> +#
> +# Get errno short string
> +#
> +def get_errno_short(err):
> +try:
> +return errno.errorcode[err]
> +except KeyError:
> +return "_unknown_"
> +
> +
> +#
> +# Format a eBPF per-cpu array entry (if the count is > 0)
> +#
> +def format_per_cpu_array(cpu_array, key=None, skip_key=None):

nit: it's no longer an array.

> +per_cpu = ""
> +
> +if key is not None:
> +total = cpu_array.sum(key).value
> +if total > 0:
> +for cpu, value in enumerate(cpu_array.getvalue(key)):
> +if value == 0:
> +continue
> +
> +per_cpu += " {}: {},".format(cpu, value)
> +else:
> +total = 0
> +total_cpu = None
> +
> +for key in cpu_array.keys():
> +if skip_key is not None and skip_key.value == key.value:
> +continue
> +
> +if total_cpu is None:
> +total_cpu = [0] * len(cpu_array.getvalue(key))
> +
> +for cpu, value in enumerate(cpu_array.getvalue(key)):
> +total_cpu[cpu] += value
> +total += value
> +
> +if total >= 0:
> +for cpu, value in enumerate(total_cpu):

Doing some further testing I got this:

Traceback (most recent call last):
  File "/home/amorenoz/src/ovs/utilities/usdt-scripts/./kernel_delay.py",
line 1612, in 
main()
  File "/home/amorenoz/src/ovs/utilities/usdt-scripts/./kernel_delay.py",
line 1571, in main
process_results(syscall_events=syscall_events,
  File "/home/amorenoz/src/ovs/utilities/usdt-scripts/./kernel_delay.py",
line 1198, in process_results
display_upcall_results()
  File "/home/amorenoz/src/ovs/utilities/usdt-scripts/./kernel_delay.py",
line 1014, in display_upcall_results
total, per_cpu = format_per_cpu_array(upcalls, skip_key=k)
 ^
  File "/home/amorenoz/src/ovs/utilities/usdt-scripts/./kernel_delay.py",
line 975, in format_per_cpu_array
for cpu, value in enumerate(total_cpu):
  ^^

Re: [ovs-dev] [PATCH] utilities: Add upcall statistics to the kernel_delay.py script.

[Adrián Moreno]

On Tue, Dec 17, 2024 at 03:30:51PM +0100, Eelco Chaudron wrote:
>
>
> On 13 Dec 2024, at 13:56, Adrián Moreno wrote:
>
> > On Mon, Nov 25, 2024 at 03:14:20PM +0100, Eelco Chaudron wrote:
> >> This patch installs a kernel return probe on ovs_dp_upcall() to record
> >> all successful and failed calls per CPU. The statistics are reported
> >> when the script exits, providing insights into the upcall behavior.
> >>
> >> This is an example output:
> >>
> >>   # UPCALL STATISTICS:
> >> Total upcalls   : 183247 [0: 8937, 2: 14464, 4: 10372, 6: 4254, 
> >> ...]
> >> Successfull upcalls : 120195 [0: 5945, 2: 6379, 4: 5966, 6: 4254, ...]
> >> Failed upcalls  : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
> >>11, EAGAIN   : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
> >>
> >> Signed-off-by: Eelco Chaudron 
> >
> > Thanks for this patch. Correlating upcall failures to kernel delays is
> > extremely useful. Some comments below.
>
> Thanks Adrian for the review. I’ll send out a v2 in a minute, but see my 
> responses below.
>
> //Eelco
>
> >> ---
> >>  utilities/automake.mk   |1
> >>  utilities/usdt-scripts/kernel_delay.py  |  127 
> >> +++
> >>  utilities/usdt-scripts/kernel_delay.rst |9 ++
> >>  3 files changed, 136 insertions(+), 1 deletion(-)
> >>
> >> diff --git a/utilities/automake.mk b/utilities/automake.mk
> >> index acc1af4e0..22260b254 100644
> >> --- a/utilities/automake.mk
> >> +++ b/utilities/automake.mk
> >> @@ -149,6 +149,7 @@ FLAKE8_PYFILES += utilities/ovs-pcap.in \
> >>utilities/ovs-pipegen.py \
> >>utilities/usdt-scripts/dpif_op_nl_monitor.py \
> >>utilities/usdt-scripts/flow_reval_monitor.py \
> >> +  utilities/usdt-scripts/kernel_delay.py \
> >>utilities/usdt-scripts/upcall_monitor.py \
> >>utilities/usdt-scripts/upcall_cost.py
> >>
> >> diff --git a/utilities/usdt-scripts/kernel_delay.py 
> >> b/utilities/usdt-scripts/kernel_delay.py
> >> index de6b0c9de..a5aa1a3d4 100755
> >> --- a/utilities/usdt-scripts/kernel_delay.py
> >> +++ b/utilities/usdt-scripts/kernel_delay.py
> >> @@ -39,6 +39,7 @@
> >>  #
> >>  import argparse
> >>  import datetime
> >> +import errno
> >>  import os
> >>  import pytz
> >>  import psutil
> >> @@ -556,6 +557,30 @@ TRACEPOINT_PROBE(irq, softirq_exit)
> >>  data->start_ns = 0;
> >>  return 0;
> >>  }
> >> +
> >> +
> >> +/*
> >> + * For measuring upcall statistics (per CPU).
> >> + */
> >> +#define MAX_ERROR 255
> >
> > Why this limit? The kernel seems to use 4095 in the PTR_ERR
> > infrastructure. Although not exposed to user applications,
> > kernel has some > 255 values. They don't seem networking related for
> > now but still.
> >
> > In order to reduce this arbitrary limit and avoid needless iteration
> > over an almost empty array, I'd make it a percpu_hash. It supports
> > "lookup_or_try_init" and "increment" so it's as easy to use from ebpf
> > and almost identically from python.
>
> Good point, I will use a per-CPU hash table instead.
>
> >> +BPF_TABLE("percpu_array", uint32_t, uint64_t, upcall_count, MAX_ERROR + 
> >> 1);
> >> +
> >> +#if 
> >> +int kretprobe__ovs_dp_upcall(struct pt_regs *ctx)
> >> +{
> >
> > When trying the script I was a bit confused that numbers didn't add up,
> > e.g:
> >
> > # UPCALL STATISTICS:
> >   Total upcalls   : 776234 [0: 204345, 1: 38773, 2: 38206, 3:
> > 38700, 4: 104040, 5: 77400, 6: 70290, 8: 69705, 9: 67410, 10: 14220,
> > 11: 53145]
> >   Successfull upcalls : 268640 [0: 72829, 1: 14556, 2: 12876, 3:
> > 13192, 4: 34227, 5: 26480, 6: 24130, 8: 23791, 9: 23372, 10: 6164, 11:
> > 17023]
> >   Failed upcalls  : 508564 [0: 131516, 1: 25187, 2: 25330, 3:
> > 25508, 4: 69813, 5: 50920, 6: 46160, 8: 45914, 9: 44038, 10: 8056, 11:
> > 36122]
> >  11, EAGAIN   : 509485 [0: 131516, 1: 26112, 2: 25330, 3:
> > 25508, 4: 69813, 5: 50920, 6: 46160, 8: 45914, 9: 44038, 10: 8056, 11:
> > 36122]
> >
> > Total of EAGAIN erros != total Failed and
> > Total Failed + Total Successfull != Total.
> >
> > I believe the reason is that events of this 

Re: [ovs-dev] [PATCH] utilities: Add upcall statistics to the kernel_delay.py script.

[Adrián Moreno]

On Mon, Nov 25, 2024 at 03:14:20PM +0100, Eelco Chaudron wrote:
> This patch installs a kernel return probe on ovs_dp_upcall() to record
> all successful and failed calls per CPU. The statistics are reported
> when the script exits, providing insights into the upcall behavior.
>
> This is an example output:
>
>   # UPCALL STATISTICS:
> Total upcalls   : 183247 [0: 8937, 2: 14464, 4: 10372, 6: 4254, ...]
> Successfull upcalls : 120195 [0: 5945, 2: 6379, 4: 5966, 6: 4254, ...]
> Failed upcalls  : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
>11, EAGAIN   : 63052 [0: 2992, 2: 8085, 4: 4406, 8: 2275, ...]
>
> Signed-off-by: Eelco Chaudron 

Thanks for this patch. Correlating upcall failures to kernel delays is
extremely useful. Some comments below.

> ---
>  utilities/automake.mk   |1
>  utilities/usdt-scripts/kernel_delay.py  |  127 
> +++
>  utilities/usdt-scripts/kernel_delay.rst |9 ++
>  3 files changed, 136 insertions(+), 1 deletion(-)
>
> diff --git a/utilities/automake.mk b/utilities/automake.mk
> index acc1af4e0..22260b254 100644
> --- a/utilities/automake.mk
> +++ b/utilities/automake.mk
> @@ -149,6 +149,7 @@ FLAKE8_PYFILES += utilities/ovs-pcap.in \
>   utilities/ovs-pipegen.py \
>   utilities/usdt-scripts/dpif_op_nl_monitor.py \
>   utilities/usdt-scripts/flow_reval_monitor.py \
> + utilities/usdt-scripts/kernel_delay.py \
>   utilities/usdt-scripts/upcall_monitor.py \
>   utilities/usdt-scripts/upcall_cost.py
>
> diff --git a/utilities/usdt-scripts/kernel_delay.py 
> b/utilities/usdt-scripts/kernel_delay.py
> index de6b0c9de..a5aa1a3d4 100755
> --- a/utilities/usdt-scripts/kernel_delay.py
> +++ b/utilities/usdt-scripts/kernel_delay.py
> @@ -39,6 +39,7 @@
>  #
>  import argparse
>  import datetime
> +import errno
>  import os
>  import pytz
>  import psutil
> @@ -556,6 +557,30 @@ TRACEPOINT_PROBE(irq, softirq_exit)
>  data->start_ns = 0;
>  return 0;
>  }
> +
> +
> +/*
> + * For measuring upcall statistics (per CPU).
> + */
> +#define MAX_ERROR 255

Why this limit? The kernel seems to use 4095 in the PTR_ERR
infrastructure. Although not exposed to user applications,
kernel has some > 255 values. They don't seem networking related for
now but still.

In order to reduce this arbitrary limit and avoid needless iteration
over an almost empty array, I'd make it a percpu_hash. It supports
"lookup_or_try_init" and "increment" so it's as easy to use from ebpf
and almost identically from python.

> +BPF_TABLE("percpu_array", uint32_t, uint64_t, upcall_count, MAX_ERROR + 1);
> +
> +#if 
> +int kretprobe__ovs_dp_upcall(struct pt_regs *ctx)
> +{

When trying the script I was a bit confused that numbers didn't add up,
e.g:

# UPCALL STATISTICS:
  Total upcalls   : 776234 [0: 204345, 1: 38773, 2: 38206, 3:
38700, 4: 104040, 5: 77400, 6: 70290, 8: 69705, 9: 67410, 10: 14220,
11: 53145]
  Successfull upcalls : 268640 [0: 72829, 1: 14556, 2: 12876, 3:
13192, 4: 34227, 5: 26480, 6: 24130, 8: 23791, 9: 23372, 10: 6164, 11:
17023]
  Failed upcalls  : 508564 [0: 131516, 1: 25187, 2: 25330, 3:
25508, 4: 69813, 5: 50920, 6: 46160, 8: 45914, 9: 44038, 10: 8056, 11:
36122]
 11, EAGAIN   : 509485 [0: 131516, 1: 26112, 2: 25330, 3:
25508, 4: 69813, 5: 50920, 6: 46160, 8: 45914, 9: 44038, 10: 8056, 11:
36122]

Total of EAGAIN erros != total Failed and
Total Failed + Total Successfull != Total.

I believe the reason is that events of this kind are still being
triggered while results are being processed. The problem seems to go
away if I add the following lines at the top of this probe:

if (!capture_enabled__())
return 0;


> +int ret = PT_REGS_RC(ctx);
> +
> +if (ret > 0)
> +ret = 0;
> +
> +ret = -ret;
> +if (ret > MAX_ERROR)
> +ret = MAX_ERROR;
> +
> +upcall_count.increment(ret);
> +return 0;
> +}
> +#endif /* For measuring upcall statistics/errors. */
>  """
>
>
> @@ -900,6 +925,95 @@ def print_timestamp(msg):
>  print(time_string)
>
>
> +#
> +# Get errno short string
> +#
> +def get_errno_short(err):
> +try:
> +return errno.errorcode[err]
> +except KeyError:
> +return "_unknown_"
> +
> +
> +#
> +# Format a bfd per cpu array entry per cpu (if the count is > 0)
> +#

s/bfd/bpf?
Also, "per cpu" twice makes the sentense a bit difficult to read.

> +def format_per_cpu_array(cpu_array, key=None, skip_key=None):
> +per_cpu = ""
> +
> +if key is not None:

"is not None" is redundant: "if key:" is equivalent.

> +total = cpu_array.sum(key).value
> +if total > 0:
> +for cpu, value in enumerate(cpu_array.getvalue(key)):
> +if value == 0:
> +continue
> +
> +per_cpu += " {}: {},".format(cpu, value)
> +else:
> +total = 0
> +total_cpu = None
> +
> +for key in cpu_array.keys():

See below.

> +# I've seen extre

Re: [ovs-dev] ovs-flowviz doc build warning

[Adrián Moreno]

On Fri, Dec 06, 2024 at 08:38:49PM +, Kevin Traynor wrote:
> On 05/12/2024 17:05, Ilya Maximets wrote:
> > On 12/5/24 15:54, Kevin Traynor wrote:
> >> Hi,
> >>
> >> I'm hitting doc build error [1] with Sphinx v1.7.6 on RHEL 8.
> >>
> >> The issue is with duplicate options for ovs-flowviz because the options
> >> are described for different formats. e.g.
> >> # grep 'option:: -h,'  ovs-flowviz.8.rst
> >> .. option:: -h, --help
> >> .. option:: -h, --heat-map
> >> .. option:: -h, --heat-map
> >> .. option:: -h, --html
> >>
> >> I tried to suppress the autosectionlabel warnings but that option was
> >> introduced in Sphinx 2.1 :( There is probably some easy if you know it
> >> fix, but I spent a bit of time searching/trying and I didn't come up
> >> with something. Any ideas ?
> >>
> >
> > Do you know from which version such non-duplication duplications are
> > correctly handled?
> >

In theory, the correct way would be to change the program using the
("program::" directive).
Unfortunately, sphinx does not have notion of subcommand. I guess
something like this should work:

---
diff --git a/Documentation/ref/ovs-flowviz.8.rst
b/Documentation/ref/ovs-flowviz.8.rst
index fc850e1d3..34bfbebde 100644
--- a/Documentation/ref/ovs-flowviz.8.rst
+++ b/Documentation/ref/ovs-flowviz.8.rst
@@ -52,7 +52,7 @@ them in one of the predefined *format*\ s.
 Options
 ===

-.. program: ovs-flowviz
+.. program:: ovs-flowviz

 .. option:: -h, --help

@@ -84,7 +84,7 @@ Options

The filtering syntax is detailed in `Filtering Syntax`_.

-.. option:: -h , --highlight 
+.. option:: -s , --highlight 

Highlight the flows that match the provided *filter* expression.

@@ -151,6 +151,8 @@ option.

 Arguments:

+.. program:: ovs-flowviz [datapath|openflow] console
+
 .. option:: -h, --heat-map

Color of the packet and byte counters to reflect their relative size.
@@ -201,6 +203,9 @@ A logical block is a set of flows that have:

 Arguments:

+
+.. program:: ovs-flowviz openflow logic
+
 .. option:: -s, --show-flows

 Show all the flows under each logical block.
@@ -277,6 +282,8 @@ same tree-like flow hierarchy that the ``tree``
format prints.

 Arguments:

+.. program:: ovs-flowviz datapath console
+
 .. option:: -h, --html

 Print the graphviz format as an svg image alongside an interactive HTML

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2] ofproto: Fix default pmd_id for ofproto/detrace.

[Adrián Moreno]

On Tue, Nov 12, 2024 at 12:56:58PM +0100, Ales Musil wrote:
> The system and netdev datapath have different default pmd_id, which
> resulted in empty output of ofproto/detrace with kernel datapath.
>
> Make sure we use the correct default pmd_id when it's not specified
> as an argument. At the same time move slightly adjusted test into
> system tests so it is tested with both datapaths.
>
> Fixes: 600125b2c380 ("ofproto: Add ofproto/detrace command to map UFIDs to 
> OpenFlow.")
> Signed-off-by: Ales Musil 

Thanks for the patch Ales. It looks good. I've tested and it does solve
the issue in kernel datapath.

Reviewed-by: Adrian Moreno 

> ---
> v2: Rebase on top of latest main.
> Address comments from Dumitru:
> - Fix typo in the commit title.
> - Simplify the mathod for pmd_id.
> - Use proper function to get dpif type.
> ---
>  ofproto/ofproto-dpif-upcall.c | 10 +++-
>  tests/ofproto-dpif.at | 56 ---
>  tests/system-traffic.at   | 86 +++
>  3 files changed, 94 insertions(+), 58 deletions(-)
>
> diff --git a/ofproto/ofproto-dpif-upcall.c b/ofproto/ofproto-dpif-upcall.c
> index e7d4c2b2c..1d8d514b1 100644
> --- a/ofproto/ofproto-dpif-upcall.c
> +++ b/ofproto/ofproto-dpif-upcall.c
> @@ -3338,8 +3338,9 @@ static void
>  upcall_unixctl_ofproto_detrace(struct unixctl_conn *conn, int argc,
> const char *argv[], void *aux OVS_UNUSED)
>  {
> -unsigned int pmd_id = NON_PMD_CORE_ID;
>  const char *key_s = argv[1];
> +const char *pmd_str = NULL;
> +unsigned int pmd_id;
>  ovs_u128 ufid;
>
>  if (odp_ufid_from_string(key_s, &ufid) <= 0) {
> @@ -3348,7 +3349,7 @@ upcall_unixctl_ofproto_detrace(struct unixctl_conn 
> *conn, int argc,
>  }
>
>  if (argc == 3) {
> -const char *pmd_str = argv[2];
> +pmd_str = argv[2];
>  if (!ovs_scan(pmd_str, "pmd=%d", &pmd_id)) {
>  unixctl_command_reply_error(conn,
>  "Invalid pmd argument format. "
> @@ -3361,6 +3362,11 @@ upcall_unixctl_ofproto_detrace(struct unixctl_conn 
> *conn, int argc,
>  struct udpif *udpif;
>
>  LIST_FOR_EACH (udpif, list_node, &all_udpifs) {
> +if (!pmd_str) {
> +const char *type = dpif_normalize_type(dpif_type(udpif->dpif));
> +pmd_id = !strcmp(type, "system") ? PMD_ID_NULL : NON_PMD_CORE_ID;
> +}
> +
>  struct udpif_key *ukey = ukey_lookup(udpif, &ufid, pmd_id);
>  if (!ukey) {
>  continue;
> diff --git a/tests/ofproto-dpif.at b/tests/ofproto-dpif.at
> index 18bd359bf..8f9eab0d2 100644
> --- a/tests/ofproto-dpif.at
> +++ b/tests/ofproto-dpif.at
> @@ -12797,62 +12797,6 @@ Datapath actions: 
> psample(group=42,cookie=0x6400c8),drop
>  OVS_VSWITCHD_STOP("/Enabling an unsupported feature is very dangerous/d")
>  AT_CLEANUP
>
> -AT_SETUP([ofproto-dpif - Dump OF rules corresponding to UFID])
> -OVS_VSWITCHD_START
> -
> -add_of_ports br0 1 2 3
> -
> -dnl Add some OpenFlow rules and groups.
> -AT_DATA([groups.txt], [dnl
> -group_id=1,type=select,selection_method=dp_hash,bucket=bucket_id:0,weight:100,actions=ct(commit,table=2,nat(dst=20.0.0.2))
> -group_id=2,type=all,bucket=resubmit(,3),bucket=resubmit(,4)
> -])
> -AT_DATA([flows.txt], [dnl
> -table=0,priority=100,cookie=0x12345678,in_port=p1,ip,nw_dst=10.0.0.2,actions=resubmit(,1)
> -table=1,priority=200,ip,actions=group:1
> -table=2,ip,actions=group:2
> -table=3,ip,actions=p2
> -table=4,ip,actions=p3
> -])
> -AT_CHECK([ovs-ofctl add-groups br0 groups.txt])
> -AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> -
> -AT_CHECK([ovs-appctl netdev-dummy/receive p1 
> 'ipv4(src=10.0.0.1,dst=10.0.0.2,proto=6),tcp(src=1,dst=2)'])
> -AT_CHECK([ovs-appctl netdev-dummy/receive p1 
> 'ipv4(src=10.0.0.1,dst=10.0.0.2,proto=6),tcp(src=1,dst=2)'])
> -AT_CHECK([ovs-appctl revalidator/wait])
> -AT_CHECK([ovs-appctl revalidator/pause])
> -
> -AT_CHECK([ovs-appctl dpctl/dump-flows | strip_used | strip_stats | 
> strip_duration | strip_dp_hash | sort], [0], [dnl
> -flow-dump from the main thread:
> -recirc_id(0),in_port(1),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(dst=10.0.0.2,frag=no),
>  packets:0, bytes:0, used:0.0s, actions:hash(l4(0)),recirc(0x1)
> -recirc_id(0x1),in_port(1),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no),
>  packets:0, bytes:0, used:0.0s, 
> actions:ct(commit,nat(dst=20.0.0.2)),recirc(0x2)
> -recirc_id(0x2),in_port(1),packet_type(ns=0,id=0),eth_type(0x0800),ipv4(frag=no),
>  packets:0, bytes:0, used:0.0s, actions:2,3
> -])
> -
> -ufid=$(ovs-appctl dpctl/dump-flows -m filter='recirc_id(0)' | parse_ufid)
> -AT_CHECK([ovs-appctl ofproto/detrace $ufid | ofctl_strip], [0], [dnl
> -cookie=0x12345678, n_packets=2, n_bytes=236, 
> priority=100,ip,in_port=1,nw_dst=10.0.0.2,actions=resubmit(,1)
> -table_id=1, n_packets=2, n_bytes=236, priority=200,ip,actions=group:1
> -])
> -
> -ufid=$(ovs-appctl dpctl/dump-flows -m filte

Re: [ovs-dev] How to operate/access ovs by Rust/Go/C?

[Adrián Moreno]

On Wed, Oct 23, 2024 at 02:34:56PM +, Klaus Ma wrote:
> Hi team,
>
> I'd like to learn OVS through the client APIs, e.g. Rust/Go/C. Is there any 
> doc/tutorial/examples for that?
>

C OVSDB APIs (IDL) are generated in the OVS tree. You can look at OVN
as an example. libopenvswitch also contains the
unixctl interface.

In Go I know of libovsdb, used by ovn-kubernetes.

In Rust there seems to be a OVSDB crate as well called "ovsdb".

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH] python: ovs: flowviz: Make datapath tree easier to read.

[Adrián Moreno]

On Fri, Oct 25, 2024 at 12:11:06PM +0200, Ilya Maximets wrote:
> On 10/23/24 16:47, Eelco Chaudron wrote:
> >
> > On 21 Oct 2024, at 21:15, Ilya Maximets wrote:
> >
> >> There are few changes here:
> >>
> >>  * Make a separate recirculation node to not appear in case there is
> >>only one flow with this id.  This saves a nesting level.
> >
> > When reviewing a large trace, I prefer consistency. If one entry has 
> > multiple
> > sub-entries, all of them should have the extra nesting to improve ease of 
> > reading.
>
> I do like consistency, but 8 nesting levels in 4-time recirculation
> do not allow me to understand what I'm looking at efficiently.
> Maybe we can add an option?  e.g. "datapath tree --compact" to hide
> single-node recirculations.  WDYT?

I guess this is very subjective. My eyes are OK with the lack of
indentation consistency. However, sometimes they miss the recirculation
block title that is present in others, i.e: [recirc_id(0x12) in_port(eth0)]

Maybe we could add the title within the same Panel(Group())?

Regarding, "--compact" I think this command has already quite a lot of
options and would prefer not to add more but if we cannot agree on a
style it's probably a good solution.

>
> >
> >>  * Put actions into the same panel as the flow matches.  This saves
> >>another nesting level.
> >
> > I liked the extra separation with the boxes. However, if we decide to 
> > remove the
> > boxes, using the theme color alone would work. If we stick with the boxes, 
> > we could
> > simply remove the indent.
>
> I don't think rich allows to not have an indent.  Since we do have
> different preferences, maybe we could use an option here too, e.g.
> make the --compact affect both the recirculation and action node?
>

Separating the actions with blocks that have many flows. Specially if
dumped with "-m" where that can make individual flows be wrapped.

Maybe adding a newline between flows and actions would help the eye
while avoiding an extra indention level?

If we cannot agree on a style that is good enough for everyone we
can add a compact option but let's at least try to find some middle
grounds.

A combination of my two proposals is this snipped:

diff --git a/python/ovs/flowviz/odp/tree.py b/python/ovs/flowviz/odp/tree.py
index cd781e330..911387cf9 100644
--- a/python/ovs/flowviz/odp/tree.py
+++ b/python/ovs/flowviz/odp/tree.py
@@ -473,18 +473,19 @@ class ConsoleTreeProcessor(FileProcessor):

 visible_blocks = list(node.visible_blocks())

-# If there is only one node with this recirc_id, don't create
-# an extra nesting level, just print the node.
-if len(visible_blocks) == 1:
-self.print_block(visible_blocks[0], parent, recirc_style)
-return
-
 node_text = Text(
 "[recirc_id({}) in_port({})]".format(
 hex(node.recirc), node.in_port
 ),
 style=recirc_style,
 )
+# If there is only one node with this recirc_id, don't create
+# an extra nesting level, just print the node.
+if len(visible_blocks) == 1:
+self.print_block(visible_blocks[0], parent, recirc_style,
+ title=node_text)
+return
+
 console_node = parent.add(
 Panel(node_text, box=TreeBox, border_style=recirc_style),
 guide_style=recirc_style
@@ -493,9 +494,13 @@ class ConsoleTreeProcessor(FileProcessor):
 for block in visible_blocks:
 self.print_block(block, console_node, recirc_style)

-def print_block(self, block, parent, style):
+def print_block(self, block, parent, style, title=None):
 # Print the flow matches and the statistics.
 flow_text = []
+if title:
+flow_text.append(title)
+flow_text.append("\n")
+
 omit_first = {
 "actions": "all",
 }
@@ -520,6 +525,7 @@ class ConsoleTreeProcessor(FileProcessor):
 act_buf.append_extra("actions: ", Style(bold=(self.style is not None)))

 self.ofconsole.format_flow(act_buf, block.flows[0].flow, omitted=omit)
+flow_text.append("\n")
 flow_text.append(act_buf.text)

 flows_node = parent.add(



> >
> >>  * Replace default panel box with a custom one, which is essentially
> >>only the side border with some angles.  This makes the output much
> >>less busy and much easier on eyes, IMO.
> >>
> >>The bottom hook looks a little awkward if there is a guide line
> >>right below it, but it doesn't seem too distracting, and I didn't
> >>find an ASCII top-half of a vertical line symbol.  This one also
> >>looks nice closing the blocks.
> >>
> >>Discontinuous lines of '|' symbols are also a smaller distraction
> >>than a continuous line of '│' symbols.
> >
> > I don't have a strong preference either way. Both options get the job done.
> > For the new mode, I'd even suggest removing the ending |, since

Re: [ovs-dev] [PATCH 1/3] python: Don't install ovs-flowviz.conf as a global data.

[Adrián Moreno]

On Mon, Oct 21, 2024 at 09:23:55PM +0200, Ilya Maximets wrote:
> Specifying data_files makes setuptools install this file to a global
> sys.prefix.  So, it ends up in both the main directory of the python
> package and in the /usr/ovs-flowviz.conf, which is not good:
>
>  warning: install_data: setup script did not provide a directory for
>   'ovs/flowviz/ovs-flowviz.conf' -- installing right in
>   'build/bdist.linux-x86_64/wheel/ovs-3.4.90.data/data'
>  adding 'ovs-3.4.90.data/data/ovs-flowviz.conf'
>
> There is no real point installing this file globally, just use the
> package_data to ensure it is included in the python package.
>
> Signed-off-by: Ilya Maximets 

Reviewed-by: Adrian Moreno 

> ---
>  python/setup.py.template | 3 +--
>  1 file changed, 1 insertion(+), 2 deletions(-)
>
> diff --git a/python/setup.py.template b/python/setup.py.template
> index 415d5c0df..b134694f1 100644
> --- a/python/setup.py.template
> +++ b/python/setup.py.template
> @@ -109,8 +109,7 @@ setup_args = dict(
>  [*flow_extras_require, 'click', 'rich', 'graphviz'],
>  },
>  scripts=["ovs/flowviz/ovs-flowviz"],
> -data_files=["ovs/flowviz/ovs-flowviz.conf"],
> -include_package_data=True,
> +package_data={'ovs.flowviz': ['ovs-flowviz.conf']},
>  )
>
>  try:
> --
> 2.46.0
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH ovn] northd: Commit ct_label.obs_point_id for blocked connections.

[Adrián Moreno]

On Tue, Oct 08, 2024 at 11:51:54AM GMT, Dumitru Ceara wrote:
> On 10/8/24 10:59, Lorenzo Bianconi wrote:
> > Considering the following configuration:
> >
> > $ovn-nbctl acl-list sw01
> > from-lport   100 (inport == "sw01-port1" && udp.dst == 5201) allow-related 
> > [after-lb]
> > from-lport10 (inport == "sw01-port1" && udp) drop [after-lb]
> >
> > $ovn-nbctl list acl
> > _uuid   : e440336a-84d3-4a6d-95a9-edd1db1c3631
> > action  : drop
> > direction   : from-lport
> > external_ids: {}
> > label   : 0
> > log : false
> > match   : "inport == \"sw01-port1\" && udp"
> > meter   : []
> > name: []
> > options : {apply-after-lb="true"}
> > priority: 10
> > sample_est  : ac6a6efc-a2e0-4d68-b5f8-8cd91113e554
> > sample_new  : 5cdad2ab-4390-4772-ac40-74aa2980c06e
> > severity: []
> > tier: 0
> >
> > _uuid   : 85ef08d7-aacc-41d7-b808-6ab011edd753
> > action  : allow-related
> > direction   : from-lport
> > external_ids: {}
> > label   : 0
> > log : false
> > match   : "inport == \"sw01-port1\" && udp.dst == 5201"
> > meter   : []
> > name: []
> > options : {apply-after-lb="true"}
> > priority: 100
> > sample_est  : 143ce7e2-fd13-4d5e-930c-133d5cf87d0d
> > sample_new  : 1d1a0a05-2a8a-4c72-ad35-77d7e2908183
> > severity: []
> > tier: 0
> >
> > If the priority-100 acl is removed, the udp traffic with destination port
> > 5201 will be dropped however ovn-controller will continue sampling the
> > existing connection with the observationPointID associated to the removed
> > acl. Fix the issue updating the ct_label.obs_point_id for the connection
> > marked with ct_mark.blocked.
> >
> > Fixes: d15b12da6fe6 ("northd: Add ACL Sampling.")
> > Repoerted-at: https://issues.redhat.com/browse/FDP-819

small typo: s/Repoerted-at/Reported-at/

> > Signed-off-by: Lorenzo Bianconi 
> >
> > Signed-off-by: Lorenzo Bianconi 
> > ---
>
> Hi Lorenzo,
>
> Could you please also add tests for these scenarios?
>
> >  northd/northd.c | 9 -
> >  1 file changed, 8 insertions(+), 1 deletion(-)
> >
> > diff --git a/northd/northd.c b/northd/northd.c
> > index 2c4703301..d5b9a54b2 100644
> > --- a/northd/northd.c
> > +++ b/northd/northd.c
> > @@ -7205,7 +7205,14 @@ consider_acl(struct lflow_table *lflows, const 
> > struct ovn_datapath *od,
> >  /* For drop ACLs just sample all packets as "new" packets. */
> >  build_acl_sample_label_action(actions, acl, acl->sample_new, NULL,
> >obs_stage);
> > -ds_put_cstr(actions, "ct_commit { ct_mark.blocked = 1; }; next;");
> > +if (acl->sample_est) {
> > +ds_put_format(actions,
> > +  "ct_commit { ct_mark.blocked = 1; "
> > +  "ct_label.obs_point_id = %"PRIu32"; }; next;",
> > +  (uint32_t) acl->sample_est->metadata);
> > +} else {
> > +ds_put_cstr(actions, "ct_commit { ct_mark.blocked = 1; }; 
> > next;");
> > +}
>
> Will this ct_commit change the value of ct_label.obs_point_id to 0?  I
> guess not.  Shouldn't we explicitly do that here?  Otherwise, if the
> drop ACL doesn't have sampling enabled, I'm afraid we'll still generate
> samples with the stale point ID.
>

Maybe a stupid question:
A zero obs_point_id (zero label) is still a valid one from OVS's pov
(probably not for the reader), right? If we cannot avoid emitting the
sample based on the mark, how would we know if the label should be
used or not?

Thanks.
Adrián

> >  ovn_lflow_add_with_hint(lflows, od, stage, priority,
> >  ds_cstr(match), ds_cstr(actions),
> >  &acl->header_, lflow_ref);
>
> Regards,
> Dumitru
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v5 13/13] documentation: Document ovs-flowviz.

[Adrián Moreno]

On Mon, Sep 23, 2024 at 02:54:18PM GMT, Adrián Moreno wrote:
> On Mon, Sep 23, 2024 at 01:00:50PM GMT, Ilya Maximets wrote:
> > On 9/23/24 12:39, Adrián Moreno wrote:
> > > On Tue, Aug 20, 2024 at 12:15:52AM GMT, Ilya Maximets wrote:
> > >> On 7/10/24 19:05, Adrian Moreno wrote:
> > >>> Add a man page for ovs-flowviz as well as a topic page with some more
> > >>> detailed examples.
> > >>>
> > >>> Signed-off-by: Adrian Moreno 
> > >>> ---
> > >>>  Documentation/automake.mk   |   4 +-
> > >>>  Documentation/conf.py   |   2 +
> > >>>  Documentation/ref/index.rst |   1 +
> > >>>  Documentation/ref/ovs-flowviz.8.rst | 535 
> > >>>  Documentation/topics/flow-visualization.rst | 314 
> > >>>  Documentation/topics/index.rst  |   1 +
> > >>>  rhel/openvswitch-fedora.spec.in |   1 +
> > >>>  rhel/openvswitch.spec.in|   1 +
> > >>>  8 files changed, 858 insertions(+), 1 deletion(-)
> > >>>  create mode 100644 Documentation/ref/ovs-flowviz.8.rst
> > >>>  create mode 100644 Documentation/topics/flow-visualization.rst
> > >>
> > >> Hi, Adrian.  Thanks for the update!  I didn't make a full review, since
> > >> Eelco did, but I have a couple comments for the docs and the output
> > >> format below.
> > >>
> > >> Best regards, Ilya Maximets.
> > >>
> > >>>
> > >>> diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> > >>> index 47d2e336a..539870aa2 100644
> > >>> --- a/Documentation/automake.mk
> > >>> +++ b/Documentation/automake.mk
> > >>> @@ -45,7 +45,7 @@ DOC_SOURCE = \
> > >>> Documentation/topics/fuzzing/ovs-fuzzing-infrastructure.rst \
> > >>> Documentation/topics/fuzzing/ovs-fuzzers.rst \
> > >>> 
> > >>> Documentation/topics/fuzzing/security-analysis-of-ovs-fuzzers.rst \
> > >>> -   Documentation/topics/testing.rst \
> > >>> +   Documentation/topics/flow-visualization.rst \
> > >>> Documentation/topics/integration.rst \
> > >>> Documentation/topics/language-bindings.rst \
> > >>> Documentation/topics/networking-namespaces.rst \
> > >>> @@ -55,6 +55,7 @@ DOC_SOURCE = \
> > >>> Documentation/topics/ovsdb-replication.rst \
> > >>> Documentation/topics/porting.rst \
> > >>> Documentation/topics/record-replay.rst \
> > >>> +   Documentation/topics/testing.rst \
> > >>> Documentation/topics/tracing.rst \
> > >>> Documentation/topics/usdt-probes.rst \
> > >>> Documentation/topics/userspace-checksum-offloading.rst \
> > >>> @@ -162,6 +163,7 @@ RST_MANPAGES = \
> > >>> ovs-actions.7.rst \
> > >>> ovs-appctl.8.rst \
> > >>> ovs-ctl.8.rst \
> > >>> +   ovs-flowviz.8.rst \
> > >>> ovs-l3ping.8.rst \
> > >>> ovs-parse-backtrace.8.rst \
> > >>> ovs-pki.8.rst \
> > >>> diff --git a/Documentation/conf.py b/Documentation/conf.py
> > >>> index 15785605a..3a82f23a7 100644
> > >>> --- a/Documentation/conf.py
> > >>> +++ b/Documentation/conf.py
> > >>> @@ -120,6 +120,8 @@ _man_pages = [
> > >>>   u'utility for configuring running Open vSwitch daemons'),
> > >>>  ('ovs-ctl.8',
> > >>>   u'OVS startup helper script'),
> > >>> +('ovs-flowviz.8',
> > >>> + u'utility for visualizing OpenFlow and datapath flows'),
> > >>>  ('ovs-l3ping.8',
> > >>>   u'check network deployment for L3 tunneling problems'),
> > >>>  ('ovs-parse-backtrace.8',
> > >>> diff --git a/Documentation/ref/index.rst b/Documentation/ref/index.rst
> > >>> index 03ada932f..7f2fe6177 100644
> > >>> --- a/Documentation/ref/index.rst
> > >>> +++ b/Documentation/ref/index.rst
> > >>> @@ -42,6 +42,7 @@ time:
> > >>> ovs-actions.7
> > >>> ovs-a

Re: [ovs-dev] [PATCH v5 13/13] documentation: Document ovs-flowviz.

[Adrián Moreno]

On Mon, Sep 23, 2024 at 01:00:50PM GMT, Ilya Maximets wrote:
> On 9/23/24 12:39, Adrián Moreno wrote:
> > On Tue, Aug 20, 2024 at 12:15:52AM GMT, Ilya Maximets wrote:
> >> On 7/10/24 19:05, Adrian Moreno wrote:
> >>> Add a man page for ovs-flowviz as well as a topic page with some more
> >>> detailed examples.
> >>>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  Documentation/automake.mk   |   4 +-
> >>>  Documentation/conf.py   |   2 +
> >>>  Documentation/ref/index.rst |   1 +
> >>>  Documentation/ref/ovs-flowviz.8.rst | 535 
> >>>  Documentation/topics/flow-visualization.rst | 314 
> >>>  Documentation/topics/index.rst  |   1 +
> >>>  rhel/openvswitch-fedora.spec.in |   1 +
> >>>  rhel/openvswitch.spec.in|   1 +
> >>>  8 files changed, 858 insertions(+), 1 deletion(-)
> >>>  create mode 100644 Documentation/ref/ovs-flowviz.8.rst
> >>>  create mode 100644 Documentation/topics/flow-visualization.rst
> >>
> >> Hi, Adrian.  Thanks for the update!  I didn't make a full review, since
> >> Eelco did, but I have a couple comments for the docs and the output
> >> format below.
> >>
> >> Best regards, Ilya Maximets.
> >>
> >>>
> >>> diff --git a/Documentation/automake.mk b/Documentation/automake.mk
> >>> index 47d2e336a..539870aa2 100644
> >>> --- a/Documentation/automake.mk
> >>> +++ b/Documentation/automake.mk
> >>> @@ -45,7 +45,7 @@ DOC_SOURCE = \
> >>>   Documentation/topics/fuzzing/ovs-fuzzing-infrastructure.rst \
> >>>   Documentation/topics/fuzzing/ovs-fuzzers.rst \
> >>>   Documentation/topics/fuzzing/security-analysis-of-ovs-fuzzers.rst \
> >>> - Documentation/topics/testing.rst \
> >>> + Documentation/topics/flow-visualization.rst \
> >>>   Documentation/topics/integration.rst \
> >>>   Documentation/topics/language-bindings.rst \
> >>>   Documentation/topics/networking-namespaces.rst \
> >>> @@ -55,6 +55,7 @@ DOC_SOURCE = \
> >>>   Documentation/topics/ovsdb-replication.rst \
> >>>   Documentation/topics/porting.rst \
> >>>   Documentation/topics/record-replay.rst \
> >>> + Documentation/topics/testing.rst \
> >>>   Documentation/topics/tracing.rst \
> >>>   Documentation/topics/usdt-probes.rst \
> >>>   Documentation/topics/userspace-checksum-offloading.rst \
> >>> @@ -162,6 +163,7 @@ RST_MANPAGES = \
> >>>   ovs-actions.7.rst \
> >>>   ovs-appctl.8.rst \
> >>>   ovs-ctl.8.rst \
> >>> + ovs-flowviz.8.rst \
> >>>   ovs-l3ping.8.rst \
> >>>   ovs-parse-backtrace.8.rst \
> >>>   ovs-pki.8.rst \
> >>> diff --git a/Documentation/conf.py b/Documentation/conf.py
> >>> index 15785605a..3a82f23a7 100644
> >>> --- a/Documentation/conf.py
> >>> +++ b/Documentation/conf.py
> >>> @@ -120,6 +120,8 @@ _man_pages = [
> >>>   u'utility for configuring running Open vSwitch daemons'),
> >>>  ('ovs-ctl.8',
> >>>   u'OVS startup helper script'),
> >>> +('ovs-flowviz.8',
> >>> + u'utility for visualizing OpenFlow and datapath flows'),
> >>>  ('ovs-l3ping.8',
> >>>   u'check network deployment for L3 tunneling problems'),
> >>>  ('ovs-parse-backtrace.8',
> >>> diff --git a/Documentation/ref/index.rst b/Documentation/ref/index.rst
> >>> index 03ada932f..7f2fe6177 100644
> >>> --- a/Documentation/ref/index.rst
> >>> +++ b/Documentation/ref/index.rst
> >>> @@ -42,6 +42,7 @@ time:
> >>> ovs-actions.7
> >>> ovs-appctl.8
> >>> ovs-ctl.8
> >>> +   ovs-flowviz.8
> >>> ovs-l3ping.8
> >>> ovs-pki.8
> >>> ovs-sim.1
> >>> diff --git a/Documentation/ref/ovs-flowviz.8.rst 
> >>> b/Documentation/ref/ovs-flowviz.8.rst
> >>> new file mode 100644
> >>> index 0..969fda9be
> >>> --- /dev/null
> >>> +++ b/Documentation/ref/ovs-flowviz.8.rst
> >>> @@ -0,0 +1,535 @@
> >>> +..
> >>> +  Licensed under the Apache License, Version 2.0 

Re: [ovs-dev] [PATCH net-next] net: ovs: fix ovs_drop_reasons error

[Adrián Moreno]

On Thu, Aug 15, 2024 at 08:22:45PM GMT, Menglong Dong wrote:
> I'm sure if I understand it correctly, but it seems that there is
> something wrong with ovs_drop_reasons.
>
> ovs_drop_reasons[0] is "OVS_DROP_LAST_ACTION", but
> OVS_DROP_LAST_ACTION == __OVS_DROP_REASON + 1, which means that
> ovs_drop_reasons[1] should be "OVS_DROP_LAST_ACTION".
>
> Fix this by initializing ovs_drop_reasons with index.
>
> Fixes: 9d802da40b7c ("net: openvswitch: add last-action drop reason")
> Signed-off-by: Menglong Dong 

The patch looks good to me. Also, tested and verified that,
without the patch, adding flow to drop packets results in:

drop at: do_execute_actions+0x197/0xb20 [openvsw (0xc0db6f97)
origin: software
input port ifindex: 8
timestamp: Tue Aug 20 10:19:17 2024 859853461 nsec
protocol: 0x800
length: 98
original length: 98
drop reason: OVS_DROP_ACTION_ERROR

With the patch, the same results in:

drop at: do_execute_actions+0x197/0xb20 [openvsw (0xc0db6f97)
origin: software
input port ifindex: 8
timestamp: Tue Aug 20 10:16:13 2024 475856608 nsec
protocol: 0x800
length: 98
original length: 98
drop reason: OVS_DROP_LAST_ACTION

Tested-by: Adrian Moreno 
Reviewed-by: Adrian Moreno 

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] Active bond member selection

[Adrián Moreno]

On Thu, Aug 08, 2024 at 05:46:01PM GMT, Jonathan Davies wrote:
> Hi ovs-dev,
>
> I'm seeing behaviour that feels like a bug, related to preservation of
> active bond member selection across OVS restarts. But the behaviour has been
> there for several years, so I wonder whether I'm missing something.
>
> The OVS DB has a bond_active_slave field. This gets read by
> port_configure_bond() into struct bond_setting's active_member_mac field.
> But this gets ignored by bond_create(), which sets bond->active_slave_mac to
> eth_addr_zero.
>
> Why is it being set to eth_addr_zero rather than to the value of
> active_member_mac from the struct bond_settings that is passed to
> bond_create()? It means that the bond_active_slave field is ignored.
>
> The behaviour was introduced in commit 30353934 ("ofproto/bond: Validate
> active-slave mac.") but I'm having a hard time understanding why it changed
> from using s->active_member_mac to eth_addr_zero when the initialisation
> logic moved to bond_create(). I'd be grateful for more context.
>
> If this is the intended behaviour, why do we bother saving the
> bond_active_slave field at all? The original rationale described in commit
> 3e5aeeb5 ("bridge: Keep bond active slave selection across OVS restart")
> still seems sound to me.
>

Hi Jonathan,

Took a look at the commits you mention and I think I agree with you. I
don't have more context unfortunately.

Looking at 30353934, I see that it's trying to solve a race condition
that could make an invalid mac end up being configured. But I don't see
a reason why it should be initialized to zero.

Maybe Ilya knows more but AFAICT, this is a bug.

Thanks.
Adrián

> Thanks,
> Jonathan
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [Patch ovn v7] Text representations for drop sampling.

[Adrián Moreno]

On Wed, Aug 07, 2024 at 11:04:23AM GMT, Dumitru Ceara wrote:
> On 8/2/24 09:22, Adrián Moreno wrote:
> > On Thu, Aug 01, 2024 at 11:11:25AM GMT, Jacob Tanenbaum wrote:
> >> Created a new column in the southbound database to hardcode a human 
> >> readable
> >> description for flows. This first use is describing why the flow is 
> >> dropping packets.
> >> The new column is called flow_desc and will create southbound database 
> >> entries like this
> >>
> >> _uuid   : 20f1897b-477e-47ae-a32c-c546d83ec097
> >> actions : 
> >> "sample(probability=65535,collector_set=123,obs_domain=1,obs_point=$cookie);
> >>  /* drop */"
> >> controller_meter: []
> >> external_ids: {source="northd.c:8721", stage-name=ls_in_l2_unknown}
> >> flow_desc   : "No L2 destination"
> >> logical_datapath: []
> >> logical_dp_group: ee3c3db5-98a2-4f34-8a84-409deae140a7
> >> match   : "outport == \"none\""
> >> pipeline: ingress
> >> priority: 50
> >> table_id: 27
> >> tags: {}
> >> hash: 0
> >>
> >> future work includes entering more flow_desc for more flows and adding
> >> flow_desc to the actions as a comment.
> >
> > Hi Jacob, apart from some nits below, I have a question: I still see a
>
> Hi Jacob, Adrian,
>
> Some more comments from my side too.
>
> > lot of drops without text representation, is this future work planned
> > for this release?
> >
> >>
> >> Signed-off-by: Jacob Tanenbaum 
> >> Suggested-by: Dumitru Ceara 
> >> Reported-at: https://issues.redhat.com/browse/FDP-307
> >> Acked-by: Ales Musil 
> >>
> >
> >
> >> ---
> >>
> >> v1: initial version
> >> v2: correct commit message
> >> make the flow_desc a char*
> >> correct a typo in the ovn-sb.xml
> >> correct the test
> >> v3: rebase issue with NEWS file
> >> v4: remove options:debug_drop_domain_id="1" from testing as we
> >> do not depend on it
> >> v5: introduce string wrapper
> >> increment ovs-sb.ovsschema version
> >> correct the testing
> >> added descriptions to more dropped packets
> >> v6: v5 was not the correct branch, this is...
> >> added descriptions to more dropped packets
> >> changed the names of the macros to be more descriptive
> >> v7: rebased, corrected a few typos, updated the ovs-sb.ovsschema
> >> checksum, and added Ales ACK
> >> ---
> >>  NEWS|  2 ++
> >>  lib/ovn-util.h  | 26 +++
> >>  northd/lflow-mgr.c  | 25 +++
> >>  northd/lflow-mgr.h  | 27 +++-
> >>  northd/northd.c | 50 -
> >>  ovn-sb.ovsschema| 10 +
> >>  ovn-sb.xml  |  5 +
> >>  tests/ovn-northd.at | 15 ++
> >>  8 files changed, 120 insertions(+), 40 deletions(-)
> >>
> >> diff --git a/NEWS b/NEWS
> >> index 136f890f5..246cc9cfd 100644
> >> --- a/NEWS
> >> +++ b/NEWS
> >> @@ -50,6 +50,8 @@ Post v24.03.0
> >>- A new LSP option "disable_garp_rarp" has been added to prevent OVN 
> >> from
> >>  sending GARP or RARP announcements when a VIF is created on a bridged
> >>  logical switch.
> >> +  - Added a new column in the southbound database "flow_desc" to provide
> >> +human readable context to flows.
> >>
> >>  OVN v24.03.0 - 01 Mar 2024
> >>  --
> >> diff --git a/lib/ovn-util.h b/lib/ovn-util.h
> >> index ae971ce5a..02bf92e62 100644
> >> --- a/lib/ovn-util.h
> >> +++ b/lib/ovn-util.h
> >> @@ -469,6 +469,32 @@ void sorted_array_apply_diff(const struct 
> >> sorted_array *a1,
> >>  bool add),
> >>   const void *arg);
> >>
> >> +/* A wrapper that holds strings */
> >> +struct string_wrapper
> >> +{
>
> The curly brace should be on the previous line.
>
> >> +char *str;
> >> +bool owns_string;
> >> +};
> >> +
> >> +#define EMPTY_STRING_WRAPPER (struct string_wrapper){NULL, fal

Re: [ovs-dev] [PATCH] bond: Fix inaccurate log info in bond_shift_load.

[Adrián Moreno]

On Fri, Aug 02, 2024 at 06:38:19PM GMT, Ilya Maximets wrote:
> On 7/2/24 16:36, Han Ding wrote:
> > When the delta is less than 1024 in bond_shift_load, it print "shift 0kB of 
> > load".
> > Like this:
> > bond dpdkbond0: shift 0kB of load (with hash 71) from nic1 to nic2 (now 
> > carrying 20650165kB and 8311662kB load, respectively)
>
> Hi, Han.  Thanks for the patch!
>
> I'm curious, is this information about movements that small
> practically useful?
>

I had the same thought.
I guess it should not happen very often given how the algorithm works but
OTOH, printing "0kB" is definitely not useful.

> >
> > Signed-off-by: Han Ding 
> > ---
> >  ofproto/bond.c | 24 +---
> >  1 file changed, 17 insertions(+), 7 deletions(-)
> >
> > diff --git a/ofproto/bond.c b/ofproto/bond.c
> > index c31869a..5b1975d 100644
> > --- a/ofproto/bond.c
> > +++ b/ofproto/bond.c
> > @@ -1192,13 +1192,23 @@ bond_shift_load(struct bond_entry *hash, struct 
> > bond_member *to)
> >  struct bond *bond = from->bond;
> >  uint64_t delta = hash->tx_bytes;
> >
> > -VLOG_INFO("bond %s: shift %"PRIu64"kB of load (with hash %"PRIdPTR") "
> > -  "from %s to %s (now carrying %"PRIu64"kB and "
> > -  "%"PRIu64"kB load, respectively)",
> > -  bond->name, delta / 1024, hash - bond->hash,
> > -  from->name, to->name,
> > -  (from->tx_bytes - delta) / 1024,
> > -  (to->tx_bytes + delta) / 1024);
> > +if (delta >= 1024) {
> > +VLOG_INFO("bond %s: shift %"PRIu64"kB of load (with hash 
> > %"PRIdPTR") "
> > +"from %s to %s (now carrying %"PRIu64"kB and "
> > +"%"PRIu64"kB load, respectively)",
> > +bond->name, delta / 1024, hash - bond->hash,
> > +from->name, to->name,
> > +(from->tx_bytes - delta) / 1024,
> > +(to->tx_bytes + delta) / 1024);
> > +} else {
> > +VLOG_INFO("bond %s: shift %"PRIu64"B of load (with hash 
> > %"PRIdPTR") "
> > +"from %s to %s (now carrying %"PRIu64"kB and "

Apart from Ilya's comment, missing one more indentation space in this
line (and all others).

Thanks.

> > +"%"PRIu64"kB load, respectively)",
> > +bond->name, delta, hash - bond->hash,
> > +from->name, to->name,
> > +(from->tx_bytes - delta) / 1024,
> > +(to->tx_bytes + delta) / 1024);
> > +}
>
> I'd suggest instead of copying the whole thing, just replace "kB"
> with another %s and use a couple of ternary operators to produce
> correct value and "kB" or "B" depending on the delta.
>
> Best regards, Ilya Maximets.
> ___
> dev mailing list
> d...@openvswitch.org
> https://mail.openvswitch.org/mailman/listinfo/ovs-dev
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [Patch ovn v7] Text representations for drop sampling.

[Adrián Moreno]

On Thu, Aug 01, 2024 at 11:11:25AM GMT, Jacob Tanenbaum wrote:
> Created a new column in the southbound database to hardcode a human readable
> description for flows. This first use is describing why the flow is dropping 
> packets.
> The new column is called flow_desc and will create southbound database 
> entries like this
>
> _uuid   : 20f1897b-477e-47ae-a32c-c546d83ec097
> actions : 
> "sample(probability=65535,collector_set=123,obs_domain=1,obs_point=$cookie); 
> /* drop */"
> controller_meter: []
> external_ids: {source="northd.c:8721", stage-name=ls_in_l2_unknown}
> flow_desc   : "No L2 destination"
> logical_datapath: []
> logical_dp_group: ee3c3db5-98a2-4f34-8a84-409deae140a7
> match   : "outport == \"none\""
> pipeline: ingress
> priority: 50
> table_id: 27
> tags: {}
> hash: 0
>
> future work includes entering more flow_desc for more flows and adding
> flow_desc to the actions as a comment.

Hi Jacob, apart from some nits below, I have a question: I still see a
lot of drops without text representation, is this future work planned
for this release?

>
> Signed-off-by: Jacob Tanenbaum 
> Suggested-by: Dumitru Ceara 
> Reported-at: https://issues.redhat.com/browse/FDP-307
> Acked-by: Ales Musil 
>


> ---
>
> v1: initial version
> v2: correct commit message
> make the flow_desc a char*
> correct a typo in the ovn-sb.xml
> correct the test
> v3: rebase issue with NEWS file
> v4: remove options:debug_drop_domain_id="1" from testing as we
> do not depend on it
> v5: introduce string wrapper
> increment ovs-sb.ovsschema version
> correct the testing
> added descriptions to more dropped packets
> v6: v5 was not the correct branch, this is...
> added descriptions to more dropped packets
> changed the names of the macros to be more descriptive
> v7: rebased, corrected a few typos, updated the ovs-sb.ovsschema
> checksum, and added Ales ACK
> ---
>  NEWS|  2 ++
>  lib/ovn-util.h  | 26 +++
>  northd/lflow-mgr.c  | 25 +++
>  northd/lflow-mgr.h  | 27 +++-
>  northd/northd.c | 50 -
>  ovn-sb.ovsschema| 10 +
>  ovn-sb.xml  |  5 +
>  tests/ovn-northd.at | 15 ++
>  8 files changed, 120 insertions(+), 40 deletions(-)
>
> diff --git a/NEWS b/NEWS
> index 136f890f5..246cc9cfd 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -50,6 +50,8 @@ Post v24.03.0
>- A new LSP option "disable_garp_rarp" has been added to prevent OVN from
>  sending GARP or RARP announcements when a VIF is created on a bridged
>  logical switch.
> +  - Added a new column in the southbound database "flow_desc" to provide
> +human readable context to flows.
>
>  OVN v24.03.0 - 01 Mar 2024
>  --
> diff --git a/lib/ovn-util.h b/lib/ovn-util.h
> index ae971ce5a..02bf92e62 100644
> --- a/lib/ovn-util.h
> +++ b/lib/ovn-util.h
> @@ -469,6 +469,32 @@ void sorted_array_apply_diff(const struct sorted_array 
> *a1,
>  bool add),
>   const void *arg);
>
> +/* A wrapper that holds strings */
> +struct string_wrapper
> +{
> +char *str;
> +bool owns_string;
> +};
> +
> +#define EMPTY_STRING_WRAPPER (struct string_wrapper){NULL, false}
> +
> +static inline struct string_wrapper
> +string_wrapper_create(char *str, bool take_ownership)
> +{
> +return (struct string_wrapper) {
> +.str = str,
> +.owns_string = take_ownership,
> +};
> +}
> +
> +static inline void
> +string_wrapper_destroy(struct string_wrapper *s)
> +{
> +if (s->owns_string) {
> +free(s->str);
> +}
> +}
> +
>  /* Utilities around properly handling exit command. */
>  struct ovn_exit_args {
>  struct unixctl_conn **conns;
> diff --git a/northd/lflow-mgr.c b/northd/lflow-mgr.c
> index b2c60b5de..6157d4dfe 100644
> --- a/northd/lflow-mgr.c
> +++ b/northd/lflow-mgr.c
> @@ -25,6 +25,7 @@
>  #include "debug.h"
>  #include "lflow-mgr.h"
>  #include "lib/ovn-parallel-hmap.h"
> +#include "lib/ovn-util.h"
>
>  VLOG_DEFINE_THIS_MODULE(lflow_mgr);
>
> @@ -36,7 +37,7 @@ static void ovn_lflow_init(struct ovn_lflow *, struct 
> ovn_datapath *od,
> uint16_t priority, char *match,
> char *actions, char *io_port,
> char *ctrl_meter, char *stage_hint,
> -   const char *where);
> +   const char *where, struct string_wrapper 
> flow_desc);
>  static struct ovn_lflow *ovn_lflow_find(const struct hmap *lflows,
>  enum ovn_stage stage,
>  uint16_t priority, const char *match,
> @@ -52,7 +53,7 @@ static struct ovn_lflow *do_ovn_lflow

Re: [ovs-dev] [PATCH ovn v4 5/9] northd: Add ACL Sampling.

[Adrián Moreno]

On Thu, Aug 01, 2024 at 03:09:37PM GMT, Ilya Maximets wrote:
> On 8/1/24 14:26, Dumitru Ceara wrote:
> > On 8/1/24 13:58, Ilya Maximets wrote:
> >> On 7/31/24 19:38, Dumitru Ceara wrote:
> >>> On 7/31/24 18:39, Ilya Maximets wrote:
>  On 7/31/24 18:17, Ilya Maximets wrote:
> > On 7/31/24 11:05, Dumitru Ceara wrote:
> >> From: Adrian Moreno 
> >>
> >> Introduce a new table called Sample where per-flow IPFIX configuration
> >> can be specified.
> >> Also, reference rows from such table from the ACL table to enable the
> >> configuration of ACL sampling. If enabled, northd will add a sample
> >> action to each ACL related logical flow.
> >>
> >> Packets that hit stateful ACLs are sampled in different ways depending
> >> whether they are initiating a new session or are just forwarded on an
> >> existing (already allowed) session.  Two new columns ("sample_new" and
> >> "sample_est") are added to the ACL table to allow for potentially
> >> different sampling rates for the two cases.
> >
> > Do we actually need to be able to sample each ACL to a different set of 
> > collectors?
> >>>
> >>> Yes, please see below.
> >>>
> > I mean, is it not enough to have a common configuration per sampling 
> > type (drop,
> > acl-new, acl-est) and two boolean fields per ACL to opt-in as we do for 
> > acl logging?
> > Or a map ACL:sample = key: enum [new,est], value: metadata ?
> >
> >
> > Something like:
> >
> >  NB_Global:
> >sample_config: map key enum [drop, acl-new, acl-est]
> >   value ref Sample_Collector_Set
> >
> >  Sample_Collector_Set:
> >ids: set key integer
> >probability: integer
> >
> >  ACL:
> >sample_metadata: map key enum [new, est]
> > value integer
> >
> >>
> >> Note: If an ACL has both sampling enabled and a label associated to it
> >> then the label value overrides the observation point ID defined in the
> >> sample configuration.  This is a side effect of the implementation
> >> (observation point IDs are stored in conntrack in the same part of the
> >> ct_label where ACL labels are also stored).  The two features
> >> (sampling and ACL labels) serve however similar purposes so it's not
> >> expected that they're both enabled together.
> >>
> >> When sampling is enabled on an ACL additional logical flows are created
> >> for that ACL (one for stateless ACLs and 3 for stateful ACLs) in the 
> >> ACL
> >> action stage of the logical pipeline.  These additional flows match on 
> >> a
> >> combination of conntrack state values and observation point id values
> >> (either against a logical register or against the stored ct_label 
> >> state)
> >> in order to determine whether the packets hitting the ACLs must be
> >> sampled or not.  This comes with a slight increase in the number of
> >> logical flows and in the number of OpenFlow rules.  The number of
> >> additional flows _does not_ depend on the original ACL match or action.
> >>
> >> New --sample-new and --sample-est optional arguments are added to the
> >> 'ovn-nbctl acl-add' command to allow configuring these new types of
> >> sampling for ACLs.
> >>
> >> An example workflow of configuring ACL samples is:
> >>   # Create Sampling_App mappings for ACL traffic types:
> >>   ovn-nbctl create Sampling_App name="acl-new-traffic-sampling" \
> >> id="42"
> >>   ovn-nbctl create sampling_app name="acl-est-traffic-sampling" \
> >>id="43"
> >>   # Create two sample collectors, one that samples all packets (c1)
> >>   # and another one that samples with a probability of 10% (c2):
> >>   c1=$(ovn-nbctl create Sample_Collector name=c1 \
> >>probability=65535 set_id=1)
> >>   c2=$(ovn-nbctl create Sample_Collector name=c2 \
> >>probability=6553 set_id=2)
> >>   # Create two sample configurations (for new and for established
> >>   # traffic):
> >>   s1=$(ovn-nbctl create sample collector="$c1 $c2" metadata=4301)
> >>   s2=$(ovn-nbctl create sample collector="$c1 $c2" metadata=4302)
> >>   # Create an ingress ACL to allow IP traffic:
> >>   ovn-nbctl --sample-new=$s1 --sample-est=$s2 acl-add ls \
> >> from-lport 1 "ip" allow-related
> >>
> >> The config above will generate IPFIX samples with:
> >> - 8 MSB of observation domain id set to 42 (Sampling_App
> >>   "acl-new-traffic-sampling" config) and observation point id
> >>   set to 4301 (Sample s1) for packets that create a new
> >>   connection
> >> - 8 MSB of observation domain id set to 43 (Sampling_app
> >>   "acl-est-traffic-sampling" config) and observation point id
> >>   set to 4302 (Sample s2)

Re: [ovs-dev] [PATCH v3 11/13] ofproto: xlate: Make sampled drops explicit.

[Adrián Moreno]

On Sat, Jul 13, 2024 at 01:06:01AM GMT, Ilya Maximets wrote:
> On 7/12/24 19:06, Adrian Moreno wrote:
> > When the flow translation results in a datapath action list whose last
> > action is an "observational" action, i.e: one generated for IPFIX,
> > sFlow or local sampling applications, the packet is actually going to be
> > dropped (and observed).
> >
> > In that case, add an explicit drop action so that drop statistics remain
> > accurate. This behavior is controlled by a configurable boolean knob
> > called "explicit_sampled_drops"
> >
> > Combine the "optimizations" and other odp_actions "tweaks" into a single
> > function.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  NEWS |   5 +
> >  ofproto/ofproto-dpif-xlate.c |  64 ++---
> >  ofproto/ofproto-dpif-xlate.h |   4 +
> >  ofproto/ofproto-dpif.c   |   6 ++
> >  ofproto/ofproto-dpif.h   |   2 +
> >  ofproto/ofproto-provider.h   |   4 +
> >  ofproto/ofproto.c|  10 ++
> >  ofproto/ofproto.h|   2 +
> >  tests/drop-stats.at  | 173 +++
> >  tests/ofproto-dpif.at|  49 ++
> >  vswitchd/bridge.c|   4 +-
> >  vswitchd/vswitch.xml |  26 +-
> >  12 files changed, 333 insertions(+), 16 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index c27727c0d..4eb73aada 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -40,6 +40,11 @@ Post-v3.3.0
> >   allows samples to be emitted locally (instead of via IPFIX) in a
> >   datapath-specific manner.  The Linux kernel datapath is the first to
> >   support this feature by using the new datapath "psample" action.
> > +   - OVSDB:
>
> We normally use OVSDB for changes in the database server, not the vswitch
> schema.  Just remove it.
>
> > + * A new configuration knob "other_config:explicit-sampled-drops" in 
> > the
> > +   Open_vSwitch table controls whether an explicit drop action shall be
> > +   added at the end of datapath flows whose last action is an
> > +   observability-driven sample action.
> >
> >
> >  v3.3.0 - 16 Feb 2024
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 0a0964348..e5234b950 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -3415,6 +3415,7 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  struct ofproto *ofproto = &ctx->xin->ofproto->up;
> >  uint32_t meter_id = ofproto->slowpath_meter_id;
> >  size_t cookie_offset = 0;
> > +size_t observe_offset;
> >
> >  /* The meter action is only used to throttle userspace actions.
> >   * If they are not needed and the sampling rate is 100%, avoid 
> > generating
> > @@ -3432,6 +3433,7 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  }
> >
> >  if (args->psample) {
> > +observe_offset = ctx->odp_actions->size;
> >  odp_put_psample_action(ctx->odp_actions,
> > args->psample->group_id,
> > (void *) &args->psample->cookie,
> > @@ -3443,6 +3445,7 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  nl_msg_put_u32(ctx->odp_actions, OVS_ACTION_ATTR_METER, 
> > meter_id);
> >  }
> >
> > +observe_offset = ctx->odp_actions->size;
> >  odp_port_t odp_port = ofp_port_to_odp_port(
> >  ctx->xbridge, ctx->xin->flow.in_port.ofp_port);
> >  uint32_t pid = dpif_port_get_pid(ctx->xbridge->dpif, odp_port);
> > @@ -3457,6 +3460,9 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  if (is_sample) {
> >  nl_msg_end_nested(ctx->odp_actions, actions_offset);
> >  nl_msg_end_nested(ctx->odp_actions, sample_offset);
> > +ctx->xout->last_observe_offset = sample_offset;
> > +} else {
> > +ctx->xout->last_observe_offset = observe_offset;
> >  }
> >
> >  return cookie_offset;
> > @@ -8066,12 +8072,16 @@ xlate_wc_finish(struct xlate_ctx *ctx)
> >  }
> >  }
> >
> > -/* This will optimize the odp actions generated. For now, it will remove
> > - * trailing clone actions that are unnecessary. */
> > +/* This will tweak the odp actions generated. For now, it will:
> > + *  - Remove trailing clone actions that are unnecessary.
> > + *  - Add an explicit drop action if the action list is empty.
> > + *  - Add an explicit drop action if the last action is an observability
> > + *sample. This tweak is controlled by a configurable knob. */
> >  static void
> > -xlate_optimize_odp_actions(struct xlate_in *xin)
> > +xlate_tweak_odp_actions(struct xlate_ctx *ctx)
> >  {
> > -struct ofpbuf *actions = xin->odp_actions;
> > +uint32_t last_observe_offset = ctx->xout->last_observe_offset;
> > +struct ofpbuf *actions = ctx->xin->odp_actions;
> >  struct nlattr *last_action = NULL;
> >  struct nlattr *a;
> >  int left;
> > @@ -8085,11 +8095,28 @@ xlate_optimize_odp_actions(struct xlate_in *xin)
> >  last

Re: [ovs-dev] [PATCH v3 07/13] ofproto-dpif-xlate-cache: Add lsample to xcache.

[Adrián Moreno]

On Sat, Jul 13, 2024 at 12:51:58AM GMT, Ilya Maximets wrote:
> On 7/12/24 19:06, Adrian Moreno wrote:
> > Add a cache entry type for local sample objects.
> > Store both the dpif_lsample reference and the collector_set_id so we can
> > quickly find the particular exporter.
> >
> > Using this mechanism, account for packet and byte statistics.
> >
> > Acked-by: Eelco Chaudron 
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-lsample.c | 18 ++
> >  ofproto/ofproto-dpif-lsample.h |  4 
> >  ofproto/ofproto-dpif-xlate-cache.c | 11 ++-
> >  ofproto/ofproto-dpif-xlate-cache.h |  6 ++
> >  ofproto/ofproto-dpif-xlate.c   | 13 +
> >  ofproto/ofproto-dpif.c |  1 +
> >  6 files changed, 52 insertions(+), 1 deletion(-)
>
> Hi, Adrian.  The code is fine here, but I don't think the change is logically
> correct.  Flow translation cache will accumulate flow statistics and it will
> not be equal to the collector statistics unless the sampling rate is 100%.
> If it's not 100% packet will go through the flow, stats will be updated, but
> they will not be smapled and so should not be reported as lsample statistics.
>
> I actually do not know how we could capture the collector's stats in this 
> case.
> In case of sFlow or IPFIX the packet are going through userspace and we can
> count them when they arrive to userspace and to the corresponding module.
> In case of psample, packets stay in the kenrel and unless we listen on psample
> we don't have a way to get the actual number of sampled packets and bytes.
>
> Should we just drop patches 7 and 8 and remove the lsample/show calls from
> the tests?

I see your point. It will always inaccurate and add little on top of
pure openflow stats aggregation which can be done by other means.

I'll remove them from the next version.

Thanks.
Adrián

>
> Best regards, Ilya Maximets.
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v2 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Fri, Jul 12, 2024 at 05:14:30PM GMT, Adrián Moreno wrote:
> On Fri, Jul 12, 2024 at 04:35:48PM GMT, Eelco Chaudron wrote:
> > On 12 Jul 2024, at 1:32, Adrian Moreno wrote:
> >
> > > When sample action gets used as a way of sampling traffic with
> > > controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> > > the controller will have to increase the number of flows to ensure each
> > > part of the pipeline contains the right metadata.
> > >
> > > As an example, if the controller decides to sample stateful traffic, it
> > > could store the computed metadata for each connection in the conntrack
> > > label. However, for established connections, a flow must be created for
> > > each different ct_label value with a sample action that contains a
> > > different hardcoded obs_domain and obs_point id.
> > >
> > > This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> > > that supports specifying the observation point and domain using an
> > > OpenFlow field reference, so now the controller can express:
> > >
> > >  sample(...
> > > obs_domain_id=NXM_NX_CT_LABEL[0..31],
> > > obs_point_id=NXM_NX_CT_LABEL[32..63]
> > > ...
> > >)
> > >
> >
> > Some comments below on the test case, the rest looks good.
> >
> > Cheers,
> >
> > Eelco
> >
> > > Signed-off-by: Adrian Moreno 
> > > ---
> > >  Documentation/ref/ovs-actions.7.rst |  12 +-
> > >  NEWS|   2 +
> > >  include/openvswitch/ofp-actions.h   |   8 +-
> > >  lib/ofp-actions.c   | 245 +---
> > >  ofproto/ofproto-dpif-xlate.c|  44 -
> > >  python/ovs/flow/ofp.py  |   8 +-
> > >  python/ovs/flow/ofp_act.py  |   6 +-
> > >  tests/ofp-actions.at|   8 +
> > >  tests/ofproto-dpif.at   |  44 +
> > >  tests/ovs-ofctl.at  |  14 ++
> > >  tests/system-traffic.at |  79 +
> > >  11 files changed, 429 insertions(+), 41 deletions(-)
> > >
> > > diff --git a/Documentation/ref/ovs-actions.7.rst 
> > > b/Documentation/ref/ovs-actions.7.rst
> > > index 80acd9070..01021dc0a 100644
> > > --- a/Documentation/ref/ovs-actions.7.rst
> > > +++ b/Documentation/ref/ovs-actions.7.rst
> > > @@ -2201,13 +2201,17 @@ The following *argument* forms are accepted:
> > >  The unsigned 32-bit integer identifier of the set of sample 
> > > collectors to
> > >  send sampled packets to.  Defaults to 0.
> > >
> > > -  ``obs_domain_id=``\ *id*
> > > +  ``obs_domain_id=``\ *value*
> > >  When sending samples to IPFIX collectors, the unsigned 32-bit integer
> > > -Observation Domain ID sent in every IPFIX flow record.  Defaults to 
> > > 0.
> > > +Observation Domain ID sent in every IPFIX flow record. The *value* 
> > > may
> > > +be specified as a 32-bit integer or a field or subfield in the syntax
> > > +described under `Field Specifications`_ above. Defaults to 0.
> > >
> > > -  ``obs_point_id=``\ *id*
> > > +  ``obs_point_id=``\ *value*
> > >  When sending samples to IPFIX collectors, the unsigned 32-bit integer
> > > -Observation Point ID sent in every IPFIX flow record.  Defaults to 0.
> > > +Observation Point ID sent in every IPFIX flow record. The *value* may
> > > +be specified as a 32-bit integer or a field or subfield in the syntax
> > > +described under `Field Specifications`_ above. Defaults to 0.
> > >
> > >``sampling_port=``\ *port*
> > >  Sample packets on *port*, which should be the ingress or egress 
> > > port.  This
> > > diff --git a/NEWS b/NEWS
> > > index 096ff4d7d..7d7461c57 100644
> > > --- a/NEWS
> > > +++ b/NEWS
> > > @@ -22,6 +22,8 @@ Post-v3.3.0
> > >   support this feature by using the new datapath "psample" action.
> > > - A new unixctl command 'lsample/show' shows packet and bytes 
> > > statistics
> > >   per local sample exporter.
> > > +   - OpenFlow: a new version of the "sample" action is introduced that 
> > > allows
> > > + the use of subfields in obs_point_id and obs_domain_id.
> > >
> > >
> > >  v3.3.0 - 16 Feb 2024
> > > diff --gi

Re: [ovs-dev] [PATCH v2 06/13] ofproto-dpif-xlate: Use psample for local sample.

[Adrián Moreno]

On Fri, Jul 12, 2024 at 04:31:38PM GMT, Eelco Chaudron wrote:
> On 12 Jul 2024, at 1:32, Adrian Moreno wrote:
>
> > Use the newly added psample action to implement OpenFlow sample() actions
> > with local sampling configuration if possible.
> >
> > A bit of refactoring in compose_sample_actions arguments helps make it a
> > bit more readable.
> >
> > Signed-off-by: Adrian Moreno 
>
> The patch looks good to me. I do have one comment based on Ilya's earlier 
> feedback. Regardless of the resolution, I'm okay with it, so I'll add my ack.
>
> Acked-by: Eelco Chaudron 
>
> > ---
> >  ofproto/ofproto-dpif-lsample.c |  16 +++
> >  ofproto/ofproto-dpif-lsample.h |   5 +
> >  ofproto/ofproto-dpif-xlate.c   | 238 ++---
> >  ofproto/ofproto-dpif-xlate.h   |   5 +-
> >  ofproto/ofproto-dpif.c |   2 +-
> >  tests/ofproto-dpif.at  | 159 ++
> >  6 files changed, 345 insertions(+), 80 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index a2b2e8059..833ce923f 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -140,6 +140,22 @@ dpif_lsample_set_options(struct dpif_lsample *lsample,
> >  return changed;
> >  }
> >
> > +/* Returns the group_id for a given collector_set_id, if it exists. */
> > +bool
> > +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> > collector_set_id,
> > +  uint32_t *group_id)
> > +{
> > +struct lsample_exporter_node *node;
> > +bool found = false;
> > +
> > +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> > +if (node) {
> > +found = true;
> > +*group_id = node->exporter.options.group_id;
> > +}
> > +return found;
> > +}
> > +
> >  struct dpif_lsample *
> >  dpif_lsample_create(void)
> >  {
> > diff --git a/ofproto/ofproto-dpif-lsample.h b/ofproto/ofproto-dpif-lsample.h
> > index a491c137d..26517a645 100644
> > --- a/ofproto/ofproto-dpif-lsample.h
> > +++ b/ofproto/ofproto-dpif-lsample.h
> > @@ -18,6 +18,7 @@
> >  #define OFPROTO_DPIF_LSAMPLE_H 1
> >
> >  #include 
> > +#include 
> >  #include 
> >
> >  struct dpif_lsample;
> > @@ -32,4 +33,8 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >const struct ofproto_lsample_options *,
> >size_t n_opts);
> >
> > +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> > +   uint32_t collector_set_id,
> > +   uint32_t *group_id);
> > +
> >  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 7c4950895..9f32982b0 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -45,6 +45,7 @@
> >  #include "nx-match.h"
> >  #include "odp-execute.h"
> >  #include "ofproto/ofproto-dpif-ipfix.h"
> > +#include "ofproto/ofproto-dpif-lsample.h"
> >  #include "ofproto/ofproto-dpif-mirror.h"
> >  #include "ofproto/ofproto-dpif-monitor.h"
> >  #include "ofproto/ofproto-dpif-sflow.h"
> > @@ -116,6 +117,7 @@ struct xbridge {
> >  struct mbridge *mbridge;  /* Mirroring. */
> >  struct dpif_sflow *sflow; /* SFlow handle, or null. */
> >  struct dpif_ipfix *ipfix; /* Ipfix handle, or null. */
> > +struct dpif_lsample *lsample; /* Local sample handle, or null. */
> >  struct netflow *netflow;  /* Netflow handle, or null. */
> >  struct stp *stp;  /* STP or null if disabled. */
> >  struct rstp *rstp;/* RSTP or null if disabled. */
> > @@ -686,6 +688,7 @@ static void xlate_xbridge_set(struct xbridge *, struct 
> > dpif *,
> >const struct mbridge *,
> >const struct dpif_sflow *,
> >const struct dpif_ipfix *,
> > +  const struct dpif_lsample *,
> >const struct netflow *,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *,
> > @@ -1069,6 +1072,7 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >const struct mbridge *mbridge,
> >const struct dpif_sflow *sflow,
> >const struct dpif_ipfix *ipfix,
> > +  const struct dpif_lsample *lsample,
> >const struct netflow *netflow,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *support,
> > @@ -1099,6 +1103,11 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >  xbridge->ipfix = dpif_ipfix_ref(ipfix);
> >  }
> >
> > +if (xbridge->lsample != lsample) {
> > +dpif_lsample_unref(xbridge->lsample);
> > +xbridge->lsample = dpif_lsample_ref(lsample);
> > +}
> > +
> >

Re: [ovs-dev] [PATCH v2 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Fri, Jul 12, 2024 at 04:35:48PM GMT, Eelco Chaudron wrote:
> On 12 Jul 2024, at 1:32, Adrian Moreno wrote:
>
> > When sample action gets used as a way of sampling traffic with
> > controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> > the controller will have to increase the number of flows to ensure each
> > part of the pipeline contains the right metadata.
> >
> > As an example, if the controller decides to sample stateful traffic, it
> > could store the computed metadata for each connection in the conntrack
> > label. However, for established connections, a flow must be created for
> > each different ct_label value with a sample action that contains a
> > different hardcoded obs_domain and obs_point id.
> >
> > This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> > that supports specifying the observation point and domain using an
> > OpenFlow field reference, so now the controller can express:
> >
> >  sample(...
> > obs_domain_id=NXM_NX_CT_LABEL[0..31],
> > obs_point_id=NXM_NX_CT_LABEL[32..63]
> > ...
> >)
> >
>
> Some comments below on the test case, the rest looks good.
>
> Cheers,
>
> Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  Documentation/ref/ovs-actions.7.rst |  12 +-
> >  NEWS|   2 +
> >  include/openvswitch/ofp-actions.h   |   8 +-
> >  lib/ofp-actions.c   | 245 +---
> >  ofproto/ofproto-dpif-xlate.c|  44 -
> >  python/ovs/flow/ofp.py  |   8 +-
> >  python/ovs/flow/ofp_act.py  |   6 +-
> >  tests/ofp-actions.at|   8 +
> >  tests/ofproto-dpif.at   |  44 +
> >  tests/ovs-ofctl.at  |  14 ++
> >  tests/system-traffic.at |  79 +
> >  11 files changed, 429 insertions(+), 41 deletions(-)
> >
> > diff --git a/Documentation/ref/ovs-actions.7.rst 
> > b/Documentation/ref/ovs-actions.7.rst
> > index 80acd9070..01021dc0a 100644
> > --- a/Documentation/ref/ovs-actions.7.rst
> > +++ b/Documentation/ref/ovs-actions.7.rst
> > @@ -2201,13 +2201,17 @@ The following *argument* forms are accepted:
> >  The unsigned 32-bit integer identifier of the set of sample collectors 
> > to
> >  send sampled packets to.  Defaults to 0.
> >
> > -  ``obs_domain_id=``\ *id*
> > +  ``obs_domain_id=``\ *value*
> >  When sending samples to IPFIX collectors, the unsigned 32-bit integer
> > -Observation Domain ID sent in every IPFIX flow record.  Defaults to 0.
> > +Observation Domain ID sent in every IPFIX flow record. The *value* may
> > +be specified as a 32-bit integer or a field or subfield in the syntax
> > +described under `Field Specifications`_ above. Defaults to 0.
> >
> > -  ``obs_point_id=``\ *id*
> > +  ``obs_point_id=``\ *value*
> >  When sending samples to IPFIX collectors, the unsigned 32-bit integer
> > -Observation Point ID sent in every IPFIX flow record.  Defaults to 0.
> > +Observation Point ID sent in every IPFIX flow record. The *value* may
> > +be specified as a 32-bit integer or a field or subfield in the syntax
> > +described under `Field Specifications`_ above. Defaults to 0.
> >
> >``sampling_port=``\ *port*
> >  Sample packets on *port*, which should be the ingress or egress port.  
> > This
> > diff --git a/NEWS b/NEWS
> > index 096ff4d7d..7d7461c57 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -22,6 +22,8 @@ Post-v3.3.0
> >   support this feature by using the new datapath "psample" action.
> > - A new unixctl command 'lsample/show' shows packet and bytes statistics
> >   per local sample exporter.
> > +   - OpenFlow: a new version of the "sample" action is introduced that 
> > allows
> > + the use of subfields in obs_point_id and obs_domain_id.
> >
> >
> >  v3.3.0 - 16 Feb 2024
> > diff --git a/include/openvswitch/ofp-actions.h 
> > b/include/openvswitch/ofp-actions.h
> > index 7b57e49ad..56dc2c147 100644
> > --- a/include/openvswitch/ofp-actions.h
> > +++ b/include/openvswitch/ofp-actions.h
> > @@ -1015,14 +1015,16 @@ enum nx_action_sample_direction {
> >
> >  /* OFPACT_SAMPLE.
> >   *
> > - * Used for NXAST_SAMPLE, NXAST_SAMPLE2, and NXAST_SAMPLE3. */
> > + * Used for NXAST_SAMPLE, NXAST_SAMPLE2, NXAST_SAMPLE3 and NXAST_SAMPLE4. 
> > */
> >  struct ofpact_sample {
> >  OFPACT_PADDED_MEMBERS(
> >  struct ofpact ofpact;
> >  uint16_t probability;   /* Always positive. */
> >  uint32_t collector_set_id;
> > -uint32_t obs_domain_id;
> > -uint32_t obs_point_id;
> > +uint32_t obs_domain_imm;
> > +struct mf_subfield obs_domain_src;
> > +uint32_t obs_point_imm;
> > +struct mf_subfield obs_point_src;
> >  ofp_port_t sampling_port;
> >  enum nx_action_sample_direction direction;
> >  );
> > diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> > index da7b1dd31..6b030bf29 100644
> > --- a/lib/

Re: [ovs-dev] [PATCH v2 11/13] ofproto: xlate: Make sampled drops explicit.

[Adrián Moreno]

On Fri, Jul 12, 2024 at 04:35:04PM GMT, Eelco Chaudron wrote:
> On 12 Jul 2024, at 1:32, Adrian Moreno wrote:
>
> > When the flow translation results in a datapath action list whose last
> > action is an "observational" action, i.e: one generated for IPFIX,
> > sFlow or local sampling applications, the packet is actually going to be
> > dropped (and observed).
> >
> > In that case, add an explicit drop action so that drop statistics remain
> > accurate.
> >
> > Combine the "optimizations" and other odp_actions "tweaks" into a single
> > function.
>
> I'm running out of time to properly review (and look for alternatives), so 
> I'll add some general comments, and let you and Ilya work out the details 
> when I'm on PTO ;)
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-xlate.c |  52 +++-
> >  ofproto/ofproto-dpif-xlate.h |   4 ++
> >  tests/drop-stats.at  | 113 +++
> >  tests/ofproto-dpif.at|  47 ++-
> >  4 files changed, 200 insertions(+), 16 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 0a0964348..15e89e6a8 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -3415,6 +3415,7 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  struct ofproto *ofproto = &ctx->xin->ofproto->up;
> >  uint32_t meter_id = ofproto->slowpath_meter_id;
> >  size_t cookie_offset = 0;
> > +size_t observe_offset;
> >
> >  /* The meter action is only used to throttle userspace actions.
> >   * If they are not needed and the sampling rate is 100%, avoid 
> > generating
> > @@ -3432,6 +3433,7 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  }
> >
> >  if (args->psample) {
> > +observe_offset = ctx->odp_actions->size;
> >  odp_put_psample_action(ctx->odp_actions,
> > args->psample->group_id,
> > (void *) &args->psample->cookie,
> > @@ -3443,6 +3445,7 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  nl_msg_put_u32(ctx->odp_actions, OVS_ACTION_ATTR_METER, 
> > meter_id);
> >  }
> >
> > +observe_offset = ctx->odp_actions->size;
> >  odp_port_t odp_port = ofp_port_to_odp_port(
> >  ctx->xbridge, ctx->xin->flow.in_port.ofp_port);
> >  uint32_t pid = dpif_port_get_pid(ctx->xbridge->dpif, odp_port);
> > @@ -3457,6 +3460,9 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  if (is_sample) {
> >  nl_msg_end_nested(ctx->odp_actions, actions_offset);
> >  nl_msg_end_nested(ctx->odp_actions, sample_offset);
> > +ctx->xout->last_observe_offset = sample_offset;
> > +} else {
> > +ctx->xout->last_observe_offset = observe_offset;
>
> Not sure why, but I do not like the idea of storing offsets in the ctx. 
> Thinking out loud without any checking; We are looping through all actions in 
> xlate_optimize_odp_actions() is there no easy way to figure it out there?

I thought of this.  The condition that must be met in order to
consider an action to be "observability" is:

Its a userspace() action used for IPFIX/sFlow OR
It's a psample() action OR
It's a sample() action that inside has (
psample action OR
userspace action usef for IPFIX/sFlow
)

So, we need to introspect the netlink attributes all the way to the
userspace action's cookie, decode it and look at the type to make sure
it's not, a metered controller action (which uses sample() as well).

The resulting code was a bit ugly with all that instrospection. So I
tried this other way which I think is cleaner and more robust since
actions have to explicitly "opt-in" to be considered observational,
rather than the check happening somewhere else.

Would you prefer the introspection alternative?

>
> >  }
> >
> >  return cookie_offset;
> > @@ -8066,12 +8072,14 @@ xlate_wc_finish(struct xlate_ctx *ctx)
> >  }
> >  }
> >
> > -/* This will optimize the odp actions generated. For now, it will remove
> > - * trailing clone actions that are unnecessary. */
> > +/* This will tweak the odp actions generated. For now, it will:
> > + *  - Remove trailing clone actions that are unnecessary.
> > + *  - Add an explicit drop action if the last action is observational or 
> > the
> > + *action list is empty. */
> >  static void
> > -xlate_optimize_odp_actions(struct xlate_in *xin)
> > +xlate_tweak_odp_actions(struct xlate_ctx *ctx)
> >  {
> > -struct ofpbuf *actions = xin->odp_actions;
> > +struct ofpbuf *actions = ctx->xin->odp_actions;
> >  struct nlattr *last_action = NULL;
> >  struct nlattr *a;
> >  int left;
> > @@ -8085,9 +8093,16 @@ xlate_optimize_odp_actions(struct xlate_in *xin)
> >  last_action = a;
> >  }
> >
> > +if (!last_action) {
> > +if (ovs_explicit_drop_action_supported(ctx->xbridge->ofproto)) {
> > +put_drop_action(actions,

Re: [ovs-dev] [PATCH v1 11/13] ofproto: xlate: Make flow-sampled drops explicit.

[Adrián Moreno]

[...]
> >> 2. This patch doesn't cover cases where sampling is not actually
> >>the last action, but further actions do not yield any datapath
> >>actions.  E.g. if you have a register resoration afterwards,
> >>which happens in automatically built pipelines like in OVN.
> >>Or if the last action after sampling is learn().  And things are
> >>getting more complicated if we take into account resubmits and
> >>processing in different bridges via patch ports.
> >
> > I agree this could be problematic. Maybe we should make sure the sample
> > is the last dp action and "fix it". A trick such as the one done for
> > sflow.
>
> These tricks are not accurate as well.  I beleive they do not track
> the information well across tunnel encaps at least.
>

I am currently testing a patch implements this for both per-bridge and
per-flow sampling. I attach the essense (i.e: not the tests) of it at
the end [1]. Can you expand on your concerns about tunnel encaps?

> >
> >>
> >> 3. If someone is sampling the drops specifically, they know where
> >>to find the packets, because they configured the collector.
> >>
> >
> > I think drop stats and samples are two different things. There are
> > typically extacted by different tools and systems.
> >
> > Besides, what about per-bridge sampling (next patch)?
> > The user enables sampling on a bridge without explicitly doing it
> > for drops and suddenly the drop statistics dissapear.
>
> I think 'user enables sampling on a bridge' counts as an explicit
> altering of the datapath pipeline.
>
> >
> >> 4. Packets are not actually dropped, they are delivered to userspace
> >>or psample.  It might make sense though to drop with a reson in
> >>case the upcall fails or psample fails to deliver to any entity
> >>and it is the last action in the datapath, but that's a kernel
> >>change to make.
> >>
> >
> > I don't want to get into another semantic discussion between consume,
> > free and drop or the dark corners of the OpenFlow protocol. For me it's
> > pretty clear that if the last action is to sample, the packet is
> > "dropped" in the sense that, from a switch' perspective, if it's not
> > forwarded/sent somewhere, it's dropped.
> >
> > I know you don't think the same :-).
> >
> > Would you then agree that the concept of dropping packets is very
> > unclear and OpenFlow does not make it easy (or even possible?) to
> > express a sampled drops and we should add an extension action to
> > explicitly drop packets?
>
> I agree that dropping packets is not clear in this case.
> I'm not a fan of OpenFlow drop action, it may cause extra logical issues.
> I'd go with a database knob instead.
>
> I'd say we can either drop these two patches for now and find a better
> solution in the next release cycle, or add an experimental global database
> knob that will control this behavior and will be disabled by default.
> Once we have better understanding how it behaves in a real world, we
> could switch the default or remove the feature if it causes issues or
> confusion.
>
> What do you think?
>

I guess this new know would be considered a new feature and therefore
not backportable to 3.4. So if this affects users (as is my suspicious),
we would have a non-fixable bug?

[1] Please consider this alternative to the two patches (it does not
include tests). FWIF, we could make this tweak configurable (default to
false) in the db.

---
diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
index 0a0964348..da1ef0b49 100644
--- a/ofproto/ofproto-dpif-xlate.c
+++ b/ofproto/ofproto-dpif-xlate.c
@@ -3415,6 +3415,7 @@ compose_sample_action(struct xlate_ctx *ctx,
 struct ofproto *ofproto = &ctx->xin->ofproto->up;
 uint32_t meter_id = ofproto->slowpath_meter_id;
 size_t cookie_offset = 0;
+size_t observe_offset;

 /* The meter action is only used to throttle userspace actions.
  * If they are not needed and the sampling rate is 100%, avoid generating
@@ -3432,6 +3433,7 @@ compose_sample_action(struct xlate_ctx *ctx,
 }

 if (args->psample) {
+observe_offset = ctx->odp_actions->size;
 odp_put_psample_action(ctx->odp_actions,
args->psample->group_id,
(void *) &args->psample->cookie,
@@ -3443,6 +3445,7 @@ compose_sample_action(struct xlate_ctx *ctx,
 nl_msg_put_u32(ctx->odp_actions, OVS_ACTION_ATTR_METER, meter_id);
 }

+observe_offset = ctx->odp_actions->size;
 odp_port_t odp_port = ofp_port_to_odp_port(
 ctx->xbridge, ctx->xin->flow.in_port.ofp_port);
 uint32_t pid = dpif_port_get_pid(ctx->xbridge->dpif, odp_port);
@@ -3457,6 +3460,9 @@ compose_sample_action(struct xlate_ctx *ctx,
 if (is_sample) {
 nl_msg_end_nested(ctx->odp_actions, actions_offset);
 nl_msg_end_nested(ctx->odp_actions, sample_offset);
+ctx->xout->last_observe_offset = sample_offset;
+} else

Re: [ovs-dev] [PATCH net-next v2] selftests: openvswitch: retry instead of sleep

[Adrián Moreno]

On Wed, Jul 10, 2024 at 11:04:59AM GMT, Adrian Moreno wrote:
> There are a couple of places where the test script "sleep"s to wait for
> some external condition to be met.
>
> This is error prone, specially in slow systems (identified in CI by
> "KSFT_MACHINE_SLOW=yes").
>
> To fix this, add a "ovs_wait" function that tries to execute a command
> a few times until it succeeds. The timeout used is set to 5s for
> "normal" systems and doubled if a slow CI machine is detected.
>
> This should make the following work:
>
> $ vng --build  \
> --config tools/testing/selftests/net/config \
> --config kernel/configs/debug.config
>
> $ vng --run . --user root -- "make -C tools/testing/selftests/ \
> KSFT_MACHINE_SLOW=yes TARGETS=net/openvswitch run_tests"
>
> Signed-off-by: Adrian Moreno 
> ---
>  .../selftests/net/openvswitch/openvswitch.sh  | 45 +++
>  .../selftests/net/openvswitch/ovs-dpctl.py|  1 +
>  2 files changed, 38 insertions(+), 8 deletions(-)
>
> diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh 
> b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> index bc71dbc18b21..cc0bfae2bafa 100755
> --- a/tools/testing/selftests/net/openvswitch/openvswitch.sh
> +++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> @@ -11,6 +11,11 @@ ksft_skip=4
>  PAUSE_ON_FAIL=no
>  VERBOSE=0
>  TRACING=0
> +WAIT_TIMEOUT=5
> +
> +if test "X$KSFT_MACHINE_SLOW" == "Xyes"; then
> + WAIT_TIMEOUT=10
> +fi
>
>  tests="
>   arp_pingeth-arp: Basic arp ping between 
> two NS
> @@ -29,6 +34,30 @@ info() {
>   [ $VERBOSE = 0 ] || echo $*
>  }
>
> +ovs_wait() {
> + info "waiting $WAIT_TIMEOUT s for: $@"
> +
> + if "$@" ; then
> + info "wait succeeded immediately"
> + return 0
> + fi
> +
> + # A quick re-check helps speed up small races in fast systems.
> + # However, fractional sleeps might not necessarily work.
> + local start=0
> + sleep 0.1 || { sleep 1; start=1; }
> +
> + for (( i=start; i + if "$@" ; then
> + info "wait succeeded after $i seconds"
> + return 0
> + fi
> + sleep 1
> + done
> + info "wait failed after $i seconds"
> + return 1
> +}
> +
>  ovs_base=`pwd`
>  sbxs=
>  sbx_add () {
> @@ -278,20 +307,19 @@ test_psample() {
>
>   # Record psample data.
>   ovs_spawn_daemon "test_psample" python3 $ovs_base/ovs-dpctl.py 
> psample-events
> + ovs_wait grep -q "listening for psample events" ${ovs_dir}/stdout
>
>   # Send a single ping.
> - sleep 1
>   ovs_sbx "test_psample" ip netns exec client ping -I c1 172.31.110.20 -c 
> 1 || return 1
> - sleep 1
>
>   # We should have received one userspace action upcall and 2 psample 
> packets.
> - grep -E "userspace action command" $ovs_dir/s0.out >/dev/null 2>&1 || 
> return 1
> + ovs_wait grep -q "userspace action command" $ovs_dir/s0.out || return 1
>
>   # client -> server samples should only contain the first 14 bytes of 
> the packet.
> - grep -E "rate:4294967295,group:1,cookie:c0ffee data:[0-9a-f]{28}$" \
> -  $ovs_dir/stdout >/dev/null 2>&1 || return 1
> - grep -E "rate:4294967295,group:2,cookie:eeff0c" \
> -  $ovs_dir/stdout >/dev/null 2>&1 || return 1
> + ovs_wait grep -qE "rate:4294967295,group:1,cookie:c0ffee 
> data:[0-9a-f]{28}$" \
> + $ovs_dir/stdout || return 1
> +
> + ovs_wait grep -q "rate:4294967295,group:2,cookie:eeff0c" 
> $ovs_dir/stdout || return 1
>
>   return 0
>  }
> @@ -711,7 +739,8 @@ test_upcall_interfaces() {
>   ovs_add_netns_and_veths "test_upcall_interfaces" ui0 upc left0 l0 \
>   172.31.110.1/24 -u || return 1
>
> - sleep 1
> + ovs_wait grep -q "listening on upcall packet handler" 
> ${ovs_dir}/left0.out
> +
>   info "sending arping"
>   ip netns exec upc arping -I l0 172.31.110.20 -c 1 \
>   >$ovs_dir/arping.stdout 2>$ovs_dir/arping.stderr
> diff --git a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py 
> b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
> index 1e15b0818074..8a0396bfaf99 100644
> --- a/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
> +++ b/tools/testing/selftests/net/openvswitch/ovs-dpctl.py
> @@ -2520,6 +2520,7 @@ class PsampleEvent(EventSocket):
>  marshal_class = psample_msg
>
>  def read_samples(self):
> +print("listening for psample events", flush=True)
>  while True:
>  try:
>  for msg in self.get():
> --
> 2.45.2
>


This patch is supposed to fix openvswitch selftests on "-dbg" machines.
However, as Simon points out, all recent rounds are failing [1]. I don't
see this patch being included in the batches and I was wondering why.

Also I see a (presumably unrelated) build error netdev/build_32bit.
Is there anything I can do?

[1]
https://netdev.bots.linux.dev/contest.htm

Re: [ovs-dev] [PATCH v1 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Thu, Jul 11, 2024 at 12:32:06AM GMT, Ilya Maximets wrote:
> On 7/10/24 23:38, Adrián Moreno wrote:
> > On Wed, Jul 10, 2024 at 11:00:43PM GMT, Ilya Maximets wrote:
> >> On 7/7/24 22:09, Adrian Moreno wrote:
> >>> When sample action gets used as a way of sampling traffic with
> >>> controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> >>> the controller will have to increase the number of flows to ensure each
> >>> part of the pipeline contains the right metadata.
> >>>
> >>> As an example, if the controller decides to sample stateful traffic, it
> >>> could store the computed metadata for each connection in the conntrack
> >>> label. However, for established connections, a flow must be created for
> >>> each different ct_label value with a sample action that contains a
> >>> different hardcoded obs_domain and obs_point id.
> >>>
> >>> This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> >>> that supports specifying the observation point and domain using an
> >>> OpenFlow field reference, so now the controller can express:
> >>>
> >>>  sample(...
> >>> obs_domain_id=NXM_NX_CT_LABEL[0..31],
> >>> obs_point_id=NXM_NX_CT_LABEL[32..63]
> >>> ...
> >>>)
> >>>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  include/openvswitch/ofp-actions.h |   8 +-
> >>>  lib/ofp-actions.c | 249 +++---
> >>>  ofproto/ofproto-dpif-xlate.c  |  55 ++-
> >>>  python/ovs/flow/ofp.py|   8 +-
> >>>  python/ovs/flow/ofp_act.py|   4 +-
> >>>  tests/ofp-actions.at  |   5 +
> >>>  tests/ofproto-dpif.at |  41 +
> >>>  tests/system-traffic.at   |  74 +
> >>>  8 files changed, 405 insertions(+), 39 deletions(-)
> >>
> >> Not a full review, it's a complicated change.  See a few comments below.
> >>
> >>>
> >>> diff --git a/include/openvswitch/ofp-actions.h 
> >>> b/include/openvswitch/ofp-actions.h
> >>> index 7b57e49ad..56dc2c147 100644
> >>> --- a/include/openvswitch/ofp-actions.h
> >>> +++ b/include/openvswitch/ofp-actions.h
> >>> @@ -1015,14 +1015,16 @@ enum nx_action_sample_direction {
> >>>
> >>>  /* OFPACT_SAMPLE.
> >>>   *
> >>> - * Used for NXAST_SAMPLE, NXAST_SAMPLE2, and NXAST_SAMPLE3. */
> >>> + * Used for NXAST_SAMPLE, NXAST_SAMPLE2, NXAST_SAMPLE3 and 
> >>> NXAST_SAMPLE4. */
> >>>  struct ofpact_sample {
> >>>  OFPACT_PADDED_MEMBERS(
> >>>  struct ofpact ofpact;
> >>>  uint16_t probability;   /* Always positive. */
> >>>  uint32_t collector_set_id;
> >>> -uint32_t obs_domain_id;
> >>> -uint32_t obs_point_id;
> >>> +uint32_t obs_domain_imm;
> >>> +struct mf_subfield obs_domain_src;
> >>> +uint32_t obs_point_imm;
> >>> +struct mf_subfield obs_point_src;
> >>>  ofp_port_t sampling_port;
> >>>  enum nx_action_sample_direction direction;
> >>>  );
> >>> diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> >>> index da7b1dd31..e329a7e3f 100644
> >>> --- a/lib/ofp-actions.c
> >>> +++ b/lib/ofp-actions.c
> >>> @@ -330,6 +330,8 @@ enum ofp_raw_action_type {
> >>>  NXAST_RAW_SAMPLE2,
> >>>  /* NX1.0+(41): struct nx_action_sample2. */
> >>>  NXAST_RAW_SAMPLE3,
> >>> +/* NX1.0+(51): struct nx_action_sample4, ... VLMFF */
> >>
> >> Why the dots are here?  The structure doesn't seem to have
> >> extra fields at the end.
> >
> > I missread the description then. I thought it was just about alignment.
> >
> >>
> >>> +NXAST_RAW_SAMPLE4,
> >>>
> >>>  /* NX1.0+(34): struct nx_action_conjunction. */
> >>>  NXAST_RAW_CONJUNCTION,
> >>> @@ -6188,6 +6190,34 @@ struct nx_action_sample2 {
> >>>   };
> >>>   OFP_ASSERT(sizeof(struct nx_action_sample2) == 32);
> >>>
> >>> +/* Action structure for NXAST_SAMPLE4
> >>> + *
> >>> + * NXAST_SAMPLE4 was added in Open vSwitch 3.4.0.  Compared to 
> >>&g

Re: [ovs-dev] [PATCH v1 11/13] ofproto: xlate: Make flow-sampled drops explicit.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 11:31:42PM GMT, Ilya Maximets wrote:
> On 7/7/24 22:09, Adrian Moreno wrote:
> > When an action set ends in a an OFP_SAMPLE action, it means the packet
> > will be dropped and sampled. Make the drop explicit in this case so that
> > drop statistics remain accurate.
> >
> > This could be done outside of the sample action, i.e: "sample(...),drop"
> > but datapaths optimize sample actions that are found in the last
> > position. So, taking into account that datapaths already report when the
> > last sample probability fails, it is safe to put the drop inside the
> > sample, i.e: "sample(...,drop)".
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-xlate.c | 16 +++--
> >  tests/drop-stats.at  | 65 
> >  tests/ofproto-dpif.at| 44 
> >  3 files changed, 123 insertions(+), 2 deletions(-)
>
> Hi, Adrian.  As we discussed before on the kernel list, I'm not sure
> this is a right change to make for a fwe reasons:
>
> 1. We do not know the user's intentions.  We do not know if they
>wanted to drop these packets or their goal was to sample them
>and it just happened to be the last action in the list, because
>they put it after the output action and not before.
>

If there are datapath actions after output action you will probably get
drops reported in both datapaths.

> 2. This patch doesn't cover cases where sampling is not actually
>the last action, but further actions do not yield any datapath
>actions.  E.g. if you have a register resoration afterwards,
>which happens in automatically built pipelines like in OVN.
>Or if the last action after sampling is learn().  And things are
>getting more complicated if we take into account resubmits and
>processing in different bridges via patch ports.

I agree this could be problematic. Maybe we should make sure the sample
is the last dp action and "fix it". A trick such as the one done for
sflow.

>
> 3. If someone is sampling the drops specifically, they know where
>to find the packets, because they configured the collector.
>

I think drop stats and samples are two different things. There are
typically extacted by different tools and systems.

Besides, what about per-bridge sampling (next patch)?
The user enables sampling on a bridge without explicitly doing it
for drops and suddenly the drop statistics dissapear.

> 4. Packets are not actually dropped, they are delivered to userspace
>or psample.  It might make sense though to drop with a reson in
>case the upcall fails or psample fails to deliver to any entity
>and it is the last action in the datapath, but that's a kernel
>change to make.
>

I don't want to get into another semantic discussion between consume,
free and drop or the dark corners of the OpenFlow protocol. For me it's
pretty clear that if the last action is to sample, the packet is
"dropped" in the sense that, from a switch' perspective, if it's not
forwarded/sent somewhere, it's dropped.

I know you don't think the same :-).

Would you then agree that the concept of dropping packets is very
unclear and OpenFlow does not make it easy (or even possible?) to
express a sampled drops and we should add an extension action to
explicitly drop packets?

> 5. Drop reporting in either datapath implementation is not 100%
>accurate anyway and requires users to know the kernel internals.
>

Shouldn't we try to make them more accurate, not less?

> Some of that also applies to the next patch in the set.
>

I can take some of the critics for this patch but for me, the next one
is bluntly fixing a bug: Per-bridge sflow/IPFIX should not break drop
statistics.

> For the patch itself, you should also check that the action set is
> actually empty (not the action list) after the sample action translation.

Aye, what about doing the sflow trick and look at the datapath flow
alone?

> There is a chance that the clone action is disabled and sample is
> used in place of a clone, I'm not sure this case is covered.

That case does not go through compose_sample_action but encodes the
datapath action directly.

> And the formatting in tests seems very inconsistent.
>

I agree, I tried to follow the drop-stats.at "style" and the result is
pretty bad.

> Best regards, Ilya Maximets.
>

Thanks.

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v1 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 11:00:43PM GMT, Ilya Maximets wrote:
> On 7/7/24 22:09, Adrian Moreno wrote:
> > When sample action gets used as a way of sampling traffic with
> > controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> > the controller will have to increase the number of flows to ensure each
> > part of the pipeline contains the right metadata.
> >
> > As an example, if the controller decides to sample stateful traffic, it
> > could store the computed metadata for each connection in the conntrack
> > label. However, for established connections, a flow must be created for
> > each different ct_label value with a sample action that contains a
> > different hardcoded obs_domain and obs_point id.
> >
> > This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> > that supports specifying the observation point and domain using an
> > OpenFlow field reference, so now the controller can express:
> >
> >  sample(...
> > obs_domain_id=NXM_NX_CT_LABEL[0..31],
> > obs_point_id=NXM_NX_CT_LABEL[32..63]
> > ...
> >)
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  include/openvswitch/ofp-actions.h |   8 +-
> >  lib/ofp-actions.c | 249 +++---
> >  ofproto/ofproto-dpif-xlate.c  |  55 ++-
> >  python/ovs/flow/ofp.py|   8 +-
> >  python/ovs/flow/ofp_act.py|   4 +-
> >  tests/ofp-actions.at  |   5 +
> >  tests/ofproto-dpif.at |  41 +
> >  tests/system-traffic.at   |  74 +
> >  8 files changed, 405 insertions(+), 39 deletions(-)
>
> Not a full review, it's a complicated change.  See a few comments below.
>
> >
> > diff --git a/include/openvswitch/ofp-actions.h 
> > b/include/openvswitch/ofp-actions.h
> > index 7b57e49ad..56dc2c147 100644
> > --- a/include/openvswitch/ofp-actions.h
> > +++ b/include/openvswitch/ofp-actions.h
> > @@ -1015,14 +1015,16 @@ enum nx_action_sample_direction {
> >
> >  /* OFPACT_SAMPLE.
> >   *
> > - * Used for NXAST_SAMPLE, NXAST_SAMPLE2, and NXAST_SAMPLE3. */
> > + * Used for NXAST_SAMPLE, NXAST_SAMPLE2, NXAST_SAMPLE3 and NXAST_SAMPLE4. 
> > */
> >  struct ofpact_sample {
> >  OFPACT_PADDED_MEMBERS(
> >  struct ofpact ofpact;
> >  uint16_t probability;   /* Always positive. */
> >  uint32_t collector_set_id;
> > -uint32_t obs_domain_id;
> > -uint32_t obs_point_id;
> > +uint32_t obs_domain_imm;
> > +struct mf_subfield obs_domain_src;
> > +uint32_t obs_point_imm;
> > +struct mf_subfield obs_point_src;
> >  ofp_port_t sampling_port;
> >  enum nx_action_sample_direction direction;
> >  );
> > diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> > index da7b1dd31..e329a7e3f 100644
> > --- a/lib/ofp-actions.c
> > +++ b/lib/ofp-actions.c
> > @@ -330,6 +330,8 @@ enum ofp_raw_action_type {
> >  NXAST_RAW_SAMPLE2,
> >  /* NX1.0+(41): struct nx_action_sample2. */
> >  NXAST_RAW_SAMPLE3,
> > +/* NX1.0+(51): struct nx_action_sample4, ... VLMFF */
>
> Why the dots are here?  The structure doesn't seem to have
> extra fields at the end.

I missread the description then. I thought it was just about alignment.

>
> > +NXAST_RAW_SAMPLE4,
> >
> >  /* NX1.0+(34): struct nx_action_conjunction. */
> >  NXAST_RAW_CONJUNCTION,
> > @@ -6188,6 +6190,34 @@ struct nx_action_sample2 {
> >   };
> >   OFP_ASSERT(sizeof(struct nx_action_sample2) == 32);
> >
> > +/* Action structure for NXAST_SAMPLE4
> > + *
> > + * NXAST_SAMPLE4 was added in Open vSwitch 3.4.0.  Compared to 
> > NXAST_SAMPLE3,
> > + * it adds support for using field specifiers for observation_domain_id and
> > + * observation_point_id. */
> > +struct nx_action_sample4 {
> > +ovs_be16 type;  /* OFPAT_VENDOR. */
> > +ovs_be16 len;   /* Length is 32. */
>
> Is the length 32?  40, I suppose.

Yep

>
> > +ovs_be32 vendor;/* NX_VENDOR_ID. */
> > +ovs_be16 subtype;   /* NXAST_SAMPLE. */
>
> NXAST_SAMPLE4 ?

Ack.

>
> > +ovs_be16 probability;   /* Fraction of packets to sample. */
> > +ovs_be32 collector_set_id;  /* ID of collector set in OVSDB. */
> > +ovs_be32 obs_domain_src;/* Source of the 
> > observation_domain_id. */
> > +union {
> > +ovs_be16 obs_domain_ofs_nbits;  /* Range to use from source field. 
> > */
> > +ovs_be32 obs_domain_imm;/* Immediate value for domain id. 
> > */
> > +};
> > +ovs_be32 obs_point_src; /* Source of the observation_point_id 
> > */
> > +union {
> > +ovs_be16 obs_point_ofs_nbits;  /* Range to use from source field. 
> > */
> > +ovs_be32 obs_point_imm;/* Immediate value for point id. */
> > +};
> > +ovs_be16 sampling_port; /* Sampling port. */
> > +uint8_t  direction; /* Sampling direction. */
> > +uint8_t  zeros[5];  

Re: [ovs-dev] [PATCH v1 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 10:26:23PM GMT, Ilya Maximets wrote:
> On 7/9/24 16:25, Eelco Chaudron wrote:
> >
> >
> > On 9 Jul 2024, at 16:17, Adrián Moreno wrote:
> >
> >> On Tue, Jul 09, 2024 at 11:46:12AM GMT, Eelco Chaudron wrote:
> >>> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
> >>>
> >>>> When sample action gets used as a way of sampling traffic with
> >>>> controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> >>>> the controller will have to increase the number of flows to ensure each
> >>>> part of the pipeline contains the right metadata.
> >>>>
> >>>> As an example, if the controller decides to sample stateful traffic, it
> >>>> could store the computed metadata for each connection in the conntrack
> >>>> label. However, for established connections, a flow must be created for
> >>>> each different ct_label value with a sample action that contains a
> >>>> different hardcoded obs_domain and obs_point id.
> >>>>
> >>>> This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> >>>> that supports specifying the observation point and domain using an
> >>>> OpenFlow field reference, so now the controller can express:
> >>>>
> >>>>  sample(...
> >>>> obs_domain_id=NXM_NX_CT_LABEL[0..31],
> >>>> obs_point_id=NXM_NX_CT_LABEL[32..63]
> >>>> ...
> >>>>)
> >>>>
> >>>> Signed-off-by: Adrian Moreno 
> >>>
> >>> See some comments inline. I’m missing the documentation update.
> >>>
> >>
> >> Yes. I noticed I missed both the NEWS and documentation update.
> >>
>
> You're also missing unit tests in ovs-ofctl.at.
>

Ack.

> >>>> index 323a58cbf..2aff48f5e 100644
> >>>> --- a/ofproto/ofproto-dpif-xlate.c
> >>>> +++ b/ofproto/ofproto-dpif-xlate.c
> >>>> @@ -5909,6 +5909,44 @@ xlate_fin_timeout(struct xlate_ctx *ctx,
> >>>>  }
> >>>>  }
> >>>>
> >>>> +static uint32_t
> >>>> +ofpact_sample_get_domain(struct xlate_ctx *ctx,
> >>>> + const struct ofpact_sample *os)
> >>>> +{
> >>>> +if (os->obs_domain_src.field) {
> >>>> +union mf_subvalue *value = xmalloc(sizeof *value);
> >>>
> >>> Would it be better to just put these 128 bytes on the stack to avoid
> >>> memory fragmentation? Same for the next function.
> >>>
> >>> Or maybe something better, we could do something like we did for
> >>> exact_match_mask?
> >>>
> >>>   const union mf_value exact_match_mask = MF_VALUE_EXACT_INITIALIZER;
> >>>   extern const union mf_value exact_match_mask;
> >>>
> >>
> >> A quick look seemed to indicate dynamic memory was preferred in this
> >> module so I assumed there was a reason for it (I do remember stack size
> >> being a problem at some point, not sure if it was here).
> >>
> >> I'll look at the exact_match_mask case.
> >
> > I guess this function is not called recursively, so we should be ok with 
> > the stack usage. But a const variable might be better (and we can fix the 
> > other place(s) where you copied this code from ;).
>
> While the function itself is not called recursively, the problem is
> that it can be inlined into the main do_xlate_actions and cause
> a significant stack usage increase even if the sampling is not used.
> So, extra investigation on what is getting inlined with different
> compilers and compiler flags is needed if we want to allocate it
> on stack.
>
> Best regards, Ilya Maximets.
>

For the next version, I have implemented Eelco's suggestion: an extern
constant all-one enum mf_subfield. That should avoid both stack increase
and memory fragmentation

Cheers,
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v1 06/13] ofproto-dpif-xlate: Use psample for local sample.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 06:52:30PM GMT, Ilya Maximets wrote:
> On 7/10/24 15:25, Adrián Moreno wrote:
> > On Wed, Jul 10, 2024 at 02:56:44PM GMT, Ilya Maximets wrote:
> >> On 7/7/24 22:08, Adrian Moreno wrote:
> >>> Use the newly added psample action to implement OpenFlow sample() actions
> >>> with local sampling configuration if possible.
> >>>
> >>> A bit of refactoring in compose_sample_actions arguments helps make it a
> >>> bit more readable.
> >>>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  ofproto/ofproto-dpif-lsample.c |  16 +++
> >>>  ofproto/ofproto-dpif-lsample.h |   5 +
> >>>  ofproto/ofproto-dpif-xlate.c   | 251 +++--
> >>>  ofproto/ofproto-dpif-xlate.h   |   5 +-
> >>>  ofproto/ofproto-dpif.c |   2 +-
> >>>  tests/ofproto-dpif.at  | 146 +++
> >>>  6 files changed, 345 insertions(+), 80 deletions(-)
> >>>
> >>> diff --git a/ofproto/ofproto-dpif-lsample.c 
> >>> b/ofproto/ofproto-dpif-lsample.c
> >>> index d675a116f..534ad96f0 100644
> >>> --- a/ofproto/ofproto-dpif-lsample.c
> >>> +++ b/ofproto/ofproto-dpif-lsample.c
> >>> @@ -140,6 +140,22 @@ dpif_lsample_set_options(struct dpif_lsample 
> >>> *lsample,
> >>>  return changed;
> >>>  }
> >>>
> >>> +/* Returns the group_id for a given collector_set_id, if it exists. */
> >>> +bool
> >>> +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> >>> collector_set_id,
> >>> +  uint32_t *group_id)
> >>> +{
> >>> +struct lsample_exporter_node *node;
> >>> +bool found = false;
> >>> +
> >>> +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> >>> +if (node) {
> >>> +found = true;
> >>> +*group_id = node->exporter.options.group_id;
> >>> +}
> >>> +return found;
> >>> +}
> >>> +
> >>>  struct dpif_lsample *
> >>>  dpif_lsample_create(void)
> >>>  {
> >>> diff --git a/ofproto/ofproto-dpif-lsample.h 
> >>> b/ofproto/ofproto-dpif-lsample.h
> >>> index bee36c9c5..9c1026551 100644
> >>> --- a/ofproto/ofproto-dpif-lsample.h
> >>> +++ b/ofproto/ofproto-dpif-lsample.h
> >>> @@ -18,6 +18,7 @@
> >>>  #define OFPROTO_DPIF_LSAMPLE_H 1
> >>>
> >>>  #include 
> >>> +#include 
> >>>  #include 
> >>>
> >>>  struct dpif_lsample;
> >>> @@ -33,4 +34,8 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >>>const struct ofproto_lsample_options *,
> >>>size_t n_opts);
> >>>
> >>> +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> >>> +   uint32_t collector_set_id,
> >>> +   uint32_t *group_id);
> >>> +
> >>>  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> >>> diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> >>> index 7c4950895..5e8113d5e 100644
> >>> --- a/ofproto/ofproto-dpif-xlate.c
> >>> +++ b/ofproto/ofproto-dpif-xlate.c
> >>> @@ -47,6 +47,7 @@
> >>>  #include "ofproto/ofproto-dpif-ipfix.h"
> >>>  #include "ofproto/ofproto-dpif-mirror.h"
> >>>  #include "ofproto/ofproto-dpif-monitor.h"
> >>> +#include "ofproto/ofproto-dpif-lsample.h"
> >>>  #include "ofproto/ofproto-dpif-sflow.h"
> >>>  #include "ofproto/ofproto-dpif-trace.h"
> >>>  #include "ofproto/ofproto-dpif-xlate-cache.h"
> >>> @@ -117,6 +118,7 @@ struct xbridge {
> >>>  struct dpif_sflow *sflow; /* SFlow handle, or null. */
> >>>  struct dpif_ipfix *ipfix; /* Ipfix handle, or null. */
> >>>  struct netflow *netflow;  /* Netflow handle, or null. */
> >>> +struct dpif_lsample *lsample; /* Local sample handle, or null. */
> >>>  struct stp *stp;  /* STP or null if disabled. */
> >>>  struct rstp *rstp;/* RSTP or null if disabled. */
> >>>
> >>> @@ -686,6 +688,7 @@ static void xlate_xbridge_set(struct xbridge *, 
&

Re: [ovs-dev] [PATCH v1 08/13] ofproto-dpif-lsample: Show stats via unixctl.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 04:28:55PM GMT, Eelco Chaudron wrote:
>
>
> On 10 Jul 2024, at 15:06, Ilya Maximets wrote:
>
> > On 7/7/24 22:09, Adrian Moreno wrote:
> >> Add a command to dump statistics per exporter.
> >>
> >> Signed-off-by: Adrian Moreno 
> >> ---
> >>  NEWS   |   2 +
> >>  ofproto/ofproto-dpif-lsample.c | 111 +
> >>  ofproto/ofproto-dpif-lsample.h |   1 +
> >>  ofproto/ofproto-dpif.c |   1 +
> >>  4 files changed, 115 insertions(+)
> >>
> >> diff --git a/NEWS b/NEWS
> >> index 15faf9fc3..1c53badea 100644
> >> --- a/NEWS
> >> +++ b/NEWS
> >> @@ -20,6 +20,8 @@ Post-v3.3.0
> >>   allows samples to be emitted locally (instead of via IPFIX) in a
> >>   datapath-specific manner. The Linux kernel datapath is the first to
> >>   support this feature by using the new datapath "psample" action.
> >> + A new unixctl command 'lsample/show' shows packet and bytes 
> >> statistics
> >> + per local sample exporter.
> >
> > Maybe add this as a separate bullet point.
> >
> >>
> >>
> >>  v3.3.0 - 16 Feb 2024
> >> diff --git a/ofproto/ofproto-dpif-lsample.c 
> >> b/ofproto/ofproto-dpif-lsample.c
> >> index 171129d5b..82a87c27d 100644
> >> --- a/ofproto/ofproto-dpif-lsample.c
> >> +++ b/ofproto/ofproto-dpif-lsample.c
> >> @@ -21,7 +21,10 @@
> >>  #include "dpif.h"
> >>  #include "hash.h"
> >>  #include "ofproto.h"
> >> +#include "ofproto-dpif.h"
> >> +#include "openvswitch/dynamic-string.h"
> >>  #include "openvswitch/thread.h"
> >> +#include "unixctl.h"
> >>
> >>  /* Dpif local sampling.
> >>   *
> >> @@ -219,3 +222,111 @@ dpif_lsample_unref(struct dpif_lsample *lsample)
> >>  dpif_lsample_destroy(lsample);
> >>  }
> >>  }
> >> +
> >> +static int
> >> +comp_exporter_collector_id(const void *a_, const void *b_)
> >> +{
> >> +const struct lsample_exporter_node *a, *b;
> >> +
> >> +a = *(struct lsample_exporter_node **) a_;
> >> +b = *(struct lsample_exporter_node **) b_;
> >> +
> >> +if (a->exporter.options.collector_set_id >
> >> +b->exporter.options.collector_set_id) {
> >> +return 1;
> >> +}
> >> +if (a->exporter.options.collector_set_id <
> >> +b->exporter.options.collector_set_id) {
> >> +return -1;
> >> +}
> >> +return 0;
> >> +}
> >> +
> >> +static void
> >> +lsample_exporter_list(struct dpif_lsample *lsample,
> >> +  struct lsample_exporter_node ***list,
> >> +  size_t *num_exporters)
> >> +{
> >> +struct lsample_exporter_node **exporter_list;
> >> +struct lsample_exporter_node *node;
> >> +size_t k = 0, n;
> >> +
> >> +n = cmap_count(&lsample->exporters);
> >> +
> >> +exporter_list = xcalloc(n, sizeof *exporter_list);
> >> +
> >> +CMAP_FOR_EACH (node, node, &lsample->exporters) {
> >> +if (k >= n) {
> >> +break;
> >> +}
> >> +exporter_list[k++] = node;
> >
> > There is no guarantee that cmap didn't change between getting the
> > number of exporters and iterating over them.  Need to either take
> > a lock or grow the array on demand.
>
> I initially thought the same, but it’s covered by the break above.
>
> Thinking about it again, we might actually not show entries that were 
> displayed before, due to a new one being earlier in the map. So I guess we 
> should probably grow the exporter_list (rather than hold the lock).
>

Right. If the map has grown since we took the size we will break before
iterating over all of them k will be equal to n we will break.
If the the map has shrinked, CMAP_FOR_EACH will finish.

AFAICS, in both cases it's safe to access the data (nodes will not be
freed until the next quiese period).

What can happen is that we don't show exactly what the current state of
the map is, e.g: we show a node that has just been deleted from the map
or we don't show a node that has just been added.

The only reason for having more than one exporter per bridge is if we
want to have multiple "observability applications" having differentt
group ids. In any case, I don't expect the number to be very high nor its
creation/deletion rate.

With all this, I considered that having sporadic slighly-inaccurate
displays was an asumable cost.

What do you think?


> >> +}
> >> +
> >> +qsort(exporter_list, k, sizeof *exporter_list, 
> >> comp_exporter_collector_id);
> >> +
> >> +*list = exporter_list;
> >> +*num_exporters = k;
> >> +}
> >> +
> >> +static void
> >> +lsample_unixctl_show(struct unixctl_conn *conn, int argc OVS_UNUSED,
> >> + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED)
> >> +{
> >> +struct lsample_exporter_node **node_list = NULL;
> >> +struct ds ds = DS_EMPTY_INITIALIZER;
> >> +const struct ofproto_dpif *ofproto;
> >> +size_t i, num;
> >> +
> >> +ofproto = ofproto_dpif_lookup_by_name(argv[1]);
> >> +if (!ofproto) {
> >> +unixctl_command_reply_error(conn, "no such 

Re: [ovs-dev] [PATCH v1 06/13] ofproto-dpif-xlate: Use psample for local sample.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 02:56:44PM GMT, Ilya Maximets wrote:
> On 7/7/24 22:08, Adrian Moreno wrote:
> > Use the newly added psample action to implement OpenFlow sample() actions
> > with local sampling configuration if possible.
> >
> > A bit of refactoring in compose_sample_actions arguments helps make it a
> > bit more readable.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-lsample.c |  16 +++
> >  ofproto/ofproto-dpif-lsample.h |   5 +
> >  ofproto/ofproto-dpif-xlate.c   | 251 +++--
> >  ofproto/ofproto-dpif-xlate.h   |   5 +-
> >  ofproto/ofproto-dpif.c |   2 +-
> >  tests/ofproto-dpif.at  | 146 +++
> >  6 files changed, 345 insertions(+), 80 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index d675a116f..534ad96f0 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -140,6 +140,22 @@ dpif_lsample_set_options(struct dpif_lsample *lsample,
> >  return changed;
> >  }
> >
> > +/* Returns the group_id for a given collector_set_id, if it exists. */
> > +bool
> > +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> > collector_set_id,
> > +  uint32_t *group_id)
> > +{
> > +struct lsample_exporter_node *node;
> > +bool found = false;
> > +
> > +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> > +if (node) {
> > +found = true;
> > +*group_id = node->exporter.options.group_id;
> > +}
> > +return found;
> > +}
> > +
> >  struct dpif_lsample *
> >  dpif_lsample_create(void)
> >  {
> > diff --git a/ofproto/ofproto-dpif-lsample.h b/ofproto/ofproto-dpif-lsample.h
> > index bee36c9c5..9c1026551 100644
> > --- a/ofproto/ofproto-dpif-lsample.h
> > +++ b/ofproto/ofproto-dpif-lsample.h
> > @@ -18,6 +18,7 @@
> >  #define OFPROTO_DPIF_LSAMPLE_H 1
> >
> >  #include 
> > +#include 
> >  #include 
> >
> >  struct dpif_lsample;
> > @@ -33,4 +34,8 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >const struct ofproto_lsample_options *,
> >size_t n_opts);
> >
> > +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> > +   uint32_t collector_set_id,
> > +   uint32_t *group_id);
> > +
> >  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 7c4950895..5e8113d5e 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -47,6 +47,7 @@
> >  #include "ofproto/ofproto-dpif-ipfix.h"
> >  #include "ofproto/ofproto-dpif-mirror.h"
> >  #include "ofproto/ofproto-dpif-monitor.h"
> > +#include "ofproto/ofproto-dpif-lsample.h"
> >  #include "ofproto/ofproto-dpif-sflow.h"
> >  #include "ofproto/ofproto-dpif-trace.h"
> >  #include "ofproto/ofproto-dpif-xlate-cache.h"
> > @@ -117,6 +118,7 @@ struct xbridge {
> >  struct dpif_sflow *sflow; /* SFlow handle, or null. */
> >  struct dpif_ipfix *ipfix; /* Ipfix handle, or null. */
> >  struct netflow *netflow;  /* Netflow handle, or null. */
> > +struct dpif_lsample *lsample; /* Local sample handle, or null. */
> >  struct stp *stp;  /* STP or null if disabled. */
> >  struct rstp *rstp;/* RSTP or null if disabled. */
> >
> > @@ -686,6 +688,7 @@ static void xlate_xbridge_set(struct xbridge *, struct 
> > dpif *,
> >const struct mbridge *,
> >const struct dpif_sflow *,
> >const struct dpif_ipfix *,
> > +  const struct dpif_lsample *,
> >const struct netflow *,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *,
> > @@ -1069,6 +1072,7 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >const struct mbridge *mbridge,
> >const struct dpif_sflow *sflow,
> >const struct dpif_ipfix *ipfix,
> > +  const struct dpif_lsample *lsample,
> >const struct netflow *netflow,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *support,
> > @@ -1099,6 +1103,11 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >  xbridge->ipfix = dpif_ipfix_ref(ipfix);
> >  }
> >
> > +if (xbridge->lsample != lsample) {
> > +dpif_lsample_unref(xbridge->lsample);
> > +xbridge->lsample = dpif_lsample_ref(lsample);
> > +}
> > +
> >  if (xbridge->stp != stp) {
> >  stp_unref(xbridge->stp);
> >  xbridge->stp = stp_ref(stp);
> > @@ -1213,9 +1222,10 @@ xlate_xbridge_copy(struct xbridge *xbridge)
> >  xlate_xbridge_set(new_xbr

Re: [ovs-dev] [PATCH v1 06/13] ofproto-dpif-xlate: Use psample for local sample.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 02:18:35PM GMT, Ilya Maximets wrote:
> On 7/9/24 16:03, Eelco Chaudron wrote:
> >
> >
> > On 9 Jul 2024, at 15:52, Adrián Moreno wrote:
> >
> >> On Tue, Jul 09, 2024 at 11:45:51AM GMT, Eelco Chaudron wrote:
> >>> On 7 Jul 2024, at 22:08, Adrian Moreno wrote:
> >>>
> >>>> Use the newly added psample action to implement OpenFlow sample() actions
> >>>> with local sampling configuration if possible.
> >>>>
> >>>> A bit of refactoring in compose_sample_actions arguments helps make it a
> >>>> bit more readable.
> >>>
> >>> Some comments below.
> >>>
> >>> Cheers,
> >>>
> >>> Eelco
> >>>
> >>>> Signed-off-by: Adrian Moreno 
> >>>> ---
> >>>>  ofproto/ofproto-dpif-lsample.c |  16 +++
> >>>>  ofproto/ofproto-dpif-lsample.h |   5 +
> >>>>  ofproto/ofproto-dpif-xlate.c   | 251 +++--
> >>>>  ofproto/ofproto-dpif-xlate.h   |   5 +-
> >>>>  ofproto/ofproto-dpif.c |   2 +-
> >>>>  tests/ofproto-dpif.at  | 146 +++
> >>>>  6 files changed, 345 insertions(+), 80 deletions(-)
> >>>>
> >>>> diff --git a/ofproto/ofproto-dpif-lsample.c 
> >>>> b/ofproto/ofproto-dpif-lsample.c
> >>>> index d675a116f..534ad96f0 100644
> >>>> --- a/ofproto/ofproto-dpif-lsample.c
> >>>> +++ b/ofproto/ofproto-dpif-lsample.c
> >>>> @@ -140,6 +140,22 @@ dpif_lsample_set_options(struct dpif_lsample 
> >>>> *lsample,
> >>>>  return changed;
> >>>>  }
> >>>>
> >>>> +/* Returns the group_id for a given collector_set_id, if it exists. */
> >>>> +bool
> >>>> +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> >>>> collector_set_id,
> >>>> +  uint32_t *group_id)
> >>>> +{
> >>>> +struct lsample_exporter_node *node;
> >>>> +bool found = false;
> >>>> +
> >>>> +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> >>>> +if (node) {
> >>>> +found = true;
> >>>> +*group_id = node->exporter.options.group_id;
> >>>> +}
> >>>> +return found;
> >>>> +}
> >>>> +
> >>>>  struct dpif_lsample *
> >>>>  dpif_lsample_create(void)
> >>>>  {
> >>>> diff --git a/ofproto/ofproto-dpif-lsample.h 
> >>>> b/ofproto/ofproto-dpif-lsample.h
> >>>> index bee36c9c5..9c1026551 100644
> >>>> --- a/ofproto/ofproto-dpif-lsample.h
> >>>> +++ b/ofproto/ofproto-dpif-lsample.h
> >>>> @@ -18,6 +18,7 @@
> >>>>  #define OFPROTO_DPIF_LSAMPLE_H 1
> >>>>
> >>>>  #include 
> >>>> +#include 
> >>>>  #include 
> >>>>
> >>>>  struct dpif_lsample;
> >>>> @@ -33,4 +34,8 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >>>>const struct ofproto_lsample_options *,
> >>>>size_t n_opts);
> >>>>
> >>>> +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> >>>> +   uint32_t collector_set_id,
> >>>> +   uint32_t *group_id);
> >>>> +
> >>>>  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> >>>> diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> >>>> index 7c4950895..5e8113d5e 100644
> >>>> --- a/ofproto/ofproto-dpif-xlate.c
> >>>> +++ b/ofproto/ofproto-dpif-xlate.c
> >>>> @@ -47,6 +47,7 @@
> >>>>  #include "ofproto/ofproto-dpif-ipfix.h"
> >>>>  #include "ofproto/ofproto-dpif-mirror.h"
> >>>>  #include "ofproto/ofproto-dpif-monitor.h"
> >>>> +#include "ofproto/ofproto-dpif-lsample.h"
> >>>
> >>> Add in alphabetical order?
> >>
> >> Ack.
> >>
> >>>
> >>>>  #include "ofproto/ofproto-dpif-sflow.h"
> >>>>  #include "ofproto/ofproto-dpif-trace.h"
> >>>>  #include "ofproto

Re: [ovs-dev] [PATCH v1 03/13] ofproto_dpif: Check for psample support.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 01:28:29PM GMT, Ilya Maximets wrote:
> On 7/7/24 22:08, Adrian Moreno wrote:
> > Only kernel datapath supports this action so add a function in dpif.c
> > that checks for that.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  lib/dpif.c |  7 +++
> >  lib/dpif.h |  1 +
> >  ofproto/ofproto-dpif.c | 43 ++
> >  ofproto/ofproto-dpif.h |  7 ++-
> >  4 files changed, 57 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/dpif.c b/lib/dpif.c
> > index 71728badc..0a964ba89 100644
> > --- a/lib/dpif.c
> > +++ b/lib/dpif.c
> > @@ -1953,6 +1953,13 @@ dpif_supports_lb_output_action(const struct dpif 
> > *dpif)
> >  return dpif_is_netdev(dpif);
> >  }
> >
> > +bool
> > +dpif_may_support_psample(const struct dpif *dpif)
> > +{
> > +/* Userspace datapath does not support this action. */
> > +return !dpif_is_netdev(dpif);
> > +}
> > +
> >  /* Meters */
> >  void
> >  dpif_meter_get_features(const struct dpif *dpif,
> > diff --git a/lib/dpif.h b/lib/dpif.h
> > index a764e8a59..6bef7d5b3 100644
> > --- a/lib/dpif.h
> > +++ b/lib/dpif.h
> > @@ -941,6 +941,7 @@ int dpif_get_pmds_for_port(const struct dpif * dpif, 
> > odp_port_t port_no,
> >  char *dpif_get_dp_version(const struct dpif *);
> >  bool dpif_supports_tnl_push_pop(const struct dpif *);
> >  bool dpif_may_support_explicit_drop_action(const struct dpif *);
> > +bool dpif_may_support_psample(const struct dpif *);
> >  bool dpif_synced_dp_layers(struct dpif *);
> >
> >  /* Log functions. */
> > diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> > index 4712d10a8..94c84d697 100644
> > --- a/ofproto/ofproto-dpif.c
> > +++ b/ofproto/ofproto-dpif.c
> > @@ -873,6 +873,12 @@ ovs_lb_output_action_supported(struct ofproto_dpif 
> > *ofproto)
> >  return ofproto->backer->rt_support.lb_output_action;
> >  }
> >
> > +bool
> > +ovs_psample_supported(struct ofproto_dpif *ofproto)
> > +{
> > +return ofproto->backer->rt_support.psample;
> > +}
> > +
> >  /* Tests whether 'backer''s datapath supports recirculation.  Only newer
> >   * datapaths support OVS_KEY_ATTR_RECIRC_ID in keys.  We need to disable 
> > some
> >   * features on older datapaths that don't support this feature.
> > @@ -1609,6 +1615,42 @@ check_add_mpls(struct dpif_backer *backer)
> >  return supported;
> >  }
> >
> > +/* Tests whether 'backer''s datapath supports the OVS_ACTION_ATTR_PSAMPLE
> > + * action. */
> > +static bool
> > +check_psample(struct dpif_backer *backer)
> > +{
> > +uint8_t cookie[OVS_PSAMPLE_COOKIE_MAX_SIZE];
> > +struct odputil_keybuf keybuf;
> > +struct ofpbuf actions;
> > +struct ofpbuf key;
> > +struct flow flow;
> > +bool supported;
> > +
> > +struct odp_flow_key_parms odp_parms = {
> > +.flow = &flow,
> > +.probe = true,
> > +};
> > +
> > +memset(&flow, 0, sizeof flow);
> > +ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> > +odp_flow_key_from_flow(&odp_parms, &key);
> > +ofpbuf_init(&actions, 64);
> > +
> > +/* Generate a random max-size cookie. */
> > +random_bytes(cookie, sizeof(cookie));
>
> Don't parenthesize the argument of sizeof.
>

Ack.

> > +
> > +odp_put_psample_action(&actions, 10, cookie, sizeof cookie);
> > +
> > +supported = dpif_may_support_psample(backer->dpif) &&
> > +dpif_probe_feature(backer->dpif, "psample", &key, &actions, NULL);
>
> This function actually installs the flow into the kernel.
> You need to make sure that no packet can match it, e.g.
> by setting a bogus eth type as a match.  Some existing
> probe functions require specific matches, some are just
> missing this part as well, but we should not blindly copy
> bad patterns.
>

I guess I was always thinking on a freshly started system but you're
right! If OVS is restarted, the installed flow can affect actual
traffic. I'll add a bogus key to make sure it's not problematic.

> > +
> > +ofpbuf_uninit(&actions);
> > +VLOG_INFO("%s: Datapath %s psample", dpif_name(backer->dpif),
>
> '%s: Datapath %s psample action'  will be more clear.
>

Ack.

> > +  supported ? "supports" : "does not support");
> > +return supported;
> > +}
> > +
> >  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)  
> >  \
> >  static bool
> >  \
> >  check_##NAME(struct dpif_backer *backer)   
> >  \
> > @@ -1698,6 +1740,7 @@ check_support(struct dpif_backer *backer)
> >  dpif_supports_lb_output_action(backer->dpif);
> >  backer->rt_support.ct_zero_snat = dpif_supports_ct_zero_snat(backer);
> >  backer->rt_support.add_mpls = check_add_mpls(backer);
> > +backer->rt_support.psample = check_psample(backer);
> >
> >  /* Flow fields. */
> >  backer->rt_support.odp.ct_state = check_ct_state(backer);
> > diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofprot

Re: [ovs-dev] [PATCH v1 02/13] odp-util: Add support OVS_ACTION_ATTR_PSAMPLE.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 12:56:57PM GMT, Ilya Maximets wrote:
> On 7/7/24 22:08, Adrian Moreno wrote:
> > Add support for parsing and formatting the new action.
> >
> > Also, flag OVS_ACTION_ATTR_SAMPLE as requiring datapath assistance if it
> > contains a nested OVS_ACTION_ATTR_PSAMPLE. The reason is that the
> > sampling rate form the parent "sample" is made available to the nested
> > "psample" by the kernel.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  include/linux/openvswitch.h  | 28 +++
> >  lib/dpif-netdev.c|  1 +
> >  lib/dpif.c   |  1 +
> >  lib/odp-execute.c| 25 +-
> >  lib/odp-util.c   | 93 
> >  lib/odp-util.h   |  3 ++
> >  ofproto/ofproto-dpif-ipfix.c |  1 +
> >  ofproto/ofproto-dpif-sflow.c |  1 +
> >  python/ovs/flow/odp.py   |  8 
> >  tests/odp.at | 16 +++
> >  10 files changed, 176 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/linux/openvswitch.h b/include/linux/openvswitch.h
> > index d9fb991ef..0023b65fb 100644
> > --- a/include/linux/openvswitch.h
> > +++ b/include/linux/openvswitch.h
> > @@ -992,6 +992,31 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
> > +/**
> > + * enum ovs_pample_attr - Attributes for %OVS_ACTION_ATTR_PSAMPLE
> > + * action.
> > + *
> > + * @OVS_PSAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > + * sample.
> > + * @OVS_PSAMPLE_ATTR_COOKIE: An optional variable-length binary cookie that
> > + * contains user-defined metadata. The maximum length is
> > + * OVS_PSAMPLE_COOKIE_MAX_SIZE bytes.
> > + *
> > + * Sends the packet to the psample multicast group with the specified 
> > group and
> > + * cookie. It is possible to combine this action with the
> > + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the sample.
> > + */
> > +enum ovs_psample_attr {
> > +OVS_PSAMPLE_ATTR_GROUP = 1,/* u32 number. */
> > +OVS_PSAMPLE_ATTR_COOKIE,   /* Optional, user specified cookie. 
> > */
> > +
> > +/* private: */
> > +__OVS_PSAMPLE_ATTR_MAX
> > +};
> > +
> > +#define OVS_PSAMPLE_ATTR_MAX (__OVS_PSAMPLE_ATTR_MAX - 1)
> > +
> >  /**
> >   * enum ovs_action_attr - Action types.
> >   *
> > @@ -1056,6 +1081,8 @@ struct check_pkt_len_arg {
> >   * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> >   * argument.
> >   * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > + * @OVS_ACTION_ATTR_PSAMPLE: Send a sample of the packet to external 
> > observers
> > + * via psample.
> >   */
> >
> >  enum ovs_action_attr {
> > @@ -1087,6 +1114,7 @@ enum ovs_action_attr {
> > OVS_ACTION_ATTR_ADD_MPLS, /* struct ovs_action_add_mpls. */
> > OVS_ACTION_ATTR_DEC_TTL,  /* Nested OVS_DEC_TTL_ATTR_*. */
> > OVS_ACTION_ATTR_DROP, /* u32 xlate_error. */
> > +   OVS_ACTION_ATTR_PSAMPLE,  /* Nested OVS_PSAMPLE_ATTR_*. */
> >
> >  #ifndef __KERNEL__
> > OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > index c7f9e1490..f0594e5f5 100644
> > --- a/lib/dpif-netdev.c
> > +++ b/lib/dpif-netdev.c
> > @@ -9519,6 +9519,7 @@ dp_execute_cb(void *aux_, struct dp_packet_batch 
> > *packets_,
> >  case OVS_ACTION_ATTR_DROP:
> >  case OVS_ACTION_ATTR_ADD_MPLS:
> >  case OVS_ACTION_ATTR_DEC_TTL:
> > +case OVS_ACTION_ATTR_PSAMPLE:
> >  case __OVS_ACTION_ATTR_MAX:
> >  OVS_NOT_REACHED();
> >  }
> > diff --git a/lib/dpif.c b/lib/dpif.c
> > index 23eb18495..71728badc 100644
> > --- a/lib/dpif.c
> > +++ b/lib/dpif.c
> > @@ -1192,6 +1192,7 @@ dpif_execute_helper_cb(void *aux_, struct 
> > dp_packet_batch *packets_,
> >  case OVS_ACTION_ATTR_TUNNEL_PUSH:
> >  case OVS_ACTION_ATTR_TUNNEL_POP:
> >  case OVS_ACTION_ATTR_USERSPACE:
> > +case OVS_ACTION_ATTR_PSAMPLE:
> >  case OVS_ACTION_ATTR_RECIRC: {
> >  struct dpif_execute execute;
> >  struct ofpbuf execute_actions;
> > diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> > index 081e4d432..15577d539 100644
> > --- a/lib/odp-execute.c
> > +++ b/lib/odp-execute.c
> > @@ -818,13 +818,13 @@ requires_datapath_assistance(const struct nlattr *a)
> >  case OVS_ACTION_ATTR_RECIRC:
> >  case OVS_ACTION_ATTR_CT:
> >  case OVS_ACTION_ATTR_METER:
> > +case OVS_ACTION_ATTR_PSAMPLE:
> >  return true;
> >
> >  case OVS_ACTION_ATTR_SET:
> >  case OVS_ACTION_ATTR_SET_MASKED:
> >  case OVS_ACTION_ATTR_PUSH_VLAN:
> >  case OVS_ACTION_ATTR_POP_VLAN:
> > -case OVS_ACTION_ATTR_SAMPLE:
> >  case OVS_ACTION_ATTR_HASH:
> >  case OVS_ACTION_ATTR_PUSH_MPLS:
> >  case OVS_ACTION_ATTR_POP_MPLS:
> > @@ -841,6 +841,28 @@ requires_datapath_assistance(const struct nlattr *a)
> >  case OVS_ACTION_ATTR_DROP:
> >  return false;
> >
> > +case OVS_ACTION_ATTR_SAMPLE: {
> > +   

Re: [ovs-dev] [PATCH v1 02/13] odp-util: Add support OVS_ACTION_ATTR_PSAMPLE.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 01:01:03AM GMT, Ilya Maximets wrote:
> On 7/9/24 21:07, Adrián Moreno wrote:
> > On Tue, Jul 09, 2024 at 04:15:12PM GMT, Eelco Chaudron wrote:
> >>
> >>
> >> On 9 Jul 2024, at 15:30, Adrián Moreno wrote:
> >>
> >>> On Tue, Jul 09, 2024 at 11:45:15AM GMT, Eelco Chaudron wrote:
> >>>> On 4 Jul 2024, at 9:52, Adrian Moreno wrote:
> >>>>
> >>>>> Add support for parsing and formatting the new action.
> >>>>>
> >>>>> Also, flag OVS_ACTION_ATTR_SAMPLE as requiring datapath assistance if it
> >>>>> contains a nested OVS_ACTION_ATTR_PSAMPLE. The reason is that the
> >>>>> sampling rate form the parent "sample" is made available to the nested
> >>>>
> >>>> form -> from
> >>>>
> >>>>> "psample" by the kernel.
> >>>>
> >>>> Two small comments below, the rest looks good.
> >>>>
> >>>>> Signed-off-by: Adrian Moreno 
> >>>>> ---
> >>>>>  include/linux/openvswitch.h  | 28 +++
> >>>>>  lib/dpif-netdev.c|  1 +
> >>>>>  lib/dpif.c   |  1 +
> >>>>>  lib/odp-execute.c| 25 +-
> >>>>>  lib/odp-util.c   | 93 
> >>>>>  lib/odp-util.h   |  3 ++
> >>>>>  ofproto/ofproto-dpif-ipfix.c |  1 +
> >>>>>  ofproto/ofproto-dpif-sflow.c |  1 +
> >>>>>  python/ovs/flow/odp.py   |  8 
> >>>>>  tests/odp.at | 16 +++
> >>>>>  10 files changed, 176 insertions(+), 1 deletion(-)
> >>>>>
> >>>>> diff --git a/include/linux/openvswitch.h b/include/linux/openvswitch.h
> >>>>> index d9fb991ef..0023b65fb 100644
> >>>>> --- a/include/linux/openvswitch.h
> >>>>> +++ b/include/linux/openvswitch.h
> >>>>> @@ -992,6 +992,31 @@ struct check_pkt_len_arg {
> >>>>>  };
> >>>>>  #endif
> >>>>>
> >>>>> +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
> >>>>> +/**
> >>>>> + * enum ovs_pample_attr - Attributes for %OVS_ACTION_ATTR_PSAMPLE
> >>>>> + * action.
> >>>>> + *
> >>>>> + * @OVS_PSAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> >>>>> + * sample.
> >>>>> + * @OVS_PSAMPLE_ATTR_COOKIE: An optional variable-length binary cookie 
> >>>>> that
> >>>>> + * contains user-defined metadata. The maximum length is
> >>>>> + * OVS_PSAMPLE_COOKIE_MAX_SIZE bytes.
> >>>>> + *
> >>>>> + * Sends the packet to the psample multicast group with the specified 
> >>>>> group and
> >>>>> + * cookie. It is possible to combine this action with the
> >>>>> + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the sample.
> >>>>> + */
> >>>>> +enum ovs_psample_attr {
> >>>>> +OVS_PSAMPLE_ATTR_GROUP = 1,/* u32 number. */
> >>>>> +OVS_PSAMPLE_ATTR_COOKIE,   /* Optional, user specified 
> >>>>> cookie. */
> >>>>> +
> >>>>> +/* private: */
> >>>>
> >>>> None of the other definitions have this private marking, so I see no 
> >>>> need to
> >>>> start adding it here.
> >>>
> >>> OK. The uAPI file has it (requested by netdev maintainers) but I guess
> >>> it's kernel-specific and this file is not a blind copy of the uAPI
> >>> anyways so I think we can remove it. I'll do on the next version.
>
> I'd keep it.  There is no point in having this header different from
> the kernel one in places where it doesn't surve any purpose.
>
> >>>
> >>>>
> >>>>> +__OVS_PSAMPLE_ATTR_MAX
> >>>>> +};
> >>>>> +
> >>>>> +#define OVS_PSAMPLE_ATTR_MAX (__OVS_PSAMPLE_ATTR_MAX - 1)
> >>>>> +
> >>>>>  /**
> >>>>>   * enum ovs_action_attr - Action types.
> >>>>>   *
> >>>>> @@ -1056,6 +1081,8 @@ struct check_pkt_len_arg {
> >>>>>   * of l3 tunnel flag in t

Re: [ovs-dev] [PATCH v1 01/13] ofproto-dpif: Allow forcing dp features.

[Adrián Moreno]

On Wed, Jul 10, 2024 at 12:13:37AM GMT, Ilya Maximets wrote:
> On 7/7/24 22:08, Adrian Moreno wrote:
> > Datapath features can be set with dpif/set-dp-features unixctl command.
> > This command is not docummented and therefore not supported in
> > production but only useful for unit tests.
> >
> > A limitation was put in place originally to avoid enabling features at
> > runtime that were disabled at boot time to avoid breaking the datapath
> > in unexpected ways.
> >
> > But, considering users should not use this command and it should only be
> > used for testing, we can assume whoever runs it knows what they are
> > doing. Therefore, the limitation should be bypass-able.
> >
> > This patch adds a "--force" flag to the unixctl command to allow
> > bypassing the mentioned limitation.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif.c | 37 +++--
> >  1 file changed, 27 insertions(+), 10 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> > index fcd7cd753..4712d10a8 100644
> > --- a/ofproto/ofproto-dpif.c
> > +++ b/ofproto/ofproto-dpif.c
> > @@ -6432,7 +6432,8 @@ display_support_field(const char *name,
> >  static bool
> >  dpif_set_support(struct dpif_backer_support *rt_support,
> >   struct dpif_backer_support *bt_support,
> > - const char *name, const char *value, struct ds *ds)
> > + const char *name, const char *value, bool force,
> > + struct ds *ds)
> >  {
> >  struct shash all_fields = SHASH_INITIALIZER(&all_fields);
> >  struct dpif_support_field *field;
> > @@ -6484,8 +6485,8 @@ dpif_set_support(struct dpif_backer_support 
> > *rt_support,
> >
> >  if (field->type == DPIF_SUPPORT_FIELD_bool) {
> >  if (!strcasecmp(value, "true")) {
> > -if (*(bool *)field->bt_ptr) {
> > -*(bool *)field->rt_ptr = true;
> > +if (*(bool *) field->bt_ptr || force) {
> > +*(bool *) field->rt_ptr = true;
>
> We discussed having a scary warning message for the force option.
> I don't see it here.
>

You're right. I forgot. Thanks.

> >  changed = true;
> >  } else {
> >  ds_put_cstr(ds, "Can not enable features not supported by 
> > the datapth");
> > @@ -6720,21 +6721,36 @@ ofproto_unixctl_dpif_set_dp_features(struct 
> > unixctl_conn *conn,
> >   void *aux OVS_UNUSED)
> >  {
> >  struct ds ds = DS_EMPTY_INITIALIZER;
> > -const char *br = argv[1];
> > +struct ofproto_dpif *ofproto;
> > +bool changed, force = false;
> >  const char *name, *value;
> > -struct ofproto_dpif *ofproto = ofproto_dpif_lookup_by_name(br);
> > -bool changed;
> > +const char *br = NULL;
> > +
> > +if (!strcmp(argv[1], "--force")) {
> > +force = true;
> > +if (argc > 2) {
> > +br = argv[2];
> > +name = argc > 3 ? argv[3] : NULL;
> > +value = argc > 4 ? argv[4] : NULL;
> > +} else {
> > +unixctl_command_reply_error(conn, "bridge not specified");
> > +return;
> > +}
> > +} else {
> > +br = argv[1];
> > +name = argc > 2 ? argv[2] : NULL;
> > +value = argc > 3 ? argv[3] : NULL;
> > +}
>
> It feels like this can be cleaner if we just eat the --force,
> argv++, argc-- and continue the processing instead of doing
> the same thing with diferent indexes in different branches.
>

Thanks for the suggestion. It does seem to yield cleaner code.

> >
> > +ofproto = ofproto_dpif_lookup_by_name(br);
> >  if (!ofproto) {
> >  unixctl_command_reply_error(conn, "no such bridge");
> >  return;
> >  }
> >
> > -name = argc > 2 ? argv[2] : NULL;
> > -value = argc > 3 ? argv[3] : NULL;
> >  changed = dpif_set_support(&ofproto->backer->rt_support,
> > &ofproto->backer->bt_support,
> > -   name, value, &ds);
> > +   name, value, force, &ds);
> >  if (changed) {
> >  xlate_set_support(ofproto, &ofproto->backer->rt_support);
> >  udpif_flush(ofproto->backer->udpif);
> > @@ -6777,7 +6793,8 @@ ofproto_unixctl_init(void)
> >  unixctl_command_register("dpif/dump-flows",
> >   "[-m] [--names | --no-names] bridge", 1, 
> > INT_MAX,
> >   ofproto_unixctl_dpif_dump_flows, NULL);
> > -unixctl_command_register("dpif/set-dp-features", "bridge", 1, 3 ,
> > +unixctl_command_register("dpif/set-dp-features",
> > + "[--force] bridge [feature] [value]", 1, 4 ,
>
> There is an extra space after '4'.  It was there before, but it's
> a good time to remove it.
>
> Also, [feature [value]] .

Ack.

>
> >   ofproto_unixctl_dpif_set_dp_features, NULL);
> >  }
> >  
>

__

Re: [ovs-dev] [PATCH v1 12/13] ofproto: xlate: Make bridge-sampled drops explicit.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:46:09AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
>
> > The decision to add or not the explicit drop action is currently based
> > on whether the resulting dp action list is empty or not.
> >
> > This is OK for most of the cases but if per-flow IPFIX/sFlow sampling
> > is enabled on the bridge, it doesn't work as expected.
> >
> > Fix this by taking into account the size of these sampling actions.
>
> Same comments as on the previous patch.
>
> > Fixes: a13a0209750c ("userspace: Improved packet drop statistics.")
> > Cc: anju.tho...@ericsson.com
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-xlate.c |  5 ++--
> >  tests/drop-stats.at  | 44 
> >  tests/ofproto-dpif.at|  4 ++--
> >  3 files changed, 49 insertions(+), 4 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 094fe5d72..323a58cbf 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -8153,6 +8153,7 @@ xlate_actions(struct xlate_in *xin, struct xlate_out 
> > *xout)
> >  uint64_t action_set_stub[1024 / 8];
> >  uint64_t frozen_actions_stub[1024 / 8];
> >  uint64_t actions_stub[256 / 8];
> > +size_t sample_actions_len = 0;
> >  struct ofpbuf scratch_actions = OFPBUF_STUB_INITIALIZER(actions_stub);
> >  struct xlate_ctx ctx = {
> >  .xin = xin,
> > @@ -8412,7 +8413,7 @@ xlate_actions(struct xlate_in *xin, struct xlate_out 
> > *xout)
> >  user_cookie_offset = compose_sflow_action(&ctx);
> >  compose_ipfix_action(&ctx, ODPP_NONE);
> >  }
> > -size_t sample_actions_len = ctx.odp_actions->size;
> > +sample_actions_len = ctx.odp_actions->size;
> >  bool ecn_drop = !tnl_process_ecn(flow);
> >
> >  if (!ecn_drop
> > @@ -8575,7 +8576,7 @@ exit:
> >  }
> >
> >  /* Install drop action if datapath supports explicit drop action. */
> > -if (xin->odp_actions && !xin->odp_actions->size &&
> > +if (xin->odp_actions && xin->odp_actions->size == sample_actions_len &&
> >  ovs_explicit_drop_action_supported(ctx.xbridge->ofproto)) {
> >  put_drop_action(xin->odp_actions, ctx.error);
>
> See other patch, do we need ctx.error here?

Here we *do* need it.

This is where it was originally placed to flag xlate errors. We are at
the end of xlate_actions main function and the one place where we have
gathered ctx.error.

The change here is not to modify how the drop is added to the action
list. What this change does is change the condition upon which it's
added. Before we were checking if the action list was empty under the
assumption that if do_xlate_actions does not fill in any action, it's a
drop.

However, if per-bridge IPFIX (or sFlow) is enabled, their corresponding
actions are added _before_ calling do_xlate_actions. That's why this
patch only makes the check compare the size of the actions with the one
it had before calling do_xlate_actions.

Calling do_xlate_actions can yield errors and we should definitely flag
them.

>
> >  }
> > diff --git a/tests/drop-stats.at b/tests/drop-stats.at
> > index 44c5cc35b..31782ac52 100644
> > --- a/tests/drop-stats.at
> > +++ b/tests/drop-stats.at
> > @@ -192,6 +192,50 @@ ovs-appctl coverage/read-counter 
> > drop_action_too_many_mpls_labels
> >  OVS_VSWITCHD_STOP(["/|WARN|/d"])
> >  AT_CLEANUP
> >
> > +AT_SETUP([drop-stats - sampling])
>
> Should this maybe be 'bridge sampling'?
>

Ack.

> > +
> > +OVS_VSWITCHD_START([dnl
> > +set bridge br0 datapath_type=dummy \
> > +protocols=OpenFlow10,OpenFlow13,OpenFlow14,OpenFlow15 -- \
> > +add-port br0 p1 -- set Interface p1 type=dummy ofport_request=1])
> > +
> > +AT_DATA([flows.txt], [dnl
> > +table=0,in_port=1,actions=drop
> > +])
> > +
> > +AT_CHECK([
> > +ovs-ofctl del-flows br0
> > +ovs-ofctl -Oopenflow13 add-flows br0 flows.txt
> > +ovs-ofctl -Oopenflow13 dump-flows br0 | ofctl_strip | sort | grep 
> > actions ], [0], [dnl
> > + in_port=1 actions=drop
> > +])
> > +
> > +AT_CHECK([ovs-vsctl -- set bridge br0 ipfix=@fix -- \
> > +--id=@fix create ipfix targets=\"127.0.0.1:4739\" \
> > +  sampling=1],
> > + [0], [ignore])
> > +
> > +AT_CHECK([
> > +ovs-appctl netdev-dummy/receive p1 
> > 'in_port(1),packet_type(ns=0,id=0),eth(src=3a:6d:d2:09:9c:ab,dst=1e:2c:e9:2a:66:9e),ipv4(src=192.168.10.10,dst=192.168.10.30,proto=1,tos=0,ttl=64,frag=no),icmp(type=8,code=0)'
> > +ovs-appctl netdev-dummy/receive p1 
> > 'in_port(1),packet_type(ns=0,id=0),eth(src=3a:6d:d2:09:9c:ab,dst=1e:2c:e9:2a:66:9e),ipv4(src=192.168.10.10,dst=192.168.10.30,proto=1,tos=0,ttl=64,frag=no),icmp(type=8,code=0)'
> > +ovs-appctl netdev-dummy/receive p1 
> > 'in_port(1),packet_type(ns=0,id=0),eth(src=3a:6d:d2:09:9c:ab,dst=1e:2c:e9:2a:66:9e),ipv4(src=192.168.10.10,dst=192.168.10.30,

Re: [ovs-dev] [PATCH v1 02/13] odp-util: Add support OVS_ACTION_ATTR_PSAMPLE.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 04:15:12PM GMT, Eelco Chaudron wrote:
>
>
> On 9 Jul 2024, at 15:30, Adrián Moreno wrote:
>
> > On Tue, Jul 09, 2024 at 11:45:15AM GMT, Eelco Chaudron wrote:
> >> On 4 Jul 2024, at 9:52, Adrian Moreno wrote:
> >>
> >>> Add support for parsing and formatting the new action.
> >>>
> >>> Also, flag OVS_ACTION_ATTR_SAMPLE as requiring datapath assistance if it
> >>> contains a nested OVS_ACTION_ATTR_PSAMPLE. The reason is that the
> >>> sampling rate form the parent "sample" is made available to the nested
> >>
> >> form -> from
> >>
> >>> "psample" by the kernel.
> >>
> >> Two small comments below, the rest looks good.
> >>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  include/linux/openvswitch.h  | 28 +++
> >>>  lib/dpif-netdev.c|  1 +
> >>>  lib/dpif.c   |  1 +
> >>>  lib/odp-execute.c| 25 +-
> >>>  lib/odp-util.c   | 93 
> >>>  lib/odp-util.h   |  3 ++
> >>>  ofproto/ofproto-dpif-ipfix.c |  1 +
> >>>  ofproto/ofproto-dpif-sflow.c |  1 +
> >>>  python/ovs/flow/odp.py   |  8 
> >>>  tests/odp.at | 16 +++
> >>>  10 files changed, 176 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/include/linux/openvswitch.h b/include/linux/openvswitch.h
> >>> index d9fb991ef..0023b65fb 100644
> >>> --- a/include/linux/openvswitch.h
> >>> +++ b/include/linux/openvswitch.h
> >>> @@ -992,6 +992,31 @@ struct check_pkt_len_arg {
> >>>  };
> >>>  #endif
> >>>
> >>> +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
> >>> +/**
> >>> + * enum ovs_pample_attr - Attributes for %OVS_ACTION_ATTR_PSAMPLE
> >>> + * action.
> >>> + *
> >>> + * @OVS_PSAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> >>> + * sample.
> >>> + * @OVS_PSAMPLE_ATTR_COOKIE: An optional variable-length binary cookie 
> >>> that
> >>> + * contains user-defined metadata. The maximum length is
> >>> + * OVS_PSAMPLE_COOKIE_MAX_SIZE bytes.
> >>> + *
> >>> + * Sends the packet to the psample multicast group with the specified 
> >>> group and
> >>> + * cookie. It is possible to combine this action with the
> >>> + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the sample.
> >>> + */
> >>> +enum ovs_psample_attr {
> >>> +OVS_PSAMPLE_ATTR_GROUP = 1,/* u32 number. */
> >>> +OVS_PSAMPLE_ATTR_COOKIE,   /* Optional, user specified 
> >>> cookie. */
> >>> +
> >>> +/* private: */
> >>
> >> None of the other definitions have this private marking, so I see no need 
> >> to
> >> start adding it here.
> >
> > OK. The uAPI file has it (requested by netdev maintainers) but I guess
> > it's kernel-specific and this file is not a blind copy of the uAPI
> > anyways so I think we can remove it. I'll do on the next version.
> >
> >>
> >>> +__OVS_PSAMPLE_ATTR_MAX
> >>> +};
> >>> +
> >>> +#define OVS_PSAMPLE_ATTR_MAX (__OVS_PSAMPLE_ATTR_MAX - 1)
> >>> +
> >>>  /**
> >>>   * enum ovs_action_attr - Action types.
> >>>   *
> >>> @@ -1056,6 +1081,8 @@ struct check_pkt_len_arg {
> >>>   * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> >>>   * argument.
> >>>   * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> >>> + * @OVS_ACTION_ATTR_PSAMPLE: Send a sample of the packet to external 
> >>> observers
> >>> + * via psample.
> >>>   */
> >>>
> >>>  enum ovs_action_attr {
> >>> @@ -1087,6 +1114,7 @@ enum ovs_action_attr {
> >>>   OVS_ACTION_ATTR_ADD_MPLS, /* struct ovs_action_add_mpls. */
> >>>   OVS_ACTION_ATTR_DEC_TTL,  /* Nested OVS_DEC_TTL_ATTR_*. */
> >>>   OVS_ACTION_ATTR_DROP, /* u32 xlate_error. */
> >>> + OVS_ACTION_ATTR_PSAMPLE,  /* Nested OVS_PSAMPLE_ATTR_*. */
> >>>
> >>>  #ifndef __KERNEL__
> >>>   OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> >>> diff --git a/lib/dpif-netdev.c b/lib/dpi

Re: [ovs-dev] [PATCH v1 11/13] ofproto: xlate: Make flow-sampled drops explicit.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 04:22:04PM GMT, Eelco Chaudron wrote:
>
>
> On 9 Jul 2024, at 16:00, Adrián Moreno wrote:
>
> > On Tue, Jul 09, 2024 at 11:46:06AM GMT, Eelco Chaudron wrote:
> >> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
> >>
> >>> When an action set ends in a an OFP_SAMPLE action, it means the packet
> >>> will be dropped and sampled. Make the drop explicit in this case so that
> >>> drop statistics remain accurate.
> >>>
> >>> This could be done outside of the sample action, i.e: "sample(...),drop"
> >>> but datapaths optimize sample actions that are found in the last
> >>> position. So, taking into account that datapaths already report when the
> >>> last sample probability fails, it is safe to put the drop inside the
> >>> sample, i.e: "sample(...,drop)".
> >>
> >> Some style comments below, as this was discussed with Ilya, I let him 
> >> comment
> >> on the feature side.
> >>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  ofproto/ofproto-dpif-xlate.c | 16 +++--
> >>>  tests/drop-stats.at  | 65 
> >>>  tests/ofproto-dpif.at| 44 
> >>>  3 files changed, 123 insertions(+), 2 deletions(-)
> >>>
> >>> diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> >>> index b9546dc5b..094fe5d72 100644
> >>> --- a/ofproto/ofproto-dpif-xlate.c
> >>> +++ b/ofproto/ofproto-dpif-xlate.c
> >>> @@ -715,6 +715,7 @@ static void xlate_xbundle_copy(struct xbridge *, 
> >>> struct xbundle *);
> >>>  static void xlate_xport_copy(struct xbridge *, struct xbundle *,
> >>>   struct xport *);
> >>>  static void xlate_xcfg_free(struct xlate_cfg *);
> >>> +static void put_drop_action(struct ofpbuf *, enum xlate_error);
> >>>  
> >>>  /* Tracing helpers. */
> >>>
> >>> @@ -3392,6 +3393,8 @@ struct sample_userspace_args {
> >>>  struct compose_sample_args {
> >>>  uint32_t probability; /* Number of packets out of
> >>> * UINT32_MAX to sample. */
> >>> +bool last;/* If it's the last action 
> >>> and a
> >>> +   * drop action must be 
> >>> added. */
> >>
> >> Should describing this means this is the last action not be enough?
> >> Maybe also rename it to be last_action, so it's more clear what this means.
> >>
> >
> > Sure "last_action" is pretty much self-explanatory.
> >
> >>>  struct sample_userspace_args *userspace;  /* Optional,
> >>> * arguments for 
> >>> userspace. */
> >>>  struct sample_psample_args *psample;  /* Optional,
> >>> @@ -3456,6 +3459,11 @@ compose_sample_action(struct xlate_ctx *ctx,
> >>>  ovs_assert(res == 0);
> >>>  }
> >>>
> >>> +if (args->last &&
> >>> +ovs_explicit_drop_action_supported(ctx->xbridge->ofproto)) {
> >>> +put_drop_action(ctx->odp_actions, ctx->error);
> >>
> >> Any reason why we use ctx->error here? Should we just not use XLATE_OK?
> >>
> >
> > I was trying to thing in a case where ctx->error would not be XLATE_OK
> > but I couldn't. Nevertheless I thought if there is such case, we should
> > definitely report the right error code, right?
>
> Well, in case of an error all actions are deleted, and only an explicit drop 
> (or no) actions are added.
>
> For the sample() action, the drop reason should always be ok, or else the 
> action should not be there. So I would prefer to make this clear by using 
> XLATE_OK.

OK. I'll change that.

>
> >>> +}
> >>> +
> >>>  if (is_sample) {
> >>>  nl_msg_end_nested(ctx->odp_actions, actions_offset);
> >>>  nl_msg_end_nested(ctx->odp_actions, sample_offset);
> >>> @@ -3490,6 +3498,7 @@ compose_sflow_action(struct xlate_ctx *ctx)
> >>>
> >>>  args.probability = dpif_sflow_get_probability(sflow);
> >>>  args.userspace = &userspace;
> >>>

Re: [ovs-dev] [PATCH v1 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:46:12AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
>
> > When sample action gets used as a way of sampling traffic with
> > controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> > the controller will have to increase the number of flows to ensure each
> > part of the pipeline contains the right metadata.
> >
> > As an example, if the controller decides to sample stateful traffic, it
> > could store the computed metadata for each connection in the conntrack
> > label. However, for established connections, a flow must be created for
> > each different ct_label value with a sample action that contains a
> > different hardcoded obs_domain and obs_point id.
> >
> > This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> > that supports specifying the observation point and domain using an
> > OpenFlow field reference, so now the controller can express:
> >
> >  sample(...
> > obs_domain_id=NXM_NX_CT_LABEL[0..31],
> > obs_point_id=NXM_NX_CT_LABEL[32..63]
> > ...
> >)
> >
> > Signed-off-by: Adrian Moreno 
>
> See some comments inline. I’m missing the documentation update.
>

Yes. I noticed I missed both the NEWS and documentation update.

> Cheers,
>
> Eelco
>
> > ---
> >  include/openvswitch/ofp-actions.h |   8 +-
> >  lib/ofp-actions.c | 249 +++---
> >  ofproto/ofproto-dpif-xlate.c  |  55 ++-
> >  python/ovs/flow/ofp.py|   8 +-
> >  python/ovs/flow/ofp_act.py|   4 +-
> >  tests/ofp-actions.at  |   5 +
> >  tests/ofproto-dpif.at |  41 +
> >  tests/system-traffic.at   |  74 +
> >  8 files changed, 405 insertions(+), 39 deletions(-)
> >
> > diff --git a/include/openvswitch/ofp-actions.h 
> > b/include/openvswitch/ofp-actions.h
> > index 7b57e49ad..56dc2c147 100644
> > --- a/include/openvswitch/ofp-actions.h
> > +++ b/include/openvswitch/ofp-actions.h
> > @@ -1015,14 +1015,16 @@ enum nx_action_sample_direction {
> >
> >  /* OFPACT_SAMPLE.
> >   *
> > - * Used for NXAST_SAMPLE, NXAST_SAMPLE2, and NXAST_SAMPLE3. */
> > + * Used for NXAST_SAMPLE, NXAST_SAMPLE2, NXAST_SAMPLE3 and NXAST_SAMPLE4. 
> > */
> >  struct ofpact_sample {
> >  OFPACT_PADDED_MEMBERS(
> >  struct ofpact ofpact;
> >  uint16_t probability;   /* Always positive. */
> >  uint32_t collector_set_id;
> > -uint32_t obs_domain_id;
> > -uint32_t obs_point_id;
> > +uint32_t obs_domain_imm;
> > +struct mf_subfield obs_domain_src;
> > +uint32_t obs_point_imm;
> > +struct mf_subfield obs_point_src;
> >  ofp_port_t sampling_port;
> >  enum nx_action_sample_direction direction;
> >  );
> > diff --git a/lib/ofp-actions.c b/lib/ofp-actions.c
> > index da7b1dd31..e329a7e3f 100644
> > --- a/lib/ofp-actions.c
> > +++ b/lib/ofp-actions.c
> > @@ -330,6 +330,8 @@ enum ofp_raw_action_type {
> >  NXAST_RAW_SAMPLE2,
> >  /* NX1.0+(41): struct nx_action_sample2. */
> >  NXAST_RAW_SAMPLE3,
> > +/* NX1.0+(51): struct nx_action_sample4, ... VLMFF */
> > +NXAST_RAW_SAMPLE4,
> >
> >  /* NX1.0+(34): struct nx_action_conjunction. */
> >  NXAST_RAW_CONJUNCTION,
> > @@ -6188,6 +6190,34 @@ struct nx_action_sample2 {
> >   };
> >   OFP_ASSERT(sizeof(struct nx_action_sample2) == 32);
> >
> > +/* Action structure for NXAST_SAMPLE4
> > + *
> > + * NXAST_SAMPLE4 was added in Open vSwitch 3.4.0.  Compared to 
> > NXAST_SAMPLE3,
> > + * it adds support for using field specifiers for observation_domain_id and
> > + * observation_point_id. */
> > +struct nx_action_sample4 {
> > +ovs_be16 type;  /* OFPAT_VENDOR. */
> > +ovs_be16 len;   /* Length is 32. */
> > +ovs_be32 vendor;/* NX_VENDOR_ID. */
> > +ovs_be16 subtype;   /* NXAST_SAMPLE. */
> > +ovs_be16 probability;   /* Fraction of packets to sample. */
> > +ovs_be32 collector_set_id;  /* ID of collector set in OVSDB. */
> > +ovs_be32 obs_domain_src;/* Source of the 
> > observation_domain_id. */
> > +union {
> > +ovs_be16 obs_domain_ofs_nbits;  /* Range to use from source field. 
> > */
> > +ovs_be32 obs_domain_imm;/* Immediate value for domain id. 
> > */
> > +};
> > +ovs_be32 obs_point_src; /* Source of the observation_point_id 
> > */
> > +union {
> > +ovs_be16 obs_point_ofs_nbits;  /* Range to use from source field. 
> > */
> > +ovs_be32 obs_point_imm;/* Immediate value for point id. */
> > +};
> > +ovs_be16 sampling_port; /* Sampling port. */
> > +uint8_t  direction; /* Sampling direction. */
> > +uint8_t  zeros[5];  /* Pad to a multiple of 8 bytes */
> > + };
>
> Maybe align all comments, and make sure all end with a dot.
>
> struct nx_action_sample4 {
>   

Re: [ovs-dev] [PATCH v1 11/13] ofproto: xlate: Make flow-sampled drops explicit.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:46:06AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
>
> > When an action set ends in a an OFP_SAMPLE action, it means the packet
> > will be dropped and sampled. Make the drop explicit in this case so that
> > drop statistics remain accurate.
> >
> > This could be done outside of the sample action, i.e: "sample(...),drop"
> > but datapaths optimize sample actions that are found in the last
> > position. So, taking into account that datapaths already report when the
> > last sample probability fails, it is safe to put the drop inside the
> > sample, i.e: "sample(...,drop)".
>
> Some style comments below, as this was discussed with Ilya, I let him comment
> on the feature side.
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-xlate.c | 16 +++--
> >  tests/drop-stats.at  | 65 
> >  tests/ofproto-dpif.at| 44 
> >  3 files changed, 123 insertions(+), 2 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index b9546dc5b..094fe5d72 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -715,6 +715,7 @@ static void xlate_xbundle_copy(struct xbridge *, struct 
> > xbundle *);
> >  static void xlate_xport_copy(struct xbridge *, struct xbundle *,
> >   struct xport *);
> >  static void xlate_xcfg_free(struct xlate_cfg *);
> > +static void put_drop_action(struct ofpbuf *, enum xlate_error);
> >  
> >  /* Tracing helpers. */
> >
> > @@ -3392,6 +3393,8 @@ struct sample_userspace_args {
> >  struct compose_sample_args {
> >  uint32_t probability; /* Number of packets out of
> > * UINT32_MAX to sample. */
> > +bool last;/* If it's the last action 
> > and a
> > +   * drop action must be 
> > added. */
>
> Should describing this means this is the last action not be enough?
> Maybe also rename it to be last_action, so it's more clear what this means.
>

Sure "last_action" is pretty much self-explanatory.

> >  struct sample_userspace_args *userspace;  /* Optional,
> > * arguments for userspace. 
> > */
> >  struct sample_psample_args *psample;  /* Optional,
> > @@ -3456,6 +3459,11 @@ compose_sample_action(struct xlate_ctx *ctx,
> >  ovs_assert(res == 0);
> >  }
> >
> > +if (args->last &&
> > +ovs_explicit_drop_action_supported(ctx->xbridge->ofproto)) {
> > +put_drop_action(ctx->odp_actions, ctx->error);
>
> Any reason why we use ctx->error here? Should we just not use XLATE_OK?
>

I was trying to thing in a case where ctx->error would not be XLATE_OK
but I couldn't. Nevertheless I thought if there is such case, we should
definitely report the right error code, right?

> > +}
> > +
> >  if (is_sample) {
> >  nl_msg_end_nested(ctx->odp_actions, actions_offset);
> >  nl_msg_end_nested(ctx->odp_actions, sample_offset);
> > @@ -3490,6 +3498,7 @@ compose_sflow_action(struct xlate_ctx *ctx)
> >
> >  args.probability = dpif_sflow_get_probability(sflow);
> >  args.userspace = &userspace;
> > +args.last = false;
> >
> >  return compose_sample_action(ctx, &args);
> >  }
> > @@ -3542,6 +3551,7 @@ compose_ipfix_action(struct xlate_ctx *ctx, 
> > odp_port_t output_odp_port)
> >
> >  args.probability = dpif_ipfix_get_bridge_exporter_probability(ipfix);
> >  args.userspace = &userspace;
> > +args.last = false;
> >
> >  compose_sample_action(ctx, &args);
> >  }
> > @@ -5974,7 +5984,8 @@ xlate_fill_ipfix_sample(struct xlate_ctx *ctx,
> >
> >  static void
> >  xlate_sample_action(struct xlate_ctx *ctx,
> > -const struct ofpact_sample *os)
> > +const struct ofpact_sample *os,
> > +bool last)
> >  {
> >  uint8_t cookie_buf[sizeof(os->obs_domain_id) + 
> > sizeof(os->obs_point_id)];
> >  struct dpif_lsample *lsample = ctx->xbridge->lsample;
> > @@ -5991,6 +6002,7 @@ xlate_sample_action(struct xlate_ctx *ctx,
> >   * the same percentage. */
> >  compose_args.probability =
> >  ((uint32_t) os->probability << 16) | os->probability;
> > +compose_args.last = last;
> >
> >  if (ipfix) {
> >  xlate_fill_ipfix_sample(ctx, os, ipfix, &userspace);
> > @@ -7726,7 +7738,7 @@ do_xlate_actions(const struct ofpact *ofpacts, size_t 
> > ofpacts_len,
> >  break;
> >
> >  case OFPACT_SAMPLE:
> > -xlate_sample_action(ctx, ofpact_get_SAMPLE(a));
> > +xlate_sample_action(ctx, ofpact_get_SAMPLE(a), last);
> >  break;
> >
> >  case OFPACT_CLONE:
> > diff --git a/tests/drop-stats.at b/tests/drop-stats.at
> > index 1d3af98da..44c5cc35b 100644
> > --

Re: [ovs-dev] [PATCH v1 10/13] tests: Test local sampling.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:46:03AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
>
> > Test simultaneous IPFIX and local sampling including slow-path.
>
> See comments below.
>
> Cheers,
>
> Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  tests/system-common-macros.at |   4 +
> >  tests/system-traffic.at   | 285 ++
> >  2 files changed, 289 insertions(+)
> >
> > diff --git a/tests/system-common-macros.at b/tests/system-common-macros.at
> > index 2a68cd664..860d6a8c9 100644
> > --- a/tests/system-common-macros.at
> > +++ b/tests/system-common-macros.at
> > @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >  # OVS_CHECK_DROP_ACTION()
> >  m4_define([OVS_CHECK_DROP_ACTION],
> >  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> > ovs-vswitchd.log])])
> > +
> > +# OVS_CHECK_PSAMPLE()
> > +m4_define([OVS_CHECK_PSAMPLE],
> > +[AT_SKIP_IF([! grep -q "Datapath supports psample" ovs-vswitchd.log])])
> > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > index 3f1a15445..ddab4ece3 100644
> > --- a/tests/system-traffic.at
> > +++ b/tests/system-traffic.at
> > @@ -9103,3 +9103,288 @@ OVS_WAIT_UNTIL([ovs-pcap p2.pcap | grep -q 
> > "m4_join([], [^],
> >
> >  OVS_TRAFFIC_VSWITCHD_STOP
> >  AT_CLEANUP
> > +
> > +AT_BANNER([local-sampling])
> > +
> > +m4_define([SAMPLE_ACTION],
> > +
> > [sample(probability=65535,collector_set_id=$1,obs_domain_id=$2,obs_point_id=$3)]dnl
> > +)
> > +
> > +AT_SETUP([psample - sanity check])
>
> Maybe call it '- ipv4' as it's equal to ipv6, or rename the IPv6 one to
> 'sanity check IPv6'

Ack.

>
> > +OVS_TRAFFIC_VSWITCHD_START()
> > +OVS_CHECK_PSAMPLE()
> > +
> > +ADD_NAMESPACES(at_ns0, at_ns1)
> > +
> > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24")
> > +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24")
> > +
> > +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> > +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> > local_group_id=10 \
> > +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> > local_group_id=12],
> > + [0], [ignore])
>
> Maybe wrap to stay <80 chars?
>
> AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> -- create Flow_Sample_Collector_Set id=1 bridge=@br0 \
>local_group_id=10 \
> -- create Flow_Sample_Collector_Set id=2 bridge=@br0 \
>local_group_id=12],
>  [0], [ignore])
>

Ack.

> > +
> > +AT_DATA([flows.txt], [dnl
> > +arp actions=NORMAL
> > +in_port=ovs-p0,ip actions=SAMPLE_ACTION(1, 2853183536, 2856341600),ovs-p1
> > +in_port=ovs-p1,ip actions=SAMPLE_ACTION(2, 3138396208, 3141554272),ovs-p0
> > +])
> > +
> > +AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> > +
> > +OVS_DAEMONIZE([ovstest test-psample > psample.out], [psample.pid])
> > +OVS_WAIT_UNTIL([grep -q "Listening for psample events" psample.out])
> > +
> > +NS_CHECK_EXEC([at_ns0], [ping -q -c 1 10.1.1.2 | FORMAT_PING], [0], [dnl
> > +1 packets transmitted, 1 received, 0% packet loss, time 0ms
> > +])
> > +
> > +AT_CHECK([ovs-appctl dpctl/dump-flows -m --names], [0], [stdout])
> > +AT_CHECK([grep -q 
> > 'actions:psample(group=10,cookie=0xaa102030aa405060),ovs-p1' stdout])
> > +AT_CHECK([grep -q 
> > 'actions:psample(group=12,cookie=0xbb102030bb405060),ovs-p0' stdout])
> > +
> > +m4_define([SAMPLE1], [m4_join([ ],
> > +[group_id=0xa],
> > +[obs_domain=0xaa102030,obs_point=0xaa405060],
> > +[.*icmp.*nw_src=10.1.1.1,nw_dst=10.1.1.2])])
> > +
> > +m4_define([SAMPLE2], [m4_join([ ],
> > +[group_id=0xc],
> > +[obs_domain=0xbb102030,obs_point=0xbb405060],
> > +[.*icmp.*nw_src=10.1.1.2,nw_dst=10.1.1.1])])
> > +
> > +OVS_WAIT_UNTIL([grep -qE 'SAMPLE1' psample.out])
> > +OVS_WAIT_UNTIL([grep -qE 'SAMPLE2' psample.out])
> > +
> > +AT_CHECK([ovs-appctl lsample/show br0], [0], [dnl
> > +Local sample statistics for bridge "br0":
> > +Collector Set ID: 1:
> > +  Group ID : 10
> > +  Total packets: 1
> > +  Total bytes  : 98
> > +
> > +Collector Set ID: 2:
> > +  Group ID : 12
> > +  Total packets: 1
> > +  Total bytes  : 98
> > +])
> > +
>
> We miss OVS_TRAFFIC_VSWITCHD_STOP() here.
>

Yes!

> > +AT_CLEANUP
> > +
> > +AT_SETUP([psample - ipv6])
>
> See comment above, if you decide to keep it ipv6, please change it to IPv6.
>
> > +OVS_TRAFFIC_VSWITCHD_START()
> > +OVS_CHECK_PSAMPLE()
> > +
> > +ADD_NAMESPACES(at_ns0, at_ns1)
> > +
> > +ADD_VETH(p0, at_ns0, br0, "fc00::1/96")
> > +ADD_VETH(p1, at_ns1, br0, "fc00::2/96")
> > +
> > +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> > +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> > local_group_id=10 \
> > +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> > local_group_id=12],
> > + [0], [ignore])
> > +
> > +AT_DATA([flows.txt], [dnl
> > +priority=100,in_port=ovs-p0,ip6,icmp6,icmpv6_type=128 
> > actions=SAMPLE_ACTIO

Re: [ovs-dev] [PATCH v1 08/13] ofproto-dpif-lsample: Show stats via unixctl.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:58AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:09, Adrian Moreno wrote:
>
> > Add a command to dump statistics per exporter.
>
> Small nit below, rest looks good to me. If this is the only change in the
> next series you can add my ACK.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  NEWS   |   2 +
> >  ofproto/ofproto-dpif-lsample.c | 111 +
> >  ofproto/ofproto-dpif-lsample.h |   1 +
> >  ofproto/ofproto-dpif.c |   1 +
> >  4 files changed, 115 insertions(+)
> >
> > diff --git a/NEWS b/NEWS
> > index 15faf9fc3..1c53badea 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -20,6 +20,8 @@ Post-v3.3.0
> >   allows samples to be emitted locally (instead of via IPFIX) in a
> >   datapath-specific manner. The Linux kernel datapath is the first to
> >   support this feature by using the new datapath "psample" action.
> > + A new unixctl command 'lsample/show' shows packet and bytes statistics
> > + per local sample exporter.
> >
> >
> >  v3.3.0 - 16 Feb 2024
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index 171129d5b..82a87c27d 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -21,7 +21,10 @@
> >  #include "dpif.h"
> >  #include "hash.h"
> >  #include "ofproto.h"
> > +#include "ofproto-dpif.h"
> > +#include "openvswitch/dynamic-string.h"
> >  #include "openvswitch/thread.h"
> > +#include "unixctl.h"
> >
> >  /* Dpif local sampling.
> >   *
> > @@ -219,3 +222,111 @@ dpif_lsample_unref(struct dpif_lsample *lsample)
> >  dpif_lsample_destroy(lsample);
> >  }
> >  }
> > +
> > +static int
> > +comp_exporter_collector_id(const void *a_, const void *b_)
> > +{
> > +const struct lsample_exporter_node *a, *b;
> > +
> > +a = *(struct lsample_exporter_node **) a_;
> > +b = *(struct lsample_exporter_node **) b_;
> > +
> > +if (a->exporter.options.collector_set_id >
> > +b->exporter.options.collector_set_id) {
> > +return 1;
> > +}
> > +if (a->exporter.options.collector_set_id <
> > +b->exporter.options.collector_set_id) {
> > +return -1;
> > +}
> > +return 0;
> > +}
> > +
> > +static void
> > +lsample_exporter_list(struct dpif_lsample *lsample,
> > +  struct lsample_exporter_node ***list,
> > +  size_t *num_exporters)
> > +{
> > +struct lsample_exporter_node **exporter_list;
> > +struct lsample_exporter_node *node;
> > +size_t k = 0, n;
> > +
> > +n = cmap_count(&lsample->exporters);
> > +
> > +exporter_list = xcalloc(n, sizeof *exporter_list);
> > +
> > +CMAP_FOR_EACH (node, node, &lsample->exporters) {
> > +if (k >= n) {
> > +break;
> > +}
> > +exporter_list[k++] = node;
> > +}
> > +
> > +qsort(exporter_list, k, sizeof *exporter_list, 
> > comp_exporter_collector_id);
> > +
> > +*list = exporter_list;
> > +*num_exporters = k;
> > +}
> > +
> > +static void
> > +lsample_unixctl_show(struct unixctl_conn *conn, int argc OVS_UNUSED,
> > + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED)
> > +{
> > +struct lsample_exporter_node **node_list = NULL;
> > +struct ds ds = DS_EMPTY_INITIALIZER;
> > +const struct ofproto_dpif *ofproto;
> > +size_t i, num;
> > +
> > +ofproto = ofproto_dpif_lookup_by_name(argv[1]);
> > +if (!ofproto) {
> > +unixctl_command_reply_error(conn, "no such bridge");
> > +return;
> > +}
> > +
> > +if (!ofproto->lsample) {
> > +unixctl_command_reply_error(conn,
> > +"no local sampling exporters 
> > configured");
> > +return;
> > +}
> > +
> > +ds_put_format(&ds, "Local sample statistics for bridge \"%s\":\n",
> > +  argv[1]);
> > +
> > +lsample_exporter_list(ofproto->lsample, &node_list, &num);
> > +
> > +for (i = 0; i < num; i++) {
> > +uint64_t n_bytes;
> > +uint64_t n_packets;
> > +
> > +struct lsample_exporter_node *node = node_list[i];
> > +
> > +atomic_read_relaxed(&node->exporter.n_packets, &n_packets);
> > +atomic_read_relaxed(&node->exporter.n_bytes, &n_bytes);
> > +
> > +if (i) {
> > +ds_put_cstr(&ds, "\n");
> > +}
> > +
> > +ds_put_format(&ds, "Collector Set ID: %"PRIu32":\n",
> > +node->exporter.options.collector_set_id);
> > +ds_put_format(&ds, "  Group ID : %"PRIu32"\n",
> > +node->exporter.options.group_id);
> > +ds_put_format(&ds, "  Total packets: %"PRIu64"\n", n_packets);
> > +ds_put_format(&ds, "  Total bytes  : %"PRIu64"\n", n_bytes);
> > +}
> > +
> > +free(node_list);
> > +unixctl_command_reply(conn, ds_cstr(&ds));
> > +ds_destroy(&ds);
> > +}
> > +
> > +void dpif_lsample_init(void)

Re: [ovs-dev] [PATCH v1 07/13] ofproto-dpif-xlate-cache: Add lsample to xcache.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:54AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:08, Adrian Moreno wrote:
>
> > Add a cache entry type for local sample objects.
> > Store both the dpif_lsample reference and the collector_set_id so we can
> > quickly find the particular exporter.
> >
> > Using this mechanism, account for packet and byte statistics.
>
> Small nit below, rest looks good to me. If this is the only change in the
> next series you can add my ACK.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-lsample.c | 18 ++
> >  ofproto/ofproto-dpif-lsample.h |  4 
> >  ofproto/ofproto-dpif-xlate-cache.c | 11 ++-
> >  ofproto/ofproto-dpif-xlate-cache.h |  6 ++
> >  ofproto/ofproto-dpif-xlate.c   | 14 ++
> >  ofproto/ofproto-dpif.c |  1 +
> >  6 files changed, 53 insertions(+), 1 deletion(-)
> >
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index 534ad96f0..171129d5b 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -18,6 +18,7 @@
> >  #include "ofproto-dpif-lsample.h"
> >
> >  #include "cmap.h"
> > +#include "dpif.h"
> >  #include "hash.h"
> >  #include "ofproto.h"
> >  #include "openvswitch/thread.h"
> > @@ -44,6 +45,8 @@ struct dpif_lsample {
> >
> >  struct lsample_exporter {
> >  struct ofproto_lsample_options options;
> > +atomic_uint64_t n_packets;
> > +atomic_uint64_t n_bytes;
> >  };
> >
> >  struct lsample_exporter_node {
> > @@ -156,6 +159,21 @@ dpif_lsample_get_group_id(struct dpif_lsample *ps, 
> > uint32_t collector_set_id,
> >  return found;
> >  }
> >
> > +void
> > +dpif_lsample_credit_stats(struct dpif_lsample *lsample,
> > +  uint32_t collector_set_id,
> > +  const struct dpif_flow_stats *stats)
> > +{
> > +struct lsample_exporter_node *node;
> > +uint64_t orig;
> > +
> > +node = dpif_lsample_find_exporter_node(lsample, collector_set_id);
> > +if (node) {
> > +atomic_add_relaxed(&node->exporter.n_packets, stats->n_packets, 
> > &orig);
> > +atomic_add_relaxed(&node->exporter.n_bytes, stats->n_bytes, &orig);
> > +}
> > +}
> > +
> >  struct dpif_lsample *
> >  dpif_lsample_create(void)
> >  {
> > diff --git a/ofproto/ofproto-dpif-lsample.h b/ofproto/ofproto-dpif-lsample.h
> > index 9c1026551..2666e5478 100644
> > --- a/ofproto/ofproto-dpif-lsample.h
> > +++ b/ofproto/ofproto-dpif-lsample.h
> > @@ -23,6 +23,7 @@
> >
> >  struct dpif_lsample;
> >  struct ofproto_lsample_options;
> > +struct dpif_flow_stats;
> >
> >  struct dpif_lsample *dpif_lsample_create(void);
> >
> > @@ -38,4 +39,7 @@ bool dpif_lsample_get_group_id(struct dpif_lsample *,
> > uint32_t collector_set_id,
> > uint32_t *group_id);
> >
> > +void dpif_lsample_credit_stats(struct dpif_lsample *,
> > +   uint32_t collector_set_id,
> > +   const struct dpif_flow_stats *);
> >  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> > diff --git a/ofproto/ofproto-dpif-xlate-cache.c 
> > b/ofproto/ofproto-dpif-xlate-cache.c
> > index 2e1fcb3a6..73d79bfc7 100644
> > --- a/ofproto/ofproto-dpif-xlate-cache.c
> > +++ b/ofproto/ofproto-dpif-xlate-cache.c
> > @@ -35,9 +35,10 @@
> >  #include "learn.h"
> >  #include "mac-learning.h"
> >  #include "netdev-vport.h"
> > +#include "ofproto/ofproto-dpif.h"
> > +#include "ofproto/ofproto-dpif-lsample.h"
> >  #include "ofproto/ofproto-dpif-mirror.h"
> >  #include "ofproto/ofproto-dpif-xlate.h"
> > -#include "ofproto/ofproto-dpif.h"
> >  #include "ofproto/ofproto-provider.h"
> >  #include "openvswitch/dynamic-string.h"
> >  #include "openvswitch/vlog.h"
> > @@ -162,6 +163,11 @@ xlate_push_stats_entry(struct xc_entry *entry,
> >  }
> >
> >  break;
> > +case XC_LSAMPLE:
> > +dpif_lsample_credit_stats(entry->lsample.lsample,
> > +  entry->lsample.collector_set_id,
> > +  stats);
> > +break;
> >  default:
> >  OVS_NOT_REACHED();
> >  }
> > @@ -245,6 +251,9 @@ xlate_cache_clear_entry(struct xc_entry *entry)
> >  break;
> >  case XC_TUNNEL_HEADER:
> >  break;
> > +case XC_LSAMPLE:
> > +dpif_lsample_unref(entry->lsample.lsample);
> > +break;
> >  default:
> >  OVS_NOT_REACHED();
> >  }
> > diff --git a/ofproto/ofproto-dpif-xlate-cache.h 
> > b/ofproto/ofproto-dpif-xlate-cache.h
> > index 0fc6d2ea6..df8115419 100644
> > --- a/ofproto/ofproto-dpif-xlate-cache.h
> > +++ b/ofproto/ofproto-dpif-xlate-cache.h
> > @@ -29,6 +29,7 @@
> >  struct bfd;
> >  struct bond;
> >  struct dpif_flow_stats;
> > +struct dpif_lsample;
> >  struct flow;
> >  struct group_dpif;
> >  struct mbridge;
> > @@ -53,6 +54,7 @@ enum xc_type {
> >  XC_GROUP,
> >  XC_TNL_NEIGH,
>

Re: [ovs-dev] [PATCH v1 06/13] ofproto-dpif-xlate: Use psample for local sample.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:51AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:08, Adrian Moreno wrote:
>
> > Use the newly added psample action to implement OpenFlow sample() actions
> > with local sampling configuration if possible.
> >
> > A bit of refactoring in compose_sample_actions arguments helps make it a
> > bit more readable.
>
> Some comments below.
>
> Cheers,
>
> Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-lsample.c |  16 +++
> >  ofproto/ofproto-dpif-lsample.h |   5 +
> >  ofproto/ofproto-dpif-xlate.c   | 251 +++--
> >  ofproto/ofproto-dpif-xlate.h   |   5 +-
> >  ofproto/ofproto-dpif.c |   2 +-
> >  tests/ofproto-dpif.at  | 146 +++
> >  6 files changed, 345 insertions(+), 80 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index d675a116f..534ad96f0 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -140,6 +140,22 @@ dpif_lsample_set_options(struct dpif_lsample *lsample,
> >  return changed;
> >  }
> >
> > +/* Returns the group_id for a given collector_set_id, if it exists. */
> > +bool
> > +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> > collector_set_id,
> > +  uint32_t *group_id)
> > +{
> > +struct lsample_exporter_node *node;
> > +bool found = false;
> > +
> > +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> > +if (node) {
> > +found = true;
> > +*group_id = node->exporter.options.group_id;
> > +}
> > +return found;
> > +}
> > +
> >  struct dpif_lsample *
> >  dpif_lsample_create(void)
> >  {
> > diff --git a/ofproto/ofproto-dpif-lsample.h b/ofproto/ofproto-dpif-lsample.h
> > index bee36c9c5..9c1026551 100644
> > --- a/ofproto/ofproto-dpif-lsample.h
> > +++ b/ofproto/ofproto-dpif-lsample.h
> > @@ -18,6 +18,7 @@
> >  #define OFPROTO_DPIF_LSAMPLE_H 1
> >
> >  #include 
> > +#include 
> >  #include 
> >
> >  struct dpif_lsample;
> > @@ -33,4 +34,8 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >const struct ofproto_lsample_options *,
> >size_t n_opts);
> >
> > +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> > +   uint32_t collector_set_id,
> > +   uint32_t *group_id);
> > +
> >  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 7c4950895..5e8113d5e 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -47,6 +47,7 @@
> >  #include "ofproto/ofproto-dpif-ipfix.h"
> >  #include "ofproto/ofproto-dpif-mirror.h"
> >  #include "ofproto/ofproto-dpif-monitor.h"
> > +#include "ofproto/ofproto-dpif-lsample.h"
>
> Add in alphabetical order?

Ack.

>
> >  #include "ofproto/ofproto-dpif-sflow.h"
> >  #include "ofproto/ofproto-dpif-trace.h"
> >  #include "ofproto/ofproto-dpif-xlate-cache.h"
> > @@ -117,6 +118,7 @@ struct xbridge {
> >  struct dpif_sflow *sflow; /* SFlow handle, or null. */
> >  struct dpif_ipfix *ipfix; /* Ipfix handle, or null. */
> >  struct netflow *netflow;  /* Netflow handle, or null. */
> > +struct dpif_lsample *lsample; /* Local sample handle, or null. */
>
> I would move above netflow, as you also do the actual init/unref before 
> netflow.


Ack.


>
> >  struct stp *stp;  /* STP or null if disabled. */
> >  struct rstp *rstp;/* RSTP or null if disabled. */
> >
> > @@ -686,6 +688,7 @@ static void xlate_xbridge_set(struct xbridge *, struct 
> > dpif *,
> >const struct mbridge *,
> >const struct dpif_sflow *,
> >const struct dpif_ipfix *,
> > +  const struct dpif_lsample *,
> >const struct netflow *,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *,
> > @@ -1069,6 +1072,7 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >const struct mbridge *mbridge,
> >const struct dpif_sflow *sflow,
> >const struct dpif_ipfix *ipfix,
> > +  const struct dpif_lsample *lsample,
> >const struct netflow *netflow,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *support,
> > @@ -1099,6 +1103,11 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >  xbridge->ipfix = dpif_ipfix_ref(ipfix);
> >  }
> >
> > +if (xbridge->lsample != lsample) {
> > +dpif_lsample_unref(xbridge->lsample);
> > +xbridge->lsample = dpif_lsample_ref(lsample);
> > +}
> > +
> >  if (xbridge->stp 

Re: [ovs-dev] [PATCH v1 05/13] vswitchd: Add local sampling to vswitchd schema.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:35AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:08, Adrian Moreno wrote:
>
> > Add as new column in the Flow_Sample_Collector_Set table named
> > "local_group_id" which enables this feature.
>
> Small nit below, if fixed add my ACK to the next revision.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  NEWS   |  4 ++
> >  vswitchd/bridge.c  | 78 +++---
> >  vswitchd/vswitch.ovsschema |  9 -
> >  vswitchd/vswitch.xml   | 40 +--
> >  4 files changed, 120 insertions(+), 11 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index e0359b759..15faf9fc3 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -16,6 +16,10 @@ Post-v3.3.0
> > per interface 'options:dpdk-lsc-interrupt' to 'false'.
> > - Python:
> >   * Added custom transaction support to the Idl via add_op().
> > +   - Local sampling is introduced. It reuses the OpenFlow sample action and
> > + allows samples to be emitted locally (instead of via IPFIX) in a
> > + datapath-specific manner. The Linux kernel datapath is the first to
> > + support this feature by using the new datapath "psample" action.
> >
> >
> >  v3.3.0 - 16 Feb 2024
> > diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c
> > index 95a65fcdc..b7db681f3 100644
> > --- a/vswitchd/bridge.c
> > +++ b/vswitchd/bridge.c
> > @@ -288,6 +288,7 @@ static void bridge_configure_mac_table(struct bridge *);
> >  static void bridge_configure_mcast_snooping(struct bridge *);
> >  static void bridge_configure_sflow(struct bridge *, int 
> > *sflow_bridge_number);
> >  static void bridge_configure_ipfix(struct bridge *);
> > +static void bridge_configure_lsample(struct bridge *);
> >  static void bridge_configure_spanning_tree(struct bridge *);
> >  static void bridge_configure_tables(struct bridge *);
> >  static void bridge_configure_dp_desc(struct bridge *);
> > @@ -989,6 +990,7 @@ bridge_reconfigure(const struct ovsrec_open_vswitch 
> > *ovs_cfg)
> >  bridge_configure_netflow(br);
> >  bridge_configure_sflow(br, &sflow_bridge_number);
> >  bridge_configure_ipfix(br);
> > +bridge_configure_lsample(br);
> >  bridge_configure_spanning_tree(br);
> >  bridge_configure_tables(br);
> >  bridge_configure_dp_desc(br);
> > @@ -1537,10 +1539,11 @@ ovsrec_ipfix_is_valid(const struct ovsrec_ipfix 
> > *ipfix)
> >  return ipfix && ipfix->n_targets > 0;
> >  }
> >
> > -/* Returns whether a Flow_Sample_Collector_Set row is valid. */
> > +/* Returns whether a Flow_Sample_Collector_Set row contains a valid IPFIX
> > + * configuration. */
> >  static bool
> > -ovsrec_fscs_is_valid(const struct ovsrec_flow_sample_collector_set *fscs,
> > - const struct bridge *br)
> > +ovsrec_fscs_is_valid_ipfix(const struct ovsrec_flow_sample_collector_set 
> > *fscs,
> > +   const struct bridge *br)
> >  {
> >  return ovsrec_ipfix_is_valid(fscs->ipfix) && fscs->bridge == br->cfg;
> >  }
> > @@ -1558,7 +1561,7 @@ bridge_configure_ipfix(struct bridge *br)
> >  const char *virtual_obs_id;
> >
> >  OVSREC_FLOW_SAMPLE_COLLECTOR_SET_FOR_EACH(fe_cfg, idl) {
> > -if (ovsrec_fscs_is_valid(fe_cfg, br)) {
> > +if (ovsrec_fscs_is_valid_ipfix(fe_cfg, br)) {
> >  n_fe_opts++;
> >  }
> >  }
> > @@ -1621,7 +1624,7 @@ bridge_configure_ipfix(struct bridge *br)
> >  fe_opts = xcalloc(n_fe_opts, sizeof *fe_opts);
> >  opts = fe_opts;
> >  OVSREC_FLOW_SAMPLE_COLLECTOR_SET_FOR_EACH(fe_cfg, idl) {
> > -if (ovsrec_fscs_is_valid(fe_cfg, br)) {
> > +if (ovsrec_fscs_is_valid_ipfix(fe_cfg, br)) {
> >  opts->collector_set_id = fe_cfg->id;
> >  sset_init(&opts->targets);
> >  sset_add_array(&opts->targets, fe_cfg->ipfix->targets,
> > @@ -1667,6 +1670,71 @@ bridge_configure_ipfix(struct bridge *br)
> >  }
> >  }
> >
> > +/* Returns whether a Flow_Sample_Collector_Set row contains a valid local
> > + * sampling configuration. */
> > +static bool
> > +ovsrec_fscs_is_valid_local(const struct ovsrec_flow_sample_collector_set 
> > *fscs,
> > +   const struct bridge *br)
> > +{
> > +return fscs->local_group_id && fscs->n_local_group_id == 1 &&
> > +   fscs->bridge == br->cfg;
> > +}
> > +
> > +/* Set local sample configuration on 'br'. */
> > +static void
> > +bridge_configure_lsample(struct bridge *br)
> > +{
> > +static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> > +const struct ovsrec_flow_sample_collector_set *fscs;
> > +struct ofproto_lsample_options *opts_array, *opts;
> > +size_t n_opts = 0;
> > +int ret;
> > +
> > +/* Iterate the Flow_Sample_Collector_Set table twice.
> > + * First to get the number of valid configuration entries, then to 
> > process
> > + * each of them and build an 

Re: [ovs-dev] [PATCH v1 04/13] ofproto: Add ofproto-dpif-lsample.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:19AM GMT, Eelco Chaudron wrote:
> On 7 Jul 2024, at 22:08, Adrian Moreno wrote:
>
> > Add a new resource in ofproto-dpif and the corresponding API in
> > ofproto_provider.h to represent and local sampling configuration.
> >
> > Signed-off-by: Adrian Moreno 
>
> Some small nits below.
>
> //Eelco
>
> > ---
> >  ofproto/automake.mk|   2 +
> >  ofproto/ofproto-dpif-lsample.c | 187 +
> >  ofproto/ofproto-dpif-lsample.h |  36 +++
> >  ofproto/ofproto-dpif.c |  38 +++
> >  ofproto/ofproto-dpif.h |   1 +
> >  ofproto/ofproto-provider.h |   9 ++
> >  ofproto/ofproto.c  |  12 +++
> >  ofproto/ofproto.h  |   8 ++
> >  8 files changed, 293 insertions(+)
> >  create mode 100644 ofproto/ofproto-dpif-lsample.c
> >  create mode 100644 ofproto/ofproto-dpif-lsample.h
> >
> > diff --git a/ofproto/automake.mk b/ofproto/automake.mk
> > index 7c08b563b..cb1361b8a 100644
> > --- a/ofproto/automake.mk
> > +++ b/ofproto/automake.mk
> > @@ -30,6 +30,8 @@ ofproto_libofproto_la_SOURCES = \
> > ofproto/ofproto-dpif.h \
> > ofproto/ofproto-dpif-ipfix.c \
> > ofproto/ofproto-dpif-ipfix.h \
> > +   ofproto/ofproto-dpif-lsample.c \
> > +   ofproto/ofproto-dpif-lsample.h \
> > ofproto/ofproto-dpif-mirror.c \
> > ofproto/ofproto-dpif-mirror.h \
> > ofproto/ofproto-dpif-monitor.c \
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > new file mode 100644
> > index 0..d675a116f
> > --- /dev/null
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -0,0 +1,187 @@
> > +/*
> > + * Copyright (c) 2024 Red Hat, Inc.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + * http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +#include 
> > +#include "ofproto-dpif-lsample.h"
> > +
> > +#include "cmap.h"
> > +#include "hash.h"
> > +#include "ofproto.h"
> > +#include "openvswitch/thread.h"
> > +
> > +/* Dpif local sampling.
> > + *
> > + * Thread safety: dpif_lsample allows lockless concurrent reads of local
> > + * sampling exporters as long as the following restrictions are met:
> > + *   1) While the last reference is being dropped, i.e: a thread is calling
> > + *  "dpif_lsample_unref" on the last reference, other threads cannot 
> > call
> > + *  "dpif_lsample_ref".
> > + *   2) Threads do not quiese while holding references to internal
> > + *  lsample_exporter objects.
> > + */
> > +
> > +struct dpif_lsample {
> > +struct cmap exporters;  /* Contains lsample_exporter_node 
> > instances
> > +   indexed by collector_set_id. */
> > +struct ovs_mutex mutex; /* Protects concurrent 
> > insertion/deletion
> > + * of exporters. */
> > +
> > +struct ovs_refcount ref_cnt;/* Controls references to this 
> > instance. */
> > +};
> > +
> > +struct lsample_exporter {
> > +struct ofproto_lsample_options options;
> > +};
> > +
> > +struct lsample_exporter_node {
> > +struct cmap_node node;  /* In dpif_lsample->exporters. */
> > +struct lsample_exporter exporter;
> > +};
> > +
> > +static void
> > +dpif_lsample_delete_exporter(struct dpif_lsample *lsample,
> > + struct lsample_exporter_node *node)
> > +{
> > +ovs_mutex_lock(&lsample->mutex);
> > +cmap_remove(&lsample->exporters, &node->node,
> > +hash_int(node->exporter.options.collector_set_id, 0));
> > +ovs_mutex_unlock(&lsample->mutex);
> > +
> > +ovsrcu_postpone(free, node);
> > +}
> > +
> > +/* Adds an exporter with the provided options which are copied. */
> > +static struct lsample_exporter_node *
> > +dpif_lsample_add_exporter(struct dpif_lsample *lsample,
> > +  const struct ofproto_lsample_options *options)
> > +{
> > +struct lsample_exporter_node *node;
> > +
> > +node = xzalloc(sizeof *node);
> > +node->exporter.options = *options;
> > +
> > +ovs_mutex_lock(&lsample->mutex);
> > +cmap_insert(&lsample->exporters, &node->node,
> > +hash_int(options->collector_set_id, 0));
> > +ovs_mutex_unlock(&lsample->mutex);
> > +
> > +return node;
> > +}
> > +
> > +static struct lsample_exporter_node *
> > +dpif_lsample_find_exporter_node(const struct dpif_lsample *lsample,
> > +const uint32_t collector_set_id)
> > +{

Re: [ovs-dev] [PATCH v1 03/13] ofproto_dpif: Check for psample support.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:17AM GMT, Eelco Chaudron wrote:
> On 4 Jul 2024, at 9:52, Adrian Moreno wrote:
>
> > Only kernel datapath supports this action so add a function in dpif.c
> > that checks for that.
>
> One small nit below, rest looks good. If this is the only change in the
> next series you can add my ACK.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  lib/dpif.c |  7 +++
> >  lib/dpif.h |  1 +
> >  ofproto/ofproto-dpif.c | 43 ++
> >  ofproto/ofproto-dpif.h |  7 ++-
> >  4 files changed, 57 insertions(+), 1 deletion(-)
> >
> > diff --git a/lib/dpif.c b/lib/dpif.c
> > index 71728badc..0a964ba89 100644
> > --- a/lib/dpif.c
> > +++ b/lib/dpif.c
> > @@ -1953,6 +1953,13 @@ dpif_supports_lb_output_action(const struct dpif 
> > *dpif)
> >  return dpif_is_netdev(dpif);
> >  }
> >
> > +bool
> > +dpif_may_support_psample(const struct dpif *dpif)
> > +{
> > +/* Userspace datapath does not support this action. */
> > +return !dpif_is_netdev(dpif);
> > +}
> > +
> >  /* Meters */
> >  void
> >  dpif_meter_get_features(const struct dpif *dpif,
> > diff --git a/lib/dpif.h b/lib/dpif.h
> > index a764e8a59..6bef7d5b3 100644
> > --- a/lib/dpif.h
> > +++ b/lib/dpif.h
> > @@ -941,6 +941,7 @@ int dpif_get_pmds_for_port(const struct dpif * dpif, 
> > odp_port_t port_no,
> >  char *dpif_get_dp_version(const struct dpif *);
> >  bool dpif_supports_tnl_push_pop(const struct dpif *);
> >  bool dpif_may_support_explicit_drop_action(const struct dpif *);
> > +bool dpif_may_support_psample(const struct dpif *);
> >  bool dpif_synced_dp_layers(struct dpif *);
> >
> >  /* Log functions. */
> > diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> > index 4712d10a8..94c84d697 100644
> > --- a/ofproto/ofproto-dpif.c
> > +++ b/ofproto/ofproto-dpif.c
> > @@ -873,6 +873,12 @@ ovs_lb_output_action_supported(struct ofproto_dpif 
> > *ofproto)
> >  return ofproto->backer->rt_support.lb_output_action;
> >  }
> >
> > +bool
> > +ovs_psample_supported(struct ofproto_dpif *ofproto)
> > +{
> > +return ofproto->backer->rt_support.psample;
> > +}
> > +
> >  /* Tests whether 'backer''s datapath supports recirculation.  Only newer
> >   * datapaths support OVS_KEY_ATTR_RECIRC_ID in keys.  We need to disable 
> > some
> >   * features on older datapaths that don't support this feature.
> > @@ -1609,6 +1615,42 @@ check_add_mpls(struct dpif_backer *backer)
> >  return supported;
> >  }
> >
> > +/* Tests whether 'backer''s datapath supports the OVS_ACTION_ATTR_PSAMPLE
> > + * action. */
> > +static bool
> > +check_psample(struct dpif_backer *backer)
> > +{
> > +uint8_t cookie[OVS_PSAMPLE_COOKIE_MAX_SIZE];
> > +struct odputil_keybuf keybuf;
> > +struct ofpbuf actions;
> > +struct ofpbuf key;
> > +struct flow flow;
> > +bool supported;
> > +
> > +struct odp_flow_key_parms odp_parms = {
> > +.flow = &flow,
> > +.probe = true,
> > +};
> > +
> > +memset(&flow, 0, sizeof flow);
> > +ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> > +odp_flow_key_from_flow(&odp_parms, &key);
> > +ofpbuf_init(&actions, 64);
> > +
> > +/* Generate a random max-size cookie. */
> > +random_bytes(cookie, sizeof(cookie));
> > +
> > +odp_put_psample_action(&actions, 10, cookie, sizeof cookie);
> > +
> > +supported = dpif_may_support_psample(backer->dpif) &&
> > +dpif_probe_feature(backer->dpif, "psample", &key, &actions, NULL);
> > +
> > +ofpbuf_uninit(&actions);
> > +VLOG_INFO("%s: Datapath %s psample", dpif_name(backer->dpif),
> > +  supported ? "supports" : "does not support");
> > +return supported;
> > +}
> > +
> >  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)  
> >  \
> >  static bool
> >  \
> >  check_##NAME(struct dpif_backer *backer)   
> >  \
> > @@ -1698,6 +1740,7 @@ check_support(struct dpif_backer *backer)
> >  dpif_supports_lb_output_action(backer->dpif);
> >  backer->rt_support.ct_zero_snat = dpif_supports_ct_zero_snat(backer);
> >  backer->rt_support.add_mpls = check_add_mpls(backer);
> > +backer->rt_support.psample = check_psample(backer);
> >
> >  /* Flow fields. */
> >  backer->rt_support.odp.ct_state = check_ct_state(backer);
> > diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
> > index d33f73df8..bc7a19dab 100644
> > --- a/ofproto/ofproto-dpif.h
> > +++ b/ofproto/ofproto-dpif.h
> > @@ -213,7 +213,10 @@ struct group_dpif *group_dpif_lookup(struct 
> > ofproto_dpif *,
> >  DPIF_SUPPORT_FIELD(bool, ct_zero_snat, "Conntrack all-zero IP SNAT")   
> >  \
> > 
> >  \
> >  /* True if the datapath supports add_mpls action. */   
> >  \
> > -DPIF_SUPPORT_FIELD(bool, add_mp

Re: [ovs-dev] [PATCH v1 02/13] odp-util: Add support OVS_ACTION_ATTR_PSAMPLE.

[Adrián Moreno]

On Tue, Jul 09, 2024 at 11:45:15AM GMT, Eelco Chaudron wrote:
> On 4 Jul 2024, at 9:52, Adrian Moreno wrote:
>
> > Add support for parsing and formatting the new action.
> >
> > Also, flag OVS_ACTION_ATTR_SAMPLE as requiring datapath assistance if it
> > contains a nested OVS_ACTION_ATTR_PSAMPLE. The reason is that the
> > sampling rate form the parent "sample" is made available to the nested
>
> form -> from
>
> > "psample" by the kernel.
>
> Two small comments below, the rest looks good.
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  include/linux/openvswitch.h  | 28 +++
> >  lib/dpif-netdev.c|  1 +
> >  lib/dpif.c   |  1 +
> >  lib/odp-execute.c| 25 +-
> >  lib/odp-util.c   | 93 
> >  lib/odp-util.h   |  3 ++
> >  ofproto/ofproto-dpif-ipfix.c |  1 +
> >  ofproto/ofproto-dpif-sflow.c |  1 +
> >  python/ovs/flow/odp.py   |  8 
> >  tests/odp.at | 16 +++
> >  10 files changed, 176 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/linux/openvswitch.h b/include/linux/openvswitch.h
> > index d9fb991ef..0023b65fb 100644
> > --- a/include/linux/openvswitch.h
> > +++ b/include/linux/openvswitch.h
> > @@ -992,6 +992,31 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
> > +/**
> > + * enum ovs_pample_attr - Attributes for %OVS_ACTION_ATTR_PSAMPLE
> > + * action.
> > + *
> > + * @OVS_PSAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > + * sample.
> > + * @OVS_PSAMPLE_ATTR_COOKIE: An optional variable-length binary cookie that
> > + * contains user-defined metadata. The maximum length is
> > + * OVS_PSAMPLE_COOKIE_MAX_SIZE bytes.
> > + *
> > + * Sends the packet to the psample multicast group with the specified 
> > group and
> > + * cookie. It is possible to combine this action with the
> > + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the sample.
> > + */
> > +enum ovs_psample_attr {
> > +OVS_PSAMPLE_ATTR_GROUP = 1,/* u32 number. */
> > +OVS_PSAMPLE_ATTR_COOKIE,   /* Optional, user specified cookie. 
> > */
> > +
> > +/* private: */
>
> None of the other definitions have this private marking, so I see no need to
> start adding it here.

OK. The uAPI file has it (requested by netdev maintainers) but I guess
it's kernel-specific and this file is not a blind copy of the uAPI
anyways so I think we can remove it. I'll do on the next version.

>
> > +__OVS_PSAMPLE_ATTR_MAX
> > +};
> > +
> > +#define OVS_PSAMPLE_ATTR_MAX (__OVS_PSAMPLE_ATTR_MAX - 1)
> > +
> >  /**
> >   * enum ovs_action_attr - Action types.
> >   *
> > @@ -1056,6 +1081,8 @@ struct check_pkt_len_arg {
> >   * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> >   * argument.
> >   * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > + * @OVS_ACTION_ATTR_PSAMPLE: Send a sample of the packet to external 
> > observers
> > + * via psample.
> >   */
> >
> >  enum ovs_action_attr {
> > @@ -1087,6 +1114,7 @@ enum ovs_action_attr {
> > OVS_ACTION_ATTR_ADD_MPLS, /* struct ovs_action_add_mpls. */
> > OVS_ACTION_ATTR_DEC_TTL,  /* Nested OVS_DEC_TTL_ATTR_*. */
> > OVS_ACTION_ATTR_DROP, /* u32 xlate_error. */
> > +   OVS_ACTION_ATTR_PSAMPLE,  /* Nested OVS_PSAMPLE_ATTR_*. */
> >
> >  #ifndef __KERNEL__
> > OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > index c7f9e1490..f0594e5f5 100644
> > --- a/lib/dpif-netdev.c
> > +++ b/lib/dpif-netdev.c
> > @@ -9519,6 +9519,7 @@ dp_execute_cb(void *aux_, struct dp_packet_batch 
> > *packets_,
> >  case OVS_ACTION_ATTR_DROP:
> >  case OVS_ACTION_ATTR_ADD_MPLS:
> >  case OVS_ACTION_ATTR_DEC_TTL:
> > +case OVS_ACTION_ATTR_PSAMPLE:
> >  case __OVS_ACTION_ATTR_MAX:
> >  OVS_NOT_REACHED();
> >  }
> > diff --git a/lib/dpif.c b/lib/dpif.c
> > index 23eb18495..71728badc 100644
> > --- a/lib/dpif.c
> > +++ b/lib/dpif.c
> > @@ -1192,6 +1192,7 @@ dpif_execute_helper_cb(void *aux_, struct 
> > dp_packet_batch *packets_,
> >  case OVS_ACTION_ATTR_TUNNEL_PUSH:
> >  case OVS_ACTION_ATTR_TUNNEL_POP:
> >  case OVS_ACTION_ATTR_USERSPACE:
> > +case OVS_ACTION_ATTR_PSAMPLE:
> >  case OVS_ACTION_ATTR_RECIRC: {
> >  struct dpif_execute execute;
> >  struct ofpbuf execute_actions;
> > diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> > index 081e4d432..15577d539 100644
> > --- a/lib/odp-execute.c
> > +++ b/lib/odp-execute.c
> > @@ -818,13 +818,13 @@ requires_datapath_assistance(const struct nlattr *a)
> >  case OVS_ACTION_ATTR_RECIRC:
> >  case OVS_ACTION_ATTR_CT:
> >  case OVS_ACTION_ATTR_METER:
> > +case OVS_ACTION_ATTR_PSAMPLE:
> >  return true;
> >
> >  case OVS_ACTION_ATTR_SET:
> >  case OVS_ACTION_ATTR_SET_MASKED:
> >  case OVS_ACTION_ATTR_PUSH_VLAN:
>

Re: [ovs-dev] [PATCH ovn 8/8] northd: Add ACL Sampling.

[Adrián Moreno]

On Mon, Jul 08, 2024 at 01:24:14PM GMT, Dumitru Ceara wrote:
> From: Adrian Moreno 
>
> Introduce a new table called Sample where per-flow IPFIX configuration
> can be specified.
> Also, reference rows from such table from the ACL table to enable the
> configuration of ACL sampling. If enabled, northd will add a sample
> action to each ACL related logical flow.
>
> Packets that hit stateful ACLs are sampled in different ways depending
> whether they are initiating a new session or are just forwarded on an
> existing (already allowed) session.  Two new columns ("sample_new" and
> "sample_est") are added to the ACL table to allow for potentially
> different sampling rates for the two cases.
>
> Note: If an ACL has both sampling enabled and a label associated to it
> then the label value overrides the observation point ID defined in the
> sample configuration.  This is a side effect of the implementation
> (observation point IDs are stored in conntrack in the same part of the
> ct_label where ACL labels are also stored).  The two features
> (sampling and ACL labels) serve however similar purposes so it's not
> expected that they're both enabled together.
>
> When sampling is enabled on an ACL additional logical flows are created
> for that ACL (one for stateless ACLs and 3 for stateful ACLs) in the ACL
> action stage of the logical pipeline.  These additional flows match on a
> combination of conntrack state values and observation point id values
> (either against a logical register or against the stored ct_label state)
> in order to determine whether the packets hitting the ACLs must be
> sampled or not.  This comes with a slight increase in the number of
> logical flows and in the number of OpenFlow rules.  The number of
> additional flows _does not_ depend on the original ACL match or action.
>
> New --sample-new and --sample-est optional arguments are added to the
> 'ovn-nbctl acl-add' command to allow configuring these new types of
> sampling for ACLs.
>
> An example workflow of configuring ACL samples is:
>   # Create Sampling_App mappings for ACL traffic types:
>   ovn-nbctl create Sampling_App name="acl-new-traffic-sampling" \
> id="42"
>   ovn-nbctl create sampling_app name="acl-est-traffic-sampling" \
>   id="43"
>   # Create two sample collectors, one that samples all packets (c1)
>   # and another one that samples with a probability of 10% (c2):
>   c1=$(ovn-nbctl create Sample_Collector name=c1 \
>probability=65535 set_id=1)
>   c2=$(ovn-nbctl create Sample_Collector name=c2 \
>probability=6553 set_id=2)
>   # Create two sample configurations (for new and for established
>   # traffic):
>   s1=$(ovn-nbctl create sample collector="$c1 $c2" metadata=4301)
>   s2=$(ovn-nbctl create sample collector="$c1 $c2" metadata=4302)
>   # Create an ingress ACL to allow IP traffic:
>   ovn-nbctl --sample-new=$s1 --sample-est=$s2 acl-add ls \
> from-lport 1 "ip" allow-related
>
> The config above will generate IPFIX samples with:
> - observation domain id set to 42 (Sampling_App
>   "acl-new-traffic-sampling" config) and observation point id
>   set to 4301 (Sample s1) for packets that create a new
>   connection
> - observation domain id set to 43 (Sampling_app
>   "acl-est-traffic-sampling" config) and observation point id
>   set to 4302 (Sample s2) for packets that are part of an already
>   existing connection
>

We assume the dp_key is 0 in this case, right? Maybe worth mentioning
it.

> Reported-at: https://issues.redhat.com/browse/FDP-305
> Signed-off-by: Adrian Moreno 
> Co-authored-by: Dumitru Ceara 
> Signed-off-by: Dumitru Ceara 
> ---
>  NEWS   |   3 +
>  include/ovn/logical-fields.h   |   2 +
>  lib/logical-fields.c   |   6 +
>  northd/northd.c| 519 +++--
>  northd/ovn-northd.8.xml|  26 ++
>  ovn-nb.ovsschema   |  44 ++-
>  ovn-nb.xml |  56 +++
>  tests/atlocal.in   |   6 +
>  tests/ovn-macros.at|   4 +
>  tests/ovn-nbctl.at |  20 +
>  tests/ovn-northd.at| 240 ++--
>  tests/ovn.at   |   3 +
>  tests/system-common-macros.at  |  11 +
>  tests/system-ovn.at| 149 +++
>  utilities/containers/fedora/Dockerfile |   1 +
>  utilities/containers/ubuntu/Dockerfile |   1 +
>  utilities/ovn-nbctl.8.xml  |   8 +-
>  utilities/ovn-nbctl.c  |  43 +-
>  18 files changed, 1079 insertions(+), 63 deletions(-)

I'm not done reviewing. I need to spend some time understanding how
locial flows are built. Not very familiar with this. Plus I want to run
some tests. However, here are some initial thoughts.

>
> diff --git a/NEWS b/NEWS
> index fcf182bc02..7899c623f2 100644
> --- a/NEWS
> +++ b/NEWS
> @@ -41,

Re: [ovs-dev] [PATCH v1] selftests: openvswitch: retry instead of sleep

[Adrián Moreno]

On Mon, Jul 08, 2024 at 09:31:58PM GMT, Ilya Maximets wrote:
> On 7/8/24 15:44, Adrian Moreno wrote:
> > There are a couple of places where the test script "sleep"s to wait for
> > some external condition to be met.
> >
> > This is error prone, specially in slow systems (identified in CI by
> > "KSFT_MACHINE_SLOW=yes").
> >
> > To fix this, add a "ovs_wait" function that tries to execute a command
> > a few times until it succeeds. The timeout used is set to 5s for
> > "normal" systems and doubled if a slow CI machine is detected.
> >
> > This should make the following work:
> >
> > $ vng --build  \
> > --config tools/testing/selftests/net/config \
> > --config kernel/configs/debug.config
> >
> > $ vng --run . --user root -- "make -C tools/testing/selftests/ \
> > KSFT_MACHINE_SLOW=yes TARGETS=net/openvswitch run_tests"
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  .../selftests/net/openvswitch/openvswitch.sh  | 49 ---
> >  .../selftests/net/openvswitch/ovs-dpctl.py|  1 +
> >  2 files changed, 42 insertions(+), 8 deletions(-)
> >
>
> Hi, Adrian.  See a small pile of nitpicks below.
>
> None of them are blocking from my perspective, except for a typo.
> Just listed them since there is a typo anyway.
>
> > diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh 
> > b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > index bc71dbc18b21..83407b42073a 100755
> > --- a/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > +++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > @@ -11,6 +11,7 @@ ksft_skip=4
> >  PAUSE_ON_FAIL=no
> >  VERBOSE=0
> >  TRACING=0
> > +WAIT_TIMEOUT=5
> >
> >  tests="
> > arp_pingeth-arp: Basic arp ping between 
> > two NS
> > @@ -29,6 +30,32 @@ info() {
> > [ $VERBOSE = 0 ] || echo $*
> >  }
> >
> > +ovs_wait() {
> > +   info "waiting $WAIT_TIMEOUT s for: $@"
> > +
> > +   "$@"
> > +   if [[ $? -eq 0 ]]; then
>
> Maybe just 'if "$@"; then' ?
>

In my head this is a bit less clean but I don't mind.

> > +   info "wait succeeded inmediately"
>
> * immediately

Thanks. Will fix the typo.

>
> > +   return 0
> > +   fi
> > +
> > +   # A quick re-check helps speed up small races in fast systems.
> > +   # However, fractional sleeps might not necessarily work.
> > +   local start=0
> > +   sleep 0.1 || { sleep 1; start=1; }
> > +
> > +   for (( i=start; i
> for i in $(seq ${start} ${WAIT_TIMEOUT}); do
>
> Will need to initialize start to 1 and 2.
>
> It works, but seems like an unnecessary use of non-POSIX constructs.

The reason why I chose this form is that I find it more robust on a
script that changes IFS. If this function is called within a block that
has changed IFS, "i" will take the entire sequence as the value for the
first iteration.

>
> > +   "$@"
> > +   if [[ $? -eq 0 ]]; then
>
> if "$@"; then
>
> > +   info "wait succeeded after $i seconds"
> > +   return 0
> > +   fi
> > +   sleep 1
> > +   done
> > +   info "wait failed after $i seconds"
> > +   return 1
> > +}
> > +
> >  ovs_base=`pwd`
> >  sbxs=
> >  sbx_add () {
> > @@ -278,20 +305,21 @@ test_psample() {
> >
> > # Record psample data.
> > ovs_spawn_daemon "test_psample" python3 $ovs_base/ovs-dpctl.py 
> > psample-events
> > +   ovs_wait grep -q "listening for psample events" ${ovs_dir}/stdout
> >
> > # Send a single ping.
> > -   sleep 1
> > ovs_sbx "test_psample" ip netns exec client ping -I c1 172.31.110.20 -c 
> > 1 || return 1
> > -   sleep 1
> >
> > # We should have received one userspace action upcall and 2 psample 
> > packets.
> > -   grep -E "userspace action command" $ovs_dir/s0.out >/dev/null 2>&1 || 
> > return 1
> > +   ovs_wait grep -q "userspace action command" $ovs_dir/s0.out
> > +   [[ $? -eq 0 ]] || return 1
>
> Why checking separately and not one the same line with || return 1 ?

IMHO, passing complex commands to a function in bash can easily get very
problematic. That's why I try to remove all pipes, redirections or
logical operators like && and ||. At least for me it removes one extra
cycle that my brain has to spend looking at quotes and figuring out if
the operand will be interpreted inside the function or outside.

> Also double brackets seem unnecessary.

That's true.

>
> >
> > # client -> server samples should only contain the first 14 bytes of 
> > the packet.
> > -   grep -E "rate:4294967295,group:1,cookie:c0ffee data:[0-9a-f]{28}$" \
> > -$ovs_dir/stdout >/dev/null 2>&1 || return 1
> > -   grep -E "rate:4294967295,group:2,cookie:eeff0c" \
> > -$ovs_dir/stdout >/dev/null 2>&1 || return 1
> > +   ovs_wait grep -qE "rate:4294967295,group:1,cookie:c0ffee 
> > data:[0-9a-f]{28}$" $ovs_dir/stdout
> > +   [[ $? -eq 0 ]] || return 1
> > +
> > +   ovs_wait grep -q "rate:4294967295,group:2,cookie:eeff0c" $ovs_dir/stdout
> > +   [[ $? -eq 0 ]] || ret

Re: [ovs-dev] [PATCH v1] selftests: openvswitch: retry instead of sleep

[Adrián Moreno]

On Mon, Jul 08, 2024 at 02:27:45PM GMT, Aaron Conole wrote:
> Adrian Moreno  writes:
>
> > There are a couple of places where the test script "sleep"s to wait for
> > some external condition to be met.
> >
> > This is error prone, specially in slow systems (identified in CI by
> > "KSFT_MACHINE_SLOW=yes").
> >
> > To fix this, add a "ovs_wait" function that tries to execute a command
> > a few times until it succeeds. The timeout used is set to 5s for
> > "normal" systems and doubled if a slow CI machine is detected.
> >
> > This should make the following work:
> >
> > $ vng --build  \
> > --config tools/testing/selftests/net/config \
> > --config kernel/configs/debug.config
> >
> > $ vng --run . --user root -- "make -C tools/testing/selftests/ \
> > KSFT_MACHINE_SLOW=yes TARGETS=net/openvswitch run_tests"
> >
> > Signed-off-by: Adrian Moreno 
> > ---
>
> Looks like this does resolve the issue in question on the -dbg
> environment:
>
> https://netdev.bots.linux.dev/contest.html?executor=vmksft-net-dbg&test=openvswitch-sh

Nice! I guess the 10s global timeout is enough for now.

>
> Thanks Adrian!  Also, thanks for including the fractional sleep.
>
> Reviewed-by: Aaron Conole 
>

Thanks.

I just realized that I've missed the target branch ("net-next") in the
subject. I'll wait a bit and respin to fix that.

Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH v1 03/13] ofproto_dpif: Check for psample support.

[Adrián Moreno]

On Mon, Jul 08, 2024 at 12:10:15PM GMT, Eelco Chaudron wrote:
>
>
> On 8 Jul 2024, at 10:37, Adrián Moreno wrote:
>
> > On Sun, Jul 07, 2024 at 10:08:55PM GMT, Adrian Moreno wrote:
> >> Only kernel datapath supports this action so add a function in dpif.c
> >> that checks for that.
> >>
> >> Signed-off-by: Adrian Moreno 
> >> ---
> >>  lib/dpif.c |  7 +++
> >>  lib/dpif.h |  1 +
> >>  ofproto/ofproto-dpif.c | 43 ++
> >>  ofproto/ofproto-dpif.h |  7 ++-
> >>  4 files changed, 57 insertions(+), 1 deletion(-)
> >>
> >
> > After Dumitru's comment on the last patch, I think we also need a
> > capability for this exposed in the OVSDB. I'll add it in the next
> > version.
>
> ACK, can you wait with the v2 until I review the RFC/v1?

Sure, thanks.

>
> >> diff --git a/lib/dpif.c b/lib/dpif.c
> >> index 71728badc..0a964ba89 100644
> >> --- a/lib/dpif.c
> >> +++ b/lib/dpif.c
> >> @@ -1953,6 +1953,13 @@ dpif_supports_lb_output_action(const struct dpif 
> >> *dpif)
> >>  return dpif_is_netdev(dpif);
> >>  }
> >>
> >> +bool
> >> +dpif_may_support_psample(const struct dpif *dpif)
> >> +{
> >> +/* Userspace datapath does not support this action. */
> >> +return !dpif_is_netdev(dpif);
> >> +}
> >> +
> >>  /* Meters */
> >>  void
> >>  dpif_meter_get_features(const struct dpif *dpif,
> >> diff --git a/lib/dpif.h b/lib/dpif.h
> >> index a764e8a59..6bef7d5b3 100644
> >> --- a/lib/dpif.h
> >> +++ b/lib/dpif.h
> >> @@ -941,6 +941,7 @@ int dpif_get_pmds_for_port(const struct dpif * dpif, 
> >> odp_port_t port_no,
> >>  char *dpif_get_dp_version(const struct dpif *);
> >>  bool dpif_supports_tnl_push_pop(const struct dpif *);
> >>  bool dpif_may_support_explicit_drop_action(const struct dpif *);
> >> +bool dpif_may_support_psample(const struct dpif *);
> >>  bool dpif_synced_dp_layers(struct dpif *);
> >>
> >>  /* Log functions. */
> >> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> >> index 4712d10a8..94c84d697 100644
> >> --- a/ofproto/ofproto-dpif.c
> >> +++ b/ofproto/ofproto-dpif.c
> >> @@ -873,6 +873,12 @@ ovs_lb_output_action_supported(struct ofproto_dpif 
> >> *ofproto)
> >>  return ofproto->backer->rt_support.lb_output_action;
> >>  }
> >>
> >> +bool
> >> +ovs_psample_supported(struct ofproto_dpif *ofproto)
> >> +{
> >> +return ofproto->backer->rt_support.psample;
> >> +}
> >> +
> >>  /* Tests whether 'backer''s datapath supports recirculation.  Only newer
> >>   * datapaths support OVS_KEY_ATTR_RECIRC_ID in keys.  We need to disable 
> >> some
> >>   * features on older datapaths that don't support this feature.
> >> @@ -1609,6 +1615,42 @@ check_add_mpls(struct dpif_backer *backer)
> >>  return supported;
> >>  }
> >>
> >> +/* Tests whether 'backer''s datapath supports the OVS_ACTION_ATTR_PSAMPLE
> >> + * action. */
> >> +static bool
> >> +check_psample(struct dpif_backer *backer)
> >> +{
> >> +uint8_t cookie[OVS_PSAMPLE_COOKIE_MAX_SIZE];
> >> +struct odputil_keybuf keybuf;
> >> +struct ofpbuf actions;
> >> +struct ofpbuf key;
> >> +struct flow flow;
> >> +bool supported;
> >> +
> >> +struct odp_flow_key_parms odp_parms = {
> >> +.flow = &flow,
> >> +.probe = true,
> >> +};
> >> +
> >> +memset(&flow, 0, sizeof flow);
> >> +ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> >> +odp_flow_key_from_flow(&odp_parms, &key);
> >> +ofpbuf_init(&actions, 64);
> >> +
> >> +/* Generate a random max-size cookie. */
> >> +random_bytes(cookie, sizeof(cookie));
> >> +
> >> +odp_put_psample_action(&actions, 10, cookie, sizeof cookie);
> >> +
> >> +supported = dpif_may_support_psample(backer->dpif) &&
> >> +dpif_probe_feature(backer->dpif, "psample", &key, &actions, NULL);
> >> +
> >> +ofpbuf_uninit(&actions);
> >> +VLOG_INFO("%s: Datapath %s psample", dpif_name(backe

Re: [ovs-dev] [PATCH v1 03/13] ofproto_dpif: Check for psample support.

[Adrián Moreno]

On Sun, Jul 07, 2024 at 10:08:55PM GMT, Adrian Moreno wrote:
> Only kernel datapath supports this action so add a function in dpif.c
> that checks for that.
>
> Signed-off-by: Adrian Moreno 
> ---
>  lib/dpif.c |  7 +++
>  lib/dpif.h |  1 +
>  ofproto/ofproto-dpif.c | 43 ++
>  ofproto/ofproto-dpif.h |  7 ++-
>  4 files changed, 57 insertions(+), 1 deletion(-)
>

After Dumitru's comment on the last patch, I think we also need a
capability for this exposed in the OVSDB. I'll add it in the next
version.

Thanks.
Adrián


> diff --git a/lib/dpif.c b/lib/dpif.c
> index 71728badc..0a964ba89 100644
> --- a/lib/dpif.c
> +++ b/lib/dpif.c
> @@ -1953,6 +1953,13 @@ dpif_supports_lb_output_action(const struct dpif *dpif)
>  return dpif_is_netdev(dpif);
>  }
>
> +bool
> +dpif_may_support_psample(const struct dpif *dpif)
> +{
> +/* Userspace datapath does not support this action. */
> +return !dpif_is_netdev(dpif);
> +}
> +
>  /* Meters */
>  void
>  dpif_meter_get_features(const struct dpif *dpif,
> diff --git a/lib/dpif.h b/lib/dpif.h
> index a764e8a59..6bef7d5b3 100644
> --- a/lib/dpif.h
> +++ b/lib/dpif.h
> @@ -941,6 +941,7 @@ int dpif_get_pmds_for_port(const struct dpif * dpif, 
> odp_port_t port_no,
>  char *dpif_get_dp_version(const struct dpif *);
>  bool dpif_supports_tnl_push_pop(const struct dpif *);
>  bool dpif_may_support_explicit_drop_action(const struct dpif *);
> +bool dpif_may_support_psample(const struct dpif *);
>  bool dpif_synced_dp_layers(struct dpif *);
>
>  /* Log functions. */
> diff --git a/ofproto/ofproto-dpif.c b/ofproto/ofproto-dpif.c
> index 4712d10a8..94c84d697 100644
> --- a/ofproto/ofproto-dpif.c
> +++ b/ofproto/ofproto-dpif.c
> @@ -873,6 +873,12 @@ ovs_lb_output_action_supported(struct ofproto_dpif 
> *ofproto)
>  return ofproto->backer->rt_support.lb_output_action;
>  }
>
> +bool
> +ovs_psample_supported(struct ofproto_dpif *ofproto)
> +{
> +return ofproto->backer->rt_support.psample;
> +}
> +
>  /* Tests whether 'backer''s datapath supports recirculation.  Only newer
>   * datapaths support OVS_KEY_ATTR_RECIRC_ID in keys.  We need to disable some
>   * features on older datapaths that don't support this feature.
> @@ -1609,6 +1615,42 @@ check_add_mpls(struct dpif_backer *backer)
>  return supported;
>  }
>
> +/* Tests whether 'backer''s datapath supports the OVS_ACTION_ATTR_PSAMPLE
> + * action. */
> +static bool
> +check_psample(struct dpif_backer *backer)
> +{
> +uint8_t cookie[OVS_PSAMPLE_COOKIE_MAX_SIZE];
> +struct odputil_keybuf keybuf;
> +struct ofpbuf actions;
> +struct ofpbuf key;
> +struct flow flow;
> +bool supported;
> +
> +struct odp_flow_key_parms odp_parms = {
> +.flow = &flow,
> +.probe = true,
> +};
> +
> +memset(&flow, 0, sizeof flow);
> +ofpbuf_use_stack(&key, &keybuf, sizeof keybuf);
> +odp_flow_key_from_flow(&odp_parms, &key);
> +ofpbuf_init(&actions, 64);
> +
> +/* Generate a random max-size cookie. */
> +random_bytes(cookie, sizeof(cookie));
> +
> +odp_put_psample_action(&actions, 10, cookie, sizeof cookie);
> +
> +supported = dpif_may_support_psample(backer->dpif) &&
> +dpif_probe_feature(backer->dpif, "psample", &key, &actions, NULL);
> +
> +ofpbuf_uninit(&actions);
> +VLOG_INFO("%s: Datapath %s psample", dpif_name(backer->dpif),
> +  supported ? "supports" : "does not support");
> +return supported;
> +}
> +
>  #define CHECK_FEATURE__(NAME, SUPPORT, FIELD, VALUE, ETHTYPE)   \
>  static bool \
>  check_##NAME(struct dpif_backer *backer)\
> @@ -1698,6 +1740,7 @@ check_support(struct dpif_backer *backer)
>  dpif_supports_lb_output_action(backer->dpif);
>  backer->rt_support.ct_zero_snat = dpif_supports_ct_zero_snat(backer);
>  backer->rt_support.add_mpls = check_add_mpls(backer);
> +backer->rt_support.psample = check_psample(backer);
>
>  /* Flow fields. */
>  backer->rt_support.odp.ct_state = check_ct_state(backer);
> diff --git a/ofproto/ofproto-dpif.h b/ofproto/ofproto-dpif.h
> index d33f73df8..bc7a19dab 100644
> --- a/ofproto/ofproto-dpif.h
> +++ b/ofproto/ofproto-dpif.h
> @@ -213,7 +213,10 @@ struct group_dpif *group_dpif_lookup(struct ofproto_dpif 
> *,
>  DPIF_SUPPORT_FIELD(bool, ct_zero_snat, "Conntrack all-zero IP SNAT")\
>  \
>  /* True if the datapath supports add_mpls action. */\
> -DPIF_SUPPORT_FIELD(bool, add_mpls, "MPLS Label add")
> +DPIF_SUPPORT_FIELD(bool, add_mpls, "MPLS Label add")\
> +\
> +/* True if the datapath supports psample action. */ \
> +DPIF_SUPPORT_FIELD(b

Re: [ovs-dev] [RFC PATCH v3 13/13] ofp-actions: Load data from fields in sample action.

[Adrián Moreno]

On Thu, Jul 04, 2024 at 11:25:34AM GMT, Dumitru Ceara wrote:
> On 7/4/24 09:52, Adrian Moreno wrote:
> > When sample action gets used as a way of sampling traffic with
> > controller-generated metadata (i.e: obs_domain_id and obs_point_id),
> > the controller will have to increase the number of flows to ensure each
> > part of the pipeline contains the right metadata.
> >
> > As an example, if the controller decides to sample stateful traffic, it
> > could store the computed metadata for each connection in the conntrack
> > label. However, for established connections, a flow must be created for
> > each different ct_label value with a sample action that contains a
> > different hardcoded obs_domain and obs_point id.
> >
> > This patch adds a new version of the NXAST_RAW_SAMPLE* action (number 4)
> > that supports specifying the observation point and domain using an
> > OpenFlow field reference, so now the controller can express:
> >
> >  sample(...
> > obs_domain_id=NXM_NX_CT_LABEL[0..31],
> > obs_point_id=NXM_NX_CT_LABEL[32..63]
> > ...
> >)
> >
> > Signed-off-by: Adrian Moreno 
> > ---
>
> Hi Adrian,
>
> Thanks a lot for working on expanding the sampling support (in the
> kernel too).
>
> I didn't review the patch (or the rest of the series) yet but one thing
> we should definitely add to the non-RFC version is a way for controllers
> to detect that this new action version is supported.
>
> In other occasions we've used the OVSDB.Datapath.Capabilities column to
> report that new actions are supported (e.g., "ct_flush" for the
> extension that allows flushing CT with a generic match).  It might make
> sense to add another capability there for this new action version.
>

Yep. I've sent v1 but it does not contain this since I wanted to do a
quick respin removing the RFC now that the kernel part landed in
net-next.

I'll include it in v2.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next 0/3] selftests: openvswitch: Address some flakes in the CI environment

[Adrián Moreno]

On Fri, Jul 05, 2024 at 09:49:12AM GMT, Aaron Conole wrote:
> Jakub Kicinski  writes:
>
> > On Tue,  2 Jul 2024 09:28:27 -0400 Aaron Conole wrote:
> >> These patches aim to make using the openvswitch testsuite more reliable.
> >> These should address the major sources of flakiness in the openvswitch
> >> test suite allowing the CI infrastructure to exercise the openvswitch
> >> module for patch series.  There should be no change for users who simply
> >> run the tests (except that patch 3/3 does make some of the debugging a bit
> >> easier by making some output more verbose).
> >
> > Hi Aaron!
> >
> > The results look solid on normal builds now, but with a debug kernel
> > the test is failing consistently:
> >
> > https://netdev.bots.linux.dev/contest.html?executor=vmksft-net-dbg&test=openvswitch-sh
>
> Yes - it shows a test case issue with the upcall and psample tests.
>
> Adrian and I discussed the correct approach would be using a wait_for
> instead of just sleeping, because it seems the dbg environment might be
> too racy.  I think he is working on a follow up to submit after the
> psample work gets merged - we were hoping not to hold that patch series
> up with more potential conflicts or merge issues if that's okay.
>

Yes. I am working on a patch to solve the failures in slow systems.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next v9 00/10] net: openvswitch: Add sample multicasting.

[Adrián Moreno]

On Thu, Jul 04, 2024 at 10:56:51AM GMT, Adrian Moreno wrote:
> ** Background **
> Currently, OVS supports several packet sampling mechanisms (sFlow,
> per-bridge IPFIX, per-flow IPFIX). These end up being translated into a
> userspace action that needs to be handled by ovs-vswitchd's handler
> threads only to be forwarded to some third party application that
> will somehow process the sample and provide observability on the
> datapath.
>
> A particularly interesting use-case is controller-driven
> per-flow IPFIX sampling where the OpenFlow controller can add metadata
> to samples (via two 32bit integers) and this metadata is then available
> to the sample-collecting system for correlation.
>
> ** Problem **
> The fact that sampled traffic share netlink sockets and handler thread
> time with upcalls, apart from being a performance bottleneck in the
> sample extraction itself, can severely compromise the datapath,
> yielding this solution unfit for highly loaded production systems.
>
> Users are left with little options other than guessing what sampling
> rate will be OK for their traffic pattern and system load and dealing
> with the lost accuracy.
>
> Looking at available infrastructure, an obvious candidated would be
> to use psample. However, it's current state does not help with the
> use-case at stake because sampled packets do not contain user-defined
> metadata.
>
> ** Proposal **
> This series is an attempt to fix this situation by extending the
> existing psample infrastructure to carry a variable length
> user-defined cookie.
>
> The main existing user of psample is tc's act_sample. It is also
> extended to forward the action's cookie to psample.
>
> Finally, a new OVS action (OVS_SAMPLE_ATTR_PSAMPLE) is created.
> It accepts a group and an optional cookie and uses psample to
> multicast the packet and the metadata.
>
> --
> v8 -> v9:
> - Rebased.
>
> v7 -> v8:
> - Rebased
> - Redirect flow insertion to /dev/null to avoid spat in test.
> - Removed inline keyword in stub execute_psample_action function.
>
> v6 -> v7:
> - Rebased
> - Fixed typo in comment.
>
> v5 -> v6:
> - Renamed emit_sample -> psample
> - Addressed unused variable and conditionally compilation of function.
>
> v4 -> v5:
> - Rebased.
> - Removed lefover enum value and wrapped some long lines in selftests.
>
> v3 -> v4:
> - Rebased.
> - Addressed Jakub's comment on private and unused nla attributes.
>
> v2 -> v3:
> - Addressed comments from Simon, Aaron and Ilya.
> - Dropped probability propagation in nested sample actions.
> - Dropped patch v2's 7/9 in favor of a userspace implementation and
> consume skb if emit_sample is the last action, same as we do with
> userspace.
> - Split ovs-dpctl.py features in independent patches.
>
> v1 -> v2:
> - Create a new action ("emit_sample") rather than reuse existing
>   "sample" one.
> - Add probability semantics to psample's sampling rate.
> - Store sampling probability in skb's cb area and use it in emit_sample.
> - Test combining "emit_sample" with "trunc"
> - Drop group_id filtering and tracepoint in psample.
>
> rfc_v2 -> v1:
> - Accommodate Ilya's comments.
> - Split OVS's attribute in two attributes and simplify internal
> handling of psample arguments.
> - Extend psample and tc with a user-defined cookie.
> - Add a tracepoint to psample to facilitate troubleshooting.
>
> rfc_v1 -> rfc_v2:
> - Use psample instead of a new OVS-only multicast group.
> - Extend psample and tc with a user-defined cookie.
>
> Adrian Moreno (10):
>   net: psample: add user cookie
>   net: sched: act_sample: add action cookie to sample
>   net: psample: skip packet copy if no listeners
>   net: psample: allow using rate as probability
>   net: openvswitch: add psample action
>   net: openvswitch: store sampling probability in cb.
>   selftests: openvswitch: add psample action
>   selftests: openvswitch: add userspace parsing
>   selftests: openvswitch: parse trunc action
>   selftests: openvswitch: add psample test
>
>  Documentation/netlink/specs/ovs_flow.yaml |  17 ++
>  include/net/psample.h |   5 +-
>  include/uapi/linux/openvswitch.h  |  31 +-
>  include/uapi/linux/psample.h  |  11 +-
>  net/openvswitch/Kconfig   |   1 +
>  net/openvswitch/actions.c |  66 -
>  net/openvswitch/datapath.h|   3 +
>  net/openvswitch/flow_netlink.c|  32 ++-
>  net/openvswitch/vport.c   |   1 +
>  net/psample/psample.c |  16 +-
>  net/sched/act_sample.c|  12 +
>  .../selftests/net/openvswitch/openvswitch.sh  | 115 +++-
>  .../selftests/net/openvswitch/ovs-dpctl.py| 272 +-
>  13 files changed, 566 insertions(+), 16 deletions(-)
>
> --
> 2.45.2
>

Hi,

Simon Horman has spotted that openvswitch.sh tests are failing in the
debug executor:

https://netdev.bots.linux.dev/contest.html?test=openvswitch-sh

The

Re: [ovs-dev] [RFC PATCH v3 00/13] Introduce local sampling with NXAST_SAMPLE action.

[Adrián Moreno]

On Thu, Jul 04, 2024 at 09:52:41AM GMT, Adrian Moreno wrote:
> (Was: Add psample support to NXAST_SAMPLE action)
>
> This is the userspace counterpart of the work being done in the kernel
> [1] which is still not merged (hence the RFC state). There, a new
> datapath action is added, called "psample".
>
> From the PoV of ovs-vswitchd, this new action is used to implement
> "local sampling". Local sampling (or lsample for short) is configured
> in a similar way as current per-flow IPFIX sampling, i.e: using the
> Flow_Sample_Collector_Set table and the NXAST_SAMPLE action.
>
> However, instead of sending the sample to an external IPFIX collector
> though the network, the sample is emitted using the new action and
> made available to locally running sample collector.
>
> The specific way emit_sample sends the sample (and the way the local
> collector shall collect it) is datapath-specific.
> Currently, currently only the Linux kernel datapath implements it using
> the psample netlink multicast group.
>
> ~~ Configuration ~~
> Local sampling is configured via a new column in the
> Flow_Sample_Collector_Set (FSCS) table called "local_sample_group".
> Configuring this value is orthogonal to also associating the FSCS
> entry to an entry in the IPFIX table.
>
> Once that entry in the OVSDB is configured, NXAST_SAMPLE actions coming
> from the controller will be translated into the following odp action:
>
>sample(sample={P}%, actions(emit_sample(group={G},cookie={C})))
>
> Where:
> P: Is the sampling probability from NXAST_SAMPLE
> G: Is the group id in the FSCS entry whose "id" matches the one in
> the NXAST_SAMPLE.
> C: Is a 64bit cookie result of concatenating the obs_domain and
> obs_point from the NXAST_SAMPLE in network order, i.e:
> "htonl(obs_domain) << 32 | htonl(obs_point)"
> Notes:
> - The parent sample action might be omitted if the probability is
>   100% and there is no IPFIX sampling that requires the use of a
>   meter.
>
> ~~ Dpif-lsample ~~
> Internally, a new object called "dpif-lsample" is introduced to track
> the configured local sampling exporters and track statistics based on
> odp flow stats (using xcache).
> It exposes the list of configured exporters and their statistics on a
> new unixctl command called "lsample/show".
>

I just realized  I forgot to add a comment explicitly stating that the
above two sections below (which translate to patches 11/13, 12/13 and
13/13) are new in this version of the RFC series.

I know this can be problematic given the late stage we're in so I'll add
a bit os context on why I added them.

> ~~ Drop monitoring ~~
> A common use-case for this action can be to sample drops. However,
> adding sample actions to drops makes the existing drop statistics
> disappear. In order to fix this, patches 11 and 12 make use of explicit
> drop actions to ensure statistics still report drops even if sampled.
>

Drop monitoring and the interaction with local (or even non-local)
sampling has been discussed in the kernel series as I originally tried
to solve the problem in the kernel. After some discussions with Ilya we
agreed to explore the solution to the problem in userspace. That is why
I feel these patches are related to the series.

In any case, IMHO, both patches fix existing bugs: Enabling sampling
(local or not, per-bridge or per-flow) should not hide drop statistics.
One visibility feature should not break an existing one.

> ~~ Extended OpenFlow sample action ~~
> Given the series aims at making sampling production ready, conntrack
> integration must be considered. A common use-case for state-full
> pipelines is to calculate the observation metadata at connection
> establishment, store it in ct_label and then use it for packets of
> established connections. However, this forces OVN to create a big number
> of OFP Flows (one per distinct cookie). Patch 13 solves this by allowing
> controllers to specify the obs_domain and point ids from another OFP
> field.
>

This is an addition that, although discussed informally, did not come
directly from the kernel series but from experimentation and interaction
with the OVN team.

It can be considered a follow-up optimization so if there is controversy
around it, I'm OK postponing it to a future release.

> ~~ Testing ~~
> The series includes an test utility program than can be executed by
> running "tests/ovstest test-psample". This utility listens
> to packets multicasted by the psample module and prints them (also
> printing the obs_domain and obs_point ids).
>
> ~~ HW Offload ~~
> tc offload is not being introduced in this series as existing sample
> or userspace actions are not currently offloadable. Also some
> improvements need to be implemented in tc for it to be feasible.
>
> ~~ DPDK datapath ~~
> By naming the action "psample" it was intentionally restricted to the
> Linux datapath only. A follow up task would be spawned to think of a
> good way of implementing local-samp

Re: [ovs-dev] [PATCH net-next v8 00/10] net: openvswitch: Add sample multicasting.

[Adrián Moreno]

On Wed, Jul 03, 2024 at 07:35:19PM GMT, Jakub Kicinski wrote:
> On Tue,  2 Jul 2024 11:53:17 +0200 Adrian Moreno wrote:
> >   selftests: openvswitch: add psample action
> >   selftests: openvswitch: add userspace parsing
> >   selftests: openvswitch: parse trunc action
> >   selftests: openvswitch: add psample test
>
> Sorry for applying things slightly out of order but without Aaron's
> patches we can't check if these work in our CI. Unfortunately, they
> conflict, could you respin one more time?

No problem. I'll respin.

Adrián

> --
> pw-bot: cr
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next v7 05/10] net: openvswitch: add psample action

[Adrián Moreno]

On Tue, Jul 02, 2024 at 01:33:01PM GMT, Simon Horman wrote:
> On Tue, Jul 02, 2024 at 11:53:01AM +0200, Ilya Maximets wrote:
> > On 7/2/24 11:37, Simon Horman wrote:
> > > On Tue, Jul 02, 2024 at 03:05:02AM -0400, Adrián Moreno wrote:
> > >> On Mon, Jul 01, 2024 at 02:23:12PM GMT, Aaron Conole wrote:
> > >>> Adrian Moreno  writes:
> > >
> > > ...
> > >
> > >>>> diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c
> > >
> > > ...
> > >
> > >>>> @@ -1299,6 +1304,39 @@ static int execute_dec_ttl(struct sk_buff *skb, 
> > >>>> struct sw_flow_key *key)
> > >>>>return 0;
> > >>>>  }
> > >>>>
> > >>>> +#if IS_ENABLED(CONFIG_PSAMPLE)
> > >>>> +static void execute_psample(struct datapath *dp, struct sk_buff *skb,
> > >>>> +  const struct nlattr *attr)
> > >>>> +{
> > >>>> +  struct psample_group psample_group = {};
> > >>>> +  struct psample_metadata md = {};
> > >>>> +  const struct nlattr *a;
> > >>>> +  int rem;
> > >>>> +
> > >>>> +  nla_for_each_attr(a, nla_data(attr), nla_len(attr), rem) {
> > >>>> +  switch (nla_type(a)) {
> > >>>> +  case OVS_PSAMPLE_ATTR_GROUP:
> > >>>> +  psample_group.group_num = nla_get_u32(a);
> > >>>> +  break;
> > >>>> +
> > >>>> +  case OVS_PSAMPLE_ATTR_COOKIE:
> > >>>> +  md.user_cookie = nla_data(a);
> > >>>> +  md.user_cookie_len = nla_len(a);
> > >>>> +  break;
> > >>>> +  }
> > >>>> +  }
> > >>>> +
> > >>>> +  psample_group.net = ovs_dp_get_net(dp);
> > >>>> +  md.in_ifindex = OVS_CB(skb)->input_vport->dev->ifindex;
> > >>>> +  md.trunc_size = skb->len - OVS_CB(skb)->cutlen;
> > >>>> +
> > >>>> +  psample_sample_packet(&psample_group, skb, 0, &md);
> > >>>> +}
> > >>>> +#else
> > >>>> +static inline void execute_psample(struct datapath *dp, struct 
> > >>>> sk_buff *skb,
> > >>>> + const struct nlattr *attr) {}
> > >>>
> > >>> I noticed that this got flagged in patchwork since it is 'static inline'
> > >>> while being part of a complete translation unit - but I also see some
> > >>> other places where that has been done.  I guess it should be just
> > >>> 'static' though.  I don't feel very strongly about it.
> > >>>
> > >>
> > >> We had a bit of discussion about this with Ilya. It seems "static
> > >> inline" is a common pattern around the kernel. The coding style
> > >> documentation says:
> > >> "Generally, inline functions are preferable to macros resembling 
> > >> functions."
> > >>
> > >> So I think this "inline" is correct but I might be missing something.
> > >
> > > Hi Adrián,
> > >
> > > TL;DR: Please remove this inline keyword
> > >
> > > For Kernel networking code at least it is strongly preferred not
> > > to use inline in .c files unless there is a demonstrable - usually
> > > performance - reason to do so. Rather, it is preferred to let the
> > > compiler decide when to inline such functions. OTOH, the inline
> > > keyword in .h files is fine.
> >
> > FWIW, the main reason for 'inline' here is not performance, but silencing
> > compiler's potential 'maybe unused' warnings:
> >
> >  Function-like macros with unused parameters should be replaced by static
> >  inline functions to avoid the issue of unused variables
> >
> > I think, the rule for static inline functions in .c files is at odds with
> > the 'Conditional Compilation' section of coding style.  The section does
> > recommend to avoid conditional function declaration in .c files, but I'm not
> > sure it is reasonable to export internal static functions for that reason.
> >
> > In this particular case we can either define a macro, which is discouraged
> > by the coding style:
> >
> >  Generally, inline functions are preferable to macros resembling functions.
> >
> > Or create a static inline function, that is against rule of no static
> > inline functions in .c files.
> >
> > Or create a simple static function and mark all the arguments as unused,
> > which kind of compliant to the coding style, but the least pretty.
>
> Hi Ilya,
>
> I guess I would lean towards the last option.
> But in any case, thanks for pointing out that this is complex:
> I had not realised that when I wrote my previous email.
>

In that case this version (v7) should be good to go? Are there any other
comments? Or is v8 preferred (the only change is between them is
dropping the "inline")?

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next v7 05/10] net: openvswitch: add psample action

[Adrián Moreno]

On Tue, Jul 02, 2024 at 10:37:26AM GMT, Simon Horman wrote:
> On Tue, Jul 02, 2024 at 03:05:02AM -0400, Adrián Moreno wrote:
> > On Mon, Jul 01, 2024 at 02:23:12PM GMT, Aaron Conole wrote:
> > > Adrian Moreno  writes:
>
> ...
>
> > > > diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c
>
> ...
>
> > > > @@ -1299,6 +1304,39 @@ static int execute_dec_ttl(struct sk_buff *skb, 
> > > > struct sw_flow_key *key)
> > > > return 0;
> > > >  }
> > > >
> > > > +#if IS_ENABLED(CONFIG_PSAMPLE)
> > > > +static void execute_psample(struct datapath *dp, struct sk_buff *skb,
> > > > +   const struct nlattr *attr)
> > > > +{
> > > > +   struct psample_group psample_group = {};
> > > > +   struct psample_metadata md = {};
> > > > +   const struct nlattr *a;
> > > > +   int rem;
> > > > +
> > > > +   nla_for_each_attr(a, nla_data(attr), nla_len(attr), rem) {
> > > > +   switch (nla_type(a)) {
> > > > +   case OVS_PSAMPLE_ATTR_GROUP:
> > > > +   psample_group.group_num = nla_get_u32(a);
> > > > +   break;
> > > > +
> > > > +   case OVS_PSAMPLE_ATTR_COOKIE:
> > > > +   md.user_cookie = nla_data(a);
> > > > +   md.user_cookie_len = nla_len(a);
> > > > +   break;
> > > > +   }
> > > > +   }
> > > > +
> > > > +   psample_group.net = ovs_dp_get_net(dp);
> > > > +   md.in_ifindex = OVS_CB(skb)->input_vport->dev->ifindex;
> > > > +   md.trunc_size = skb->len - OVS_CB(skb)->cutlen;
> > > > +
> > > > +   psample_sample_packet(&psample_group, skb, 0, &md);
> > > > +}
> > > > +#else
> > > > +static inline void execute_psample(struct datapath *dp, struct sk_buff 
> > > > *skb,
> > > > +  const struct nlattr *attr) {}
> > >
> > > I noticed that this got flagged in patchwork since it is 'static inline'
> > > while being part of a complete translation unit - but I also see some
> > > other places where that has been done.  I guess it should be just
> > > 'static' though.  I don't feel very strongly about it.
> > >
> >
> > We had a bit of discussion about this with Ilya. It seems "static
> > inline" is a common pattern around the kernel. The coding style
> > documentation says:
> > "Generally, inline functions are preferable to macros resembling functions."
> >
> > So I think this "inline" is correct but I might be missing something.
>
> Hi Adrián,
>
> TL;DR: Please remove this inline keyword
>
> For Kernel networking code at least it is strongly preferred not
> to use inline in .c files unless there is a demonstrable - usually
> performance - reason to do so. Rather, it is preferred to let the
> compiler decide when to inline such functions. OTOH, the inline
> keyword in .h files is fine.
>

Ok. I'll send a new version.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next v7 05/10] net: openvswitch: add psample action

[Adrián Moreno]

On Tue, Jul 02, 2024 at 03:05:02AM GMT, Adrián Moreno wrote:
> On Mon, Jul 01, 2024 at 02:23:12PM GMT, Aaron Conole wrote:
> > Adrian Moreno  writes:
> >
> > > Add support for a new action: psample.
> > >
> > > This action accepts a u32 group id and a variable-length cookie and uses
> > > the psample multicast group to make the packet available for
> > > observability.
> > >
> > > The maximum length of the user-defined cookie is set to 16, same as
> > > tc_cookie, to discourage using cookies that will not be offloadable.
> > >
> > > Acked-by: Eelco Chaudron 
> > > Signed-off-by: Adrian Moreno 
> > > ---
> >
> > Hi Adrian,
> >
> > Just some nits below.
> >
> > >  Documentation/netlink/specs/ovs_flow.yaml | 17 
> > >  include/uapi/linux/openvswitch.h  | 28 ++
> > >  net/openvswitch/Kconfig   |  1 +
> > >  net/openvswitch/actions.c | 47 +++
> > >  net/openvswitch/flow_netlink.c| 32 ++-
> > >  5 files changed, 124 insertions(+), 1 deletion(-)
> > >
> > > diff --git a/Documentation/netlink/specs/ovs_flow.yaml 
> > > b/Documentation/netlink/specs/ovs_flow.yaml
> > > index 4fdfc6b5cae9..46f5d1cd8a5f 100644
> > > --- a/Documentation/netlink/specs/ovs_flow.yaml
> > > +++ b/Documentation/netlink/specs/ovs_flow.yaml
> > > @@ -727,6 +727,12 @@ attribute-sets:
> > >  name: dec-ttl
> > >  type: nest
> > >  nested-attributes: dec-ttl-attrs
> > > +  -
> > > +name: psample
> > > +type: nest
> > > +nested-attributes: psample-attrs
> > > +doc: |
> > > +  Sends a packet sample to psample for external observation.
> > >-
> > >  name: tunnel-key-attrs
> > >  enum-name: ovs-tunnel-key-attr
> > > @@ -938,6 +944,17 @@ attribute-sets:
> > >-
> > >  name: gbp
> > >  type: u32
> > > +  -
> > > +name: psample-attrs
> > > +enum-name: ovs-psample-attr
> > > +name-prefix: ovs-psample-attr-
> > > +attributes:
> > > +  -
> > > +name: group
> > > +type: u32
> > > +  -
> > > +name: cookie
> > > +type: binary
> > >
> > >  operations:
> > >name-prefix: ovs-flow-cmd-
> > > diff --git a/include/uapi/linux/openvswitch.h 
> > > b/include/uapi/linux/openvswitch.h
> > > index efc82c318fa2..3dd653748725 100644
> > > --- a/include/uapi/linux/openvswitch.h
> > > +++ b/include/uapi/linux/openvswitch.h
> > > @@ -914,6 +914,31 @@ struct check_pkt_len_arg {
> > >  };
> > >  #endif
> > >
> > > +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
> > > +/**
> > > + * enum ovs_psample_attr - Attributes for %OVS_ACTION_ATTR_PSAMPLE
> > > + * action.
> > > + *
> > > + * @OVS_PSAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > > + * sample.
> > > + * @OVS_PSAMPLE_ATTR_COOKIE: An optional variable-length binary cookie 
> > > that
> > > + * contains user-defined metadata. The maximum length is
> > > + * OVS_PSAMPLE_COOKIE_MAX_SIZE bytes.
> > > + *
> > > + * Sends the packet to the psample multicast group with the specified 
> > > group and
> > > + * cookie. It is possible to combine this action with the
> > > + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the sample.
> > > + */
> > > +enum ovs_psample_attr {
> > > + OVS_PSAMPLE_ATTR_GROUP = 1, /* u32 number. */
> > > + OVS_PSAMPLE_ATTR_COOKIE,/* Optional, user specified cookie. */
> > > +
> > > + /* private: */
> > > + __OVS_PSAMPLE_ATTR_MAX
> > > +};
> > > +
> > > +#define OVS_PSAMPLE_ATTR_MAX (__OVS_PSAMPLE_ATTR_MAX - 1)
> > > +
> > >  /**
> > >   * enum ovs_action_attr - Action types.
> > >   *
> > > @@ -966,6 +991,8 @@ struct check_pkt_len_arg {
> > >   * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> > >   * argument.
> > >   * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > > + * @OVS_ACTION_ATTR_PSAMPLE: Send a sample of the packet to external 
> > > observers
> > > + * via psample.
> > >   *
> > >   * Only a single header can be set wi

Re: [ovs-dev] [PATCH net-next v7 10/10] selftests: openvswitch: add psample test

[Adrián Moreno]

On Mon, Jul 01, 2024 at 02:38:44PM GMT, Aaron Conole wrote:
> Adrian Moreno  writes:
>
> > Add a test to verify sampling packets via psample works.
> >
> > In order to do that, create a subcommand in ovs-dpctl.py to listen to
> > on the psample multicast group and print samples.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  .../selftests/net/openvswitch/openvswitch.sh  | 115 +-
> >  .../selftests/net/openvswitch/ovs-dpctl.py|  73 ++-
> >  2 files changed, 182 insertions(+), 6 deletions(-)
> >
> > diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh 
> > b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > index 15bca0708717..02a366e01004 100755
> > --- a/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > +++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > @@ -20,7 +20,8 @@ tests="
> > nat_related_v4  ip4-nat-related: ICMP related 
> > matches work with SNAT
> > netlink_checks  ovsnl: validate netlink attrs 
> > and settings
> > upcall_interfaces   ovs: test the upcall interfaces
> > -   drop_reason drop: test drop reasons are 
> > emitted"
> > +   drop_reason drop: test drop reasons are 
> > emitted
> > +   psample psample: Sampling packets with 
> > psample"
> >
> >  info() {
> >  [ $VERBOSE = 0 ] || echo $*
> > @@ -102,12 +103,21 @@ ovs_netns_spawn_daemon() {
> > shift
> > netns=$1
> > shift
> > -   info "spawning cmd: $*"
> > -   ip netns exec $netns $*  >> $ovs_dir/stdout  2>> $ovs_dir/stderr &
> > +   if [ "$netns" == "_default" ]; then
> > +   $*  >> $ovs_dir/stdout  2>> $ovs_dir/stderr &
> > +   else
> > +   ip netns exec $netns $*  >> $ovs_dir/stdout  2>> 
> > $ovs_dir/stderr &
> > +   fi
> > pid=$!
> > ovs_sbx "$sbx" on_exit "kill -TERM $pid 2>/dev/null"
> >  }
> >
> > +ovs_spawn_daemon() {
> > +   sbx=$1
> > +   shift
> > +   ovs_netns_spawn_daemon $sbx "_default" $*
> > +}
> > +
> >  ovs_add_netns_and_veths () {
> > info "Adding netns attached: sbx:$1 dp:$2 {$3, $4, $5}"
> > ovs_sbx "$1" ip netns add "$3" || return 1
> > @@ -170,6 +180,19 @@ ovs_drop_reason_count()
> > return `echo "$perf_output" | grep "$pattern" | wc -l`
> >  }
> >
> > +ovs_test_flow_fails () {
> > +   ERR_MSG="Flow actions may not be safe on all matching packets"
> > +
> > +   PRE_TEST=$(dmesg | grep -c "${ERR_MSG}")
> > +   ovs_add_flow $@ &> /dev/null $@ && return 1
> > +   POST_TEST=$(dmesg | grep -c "${ERR_MSG}")
> > +
> > +   if [ "$PRE_TEST" == "$POST_TEST" ]; then
> > +   return 1
> > +   fi
> > +   return 0
> > +}
> > +
> >  usage() {
> > echo
> > echo "$0 [OPTIONS] [TEST]..."
> > @@ -184,6 +207,92 @@ usage() {
> > exit 1
> >  }
> >
> > +
> > +# psample test
> > +# - use psample to observe packets
> > +test_psample() {
> > +   sbx_add "test_psample" || return $?
> > +
> > +   # Add a datapath with per-vport dispatching.
> > +   ovs_add_dp "test_psample" psample -V 2:1 || return 1
> > +
> > +   info "create namespaces"
> > +   ovs_add_netns_and_veths "test_psample" "psample" \
> > +   client c0 c1 172.31.110.10/24 -u || return 1
> > +   ovs_add_netns_and_veths "test_psample" "psample" \
> > +   server s0 s1 172.31.110.20/24 -u || return 1
> > +
> > +   # Check if psample actions can be configured.
> > +   ovs_add_flow "test_psample" psample \
> > +   'in_port(1),eth(),eth_type(0x0806),arp()' 'psample(group=1)'
>
> Might be good to redirect this stdout/stderr line to /dev/null -
> otherwise on an unsupported system there will be the following extra
> splat:
>
>   Traceback (most recent call last):
> File 
> "/home/aconole/git/linux/tools/testing/selftests/net/openvswitch/ovs-dpctl.py",
>  line 2774, in 
>   sys.exit(main(sys.argv))
>...
> File "/usr/lib/python3.12/site-packages/pyroute2/netlink/nlsocket.py", 
> line 489, in get
>   raise msg['header']['error']
>   pyroute2.netlink.exceptions.NetlinkError: (22, 'Invalid argument')
>

I thought knowing the return value was kind of useful but sure, we can
redirect it to /dev/null.

> > +   if [ $? == 1 ]; then
> > +   info "no support for psample - skipping"
> > +   ovs_exit_sig
> > +   return $ksft_skip
> > +   fi
> > +
> > +   ovs_del_flows "test_psample" psample
> > +
> > +   # Test action verification.
> > +   OLDIFS=$IFS
> > +   IFS='*'
> > +   min_key='in_port(1),eth(),eth_type(0x0800),ipv4()'
> > +   for testcase in \
> > +   "cookie to 
> > large"*"psample(group=1,cookie=1615141312111009080706050403020100)" \
> > +   "no group with cookie"*"psample(cookie=abcd)" \
> > +   "no group"*"psample()";
> > +   do
> > +   set -- $testcase;
> > +   ovs_test_flow_fails "test_psample" psample $min_key $2
> > +   if [ $? == 1 ]; then
> > +   in

Re: [ovs-dev] [PATCH net-next v7 05/10] net: openvswitch: add psample action

[Adrián Moreno]

On Mon, Jul 01, 2024 at 02:23:12PM GMT, Aaron Conole wrote:
> Adrian Moreno  writes:
>
> > Add support for a new action: psample.
> >
> > This action accepts a u32 group id and a variable-length cookie and uses
> > the psample multicast group to make the packet available for
> > observability.
> >
> > The maximum length of the user-defined cookie is set to 16, same as
> > tc_cookie, to discourage using cookies that will not be offloadable.
> >
> > Acked-by: Eelco Chaudron 
> > Signed-off-by: Adrian Moreno 
> > ---
>
> Hi Adrian,
>
> Just some nits below.
>
> >  Documentation/netlink/specs/ovs_flow.yaml | 17 
> >  include/uapi/linux/openvswitch.h  | 28 ++
> >  net/openvswitch/Kconfig   |  1 +
> >  net/openvswitch/actions.c | 47 +++
> >  net/openvswitch/flow_netlink.c| 32 ++-
> >  5 files changed, 124 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/netlink/specs/ovs_flow.yaml 
> > b/Documentation/netlink/specs/ovs_flow.yaml
> > index 4fdfc6b5cae9..46f5d1cd8a5f 100644
> > --- a/Documentation/netlink/specs/ovs_flow.yaml
> > +++ b/Documentation/netlink/specs/ovs_flow.yaml
> > @@ -727,6 +727,12 @@ attribute-sets:
> >  name: dec-ttl
> >  type: nest
> >  nested-attributes: dec-ttl-attrs
> > +  -
> > +name: psample
> > +type: nest
> > +nested-attributes: psample-attrs
> > +doc: |
> > +  Sends a packet sample to psample for external observation.
> >-
> >  name: tunnel-key-attrs
> >  enum-name: ovs-tunnel-key-attr
> > @@ -938,6 +944,17 @@ attribute-sets:
> >-
> >  name: gbp
> >  type: u32
> > +  -
> > +name: psample-attrs
> > +enum-name: ovs-psample-attr
> > +name-prefix: ovs-psample-attr-
> > +attributes:
> > +  -
> > +name: group
> > +type: u32
> > +  -
> > +name: cookie
> > +type: binary
> >
> >  operations:
> >name-prefix: ovs-flow-cmd-
> > diff --git a/include/uapi/linux/openvswitch.h 
> > b/include/uapi/linux/openvswitch.h
> > index efc82c318fa2..3dd653748725 100644
> > --- a/include/uapi/linux/openvswitch.h
> > +++ b/include/uapi/linux/openvswitch.h
> > @@ -914,6 +914,31 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
> > +/**
> > + * enum ovs_psample_attr - Attributes for %OVS_ACTION_ATTR_PSAMPLE
> > + * action.
> > + *
> > + * @OVS_PSAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > + * sample.
> > + * @OVS_PSAMPLE_ATTR_COOKIE: An optional variable-length binary cookie that
> > + * contains user-defined metadata. The maximum length is
> > + * OVS_PSAMPLE_COOKIE_MAX_SIZE bytes.
> > + *
> > + * Sends the packet to the psample multicast group with the specified 
> > group and
> > + * cookie. It is possible to combine this action with the
> > + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the sample.
> > + */
> > +enum ovs_psample_attr {
> > +   OVS_PSAMPLE_ATTR_GROUP = 1, /* u32 number. */
> > +   OVS_PSAMPLE_ATTR_COOKIE,/* Optional, user specified cookie. */
> > +
> > +   /* private: */
> > +   __OVS_PSAMPLE_ATTR_MAX
> > +};
> > +
> > +#define OVS_PSAMPLE_ATTR_MAX (__OVS_PSAMPLE_ATTR_MAX - 1)
> > +
> >  /**
> >   * enum ovs_action_attr - Action types.
> >   *
> > @@ -966,6 +991,8 @@ struct check_pkt_len_arg {
> >   * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> >   * argument.
> >   * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > + * @OVS_ACTION_ATTR_PSAMPLE: Send a sample of the packet to external 
> > observers
> > + * via psample.
> >   *
> >   * Only a single header can be set with a single %OVS_ACTION_ATTR_SET.  
> > Not all
> >   * fields within a header are modifiable, e.g. the IPv4 protocol and 
> > fragment
> > @@ -1004,6 +1031,7 @@ enum ovs_action_attr {
> > OVS_ACTION_ATTR_ADD_MPLS, /* struct ovs_action_add_mpls. */
> > OVS_ACTION_ATTR_DEC_TTL,  /* Nested OVS_DEC_TTL_ATTR_*. */
> > OVS_ACTION_ATTR_DROP, /* u32 error code. */
> > +   OVS_ACTION_ATTR_PSAMPLE,  /* Nested OVS_PSAMPLE_ATTR_*. */
> >
> > __OVS_ACTION_ATTR_MAX,/* Nothing past this will be accepted
> >* from userspace. */
> > diff --git a/net/openvswitch/Kconfig b/net/openvswitch/Kconfig
> > index 29a7081858cd..2535f3f9f462 100644
> > --- a/net/openvswitch/Kconfig
> > +++ b/net/openvswitch/Kconfig
> > @@ -10,6 +10,7 @@ config OPENVSWITCH
> >(NF_CONNTRACK && ((!NF_DEFRAG_IPV6 || NF_DEFRAG_IPV6) && \
> >  (!NF_NAT || NF_NAT) && \
> >  (!NETFILTER_CONNCOUNT || 
> > NETFILTER_CONNCOUNT)))
> > +   depends on PSAMPLE || !PSAMPLE
> > select LIBCRC32C
> > select MPLS
> > select NET_MPLS_GSO
> > diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions.c
> > index 964225580824..a035

Re: [ovs-dev] [PATCH net-next v7 05/10] net: openvswitch: add psample action

[Adrián Moreno]

On Mon, Jul 01, 2024 at 01:40:39PM GMT, Michal Kubiak wrote:
> On Sun, Jun 30, 2024 at 09:57:26PM +0200, Adrian Moreno wrote:
> > Add support for a new action: psample.
> >
> > This action accepts a u32 group id and a variable-length cookie and uses
> > the psample multicast group to make the packet available for
> > observability.
> >
> > The maximum length of the user-defined cookie is set to 16, same as
> > tc_cookie, to discourage using cookies that will not be offloadable.
> >
> > Acked-by: Eelco Chaudron 
> > Signed-off-by: Adrian Moreno 
> > ---
> >  Documentation/netlink/specs/ovs_flow.yaml | 17 
> >  include/uapi/linux/openvswitch.h  | 28 ++
> >  net/openvswitch/Kconfig   |  1 +
> >  net/openvswitch/actions.c | 47 +++
> >  net/openvswitch/flow_netlink.c| 32 ++-
> >  5 files changed, 124 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/netlink/specs/ovs_flow.yaml 
> > b/Documentation/netlink/specs/ovs_flow.yaml
> > index 4fdfc6b5cae9..46f5d1cd8a5f 100644
> > --- a/Documentation/netlink/specs/ovs_flow.yaml
> > +++ b/Documentation/netlink/specs/ovs_flow.yaml
> > @@ -727,6 +727,12 @@ attribute-sets:
> >  name: dec-ttl
> >  type: nest
> >  nested-attributes: dec-ttl-attrs
> > +  -
> > +name: psample
> > +type: nest
> > +nested-attributes: psample-attrs
> > +doc: |
> > +  Sends a packet sample to psample for external observation.
> >-
> >  name: tunnel-key-attrs
> >  enum-name: ovs-tunnel-key-attr
> > @@ -938,6 +944,17 @@ attribute-sets:
> >-
> >  name: gbp
> >  type: u32
> > +  -
> > +name: psample-attrs
> > +enum-name: ovs-psample-attr
> > +name-prefix: ovs-psample-attr-
> > +attributes:
> > +  -
> > +name: group
> > +type: u32
> > +  -
> > +name: cookie
> > +type: binary
> >
> >  operations:
> >name-prefix: ovs-flow-cmd-
> > diff --git a/include/uapi/linux/openvswitch.h 
> > b/include/uapi/linux/openvswitch.h
> > index efc82c318fa2..3dd653748725 100644
> > --- a/include/uapi/linux/openvswitch.h
> > +++ b/include/uapi/linux/openvswitch.h
> > @@ -914,6 +914,31 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_PSAMPLE_COOKIE_MAX_SIZE 16
>
> In your patch #2 you use "TC_COOKIE_MAX_SIZE" as an array size for your
> cookie. I know that now OVS_PSAMPLE_COOKIE_MAX_SIZE == TC_COOKIE_MAX_SIZE,
> so this size will be validated correctly.
> But how likely is that those 2 constants will have different values in the
> future?
> Would it be reasonable to create more strict dependency between those
> macros, e.g.:
>
> #define OVS_PSAMPLE_COOKIE_MAX_SIZE TC_COOKIE_MAX_SIZE
>
> or, at least, add a comment that the size shouldn't be bigger than
> TC_COOKIE_MAX_SIZE?
> I'm just considering the risk of exceeding the array from the patch #2 when
> somebody increases OVS_PSAMPLE_COOKIE_MAX_SIZE in the future.
>
> Thanks,
> Michal
>

Hi Michal,

Thanks for sharing your thoughts.

I tried to keep the dependency between both cookie sizes loose.

I don't want a change in TC_COOKIE_MAX_SIZE to inadvertently modify
OVS_PSAMPLE_COOKIE_MAX_SIZE because OVS might not need a bigger cookie
and even if it does, backwards compatibility needs to be guaranteed:
meaning OVS userspace will have to detect the new size and use it or
fall back to a smaller cookie for older kernels. All this needs to be
known and worked on in userspace.

On the other hand, I intentionally made OVS's "psample" action as
similar as possible to act_psample, including setting the same cookie
size to begin with. The reason is that I think we should try to implement
tc-flower offloading of this action using act_sample, plus 16 seemed a
very reasonable max value.

When we decide to support offloading the "psample" action, this must
be done entirely in userspace. OVS must create a act_sample action
(instead of the OVS "psample" one) via netlink. In no circumstances the
openvswitch kmod interacts with tc directly.

Now, back to your concern. If OVS_PSAMPLE_COOKIE_MAX_SIZE grows and
TC_COOKIE_MAX_SIZE does not *and* we already support offloading this
action to tc, the only consequence is that OVS userspace has a
problem because the tc's netlink interface will reject cookies larger
than TC_COOKIE_MAX_SIZE [1].
This guarantees that the array in patch #2 is never overflown.

OVS will have to deal with the different sizes and try to squeeze the
data into TC_COOKIE_MAX_SIZE or fail to offload the action altogether.

Psample does not have a size limit so different parts of the kernel can
use psample with different internal max-sizes without any restriction.

I hope this clears your concerns.

[1] https://github.com/torvalds/linux/blob/master/net/sched/act_api.c#L1299

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://m

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Mon, Jul 01, 2024 at 01:30:12PM GMT, Eelco Chaudron wrote:
>
>
> On 28 Jun 2024, at 18:40, Adrián Moreno wrote:
>
> > On Wed, Jun 26, 2024 at 02:11:51PM GMT, Eelco Chaudron wrote:
> >> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
> >>
> >>> Use the newly added emit_sample to implement OpenFlow sample() actions
> >>> with local sampling configuration.
> >>
> >> See some comments below.
> >>
> >> Cheers,
> >>
> >> Eelco
> >>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  ofproto/ofproto-dpif-lsample.c |  17 
> >>>  ofproto/ofproto-dpif-lsample.h |   6 ++
> >>>  ofproto/ofproto-dpif-xlate.c   | 163 -
> >>>  ofproto/ofproto-dpif-xlate.h   |   3 +-
> >>>  ofproto/ofproto-dpif.c |   2 +-
> >>>  5 files changed, 144 insertions(+), 47 deletions(-)
> >>>
> >>> diff --git a/ofproto/ofproto-dpif-lsample.c 
> >>> b/ofproto/ofproto-dpif-lsample.c
> >>> index 7bdafac19..2c0b5da89 100644
> >>> --- a/ofproto/ofproto-dpif-lsample.c
> >>> +++ b/ofproto/ofproto-dpif-lsample.c
> >>> @@ -139,6 +139,23 @@ dpif_lsample_set_options(struct dpif_lsample 
> >>> *lsample,
> >>>  return changed;
> >>>  }
> >>>
> >>> +/* Returns the group_id of the exporter with the given collector_set_id, 
> >>> if it
> >>> + * exists. */
> >>
> >> nit: The below will fit on one line
> >>
> >> /* Returns the exporter group_id for a given collector_set_id, if it 
> >> exists. */
> >>
> >
> > Ack.
> >
> >>> +bool
> >>> +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> >>> collector_set_id,
> >>> +  uint32_t *group_id)
> >>> +{
> >>> +struct lsample_exporter_node *node;
> >>> +bool found = false;
> >>> +
> >>> +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> >>> +if (node) {
> >>> +found = true;
> >>> +*group_id = node->exporter.options.group_id;
> >>> +}
> >>> +return found;
> >>> +}
> >>> +
> >>>  struct dpif_lsample *
> >>>  dpif_lsample_create(void)
> >>>  {
> >>> diff --git a/ofproto/ofproto-dpif-lsample.h 
> >>> b/ofproto/ofproto-dpif-lsample.h
> >>> index c23ea8372..f13425575 100644
> >>> --- a/ofproto/ofproto-dpif-lsample.h
> >>> +++ b/ofproto/ofproto-dpif-lsample.h
> >>> @@ -19,6 +19,7 @@
> >>>
> >>>  #include 
> >>>  #include 
> >>> +#include 
> >>
> >> Maybe add in alphabetical order.
> >>
> >
> > Ack.
> >
> >>>  struct dpif_lsample;
> >>>  struct ofproto_lsample_options;
> >>> @@ -31,4 +32,9 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >>>const struct ofproto_lsample_options *,
> >>>size_t n_opts);
> >>>
> >>> +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> >>> +   uint32_t collector_set_id,
> >>> +   uint32_t *group_id);
> >>> +
> >>>  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> >>> +
> >>
> >> Accedantely added a newline?
> >>
> >
> > Ack.
> >
> >>> diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> >>> index 7c4950895..5bd215d31 100644
> >>> --- a/ofproto/ofproto-dpif-xlate.c
> >>> +++ b/ofproto/ofproto-dpif-xlate.c
> >>> @@ -47,6 +47,7 @@
> >>>  #include "ofproto/ofproto-dpif-ipfix.h"
> >>>  #include "ofproto/ofproto-dpif-mirror.h"
> >>>  #include "ofproto/ofproto-dpif-monitor.h"
> >>> +#include "ofproto/ofproto-dpif-lsample.h"
> >>>  #include "ofproto/ofproto-dpif-sflow.h"
> >>>  #include "ofproto/ofproto-dpif-trace.h"
> >>>  #include "ofproto/ofproto-dpif-xlate-cache.h"
> >>> @@ -117,6 +118,7 @@ struct xbridge {
> >>>  struct dpif_sflow *sflow; /* SFlow handle, or null. */
> >>>  struct dpif_ipfix *ipfix; /* Ipfix handle, or null.

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jul 01, 2024 at 01:33:23PM GMT, Eelco Chaudron wrote:
>
>
> On 28 Jun 2024, at 16:11, Adrián Moreno wrote:
>
> > On Wed, Jun 26, 2024 at 02:15:02PM GMT, Eelco Chaudron wrote:
> >> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
> >>
> >>> Test simultaneous IPFIX and local sampling including slow-path.
> >>
> >> I guess Ilya's comments on this patch summarize most of my comments. I had
> >> one small additional question. See below.
> >>
> >> //Eelco
> >>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  tests/system-common-macros.at |   4 ++
> >>>  tests/system-traffic.at   | 105 ++
> >>>  2 files changed, 109 insertions(+)
> >>>
> >>> diff --git a/tests/system-common-macros.at b/tests/system-common-macros.at
> >>> index 2a68cd664..22b8885e4 100644
> >>> --- a/tests/system-common-macros.at
> >>> +++ b/tests/system-common-macros.at
> >>> @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >>>  # OVS_CHECK_DROP_ACTION()
> >>>  m4_define([OVS_CHECK_DROP_ACTION],
> >>>  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> >>> ovs-vswitchd.log])])
> >>> +
> >>> +# OVS_CHECK_EMIT_SAMPLE()
> >>> +m4_define([OVS_CHECK_EMIT_SAMPLE],
> >>> +[AT_SKIP_IF([! grep -q "Datapath supports emit_sample" 
> >>> ovs-vswitchd.log])])
> >>> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> >>> index bd7647cbe..babc56b56 100644
> >>> --- a/tests/system-traffic.at
> >>> +++ b/tests/system-traffic.at
> >>> @@ -8977,3 +8977,108 @@ OVS_WAIT_UNTIL([cat p2.pcap | grep -E "0x0050: 
> >>> * * *5002 *2000 *b85e *00
> >>>
> >>>  OVS_TRAFFIC_VSWITCHD_STOP
> >>>  AT_CLEANUP
> >>> +
> >>> +AT_SETUP([emit_sample])
> >>> +OVS_TRAFFIC_VSWITCHD_START()
> >>> +OVS_CHECK_EMIT_SAMPLE()
> >>> +
> >>> +ADD_NAMESPACES(at_ns0, at_ns1)
> >>> +NS_CHECK_EXEC([at_ns0], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], 
> >>> [0], [ignore])
> >>> +NS_CHECK_EXEC([at_ns1], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], 
> >>> [0], [ignore])
> >>> +
> >>> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24", "e4:11:22:33:44:55")
> >>> +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24", "e4:11:22:33:44:66")
> >>> +
> >>> +NS_CHECK_EXEC([at_ns0], [arp -s 10.1.1.2 e4:11:22:33:44:66])
> >>> +NS_CHECK_EXEC([at_ns1], [arp -s 10.1.1.1 e4:11:22:33:44:55])
> >>> +
> >>> +
> >>> +AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg 
> >>> ofproto_dpif_upcall:dbg])
> >>> +
> >>> +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> >>> +-- --id=@ipfix create IPFIX 
> >>> targets=\"127.0.0.1:4739\" \
> >>> +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> >>> ipfix=@ipfix, local_sample_group=10 \
> >>> +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> >>> ipfix=@ipfix, local_sample_group=12],
> >>> + [0], [ignore])
> >>> +
> >>> +AT_DATA([flows.txt], [dnl
> >>> +in_port=ovs-p0,ip 
> >>> actions=sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918),output(port=ovs-p1,max_len=100)
> >>> +in_port=ovs-p1,ip 
> >>> actions=sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377),output(port=ovs-p0,max_len=100)
> >>> +])
> >>> +
> >>> +AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> >>> +
> >>> +OVS_DAEMONIZE([ovstest test-psample > psample.out], [psample1.pid])
> >>> +
> >>> +NS_CHECK_EXEC([at_ns0], [ping -q -c 1 10.1.1.2 | FORMAT_PING], [0], [dnl
> >>> +1 packets transmitted, 1 received, 0% packet loss, time 0ms
> >>> +])
> >>> +
> >>> +m4_define([SAMPLE1], [group_id=0xa 
> >>> obs_domain=0x,obs_point=0x 
> >>> .*icmp.*nw_src=10.1.1.1,nw_dst=10.1.1.2])
> >>> +m4_define([SAMPLE2], [group_id=0xc 
> >>> obs_domain=0x,obs_point=0x 
> >>> .*icmp.*nw_src=10.1.1.2,nw_dst=10.1.1.1])
> &

Re: [ovs-dev] [PATCH net-next v6 00/10] net: openvswitch: Add sample multicasting.

[Adrián Moreno]

On Fri, Jun 28, 2024 at 01:05:36PM GMT, Adrian Moreno wrote:
> ** Background **
> Currently, OVS supports several packet sampling mechanisms (sFlow,
> per-bridge IPFIX, per-flow IPFIX). These end up being translated into a
> userspace action that needs to be handled by ovs-vswitchd's handler
> threads only to be forwarded to some third party application that
> will somehow process the sample and provide observability on the
> datapath.
>
> A particularly interesting use-case is controller-driven
> per-flow IPFIX sampling where the OpenFlow controller can add metadata
> to samples (via two 32bit integers) and this metadata is then available
> to the sample-collecting system for correlation.
>
> ** Problem **
> The fact that sampled traffic share netlink sockets and handler thread
> time with upcalls, apart from being a performance bottleneck in the
> sample extraction itself, can severely compromise the datapath,
> yielding this solution unfit for highly loaded production systems.
>
> Users are left with little options other than guessing what sampling
> rate will be OK for their traffic pattern and system load and dealing
> with the lost accuracy.
>
> Looking at available infrastructure, an obvious candidated would be
> to use psample. However, it's current state does not help with the
> use-case at stake because sampled packets do not contain user-defined
> metadata.
>
> ** Proposal **
> This series is an attempt to fix this situation by extending the
> existing psample infrastructure to carry a variable length
> user-defined cookie.
>
> The main existing user of psample is tc's act_sample. It is also
> extended to forward the action's cookie to psample.
>
> Finally, a new OVS action (OVS_SAMPLE_ATTR_EMIT_SAMPLE) is created.
> It accepts a group and an optional cookie and uses psample to
> multicast the packet and the metadata.
>
> --
> v5 -> v6:
> - Renamed emit_sample -> psample
> - Addressed unused variable and conditionally compilation of function.
>
> v4 -> v5:
> - Rebased.
> - Removed lefover enum value and wrapped some long lines in selftests.
>
> v3 -> v4:
> - Rebased.
> - Addressed Jakub's comment on private and unused nla attributes.
>
> v2 -> v3:
> - Addressed comments from Simon, Aaron and Ilya.
> - Dropped probability propagation in nested sample actions.
> - Dropped patch v2's 7/9 in favor of a userspace implementation and
> consume skb if emit_sample is the last action, same as we do with
> userspace.
> - Split ovs-dpctl.py features in independent patches.
>
> v1 -> v2:
> - Create a new action ("emit_sample") rather than reuse existing
>   "sample" one.
> - Add probability semantics to psample's sampling rate.
> - Store sampling probability in skb's cb area and use it in emit_sample.
> - Test combining "emit_sample" with "trunc"
> - Drop group_id filtering and tracepoint in psample.
>
> rfc_v2 -> v1:
> - Accommodate Ilya's comments.
> - Split OVS's attribute in two attributes and simplify internal
> handling of psample arguments.
> - Extend psample and tc with a user-defined cookie.
> - Add a tracepoint to psample to facilitate troubleshooting.
>
> rfc_v1 -> rfc_v2:
> - Use psample instead of a new OVS-only multicast group.
> - Extend psample and tc with a user-defined cookie.
>
>
> Adrian Moreno (10):
>   net: psample: add user cookie
>   net: sched: act_sample: add action cookie to sample
>   net: psample: skip packet copy if no listeners
>   net: psample: allow using rate as probability
>   net: openvswitch: add psample action
>   net: openvswitch: store sampling probability in cb.
>   selftests: openvswitch: add psample action
>   selftests: openvswitch: add userspace parsing
>   selftests: openvswitch: parse trunc action
>   selftests: openvswitch: add psample test
>
>  Documentation/netlink/specs/ovs_flow.yaml |  17 ++
>  include/net/psample.h |   5 +-
>  include/uapi/linux/openvswitch.h  |  31 +-
>  include/uapi/linux/psample.h  |  11 +-
>  net/openvswitch/Kconfig   |   1 +
>  net/openvswitch/actions.c |  65 -
>  net/openvswitch/datapath.h|   3 +
>  net/openvswitch/flow_netlink.c|  32 ++-
>  net/openvswitch/vport.c   |   1 +
>  net/psample/psample.c |  16 +-
>  net/sched/act_sample.c|  12 +
>  .../selftests/net/openvswitch/openvswitch.sh  | 115 +++-
>  .../selftests/net/openvswitch/ovs-dpctl.py| 272 +-
>  13 files changed, 565 insertions(+), 16 deletions(-)
>
> --
> 2.45.2
>

Patchwork says this patch is not applying on net-next. I'll wait for
some reviews and rebase+resubmit it later tonight or tomorrow.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:11:51PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Use the newly added emit_sample to implement OpenFlow sample() actions
> > with local sampling configuration.
>
> See some comments below.
>
> Cheers,
>
> Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-lsample.c |  17 
> >  ofproto/ofproto-dpif-lsample.h |   6 ++
> >  ofproto/ofproto-dpif-xlate.c   | 163 -
> >  ofproto/ofproto-dpif-xlate.h   |   3 +-
> >  ofproto/ofproto-dpif.c |   2 +-
> >  5 files changed, 144 insertions(+), 47 deletions(-)
> >
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index 7bdafac19..2c0b5da89 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -139,6 +139,23 @@ dpif_lsample_set_options(struct dpif_lsample *lsample,
> >  return changed;
> >  }
> >
> > +/* Returns the group_id of the exporter with the given collector_set_id, 
> > if it
> > + * exists. */
>
> nit: The below will fit on one line
>
> /* Returns the exporter group_id for a given collector_set_id, if it exists. 
> */
>

Ack.

> > +bool
> > +dpif_lsample_get_group_id(struct dpif_lsample *ps, uint32_t 
> > collector_set_id,
> > +  uint32_t *group_id)
> > +{
> > +struct lsample_exporter_node *node;
> > +bool found = false;
> > +
> > +node = dpif_lsample_find_exporter_node(ps, collector_set_id);
> > +if (node) {
> > +found = true;
> > +*group_id = node->exporter.options.group_id;
> > +}
> > +return found;
> > +}
> > +
> >  struct dpif_lsample *
> >  dpif_lsample_create(void)
> >  {
> > diff --git a/ofproto/ofproto-dpif-lsample.h b/ofproto/ofproto-dpif-lsample.h
> > index c23ea8372..f13425575 100644
> > --- a/ofproto/ofproto-dpif-lsample.h
> > +++ b/ofproto/ofproto-dpif-lsample.h
> > @@ -19,6 +19,7 @@
> >
> >  #include 
> >  #include 
> > +#include 
>
> Maybe add in alphabetical order.
>

Ack.

> >  struct dpif_lsample;
> >  struct ofproto_lsample_options;
> > @@ -31,4 +32,9 @@ bool dpif_lsample_set_options(struct dpif_lsample *,
> >const struct ofproto_lsample_options *,
> >size_t n_opts);
> >
> > +bool dpif_lsample_get_group_id(struct dpif_lsample *,
> > +   uint32_t collector_set_id,
> > +   uint32_t *group_id);
> > +
> >  #endif /* OFPROTO_DPIF_LSAMPLE_H */
> > +
>
> Accedantely added a newline?
>

Ack.

> > diff --git a/ofproto/ofproto-dpif-xlate.c b/ofproto/ofproto-dpif-xlate.c
> > index 7c4950895..5bd215d31 100644
> > --- a/ofproto/ofproto-dpif-xlate.c
> > +++ b/ofproto/ofproto-dpif-xlate.c
> > @@ -47,6 +47,7 @@
> >  #include "ofproto/ofproto-dpif-ipfix.h"
> >  #include "ofproto/ofproto-dpif-mirror.h"
> >  #include "ofproto/ofproto-dpif-monitor.h"
> > +#include "ofproto/ofproto-dpif-lsample.h"
> >  #include "ofproto/ofproto-dpif-sflow.h"
> >  #include "ofproto/ofproto-dpif-trace.h"
> >  #include "ofproto/ofproto-dpif-xlate-cache.h"
> > @@ -117,6 +118,7 @@ struct xbridge {
> >  struct dpif_sflow *sflow; /* SFlow handle, or null. */
> >  struct dpif_ipfix *ipfix; /* Ipfix handle, or null. */
> >  struct netflow *netflow;  /* Netflow handle, or null. */
> > +struct dpif_lsample *lsample; /* Local sample handle, or null. */
> >  struct stp *stp;  /* STP or null if disabled. */
> >  struct rstp *rstp;/* RSTP or null if disabled. */
> >
> > @@ -687,6 +689,7 @@ static void xlate_xbridge_set(struct xbridge *, struct 
> > dpif *,
> >const struct dpif_sflow *,
> >const struct dpif_ipfix *,
> >const struct netflow *,
> > +  const struct dpif_lsample *,
> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *,
> >const struct xbridge_addr *);
> > @@ -1070,6 +1073,7 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >const struct dpif_sflow *sflow,
> >const struct dpif_ipfix *ipfix,
> >const struct netflow *netflow,
> > +  const struct dpif_lsample *lsample,
>
> nit: I would move above netflow, as you also do the actual init/unref before
>  netflow.

Ack.

> >bool forward_bpdu, bool has_in_band,
> >const struct dpif_backer_support *support,
> >const struct xbridge_addr *addr)
> > @@ -1099,6 +1103,11 @@ xlate_xbridge_set(struct xbridge *xbridge,
> >  xbridge->ipfix = dpif_ipfix_ref(ipfix);
> >  }
> >
> > +if (xbridge->lsample != lsample) {
> > +dpif_lsample_unref(xbridge->lsample);
> > +xbridge->lsample = dpif_lsample_re

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:15:02PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Test simultaneous IPFIX and local sampling including slow-path.
>
> I guess Ilya's comments on this patch summarize most of my comments. I had
> one small additional question. See below.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  tests/system-common-macros.at |   4 ++
> >  tests/system-traffic.at   | 105 ++
> >  2 files changed, 109 insertions(+)
> >
> > diff --git a/tests/system-common-macros.at b/tests/system-common-macros.at
> > index 2a68cd664..22b8885e4 100644
> > --- a/tests/system-common-macros.at
> > +++ b/tests/system-common-macros.at
> > @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >  # OVS_CHECK_DROP_ACTION()
> >  m4_define([OVS_CHECK_DROP_ACTION],
> >  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> > ovs-vswitchd.log])])
> > +
> > +# OVS_CHECK_EMIT_SAMPLE()
> > +m4_define([OVS_CHECK_EMIT_SAMPLE],
> > +[AT_SKIP_IF([! grep -q "Datapath supports emit_sample" 
> > ovs-vswitchd.log])])
> > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > index bd7647cbe..babc56b56 100644
> > --- a/tests/system-traffic.at
> > +++ b/tests/system-traffic.at
> > @@ -8977,3 +8977,108 @@ OVS_WAIT_UNTIL([cat p2.pcap | grep -E "0x0050: 
> > * * *5002 *2000 *b85e *00
> >
> >  OVS_TRAFFIC_VSWITCHD_STOP
> >  AT_CLEANUP
> > +
> > +AT_SETUP([emit_sample])
> > +OVS_TRAFFIC_VSWITCHD_START()
> > +OVS_CHECK_EMIT_SAMPLE()
> > +
> > +ADD_NAMESPACES(at_ns0, at_ns1)
> > +NS_CHECK_EXEC([at_ns0], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], [0], 
> > [ignore])
> > +NS_CHECK_EXEC([at_ns1], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], [0], 
> > [ignore])
> > +
> > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24", "e4:11:22:33:44:55")
> > +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24", "e4:11:22:33:44:66")
> > +
> > +NS_CHECK_EXEC([at_ns0], [arp -s 10.1.1.2 e4:11:22:33:44:66])
> > +NS_CHECK_EXEC([at_ns1], [arp -s 10.1.1.1 e4:11:22:33:44:55])
> > +
> > +
> > +AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg 
> > ofproto_dpif_upcall:dbg])
> > +
> > +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> > +-- --id=@ipfix create IPFIX targets=\"127.0.0.1:4739\" 
> > \
> > +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> > ipfix=@ipfix, local_sample_group=10 \
> > +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> > ipfix=@ipfix, local_sample_group=12],
> > + [0], [ignore])
> > +
> > +AT_DATA([flows.txt], [dnl
> > +in_port=ovs-p0,ip 
> > actions=sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918),output(port=ovs-p1,max_len=100)
> > +in_port=ovs-p1,ip 
> > actions=sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377),output(port=ovs-p0,max_len=100)
> > +])
> > +
> > +AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> > +
> > +OVS_DAEMONIZE([ovstest test-psample > psample.out], [psample1.pid])
> > +
> > +NS_CHECK_EXEC([at_ns0], [ping -q -c 1 10.1.1.2 | FORMAT_PING], [0], [dnl
> > +1 packets transmitted, 1 received, 0% packet loss, time 0ms
> > +])
> > +
> > +m4_define([SAMPLE1], [group_id=0xa 
> > obs_domain=0x,obs_point=0x 
> > .*icmp.*nw_src=10.1.1.1,nw_dst=10.1.1.2])
> > +m4_define([SAMPLE2], [group_id=0xc 
> > obs_domain=0x,obs_point=0x 
> > .*icmp.*nw_src=10.1.1.2,nw_dst=10.1.1.1])
> > +AT_CHECK([grep -E 'SAMPLE1' psample.out >/dev/null])
> > +AT_CHECK([grep -E 'SAMPLE2' psample.out >/dev/null])
> > +
> > +AT_CHECK([ovs-ofctl dump-ipfix-flow br0 | sed 's/tx pkts=[[0-9]]*/tx 
> > pkts=24/' | sed 's/tx errs=[[0-9]]*/tx errs=0/'], [0], [dnl
>
> Why are we making tx pkts always 24, and are we waiving any potential tx 
> errors?
> Is this something you have seen not being consistent, if so any idea why?
>

Yes. By the time we request the statistics, IPFIX cache should contain
the flows but they might not have been sent yet. When they are sent they
will fail (because IPFIX target is localhost and it returns the socket
returns an error if the listening socket does not exist) so the number
of errors might vary.

I can look deeper into making it more consistent but it looked like a
quick workaround.

> > +NXST_IPFIX_FLOW reply (xid=0x2): 2 ids
> > +  id   1: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, 
> > tx pkts=24
> > +  pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
> > +  id   2: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, 
> > tx pkts=24
> > +  pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
> > +])
> > +
> > +AT_CHECK([ovs-appctl lsample/show br0], [0], [dnl
> > +local sample statistics for bridge "br0":
> > +- Collector Set ID: 1
> > +  Local Sample Group: 10
> > +  Total number of bytes: 98
> > +  Total number of packets: 1
> > +
> > +- Collector Set ID: 2
> > +  Local S

Re: [ovs-dev] [RFC v2 8/9] tests: Add test-psample testing utility.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:14:27PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > This simple program reads from psample and prints the packets to stdout.
>
> Some comments below.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  include/linux/automake.mk |   1 +
> >  include/linux/psample.h   |  68 +
> >  tests/automake.mk |   3 +-
> >  tests/test-psample.c  | 282 ++
> >  4 files changed, 353 insertions(+), 1 deletion(-)
> >  create mode 100644 include/linux/psample.h
> >  create mode 100644 tests/test-psample.c
> >
> > diff --git a/include/linux/automake.mk b/include/linux/automake.mk
> > index cdae5eedc..ac306b53c 100644
> > --- a/include/linux/automake.mk
> > +++ b/include/linux/automake.mk
> > @@ -3,6 +3,7 @@ noinst_HEADERS += \
> > include/linux/netfilter/nf_conntrack_sctp.h \
> > include/linux/openvswitch.h \
> > include/linux/pkt_cls.h \
> > +   include/linux/psample.h \
> > include/linux/gen_stats.h \
> > include/linux/tc_act/tc_mpls.h \
> > include/linux/tc_act/tc_pedit.h \
> > diff --git a/include/linux/psample.h b/include/linux/psample.h
> > new file mode 100644
> > index 0..d5761b730
> > --- /dev/null
> > +++ b/include/linux/psample.h
> > @@ -0,0 +1,68 @@
> > +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */
> > +#ifndef __LINUX_PSAMPLE_H
> > +#define __LINUX_PSAMPLE_H
> > +
> > +enum {
> > +   PSAMPLE_ATTR_IIFINDEX,
> > +   PSAMPLE_ATTR_OIFINDEX,
> > +   PSAMPLE_ATTR_ORIGSIZE,
> > +   PSAMPLE_ATTR_SAMPLE_GROUP,
> > +   PSAMPLE_ATTR_GROUP_SEQ,
> > +   PSAMPLE_ATTR_SAMPLE_RATE,
> > +   PSAMPLE_ATTR_DATA,
> > +   PSAMPLE_ATTR_GROUP_REFCOUNT,
> > +   PSAMPLE_ATTR_TUNNEL,
> > +
> > +   PSAMPLE_ATTR_PAD,
> > +   PSAMPLE_ATTR_OUT_TC,/* u16 */
> > +   PSAMPLE_ATTR_OUT_TC_OCC,/* u64, bytes */
> > +   PSAMPLE_ATTR_LATENCY,   /* u64, nanoseconds */
> > +   PSAMPLE_ATTR_TIMESTAMP, /* u64, nanoseconds */
> > +   PSAMPLE_ATTR_PROTO, /* u16 */
> > +   PSAMPLE_ATTR_USER_COOKIE,   /* binary, user provided data */
> > +   PSAMPLE_ATTR_SAMPLE_PROBABILITY,/* no argument, interpret rate in
> > +* PSAMPLE_ATTR_SAMPLE_RATE as a
> > +* probability scaled 0 - U32_MAX.
> > +*/
> > +
> > +   __PSAMPLE_ATTR_MAX
> > +};
> > +
> > +enum psample_command {
> > +   PSAMPLE_CMD_SAMPLE,
> > +   PSAMPLE_CMD_GET_GROUP,
> > +   PSAMPLE_CMD_NEW_GROUP,
> > +   PSAMPLE_CMD_DEL_GROUP,
> > +   PSAMPLE_CMD_SAMPLE_FILTER_SET,
> > +};
> > +
> > +enum psample_tunnel_key_attr {
> > +   PSAMPLE_TUNNEL_KEY_ATTR_ID, /* be64 Tunnel ID */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_IPV4_SRC,   /* be32 src IP address. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_IPV4_DST,   /* be32 dst IP address. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_TOS,/* u8 Tunnel IP ToS. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_TTL,/* u8 Tunnel IP TTL. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_DONT_FRAGMENT,  /* No argument, set DF. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_CSUM,   /* No argument. CSUM 
> > packet. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_OAM,/* No argument. OAM frame.  
> > */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_GENEVE_OPTS,/* Array of Geneve options. 
> > */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_TP_SRC, /* be16 src Transport Port. 
> > */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_TP_DST, /* be16 dst Transport Port. 
> > */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_VXLAN_OPTS, /* Nested VXLAN opts* */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_IPV6_SRC,   /* struct in6_addr src IPv6 
> > address. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_IPV6_DST,   /* struct in6_addr dst IPv6 
> > address. */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_PAD,
> > +   PSAMPLE_TUNNEL_KEY_ATTR_ERSPAN_OPTS,/* struct erspan_metadata */
> > +   PSAMPLE_TUNNEL_KEY_ATTR_IPV4_INFO_BRIDGE,   /* No argument. 
> > IPV4_INFO_BRIDGE mode.*/
> > +   __PSAMPLE_TUNNEL_KEY_ATTR_MAX
> > +};
> > +
> > +/* Can be overridden at runtime by module option */
> > +#define PSAMPLE_ATTR_MAX (__PSAMPLE_ATTR_MAX - 1)
> > +
> > +#define PSAMPLE_NL_MCGRP_CONFIG_NAME "config"
> > +#define PSAMPLE_NL_MCGRP_SAMPLE_NAME "packets"
> > +#define PSAMPLE_GENL_NAME "psample"
> > +#define PSAMPLE_GENL_VERSION 1
> > +
> > +#endif
> > diff --git a/tests/automake.mk b/tests/automake.mk
> > index 04f48f2d8..edfc2cb33 100644
> > --- a/tests/automake.mk
> > +++ b/tests/automake.mk
> > @@ -499,7 +499,8 @@ endif
> >  if LINUX
> >  tests_ovstest_SOURCES += \
> > tests/test-netlink-conntrack.c \
> > -   tests/test-netlink-policy.c
> > +   tests/test-netlink-policy.c \
> > +   tests/test-psample.c
> >  endif
> >
> >  tests_ovstest_LDADD = lib/libopenvswitch.la
> > diff --git a/tests/test-psample.c b/tests/test-psample.c
> > new file mode 100644
> > index 0..f04d903

Re: [ovs-dev] [RFC v2 7/9] ofproto-dpif-lsample: Show stats via unixctl.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:13:40PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Add a command to dump statistics per exporter.
>
> Some comments below.
>
> //Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  NEWS   |   2 +
> >  ofproto/ofproto-dpif-lsample.c | 113 +
> >  ofproto/ofproto-dpif-lsample.h |   1 +
> >  ofproto/ofproto-dpif.c |   1 +
> >  4 files changed, 117 insertions(+)
> >
> > diff --git a/NEWS b/NEWS
> > index 1c05a7120..a443f9fe1 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -11,6 +11,8 @@ Post-v3.3.0
> >   allows samples to be emitted locally (instead of via IPFIX) in a
> >   datapath-specific manner via the new datapath action called 
> > "emit_sample".
> >   Linux kernel datapath is the first to support this feature.
> > + A new unixctl command 'lsample/show' shows packet and bytes statistics
> > + per local sample exporter.
> >
> >
> >  v3.3.0 - 16 Feb 2024
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > index 0c71e354d..e31dabac4 100644
> > --- a/ofproto/ofproto-dpif-lsample.c
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -21,7 +21,10 @@
> >  #include "dpif.h"
> >  #include "hash.h"
> >  #include "ofproto.h"
> > +#include "ofproto-dpif.h"
> > +#include "openvswitch/dynamic-string.h"
> >  #include "openvswitch/thread.h"
> > +#include "unixctl.h"
> >
> >  /* Dpif local sampling.
> >   *
> > @@ -218,3 +221,113 @@ dpif_lsample_unref(struct dpif_lsample *lsample)
> >  dpif_lsample_destroy(lsample);
> >  }
> >  }
> > +
> > +static int
> > +compare_exporter_list(const void *a_, const void *b_)
>
> We are not comparing a list here, so we should choose a more appropriate name,
> maybe something like;  compare_exporter_collector_set_id(), or if you want it
> shorter, maybe comp_exporter_collector_id(), use you fantasy (but not to
> much ;).
>
> > +{
> > +const struct lsample_exporter_node *a, *b;
> > +
> > +a = *(struct lsample_exporter_node **)a_;
> > +b = *(struct lsample_exporter_node **)b_;
>
> Fix space around casting.
>

Ack.

> > +
> > +if (a->exporter.options.collector_set_id >
> > +b->exporter.options.collector_set_id) {
> > +return 1;
> > +}
> > +if (a->exporter.options.collector_set_id <
> > +b->exporter.options.collector_set_id) {
> > +return -1;
> > +}
> > +return 0;
> > +}
> > +
> > +static void
> > +lsample_exporter_list(struct dpif_lsample *lsample,
> > +  struct lsample_exporter_node ***list,
> > +  size_t *num_exporters)
> > +{
> > +struct lsample_exporter_node *node;
> > +struct lsample_exporter_node **exporter_list;
>
> Reverse Christmas tree order.
>

Ack.


> > +size_t k = 0, n;
> > +
> > +n = cmap_count(&lsample->exporters);
> > +
> > +exporter_list = xcalloc(n, sizeof *exporter_list);
> > +
> > +CMAP_FOR_EACH (node, node, &lsample->exporters) {
> > +if (k >= n) {
> > +break;
> > +}
> > +exporter_list[k++] = node;
> > +}
> > +
> > +qsort(exporter_list, k, sizeof *exporter_list, compare_exporter_list);
> > +
> > +*list = exporter_list;
> > +*num_exporters = k;
> > +}
> > +
> > +static void
> > +lsample_unixctl_show(struct unixctl_conn *conn, int argc OVS_UNUSED,
> > + const char *argv[] OVS_UNUSED, void *aux OVS_UNUSED)
> > +{
> > +struct lsample_exporter_node **node_list = NULL;
> > +struct lsample_exporter_node *node;
> > +struct ds ds = DS_EMPTY_INITIALIZER;
>
> Reverse Christmas tree? Or even better move the *node definition inside the
> for loop below.
>

Ack.

> > +const struct ofproto_dpif *ofproto;
> > +size_t i, num;
> > +
> > +ofproto = ofproto_dpif_lookup_by_name(argv[1]);
> > +if (!ofproto) {
> > +unixctl_command_reply_error(conn, "no such bridge");
> > +return;
> > +}
> > +
> > +if (!ofproto->lsample) {
> > +unixctl_command_reply_error(conn, "no local sampling exporters "
> > +"configured");
>
> unixctl_command_reply_error(conn,
> "no local sampling exporters configured");
>
>
> > +return;
> > +}
> > +
> > +ds_put_format(&ds, "local sample statistics for bridge \"%s\":\n",
>
> Maybe capital L for Local?
>

Ack.

> > + argv[1]);
> > +
> > +lsample_exporter_list(ofproto->lsample, &node_list, &num);
> > +
> > +for (i = 0; i < num; i++) {
> > +uint64_t n_bytes;
> > +uint64_t n_packets;
>
> Reverse Christmas tree
>

Ack.

> > +
> > +node = node_list[i];
> > +
> > +atomic_read_relaxed(&node->exporter.n_packets, &n_packets);
> > +atomic_read_relaxed(&node->exporter.n_bytes, &n_bytes);
> > +
> > +if (i) {
> > +ds_put_cstr(&ds, "\n");
> > +}
> > +
> > +  

Re: [ovs-dev] [RFC v2 4/9] vswitchd: Add local sampling to vswitchd schema.

[Adrián Moreno]

On Thu, Jun 27, 2024 at 03:57:03PM GMT, Eelco Chaudron wrote:
>
>
> On 27 Jun 2024, at 13:37, Adrián Moreno wrote:
>
> > On Wed, Jun 26, 2024 at 02:11:00PM GMT, Eelco Chaudron wrote:
> >> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
> >>
> >>> Add as new column in the Flow_Sample_Collector_Set table named
> >>> "local_sample_group" which enables this feature.
> >>>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  NEWS   |  4 ++
> >>>  vswitchd/bridge.c  | 78 +++---
> >>>  vswitchd/vswitch.ovsschema |  9 -
> >>>  vswitchd/vswitch.xml   | 39 +--
> >>>  4 files changed, 119 insertions(+), 11 deletions(-)
> >>>
> >>> diff --git a/NEWS b/NEWS
> >>> index b92cec532..1c05a7120 100644
> >>> --- a/NEWS
> >>> +++ b/NEWS
> >>> @@ -7,6 +7,10 @@ Post-v3.3.0
> >>> - The primary development branch has been renamed from 'master' to 
> >>> 'main'.
> >>>   The OVS tree remains hosted on GitHub.
> >>>https://github.com/openvswitch/ovs.git
> >>> +   - Local sampling is introduced. It reuses the OpenFlow sample action 
> >>> and
> >>> + allows samples to be emitted locally (instead of via IPFIX) in a
> >>> + datapath-specific manner via the new datapath action called 
> >>> "emit_sample".
> >>> + Linux kernel datapath is the first to support this feature.
> >>>
> >>>
> >>>  v3.3.0 - 16 Feb 2024
> >>> diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c
> >>> index 95a65fcdc..cd7dc6646 100644
> >>> --- a/vswitchd/bridge.c
> >>> +++ b/vswitchd/bridge.c
> >>> @@ -288,6 +288,7 @@ static void bridge_configure_mac_table(struct bridge 
> >>> *);
> >>>  static void bridge_configure_mcast_snooping(struct bridge *);
> >>>  static void bridge_configure_sflow(struct bridge *, int 
> >>> *sflow_bridge_number);
> >>>  static void bridge_configure_ipfix(struct bridge *);
> >>> +static void bridge_configure_lsample(struct bridge *);
> >>>  static void bridge_configure_spanning_tree(struct bridge *);
> >>>  static void bridge_configure_tables(struct bridge *);
> >>>  static void bridge_configure_dp_desc(struct bridge *);
> >>> @@ -989,6 +990,7 @@ bridge_reconfigure(const struct ovsrec_open_vswitch 
> >>> *ovs_cfg)
> >>>  bridge_configure_netflow(br);
> >>>  bridge_configure_sflow(br, &sflow_bridge_number);
> >>>  bridge_configure_ipfix(br);
> >>> +bridge_configure_lsample(br);
> >>>  bridge_configure_spanning_tree(br);
> >>>  bridge_configure_tables(br);
> >>>  bridge_configure_dp_desc(br);
> >>> @@ -1537,10 +1539,11 @@ ovsrec_ipfix_is_valid(const struct ovsrec_ipfix 
> >>> *ipfix)
> >>>  return ipfix && ipfix->n_targets > 0;
> >>>  }
> >>>
> >>> -/* Returns whether a Flow_Sample_Collector_Set row is valid. */
> >>> +/* Returns whether a Flow_Sample_Collector_Set row constains valid IPFIX
> >>
> >> constains -> 'contains a'
> >>
> >
> > Ack.
> >
> >>> + * configuration. */
> >>>  static bool
> >>> -ovsrec_fscs_is_valid(const struct ovsrec_flow_sample_collector_set *fscs,
> >>> - const struct bridge *br)
> >>> +ovsrec_fscs_is_valid_ipfix(const struct ovsrec_flow_sample_collector_set 
> >>> *fscs,
> >>> +   const struct bridge *br)
> >>>  {
> >>>  return ovsrec_ipfix_is_valid(fscs->ipfix) && fscs->bridge == br->cfg;
> >>>  }
> >>> @@ -1558,7 +1561,7 @@ bridge_configure_ipfix(struct bridge *br)
> >>>  const char *virtual_obs_id;
> >>>
> >>>  OVSREC_FLOW_SAMPLE_COLLECTOR_SET_FOR_EACH(fe_cfg, idl) {
> >>> -if (ovsrec_fscs_is_valid(fe_cfg, br)) {
> >>> +if (ovsrec_fscs_is_valid_ipfix(fe_cfg, br)) {
> >>>  n_fe_opts++;
> >>>  }
> >>>  }
> >>> @@ -1621,7 +1624,7 @@ bridge_configure_ipfix(struct bridge *br)
> >>>  fe_opts = xcalloc(n_fe_opts, sizeof *fe_opts);

Re: [ovs-dev] [RFC v2 1/9] odp-util: Add support OVS_ACTION_ATTR_EMIT_SAMPLE.

[Adrián Moreno]

On Thu, Jun 27, 2024 at 03:42:38PM GMT, Eelco Chaudron wrote:
>
>
> On 27 Jun 2024, at 12:45, Adrián Moreno wrote:
>
> > On Wed, Jun 26, 2024 at 02:08:27PM GMT, Eelco Chaudron wrote:
> >> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
> >>
> >>> Add support for parsing and formatting the new action.
> >>>
> >>> Also, flag OVS_ACTION_ATTR_SAMPLE as requiring datapath assistance if it
> >>> contains a nested OVS_ACTION_ATTR_EMIT_SAMPLE. The reason is that the
> >>> sampling rate form the parent "sample" is made available to the nested
> >>> "emit_sample" by the kernel.
> >>
> >> Hi Adrian,
> >>
> >> Thanks for this series! This email kicks off my review of the series,
> >> see comments below and in the other patches.
> >>
> >> Cheers,
> >>
> >> Eelco
> >>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  include/linux/openvswitch.h  |  25 +
> >>>  lib/dpif-netdev.c|   1 +
> >>>  lib/dpif.c   |   1 +
> >>>  lib/odp-execute.c|  25 -
> >>>  lib/odp-util.c   | 103 +++
> >>>  lib/odp-util.h   |   3 +
> >>>  ofproto/ofproto-dpif-ipfix.c |   1 +
> >>>  ofproto/ofproto-dpif-sflow.c |   1 +
> >>>  python/ovs/flow/odp.py   |   8 +++
> >>>  tests/odp.at |  16 ++
> >>>  10 files changed, 183 insertions(+), 1 deletion(-)
> >>>
> >>> diff --git a/include/linux/openvswitch.h b/include/linux/openvswitch.h
> >>> index d9fb991ef..b4e0647bd 100644
> >>> --- a/include/linux/openvswitch.h
> >>> +++ b/include/linux/openvswitch.h
> >>> @@ -992,6 +992,30 @@ struct check_pkt_len_arg {
> >>>  };
> >>>  #endif
> >>>
> >>> +#define OVS_EMIT_SAMPLE_COOKIE_MAX_SIZE 16
> >>> +/**
> >>> + * enum ovs_emit_sample_attr - Attributes for 
> >>> %OVS_ACTION_ATTR_EMIT_SAMPLE
> >>> + * action.
> >>> + *
> >>> + * @OVS_EMIT_SAMPLE_ATTR_GROUP: 32-bit number to identify the source of 
> >>> the
> >>> + * sample.
> >>> + * @OVS_EMIT_SAMPLE_ATTR_COOKIE: A variable-length binary cookie that 
> >>> contains
> >>> + * user-defined metadata. The maximum length is 16 bytes.
> >>> + *
> >>> + * Sends the packet to the psample multicast group with the specified 
> >>> group and
> >>> + * cookie. It is possible to combine this action with the
> >>> + * %OVS_ACTION_ATTR_TRUNC to limit the size of the packet being emitted.
> >>
> >> I'll start the discussion again on the naming. The name "emit_sample()"
> >> does not seem appropriate. This function's primary role is to copy the
> >> packet and send it to a local collector, which varies depending on the
> >> datapath. For the kernel datapath, this collector is psample, while for
> >> userspace, it will likely be some kind of probe. This action is distinct
> >> from the sample() action by design; it is a standalone action that can
> >> be combined with others.
> >>
> >> Furthermore, the action itself does not involve taking a sample; it
> >> consistently pushes the packet to the local collector. Therefore, I
> >> suggest renaming "emit_sample()" to "emit_local()". This same goes for
> >> all the derivative ATTR naming.
> >>
> >
> > Let's discuss this in the kernel ml.
> >
> >>> + */
> >>> +enum ovs_emit_sample_attr {
> >>> + OVS_EMIT_SAMPLE_ATTR_UNPSEC,
> >>> + OVS_EMIT_SAMPLE_ATTR_GROUP, /* u32 number. */
> >>> + OVS_EMIT_SAMPLE_ATTR_COOKIE,/* Optional, user specified cookie. */
> >>
> >> Fix comment alignment? Maybe also change the order to be alphabetical?
> >>
> >
> > What do you mean by comment alignment?
>
> Well you are using tabs to index which might result in it looking like this:
>
> + OVS_EMIT_SAMPLE_ATTR_GROUP, /* u32 number. */
> + OVS_EMIT_SAMPLE_ATTR_COOKIE,/* Optional, user specified cookie. */
>
> (Most) of the other comments use spaces after the definition.
>
> >>> + __OVS_EMIT_SAMPLE_ATTR_MAX
> >>> +};
> >>> +
> >>> +#define OVS_EMIT_SAMPLE_ATTR_MAX (__OVS_EMIT_SAMPLE_ATTR_MAX - 1)
> >>> +
> >

Re: [ovs-dev] [PATCH net-next v5 05/10] net: openvswitch: add emit_sample action

[Adrián Moreno]

On Thu, Jun 27, 2024 at 09:30:08AM GMT, Aaron Conole wrote:
> Ilya Maximets  writes:
>
> > On 6/27/24 12:15, Adrián Moreno wrote:
> >> On Thu, Jun 27, 2024 at 11:31:41AM GMT, Eelco Chaudron wrote:
> >>>
> >>>
> >>> On 27 Jun 2024, at 11:23, Ilya Maximets wrote:
> >>>
> >>>> On 6/27/24 11:14, Eelco Chaudron wrote:
> >>>>>
> >>>>>
> >>>>> On 27 Jun 2024, at 10:36, Ilya Maximets wrote:
> >>>>>
> >>>>>> On 6/27/24 09:52, Adrián Moreno wrote:
> >>>>>>> On Thu, Jun 27, 2024 at 09:06:46AM GMT, Eelco Chaudron wrote:
> >>>>>>>>
> >>>>>>>>
> >>>>>>>> On 26 Jun 2024, at 22:34, Adrián Moreno wrote:
> >>>>>>>>
> >>>>>>>>> On Wed, Jun 26, 2024 at 04:28:17PM GMT, Eelco Chaudron wrote:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> On 25 Jun 2024, at 22:51, Adrian Moreno wrote:
> >>>>>>>>>>
> >>>>>>>>>>> Add support for a new action: emit_sample.
> >>>>>>>>>>>
> >>>>>>>>>>> This action accepts a u32 group id and a variable-length cookie 
> >>>>>>>>>>> and uses
> >>>>>>>>>>> the psample multicast group to make the packet available for
> >>>>>>>>>>> observability.
> >>>>>>>>>>>
> >>>>>>>>>>> The maximum length of the user-defined cookie is set to 16, same 
> >>>>>>>>>>> as
> >>>>>>>>>>> tc_cookie, to discourage using cookies that will not be 
> >>>>>>>>>>> offloadable.
> >>>>>>>>>>
> >>>>>>>>>> I’ll add the same comment as I had in the user space part, and that
> >>>>>>>>>> is that I feel from an OVS perspective this action should be called
> >>>>>>>>>> emit_local() instead of emit_sample() to make it Datapath 
> >>>>>>>>>> independent.
> >>>>>>>>>> Or quoting the earlier comment:
> >>>>>>>>>>
> >>>>>>>>>>
> >>>>>>>>>> “I’ll start the discussion again on the naming. The name 
> >>>>>>>>>> "emit_sample()"
> >>>>>>>>>> does not seem appropriate. This function's primary role is to copy 
> >>>>>>>>>> the
> >>>>>>>>>> packet and send it to a local collector, which varies depending on 
> >>>>>>>>>> the
> >>>>>>>>>> datapath. For the kernel datapath, this collector is psample, 
> >>>>>>>>>> while for
> >>>>>>>>>> userspace, it will likely be some kind of probe. This action is 
> >>>>>>>>>> distinct
> >>>>>>>>>> from the sample() action by design; it is a standalone action that 
> >>>>>>>>>> can
> >>>>>>>>>> be combined with others.
> >>>>>>>>>>
> >>>>>>>>>> Furthermore, the action itself does not involve taking a sample; it
> >>>>>>>>>> consistently pushes the packet to the local collector. Therefore, I
> >>>>>>>>>> suggest renaming "emit_sample()" to "emit_local()". This same goes 
> >>>>>>>>>> for
> >>>>>>>>>> all the derivative ATTR naming.”
> >>>>>>>>>>
> >>>>>>>>>
> >>>>>>>>> This is a blurry semantic area.
> >>>>>>>>> IMO, "sample" is the act of extracting (potentially a piece of)
> >>>>>>>>> someting, in this case, a packet. It is common to only take some 
> >>>>>>>>> packets
> >>>>>>>>> as samples, so this action usually comes with some kind of "rate", 
> >>>>>>>>> but
> >>>>>>>>> e

Re: [ovs-dev] [RFC v2 4/9] vswitchd: Add local sampling to vswitchd schema.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:11:00PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Add as new column in the Flow_Sample_Collector_Set table named
> > "local_sample_group" which enables this feature.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  NEWS   |  4 ++
> >  vswitchd/bridge.c  | 78 +++---
> >  vswitchd/vswitch.ovsschema |  9 -
> >  vswitchd/vswitch.xml   | 39 +--
> >  4 files changed, 119 insertions(+), 11 deletions(-)
> >
> > diff --git a/NEWS b/NEWS
> > index b92cec532..1c05a7120 100644
> > --- a/NEWS
> > +++ b/NEWS
> > @@ -7,6 +7,10 @@ Post-v3.3.0
> > - The primary development branch has been renamed from 'master' to 
> > 'main'.
> >   The OVS tree remains hosted on GitHub.
> > https://github.com/openvswitch/ovs.git
> > +   - Local sampling is introduced. It reuses the OpenFlow sample action and
> > + allows samples to be emitted locally (instead of via IPFIX) in a
> > + datapath-specific manner via the new datapath action called 
> > "emit_sample".
> > + Linux kernel datapath is the first to support this feature.
> >
> >
> >  v3.3.0 - 16 Feb 2024
> > diff --git a/vswitchd/bridge.c b/vswitchd/bridge.c
> > index 95a65fcdc..cd7dc6646 100644
> > --- a/vswitchd/bridge.c
> > +++ b/vswitchd/bridge.c
> > @@ -288,6 +288,7 @@ static void bridge_configure_mac_table(struct bridge *);
> >  static void bridge_configure_mcast_snooping(struct bridge *);
> >  static void bridge_configure_sflow(struct bridge *, int 
> > *sflow_bridge_number);
> >  static void bridge_configure_ipfix(struct bridge *);
> > +static void bridge_configure_lsample(struct bridge *);
> >  static void bridge_configure_spanning_tree(struct bridge *);
> >  static void bridge_configure_tables(struct bridge *);
> >  static void bridge_configure_dp_desc(struct bridge *);
> > @@ -989,6 +990,7 @@ bridge_reconfigure(const struct ovsrec_open_vswitch 
> > *ovs_cfg)
> >  bridge_configure_netflow(br);
> >  bridge_configure_sflow(br, &sflow_bridge_number);
> >  bridge_configure_ipfix(br);
> > +bridge_configure_lsample(br);
> >  bridge_configure_spanning_tree(br);
> >  bridge_configure_tables(br);
> >  bridge_configure_dp_desc(br);
> > @@ -1537,10 +1539,11 @@ ovsrec_ipfix_is_valid(const struct ovsrec_ipfix 
> > *ipfix)
> >  return ipfix && ipfix->n_targets > 0;
> >  }
> >
> > -/* Returns whether a Flow_Sample_Collector_Set row is valid. */
> > +/* Returns whether a Flow_Sample_Collector_Set row constains valid IPFIX
>
> constains -> 'contains a'
>

Ack.

> > + * configuration. */
> >  static bool
> > -ovsrec_fscs_is_valid(const struct ovsrec_flow_sample_collector_set *fscs,
> > - const struct bridge *br)
> > +ovsrec_fscs_is_valid_ipfix(const struct ovsrec_flow_sample_collector_set 
> > *fscs,
> > +   const struct bridge *br)
> >  {
> >  return ovsrec_ipfix_is_valid(fscs->ipfix) && fscs->bridge == br->cfg;
> >  }
> > @@ -1558,7 +1561,7 @@ bridge_configure_ipfix(struct bridge *br)
> >  const char *virtual_obs_id;
> >
> >  OVSREC_FLOW_SAMPLE_COLLECTOR_SET_FOR_EACH(fe_cfg, idl) {
> > -if (ovsrec_fscs_is_valid(fe_cfg, br)) {
> > +if (ovsrec_fscs_is_valid_ipfix(fe_cfg, br)) {
> >  n_fe_opts++;
> >  }
> >  }
> > @@ -1621,7 +1624,7 @@ bridge_configure_ipfix(struct bridge *br)
> >  fe_opts = xcalloc(n_fe_opts, sizeof *fe_opts);
> >  opts = fe_opts;
> >  OVSREC_FLOW_SAMPLE_COLLECTOR_SET_FOR_EACH(fe_cfg, idl) {
> > -if (ovsrec_fscs_is_valid(fe_cfg, br)) {
> > +if (ovsrec_fscs_is_valid_ipfix(fe_cfg, br)) {
> >  opts->collector_set_id = fe_cfg->id;
> >  sset_init(&opts->targets);
> >  sset_add_array(&opts->targets, fe_cfg->ipfix->targets,
> > @@ -1667,6 +1670,71 @@ bridge_configure_ipfix(struct bridge *br)
> >  }
> >  }
> >
> > +/* Returns whether a Flow_Sample_Collector_Set row contains valid local
> > + * sampling configuraiton. */
>
> ...row contains a valid local...
>++
>
> configuraiton -> configuration

Ack.

>
> > +static bool
> > +ovsrec_fscs_is_valid_local(const struct ovsrec_flow_sample_collector_set 
> > *fscs,
> > +   const struct bridge *br)
> > +{
> > +return fscs->local_sample_group && fscs->n_local_sample_group == 1 &&
> > +   fscs->bridge == br->cfg;
> > +}
> > +
> > +/* Set local sample configuration on 'br'. */
> > +static void
> > +bridge_configure_lsample(struct bridge *br)
> > +{
> > +static struct vlog_rate_limit rl = VLOG_RATE_LIMIT_INIT(1, 5);
> > +const struct ovsrec_flow_sample_collector_set *fscs;
> > +struct ofproto_lsample_options *opts_array, *opts;
> > +size_t n_opts = 0;
> > +int ret;
> > +
> > +/* Iterate the Flow_Sample_Collector_Set table twice.
> > + * First

Re: [ovs-dev] [RFC v2 3/9] ofproto: Add ofproto-dpif-lsample.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:10:27PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Add a new resource in ofproto-dpif and the corresponding API in
> > ofproto_provider.h to represent and local sampling configuration.
> >
> > Signed-off-by: Adrian Moreno 
>
> See some comments below.
>
> Cheers,
>
> Eelco
>
> > ---
> >  ofproto/automake.mk|   2 +
> >  ofproto/ofproto-dpif-lsample.c | 185 +
> >  ofproto/ofproto-dpif-lsample.h |  34 ++
> >  ofproto/ofproto-dpif.c |  37 +++
> >  ofproto/ofproto-dpif.h |   1 +
> >  ofproto/ofproto-provider.h |   9 ++
> >  ofproto/ofproto.c  |  12 +++
> >  ofproto/ofproto.h  |   8 ++
> >  8 files changed, 288 insertions(+)
> >  create mode 100644 ofproto/ofproto-dpif-lsample.c
> >  create mode 100644 ofproto/ofproto-dpif-lsample.h
> >
> > diff --git a/ofproto/automake.mk b/ofproto/automake.mk
> > index 7c08b563b..fd39bf561 100644
> > --- a/ofproto/automake.mk
> > +++ b/ofproto/automake.mk
> > @@ -34,6 +34,8 @@ ofproto_libofproto_la_SOURCES = \
> > ofproto/ofproto-dpif-mirror.h \
> > ofproto/ofproto-dpif-monitor.c \
> > ofproto/ofproto-dpif-monitor.h \
> > +   ofproto/ofproto-dpif-lsample.c \
> > +   ofproto/ofproto-dpif-lsample.h \
>
> Guess these need to move before the dpif-m* files.
>

Ack

> > ofproto/ofproto-dpif-rid.c \
> > ofproto/ofproto-dpif-rid.h \
> > ofproto/ofproto-dpif-sflow.c \
> > diff --git a/ofproto/ofproto-dpif-lsample.c b/ofproto/ofproto-dpif-lsample.c
> > new file mode 100644
> > index 0..7bdafac19
> > --- /dev/null
> > +++ b/ofproto/ofproto-dpif-lsample.c
> > @@ -0,0 +1,185 @@
> > +/*
> > + * Copyright (c) 2024 Red Hat, Inc.
> > + *
> > + * Licensed under the Apache License, Version 2.0 (the "License");
> > + * you may not use this file except in compliance with the License.
> > + * You may obtain a copy of the License at:
> > + *
> > + * http://www.apache.org/licenses/LICENSE-2.0
> > + *
> > + * Unless required by applicable law or agreed to in writing, software
> > + * distributed under the License is distributed on an "AS IS" BASIS,
> > + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
> > + * See the License for the specific language governing permissions and
> > + * limitations under the License.
> > + */
> > +
> > +#include 
> > +#include "ofproto-dpif-lsample.h"
> > +
> > +#include "cmap.h"
> > +#include "hash.h"
> > +#include "ofproto.h"
> > +#include "openvswitch/thread.h"
> > +
> > +/* Dpif local sampling.
> > + *
> > + * Thread safety: dpif_lsample allows lockless concurrent reads of local
> > + * sampling exporters as long as the following restrictions are met:
> > + *   1) While the last reference is being dropped, i.e: a thread is calling
> > + *  "dpif_lsample_unref" on the last reference, other threads cannot 
> > call
> > + *  "dpif_lsample_ref".
> > + *   2) Threads do not quiese while holding references to internal
> > + *  lsample_exporter objects.
> > + */
> > +
> > +struct dpif_lsample {
> > +struct cmap exporters;  /* Contains lsample_exporter_node 
> > instances
> > +   indexed by collector_set_id. */
> > +struct ovs_mutex mutex; /* Protects concurrent 
> > insertion/deletion
> > + * of exporters. */
> > +
> > +struct ovs_refcount ref_cnt;/* Controls references to this 
> > instance. */
> > +};
> > +
> > +struct lsample_exporter {
> > +struct ofproto_lsample_options options;
> > +};
> > +
> > +struct lsample_exporter_node {
> > +struct cmap_node node;  /* In dpif_lsample->exporters. */
> > +struct lsample_exporter exporter;
> > +};
> > +
> > +static void
> > +dpif_lsample_delete_exporter(struct dpif_lsample *lsample,
> > + struct lsample_exporter_node *node)
> > +{
> > +ovs_mutex_lock(&lsample->mutex);
> > +cmap_remove(&lsample->exporters, &node->node,
> > +hash_int(node->exporter.options.collector_set_id, 0));
> > +ovs_mutex_unlock(&lsample->mutex);
> > +
> > +ovsrcu_postpone(free, node);
> > +}
> > +
> > +/* Adds an exporter with the provided options which are copied. */
> > +static struct lsample_exporter_node *
> > +dpif_lsample_add_exporter(struct dpif_lsample *lsample,
> > +  const struct ofproto_lsample_options *options)
> > +{
> > +struct lsample_exporter_node *node;
>
> New line between definitions and code.
>

Ack.

> > +node = xzalloc(sizeof *node);
> > +node->exporter.options = *options;
> > +
> > +ovs_mutex_lock(&lsample->mutex);
> > +cmap_insert(&lsample->exporters, &node->node,
> > +hash_int(options->collector_set_id, 0));
> > +ovs_mutex_unlock(&lsample->mutex);
> > +
> > +return node;
> > +}
> > +
> > +static struct lsample_exporter_node *
> > +dpif_lsample_find_expo

Re: [ovs-dev] [RFC v2 1/9] odp-util: Add support OVS_ACTION_ATTR_EMIT_SAMPLE.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 02:08:27PM GMT, Eelco Chaudron wrote:
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Add support for parsing and formatting the new action.
> >
> > Also, flag OVS_ACTION_ATTR_SAMPLE as requiring datapath assistance if it
> > contains a nested OVS_ACTION_ATTR_EMIT_SAMPLE. The reason is that the
> > sampling rate form the parent "sample" is made available to the nested
> > "emit_sample" by the kernel.
>
> Hi Adrian,
>
> Thanks for this series! This email kicks off my review of the series,
> see comments below and in the other patches.
>
> Cheers,
>
> Eelco
>
> > Signed-off-by: Adrian Moreno 
> > ---
> >  include/linux/openvswitch.h  |  25 +
> >  lib/dpif-netdev.c|   1 +
> >  lib/dpif.c   |   1 +
> >  lib/odp-execute.c|  25 -
> >  lib/odp-util.c   | 103 +++
> >  lib/odp-util.h   |   3 +
> >  ofproto/ofproto-dpif-ipfix.c |   1 +
> >  ofproto/ofproto-dpif-sflow.c |   1 +
> >  python/ovs/flow/odp.py   |   8 +++
> >  tests/odp.at |  16 ++
> >  10 files changed, 183 insertions(+), 1 deletion(-)
> >
> > diff --git a/include/linux/openvswitch.h b/include/linux/openvswitch.h
> > index d9fb991ef..b4e0647bd 100644
> > --- a/include/linux/openvswitch.h
> > +++ b/include/linux/openvswitch.h
> > @@ -992,6 +992,30 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_EMIT_SAMPLE_COOKIE_MAX_SIZE 16
> > +/**
> > + * enum ovs_emit_sample_attr - Attributes for %OVS_ACTION_ATTR_EMIT_SAMPLE
> > + * action.
> > + *
> > + * @OVS_EMIT_SAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > + * sample.
> > + * @OVS_EMIT_SAMPLE_ATTR_COOKIE: A variable-length binary cookie that 
> > contains
> > + * user-defined metadata. The maximum length is 16 bytes.
> > + *
> > + * Sends the packet to the psample multicast group with the specified 
> > group and
> > + * cookie. It is possible to combine this action with the
> > + * %OVS_ACTION_ATTR_TRUNC to limit the size of the packet being emitted.
>
> I'll start the discussion again on the naming. The name "emit_sample()"
> does not seem appropriate. This function's primary role is to copy the
> packet and send it to a local collector, which varies depending on the
> datapath. For the kernel datapath, this collector is psample, while for
> userspace, it will likely be some kind of probe. This action is distinct
> from the sample() action by design; it is a standalone action that can
> be combined with others.
>
> Furthermore, the action itself does not involve taking a sample; it
> consistently pushes the packet to the local collector. Therefore, I
> suggest renaming "emit_sample()" to "emit_local()". This same goes for
> all the derivative ATTR naming.
>

Let's discuss this in the kernel ml.

> > + */
> > +enum ovs_emit_sample_attr {
> > +   OVS_EMIT_SAMPLE_ATTR_UNPSEC,
> > +   OVS_EMIT_SAMPLE_ATTR_GROUP, /* u32 number. */
> > +   OVS_EMIT_SAMPLE_ATTR_COOKIE,/* Optional, user specified cookie. */
>
> Fix comment alignment? Maybe also change the order to be alphabetical?
>

What do you mean by comment alignment?


> > +   __OVS_EMIT_SAMPLE_ATTR_MAX
> > +};
> > +
> > +#define OVS_EMIT_SAMPLE_ATTR_MAX (__OVS_EMIT_SAMPLE_ATTR_MAX - 1)
> > +
> > +
> >  /**
> >   * enum ovs_action_attr - Action types.
> >   *
> > @@ -1087,6 +,7 @@ enum ovs_action_attr {
> > OVS_ACTION_ATTR_ADD_MPLS, /* struct ovs_action_add_mpls. */
> > OVS_ACTION_ATTR_DEC_TTL,  /* Nested OVS_DEC_TTL_ATTR_*. */
> > OVS_ACTION_ATTR_DROP, /* u32 xlate_error. */
> > +   OVS_ACTION_ATTR_EMIT_SAMPLE,  /* Nested OVS_EMIT_SAMPLE_ATTR_*. */
> >
> >  #ifndef __KERNEL__
> > OVS_ACTION_ATTR_TUNNEL_PUSH,   /* struct ovs_action_push_tnl*/
> > diff --git a/lib/dpif-netdev.c b/lib/dpif-netdev.c
> > index c7f9e1490..c1171890c 100644
> > --- a/lib/dpif-netdev.c
> > +++ b/lib/dpif-netdev.c
> > @@ -9519,6 +9519,7 @@ dp_execute_cb(void *aux_, struct dp_packet_batch 
> > *packets_,
> >  case OVS_ACTION_ATTR_DROP:
> >  case OVS_ACTION_ATTR_ADD_MPLS:
> >  case OVS_ACTION_ATTR_DEC_TTL:
> > +case OVS_ACTION_ATTR_EMIT_SAMPLE:
> >  case __OVS_ACTION_ATTR_MAX:
> >  OVS_NOT_REACHED();
> >  }
> > diff --git a/lib/dpif.c b/lib/dpif.c
> > index 23eb18495..489d6a095 100644
> > --- a/lib/dpif.c
> > +++ b/lib/dpif.c
> > @@ -1192,6 +1192,7 @@ dpif_execute_helper_cb(void *aux_, struct 
> > dp_packet_batch *packets_,
> >  case OVS_ACTION_ATTR_TUNNEL_PUSH:
> >  case OVS_ACTION_ATTR_TUNNEL_POP:
> >  case OVS_ACTION_ATTR_USERSPACE:
> > +case OVS_ACTION_ATTR_EMIT_SAMPLE:
> >  case OVS_ACTION_ATTR_RECIRC: {
> >  struct dpif_execute execute;
> >  struct ofpbuf execute_actions;
> > diff --git a/lib/odp-execute.c b/lib/odp-execute.c
> > index 081e4d432..967abfd0a 100644
> > --- a/lib/odp-execute.c
> > +++ b/lib/odp-execute.c
> > @@ -818,13 +818,13 @@ requires_datapath_assistan

Re: [ovs-dev] [PATCH net-next v5 05/10] net: openvswitch: add emit_sample action

[Adrián Moreno]

On Thu, Jun 27, 2024 at 11:31:41AM GMT, Eelco Chaudron wrote:
>
>
> On 27 Jun 2024, at 11:23, Ilya Maximets wrote:
>
> > On 6/27/24 11:14, Eelco Chaudron wrote:
> >>
> >>
> >> On 27 Jun 2024, at 10:36, Ilya Maximets wrote:
> >>
> >>> On 6/27/24 09:52, Adrián Moreno wrote:
> >>>> On Thu, Jun 27, 2024 at 09:06:46AM GMT, Eelco Chaudron wrote:
> >>>>>
> >>>>>
> >>>>> On 26 Jun 2024, at 22:34, Adrián Moreno wrote:
> >>>>>
> >>>>>> On Wed, Jun 26, 2024 at 04:28:17PM GMT, Eelco Chaudron wrote:
> >>>>>>>
> >>>>>>>
> >>>>>>> On 25 Jun 2024, at 22:51, Adrian Moreno wrote:
> >>>>>>>
> >>>>>>>> Add support for a new action: emit_sample.
> >>>>>>>>
> >>>>>>>> This action accepts a u32 group id and a variable-length cookie and 
> >>>>>>>> uses
> >>>>>>>> the psample multicast group to make the packet available for
> >>>>>>>> observability.
> >>>>>>>>
> >>>>>>>> The maximum length of the user-defined cookie is set to 16, same as
> >>>>>>>> tc_cookie, to discourage using cookies that will not be offloadable.
> >>>>>>>
> >>>>>>> I’ll add the same comment as I had in the user space part, and that
> >>>>>>> is that I feel from an OVS perspective this action should be called
> >>>>>>> emit_local() instead of emit_sample() to make it Datapath independent.
> >>>>>>> Or quoting the earlier comment:
> >>>>>>>
> >>>>>>>
> >>>>>>> “I’ll start the discussion again on the naming. The name 
> >>>>>>> "emit_sample()"
> >>>>>>> does not seem appropriate. This function's primary role is to copy the
> >>>>>>> packet and send it to a local collector, which varies depending on the
> >>>>>>> datapath. For the kernel datapath, this collector is psample, while 
> >>>>>>> for
> >>>>>>> userspace, it will likely be some kind of probe. This action is 
> >>>>>>> distinct
> >>>>>>> from the sample() action by design; it is a standalone action that can
> >>>>>>> be combined with others.
> >>>>>>>
> >>>>>>> Furthermore, the action itself does not involve taking a sample; it
> >>>>>>> consistently pushes the packet to the local collector. Therefore, I
> >>>>>>> suggest renaming "emit_sample()" to "emit_local()". This same goes for
> >>>>>>> all the derivative ATTR naming.”
> >>>>>>>
> >>>>>>
> >>>>>> This is a blurry semantic area.
> >>>>>> IMO, "sample" is the act of extracting (potentially a piece of)
> >>>>>> someting, in this case, a packet. It is common to only take some 
> >>>>>> packets
> >>>>>> as samples, so this action usually comes with some kind of "rate", but
> >>>>>> even if the rate is 1, it's still sampling in this context.
> >>>>>>
> >>>>>> OTOH, OVS kernel design tries to be super-modular and define small
> >>>>>> combinable actions, so the rate or probability generation is done with
> >>>>>> another action which is (IMHO unfortunately) named "sample".
> >>>>>>
> >>>>>> With that interpretation of the term it would actually make more sense
> >>>>>> to rename "sample" to something like "random" (of course I'm not
> >>>>>> suggestion we do it). "sample" without any nested action that actually
> >>>>>> sends the packet somewhere is not sampling, it's just doing something 
> >>>>>> or
> >>>>>> not based on a probability. Where as "emit_sample" is sampling even if
> >>>>>> it's not nested inside a "sample".
> >>>>>
> >>>>> You're assuming we are extracting a packet for sampling, but this 
> >>>>> function
> &

Re: [ovs-dev] [PATCH net-next v5 10/10] selftests: openvswitch: add emit_sample test

[Adrián Moreno]

On Wed, Jun 26, 2024 at 01:15:21PM GMT, Ilya Maximets wrote:
> On 6/25/24 22:51, Adrian Moreno wrote:
> > Add a test to verify sampling packets via psample works.
> >
> > In order to do that, create a subcommand in ovs-dpctl.py to listen to
> > on the psample multicast group and print samples.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  .../selftests/net/openvswitch/openvswitch.sh  | 114 +-
> >  .../selftests/net/openvswitch/ovs-dpctl.py|  73 ++-
> >  2 files changed, 181 insertions(+), 6 deletions(-)
> >
> > diff --git a/tools/testing/selftests/net/openvswitch/openvswitch.sh 
> > b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > index 15bca0708717..aeb9bee772be 100755
> > --- a/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > +++ b/tools/testing/selftests/net/openvswitch/openvswitch.sh
> > @@ -20,7 +20,8 @@ tests="
> > nat_related_v4  ip4-nat-related: ICMP related 
> > matches work with SNAT
> > netlink_checks  ovsnl: validate netlink attrs 
> > and settings
> > upcall_interfaces   ovs: test the upcall interfaces
> > -   drop_reason drop: test drop reasons are 
> > emitted"
> > +   drop_reason drop: test drop reasons are 
> > emitted
> > +   emit_sample emit_sample: Sampling packets 
> > with psample"
>
> There is an extra space character right after emit_sample word.
> This makes './openvswitch.sh emit_sample' to not run the test,
> because 'emit_sample' != 'emit_sample '.
>

Wow, good catch! I'll get rid of that space.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next v5 05/10] net: openvswitch: add emit_sample action

[Adrián Moreno]

On Thu, Jun 27, 2024 at 09:06:46AM GMT, Eelco Chaudron wrote:
>
>
> On 26 Jun 2024, at 22:34, Adrián Moreno wrote:
>
> > On Wed, Jun 26, 2024 at 04:28:17PM GMT, Eelco Chaudron wrote:
> >>
> >>
> >> On 25 Jun 2024, at 22:51, Adrian Moreno wrote:
> >>
> >>> Add support for a new action: emit_sample.
> >>>
> >>> This action accepts a u32 group id and a variable-length cookie and uses
> >>> the psample multicast group to make the packet available for
> >>> observability.
> >>>
> >>> The maximum length of the user-defined cookie is set to 16, same as
> >>> tc_cookie, to discourage using cookies that will not be offloadable.
> >>
> >> I’ll add the same comment as I had in the user space part, and that
> >> is that I feel from an OVS perspective this action should be called
> >> emit_local() instead of emit_sample() to make it Datapath independent.
> >> Or quoting the earlier comment:
> >>
> >>
> >> “I’ll start the discussion again on the naming. The name "emit_sample()"
> >> does not seem appropriate. This function's primary role is to copy the
> >> packet and send it to a local collector, which varies depending on the
> >> datapath. For the kernel datapath, this collector is psample, while for
> >> userspace, it will likely be some kind of probe. This action is distinct
> >> from the sample() action by design; it is a standalone action that can
> >> be combined with others.
> >>
> >> Furthermore, the action itself does not involve taking a sample; it
> >> consistently pushes the packet to the local collector. Therefore, I
> >> suggest renaming "emit_sample()" to "emit_local()". This same goes for
> >> all the derivative ATTR naming.”
> >>
> >
> > This is a blurry semantic area.
> > IMO, "sample" is the act of extracting (potentially a piece of)
> > someting, in this case, a packet. It is common to only take some packets
> > as samples, so this action usually comes with some kind of "rate", but
> > even if the rate is 1, it's still sampling in this context.
> >
> > OTOH, OVS kernel design tries to be super-modular and define small
> > combinable actions, so the rate or probability generation is done with
> > another action which is (IMHO unfortunately) named "sample".
> >
> > With that interpretation of the term it would actually make more sense
> > to rename "sample" to something like "random" (of course I'm not
> > suggestion we do it). "sample" without any nested action that actually
> > sends the packet somewhere is not sampling, it's just doing something or
> > not based on a probability. Where as "emit_sample" is sampling even if
> > it's not nested inside a "sample".
>
> You're assuming we are extracting a packet for sampling, but this function
> can be used for various other purposes. For instance, it could handle the
> packet outside of the OVS pipeline through an eBPF program (so we are not
> taking a sample, but continue packet processing outside of the OVS
> pipeline). Calling it emit_sampling() in such cases could be very
> confusing.
>

Well, I guess that would be clearly abusing the action. You could say
that of anything really. You could hook into skb_consume and continue
processing the skb but that doesn't change the intended behavior of the
drop action.

The intended behavior of the action is sampling, as is the intended
behavior of "psample".

> > Having said that, I don't have a super strong favor for "emit_sample". I'm
> > OK with "emit_local" or "emit_packet" or even just "emit".
> > I don't think any term will fully satisfy everyone so I hope we can find
> > a reasonable compromise.
>
> My preference would be emit_local() as we hand it off to some local
> datapath entity.
>

I'm OK removing the controversial term. Let's see what others think.

> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  Documentation/netlink/specs/ovs_flow.yaml | 17 +
> >>>  include/uapi/linux/openvswitch.h  | 28 ++
> >>>  net/openvswitch/Kconfig   |  1 +
> >>>  net/openvswitch/actions.c | 45 +++
> >>>  net/openvswitch/flow_netlink.c| 33 -
> >>>  5 files changed, 123 insertions(+), 1 deletion(-)
>

Re: [ovs-dev] [PATCH net-next v5 05/10] net: openvswitch: add emit_sample action

[Adrián Moreno]

On Wed, Jun 26, 2024 at 04:28:17PM GMT, Eelco Chaudron wrote:
>
>
> On 25 Jun 2024, at 22:51, Adrian Moreno wrote:
>
> > Add support for a new action: emit_sample.
> >
> > This action accepts a u32 group id and a variable-length cookie and uses
> > the psample multicast group to make the packet available for
> > observability.
> >
> > The maximum length of the user-defined cookie is set to 16, same as
> > tc_cookie, to discourage using cookies that will not be offloadable.
>
> I’ll add the same comment as I had in the user space part, and that
> is that I feel from an OVS perspective this action should be called
> emit_local() instead of emit_sample() to make it Datapath independent.
> Or quoting the earlier comment:
>
>
> “I’ll start the discussion again on the naming. The name "emit_sample()"
> does not seem appropriate. This function's primary role is to copy the
> packet and send it to a local collector, which varies depending on the
> datapath. For the kernel datapath, this collector is psample, while for
> userspace, it will likely be some kind of probe. This action is distinct
> from the sample() action by design; it is a standalone action that can
> be combined with others.
>
> Furthermore, the action itself does not involve taking a sample; it
> consistently pushes the packet to the local collector. Therefore, I
> suggest renaming "emit_sample()" to "emit_local()". This same goes for
> all the derivative ATTR naming.”
>

This is a blurry semantic area.
IMO, "sample" is the act of extracting (potentially a piece of)
someting, in this case, a packet. It is common to only take some packets
as samples, so this action usually comes with some kind of "rate", but
even if the rate is 1, it's still sampling in this context.

OTOH, OVS kernel design tries to be super-modular and define small
combinable actions, so the rate or probability generation is done with
another action which is (IMHO unfortunately) named "sample".

With that interpretation of the term it would actually make more sense
to rename "sample" to something like "random" (of course I'm not
suggestion we do it). "sample" without any nested action that actually
sends the packet somewhere is not sampling, it's just doing something or
not based on a probability. Where as "emit_sample" is sampling even if
it's not nested inside a "sample".

Having said that, I don't have a super strong favor for "emit_sample". I'm
OK with "emit_local" or "emit_packet" or even just "emit".
I don't think any term will fully satisfy everyone so I hope we can find
a reasonable compromise.


> > Signed-off-by: Adrian Moreno 
> > ---
> >  Documentation/netlink/specs/ovs_flow.yaml | 17 +
> >  include/uapi/linux/openvswitch.h  | 28 ++
> >  net/openvswitch/Kconfig   |  1 +
> >  net/openvswitch/actions.c | 45 +++
> >  net/openvswitch/flow_netlink.c| 33 -
> >  5 files changed, 123 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/netlink/specs/ovs_flow.yaml 
> > b/Documentation/netlink/specs/ovs_flow.yaml
> > index 4fdfc6b5cae9..a7ab5593a24f 100644
> > --- a/Documentation/netlink/specs/ovs_flow.yaml
> > +++ b/Documentation/netlink/specs/ovs_flow.yaml
> > @@ -727,6 +727,12 @@ attribute-sets:
> >  name: dec-ttl
> >  type: nest
> >  nested-attributes: dec-ttl-attrs
> > +  -
> > +name: emit-sample
> > +type: nest
> > +nested-attributes: emit-sample-attrs
> > +doc: |
> > +  Sends a packet sample to psample for external observation.
> >-
> >  name: tunnel-key-attrs
> >  enum-name: ovs-tunnel-key-attr
> > @@ -938,6 +944,17 @@ attribute-sets:
> >-
> >  name: gbp
> >  type: u32
> > +  -
> > +name: emit-sample-attrs
> > +enum-name: ovs-emit-sample-attr
> > +name-prefix: ovs-emit-sample-attr-
> > +attributes:
> > +  -
> > +name: group
> > +type: u32
> > +  -
> > +name: cookie
> > +type: binary
> >
> >  operations:
> >name-prefix: ovs-flow-cmd-
> > diff --git a/include/uapi/linux/openvswitch.h 
> > b/include/uapi/linux/openvswitch.h
> > index efc82c318fa2..8cfa1b3f6b06 100644
> > --- a/include/uapi/linux/openvswitch.h
> > +++ b/include/uapi/linux/openvswitch.h
> > @@ -914,6 +914,31 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_EMIT_SAMPLE_COOKIE_MAX_SIZE 16
> > +/**
> > + * enum ovs_emit_sample_attr - Attributes for %OVS_ACTION_ATTR_EMIT_SAMPLE
> > + * action.
> > + *
> > + * @OVS_EMIT_SAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > + * sample.
> > + * @OVS_EMIT_SAMPLE_ATTR_COOKIE: A variable-length binary cookie that 
> > contains
> > + * user-defined metadata. The maximum length is 
> > OVS_EMIT_SAMPLE_COOKIE_MAX_SIZE
> > + * bytes.
> > + *
> > + * Sends the packet to the psample multicast group with the specified 
> > group and
> > + * cookie. It is possible 

Re: [ovs-dev] [PATCH net-next v5 05/10] net: openvswitch: add emit_sample action

[Adrián Moreno]

On Wed, Jun 26, 2024 at 12:51:19PM GMT, Ilya Maximets wrote:
> On 6/25/24 22:51, Adrian Moreno wrote:
> > Add support for a new action: emit_sample.
> >
> > This action accepts a u32 group id and a variable-length cookie and uses
> > the psample multicast group to make the packet available for
> > observability.
> >
> > The maximum length of the user-defined cookie is set to 16, same as
> > tc_cookie, to discourage using cookies that will not be offloadable.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  Documentation/netlink/specs/ovs_flow.yaml | 17 +
> >  include/uapi/linux/openvswitch.h  | 28 ++
> >  net/openvswitch/Kconfig   |  1 +
> >  net/openvswitch/actions.c | 45 +++
> >  net/openvswitch/flow_netlink.c| 33 -
> >  5 files changed, 123 insertions(+), 1 deletion(-)
> >
> > diff --git a/Documentation/netlink/specs/ovs_flow.yaml 
> > b/Documentation/netlink/specs/ovs_flow.yaml
> > index 4fdfc6b5cae9..a7ab5593a24f 100644
> > --- a/Documentation/netlink/specs/ovs_flow.yaml
> > +++ b/Documentation/netlink/specs/ovs_flow.yaml
> > @@ -727,6 +727,12 @@ attribute-sets:
> >  name: dec-ttl
> >  type: nest
> >  nested-attributes: dec-ttl-attrs
> > +  -
> > +name: emit-sample
> > +type: nest
> > +nested-attributes: emit-sample-attrs
> > +doc: |
> > +  Sends a packet sample to psample for external observation.
> >-
> >  name: tunnel-key-attrs
> >  enum-name: ovs-tunnel-key-attr
> > @@ -938,6 +944,17 @@ attribute-sets:
> >-
> >  name: gbp
> >  type: u32
> > +  -
> > +name: emit-sample-attrs
> > +enum-name: ovs-emit-sample-attr
> > +name-prefix: ovs-emit-sample-attr-
> > +attributes:
> > +  -
> > +name: group
> > +type: u32
> > +  -
> > +name: cookie
> > +type: binary
> >
> >  operations:
> >name-prefix: ovs-flow-cmd-
> > diff --git a/include/uapi/linux/openvswitch.h 
> > b/include/uapi/linux/openvswitch.h
> > index efc82c318fa2..8cfa1b3f6b06 100644
> > --- a/include/uapi/linux/openvswitch.h
> > +++ b/include/uapi/linux/openvswitch.h
> > @@ -914,6 +914,31 @@ struct check_pkt_len_arg {
> >  };
> >  #endif
> >
> > +#define OVS_EMIT_SAMPLE_COOKIE_MAX_SIZE 16
> > +/**
> > + * enum ovs_emit_sample_attr - Attributes for %OVS_ACTION_ATTR_EMIT_SAMPLE
> > + * action.
> > + *
> > + * @OVS_EMIT_SAMPLE_ATTR_GROUP: 32-bit number to identify the source of the
> > + * sample.
> > + * @OVS_EMIT_SAMPLE_ATTR_COOKIE: A variable-length binary cookie that 
> > contains
> > + * user-defined metadata. The maximum length is 
> > OVS_EMIT_SAMPLE_COOKIE_MAX_SIZE
> > + * bytes.
> > + *
> > + * Sends the packet to the psample multicast group with the specified 
> > group and
> > + * cookie. It is possible to combine this action with the
> > + * %OVS_ACTION_ATTR_TRUNC action to limit the size of the packet being 
> > emitted.
> > + */
> > +enum ovs_emit_sample_attr {
> > +   OVS_EMIT_SAMPLE_ATTR_GROUP = 1, /* u32 number. */
> > +   OVS_EMIT_SAMPLE_ATTR_COOKIE,/* Optional, user specified cookie. */
> > +
> > +   /* private: */
> > +   __OVS_EMIT_SAMPLE_ATTR_MAX
> > +};
> > +
> > +#define OVS_EMIT_SAMPLE_ATTR_MAX (__OVS_EMIT_SAMPLE_ATTR_MAX - 1)
> > +
> >  /**
> >   * enum ovs_action_attr - Action types.
> >   *
> > @@ -966,6 +991,8 @@ struct check_pkt_len_arg {
> >   * of l3 tunnel flag in the tun_flags field of OVS_ACTION_ATTR_ADD_MPLS
> >   * argument.
> >   * @OVS_ACTION_ATTR_DROP: Explicit drop action.
> > + * @OVS_ACTION_ATTR_EMIT_SAMPLE: Send a sample of the packet to external
> > + * observers via psample.
> >   *
> >   * Only a single header can be set with a single %OVS_ACTION_ATTR_SET.  
> > Not all
> >   * fields within a header are modifiable, e.g. the IPv4 protocol and 
> > fragment
> > @@ -1004,6 +1031,7 @@ enum ovs_action_attr {
> > OVS_ACTION_ATTR_ADD_MPLS, /* struct ovs_action_add_mpls. */
> > OVS_ACTION_ATTR_DEC_TTL,  /* Nested OVS_DEC_TTL_ATTR_*. */
> > OVS_ACTION_ATTR_DROP, /* u32 error code. */
> > +   OVS_ACTION_ATTR_EMIT_SAMPLE,  /* Nested OVS_EMIT_SAMPLE_ATTR_*. */
> >
> > __OVS_ACTION_ATTR_MAX,/* Nothing past this will be accepted
> >* from userspace. */
> > diff --git a/net/openvswitch/Kconfig b/net/openvswitch/Kconfig
> > index 29a7081858cd..2535f3f9f462 100644
> > --- a/net/openvswitch/Kconfig
> > +++ b/net/openvswitch/Kconfig
> > @@ -10,6 +10,7 @@ config OPENVSWITCH
> >(NF_CONNTRACK && ((!NF_DEFRAG_IPV6 || NF_DEFRAG_IPV6) && \
> >  (!NF_NAT || NF_NAT) && \
> >  (!NETFILTER_CONNCOUNT || 
> > NETFILTER_CONNCOUNT)))
> > +   depends on PSAMPLE || !PSAMPLE
> > select LIBCRC32C
> > select MPLS
> > select NET_MPLS_GSO
> > diff --git a/net/openvswitch/actions.c b/net/openvswitch/actions

Re: [ovs-dev] [PATCH net-next v5 02/10] net: sched: act_sample: add action cookie to sample

[Adrián Moreno]

On Wed, Jun 26, 2024 at 04:28:01PM GMT, Eelco Chaudron wrote:
>
>
> On 25 Jun 2024, at 22:51, Adrian Moreno wrote:
>
> > If the action has a user_cookie, pass it along to the sample so it can
> > be easily identified.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  net/sched/act_sample.c | 12 
> >  1 file changed, 12 insertions(+)
> >
> > diff --git a/net/sched/act_sample.c b/net/sched/act_sample.c
> > index a69b53d54039..2ceb4d141b71 100644
> > --- a/net/sched/act_sample.c
> > +++ b/net/sched/act_sample.c
> > @@ -167,7 +167,9 @@ TC_INDIRECT_SCOPE int tcf_sample_act(struct sk_buff 
> > *skb,
> >  {
> > struct tcf_sample *s = to_sample(a);
> > struct psample_group *psample_group;
> > +   u8 cookie_data[TC_COOKIE_MAX_SIZE];
> > struct psample_metadata md = {};
> > +   struct tc_cookie *user_cookie;
> > int retval;
> >
> > tcf_lastuse_update(&s->tcf_tm);
> > @@ -189,6 +191,16 @@ TC_INDIRECT_SCOPE int tcf_sample_act(struct sk_buff 
> > *skb,
> > if (skb_at_tc_ingress(skb) && tcf_sample_dev_ok_push(skb->dev))
> > skb_push(skb, skb->mac_len);
> >
> > +   rcu_read_lock();
> > +   user_cookie = rcu_dereference(a->user_cookie);
> > +   if (user_cookie) {
> > +   memcpy(cookie_data, user_cookie->data,
> > +  user_cookie->len);
>
> Maybe I’m over paranoid, but can we assume user_cookie->len, will not be 
> larger than TC_COOKIE_MAX_SIZE?
> Or should we do something like min(user_cookie->len, sizeof(cookie_data))
>

I think it's good to be paranoid with this kind of things. I do,
however, think it should be safe to use. The cookie is extracted from
the netlink attribute directly and its length is verified with the
nla_policy [1]. So nothing that comes into the kernel should be larger
than TC_COOKIE_MAX_SIZE.

I guess if there is some previous bug that allows for the size to get
corrupted, then this might happen but doing those kind of checks in the
fast path seems a bit excessive. For example, Ilya argued in v2 [2] that
we should avoid zeroing "u8 cookie_data[TC_COOKIE_MAX_SIZE]" to safe the
unneeded cycles.

[1] 
https://github.com/torvalds/linux/blob/55027e689933ba2e64f3d245fb1ff185b3e7fc81/net/sched/act_api.c#L1299
[2] 
https://patchwork.kernel.org/project/netdevbpf/patch/20240603185647.2310748-3-amore...@redhat.com/

Thanks.
Adrián

> > +   md.user_cookie = cookie_data;
> > +   md.user_cookie_len = user_cookie->len;
> > +   }
> > +   rcu_read_unlock();
> > +
> > md.trunc_size = s->truncate ? s->trunc_size : skb->len;
> > psample_sample_packet(psample_group, skb, s->rate, &md);
> >
> > --
> > 2.45.1
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Wed, Jun 26, 2024 at 11:28:57AM GMT, Ilya Maximets wrote:
> On 6/26/24 07:14, Adrián Moreno wrote:
> > On Tue, Jun 25, 2024 at 11:04:42PM GMT, Ilya Maximets wrote:
> >> On 6/25/24 22:53, Adrián Moreno wrote:
> >>> On Mon, Jun 24, 2024 at 07:43:32PM GMT, Adrián Moreno wrote:
> >>>> On Mon, Jun 24, 2024 at 04:05:36PM GMT, Ilya Maximets wrote:
> >>>>> On 6/24/24 15:19, Adrián Moreno wrote:
> >>>>>> On Mon, Jun 24, 2024 at 01:37:44PM GMT, Ilya Maximets wrote:
> >>>>>>> On 6/5/24 22:23, Adrian Moreno wrote:
> >>>>>>>> Use the newly added emit_sample to implement OpenFlow sample() 
> >>>>>>>> actions
> >>>>>>>> with local sampling configuration.
> >>>>>>>>
> >>>>>>>> Signed-off-by: Adrian Moreno 
> >>>>>>>> ---
> >>>>>>>>  ofproto/ofproto-dpif-lsample.c |  17 
> >>>>>>>>  ofproto/ofproto-dpif-lsample.h |   6 ++
> >>>>>>>>  ofproto/ofproto-dpif-xlate.c   | 163 
> >>>>>>>> -
> >>>>>>>>  ofproto/ofproto-dpif-xlate.h   |   3 +-
> >>>>>>>>  ofproto/ofproto-dpif.c |   2 +-
> >>>>>>>>  5 files changed, 144 insertions(+), 47 deletions(-)
> >>>>>>>
> >>>>>>> Not a full review, just a note that this patch needs tests in 
> >>>>>>> ofproto-dpif.at
> >>>>>>> that would check that we generate correct datapath flows.
> >>>>>>>
> >>>>>>
> >>>>>> I thought about that, but ofproto-dpif.at is based on dummy datapat
> >>>>>> (userspace) which does not support this action. The configuration will
> >>>>>> be ignored and the datapath actions will not be generated. That's why I
> >>>>>> relied on system-traffic.at tests. Do you see any way around this?
> >>>>>
> >>>>> We don't need actual datapath support if we're not sending any actual
> >>>>> trafic.  You should be able to just turn on the capability and check
> >>>>> that ofproto/trace generates correct actions.
> >>>>>
> >>>>> We test kernel tunnels this way, for example.  See the macro
> >>>>> OVS_VSWITCHD_DISABLE_TUNNEL_PUSH_POP.  And some other similar checks.
> >>>>>
> >>>>
> >>>
> >>> Tried it but it doesn't seem to work. I now remember I hit this issue
> >>> originally :-).
> >>> We cannot enable a capability beyond the datapath's "boot time"
> >>> capabilities.
> >>>
> >>> If you try to run
> >>> "ovs-appctl dpif/set-dp-features br0 emit_sample true", the command
> >>> fails with:
> >>> "Can not enable features not supported by the datapth"
> >>>
> >>> The safeguard does make sense in general (prevents users from
> >>> knee-capping themselves) but for testing maybe we want to force it?
> >>>
> >>> OTOH, marking this feature as supported in the userspace datapath would
> >>> not be catastrophic, we could just warn the action is not supported
> >>> in odp_execute(), or consider VLOG_DBG_RL as current implementation
> >>> until we come up with a good one. However, I cannot of a good way of
> >>> of properly reporting this to the user beforehand so I think it could
> >>> still cause a lot of confusion.
> >>>
> >>> Thoughts?
> >>
> >> Hmm.  This command is generally for testing only or experiments.
> >> It's not documented and hence we do not support changing datapath
> >> features in runtime.
> >
> > Oh! It's true that it's docummented, although it does appear in
> > "list-commands". I thought we supported disabling features, I guess we
> > might on a case-by-case basis and through OVSDB.
> >
> >>
> >> It should be fine to add another command like force-dp-features to
> >> overcome the restriction.  Just make sure that it is not documented
> >> and doesn't appear in the list-commands output.
> >>
> >
> > If "set-dp-features" is already an exceptional path, can't we just add a
> > "--force" option to it?
>
> I guess, it's fine to add a flag.  Just don't document it.  But, please,
> add a scary warning in this case saying that unsupported feature is enabled
> and the behavior is unpredictable from this point on.  Tests can ignore the
> warning.
>

I will not documment it but I don't think we should remove from
"list-commands" either.

And sure, I'll add a scary warning.

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Tue, Jun 25, 2024 at 11:04:42PM GMT, Ilya Maximets wrote:
> On 6/25/24 22:53, Adrián Moreno wrote:
> > On Mon, Jun 24, 2024 at 07:43:32PM GMT, Adrián Moreno wrote:
> >> On Mon, Jun 24, 2024 at 04:05:36PM GMT, Ilya Maximets wrote:
> >>> On 6/24/24 15:19, Adrián Moreno wrote:
> >>>> On Mon, Jun 24, 2024 at 01:37:44PM GMT, Ilya Maximets wrote:
> >>>>> On 6/5/24 22:23, Adrian Moreno wrote:
> >>>>>> Use the newly added emit_sample to implement OpenFlow sample() actions
> >>>>>> with local sampling configuration.
> >>>>>>
> >>>>>> Signed-off-by: Adrian Moreno 
> >>>>>> ---
> >>>>>>  ofproto/ofproto-dpif-lsample.c |  17 
> >>>>>>  ofproto/ofproto-dpif-lsample.h |   6 ++
> >>>>>>  ofproto/ofproto-dpif-xlate.c   | 163 -
> >>>>>>  ofproto/ofproto-dpif-xlate.h   |   3 +-
> >>>>>>  ofproto/ofproto-dpif.c |   2 +-
> >>>>>>  5 files changed, 144 insertions(+), 47 deletions(-)
> >>>>>
> >>>>> Not a full review, just a note that this patch needs tests in 
> >>>>> ofproto-dpif.at
> >>>>> that would check that we generate correct datapath flows.
> >>>>>
> >>>>
> >>>> I thought about that, but ofproto-dpif.at is based on dummy datapat
> >>>> (userspace) which does not support this action. The configuration will
> >>>> be ignored and the datapath actions will not be generated. That's why I
> >>>> relied on system-traffic.at tests. Do you see any way around this?
> >>>
> >>> We don't need actual datapath support if we're not sending any actual
> >>> trafic.  You should be able to just turn on the capability and check
> >>> that ofproto/trace generates correct actions.
> >>>
> >>> We test kernel tunnels this way, for example.  See the macro
> >>> OVS_VSWITCHD_DISABLE_TUNNEL_PUSH_POP.  And some other similar checks.
> >>>
> >>
> >
> > Tried it but it doesn't seem to work. I now remember I hit this issue
> > originally :-).
> > We cannot enable a capability beyond the datapath's "boot time"
> > capabilities.
> >
> > If you try to run
> > "ovs-appctl dpif/set-dp-features br0 emit_sample true", the command
> > fails with:
> > "Can not enable features not supported by the datapth"
> >
> > The safeguard does make sense in general (prevents users from
> > knee-capping themselves) but for testing maybe we want to force it?
> >
> > OTOH, marking this feature as supported in the userspace datapath would
> > not be catastrophic, we could just warn the action is not supported
> > in odp_execute(), or consider VLOG_DBG_RL as current implementation
> > until we come up with a good one. However, I cannot of a good way of
> > of properly reporting this to the user beforehand so I think it could
> > still cause a lot of confusion.
> >
> > Thoughts?
>
> Hmm.  This command is generally for testing only or experiments.
> It's not documented and hence we do not support changing datapath
> features in runtime.

Oh! It's true that it's docummented, although it does appear in
"list-commands". I thought we supported disabling features, I guess we
might on a case-by-case basis and through OVSDB.

>
> It should be fine to add another command like force-dp-features to
> overcome the restriction.  Just make sure that it is not documented
> and doesn't appear in the list-commands output.
>

If "set-dp-features" is already an exceptional path, can't we just add a
"--force" option to it?

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 07:43:32PM GMT, Adrián Moreno wrote:
> On Mon, Jun 24, 2024 at 04:05:36PM GMT, Ilya Maximets wrote:
> > On 6/24/24 15:19, Adrián Moreno wrote:
> > > On Mon, Jun 24, 2024 at 01:37:44PM GMT, Ilya Maximets wrote:
> > >> On 6/5/24 22:23, Adrian Moreno wrote:
> > >>> Use the newly added emit_sample to implement OpenFlow sample() actions
> > >>> with local sampling configuration.
> > >>>
> > >>> Signed-off-by: Adrian Moreno 
> > >>> ---
> > >>>  ofproto/ofproto-dpif-lsample.c |  17 
> > >>>  ofproto/ofproto-dpif-lsample.h |   6 ++
> > >>>  ofproto/ofproto-dpif-xlate.c   | 163 -
> > >>>  ofproto/ofproto-dpif-xlate.h   |   3 +-
> > >>>  ofproto/ofproto-dpif.c |   2 +-
> > >>>  5 files changed, 144 insertions(+), 47 deletions(-)
> > >>
> > >> Not a full review, just a note that this patch needs tests in 
> > >> ofproto-dpif.at
> > >> that would check that we generate correct datapath flows.
> > >>
> > >
> > > I thought about that, but ofproto-dpif.at is based on dummy datapat
> > > (userspace) which does not support this action. The configuration will
> > > be ignored and the datapath actions will not be generated. That's why I
> > > relied on system-traffic.at tests. Do you see any way around this?
> >
> > We don't need actual datapath support if we're not sending any actual
> > trafic.  You should be able to just turn on the capability and check
> > that ofproto/trace generates correct actions.
> >
> > We test kernel tunnels this way, for example.  See the macro
> > OVS_VSWITCHD_DISABLE_TUNNEL_PUSH_POP.  And some other similar checks.
> >
>

Tried it but it doesn't seem to work. I now remember I hit this issue
originally :-).
We cannot enable a capability beyond the datapath's "boot time"
capabilities.

If you try to run
"ovs-appctl dpif/set-dp-features br0 emit_sample true", the command
fails with:
"Can not enable features not supported by the datapth"

The safeguard does make sense in general (prevents users from
knee-capping themselves) but for testing maybe we want to force it?

OTOH, marking this feature as supported in the userspace datapath would
not be catastrophic, we could just warn the action is not supported
in odp_execute(), or consider VLOG_DBG_RL as current implementation
until we come up with a good one. However, I cannot of a good way of
of properly reporting this to the user beforehand so I think it could
still cause a lot of confusion.

Thoughts?

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [PATCH net-next v4 04/10] net: psample: allow using rate as probability

[Adrián Moreno]

On Tue, Jun 25, 2024 at 01:17:19PM GMT, Paolo Abeni wrote:
> On Fri, 2024-06-21 at 12:10 +0200, Adrian Moreno wrote:
> > diff --git a/include/uapi/linux/tc_act/tc_sample.h 
> > b/include/uapi/linux/tc_act/tc_sample.h
> > index fee1bcc20793..7ee0735e7b38 100644
> > --- a/include/uapi/linux/tc_act/tc_sample.h
> > +++ b/include/uapi/linux/tc_act/tc_sample.h
> > @@ -18,6 +18,7 @@ enum {
> > TCA_SAMPLE_TRUNC_SIZE,
> > TCA_SAMPLE_PSAMPLE_GROUP,
> > TCA_SAMPLE_PAD,
> > +   TCA_SAMPLE_PROBABILITY,
> > __TCA_SAMPLE_MAX
> >  };
> >  #define TCA_SAMPLE_MAX (__TCA_SAMPLE_MAX - 1)
>
> I believe Ilya's comment on v3 is correct, this chunk looks unrelated
> and unneeded. I guess you can drop it? Or am I missing something?
>

Thanks both for spotting it. I'll send v5 without it.

> Thanks,
>
> Paolo
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 08:26:44PM GMT, Ilya Maximets wrote:
> On 6/24/24 20:00, Adrián Moreno wrote:
> > On Mon, Jun 24, 2024 at 02:03:00PM GMT, Ilya Maximets wrote:
> > [...]
> >>
> >> Why are we adding IPFIX?  This should work without IPFIX.
> >>
> >> Having both together can be a separate test.
> >>
> >>> +
> >>> +AT_DATA([flows.txt], [dnl
> >>> +in_port=ovs-p0,ip 
> >>> actions=sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918),output(port=ovs-p1,max_len=100)
> >>> +in_port=ovs-p1,ip 
> >>> actions=sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377),output(port=ovs-p0,max_len=100)
> >>
> >> It should be possible to wrap these lines a little with dnl.
> >
> > I'm curious about wrapping requirements for flows in tests.
> > None of them seem to be wrapped, splitting matches and actions won't
> > make lines much shorter and having more exotic wrapped & indented
> > action arguments seem strage to read (for eyes made to read flows in
> > a single line). What wrapping do you have in mind?
>
> No hard requirements, but I find lines that long hard to read
> since they are going out of the window width for me and getting
> wrapped in random places.
>
>
> AT_DATA([flows.txt], [dnl
> in_port=ovs-p0,ip dnl
> actions=dnl
>  sample( dnl
>   probability=65535, dnl
>   collector_set_id=1,dnl
>   obs_domain_id=1431655765,  dnl
>   obs_point_id=1717986918),  dnl
>  output(port=ovs-p1,max_len=100)
> ...
>
> I agree that above is not very readable...
>
> Or
>
> AT_DATA([flows.txt], [dnl
> m4_join([], [
>   [in_port=ovs-p0,ip], [ ],
>   [actions=],
>
> [sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918)],
>[output(port=ovs-p1,max_len=100)]
> ]
> m4_join([], [
>   [in_port=ovs-p1,ip], [ ],
>   [actions=],
>
> [sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377)],
>[output(port=ovs-p0,max_len=100)]
> ]
> )
>
> Or maybe just shorten the lines with a macro:
>
> m4_define([SAMPLE_ACTION],
>  
> [sample(probability=65535,collector_set_id=$1,obs_domain_id=$2,obs_point_id=$3)]
> )
>
> AT_DATA([flows.txt], [dnl
> in_port=ovs-p0,ip actions=SAMPLE_ACTION(1, 1431655765, 1717986918), 
> output(port=ovs-p1,max_len=100)
> in_port=ovs-p1,ip actions=SAMPLE_ACTION(2, 2290649224, 2576980377), 
> output(port=ovs-p0,max_len=100)
> ])
>

I prefer this one. I'll give it a try.

Thanks.
Adrián

> (Haven't tested any of these)
>
> Best regards, Ilya Maximets.
>

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 07:57:28PM GMT, Ilya Maximets wrote:
> On 6/24/24 19:38, Adrián Moreno wrote:
> > On Mon, Jun 24, 2024 at 04:15:50PM GMT, Ilya Maximets wrote:
> >> On 6/24/24 15:14, Adrián Moreno wrote:
> >>> On Mon, Jun 24, 2024 at 02:03:00PM GMT, Ilya Maximets wrote:
> >>>> On 6/5/24 22:23, Adrian Moreno wrote:
> >>>>> Test simultaneous IPFIX and local sampling including slow-path.
> >>>>>
> >>>>> Signed-off-by: Adrian Moreno 
> >>>>> ---
> >>>>>  tests/system-common-macros.at |   4 ++
> >>>>>  tests/system-traffic.at   | 105 ++
> >>>>>  2 files changed, 109 insertions(+)
> >>>>>
> >>>>
> >>>> Not a full review, but see a few comments below.
> >>>>
> >>>>> diff --git a/tests/system-common-macros.at 
> >>>>> b/tests/system-common-macros.at
> >>>>> index 2a68cd664..22b8885e4 100644
> >>>>> --- a/tests/system-common-macros.at
> >>>>> +++ b/tests/system-common-macros.at
> >>>>> @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >>>>>  # OVS_CHECK_DROP_ACTION()
> >>>>>  m4_define([OVS_CHECK_DROP_ACTION],
> >>>>>  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> >>>>> ovs-vswitchd.log])])
> >>>>> +
> >>>>> +# OVS_CHECK_EMIT_SAMPLE()
> >>>>> +m4_define([OVS_CHECK_EMIT_SAMPLE],
> >>>>> +[AT_SKIP_IF([! grep -q "Datapath supports emit_sample" 
> >>>>> ovs-vswitchd.log])])
> >>>>> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> >>>>> index bd7647cbe..babc56b56 100644
> >>>>> --- a/tests/system-traffic.at
> >>>>> +++ b/tests/system-traffic.at
> >>>>> @@ -8977,3 +8977,108 @@ OVS_WAIT_UNTIL([cat p2.pcap | grep -E "0x0050: 
> >>>>> * * *5002 *2000 *b85e *00
> >>>>>
> >>>>>  OVS_TRAFFIC_VSWITCHD_STOP
> >>>>>  AT_CLEANUP
> >>>>> +
> >>>>> +AT_SETUP([emit_sample])
> >>>>
> >>>> Other tests have a test group name at the beginning.
> >>>> You're also adding the test into NSH section.  Add a new section, e.g.,
> >>>> 'sampling' for this with AT_BANNER.
> >>>>
> >>>
> >>> Thanks. Will do.
> >>>
> >>>>> +OVS_TRAFFIC_VSWITCHD_START()
> >>>>> +OVS_CHECK_EMIT_SAMPLE()
> >>>>> +
> >>>>> +ADD_NAMESPACES(at_ns0, at_ns1)
> >>>>> +NS_CHECK_EXEC([at_ns0], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], 
> >>>>> [0], [ignore])
> >>>>> +NS_CHECK_EXEC([at_ns1], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], 
> >>>>> [0], [ignore])
> >>>>
> >>>> This looks strange, don't disable IPv6.  Appropriate datapath actions
> >>>> should not forward IPv6 traffic, if that is a desired behavior.
> >>>>
> >>>
> >>> I found tests cleaner if automatic ipv6 traffic is not sent by ports.
> >>> But sure, I can deal with that with OpenFlow.
> >>
> >> Would be good to have two separate tests.  One for IPv4 and one for IPv6.
> >>
> >>>
> >>>> Though, I guess, it would be nice to test capturing both IPv4 and IPv6.
> >>>>
> >>>>> +
> >>>>> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24", "e4:11:22:33:44:55")
> >>>>> +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24", "e4:11:22:33:44:66")
> >>>>> +
> >>>>> +NS_CHECK_EXEC([at_ns0], [arp -s 10.1.1.2 e4:11:22:33:44:66])
> >>>>> +NS_CHECK_EXEC([at_ns1], [arp -s 10.1.1.1 e4:11:22:33:44:55])
> >>>>> +
> >>>>> +
> >>>>> +AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg 
> >>>>> ofproto_dpif_upcall:dbg])
> >>>>> +
> >>>>> +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> >>>>> +-- --id=@ipfix create IPFIX 
> >>>>> targets=\"127.0.0.1:4739\" \
> >>>>> +-- create Flow_Sample_Collector_Set id=1 
> >>>>> bridge=@br0

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 02:03:00PM GMT, Ilya Maximets wrote:
[...]
>
> Why are we adding IPFIX?  This should work without IPFIX.
>
> Having both together can be a separate test.
>
> > +
> > +AT_DATA([flows.txt], [dnl
> > +in_port=ovs-p0,ip 
> > actions=sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918),output(port=ovs-p1,max_len=100)
> > +in_port=ovs-p1,ip 
> > actions=sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377),output(port=ovs-p0,max_len=100)
>
> It should be possible to wrap these lines a little with dnl.

I'm curious about wrapping requirements for flows in tests.
None of them seem to be wrapped, splitting matches and actions won't
make lines much shorter and having more exotic wrapped & indented
action arguments seem strage to read (for eyes made to read flows in
a single line). What wrapping do you have in mind?

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 04:05:36PM GMT, Ilya Maximets wrote:
> On 6/24/24 15:19, Adrián Moreno wrote:
> > On Mon, Jun 24, 2024 at 01:37:44PM GMT, Ilya Maximets wrote:
> >> On 6/5/24 22:23, Adrian Moreno wrote:
> >>> Use the newly added emit_sample to implement OpenFlow sample() actions
> >>> with local sampling configuration.
> >>>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  ofproto/ofproto-dpif-lsample.c |  17 
> >>>  ofproto/ofproto-dpif-lsample.h |   6 ++
> >>>  ofproto/ofproto-dpif-xlate.c   | 163 -
> >>>  ofproto/ofproto-dpif-xlate.h   |   3 +-
> >>>  ofproto/ofproto-dpif.c |   2 +-
> >>>  5 files changed, 144 insertions(+), 47 deletions(-)
> >>
> >> Not a full review, just a note that this patch needs tests in 
> >> ofproto-dpif.at
> >> that would check that we generate correct datapath flows.
> >>
> >
> > I thought about that, but ofproto-dpif.at is based on dummy datapat
> > (userspace) which does not support this action. The configuration will
> > be ignored and the datapath actions will not be generated. That's why I
> > relied on system-traffic.at tests. Do you see any way around this?
>
> We don't need actual datapath support if we're not sending any actual
> trafic.  You should be able to just turn on the capability and check
> that ofproto/trace generates correct actions.
>
> We test kernel tunnels this way, for example.  See the macro
> OVS_VSWITCHD_DISABLE_TUNNEL_PUSH_POP.  And some other similar checks.
>

OK. Thanks for the pointer. I'll give it a try.

Adrián.

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 04:15:50PM GMT, Ilya Maximets wrote:
> On 6/24/24 15:14, Adrián Moreno wrote:
> > On Mon, Jun 24, 2024 at 02:03:00PM GMT, Ilya Maximets wrote:
> >> On 6/5/24 22:23, Adrian Moreno wrote:
> >>> Test simultaneous IPFIX and local sampling including slow-path.
> >>>
> >>> Signed-off-by: Adrian Moreno 
> >>> ---
> >>>  tests/system-common-macros.at |   4 ++
> >>>  tests/system-traffic.at   | 105 ++
> >>>  2 files changed, 109 insertions(+)
> >>>
> >>
> >> Not a full review, but see a few comments below.
> >>
> >>> diff --git a/tests/system-common-macros.at b/tests/system-common-macros.at
> >>> index 2a68cd664..22b8885e4 100644
> >>> --- a/tests/system-common-macros.at
> >>> +++ b/tests/system-common-macros.at
> >>> @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >>>  # OVS_CHECK_DROP_ACTION()
> >>>  m4_define([OVS_CHECK_DROP_ACTION],
> >>>  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> >>> ovs-vswitchd.log])])
> >>> +
> >>> +# OVS_CHECK_EMIT_SAMPLE()
> >>> +m4_define([OVS_CHECK_EMIT_SAMPLE],
> >>> +[AT_SKIP_IF([! grep -q "Datapath supports emit_sample" 
> >>> ovs-vswitchd.log])])
> >>> diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> >>> index bd7647cbe..babc56b56 100644
> >>> --- a/tests/system-traffic.at
> >>> +++ b/tests/system-traffic.at
> >>> @@ -8977,3 +8977,108 @@ OVS_WAIT_UNTIL([cat p2.pcap | grep -E "0x0050: 
> >>> * * *5002 *2000 *b85e *00
> >>>
> >>>  OVS_TRAFFIC_VSWITCHD_STOP
> >>>  AT_CLEANUP
> >>> +
> >>> +AT_SETUP([emit_sample])
> >>
> >> Other tests have a test group name at the beginning.
> >> You're also adding the test into NSH section.  Add a new section, e.g.,
> >> 'sampling' for this with AT_BANNER.
> >>
> >
> > Thanks. Will do.
> >
> >>> +OVS_TRAFFIC_VSWITCHD_START()
> >>> +OVS_CHECK_EMIT_SAMPLE()
> >>> +
> >>> +ADD_NAMESPACES(at_ns0, at_ns1)
> >>> +NS_CHECK_EXEC([at_ns0], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], 
> >>> [0], [ignore])
> >>> +NS_CHECK_EXEC([at_ns1], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], 
> >>> [0], [ignore])
> >>
> >> This looks strange, don't disable IPv6.  Appropriate datapath actions
> >> should not forward IPv6 traffic, if that is a desired behavior.
> >>
> >
> > I found tests cleaner if automatic ipv6 traffic is not sent by ports.
> > But sure, I can deal with that with OpenFlow.
>
> Would be good to have two separate tests.  One for IPv4 and one for IPv6.
>
> >
> >> Though, I guess, it would be nice to test capturing both IPv4 and IPv6.
> >>
> >>> +
> >>> +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24", "e4:11:22:33:44:55")
> >>> +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24", "e4:11:22:33:44:66")
> >>> +
> >>> +NS_CHECK_EXEC([at_ns0], [arp -s 10.1.1.2 e4:11:22:33:44:66])
> >>> +NS_CHECK_EXEC([at_ns1], [arp -s 10.1.1.1 e4:11:22:33:44:55])
> >>> +
> >>> +
> >>> +AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg 
> >>> ofproto_dpif_upcall:dbg])
> >>> +
> >>> +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> >>> +-- --id=@ipfix create IPFIX 
> >>> targets=\"127.0.0.1:4739\" \
> >>> +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> >>> ipfix=@ipfix, local_sample_group=10 \
> >>> +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> >>> ipfix=@ipfix, local_sample_group=12],
> >>> + [0], [ignore])
> >>
> >> Why are we adding IPFIX?  This should work without IPFIX.
> >>
> >
> > It is intentional. I want to test both features work simulteaneously.
>
> Tests should be simple.  At least, we should have at least one very simple
> test, so if it fails we know that there is an issue in the basic logic.
>
> It's valuable to test a combination of the two, but it should be a separate
> test.  So, 3 tests so far: IPv4, IPv6, psample+ipfix.  Maybe the 4th separate
> test for the slow path.
>

Ack.

> >
> >&g

Re: [ovs-dev] [RFC v2 5/9] ofproto-dpif-xlate: Use emit_sample for local sample.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 01:37:44PM GMT, Ilya Maximets wrote:
> On 6/5/24 22:23, Adrian Moreno wrote:
> > Use the newly added emit_sample to implement OpenFlow sample() actions
> > with local sampling configuration.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  ofproto/ofproto-dpif-lsample.c |  17 
> >  ofproto/ofproto-dpif-lsample.h |   6 ++
> >  ofproto/ofproto-dpif-xlate.c   | 163 -
> >  ofproto/ofproto-dpif-xlate.h   |   3 +-
> >  ofproto/ofproto-dpif.c |   2 +-
> >  5 files changed, 144 insertions(+), 47 deletions(-)
>
> Not a full review, just a note that this patch needs tests in ofproto-dpif.at
> that would check that we generate correct datapath flows.
>

I thought about that, but ofproto-dpif.at is based on dummy datapat
(userspace) which does not support this action. The configuration will
be ignored and the datapath actions will not be generated. That's why I
relied on system-traffic.at tests. Do you see any way around this?

Thanks.
Adrián

___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 02:03:00PM GMT, Ilya Maximets wrote:
> On 6/5/24 22:23, Adrian Moreno wrote:
> > Test simultaneous IPFIX and local sampling including slow-path.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  tests/system-common-macros.at |   4 ++
> >  tests/system-traffic.at   | 105 ++
> >  2 files changed, 109 insertions(+)
> >
>
> Not a full review, but see a few comments below.
>
> > diff --git a/tests/system-common-macros.at b/tests/system-common-macros.at
> > index 2a68cd664..22b8885e4 100644
> > --- a/tests/system-common-macros.at
> > +++ b/tests/system-common-macros.at
> > @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >  # OVS_CHECK_DROP_ACTION()
> >  m4_define([OVS_CHECK_DROP_ACTION],
> >  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> > ovs-vswitchd.log])])
> > +
> > +# OVS_CHECK_EMIT_SAMPLE()
> > +m4_define([OVS_CHECK_EMIT_SAMPLE],
> > +[AT_SKIP_IF([! grep -q "Datapath supports emit_sample" 
> > ovs-vswitchd.log])])
> > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > index bd7647cbe..babc56b56 100644
> > --- a/tests/system-traffic.at
> > +++ b/tests/system-traffic.at
> > @@ -8977,3 +8977,108 @@ OVS_WAIT_UNTIL([cat p2.pcap | grep -E "0x0050: 
> > * * *5002 *2000 *b85e *00
> >
> >  OVS_TRAFFIC_VSWITCHD_STOP
> >  AT_CLEANUP
> > +
> > +AT_SETUP([emit_sample])
>
> Other tests have a test group name at the beginning.
> You're also adding the test into NSH section.  Add a new section, e.g.,
> 'sampling' for this with AT_BANNER.
>

Thanks. Will do.

> > +OVS_TRAFFIC_VSWITCHD_START()
> > +OVS_CHECK_EMIT_SAMPLE()
> > +
> > +ADD_NAMESPACES(at_ns0, at_ns1)
> > +NS_CHECK_EXEC([at_ns0], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], [0], 
> > [ignore])
> > +NS_CHECK_EXEC([at_ns1], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], [0], 
> > [ignore])
>
> This looks strange, don't disable IPv6.  Appropriate datapath actions
> should not forward IPv6 traffic, if that is a desired behavior.
>

I found tests cleaner if automatic ipv6 traffic is not sent by ports.
But sure, I can deal with that with OpenFlow.

> Though, I guess, it would be nice to test capturing both IPv4 and IPv6.
>
> > +
> > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24", "e4:11:22:33:44:55")
> > +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24", "e4:11:22:33:44:66")
> > +
> > +NS_CHECK_EXEC([at_ns0], [arp -s 10.1.1.2 e4:11:22:33:44:66])
> > +NS_CHECK_EXEC([at_ns1], [arp -s 10.1.1.1 e4:11:22:33:44:55])
> > +
> > +
> > +AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg 
> > ofproto_dpif_upcall:dbg])
> > +
> > +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> > +-- --id=@ipfix create IPFIX targets=\"127.0.0.1:4739\" 
> > \
> > +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> > ipfix=@ipfix, local_sample_group=10 \
> > +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> > ipfix=@ipfix, local_sample_group=12],
> > + [0], [ignore])
>
> Why are we adding IPFIX?  This should work without IPFIX.
>

It is intentional. I want to test both features work simulteaneously.

> Having both together can be a separate test.
>
> > +
> > +AT_DATA([flows.txt], [dnl
> > +in_port=ovs-p0,ip 
> > actions=sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918),output(port=ovs-p1,max_len=100)
> > +in_port=ovs-p1,ip 
> > actions=sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377),output(port=ovs-p0,max_len=100)
>
> It should be possible to wrap these lines a little with dnl.
>
> Also, please make IDs non-symmetric, so we can't check that the
> byte order is correct.
>

Originally, I thought just concatenating the values in native endianness
as a kind of replacement for the user_action_cookie (hence the symmetric
values). But I'm having second thoughts. I plan to change it in the next
version and store it in "network" order. I will change the test to
verify it accordingly.

> > +])
> > +
> > +AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> > +
> > +OVS_DAEMONIZE([ovstest test-psample > psample.out], [psample1.pid])
>
> Should we wait for the test app to start listening?  i.e. there might be
> a race between the listener and the ping below.
>

I guess so.

> > +
> > +NS_CHECK_EXEC([at_ns0], [ping -q -c 1 10.1.1.2 | FORMAT_PING], [0], [dnl
> > +1 packets transmitted, 1 received, 0% packet loss, time 0ms
> > +])
>
> We may need to add -W to the ping.
>

Ack.

> > +
> > +m4_define([SAMPLE1], [group_id=0xa 
> > obs_domain=0x,obs_point=0x 
> > .*icmp.*nw_src=10.1.1.1,nw_dst=10.1.1.2])
> > +m4_define([SAMPLE2], [group_id=0xc 
> > obs_domain=0x,obs_point=0x 
> > .*icmp.*nw_src=10.1.1.2,nw_dst=10.1.1.1])
>
> May use m4_join to wrap these lines.
>
> > +AT_CHECK([grep -E 'SAMPLE1' psample.out >/dev/null])
> > +AT_CHECK([grep -E 'SAMPLE2' psample.out >/dev/null])
>
> 'grep -q' to avoid the outpu

Re: [ovs-dev] [RFC v2 9/9] tests: Test local sampling.

[Adrián Moreno]

On Mon, Jun 24, 2024 at 01:23:55PM GMT, Eelco Chaudron wrote:
>
>
> On 5 Jun 2024, at 22:23, Adrian Moreno wrote:
>
> > Test simultaneous IPFIX and local sampling including slow-path.
> >
> > Signed-off-by: Adrian Moreno 
> > ---
> >  tests/system-common-macros.at |   4 ++
> >  tests/system-traffic.at   | 105 ++
> >  2 files changed, 109 insertions(+)
> >
> > diff --git a/tests/system-common-macros.at b/tests/system-common-macros.at
> > index 2a68cd664..22b8885e4 100644
> > --- a/tests/system-common-macros.at
> > +++ b/tests/system-common-macros.at
> > @@ -378,3 +378,7 @@ m4_define([OVS_CHECK_GITHUB_ACTION],
> >  # OVS_CHECK_DROP_ACTION()
> >  m4_define([OVS_CHECK_DROP_ACTION],
> >  [AT_SKIP_IF([! grep -q "Datapath supports explicit drop action" 
> > ovs-vswitchd.log])])
> > +
> > +# OVS_CHECK_EMIT_SAMPLE()
> > +m4_define([OVS_CHECK_EMIT_SAMPLE],
> > +[AT_SKIP_IF([! grep -q "Datapath supports emit_sample" 
> > ovs-vswitchd.log])])
> > diff --git a/tests/system-traffic.at b/tests/system-traffic.at
> > index bd7647cbe..babc56b56 100644
> > --- a/tests/system-traffic.at
> > +++ b/tests/system-traffic.at
> > @@ -8977,3 +8977,108 @@ OVS_WAIT_UNTIL([cat p2.pcap | grep -E "0x0050: 
> > * * *5002 *2000 *b85e *00
> >
> >  OVS_TRAFFIC_VSWITCHD_STOP
> >  AT_CLEANUP
> > +
> > +AT_SETUP([emit_sample])
> > +OVS_TRAFFIC_VSWITCHD_START()
> > +OVS_CHECK_EMIT_SAMPLE()
> > +
> > +ADD_NAMESPACES(at_ns0, at_ns1)
> > +NS_CHECK_EXEC([at_ns0], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], [0], 
> > [ignore])
> > +NS_CHECK_EXEC([at_ns1], [sysctl -w net.ipv6.conf.all.disable_ipv6=1], [0], 
> > [ignore])
> > +
> > +ADD_VETH(p0, at_ns0, br0, "10.1.1.1/24", "e4:11:22:33:44:55")
> > +ADD_VETH(p1, at_ns1, br0, "10.1.1.2/24", "e4:11:22:33:44:66")
> > +
> > +NS_CHECK_EXEC([at_ns0], [arp -s 10.1.1.2 e4:11:22:33:44:66])
> > +NS_CHECK_EXEC([at_ns1], [arp -s 10.1.1.1 e4:11:22:33:44:55])
> > +
> > +
> > +AT_CHECK([ovs-appctl vlog/set dpif:dbg dpif_netdev:dbg 
> > ofproto_dpif_upcall:dbg])
> > +
> > +AT_CHECK([ovs-vsctl -- --id=@br0 get Bridge br0 \
> > +-- --id=@ipfix create IPFIX targets=\"127.0.0.1:4739\" 
> > \
> > +-- create Flow_Sample_Collector_Set id=1 bridge=@br0 
> > ipfix=@ipfix, local_sample_group=10 \
> > +-- create Flow_Sample_Collector_Set id=2 bridge=@br0 
> > ipfix=@ipfix, local_sample_group=12],
> > + [0], [ignore])
> > +
> > +AT_DATA([flows.txt], [dnl
> > +in_port=ovs-p0,ip 
> > actions=sample(probability=65535,collector_set_id=1,obs_domain_id=1431655765,obs_point_id=1717986918),output(port=ovs-p1,max_len=100)
> > +in_port=ovs-p1,ip 
> > actions=sample(probability=65535,collector_set_id=2,obs_domain_id=2290649224,obs_point_id=2576980377),output(port=ovs-p0,max_len=100)
> > +])
> > +
> > +AT_CHECK([ovs-ofctl add-flows br0 flows.txt])
> > +
> > +OVS_DAEMONIZE([ovstest test-psample > psample.out], [psample1.pid])
> > +
> > +NS_CHECK_EXEC([at_ns0], [ping -q -c 1 10.1.1.2 | FORMAT_PING], [0], [dnl
> > +1 packets transmitted, 1 received, 0% packet loss, time 0ms
> > +])
> > +
> > +m4_define([SAMPLE1], [group_id=0xa 
> > obs_domain=0x,obs_point=0x 
> > .*icmp.*nw_src=10.1.1.1,nw_dst=10.1.1.2])
> > +m4_define([SAMPLE2], [group_id=0xc 
> > obs_domain=0x,obs_point=0x 
> > .*icmp.*nw_src=10.1.1.2,nw_dst=10.1.1.1])
> > +AT_CHECK([grep -E 'SAMPLE1' psample.out >/dev/null])
> > +AT_CHECK([grep -E 'SAMPLE2' psample.out >/dev/null])
> > +
> > +AT_CHECK([ovs-ofctl dump-ipfix-flow br0 | sed 's/tx pkts=[[0-9]]*/tx 
> > pkts=24/' | sed 's/tx errs=[[0-9]]*/tx errs=0/'], [0], [dnl
> > +NXST_IPFIX_FLOW reply (xid=0x2): 2 ids
> > +  id   1: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, 
> > tx pkts=24
> > +  pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
> > +  id   2: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, 
> > tx pkts=24
> > +  pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
> > +])
>
> This is not a review, just a heads up so you are aware that this test fails 
> sometimes:
>
> +++ 
> /home/echaudron/Documents/review/ovs_adrian_psample/tests/system-kmod-testsuite.dir/at-groups/178/stdout
>   2024-06-24 13:05:35.05700 +0200
> @@ -1,6 +1,6 @@
>  NXST_IPFIX_FLOW reply (xid=0x2): 2 ids
> -  id   1: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, tx 
> pkts=24
> -  pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
>id   2: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, tx 
> pkts=24
>pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
> +  id   1: flows=1, current flows=0, sampled pkts=1, ipv4 ok=1, ipv6 ok=0, tx 
> pkts=24
> +  pkts errs=0, ipv4 errs=0, ipv6 errs=0, tx errs=0
>
> The order does not seem to be fixed all the time (same for the one below).
>

Ugh, I assumed the output would be ordered. I'll deal with it manually
in the test.

> //Eelco
>
> > +
> > +AT_CHECK([ovs-