Re: [ovs-dev] [PATCH v10] netdev-dpdk: add control plane protection support
Kevin Traynor, May 24, 2023 at 16:06: > Hmm, not sure on this one. I have a feeling that having a 'hash' mode > for tx-steering that only applies to vhost devices, and 'hash' mode > for rx-steering that only applies to NICs means people will miss the > subtlety and try to enable the wrong hash mode on the wrong device :-) > So 'rss' might make it more distinct. "rss" is fine then. > Any reason for '+' ? I think commas are used more frequently. If it > needed to be extended in future for some extra config, then adding > 'key:value' pairs would be seamlessly. e.g. =rss:, > lacp: The "rss" item should be mandatory anyway. There should be no way to disable it. I assume that it is what Ilya meant with these "+" symbols. That would leave us with these examples: # lacp+bfd on separate queue, rss on other queues options:rx-steering=rss,lacp,bfd # same options:rx-steering=lacp,bfd,rss # only regular rss, same as no config at all options:rx-steering=rss # invalid configurations options:rx-steering=lacp options:rx-steering= options:rx-steering=bfd,lacp What do you think? ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v10] netdev-dpdk: add control plane protection support
Ilya Maximets, May 23, 2023 at 22:04:
> 'rxq-isolate' will be confused with 'other_config:pmd-rxq-isolate'.
> Same likely goes for the 'isolated-rxq'.
>
> 'rxq-steernig' may be confused with 'other_config:tx-steering'.
> But this can be argued that it's essentially similar functionality,
> so should be named similarly. Maybe we can double down on that and
> use options:rx-steernig=rss|rss+lacp|... with 'rss' being a default
> configuration?
I like that idea. Instead of "rss", how about "hash" to match what
tx-steering is using?
So the value of options:rx-steering would be "hash" followed by an
arbitrary list of preset protocols (for now, only "lacp") all separated
by "+". It may also open the door for other base steering methods than
"hash" ("rr" for round-robin some day?).
If that is OK, I can prepare a v11 with everything renamed.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v10] netdev-dpdk: add control plane protection support
Aaron Conole, May 23, 2023 at 15:32: > I think one issue I have with this is that the name is a bit > misleading. Control plane, from OVS perspective, would be like OpenFlow > communications. This is more like a traffic steering mechanism. > > Maybe it would help to call it something like "traffic-based-rps" or > something like that (but not clear what would be best). It is really a > way to steer specific traffic to a distinct rxq. > > WDYT? Actually, "packet-steering" was one of the first ideas I had for this feature, but I thought it may be confusing. I am weary of reusing the "rps" acronym as it may be confused with a linux specific feature: https://docs.kernel.org/networking/scaling.html#rps-receive-packet-steering The feature introduced in this patch is relying on RTE flow which has nothing to do with Linux. However, I agree that the name should reflect that we are steering traffic into a dedicated rxq. How about these ideas? options:isolated-rxq=lacp,... options:rxq-isolate=lacp,... options:rxq-steering=lacp,... I personally prefer "rxq-isolate". ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v10] netdev-dpdk: add control plane protection support
r, it does not work by
limiting the rate of traffic explicitly. It only guarantees that some
protocols have a lower chance of being dropped because the PMD cores
cannot keep up with regular traffic.
The choice of protocols is limited on purpose. This is not meant to be
configurable by users. Some limited configurability could be considered
in the future but it would expose to more potential issues if users are
accidentally redirecting all traffic in the control plane queue.
Cc: Anthony Harivel
Cc: Christophe Fontaine
Cc: David Marchand
Cc: Kevin Traynor
Signed-off-by: Robin Jarry
---
v9 -> v10:
* If hw-offload is enabled, cp-protection will now be forcibly disabled
on all ports.
Documentation/topics/dpdk/phy.rst | 83
NEWS | 3 +
lib/netdev-dpdk.c | 310 +-
vswitchd/vswitch.xml | 27 +++
4 files changed, 421 insertions(+), 2 deletions(-)
diff --git a/Documentation/topics/dpdk/phy.rst
b/Documentation/topics/dpdk/phy.rst
index 4b0fe8dded3a..733c4251bca9 100644
--- a/Documentation/topics/dpdk/phy.rst
+++ b/Documentation/topics/dpdk/phy.rst
@@ -131,6 +131,89 @@ possible with DPDK acceleration. It is possible to
configure multiple Rx queues
for ``dpdk`` ports, thus ensuring this is not a bottleneck for performance. For
information on configuring PMD threads, refer to :doc:`pmd`.
+Control Plane Protection
+
+
+.. warning:: This feature is experimental.
+
+Some control protocols are used to maintain link status between forwarding
+engines. In SDN environments, these packets share the same physical network
+than the user data traffic.
+
+When the system is not sized properly, the PMD threads may not be able to
+process all incoming traffic from the configured Rx queues. When a signaling
+packet of such protocols is dropped, it can cause link flapping, worsening the
+situation.
+
+Some physical NICs can be programmed to put these protocols in a dedicated
+hardware Rx queue using the rte_flow__ API.
+
+__ https://doc.dpdk.org/guides-22.11/prog_guide/rte_flow.html
+
+The currently supported control plane protocols are:
+
+``lacp``
+ `Link Aggregation Control Protocol`__. Ether type ``0x8809``.
+
+ __ https://www.ieee802.org/3/ad/public/mar99/seaman_1_0399.pdf
+
+.. warning::
+
+ This feature is not compatible with all NICs. Refer to the DPDK
+ `compatibilty matrix`__ and vendor documentation for more details.
+
+ __ https://doc.dpdk.org/guides-22.11/nics/overview.html
+
+Control plane protection must be enabled on specific protocols per port. The
+``cp-protection`` option requires a coma separated list of protocol names::
+
+ $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \
+options:dpdk-devargs=:01:00.0 options:n_rxq=2 \
+options:cp-protection=lacp
+
+.. note::
+
+ If multiple Rx queues are already configured, regular RSS (Receive Side
+ Scaling) queue balancing is done on all but the extra control plane
+ protection queue.
+
+.. tip::
+
+ You can check if control plane protection is supported on a port with the
+ following command::
+
+ $ ovs-vsctl get interface dpdk-p0 status
+ {cp_protection_queue="2", driver_name=..., rss_queues="0-1"}
+
+ This will also show in ``ovs-vswitchd.log``::
+
+ INFO|dpdk-p0: cp-protection: redirecting lacp traffic to queue 2
+ INFO|dpdk-p0: cp-protection: applying rss on queues 0-1
+
+ If the hardware does not support redirecting control plane traffic to
+ a dedicated queue, it will be explicit::
+
+ $ ovs-vsctl get interface dpdk-p0 status
+ {cp_protection=unsupported, driver_name=...}
+
+ More details can often be found in ``ovs-vswitchd.log``::
+
+ WARN|dpdk-p0: cp-protection: failed to add lacp flow: Unsupported pattern
+
+To disable control plane protection on a port, use the following command::
+
+ $ ovs-vsctl remove Interface dpdk-p0 options cp-protection
+
+You can see that it has been disabled in ``ovs-vswitchd.log``::
+
+ INFO|dpdk-p0: cp-protection: disabled
+
+.. warning::
+
+ This feature is mutually exclusive with ``other_options:hw-offload`` as it
+ may conflict with the offloaded RTE flows. If both are enabled,
+ ``cp-protection`` will be forcibly disabled.
+
.. _dpdk-phy-flow-control:
Flow Control
diff --git a/NEWS b/NEWS
index b6418c36e95e..43338edaf5aa 100644
--- a/NEWS
+++ b/NEWS
@@ -21,6 +21,9 @@ Post-v3.1.0
* ovs-vswitchd will keep the CAP_SYS_RAWIO capability when started
with the --hw-rawio-access command line option. This allows the
process extra privileges when mapping physical interconnect memory.
+ * New experimental "cp-protection=" option to redirect certain
+ protocols (for now, only LACP) to a dedicated hardware queue using
+ RTE flow.
- SRv6 Tunnel Protocol
* Added support for userspace datapath (only).
diff --git
Re: [ovs-dev] [PATCH v9] netdev-dpdk: add control plane protection support
Kevin Traynor, Mar 23, 2023 at 15:27: > Hi Robin, > > Regarding having both features enabled, I think it's an issue that it's > chronological based if they are enabled while running. It introduces > another variable that might confuse things. > > For example, the operation could be changed from cp-proto to hw-offload > on a port by restarting OVS, which would probably be unexpected by a > user. I mentioned it while chatting to Ilya and he agreed that same > state in ovsdb should mean same state in ovs-vswitchd. > > So that would mean having a binary priority between the two features and > removing one if the higher priority one was later enabled (either > globally or per-port?). > > Whatever the co-existance (or not) is, I think it's better to resolve it > in mail first to avoid you having to rework code over again. I don't > think it needs to be super-smart as these are experimental features, > just needs to be consistent and clearly documented for the user. > > Code wise, I've tested previous versions and I think the code is in > pretty good shape overall. I'll do another pass review/testing when the > hwol/cp-prot prio is resolved. > > thanks, > Kevin. Hi Kevin, sorry not to have replied earlier, I got caught up in other issues :) I agree that having a deterministic priority between rte flow offload and control plane protection is a must have. However, I am not sure how to implement it in the current state of things. The main issue is that cp-protection is dpdk specific whereas hw-offload is in the abstract netdev layer. There is no way to check the state of cp-protection from netdev-offload.c. Maybe I could expose a minimal generic API in netdev.h to determine if hw-offload can be enabled for a specific device or not. And implement it for dpdk, based on the value of cp-protection. What do you think? ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v9] netdev-dpdk: add control plane protection support
Hi Aaron, Thanks for your feedback. Aaron Conole, Mar 07, 2023 at 19:57: > I'm concerned about this - this is a negative interference with rte_flow > offload. And rte_flow offload would also just alleviate these problems, > yes? Maybe we should encourage a user to just turn on flow offloading? Indeed this feature will not get along with rte_flow offload, but as you said, if your hardware and flow pipeline are compatible with rte_flow offload, then, you probably don't need this feature since your PMD threads will only handle a limited portion of the data plane traffic. The CP protection flows are very basic (for LACP, only matching the ether type, no VLAN support needed, etc.) and they should be supported by a broad range of NICs (even Intel 82599 Niantic from 2013). This feature is mostly aimed at pure DPDK (without extended rte_flow offload support) where LACP is mixed in the same queues and processed by the same CPUs than data plane traffic. > Additionally, it doesn't seem to have a good solution for kernel side. For non-DPDK use cases, I see two separate paths. Please do correct me if I am wrong: 1) pure kernel (not tested, only from ethtool(8)) ethtool -L $dev rx $((rxqs + 1)) ethtool -X $dev equal $((rxqs - 1)) ethtool -U $dev flow-type ether proto 0x8809 m 0x action $rxqs 2) tc flow offload Only mlx5 supported hardware is concerned and there should be direct support for hardware bond offload (LACP handled in the NIC). > And I worry about adding flows into the system that the ofproto layer > doesn't know about but will interact with - but maybe I'm just being > paranoid. There should be no overlap with the ofproto layer. The CP protection flows only route certain packets into certain RXQs. The packets are still processed by the same pipeline. I understand your reservations as this involves intrusive elements in the packet flow and does not play well with RTE flow offload. However, actual openflow pipelines (like the ones in OVN deployments) are supported in RTE flow by a limited and recent set of hardware. Users with existing deployments and older hardware cannot benefit from this. This feature could allow making these environments more resilient and less prone to link flapping. Also, given that cp-protection is disabled by default, there will be no conflict with DPDK hw-offload when there is hardware support. What do you think? ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [RFC] dpdk: Allow retaining cap_sys_rawio privileges
Salut Gaëtan,
Gaetan Rivet, Feb 23, 2023 at 22:33:
> I've looked at your patch Robin and the offloads you insert in
> dpdk_cp_prot_add_flow use the following:
>
> const struct rte_flow_attr attr = { .ingress = 1 };
>
> implicitly setting transfer and group to 0. If either of those had
> been non-zero instead, cap_sys_rawio would be required.
Oh I was not aware that this would change anything. Is there some
document/code snippet/anything that explains why is that so? Is that
specific to the mlx5 driver?
Thanks!
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [RFC] dpdk: Allow retaining cap_sys_rawio privileges
Aaron Conole, Feb 23, 2023 at 22:09: > Thanks for taking a look. You're saying that you tested without this > patch applied, yes? That could be. I only know of one hardware which > requires CAP_SYS_RAWIO for rte_flow to function. Yes that is correct, I tested *without* this patch applied and with a non-root user (ovs-vswitchd linked with libcap-ng). ovs-ctl --ovs-user="openvswitch:hugetlbfs" start The basic RTE flow rules (matching of the ether type field and redirect to a specific queue) were created without errors returned with both NICs I had available (Intel X710 and Mellanox ConnectX-5 Ex) cp-protection: redirected lacp traffic to rx queue 1 cp-protection: redirected other traffic to rx queue 0 ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v9] netdev-dpdk: add control plane protection support
For the record, I have tested this feature as a non-privileged user: ovs-ctl --ovs-user="openvswitch:hugetlbfs" start With the proper UDEV rules for vfio based devices, I have successfully configured cp-protection=lacp on i40e (Intel X710 NICs) and mlx5 (ConnectX-5 Ex NICs) drivers. cp-protection: redirected lacp traffic to rx queue 1 cp-protection: redirected other traffic to rx queue 0 ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [RFC] dpdk: Allow retaining cap_sys_rawio privileges
Aaron Conole, Feb 22, 2023 at 18:07: > Open vSwitch generally tries to let the underlying operating system > managed the low level details of hardware, for example DMA mapping, > bus arbitration, etc. However, when using DPDK, the underlying > operating system yields control of many of these details to userspace > for management. > > In the case of some DPDK port drivers, configuring rte_flow or even > allocating resources may require access to iopl/ioperm calls, which > are guarded by the CAP_SYS_RAWIO privilege on linux systems. These > calls are dangerous, and can allow a process to completely compromise > a system. However, they are needed in the case of some userspace > driver code which manages the hardware (for example, the mlx > implementation of backend support for rte_flow). > > Here, we create an opt-in flag passed to the command line to allow > this access. We need to do this before ever accessing the database, > because we want to drop all privileges asap, and cannot wait for > a connection to the database to be established and functional before > dropping. There may be distribution specific ways to do capability > management as well (using for example, systemd), but they are not > as universal to the vswitchd as a flag. > > Signed-off-by: Aaron Conole Hi Aaron, I briefly tested the injection of a basic RTE flow rule (see my control plane protection patch here for more details[1]). With a non-root user, there are no permission issues that I can see. I have tested with both the i40e (X710) and mlx5 (ConnectX-5 Ex) drivers. Maybe CAP_SYS_RAWIO is only required for more advanced flows but I am surprised that I didn't encounter the issue for neither a vfio-pci based driver nor with a bifurcated driver. [1]: http://patchwork.ozlabs.org/project/openvswitch/patch/20230222154321.23421-1-rja...@redhat.com/ ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v9] netdev-dpdk: add control plane protection support
ture may be considered as "QoS". However, it does not work by
limiting the rate of traffic explicitly. It only guarantees that some
protocols have a lower chance of being dropped because the PMD cores
cannot keep up with regular traffic.
The choice of protocols is limited on purpose. This is not meant to be
configurable by users. Some limited configurability could be considered
in the future but it would expose to more potential issues if users are
accidentally redirecting all traffic in the control plane queue.
Cc: Anthony Harivel
Cc: Christophe Fontaine
Cc: David Marchand
Cc: Kevin Traynor
Signed-off-by: Robin Jarry
---
v8 -> v9:
* Rebased on cf288fdfe2bf ("AUTHORS: Add Liang Mancang and Viacheslav
Galaktionov.")
* Reset rte_flow_error struct before passing it to functions.
* Refined some comments.
* Updated check for hw-offload on a per-port basis. That way, if a port
already has cp-protection enabled, hw-offload will not be enabled on
it but cp-protection will continue to work until next restart.
However, On next restart, hw-offload will be checked first and
therefore cp-protection will be disabled on all ports.
Unless there are significant reserves about this patch. Would it be ok
to include it for 3.2?
Thanks!
Documentation/topics/dpdk/phy.rst | 77
NEWS | 4 +
lib/netdev-dpdk.c | 310 +-
vswitchd/vswitch.xml | 26 +++
4 files changed, 414 insertions(+), 3 deletions(-)
diff --git a/Documentation/topics/dpdk/phy.rst
b/Documentation/topics/dpdk/phy.rst
index 4b0fe8dded3a..518b67134639 100644
--- a/Documentation/topics/dpdk/phy.rst
+++ b/Documentation/topics/dpdk/phy.rst
@@ -131,6 +131,83 @@ possible with DPDK acceleration. It is possible to
configure multiple Rx queues
for ``dpdk`` ports, thus ensuring this is not a bottleneck for performance. For
information on configuring PMD threads, refer to :doc:`pmd`.
+Control Plane Protection
+
+
+.. warning:: This feature is experimental.
+
+Some control protocols are used to maintain link status between forwarding
+engines. In SDN environments, these packets share the same physical network
+than the user data traffic.
+
+When the system is not sized properly, the PMD threads may not be able to
+process all incoming traffic from the configured Rx queues. When a signaling
+packet of such protocols is dropped, it can cause link flapping, worsening the
+situation.
+
+Some physical NICs can be programmed to put these protocols in a dedicated
+hardware Rx queue using the rte_flow__ API.
+
+__ https://doc.dpdk.org/guides-22.11/prog_guide/rte_flow.html
+
+The currently supported control plane protocols are:
+
+``lacp``
+ `Link Aggregation Control Protocol`__. Ether type ``0x8809``.
+
+ __ https://www.ieee802.org/3/ad/public/mar99/seaman_1_0399.pdf
+
+.. warning::
+
+ This feature is not compatible with all NICs. Refer to the DPDK
+ `compatibilty matrix`__ and vendor documentation for more details.
+
+ __ https://doc.dpdk.org/guides-22.11/nics/overview.html
+
+Control plane protection must be enabled on specific protocols per port. The
+``cp-protection`` option requires a coma separated list of protocol names::
+
+ $ ovs-vsctl add-port br0 dpdk-p0 -- set Interface dpdk-p0 type=dpdk \
+options:dpdk-devargs=:01:00.0 options:n_rxq=2 \
+options:cp-protection=lacp
+
+.. note::
+
+ If multiple Rx queues are already configured, regular RSS (Receive Side
+ Scaling) queue balancing is done on all but the extra control plane
+ protection queue.
+
+.. tip::
+
+ You can check if control plane protection is supported on a port with the
+ following command::
+
+ $ ovs-vsctl get interface dpdk-p0 status
+ {cp_protection_queue="2", driver_name=..., rss_queues="0-1"}
+
+ This will also show in ``ovs-vswitchd.log``::
+
+ INFO|dpdk-p0: cp-protection: redirecting lacp traffic to queue 2
+ INFO|dpdk-p0: cp-protection: applying rss on queues 0-1
+
+ If the hardware does not support redirecting control plane traffic to
+ a dedicated queue, it will be explicit::
+
+ $ ovs-vsctl get interface dpdk-p0 status
+ {cp_protection=unsupported, driver_name=...}
+
+ More details can often be found in ``ovs-vswitchd.log``::
+
+ WARN|dpdk-p0: cp-protection: failed to add lacp flow: Unsupported pattern
+
+To disable control plane protection on a port, use the following command::
+
+ $ ovs-vsctl remove Interface dpdk-p0 options cp-protection
+
+You can see that it has been disabled in ``ovs-vswitchd.log``::
+
+ INFO|dpdk-p0: cp-protection: disabled
+
.. _dpdk-phy-flow-control:
Flow Control
diff --git a/NEWS b/NEWS
index 85b34962145e..22505ea9d4ad 100644
--- a/NEWS
+++ b/NEWS
@@ -10,6 +10,10 @@ Post-v3.1.0
in order to create OVSDB sockets with access mode of 0770.
- QoS:
* Added new configuration option 'jit
Bug#1028779: buildbot: FTBFS: dh_auto_test: error: pybuild --test -i python{version} -p 3.11 --system=custom "--test-args=PYTHONPATH=pkg:{destdir}/{install_dir} PATH=\$PATH:{destdir}/usr/bin trial3 --
Nilesh Patra, Feb 05, 2023 at 12:12: Hi Robin, buildbot is marked for removal on Feb 16. Do you intend to make an upload? Hi Nilesh, sorry I had forgotten about that. It looks like Bastien has uploaded 3.7.0-1 with a fix.
Bug#1028779: buildbot: FTBFS: dh_auto_test: error: pybuild --test -i python{version} -p 3.11 --system=custom "--test-args=PYTHONPATH=pkg:{destdir}/{install_dir} PATH=\$PATH:{destdir}/usr/bin trial3 --
Nilesh Patra, Feb 05, 2023 at 12:12: Hi Robin, buildbot is marked for removal on Feb 16. Do you intend to make an upload? Hi Nilesh, sorry I had forgotten about that. It looks like Bastien has uploaded 3.7.0-1 with a fix.
Re: [ovs-dev] [PATCH v8] netdev-dpdk: add control plane protection support
Hi Kevin,
Kevin Traynor, Jan 17, 2023 at 19:38:
> I saw in patchwork that there was a CI fail but I didn't examine it.
> Perhaps you could check and confirm about it.
Yes I saw it and I didn't understand what went wrong:
2023-01-13T15:44:29.422Z|1|dpif_netdev(revalidator75)|ERR|internal error
parsing flow key
skb_priority(0),skb_mark(0),ct_state(0),ct_zone(0),ct_mark(0),ct_label(0),recirc_id(0),dp_hash(0),in_port(2),packet_type(ns=0,id=0),eth(src=5a:51:ee:7c:af:0d,dst=33:33:ff:7c:af:0d),eth_type(0x88a8),vlan(vid=4094,pcp=0),encap(eth_type(0x8100),vlan(vid=100,pcp=0),encap(eth_type(0x86dd),ipv6(src=::,dst=ff02::1:ff7c:af0d,label=0,proto=58,tclass=0,hlimit=255,frag=no),icmpv6(type=135,code=0),nd(target=fe80::5851:eeff:fe7c:af0d,sll=00:00:00:00:00:00,tll=00:00:00:00:00:00)))
2023-01-13T15:44:29.422Z|2|dpif(revalidator75)|WARN|netdev at ovs-netdev:
failed to put[modify] (Invalid argument)
ufid:8ad5a583-080e-4333-9645-095cc328f6c3
skb_priority(0/0),skb_mark(0/0),ct_state(0/0),ct_zone(0/0),ct_mark(0/0),ct_label(0/0),recirc_id(0),dp_hash(0/0),in_port(2),packet_type(ns=0,id=0),eth(src=5a:51:ee:7c:af:0d,dst=33:33:ff:7c:af:0d),eth_type(0x88a8),vlan(vid=4094,pcp=0/0x0),encap(eth_type(0x8100),vlan(vid=100/0x0,pcp=0/0x0),encap(eth_type(0x86dd),ipv6(src=::/::,dst=ff02::1:ff7c:af0d/::,label=0/0,proto=58/0,tclass=0/0,hlimit=255/0,frag=no),icmpv6(type=135/0,code=0/0),nd(target=fe80::5851:eeff:fe7c:af0d/::,sll=00:00:00:00:00:00/00:00:00:00:00:00,tll=00:00:00:00:00:00/00:00:00:00:00:00))),
actions:drop
151. system-traffic.at:7110: 151. 802.1ad - vlan_limit
(system-traffic.at:7110): FAILED (system-traffic.at:7135)
https://mail.openvswitch.org/pipermail/ovs-build/2023-January/027852.html
It does not seem related to this patch but I could be wrong.
> > It cannot be enabled when other_config:hw-offload=true as it may
> > conflict with the offloaded RTE flows. Similarly, if hw-offload is
> > enabled while some ports already have cp-protection enabled, RTE
> > flow offloading will be disabled on these ports.
>
> I'm not sure if you are referring the features hw-offload and/or
> cp-prot when you say RTE flow offloading will be disabled on these
> ports.
>
> What I see is that if cp-prot is enabled on a port and then hw-offload
> is enabled by the user, hw-offload will not enable because of cp-prot
> AND cp-prot will also be disabled by virtue the user trying to enable
> hw-offload.
>
> I was not expecting that last part, even if they are mutually
> exclusive features I thought one would work. Perhaps that is
> intentional and what you meant above, I wasn't 100% clear.
This was not intentional. To be honest, I did not run extensive tests
with hw-offload. I will have a look at this use case and see if I can at
least leave cp-protection enabled.
It may not be possible given that hw-offload is not enabled immediately
upon the ovsdb update but asynchronously. Also, since cp-protection is
a dpdk specific feature, and hw-offload is dpif-netdev, I'm not sure if
there is a way to check anything when the user enables hw-offload.
> Anyway, I understand this is just experimental and there is a lot of
> scenarios to cover between the features (e.g. what if one of the
> features could not rte_flow_validate(), could the other be used then?)
>
> I think you would need to them tease out in the future and if they
> stay mutually exclusive that there is a clear precedence either by
> feature or order enabled etc. You might even need to put a table in
> docs to show the user.
I think that making both features work without interference would not be
easy. We would need a way to ensure that the cp-protection flow happens
before the RSS flow that is inserted by hw-offload when a datapath flow
cannot be offloaded because of missing matcher/action support in hw.
That could be done via flow priorities/tables but these are only
supported by a very limited number of NICs. That RSS flow would also
need to be altered to only redirect on RXQs that are not reserved by the
cp-protection flow.
> > +/* User input for n_rxq (see netdev_dpdk_reconfigure). */
>
> This comment is nice. minor: 'requested_n_rxq' could get a comment now
> that it is no longer the user requested value and isn't the same as
> the other requested_* members anymore.
Sure that makes sense.
> I have seen that ixgbe sets the error struct when returning success
> which seems a violation of the rte_flow_validation/create API ("PMDs
> initialize this structure in case of error only".)
>
> It's ok because you rightly ignore the error message, but it got me
> thinking that we should reset/validate the error messages. See below.
>
> +set_error(, RTE_FLOW_ERROR_TYPE_NONE);
Ack, I will add this.
Thanks for the review.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v8] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between forwarding engines (e.g. LACP). When the system is not sized properly, the PMD threads may not be able to process all incoming traffic from the configured Rx queues. When a signaling packet of such protocols is dropped, it can cause link flapping, worsening the situation. Use the RTE flow API to redirect these protocols into a dedicated Rx queue. The assumption is made that the ratio between control protocol traffic and user data traffic is very low and thus this dedicated Rx queue will never get full. The RSS redirection table is re-programmed to only use the other Rx queues. The RSS table size is stored in the netdev_dpdk structure at port initialization to avoid requesting the information again when changing the port configuration. The additional Rx queue will be assigned a PMD core like any other Rx queue. Polling that extra queue may introduce increased latency and a slight performance penalty at the benefit of preventing link flapping. This feature must be enabled per port on specific protocols via the cp-protection option. This option takes a coma-separated list of protocol names. It is only supported on ethernet ports. This feature is experimental. If the user has already configured multiple Rx queues on the port, an additional one will be allocated for control plane packets. If the hardware cannot satisfy the requested number of requested Rx queues, the last Rx queue will be assigned for control plane. If only one Rx queue is available, the cp-protection feature will be disabled. If the hardware does not support the RTE flow matchers/actions, the feature will be disabled. It cannot be enabled when other_config:hw-offload=true as it may conflict with the offloaded RTE flows. Similarly, if hw-offload is enabled while some ports already have cp-protection enabled, RTE flow offloading will be disabled on these ports. Example use: ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \ set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \ set interface phy0 options:cp-protection=lacp -- \ set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \ set interface phy1 options:cp-protection=lacp As a starting point, only one protocol is supported: LACP. Other protocols can be added in the future. NIC compatibility should be checked. To validate that this works as intended, I used a traffic generator to generate random traffic slightly above the machine capacity at line rate on a two ports bond interface. OVS is configured to receive traffic on two VLANs and pop/push them in a br-int bridge based on tags set on patch ports. +--+ | DUT | |++| || br-int || default flow, action=NORMAL |||| || patch10patch11 || |+---|---|+| || | | |+---|---|+| || patch00patch01 || || tag:10tag:20 || |||| || br-phy || default flow, action=NORMAL |||| || bond0|| balance-slb, lacp=passive, lacp-time=fast ||phy0 phy1 || |+--|-|---+| +---|-|+ | | +---|-|+ | port0 port1 | balance L3/L4, lacp=active, lacp-time=fast | lag | mode trunk VLANs 10, 20 | | |switch| | | | vlan 10vlan 20 | mode access | port2 port3 | +-|--|-+ | | +-|--|-+ | port0 port1 | Random traffic that is properly balanced | | across the bond ports in both directions. | traffic generator | +--+ Without cp-protection, the bond0 links are randomly switching to "defaulted" when one of the LACP packets sent by the switch is dropped because the RX queues are full and the PMD threads did not process them fast enough. When that happens, all traffic must go through a single link which causes above line rate traffic to be dropped. When cp-protection is enabled, no LACP packet is dropped and the bond links remain enabled at all times, maximizing the throughput. This feature may be considered as "QoS". However, it does not work by limiting the rate of traffic explicitly. It only guarantees that some protocols have a lower chance of being dropped because the PMD cores cannot keep up with regular traffic. The choice of protocols is limited on purpose. This is not meant to be configurable by users. Some limited configurability could be considered in the future but it would expose to more potential issues if users are accidentally redirecting all traffic in the control plane queue. Cc: Christophe Fontaine Cc: Kevin Traynor Cc: David Marchand Signed-off-by: Robin Jarry --- v6 -> v
Re: [ovs-dev] [PATCH v7] netdev-dpdk: add control plane protection support
Kevin Traynor, Jan 12, 2023 at 14:56: > > +(void) dpdk_cp_prot_rss_configure(dev, dev->up.n_rxq); > > I'm wondering is this needed? It is allowing traffic go to the extra > cp-proto queue that the user has not request for general traffic until > later when it is stopped. I don't see a need to use the extra queue > and I'm not sure that we should. e.g: > > # /root/ovs/utilities/ovs-vsctl set Interface myport options:n_rxq=4 > 2023-01-12T13:03:04Z|00175|netdev_dpdk|DBG|myport: cp-protection: reset flows > 2023-01-12T13:03:04Z|00176|netdev_dpdk|INFO|Port 0: 24:6e:96:71:fe:28 > 2023-01-12T13:03:04Z|00177|netdev_dpdk|INFO|myport: cp-protection: redirected > lacp traffic to rx queue 4 > 2023-01-12T13:03:04Z|00178|netdev_dpdk|INFO|myport: cp-protection: applied > rss on rx queue 0-3 > ... > # /root/ovs/utilities/ovs-vsctl remove interface myport options cp-protection > > 5368│ > 5369├>(void) dpdk_cp_prot_rss_configure(dev, dev->up.n_rxq); > 5370│ } > > dev->up.n_rxq = 5 This is weird, it should have been 4 when cp protection is disabled. I will have a closer look. > > +dev->requested_n_rxq = dev->user_n_rxq; > > +goto retry; > > A retry through this function is doing a double free on dev->tx > _q which gives an error message: > > 2023-01-12T12:27:13Z|00104|dpdk|ERR|EAL: Error: Invalid memory Indeed, I had missed that. I'll fix it for v8. Thanks. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v7] netdev-dpdk: add control plane protection support
Kevin Traynor, Jan 13, 2023 at 12:05:
> Hi Robin - not sure if you saw the test report from Intel:
> https://mail.openvswitch.org/pipermail/ovs-build/2023-January/027792.html
>
> There is a problem with adding vhost port, existing before cp-prot is
> enabled. see below,
>
> 5500│ /* Enable TX queue 0 by default if it wasn't disabled. */
> 5501├>if (dev->tx_q[0].map == OVS_VHOST_QUEUE_MAP_UNKNOWN) {
> 5502│ dev->tx_q[0].map = 0;
> 5503│ }
>
> (gdb) p dev->tx_q
> $1 = (struct dpdk_tx_queue *) 0x0
Damn, I'll fix that. Thanks for testing.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v7] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between forwarding engines (e.g. LACP). When the system is not sized properly, the PMD threads may not be able to process all incoming traffic from the configured Rx queues. When a signaling packet of such protocols is dropped, it can cause link flapping, worsening the situation. Use the RTE flow API to redirect these protocols into a dedicated Rx queue. The assumption is made that the ratio between control protocol traffic and user data traffic is very low and thus this dedicated Rx queue will never get full. The RSS redirection table is re-programmed to only use the other Rx queues. The RSS table size is stored in the netdev_dpdk structure at port initialization to avoid requesting the information again when changing the port configuration. The additional Rx queue will be assigned a PMD core like any other Rx queue. Polling that extra queue may introduce increased latency and a slight performance penalty at the benefit of preventing link flapping. This feature must be enabled per port on specific protocols via the cp-protection option. This option takes a coma-separated list of protocol names. It is only supported on ethernet ports. This feature is experimental. If the user has already configured multiple Rx queues on the port, an additional one will be allocated for control plane packets. If the hardware cannot satisfy the requested number of requested Rx queues, the last Rx queue will be assigned for control plane. If only one Rx queue is available, the cp-protection feature will be disabled. If the hardware does not support the RTE flow matchers/actions, the feature will be disabled. It cannot be enabled when other_config:hw-offload=true as it may conflict with the offloaded RTE flows. Similarly, if hw-offload is enabled while some ports already have cp-protection enabled, RTE flow offloading will be disabled on these ports. Example use: ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \ set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \ set interface phy0 options:cp-protection=lacp -- \ set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \ set interface phy1 options:cp-protection=lacp As a starting point, only one protocol is supported: LACP. Other protocols can be added in the future. NIC compatibility should be checked. To validate that this works as intended, I used a traffic generator to generate random traffic slightly above the machine capacity at line rate on a two ports bond interface. OVS is configured to receive traffic on two VLANs and pop/push them in a br-int bridge based on tags set on patch ports. +--+ | DUT | |++| || br-int || default flow, action=NORMAL |||| || patch10patch11 || |+---|---|+| || | | |+---|---|+| || patch00patch01 || || tag:10tag:20 || |||| || br-phy || default flow, action=NORMAL |||| || bond0|| balance-slb, lacp=passive, lacp-time=fast ||phy0 phy1 || |+--|-|---+| +---|-|+ | | +---|-|+ | port0 port1 | balance L3/L4, lacp=active, lacp-time=fast | lag | mode trunk VLANs 10, 20 | | |switch| | | | vlan 10vlan 20 | mode access | port2 port3 | +-|--|-+ | | +-|--|-+ | port0 port1 | Random traffic that is properly balanced | | across the bond ports in both directions. | traffic generator | +--+ Without cp-protection, the bond0 links are randomly switching to "defaulted" when one of the LACP packets sent by the switch is dropped because the RX queues are full and the PMD threads did not process them fast enough. When that happens, all traffic must go through a single link which causes above line rate traffic to be dropped. When cp-protection is enabled, no LACP packet is dropped and the bond links remain enabled at all times, maximizing the throughput. This feature may be considered as "QoS". However, it does not work by limiting the rate of traffic explicitly. It only guarantees that some protocols have a lower chance of being dropped because the PMD cores cannot keep up with regular traffic. The choice of protocols is limited on purpose. This is not meant to be configurable by users. Some limited configurability could be considered in the future but it would expose to more potential issues if users are accidentally redirecting all traffic in the control plane queue. Cc: Christophe Fontaine Cc: Kevin Traynor Cc: David Marchand Signed-off-by: Robin Jarry --- v6 ->
Re: [ovs-dev] [PATCH v3 2/2] dpif-netdev: Add load based PMD sleeping.
unt of sleep time was 9.15 seconds > and > +the average sleep time per iteration was 46 microseconds:: > + > + - sleep iterations: 153994 ( 76.8 % of iterations) > + Sleep time: 9159399 us ( 46 us/iteration avg.) > + > +.. note:: > + > +If there is a sudden spike of packets while the PMD thread is sleeping > and > +the processor is in a low-power state it may result in some lost packets > or > +extra latency before the PMD thread returns to processing packets at full > +rate. > + > +.. note:: > + > + Default Linux kernel hrtimer resolution is set to 50 microseconds so this > +will add overhead to requested sleep time. I wonder if it would make sense to round up to the nearest hrtimer resolution (if such info can be retrieved at runtime). Cheers, Reviewed-by: Robin Jarry ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v6] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between forwarding engines (e.g. LACP). When the system is not sized properly, the PMD threads may not be able to process all incoming traffic from the configured Rx queues. When a signaling packet of such protocols is dropped, it can cause link flapping, worsening the situation. Use the RTE flow API to redirect these protocols into a dedicated Rx queue. The assumption is made that the ratio between control protocol traffic and user data traffic is very low and thus this dedicated Rx queue will never get full. The RSS redirection table is re-programmed to only use the other Rx queues. The RSS table size is stored in the netdev_dpdk structure at port initialization to avoid requesting the information again when changing the port configuration. The additional Rx queue will be assigned a PMD core like any other Rx queue. Polling that extra queue may introduce increased latency and a slight performance penalty at the benefit of preventing link flapping. This feature must be enabled per port on specific protocols via the cp-protection option. This option takes a coma-separated list of protocol names. It is only supported on ethernet ports. This feature is experimental. If the user has already configured multiple Rx queues on the port, an additional one will be allocated for control plane packets. If the hardware cannot satisfy the requested number of requested Rx queues, the last Rx queue will be assigned for control plane. If only one Rx queue is available, the cp-protection feature will be disabled. If the hardware does not support the RTE flow matchers/actions, the feature will be disabled. It cannot be enabled when other_config:hw-offload=true as it may conflict with the offloaded RTE flows. Similarly, if hw-offload is enabled while some ports already have cp-protection enabled, RTE flow offloading will be disabled on these ports. Example use: ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \ set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \ set interface phy0 options:cp-protection=lacp -- \ set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \ set interface phy1 options:cp-protection=lacp As a starting point, only one protocol is supported: LACP. Other protocols can be added in the future. NIC compatibility should be checked. To validate that this works as intended, I used a traffic generator to generate random traffic slightly above the machine capacity at line rate on a two ports bond interface. OVS is configured to receive traffic on two VLANs and pop/push them in a br-int bridge based on tags set on patch ports. +--+ | DUT | |++| || br-int || default flow, action=NORMAL |||| || patch10patch11 || |+---|---|+| || | | |+---|---|+| || patch00patch01 || || tag:10tag:20 || |||| || br-phy || default flow, action=NORMAL |||| || bond0|| balance-slb, lacp=passive, lacp-time=fast ||phy0 phy1 || |+--|-|---+| +---|-|+ | | +---|-|+ | port0 port1 | balance L3/L4, lacp=active, lacp-time=fast | lag | mode trunk VLANs 10, 20 | | |switch| | | | vlan 10vlan 20 | mode access | port2 port3 | +-|--|-+ | | +-|--|-+ | port0 port1 | Random traffic that is properly balanced | | across the bond ports in both directions. | traffic generator | +--+ Without cp-protection, the bond0 links are randomly switching to "defaulted" when one of the LACP packets sent by the switch is dropped because the RX queues are full and the PMD threads did not process them fast enough. When that happens, all traffic must go through a single link which causes above line rate traffic to be dropped. When cp-protection is enabled, no LACP packet is dropped and the bond links remain enabled at all times, maximizing the throughput. This feature may be considered as "QoS". However, it does not work by limiting the rate of traffic explicitly. It only guarantees that some protocols have a lower chance of being dropped because the PMD cores cannot keep up with regular traffic. The choice of protocols is limited on purpose. This is not meant to be configurable by users. Some limited configurability could be considered in the future but it would expose to more potential issues if users are accidentally redirecting all traffic in the control plane queue. Cc: Christophe Fontaine Cc: Kevin Traynor Cc: David Marchand Signed-off-by: Robin Jarry ---
Re: [ovs-dev] [PATCH v5] netdev-dpdk: add control plane protection support
Kevin Traynor, Dec 21, 2022 at 17:35:
> Hi Robin,
>
> I did a bit of testing and some comments. I tested out applying a
> config. I wasn't able to check for lacp traffic, but I did see the extra
> rxq being added and rss working as expected on the other rx queues.
>
> One issue I found is where flow cannot be applied. It is recovering but
> when there is a reconfigure, everytime it uses n_rxq+1, and then
> reconfigures again with n_rxq.
>
> For example (flow normally ok for mlx, i forced a fail):
> $ ovs-vsctl get Interface myport status
> {bus_info="bus_name=pci, vendor_id=15b3, device_id=1017",
> cp_protection=unsupported, driver_name=mlx5_pci, if_descr="DPDK 22.11.1
> mlx5_pci", if_type="6", link_speed="25Gbps", max_hash_mac_addrs="0",
> max_mac_addrs="128", max_rx_pktlen="1518", max_rx_queues="1024",
> max_tx_queues="1024", max_vfs="0", max_vmdq_pools="0",
> min_rx_bufsize="32", numa_id="0", port_no="2"}
>
> $ ovs-vsctl get Interface myport options:n_rxq
> "3"
>
> change some config unrelated with rxq (snipped irrelevant parts):
> $ ovs-vsctl set Open_vSwitch . other_config:pmd-cpu-mask=100
>
> INFO|Performing pmd to rx queue assignment using cycles algorithm.
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 2
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 1
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 0
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 3
> INFO|Port 2: 04:3f:72:c2:07:b8
> INFO|Performing pmd to rx queue assignment using cycles algorithm.
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 0
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 1
> INFO|Core 8 on numa node 0 assigned port 'myport' rx queue 2
>
> If I remove the option:cp-protection, things return to normal.
Hmm, that is weird, I'll have a closer look.
> It would also be nice to document how to remove it from a port i.e.
> ovs-vsctl remove interface dpdk-p0 options cp-protection
>
> I was able to add/remove/add and it seemed fine. At present it only
> gives a dbg message:
> 2022-12-21T16:00:43Z|00168|netdev_dpdk|DBG|myport: cp-protection: reset
> flows and then it shows the reassignment of rxqs, so perhaps a message
> at info level indicating cp-protection is being removed for that port
> would be useful.
Ack, I will add a hint in the docs and some explicit info message
stating that cp protection is now disabled.
> I notice that if I set cp-prot, then enable hw-offload, I get:
> |WARN|myport: hw-offload is mutually exclusive with cp-protection
>
> It removes the cp-prot, but only mentions in debug log, so it's not
> clear. I didn't check operation of hw-offload.
>
> Also saw that if I set hw-offload to false, cp-prot cannot operate as it
> just checks for presence of hw-offload entry and rejects cp-prot even if
> hw-offload=false.
>
> The main thing is cp-prot does not enable when hw-offload=true and that
> is there at present. So probably just better logs can do for now. The
> other combinations i'm testing are more test cases than realistic cases,
> so as it's experimental perhaps it's fine to work on the different
> combinations and more fine grained controls (e.g. hw-offload=false) later.
This is probably related to the fact that hw-offload cannot be disabled
entirely without restarting vswitchd (unrelated to my patch). This is
what I noticed but I may have got it wrong.
I'll have a look if this can be improved.
Thanks a lot for more testing!
Joyeux Noël & Bonne année, all :)
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v5] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between forwarding engines (e.g. LACP). When the system is not sized properly, the PMD threads may not be able to process all incoming traffic from the configured Rx queues. When a signaling packet of such protocols is dropped, it can cause link flapping, worsening the situation. Use the RTE flow API to redirect these protocols into a dedicated Rx queue. The assumption is made that the ratio between control protocol traffic and user data traffic is very low and thus this dedicated Rx queue will never get full. The RSS redirection table is re-programmed to only use the other Rx queues. The RSS table size is stored in the netdev_dpdk structure at port initialization to avoid requesting the information again when changing the port configuration. The additional Rx queue will be assigned a PMD core like any other Rx queue. Polling that extra queue may introduce increased latency and a slight performance penalty at the benefit of preventing link flapping. This feature must be enabled per port on specific protocols via the cp-protection option. This option takes a coma-separated list of protocol names. It is only supported on ethernet ports. If the user has already configured multiple Rx queues on the port, an additional one will be allocated for control plane packets. If the hardware cannot satisfy the requested number of requested Rx queues, the last Rx queue will be assigned for control plane. If only one Rx queue is available, the cp-protection feature will be disabled. If the hardware does not support the RTE flow matchers/actions, the feature will be disabled. It cannot be enabled when other_config:hw-offload=true as it may conflict with the offloaded RTE flows. Similarly, if hw-offload is enabled while some ports already have cp-protection enabled, RTE flow offloading will be disabled on these ports. Example use: ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \ set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \ set interface phy0 options:cp-protection=lacp -- \ set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \ set interface phy1 options:cp-protection=lacp As a starting point, only one protocol is supported: LACP. Other protocols can be added in the future. NIC compatibility should be checked. To validate that this works as intended, I used a traffic generator to generate random traffic slightly above the machine capacity at line rate on a two ports bond interface. OVS is configured to receive traffic on two VLANs and pop/push them in a br-int bridge based on tags set on patch ports. +--+ | DUT | |++| || br-int || default flow, action=NORMAL |||| || patch10patch11 || |+---|---|+| || | | |+---|---|+| || patch00patch01 || || tag:10tag:20 || |||| || br-phy || default flow, action=NORMAL |||| || bond0|| balance-slb, lacp=passive, lacp-time=fast ||phy0 phy1 || |+--|-|---+| +---|-|+ | | +---|-|+ | port0 port1 | balance L3/L4, lacp=active, lacp-time=fast | lag | mode trunk VLANs 10, 20 | | |switch| | | | vlan 10vlan 20 | mode access | port2 port3 | +-|--|-+ | | +-|--|-+ | port0 port1 | Random traffic that is properly balanced | | across the bond ports in both directions. | traffic generator | +--+ Without cp-protection, the bond0 links are randomly switching to "defaulted" when one of the LACP packets sent by the switch is dropped because the RX queues are full and the PMD threads did not process them fast enough. When that happens, all traffic must go through a single link which causes above line rate traffic to be dropped. When cp-protection is enabled, no LACP packet is dropped and the bond links remain enabled at all times, maximizing the throughput. This feature may be considered as "QoS". However, it does not work by limiting the rate of traffic explicitly. It only guarantees that some protocols have a lower chance of being dropped because the PMD cores cannot keep up with regular traffic. The choice of protocols is limited on purpose. This is not meant to be configurable by users. Some limited configurability could be considered in the future but it would expose to more potential issues if users are accidentally redirecting all traffic in the control plane queue. Cc: Christophe Fontaine Cc: Kevin Traynor Cc: David Marchand Signed-off-by: Robin Jarry --- v4 -> v5: * Added NEWS entry * Updated
[ovs-dev] [PATCH v4] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between
forwarding engines (e.g. LACP). When the system is not sized properly,
the PMD threads may not be able to process all incoming traffic from the
configured Rx queues. When a signaling packet of such protocols is
dropped, it can cause link flapping, worsening the situation.
Use the RTE flow API to redirect these protocols into a dedicated Rx
queue. The assumption is made that the ratio between control protocol
traffic and user data traffic is very low and thus this dedicated Rx
queue will never get full. The RSS redirection table is re-programmed to
only use the other Rx queues. The RSS table size is stored in the
netdev_dpdk structure at port initialization to avoid requesting the
information again when changing the port configuration.
The additional Rx queue will be assigned a PMD core like any other Rx
queue. Polling that extra queue may introduce increased latency and
a slight performance penalty at the benefit of preventing link flapping.
This feature must be enabled per port on specific protocols via the
cp-protection option. This option takes a coma-separated list of
protocol names. It is only supported on ethernet ports.
If the user has already configured multiple Rx queues on the port, an
additional one will be allocated for control plane packets. If the
hardware cannot satisfy the requested number of requested Rx queues, the
last Rx queue will be assigned for control plane. If only one Rx queue
is available, the cp-protection feature will be disabled. If the
hardware does not support the RTE flow matchers/actions, the feature
will be disabled.
It cannot be enabled when other_config:hw-offload=true as it may
conflict with the offloaded RTE flows. Similarly, if hw-offload is
enabled while some ports already have cp-protection enabled, the RTE
flow offloading will be disabled on these ports.
Example use:
ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \
set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \
set interface phy0 options:cp-protection=lacp -- \
set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \
set interface phy1 options:cp-protection=lacp
As a starting point, only one protocol is supported: LACP. Other
protocols can be added in the future. NIC compatibility should be
checked.
To validate that this works as intended, I used a traffic generator to
generate random traffic slightly above the machine capacity at line rate
on a two ports bond interface. OVS is configured to receive traffic on
two VLANs and pop/push them in a br-int bridge based on tags set on
patch ports.
+--+
| DUT |
|++|
|| br-int || default flow, action=NORMAL
||||
|| patch10patch11 ||
|+---|---|+|
|| | |
|+---|---|+|
|| patch00patch01 ||
|| tag:10tag:20 ||
||||
|| br-phy || default flow, action=NORMAL
||||
|| bond0|| balance-slb, lacp=passive, lacp-time=fast
||phy0 phy1 ||
|+--|-|---+|
+---|-|+
| |
+---|-|+
| port0 port1 | balance L3/L4, lacp=active, lacp-time=fast
| lag | mode trunk VLANs 10, 20
| |
|switch|
| |
| vlan 10vlan 20 | mode access
| port2 port3 |
+-|--|-+
| |
+-|--|-+
| port0 port1 | Random traffic that is properly balanced
| | across the bond ports in both directions.
| traffic generator |
+--+
Without cp-protection, the bond0 links are randomly switching to
"defaulted" when one of the LACP packets sent by the switch is dropped
because the RX queues are full and the PMD threads did not process them
fast enough. When that happens, all traffic must go through a single
link which causes above line rate traffic to be dropped.
When cp-protection is enabled, no LACP packet is dropped and the bond
links remain enabled at all times, maximizing the throughput.
This feature may be considered as "QoS". However, it does not work by
limiting the rate of traffic explicitly. It only guarantees that some
protocols have a lower chance of being dropped because the PMD cores
cannot keep up with regular traffic.
The choice of protocols is limited on purpose. This is not meant to be
configurable by users. Some limited configurability could be considered
in the future but it would expose to more potential issues if users are
accidentally redirecting all traffic in the control plane queue.
Cc: Christophe Fontaine
Cc: Kevin Traynor
Signed-off-by: Robin Jarry
---
v3 -> v4:
* Rebased on master 739bcf2263b3 ("o
Re: [ovs-dev] mlx5 rte_eth_dev_info.reta_size value
Robin Jarry, Dec 05, 2022 at 16:57:
> Hi Ori,
>
> While working on a patch for OvS[1], I have tried to reconfigure the
> redirection table using the code examples that are layout around in
> testpmd and other places.
>
> [1]:
> http://patchwork.ozlabs.org/project/openvswitch/patch/20221021145308.141933-1-rja...@redhat.com/
>
> Here is a stripped down version of the code I use:
>
> int update_reta(int port_id, int num_rxq)
> {
>struct rte_eth_rss_reta_entry64 *conf;
>struct rte_eth_dev_info info;
>size_t conf_size;
>int err;
>
>rte_eth_dev_info_get(port_id, );
>conf_size = (info.reta_size / RTE_ETH_RETA_GROUP_SIZE) * sizeof(*conf);
>conf = malloc(conf_size);
>memset(conf, 0, conf_size);
>
>for (uint16_t i = 0; i < info.reta_size; i++) {
> uint16_t idx = i / RTE_ETH_RETA_GROUP_SIZE;
> uint16_t shift = i % RTE_ETH_RETA_GROUP_SIZE;
> reta_conf[idx].mask |= 1ULL << shift;
> reta_conf[idx].reta[shift] = i % num_rxq;
>}
>err = rte_eth_dev_rss_reta_update(port_id, conf, info.reta_size);
>free(conf);
>
>return err;
> }
>
> This works well for i40e and ice drivers but I get very confusing
> reta_size values with mlx5.
>
> mlx5_ethdev.c
>
> 333├>info->reta_size = priv->reta_idx_n ?
> 334│ priv->reta_idx_n : config->ind_table_max_size;
>
> (gdb) p priv->reta_idx_n
> $5 = 2
> (gdb) p config->ind_table_max_size
> $6 = 512
>
> Obviously, info.reta_size / RTE_ETH_RETA_GROUP_SIZE = 1 / 512 = 0
>
> From what I had understood info.reta_size should be a multiple of
> RTE_ETH_RETA_GROUP_SIZE. This is what I can observe with i40e and ice at
> least. Is it possible that the mlx5 driver has an issue there?
>
> I found this commit[2] from 2015 that may have introduced an issue but
> I am surprised that no one has ever encountered that before me. The
> suspicious code bit is:
>
> +/* If the requested number of RX queues is not a power of two, use the
> + * maximum indirection table size for better balancing.
> + * The result is always rounded to the next power of two. */
> +reta_idx_n = (1 << log2above((rxqs_n & (rxqs_n - 1)) ?
> +priv->ind_table_max_size :
> +rxqs_n));
>
> When rxqs_n == 2, reta_idx_n is initialized to 2 as well.
>
> [2]: https://git.dpdk.org/dpdk/commit/?id=634efbc2c8c05
>
> If you can provide any help, that would be much appreciated.
>
> Thanks!
To make sure, I have written a simple program that reuses log2above:
#include
static unsigned int log2above(unsigned int v)
{
unsigned int l, r;
for (l = 0, r = 0; (v >> 1); ++l, v >>= 1)
r |= (v & 1);
return l + r;
}
void main(void)
{
for (unsigned n = 1; n < 16; n++) {
printf("n_rxq=%d -> reta_size=%d\n", n,
1 << log2above((n & (n - 1)) ? 512 : n));
}
}
Running this yields:
n_rxq=1 -> reta_size=1
n_rxq=2 -> reta_size=2
n_rxq=3 -> reta_size=512
n_rxq=4 -> reta_size=4
n_rxq=5 -> reta_size=512
n_rxq=6 -> reta_size=512
n_rxq=7 -> reta_size=512
n_rxq=8 -> reta_size=8
n_rxq=9 -> reta_size=512
n_rxq=10 -> reta_size=512
n_rxq=11 -> reta_size=512
n_rxq=12 -> reta_size=512
n_rxq=13 -> reta_size=512
n_rxq=14 -> reta_size=512
n_rxq=15 -> reta_size=512
There is obviously something wrong and I am not sure what was the
original intention. So I don't know what to do to fix this.
I added Nelio in the thread. Maybe he can help :)
Cheers.
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] mlx5 rte_eth_dev_info.reta_size value
Hi Ori,
While working on a patch for OvS[1], I have tried to reconfigure the
redirection table using the code examples that are layout around in
testpmd and other places.
[1]:
http://patchwork.ozlabs.org/project/openvswitch/patch/20221021145308.141933-1-rja...@redhat.com/
Here is a stripped down version of the code I use:
int update_reta(int port_id, int num_rxq)
{
struct rte_eth_rss_reta_entry64 *conf;
struct rte_eth_dev_info info;
size_t conf_size;
int err;
rte_eth_dev_info_get(port_id, );
conf_size = (info.reta_size / RTE_ETH_RETA_GROUP_SIZE) * sizeof(*conf);
conf = malloc(conf_size);
memset(conf, 0, conf_size);
for (uint16_t i = 0; i < info.reta_size; i++) {
uint16_t idx = i / RTE_ETH_RETA_GROUP_SIZE;
uint16_t shift = i % RTE_ETH_RETA_GROUP_SIZE;
reta_conf[idx].mask |= 1ULL << shift;
reta_conf[idx].reta[shift] = i % num_rxq;
}
err = rte_eth_dev_rss_reta_update(port_id, conf, info.reta_size);
free(conf);
return err;
}
This works well for i40e and ice drivers but I get very confusing
reta_size values with mlx5.
mlx5_ethdev.c
333├>info->reta_size = priv->reta_idx_n ?
334│ priv->reta_idx_n : config->ind_table_max_size;
(gdb) p priv->reta_idx_n
$5 = 2
(gdb) p config->ind_table_max_size
$6 = 512
Obviously, info.reta_size / RTE_ETH_RETA_GROUP_SIZE = 1 / 512 = 0
From what I had understood info.reta_size should be a multiple of
RTE_ETH_RETA_GROUP_SIZE. This is what I can observe with i40e and ice at
least. Is it possible that the mlx5 driver has an issue there?
I found this commit[2] from 2015 that may have introduced an issue but
I am surprised that no one has ever encountered that before me. The
suspicious code bit is:
+/* If the requested number of RX queues is not a power of two, use the
+ * maximum indirection table size for better balancing.
+ * The result is always rounded to the next power of two. */
+reta_idx_n = (1 << log2above((rxqs_n & (rxqs_n - 1)) ?
+priv->ind_table_max_size :
+rxqs_n));
When rxqs_n == 2, reta_idx_n is initialized to 2 as well.
[2]: https://git.dpdk.org/dpdk/commit/?id=634efbc2c8c05
If you can provide any help, that would be much appreciated.
Thanks!
--
Robin Jarry
Principal Software Engineer
Red Hat
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v3] netdev-dpdk: add control plane protection support
Hi Kevin, Kevin Traynor, Nov 11, 2022 at 19:15: > > +reta_conf_size = (dev->reta_size / RTE_ETH_RETA_GROUP_SIZE) > > +* sizeof(struct rte_eth_rss_reta_entry64); > > In dpdk_eth_dev_init, we get reta_size from driver, > > mlx5_ethdev.c > > 333├>info->reta_size = priv->reta_idx_n ? > 334│ priv->reta_idx_n : config->ind_table_max_size; > > (gdb) p priv->reta_idx_n > $5 = 1 > (gdb) p config->ind_table_max_size > $6 = 512 > > and store: > dev->reta_size = info.reta_size; > > Now we use it, > dev->reta_size = 1 / RTE_ETH_RETA_GROUP_SIZE (64) > but it results in reta_conf_size = 0 > > > +reta_conf = xmalloc(reta_conf_size); > > xmalloc only allocates 1 byte (void *p = malloc(size ? size : 1);) Hmm indeed this is bad :) Since reworking the RSS fallback I may not have re-tested with CX-5, only with Intel NICs. I am quite surprised of this return value. Could it be an issue with the mlx5 dpdk driver? Thanks for testing! ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH v3] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between forwarding engines (e.g. LACP). When the system is not sized properly, the PMD threads may not be able to process all incoming traffic from the configured Rx queues. When a signaling packet of such protocols is dropped, it can cause link flapping, worsening the situation. Use the RTE flow API to redirect these protocols into a dedicated Rx queue. The assumption is made that the ratio between control protocol traffic and user data traffic is very low and thus this dedicated Rx queue will never get full. The RSS redirection table is re-programmed to only use the other Rx queues. The RSS table size is stored in the netdev_dpdk structure at port initialization to avoid requesting the information again when changing the port configuration. The additional Rx queue will be assigned a PMD core like any other Rx queue. Polling that extra queue may introduce increased latency and a slight performance penalty at the benefit of preventing link flapping. This feature must be enabled per port on specific protocols via the cp-protection option. This option takes a coma-separated list of protocol names. It is only supported on ethernet ports. If the user has already configured multiple Rx queues on the port, an additional one will be allocated for control plane packets. If the hardware cannot satisfy the requested number of requested Rx queues, the last Rx queue will be assigned for control plane. If only one Rx queue is available, the cp-protection feature will be disabled. If the hardware does not support the RTE flow matchers/actions, the feature will be disabled. It cannot be enabled when other_config:hw-offload=true as it may conflict with the offloaded RTE flows. Similarly, if hw-offload is enabled while some ports already have cp-protection enabled, the RTE flow offloading will be disabled on these ports. Example use: ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \ set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \ set interface phy0 options:cp-protection=lacp -- \ set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \ set interface phy1 options:cp-protection=lacp As a starting point, only one protocol is supported: LACP. Other protocols can be added in the future. NIC compatibility should be checked. To validate that this works as intended, I used a traffic generator to generate random traffic slightly above the machine capacity at line rate on a two ports bond interface. OVS is configured to receive traffic on two VLANs and pop/push them in a br-int bridge based on tags set on patch ports. +--+ | DUT | |++| || br-int || default flow, action=NORMAL |||| || patch10patch11 || |+---|---|+| || | | |+---|---|+| || patch00patch01 || || tag:10tag:20 || |||| || br-phy || default flow, action=NORMAL |||| || bond0|| balance-slb, lacp=passive, lacp-time=fast ||phy0 phy1 || |+--|-|---+| +---|-|+ | | +---|-|+ | port0 port1 | balance L3/L4, lacp=active, lacp-time=fast | lag | mode trunk VLANs 10, 20 | | |switch| | | | vlan 10vlan 20 | mode access | port2 port3 | +-|--|-+ | | +-|--|-+ | port0 port1 | Random traffic that is properly balanced | | across the bond ports in both directions. | traffic generator | +--+ Without cp-protection, the bond0 links are randomly switching to "defaulted" when one of the LACP packets sent by the switch is dropped because the RX queues are full and the PMD threads did not process them fast enough. When that happens, all traffic must go through a single link which causes above line rate traffic to be dropped. When cp-protection is enabled, no LACP packet is dropped and the bond links remain enabled at all times, maximizing the throughput. This feature may be considered as "QoS". However, it does not work by limiting the rate of traffic explicitly. It only guarantees that some protocols have a lower chance of being dropped because the PMD cores cannot keep up with regular traffic. The choice of protocols is limited on purpose. This is not meant to be configurable by users. Some limited configurability could be considered in the future but it would expose to more potential issues if users are accidentally redirecting all traffic in the control plane queue. Cc: Christophe Fontaine Cc: Kevin Traynor Signed-off-by: Robin Jarry --- v2 -> v3: * Added dry_run validation that rte_flows are a
[PATCH v2] cli: add options --offset and --limit to notmuch show
notmuch search does not output header values. However, when browsing
through a large email corpus, it can be time saving to be able to
paginate without running notmuch show for each message/thread.
Add --offset and --limit options to notmuch show. This is inspired from
commit 796b629c3b82 ("cli: add options --offset and --limit to notmuch
search").
Update man page, shell completion and add a test case to ensure it works
as expected.
Cc: Tim Culverhouse
Cc: Tomi Ollila
Signed-off-by: Robin Jarry
---
v1 -> v2:
- fixed typo in print_status_query (search -> show)
- changed all shell variable expansions without braces
- removed the bash-specific 'function' keyword
completion/notmuch-completion.bash | 2 +-
completion/zsh/_notmuch| 2 +
doc/man1/notmuch-show.rst | 9
notmuch-client.h | 2 +
notmuch-show.c | 49 +++---
test/T131-show-limiting.sh | 81 ++
6 files changed, 138 insertions(+), 7 deletions(-)
create mode 100755 test/T131-show-limiting.sh
diff --git a/completion/notmuch-completion.bash
b/completion/notmuch-completion.bash
index 0022b54bff5d..3748846edf83 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -530,7 +530,7 @@ _notmuch_show()
! $split &&
case "${cur}" in
-*)
- local options="--entire-thread= --format= --exclude= --body=
--format-version= --part= --verify --decrypt= --include-html
${_notmuch_shared_options}"
+ local options="--entire-thread= --format= --exclude= --body=
--format-version= --part= --verify --decrypt= --include-html --limit= --offset=
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch
index e207d90b7202..0bdd7f772a7a 100644
--- a/completion/zsh/_notmuch
+++ b/completion/zsh/_notmuch
@@ -245,6 +245,8 @@ _notmuch_show() {
'--exclude=[respect excluded tags setting]:exclude tags:(true false)' \
'--body=[output body]:output body content:(true false)' \
'--include-html[include text/html parts in the output]' \
+'--limit=[limit the number of displayed results]:limit: ' \
+'--offset=[skip displaying the first N results]:offset: ' \
'*::search term:_notmuch_search_term'
}
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 2c0a0de6ad16..c13d94de0244 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -130,6 +130,15 @@ Supported options for **show** include
By default, results will be displayed in reverse chronological
order, (that is, the newest results will be displayed first).
+.. option:: --offset=[-]N
+
+ Skip displaying the first N results. With the leading '-', start
+ at the Nth result from the end.
+
+.. option:: --limit=N
+
+ Limit the number of displayed results to N.
+
.. option:: --verify
Compute and report the validity of any MIME cryptographic
diff --git a/notmuch-client.h b/notmuch-client.h
index 21b49908ae24..1a87240d3c21 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -77,6 +77,8 @@ typedef struct notmuch_show_params {
bool output_body;
int duplicate;
int part;
+int offset;
+int limit;
_notmuch_crypto_t crypto;
bool include_html;
GMimeStream *out_stream;
diff --git a/notmuch-show.c b/notmuch-show.c
index ee9efa7448d7..7fb40ce9ab5d 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1159,6 +1159,18 @@ do_show_threaded (void *ctx,
notmuch_thread_t *thread;
notmuch_messages_t *messages;
notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;
+int i;
+
+if (params->offset < 0) {
+ unsigned count;
+ notmuch_status_t s = notmuch_query_count_threads (query, );
+ if (print_status_query ("notmuch show", query, s))
+ return 1;
+
+ params->offset += count;
+ if (params->offset < 0)
+ params->offset = 0;
+}
status = notmuch_query_search_threads (query, );
if (print_status_query ("notmuch show", query, status))
@@ -1166,11 +1178,16 @@ do_show_threaded (void *ctx,
sp->begin_list (sp);
-for (;
-notmuch_threads_valid (threads);
-notmuch_threads_move_to_next (threads)) {
+for (i = 0;
+notmuch_threads_valid (threads) && (params->limit < 0 || i <
params->offset + params->limit);
+notmuch_threads_move_to_next (threads), i++) {
thread = notmuch_threads_get (threads);
+ if (i < params->offset) {
+ notmuch_thread_destroy (thread);
+ continue;
+ }
+
messages = notmuch_thread_get_toplevel_messages (thread);
if (messages == NULL)
@@ -1201,6 +1218,18 @@ do_show_u
Re: [PATCH] cli: add options --offset and --limit to notmuch show
Hi Tomi,
Tomi Ollila, Oct 12, 2022 at 21:39:
> > diff --git a/notmuch-show.c b/notmuch-show.c
> > index ee9efa7448d7..ad31e0123268 100644
> > --- a/notmuch-show.c
> > +++ b/notmuch-show.c
> > @@ -1159,6 +1159,18 @@ do_show_threaded (void *ctx,
> > notmuch_thread_t *thread;
> > notmuch_messages_t *messages;
> > notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;
> > +int i;
> > +
> > +if (params->offset < 0) {
> > + unsigned count;
> > + notmuch_status_t s = notmuch_query_count_threads (query, );
> > + if (print_status_query ("notmuch search", query, s))
> > + return 1;
> > +
> > + params->offset += count;
> > + if (params->offset < 0)
>
> this check and setting it to 0 is mystic to me, probably same code as in
> search (?) probably it is good (?) (will not comment the same below)
Yes, I copy/pasted that code from notmuch-search.c. I believe this to
handle the case where:
--limit=N && --offset=-M && M > count
After adding count to offset, you may end up with a negative offset
which makes the loop exit condition invalid and the --limit value would
not be enforced as expected:
i < params->offset + params->limit
> > diff --git a/test/T131-show-limiting.sh b/test/T131-show-limiting.sh
> > new file mode 100755
> > index ..a3da35944a3e
> > --- /dev/null
> > +++ b/test/T131-show-limiting.sh
> > @@ -0,0 +1,81 @@
> > +#!/usr/bin/env bash
> > +test_description='"notmuch show" --offset and --limit parameters'
> > +. $(dirname "$0")/test-lib.sh || exit 1
> > +
> > +add_email_corpus
> > +
> > +function show() {
>
> 'function' not used in other function defitions in other files, so this is
> inconsistent (otherwise the content looks "better" than what I see usual ;D)
I concur that this is a bash-only construct. I could remove the function
keyword and we would have the same result.
> > +test_begin_subtest "${outp}: offset = 0"
>
> inconsistent ${outp} (where $outp used elsewhere) ...
Indeed. I had removed all the ${} except this one. For consistency with
the other test scripts, I could add them back everywhere. I don't mind
either way.
I'll hold off for other remarks before sending a v2.
Cheers,
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] cli: add options --offset and --limit to notmuch show
notmuch search does not output header values. However, when browsing
through a large email corpus, it can be time saving to be able to
paginate without running notmuch show for each message/thread.
Add --offset and --limit options to notmuch show. This is inspired from
commit 796b629c3b82 ("cli: add options --offset and --limit to notmuch
search").
Update man page, shell completion and add a test case to ensure it works
as expected.
Cc: Tim Culverhouse
Signed-off-by: Robin Jarry
---
completion/notmuch-completion.bash | 2 +-
completion/zsh/_notmuch| 2 +
doc/man1/notmuch-show.rst | 9
notmuch-client.h | 2 +
notmuch-show.c | 49 +++---
test/T131-show-limiting.sh | 81 ++
6 files changed, 138 insertions(+), 7 deletions(-)
create mode 100755 test/T131-show-limiting.sh
diff --git a/completion/notmuch-completion.bash
b/completion/notmuch-completion.bash
index 0022b54bff5d..3748846edf83 100644
--- a/completion/notmuch-completion.bash
+++ b/completion/notmuch-completion.bash
@@ -530,7 +530,7 @@ _notmuch_show()
! $split &&
case "${cur}" in
-*)
- local options="--entire-thread= --format= --exclude= --body=
--format-version= --part= --verify --decrypt= --include-html
${_notmuch_shared_options}"
+ local options="--entire-thread= --format= --exclude= --body=
--format-version= --part= --verify --decrypt= --include-html --limit= --offset=
${_notmuch_shared_options}"
compopt -o nospace
COMPREPLY=( $(compgen -W "$options" -- ${cur}) )
;;
diff --git a/completion/zsh/_notmuch b/completion/zsh/_notmuch
index e207d90b7202..0bdd7f772a7a 100644
--- a/completion/zsh/_notmuch
+++ b/completion/zsh/_notmuch
@@ -245,6 +245,8 @@ _notmuch_show() {
'--exclude=[respect excluded tags setting]:exclude tags:(true false)' \
'--body=[output body]:output body content:(true false)' \
'--include-html[include text/html parts in the output]' \
+'--limit=[limit the number of displayed results]:limit: ' \
+'--offset=[skip displaying the first N results]:offset: ' \
'*::search term:_notmuch_search_term'
}
diff --git a/doc/man1/notmuch-show.rst b/doc/man1/notmuch-show.rst
index 2c0a0de6ad16..c13d94de0244 100644
--- a/doc/man1/notmuch-show.rst
+++ b/doc/man1/notmuch-show.rst
@@ -130,6 +130,15 @@ Supported options for **show** include
By default, results will be displayed in reverse chronological
order, (that is, the newest results will be displayed first).
+.. option:: --offset=[-]N
+
+ Skip displaying the first N results. With the leading '-', start
+ at the Nth result from the end.
+
+.. option:: --limit=N
+
+ Limit the number of displayed results to N.
+
.. option:: --verify
Compute and report the validity of any MIME cryptographic
diff --git a/notmuch-client.h b/notmuch-client.h
index 21b49908ae24..1a87240d3c21 100644
--- a/notmuch-client.h
+++ b/notmuch-client.h
@@ -77,6 +77,8 @@ typedef struct notmuch_show_params {
bool output_body;
int duplicate;
int part;
+int offset;
+int limit;
_notmuch_crypto_t crypto;
bool include_html;
GMimeStream *out_stream;
diff --git a/notmuch-show.c b/notmuch-show.c
index ee9efa7448d7..ad31e0123268 100644
--- a/notmuch-show.c
+++ b/notmuch-show.c
@@ -1159,6 +1159,18 @@ do_show_threaded (void *ctx,
notmuch_thread_t *thread;
notmuch_messages_t *messages;
notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;
+int i;
+
+if (params->offset < 0) {
+ unsigned count;
+ notmuch_status_t s = notmuch_query_count_threads (query, );
+ if (print_status_query ("notmuch search", query, s))
+ return 1;
+
+ params->offset += count;
+ if (params->offset < 0)
+ params->offset = 0;
+}
status = notmuch_query_search_threads (query, );
if (print_status_query ("notmuch show", query, status))
@@ -1166,11 +1178,16 @@ do_show_threaded (void *ctx,
sp->begin_list (sp);
-for (;
-notmuch_threads_valid (threads);
-notmuch_threads_move_to_next (threads)) {
+for (i = 0;
+notmuch_threads_valid (threads) && (params->limit < 0 || i <
params->offset + params->limit);
+notmuch_threads_move_to_next (threads), i++) {
thread = notmuch_threads_get (threads);
+ if (i < params->offset) {
+ notmuch_thread_destroy (thread);
+ continue;
+ }
+
messages = notmuch_thread_get_toplevel_messages (thread);
if (messages == NULL)
@@ -1201,6 +1218,18 @@ do_show_unthreaded (void *ctx,
notmuch_message_t *message;
notmuch_status_t status, res = NOTMUCH_STATUS_SUCCESS;
notmuch_bool_t excluded;
+int i;
+
+if (params->offset < 0) {
+
Re: [PATCH] cli: add options --offset and --limit to notmuch show
Robin Jarry, Oct 12, 2022 at 00:19:
> + if (print_status_query ("notmuch search", query, s))
I just realized that I copy pasted code from notmuch-search.c and did
not updated everything on the way...
I'll hold before sending a v2 if there are other remarks or changes
required.
Cheers all.
___
notmuch mailing list -- notmuch@notmuchmail.org
To unsubscribe send an email to notmuch-le...@notmuchmail.org
[PATCH] css: make diff colors more accessible
The colors used to display patch diffs are confusing. The context color
is very similar to the added line color and the contrast between added
and removed lines is very low.
Originally, the choice of purple/blue (instead of the more common
red/green palette) may have been made with colorblindness accessibility
in mind. However, after inspecting the current colors with
colorblindness "simulators", I found that the low contrast was
consistent no matter what vision deficiency (if any) you might have.
Update the colors to use a more common red/green palette. Add background
colors to increase contrast for colorblind people. Use less confusing
colors for context and diff hunks. Use normal line height to prevent
background colors from overlapping. Use a different color for email
quotes (blue) to avoid confusion with added lines.
I have made a compilation of the current and updated color palette
previews for normal vision and all common color deficiencies. I also
included the same diff as seen from Github interface for reference.
Link: http://files.diabeteman.com/patchwork-diff-colors/
Signed-off-by: Robin Jarry
---
htdocs/css/style.css | 16
...-colors-more-accessible-82eda58a89984d46.yaml | 5 +
2 files changed, 13 insertions(+), 8 deletions(-)
create mode 100644
releasenotes/notes/make-diff-colors-more-accessible-82eda58a89984d46.yaml
diff --git a/htdocs/css/style.css b/htdocs/css/style.css
index 9156aa6ee073..1a739510924c 100644
--- a/htdocs/css/style.css
+++ b/htdocs/css/style.css
@@ -17,7 +17,7 @@ h2 a, h2 span {
}
pre {
-line-height: 110%;
+line-height: normal;
background-color: white;
border-radius: 0;
}
@@ -354,15 +354,15 @@ button[class^=comment-action] {
}
.quote {
-color: #007f00;
+color: #365cb5;
}
-span.p_header{ color: #2e8b57; font-weight: bold; }
-span.p_chunk{ color: #a52a2a; font-weight: bold; }
-span.p_context{ color: #a020f0; }
-span.p_add{ color: #008b8b; }
-span.p_del{ color: #6a5acd; }
-span.p_mod{ color: #ff; }
+span.p_header{ font-weight: bold; }
+span.p_chunk{ color: #329fb0; font-weight: bold; }
+span.p_context{ }
+span.p_add{ color: #1b9d09; background-color: #edffed; }
+span.p_del{ color: #c80101; background-color: #ffe2e2; }
+span.p_mod{ color: #a020f0; }
.acked-by {
color: #2d4566;
diff --git
a/releasenotes/notes/make-diff-colors-more-accessible-82eda58a89984d46.yaml
b/releasenotes/notes/make-diff-colors-more-accessible-82eda58a89984d46.yaml
new file mode 100644
index ..f65995e51d9a
--- /dev/null
+++ b/releasenotes/notes/make-diff-colors-more-accessible-82eda58a89984d46.yaml
@@ -0,0 +1,5 @@
+---
+other:
+ - |
+The patch diff color palette was modified to make it more accessible for
+all users, including those with common color deficiencies.
--
2.37.3
___
Patchwork mailing list
Patchwork@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/patchwork
Re: [ovs-dev] [RFC PATCH] netdev-dpdk: add control plane protection support
Hi Thilak, Thilak Raj Surendra Babu, Sep 22, 2022 at 07:16: > Could we call rte_flow_validate() before calling flow_create ? At first, I thought it would not be necessary hence skipped the validation and I am only relying on the driver to reject the rule if not supported. However, it may be a bit more robust in case the last rss flow is not supported. We would not have to flush other flows. > Can we not use isolate the Queue using rte_flow_isolate()? >From what I understand, rte_flow_isolate is very intrusive. https://doc.dpdk.org/guides-21.11/prog_guide/rte_flow.html#flow-isolated-mode It may not play well with the dpdk rte flow offload subsystem. Thanks for reviewing. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [RFC PATCH] netdev-dpdk: add control plane protection support
Some control protocols are used to maintain link status between forwarding engines (e.g. LACP). When the system is not sized properly, the PMD threads may not be able to process all incoming traffic from the configured Rx queues. When a signaling packet of such protocols is dropped, it can cause link flapping, worsening the situation. Use the RTE flow API to redirect these protocols into a dedicated Rx queue. The assumption is made that the ratio between control protocol traffic and user data traffic is very low and thus this dedicated Rx queue will never get full. Regular RSS is done on all other Rx queues. The additional Rx queue will be assigned a PMD core like any other Rx queue. Polling that extra queue may introduce increased latency and a slight performance penalty at the benefit of preventing link flapping. This feature must be enabled per port on specific protocols via the cp-protection option. This option takes a coma-separated list of protocol names. Example: ovs-vsctl add-bond br-phy bond0 phy0 phy1 -- \ set interface phy0 type=dpdk options:dpdk-devargs=:ca:00.0 -- \ set interface phy0 options:cp-protection=lacp -- \ set interface phy1 type=dpdk options:dpdk-devargs=:ca:00.1 -- \ set interface phy1 options:cp-protection=lacp As a starting point, only one protocol is supported: LACP. Other protocols can be added in the future. NIC compatibility should be checked. To validate that this works as intended, I used a traffic generator to generate random traffic slightly above the machine capacity at line rate on a two ports bond interface. OVS is configured to receive traffic on two VLANs and pop/push them in a br-int bridge based on tags set on patch ports. +--+ | DUT | |++| || br-int || default flow, action=NORMAL |||| || patch10patch11 || |+---|---|+| || | | |+---|---|+| || patch00patch01 || || tag:10tag:20 || |||| || br-phy || default flow, action=NORMAL |||| || bond0|| balance-slb, lacp=passive, lacp-time=fast ||phy0 phy1 || |+--|-|---+| +---|-|+ | | +---|-|+ | port0 port1 | balance L3/L4, lacp=active, lacp-time=fast | lag | mode trunk VLANs 10, 20 | | |switch| | | | vlan 10vlan 20 | mode access | port2 port3 | +-|--|-+ | | +-|--|-+ | port0 port1 | Random traffic that is properly balanced | | across the bond ports in both directions. | traffic generator | +--+ Without cp-protection, the bond0 links are randomly switching to "defaulted" when one of the LACP packets sent by the switch is dropped because the RX queues are full and the PMD threads did not process them fast enough. When that happens, all traffic must go through a single link which causes above line rate traffic to be dropped. When cp-protection is enabled, no LACP packet is dropped and the bond links remain enabled at all times, maximizing the throughput. Notes: * This feature may be considered as "QoS". However, it does not work by limiting the rate of traffic explicitly. It only guarantees that some protocols have a lower chance of being dropped because the PMD cores cannot keep up with regular traffic. * The choice of protocols is limited on purpose. This is not meant to be configurable by users. Some limited configurability could be considered in the future but it would expose to more potential issues if users are accidentally redirecting all traffic in the control plane queue. * Encapsulated traffic (into VXLAN, GRE, Geneve) is not redirected into that extra queue. This may be addressed in the future as well. * For now, I validated that this works on Mellanox Connect-X 5 and Intel E810 NICs. Intel X710 does not support the match all flows. Validation on other NICs should be done. * Unit tests may not be possible here. The netdev-dummy driver does not share code with netdev-dpdk. Cc: Christophe Fontaine Cc: Kevin Traynor Signed-off-by: Robin Jarry --- Documentation/topics/dpdk/phy.rst | 41 ++ lib/netdev-dpdk.c | 235 ++ lib/netdev-dpdk.h | 5 + lib/netdev-offload-dpdk.c | 18 ++- lib/netdev-provider.h | 7 + 5 files changed, 302 insertions(+), 4 deletions(-) diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index 937f4c40e5a8..00b3887c70bd 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -131,6 +131,47 @
Bug#1019152: python-bonsai: flaky autopkgtest on armhf and armel: timeout too short?
Paul Gevers, Sep 19, 2022 at 21:52: > Architecture: !armel !armhf > is supported. Awesome. I'll upload a new version with that fix then. Thanks!
Bug#1019152: python-bonsai: flaky autopkgtest on armhf and armel: timeout too short?
Paul Gevers, Sep 19, 2022 at 21:52: > Architecture: !armel !armhf > is supported. Awesome. I'll upload a new version with that fix then. Thanks!
Bug#1019152: python-bonsai: flaky autopkgtest on armhf and armel: timeout too short?
Paul Gevers, Sep 19, 2022 at 21:13: > I consider tests marked flaky as not so useful. Obviously if you'll > look at it from time to time it's OK, but if no human is going to > inspect it, it's smarter to just skip those architectures. Doing the > latter is also easier than the former as we have the "Architecture" > field for that, while flaky per arch you'd need to implement yourself. I can live with disabling these tests on armel and armhf. If I understood correctly, the Architecture field in d/t/control does not support exclusions and it requires to set what arch is supported. I did not found any explicit docs on that matter. Is there a way to list multiple architectures?
Bug#1019152: python-bonsai: flaky autopkgtest on armhf and armel: timeout too short?
Paul Gevers, Sep 19, 2022 at 21:13: > I consider tests marked flaky as not so useful. Obviously if you'll > look at it from time to time it's OK, but if no human is going to > inspect it, it's smarter to just skip those architectures. Doing the > latter is also easier than the former as we have the "Architecture" > field for that, while flaky per arch you'd need to implement yourself. I can live with disabling these tests on armel and armhf. If I understood correctly, the Architecture field in d/t/control does not support exclusions and it requires to set what arch is supported. I did not found any explicit docs on that matter. Is there a way to list multiple architectures?
Bug#1019152: python-bonsai: flaky autopkgtest on armhf and armel: timeout too short?
Hi Paul, sorry for the delay. I am getting to work on this issue. I am not sure how to increase the timeouts for these tests. Would it be acceptable to mark these tests as flaky only on armel and armhf?
Bug#1019152: python-bonsai: flaky autopkgtest on armhf and armel: timeout too short?
Hi Paul, sorry for the delay. I am getting to work on this issue. I am not sure how to increase the timeouts for these tests. Would it be acceptable to mark these tests as flaky only on armel and armhf?
Bug#1019894: ITP: golang-sourcehut-rockorager-tcell-term -- TODO
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-sourcehut-rockorager-tcell-term Version : 0.1.0-1 Upstream Author : Tim Culverhouse * URL : https://git.sr.ht/~rockorager/tcell-term * License : expat Programming Lang: Go Description : A virtual terminal widget for tcell tcell-term implements the native tcell Widget interface. This is a new dependency for aerc.
Bug#1019894: ITP: golang-sourcehut-rockorager-tcell-term -- TODO
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-sourcehut-rockorager-tcell-term Version : 0.1.0-1 Upstream Author : Tim Culverhouse * URL : https://git.sr.ht/~rockorager/tcell-term * License : expat Programming Lang: Go Description : A virtual terminal widget for tcell tcell-term implements the native tcell Widget interface. This is a new dependency for aerc.
Bug#1019894: ITP: golang-sourcehut-rockorager-tcell-term -- TODO
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-sourcehut-rockorager-tcell-term Version : 0.1.0-1 Upstream Author : Tim Culverhouse * URL : https://git.sr.ht/~rockorager/tcell-term * License : expat Programming Lang: Go Description : A virtual terminal widget for tcell tcell-term implements the native tcell Widget interface. This is a new dependency for aerc.
Bug#1019894: ITP: golang-sourcehut-rockorager-tcell-term -- TODO
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-sourcehut-rockorager-tcell-term Version : 0.1.0-1 Upstream Author : Tim Culverhouse * URL : https://git.sr.ht/~rockorager/tcell-term * License : expat Programming Lang: Go Description : A virtual terminal widget for tcell tcell-term implements the native tcell Widget interface. This is a new dependency for aerc.
[ovs-dev] [PATCH v3] netdev-dpdk: fix tx_dropped counters value
Packets that could not be transmitted because the TXQ are full should be
taken into account in the global ovs_tx_failure_drops as it was the case
before commit 29b94e12d57d ("netdev-dpdk: Refactor the DPDK transmit
path.").
netdev_dpdk_eth_tx_burst() returns the number of packets that were *not*
transmitted. Add that number to stats.tx_failure_drops and only include
the packets that were dropped in previous steps afterwards.
Fixes: 29b94e12d57d ("netdev-dpdk: Refactor the DPDK transmit path.")
Signed-off-by: Robin Jarry
---
Notes:
v2 -> v3: fixed double count of tx_failure_drops
lib/netdev-dpdk.c | 6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index 0dd655507b50..ff241e02fe48 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -2882,9 +2882,9 @@ netdev_dpdk_eth_send(struct netdev *netdev, int qid,
cnt = netdev_dpdk_common_send(netdev, batch, );
-dropped = batch_cnt - cnt;
-
-dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, cnt);
+dropped = netdev_dpdk_eth_tx_burst(dev, qid, pkts, cnt);
+stats.tx_failure_drops += dropped;
+dropped += batch_cnt - cnt;
if (OVS_UNLIKELY(dropped)) {
struct netdev_dpdk_sw_stats *sw_stats = dev->sw_stats;
--
2.37.2
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Re: [ovs-dev] [PATCH v2] netdev-dpdk: fix tx_dropped counters value
Hi Mike, Mike Pattrick, Aug 31, 2022 at 18:09: > On Wed, Aug 3, 2022 at 4:58 AM Robin Jarry wrote: > > dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, cnt); > > +stats.tx_failure_drops += dropped; > > Hello Robin, > > I think this will double count tx_failure_drops if there were any in > netdev_dpdk_common_send(). You are correct, I will fix this. > > +dropped = stats.tx_mtu_exceeded_drops + > > + stats.tx_qos_drops + > > + stats.tx_failure_drops + > > + stats.tx_invalid_hwol_drops; > > I'm not too clear on why we have to recalculate this, what case would > ccase dropped to be incorrect here. I'll change the logic a bit. This is awkward. Thanks for reviewing! ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Bug#1017247: golang-github-gatherstars-com-jwz: FTBFS: dh_auto_test: error: cd _build && go test -vet=off -v -p 8 github.com/gatherstars-com/jwz github.com/gatherstars-com/jwz/examples/visualize retur
Hey Nilesh, Nilesh Patra, Aug 25, 2022 at 14:20: > This bug is causing an autoremoval warning for aerc. There does not seem to > be a fix > upstream about this. I am not sure what exactly is triggering this, but my > hunch > is it might be related to change in sort function with golang 1.19. > (This works fine with go-1.18) > > Can I ask you to take a look at it? I pushed a fix on salsa: https://salsa.debian.org/go-team/packages/golang-github-gatherstars-com-jwz/-/commit/842c69125282bdfb2725325d91d8002ce8f86891 This patch was submitted upstream: https://github.com/gatherstars-com/jwz/pull/2 If you want I can upload but you'll need to give me permission for golang-github-gatherstars-com-jwz :-) Cheers,
Bug#1017247: golang-github-gatherstars-com-jwz: FTBFS: dh_auto_test: error: cd _build && go test -vet=off -v -p 8 github.com/gatherstars-com/jwz github.com/gatherstars-com/jwz/examples/visualize retur
Hey Nilesh, Nilesh Patra, Aug 25, 2022 at 14:20: > This bug is causing an autoremoval warning for aerc. There does not seem to > be a fix > upstream about this. I am not sure what exactly is triggering this, but my > hunch > is it might be related to change in sort function with golang 1.19. > (This works fine with go-1.18) > > Can I ask you to take a look at it? I pushed a fix on salsa: https://salsa.debian.org/go-team/packages/golang-github-gatherstars-com-jwz/-/commit/842c69125282bdfb2725325d91d8002ce8f86891 This patch was submitted upstream: https://github.com/gatherstars-com/jwz/pull/2 If you want I can upload but you'll need to give me permission for golang-github-gatherstars-com-jwz :-) Cheers,
[ovs-dev] [PATCH v2] netdev-dpdk: fix tx_dropped counters value
Packets that could not be transmitted because the TXQ are full should be
taken into account in the global ovs_tx_failure_drops as it was the case
before commit 29b94e12d57d ("netdev-dpdk: Refactor the DPDK transmit
path.").
Also, the global tx_dropped should take all ovs_*_tx_drops counters into
account.
Fixes: 29b94e12d57d ("netdev-dpdk: Refactor the DPDK transmit path.")
Signed-off-by: Robin Jarry
---
v1 -> v2: fixed build error (last minute copy paste without testing...)
lib/netdev-dpdk.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index 0dd655507b50..84f0fbf24355 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -2883,8 +2883,12 @@ netdev_dpdk_eth_send(struct netdev *netdev, int qid,
cnt = netdev_dpdk_common_send(netdev, batch, );
dropped = batch_cnt - cnt;
-
dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, cnt);
+stats.tx_failure_drops += dropped;
+dropped = stats.tx_mtu_exceeded_drops +
+ stats.tx_qos_drops +
+ stats.tx_failure_drops +
+ stats.tx_invalid_hwol_drops;
if (OVS_UNLIKELY(dropped)) {
struct netdev_dpdk_sw_stats *sw_stats = dev->sw_stats;
--
2.37.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH] netdev-dpdk: fix tx_dropped counters value
Packets that could not be transmitted because the TXQ are full should be
taken into account in the global ovs_tx_failure_drops as it was the case
before commit 29b94e12d57d ("netdev-dpdk: Refactor the DPDK transmit
path.").
Also, the global tx_dropped should take all ovs_*_tx_drops counters into
account.
Fixes: 29b94e12d57d ("netdev-dpdk: Refactor the DPDK transmit path.")
Signed-off-by: Robin Jarry
---
I am not 100% sure about the tx_dropped global counter but now it is
aligned with vhost ports.
lib/netdev-dpdk.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c
index 0dd655507b50..17e9268c9216 100644
--- a/lib/netdev-dpdk.c
+++ b/lib/netdev-dpdk.c
@@ -2883,8 +2883,12 @@ netdev_dpdk_eth_send(struct netdev *netdev, int qid,
cnt = netdev_dpdk_common_send(netdev, batch, );
dropped = batch_cnt - cnt;
-
dropped += netdev_dpdk_eth_tx_burst(dev, qid, pkts, cnt);
+stats.tx_failure_drops += dropped;
+dropped = sw_stats_add->tx_mtu_exceeded_drops +
+ sw_stats_add->tx_qos_drops +
+ sw_stats_add->tx_failure_drops +
+ sw_stats_add->tx_invalid_hwol_drops;
if (OVS_UNLIKELY(dropped)) {
struct netdev_dpdk_sw_stats *sw_stats = dev->sw_stats;
--
2.37.1
___
dev mailing list
d...@openvswitch.org
https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Bug#1016453: python-tornado breaks python-bonsai autopkgtest: 'TornadoLDAPConnectionTest' object has no attribute 'should_close_asyncio_loop'
Hi Paul, I have checked in bonsai code base, and there is no reference to should_close_asyncio_loop. It looks like a tornado thing. However, looking at the full error trace, I can see that: > ldapwhoami: unrecognized option -� > Issue LDAP Who am I? operation to request user's authzid > > usage: ldapwhoami [options] > Common options: > -d level set LDAP debugging level to `level' > -D binddn bind DN > [...] Which may cause the tests to be aborted early and cause should_close_asyncio_loop to be accessed[1] before it is defined[2] in tornado. [1]: https://salsa.debian.org/python-team/packages/python-tornado/-/blob/master/tornado/testing.py#L282 [2]: https://salsa.debian.org/python-team/packages/python-tornado/-/blob/master/tornado/testing.py#L204-230 There were a few unreleased patches from bonsai[3], I'll make a new release. [3]: https://salsa.debian.org/python-team/packages/python-bonsai/-/commit/22d2533ab8094a299d1816a46917070e2b251265 In the meantime, I think that the should_close_asyncio_loop attribute should be defined in tornado.testing.AsyncTestCase.__init__() instead of in tornado.testing.AsyncTestCase.setUp().
Bug#1016453: python-tornado breaks python-bonsai autopkgtest: 'TornadoLDAPConnectionTest' object has no attribute 'should_close_asyncio_loop'
Hi Paul, I have checked in bonsai code base, and there is no reference to should_close_asyncio_loop. It looks like a tornado thing. However, looking at the full error trace, I can see that: > ldapwhoami: unrecognized option -� > Issue LDAP Who am I? operation to request user's authzid > > usage: ldapwhoami [options] > Common options: > -d level set LDAP debugging level to `level' > -D binddn bind DN > [...] Which may cause the tests to be aborted early and cause should_close_asyncio_loop to be accessed[1] before it is defined[2] in tornado. [1]: https://salsa.debian.org/python-team/packages/python-tornado/-/blob/master/tornado/testing.py#L282 [2]: https://salsa.debian.org/python-team/packages/python-tornado/-/blob/master/tornado/testing.py#L204-230 There were a few unreleased patches from bonsai[3], I'll make a new release. [3]: https://salsa.debian.org/python-team/packages/python-bonsai/-/commit/22d2533ab8094a299d1816a46917070e2b251265 In the meantime, I think that the should_close_asyncio_loop attribute should be defined in tornado.testing.AsyncTestCase.__init__() instead of in tornado.testing.AsyncTestCase.setUp().
Robin Jarry: Declaration of intent to become a DM
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Hi all, I would like to apply to change my status in Debian to Debian Maintainer. I have worked on buildbot and aerc for a while and I would like to be able to upload my own packages without a sponsor. Thanks. -BEGIN PGP SIGNATURE- iHUEARYKAB0WIQTYtqmQfDq3IEKLYGhicY4NZm/DNQUCYtUsfgAKCRBicY4NZm/D Nd+oAQC2TU8NosVmyBE6yyXObpZGRbrbs2wKove1Dv5V0wGCEgEA8NqCWa+yAF3V dTf0mwm69+u9QTAdQXSRtYF5pwslNw8= =GnVk -END PGP SIGNATURE- Robin Jarry (via nm.debian.org) For details and to comment, visit https://nm.debian.org/process/1092/ -- https://nm.debian.org/process/1092/
Re: RFS: aerc
Nilesh Patra, Jul 15, 2022 at 18:48: > Uploaded, thank you. Thanks! > Was the fix int64 v/s int32 thingy only -- just a type conversion? > > Admittedly I didn't go through the code. Here is the upstream patch: https://lists.sr.ht/~rjarry/aerc-devel/patches/33896 There are integers involved, however I did not pay too much attention to the fix itself.
Re: RFS: aerc
Nilesh Patra, Jul 15, 2022 at 18:04: > The CI for i386 is failing due to one failing test, see[1] > (Please check once before sending RFS :))) > > Can you fix this? Otherwise this package won't migrate to testing. > > [1]: https://salsa.debian.org/go-team/packages/aerc/-/jobs/2999193 Hi Nilesh, I have included an upstream patch that fixes the i386 tests. However the piuparts job is still failing. I have looked at the log and I don't understand what is wrong. https://salsa.debian.org/go-team/packages/aerc/-/jobs/2999245 0m20.6s ERROR: FAIL: After purging files have disappeared: /etc/apt/sources.list.d/debian.sources not owned 0m20.6s ERROR: FAIL: Installation and purging test. Maybe there is another error related to aerc itself but I could not find it.
RFS: aerc
Hi all, could a kind soul upload aerc on my behalf? Thanks! -- Robin
Re: Sponsor upload request: golang-github-emersion-go-mbox
Nilesh Patra, Jul 04, 2022 at 16:17: > Please do the following: > - mention the license of 'reader_test.go' in your d/copyright. > - lintian error: "E: golang-github-emersion-go-mbox-dev: > extended-description-is-empty" > - out-of-date-standards-version 4.6.0 (released 2021-08-18) (current is > 4.6.1.0) Done. Thanks for reviewing!
Sponsor upload request: golang-github-emersion-go-mbox
Hi Nilesh, all, I am preparing for packaging the next release of aerc: https://git.sr.ht/~rjarry/aerc https://salsa.debian.org/go-team/packages/aerc/ It requires a new dependency which is not available in Debian yet: github.com/emersion/go-mboxhttps://bugs.debian.org/1014313 Using dh-make-golang, I have prepared the package on salsa: https://salsa.debian.org/go-team/packages/golang-github-emersion-go-mbox Could a kind soul please upload them to NEW on my behalf? Thanks a bunch.
Bug#1014313: ITP: golang-github-emersion-go-mbox -- Package mbox parses the mbox file format into messages and formats messages into mbox files
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-mbox Version : 1.0.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-mbox * License : Expat Programming Lang: Go Description : Package mbox parses the mbox file format into messages and formats messages into mbox files New dependency for aerc.
Bug#1014313: ITP: golang-github-emersion-go-mbox -- Package mbox parses the mbox file format into messages and formats messages into mbox files
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-mbox Version : 1.0.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-mbox * License : Expat Programming Lang: Go Description : Package mbox parses the mbox file format into messages and formats messages into mbox files New dependency for aerc.
Bug#1014313: ITP: golang-github-emersion-go-mbox -- Package mbox parses the mbox file format into messages and formats messages into mbox files
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-mbox Version : 1.0.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-mbox * License : Expat Programming Lang: Go Description : Package mbox parses the mbox file format into messages and formats messages into mbox files New dependency for aerc.
Bug#1014313: ITP: golang-github-emersion-go-mbox -- Package mbox parses the mbox file format into messages and formats messages into mbox files
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-mbox Version : 1.0.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-mbox * License : Expat Programming Lang: Go Description : Package mbox parses the mbox file format into messages and formats messages into mbox files New dependency for aerc.
Re: Request for sponsor uploads
Nilesh Patra, Jun 29, 2022 at 10:23: > Please add in yourself to uploaders for aerc if you'd like. I will. > Uploaded. However please take a look at my commits. Awesome, thanks!
Request for sponsor uploads
Hi Nilesh, all, I am preparing for packaging the next release of aerc: https://git.sr.ht/~rjarry/aerc https://salsa.debian.org/go-team/packages/aerc/ It requires three new dependencies which are not available in Debian yet: * github.com/arran4/golang-ical https://bugs.debian.org/1013927 * github.com/emersion/go-msgauthhttps://bugs.debian.org/1013928 - github.com/emersion/go-milter https://bugs.debian.org/1013966 Using dh-make-golang, I have prepared all three packages on salsa: https://salsa.debian.org/go-team/packages/golang-github-arran4-golang-ical https://salsa.debian.org/go-team/packages/golang-github-emersion-go-msgauth https://salsa.debian.org/go-team/packages/golang-github-emersion-go-milter Could a kind soul please upload them to NEW on my behalf? Thanks a bunch. -- Robin
Bug#1013966: ITP: golang-github-emersion-go-milter -- Go library to write mail filters
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-milter Version : 0.3.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-milter * License : BSD-2-clause Programming Lang: Go Description : Go library to write mail filters go-milter . GoDoc (https://godoc.org/github.com/emersion/go-milter) builds.sr.ht status (https://builds.sr.ht/~emersion/go-milter/commits?) . A Go library to write mail filters. . License . BSD 2-Clause Dependency of golang-github-emersion-go-msgauth, indirect dependency of aerc.
Bug#1013966: ITP: golang-github-emersion-go-milter -- Go library to write mail filters
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-milter Version : 0.3.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-milter * License : BSD-2-clause Programming Lang: Go Description : Go library to write mail filters go-milter . GoDoc (https://godoc.org/github.com/emersion/go-milter) builds.sr.ht status (https://builds.sr.ht/~emersion/go-milter/commits?) . A Go library to write mail filters. . License . BSD 2-Clause Dependency of golang-github-emersion-go-msgauth, indirect dependency of aerc.
Bug#1013966: ITP: golang-github-emersion-go-milter -- Go library to write mail filters
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-milter Version : 0.3.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-milter * License : BSD-2-clause Programming Lang: Go Description : Go library to write mail filters go-milter . GoDoc (https://godoc.org/github.com/emersion/go-milter) builds.sr.ht status (https://builds.sr.ht/~emersion/go-milter/commits?) . A Go library to write mail filters. . License . BSD 2-Clause Dependency of golang-github-emersion-go-msgauth, indirect dependency of aerc.
Bug#1013966: ITP: golang-github-emersion-go-milter -- Go library to write mail filters
Package: wnpp Severity: wishlist Owner: Robin Jarry * Package name: golang-github-emersion-go-milter Version : 0.3.3-1 Upstream Author : Simon Ser * URL : https://github.com/emersion/go-milter * License : BSD-2-clause Programming Lang: Go Description : Go library to write mail filters go-milter . GoDoc (https://godoc.org/github.com/emersion/go-milter) builds.sr.ht status (https://builds.sr.ht/~emersion/go-milter/commits?) . A Go library to write mail filters. . License . BSD 2-Clause Dependency of golang-github-emersion-go-msgauth, indirect dependency of aerc.
Bug#1013927: ITP: golang-github-arran4-golang-ical -- A ICS / ICal parser and serialiser for Golang.
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-arran4-golang-ical
Version : 0.0~git20220517.fd89fefb0182-1
Upstream Author : Arran Ubels
* URL : https://github.com/arran4/golang-ical
* License : Apache-2.0
Programming Lang: Go
Description : A ICS / ICal parser and serialiser for Golang.
golang-ical
.
A ICS / ICal parser and serialiser for Golang.
.
Because the other libraries didn't quite do what I needed.
.
Usage, parsing:
.
cal, err := ParseCalendar(strings.NewReader(input))
.
.
Creating:
.
cal := ics.NewCalendar()
cal.SetMethod(ics.MethodRequest)
event := cal.AddEvent(fmt.Sprintf("id@domain",
p.SessionKey.IntID()))
event.SetCreatedTime(time.Now())
event.SetDtStampTime(time.Now())
event.SetModifiedAt(time.Now())
event.SetStartAt(time.Now())
event.SetEndAt(time.Now())
event.SetSummary("Summary")
event.SetLocation("Address")
event.SetDescription("Description")
event.SetURL("https://URL/;)
event.AddRrule(fmt.Sprintf("FREQ=YEARLY;BYMONTH=%d;BYMONTHDAY=%d",
time.Now().Month(), time.Now().Day()))
event.SetOrganizer("sender@domain", ics.WithCN("This Machine"))
event.AddAttendee("reciever or participant",
ics.CalendarUserTypeIndividual, ics.ParticipationStatusNeedsAction,
ics.ParticipationRoleReqParticipant, ics.WithRSVP(true))
return cal.Serialize()
.
Helper methods created as needed feel free to send a P.R. with more.
This is a new build dependency for aerc.
Bug#1013928: ITP: golang-github-emersion-go-msgauth -- A Go library and tools for DKIM, DMARC and Authentication-Results
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-emersion-go-msgauth
Version : 0.6.6-1
Upstream Author : Simon Ser
* URL : https://github.com/emersion/go-msgauth
* License : Expat
Programming Lang: Go
Description : A Go library and tools for DKIM, DMARC and
Authentication-Results
go-msgauth
.
godocs.io (https://godocs.io/github.com/emersion/go-msgauth) builds.sr.ht
status (https://builds.sr.ht/~emersion/go-msgauth/commits/master)
.
A Go library and tools to authenticate e-mails:
.
* Create and verify DKIM signatures
(https://tools.ietf.org/html/rfc6376)
* Create and parse Authentication-Results header fields
(https://tools.ietf.org/html/rfc7601)
* Fetch DMARC (http://tools.ietf.org/html/rfc7489) records
.
DKIM godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dkim)
.
Sign
.
r := strings.NewReader(mailString)
.
options := {
Domain: "example.org",
Selector: "brisbane",
Signer: privateKey,
}
.
var b bytes.Buffer
if err := dkim.Sign(, r, options); err != nil {
log.Fatal(err)
}
.
Verify
.
r := strings.NewReader(mailString)
.
verifications, err := dkim.Verify(r)
if err != nil {
log.Fatal(err)
}
.
for _, v := range verifications {
if v.Err == nil {
log.Println("Valid signature for:", v.Domain)
} else {
log.Println("Invalid signature for:", v.Domain, v.Err)
}
}
.
FAQ
.
**Why can't I verify a mail.Message directly?** A mail.Message header is
already parsed, and whitespace characters (especially continuation
lines) are removed. Thus, the signature computed from the parsed header
is not the same as the one computed from the raw header.
.
**How can I publish my public key?** You have to add a TXT record to
your DNS zone. See RFC 6376 appendix C
(https://tools.ietf.org/html/rfc6376#appendix-C). You can use the dkim-
keygen tool included in go-msgauth to generate the key and the TXT
record.
.
Authentication-Results godocs.io
(https://godocs.io/github.com/emersion/go-msgauth/authres)
.
// Format
results := []authres.Result{
{Value: authres.ResultPass, From: "example.net"},
{Value: authres.ResultPass, Auth:
"sen...@example.com"},
}
s := authres.Format("example.com", results)
log.Println(s)
.
// Parse
identifier, results, err := authres.Parse(s)
if err != nil {
log.Fatal(err)
}
.
log.Println(identifier, results)
.
DMARC godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dmarc)
.
See the GoDoc page.
.
Tools
.
A few tools are included in go-msgauth:
.
* dkim-keygen: generate a DKIM key
* dkim-milter: a mail filter to sign and verify DKIM signatures
* dkim-verify: verify a DKIM-signed email
* dmarc-lookup: lookup the DMARC policy of a domain
.
License
.
MIT
This is a new build dependency for aerc.
Bug#1013928: ITP: golang-github-emersion-go-msgauth -- A Go library and tools for DKIM, DMARC and Authentication-Results
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-emersion-go-msgauth
Version : 0.6.6-1
Upstream Author : Simon Ser
* URL : https://github.com/emersion/go-msgauth
* License : Expat
Programming Lang: Go
Description : A Go library and tools for DKIM, DMARC and
Authentication-Results
go-msgauth
.
godocs.io (https://godocs.io/github.com/emersion/go-msgauth) builds.sr.ht
status (https://builds.sr.ht/~emersion/go-msgauth/commits/master)
.
A Go library and tools to authenticate e-mails:
.
* Create and verify DKIM signatures
(https://tools.ietf.org/html/rfc6376)
* Create and parse Authentication-Results header fields
(https://tools.ietf.org/html/rfc7601)
* Fetch DMARC (http://tools.ietf.org/html/rfc7489) records
.
DKIM godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dkim)
.
Sign
.
r := strings.NewReader(mailString)
.
options := {
Domain: "example.org",
Selector: "brisbane",
Signer: privateKey,
}
.
var b bytes.Buffer
if err := dkim.Sign(, r, options); err != nil {
log.Fatal(err)
}
.
Verify
.
r := strings.NewReader(mailString)
.
verifications, err := dkim.Verify(r)
if err != nil {
log.Fatal(err)
}
.
for _, v := range verifications {
if v.Err == nil {
log.Println("Valid signature for:", v.Domain)
} else {
log.Println("Invalid signature for:", v.Domain, v.Err)
}
}
.
FAQ
.
**Why can't I verify a mail.Message directly?** A mail.Message header is
already parsed, and whitespace characters (especially continuation
lines) are removed. Thus, the signature computed from the parsed header
is not the same as the one computed from the raw header.
.
**How can I publish my public key?** You have to add a TXT record to
your DNS zone. See RFC 6376 appendix C
(https://tools.ietf.org/html/rfc6376#appendix-C). You can use the dkim-
keygen tool included in go-msgauth to generate the key and the TXT
record.
.
Authentication-Results godocs.io
(https://godocs.io/github.com/emersion/go-msgauth/authres)
.
// Format
results := []authres.Result{
{Value: authres.ResultPass, From: "example.net"},
{Value: authres.ResultPass, Auth:
"sen...@example.com"},
}
s := authres.Format("example.com", results)
log.Println(s)
.
// Parse
identifier, results, err := authres.Parse(s)
if err != nil {
log.Fatal(err)
}
.
log.Println(identifier, results)
.
DMARC godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dmarc)
.
See the GoDoc page.
.
Tools
.
A few tools are included in go-msgauth:
.
* dkim-keygen: generate a DKIM key
* dkim-milter: a mail filter to sign and verify DKIM signatures
* dkim-verify: verify a DKIM-signed email
* dmarc-lookup: lookup the DMARC policy of a domain
.
License
.
MIT
This is a new build dependency for aerc.
Bug#1013928: ITP: golang-github-emersion-go-msgauth -- A Go library and tools for DKIM, DMARC and Authentication-Results
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-emersion-go-msgauth
Version : 0.6.6-1
Upstream Author : Simon Ser
* URL : https://github.com/emersion/go-msgauth
* License : Expat
Programming Lang: Go
Description : A Go library and tools for DKIM, DMARC and
Authentication-Results
go-msgauth
.
godocs.io (https://godocs.io/github.com/emersion/go-msgauth) builds.sr.ht
status (https://builds.sr.ht/~emersion/go-msgauth/commits/master)
.
A Go library and tools to authenticate e-mails:
.
* Create and verify DKIM signatures
(https://tools.ietf.org/html/rfc6376)
* Create and parse Authentication-Results header fields
(https://tools.ietf.org/html/rfc7601)
* Fetch DMARC (http://tools.ietf.org/html/rfc7489) records
.
DKIM godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dkim)
.
Sign
.
r := strings.NewReader(mailString)
.
options := {
Domain: "example.org",
Selector: "brisbane",
Signer: privateKey,
}
.
var b bytes.Buffer
if err := dkim.Sign(, r, options); err != nil {
log.Fatal(err)
}
.
Verify
.
r := strings.NewReader(mailString)
.
verifications, err := dkim.Verify(r)
if err != nil {
log.Fatal(err)
}
.
for _, v := range verifications {
if v.Err == nil {
log.Println("Valid signature for:", v.Domain)
} else {
log.Println("Invalid signature for:", v.Domain, v.Err)
}
}
.
FAQ
.
**Why can't I verify a mail.Message directly?** A mail.Message header is
already parsed, and whitespace characters (especially continuation
lines) are removed. Thus, the signature computed from the parsed header
is not the same as the one computed from the raw header.
.
**How can I publish my public key?** You have to add a TXT record to
your DNS zone. See RFC 6376 appendix C
(https://tools.ietf.org/html/rfc6376#appendix-C). You can use the dkim-
keygen tool included in go-msgauth to generate the key and the TXT
record.
.
Authentication-Results godocs.io
(https://godocs.io/github.com/emersion/go-msgauth/authres)
.
// Format
results := []authres.Result{
{Value: authres.ResultPass, From: "example.net"},
{Value: authres.ResultPass, Auth:
"sen...@example.com"},
}
s := authres.Format("example.com", results)
log.Println(s)
.
// Parse
identifier, results, err := authres.Parse(s)
if err != nil {
log.Fatal(err)
}
.
log.Println(identifier, results)
.
DMARC godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dmarc)
.
See the GoDoc page.
.
Tools
.
A few tools are included in go-msgauth:
.
* dkim-keygen: generate a DKIM key
* dkim-milter: a mail filter to sign and verify DKIM signatures
* dkim-verify: verify a DKIM-signed email
* dmarc-lookup: lookup the DMARC policy of a domain
.
License
.
MIT
This is a new build dependency for aerc.
Bug#1013927: ITP: golang-github-arran4-golang-ical -- A ICS / ICal parser and serialiser for Golang.
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-arran4-golang-ical
Version : 0.0~git20220517.fd89fefb0182-1
Upstream Author : Arran Ubels
* URL : https://github.com/arran4/golang-ical
* License : Apache-2.0
Programming Lang: Go
Description : A ICS / ICal parser and serialiser for Golang.
golang-ical
.
A ICS / ICal parser and serialiser for Golang.
.
Because the other libraries didn't quite do what I needed.
.
Usage, parsing:
.
cal, err := ParseCalendar(strings.NewReader(input))
.
.
Creating:
.
cal := ics.NewCalendar()
cal.SetMethod(ics.MethodRequest)
event := cal.AddEvent(fmt.Sprintf("id@domain",
p.SessionKey.IntID()))
event.SetCreatedTime(time.Now())
event.SetDtStampTime(time.Now())
event.SetModifiedAt(time.Now())
event.SetStartAt(time.Now())
event.SetEndAt(time.Now())
event.SetSummary("Summary")
event.SetLocation("Address")
event.SetDescription("Description")
event.SetURL("https://URL/;)
event.AddRrule(fmt.Sprintf("FREQ=YEARLY;BYMONTH=%d;BYMONTHDAY=%d",
time.Now().Month(), time.Now().Day()))
event.SetOrganizer("sender@domain", ics.WithCN("This Machine"))
event.AddAttendee("reciever or participant",
ics.CalendarUserTypeIndividual, ics.ParticipationStatusNeedsAction,
ics.ParticipationRoleReqParticipant, ics.WithRSVP(true))
return cal.Serialize()
.
Helper methods created as needed feel free to send a P.R. with more.
This is a new build dependency for aerc.
Bug#1013927: ITP: golang-github-arran4-golang-ical -- A ICS / ICal parser and serialiser for Golang.
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-arran4-golang-ical
Version : 0.0~git20220517.fd89fefb0182-1
Upstream Author : Arran Ubels
* URL : https://github.com/arran4/golang-ical
* License : Apache-2.0
Programming Lang: Go
Description : A ICS / ICal parser and serialiser for Golang.
golang-ical
.
A ICS / ICal parser and serialiser for Golang.
.
Because the other libraries didn't quite do what I needed.
.
Usage, parsing:
.
cal, err := ParseCalendar(strings.NewReader(input))
.
.
Creating:
.
cal := ics.NewCalendar()
cal.SetMethod(ics.MethodRequest)
event := cal.AddEvent(fmt.Sprintf("id@domain",
p.SessionKey.IntID()))
event.SetCreatedTime(time.Now())
event.SetDtStampTime(time.Now())
event.SetModifiedAt(time.Now())
event.SetStartAt(time.Now())
event.SetEndAt(time.Now())
event.SetSummary("Summary")
event.SetLocation("Address")
event.SetDescription("Description")
event.SetURL("https://URL/;)
event.AddRrule(fmt.Sprintf("FREQ=YEARLY;BYMONTH=%d;BYMONTHDAY=%d",
time.Now().Month(), time.Now().Day()))
event.SetOrganizer("sender@domain", ics.WithCN("This Machine"))
event.AddAttendee("reciever or participant",
ics.CalendarUserTypeIndividual, ics.ParticipationStatusNeedsAction,
ics.ParticipationRoleReqParticipant, ics.WithRSVP(true))
return cal.Serialize()
.
Helper methods created as needed feel free to send a P.R. with more.
This is a new build dependency for aerc.
Bug#1013928: ITP: golang-github-emersion-go-msgauth -- A Go library and tools for DKIM, DMARC and Authentication-Results
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-emersion-go-msgauth
Version : 0.6.6-1
Upstream Author : Simon Ser
* URL : https://github.com/emersion/go-msgauth
* License : Expat
Programming Lang: Go
Description : A Go library and tools for DKIM, DMARC and
Authentication-Results
go-msgauth
.
godocs.io (https://godocs.io/github.com/emersion/go-msgauth) builds.sr.ht
status (https://builds.sr.ht/~emersion/go-msgauth/commits/master)
.
A Go library and tools to authenticate e-mails:
.
* Create and verify DKIM signatures
(https://tools.ietf.org/html/rfc6376)
* Create and parse Authentication-Results header fields
(https://tools.ietf.org/html/rfc7601)
* Fetch DMARC (http://tools.ietf.org/html/rfc7489) records
.
DKIM godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dkim)
.
Sign
.
r := strings.NewReader(mailString)
.
options := {
Domain: "example.org",
Selector: "brisbane",
Signer: privateKey,
}
.
var b bytes.Buffer
if err := dkim.Sign(, r, options); err != nil {
log.Fatal(err)
}
.
Verify
.
r := strings.NewReader(mailString)
.
verifications, err := dkim.Verify(r)
if err != nil {
log.Fatal(err)
}
.
for _, v := range verifications {
if v.Err == nil {
log.Println("Valid signature for:", v.Domain)
} else {
log.Println("Invalid signature for:", v.Domain, v.Err)
}
}
.
FAQ
.
**Why can't I verify a mail.Message directly?** A mail.Message header is
already parsed, and whitespace characters (especially continuation
lines) are removed. Thus, the signature computed from the parsed header
is not the same as the one computed from the raw header.
.
**How can I publish my public key?** You have to add a TXT record to
your DNS zone. See RFC 6376 appendix C
(https://tools.ietf.org/html/rfc6376#appendix-C). You can use the dkim-
keygen tool included in go-msgauth to generate the key and the TXT
record.
.
Authentication-Results godocs.io
(https://godocs.io/github.com/emersion/go-msgauth/authres)
.
// Format
results := []authres.Result{
{Value: authres.ResultPass, From: "example.net"},
{Value: authres.ResultPass, Auth:
"sen...@example.com"},
}
s := authres.Format("example.com", results)
log.Println(s)
.
// Parse
identifier, results, err := authres.Parse(s)
if err != nil {
log.Fatal(err)
}
.
log.Println(identifier, results)
.
DMARC godocs.io (https://godocs.io/github.com/emersion/go-msgauth/dmarc)
.
See the GoDoc page.
.
Tools
.
A few tools are included in go-msgauth:
.
* dkim-keygen: generate a DKIM key
* dkim-milter: a mail filter to sign and verify DKIM signatures
* dkim-verify: verify a DKIM-signed email
* dmarc-lookup: lookup the DMARC policy of a domain
.
License
.
MIT
This is a new build dependency for aerc.
Bug#1013927: ITP: golang-github-arran4-golang-ical -- A ICS / ICal parser and serialiser for Golang.
Package: wnpp
Severity: wishlist
Owner: Robin Jarry
* Package name: golang-github-arran4-golang-ical
Version : 0.0~git20220517.fd89fefb0182-1
Upstream Author : Arran Ubels
* URL : https://github.com/arran4/golang-ical
* License : Apache-2.0
Programming Lang: Go
Description : A ICS / ICal parser and serialiser for Golang.
golang-ical
.
A ICS / ICal parser and serialiser for Golang.
.
Because the other libraries didn't quite do what I needed.
.
Usage, parsing:
.
cal, err := ParseCalendar(strings.NewReader(input))
.
.
Creating:
.
cal := ics.NewCalendar()
cal.SetMethod(ics.MethodRequest)
event := cal.AddEvent(fmt.Sprintf("id@domain",
p.SessionKey.IntID()))
event.SetCreatedTime(time.Now())
event.SetDtStampTime(time.Now())
event.SetModifiedAt(time.Now())
event.SetStartAt(time.Now())
event.SetEndAt(time.Now())
event.SetSummary("Summary")
event.SetLocation("Address")
event.SetDescription("Description")
event.SetURL("https://URL/;)
event.AddRrule(fmt.Sprintf("FREQ=YEARLY;BYMONTH=%d;BYMONTHDAY=%d",
time.Now().Month(), time.Now().Day()))
event.SetOrganizer("sender@domain", ics.WithCN("This Machine"))
event.AddAttendee("reciever or participant",
ics.CalendarUserTypeIndividual, ics.ParticipationStatusNeedsAction,
ics.ParticipationRoleReqParticipant, ics.WithRSVP(true))
return cal.Serialize()
.
Helper methods created as needed feel free to send a P.R. with more.
This is a new build dependency for aerc.
Bug#1004344: how to generate list of JS deps to package
Martin, Jun 20, 2022 at 15:49: > See https://wiki.debian.org/Javascript/Nodejs/Tasks/ > > The tool generates the source code for a wiki page, e.g. like this one: Hi, I ran the script on buildbot-build-common which is not a real installable package, only a toolkit used by www plugins (it took more than 2 hours). https://paste.sr.ht/~rjarry/2197906bc7f706e726dddfe3071a27b0a82c5f80 I cannot subscribe to the wiki to create a new page: "Automatic account creation disabled to stop spammers signing up." In any case, there seem to be a lot of missing packages. This does not look like a trivial task :) Cheers, Robin
Re: [ovs-dev] User space connection tracking benchmarks
Paolo Valerio, Jun 05, 2022 at 19:37: > Just a note that may be useful. > After some tests, I noticed that establishing e.g. two TCP connections, > and leaving the first one idle after 3whs, once the second connection > expires (after moving to TIME_WAIT as a result of termination), the > second doesn't get evicted until any event gets scheduled for the first. > > ovs-appctl dpctl/dump-conntrack -s > tcp,orig=(src=10.1.1.1,dst=10.1.1.2,sport=9090,dport=8080),reply=(src=10.1.1.2,dst=10.1.1.1,sport=8080,dport=9090),zone=1,timeout=84576,protoinfo=(state=ESTABLISHED) > tcp,orig=(src=10.1.1.1,dst=10.1.1.2,sport=9091,dport=8080),reply=(src=10.1.1.2,dst=10.1.1.1,sport=8080,dport=9091),zone=1,timeout=0,protoinfo=(state=TIME_WAIT) > > This may be somewhat related to your results as during the > test, the number of connections may reach the limit so apparently reducing > the performances. Indeed, there was an issue in my test procedure. Due to the way T-Rex generates connections, it is easy to fill the conntrack table after a few iterations, making the test results inconsistent. Also, the flows which I had configured were not correct. There was an extraneous action=NORMAL flow at the end. When the conntrack table is full and a new packet cannot be tracked, it is marked as +trk+inv and not dropped. This behaviour is specific to the userspace datapath. The linux kernel datapath seems to drop the packet when it cannot be added to connection tracking. Gaëtan's series (v4) seems less resilient to the conntrack table being full, especially when there is more than one PMD core. I have changed the t-rex script to allow running arbitrary commands in between traffic iterations. This is leveraged to flush the conntrack table and run each iteration in the same conditions. https://github.com/cisco-system-traffic-generator/trex-core/blob/v2.98/scripts/cps_ndr.py To avoid filling the conntrack table, the max size was increased to 50M. The DUT configuration can be summarized as the following: ovs-vsctl set open_vswitch . other_config:dpdk-init=true ovs-vsctl set open_vswitch . other_config:pmd-cpu-mask="0x15554" ovs-vsctl add-br br0 -- set bridge br0 datapath_type=netdev ovs-vsctl add-port br0 pf0 -- set Interface pf0 type=dpdk \ options:dpdk-devargs=:3b:00.0 options:n_rxq=4 options:n_rxq_desc=4096 ovs-vsctl add-port br0 pf1 -- set Interface pf1 type=dpdk \ options:dpdk-devargs=:3b:00.1 options:n_rxq=4 options:n_rxq_desc=4096 ovs-appctl dpctl/ct-set-maxconns 5000 ovs-ofctl add-flow br0 "table=0,priority=10,ip,ct_state=-trk,actions=ct(table=0)" ovs-ofctl add-flow br0 "table=0,priority=10,ip,ct_state=+trk+new,actions=ct(commit),NORMAL" ovs-ofctl add-flow br0 "table=0,priority=10,ip,ct_state=+trk+est,actions=NORMAL" ovs-ofctl add-flow br0 "table=0,priority=0,actions=drop" Short Lived Connections --- ./cps_ndr.py --sample-time 10 --max-iterations 8 --error-threshold 0.01 \ --reset-command "ssh $dut ovs-appctl dpctl/flush-conntrack" \ --udp-percent 1 --num-messages 1 --message-size 20 --server-wait 0 \ --min-cps 10k --max-cps 600k == === == == === Series Num. Flows CPSPPSBPS == === == == === Baseline 40.1K 79.3K/s556Kp/s347Mb/s Gaetan v1 60.5K 121K/s 837Kp/s522Mb/s Gaetan v4 61.4K 122K/s 850Kp/s530Mb/s Paolo 377K756K/s 5.3Mp/s3.3Gb/s == === == == === Even after fixing the test procedure, Paolo's series still performs a lot better with short lived connections. Long Lived Connections -- ./cps_ndr.py --sample-time 30 --max-iterations 8 --error-threshold 0.01 \ --reset-command "ssh $dut ovs-appctl dpctl/flush-conntrack" \ --udp-percent 1 --num-messages 500 --message-size 20 --server-wait 50 \ --min-cps 100 --max-cps 10k == === == == === Series Num. Flows CPSPPSBPS == === == == === Baseline 17.4K 504/s 633Kp/s422Mb/s Gaetan v1 80.4K 3.1K/s 4.6Mp/s3.0Gb/s Gaetan v4 139K5.4K/s 8.2Mp/s5.4Gb/s Paolo 132K5.2K/s 7.7Mp/s5.2Gb/s == === == == === Thanks to Paolo for his help on this second round of tests. ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] User space connection tracking benchmarks
Hi all, I have been working on some benchmarks of user space connection tracking. I wanted to give some feedback on the results so that I got on two patch series that were submitted by Paolo and Gaëtan a while ago. In this intent, I have written a small script that makes use of T-Rex ASTF API to generate arbitrary TCP and UDP connections: https://github.com/rh-nfv-int/trex-core/blob/master/scripts/conntrack_ndr.py TL;DR: Both series show significant improvement in conntrack creation and destruction for a single core (around +240%). However, there seem to be scalability issues with multiple cores in Gaëtan's series. Detailed benchmark procedure and results: https://gist.github.com/rjarry/efe91f14a9bda4eb287592f696dbd123 Gaëtan Rivet's series: https://patchwork.ozlabs.org/project/openvswitch/list/?series=292039 Paolo Valerio's series (first 3 patches are missing from the web ui): https://patchwork.ozlabs.org/project/openvswitch/list/?series=291239 Cheers, -- Robin ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
[ovs-dev] [PATCH] Documentation: use new syntax for dpdk port representors
Since DPDK 21.05, the representor identifier now handles a relative VF offset. The legacy representor ID seems only valid in certain cases (first dpdk port). Link: https://github.com/DPDK/dpdk/commit/cebf7f17159a8 Signed-off-by: Robin Jarry --- Documentation/topics/dpdk/phy.rst | 12 ++-- lib/netdev-dpdk.c | 2 +- 2 files changed, 7 insertions(+), 7 deletions(-) diff --git a/Documentation/topics/dpdk/phy.rst b/Documentation/topics/dpdk/phy.rst index 937f4c40e5a8..8fc34a378cb8 100644 --- a/Documentation/topics/dpdk/phy.rst +++ b/Documentation/topics/dpdk/phy.rst @@ -267,7 +267,7 @@ Representors are multi devices created on top of one PF. For more information, refer to the `DPDK documentation`__. -__ https://doc.dpdk.org/guides-21.11/prog_guide/switch_representation.html +__ https://doc.dpdk.org/guides-21.11/prog_guide/switch_representation.html#port-representors Prior to port representors there was a one-to-one relationship between the PF and the eth device. With port representors the relationship becomes one PF to @@ -287,18 +287,18 @@ address in devargs. For an existing bridge called ``br0`` and PCI address When configuring a VF-based port, DPDK uses an extended devargs syntax which has the following format:: -BDBF,representor=[] +BDBF,representor= This syntax shows that a representor is an enumerated eth device (with -a representor ID) which uses the PF PCI address. -The following commands add representors 3 and 5 using PCI device address +a representor identifier) which uses the PF PCI address. +The following commands add representors of VF 3 and 5 using PCI device address ``:08:00.0``:: $ ovs-vsctl add-port br0 dpdk-rep3 -- set Interface dpdk-rep3 type=dpdk \ - options:dpdk-devargs=:08:00.0,representor=[3] + options:dpdk-devargs=:08:00.0,representor=vf3 $ ovs-vsctl add-port br0 dpdk-rep5 -- set Interface dpdk-rep5 type=dpdk \ - options:dpdk-devargs=:08:00.0,representor=[5] + options:dpdk-devargs=:08:00.0,representor=vf5 .. important:: diff --git a/lib/netdev-dpdk.c b/lib/netdev-dpdk.c index e0a2dccf5ce1..95f491081a11 100644 --- a/lib/netdev-dpdk.c +++ b/lib/netdev-dpdk.c @@ -1772,7 +1772,7 @@ static dpdk_port_t netdev_dpdk_get_port_by_devargs(const char *devargs) } /* - * Normally, a PCI id (optionally followed by a representor number) + * Normally, a PCI id (optionally followed by a representor identifier) * is enough for identifying a specific DPDK port. * However, for some NICs having multiple ports sharing the same PCI * id, using PCI id won't work then. -- 2.35.1 ___ dev mailing list d...@openvswitch.org https://mail.openvswitch.org/mailman/listinfo/ovs-dev
Bug#1006955: ITP: python-txrequests -- Asynchronous Python HTTP Requests for Humans using twisted
Package: wnpp Severity: wishlist Owner: Robin Jarry X-Debbugs-Cc: debian-de...@lists.debian.org * Package name: python-txrequests Version : 0.9.6 Upstream Author : Pierre Tardy * URL : https://github.com/tardyp/txrequests * License : Apache-2.0 Programming Lang: Python Description : Asynchronous Python HTTP Requests for Humans using twisted Small add-on for the python requests http library. Makes use twisted's ThreadPool, so that the requests'API returns deferred objects. This is a dependency of buildbot. Required to update to 3.4.1 and later. I intend on maintaining this package in the context of the Debian Python Team. I will need a sponsor for uploads after pushing on Salsa. Thanks!
Bug#1006955: ITP: python-txrequests -- Asynchronous Python HTTP Requests for Humans using twisted
Package: wnpp Severity: wishlist Owner: Robin Jarry X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: python-txrequests Version : 0.9.6 Upstream Author : Pierre Tardy * URL : https://github.com/tardyp/txrequests * License : Apache-2.0 Programming Lang: Python Description : Asynchronous Python HTTP Requests for Humans using twisted Small add-on for the python requests http library. Makes use twisted's ThreadPool, so that the requests'API returns deferred objects. This is a dependency of buildbot. Required to update to 3.4.1 and later. I intend on maintaining this package in the context of the Debian Python Team. I will need a sponsor for uploads after pushing on Salsa. Thanks!
Bug#1006955: ITP: python-txrequests -- Asynchronous Python HTTP Requests for Humans using twisted
Package: wnpp Severity: wishlist Owner: Robin Jarry X-Debbugs-Cc: debian-de...@lists.debian.org * Package name: python-txrequests Version : 0.9.6 Upstream Author : Pierre Tardy * URL : https://github.com/tardyp/txrequests * License : Apache-2.0 Programming Lang: Python Description : Asynchronous Python HTTP Requests for Humans using twisted Small add-on for the python requests http library. Makes use twisted's ThreadPool, so that the requests'API returns deferred objects. This is a dependency of buildbot. Required to update to 3.4.1 and later. I intend on maintaining this package in the context of the Debian Python Team. I will need a sponsor for uploads after pushing on Salsa. Thanks!
Bug#1004344: buildbot: Fails to load the web interface
Hi, this is a known issue. The buildbot-www package cannot be integrated in Debian due to tons of missing JS dependencies. See buildbot(7) and bug #883529 for more details. Since then, buildbot has changed from coffeescript+gulp to plain JS+webpack. This should make it easier to package the missing dependencies and support packaging the www plugins. Still far from a trivial task unfortunately... I am not even sure how to proceed. I would really appreciate some help on the matter. In the meantime, you may install buildbot-www via pip (although this is not a very nice workaround). Thanks.
Re: Review of python-bonsai
Hi Louis-Philippe, I think I have addressed all your comments. I have pushed fixes in a temporary branch. I'll integrate them on debian/master once the review is finished: https://salsa.debian.org/python-team/packages/python-bonsai/-/commits/review/ See my replies below: Louis-Philippe Véronneau, Dec 13, 2021 at 23:07: > * I'm curious to why you need to set "export LC_ALL = C.UTF-8". This was a leftover from another package. It is useless. > 5. d/tests: > > I don't have an autopkgtests setup that has machine-level isolation. You > ran that code and it works? Yes, here is the build log: http://files.diabeteman.com/juoghahf0teG3Phi/python-bonsai_1.3.0-1_amd64.build.txt I tried to reproduce the test environment which is prepared for the github CI actions. This involves generating a docker container with an LDAP server installed and configured, running that container with elevated privileges and run the test suite to talk to that container. It also does some tc voodoo to force timeouts on the TCP connections (see .ci/delay.py). I did not manage to run autopkgtests with containers but since the test needs to run multiple services, a virtual machine made more sense to me. Also, the use of tc may be a bit too much for a container (I'm not sure about the limitations, depending on the host). Some tests are explicitly disabled because I could not get them to work. I will ping the upstream developer with this when I get a chance. > 7. Upstream code > > Have you read the upstream code? It's something you should do (and you > should read all the changes for each new update). Not that you have to > do a proper security audit, but you should go through the code to be > sure there's no obvious or dangerous things in there. I have read most of the python code and it looks well written, documented and tested. C python extensions code is always rather obscure but I did not see anything suspicious. I am no expert in libldap2 however. The project has been around for some years now. It looks stable enough to go in Debian in my opinion. Thanks a lot for your time!
Bug#1001417: ITP: bonsai -- Python3 asyncio-compatible LDAP library
Package: wnpp Severity: wishlist Owner: Robin Jarry X-Debbugs-Cc: debian-devel@lists.debian.org * Package name: bonsai Version : 1.3.0 Upstream Author : noirello * URL : https://github.com/noirello/bonsai * License : MIT Programming Lang: Python 3 Description : Python3 asyncio-compatible LDAP library Bonsai is library for handling LDAP operations in Python. Uses libldap2 on UNIX platforms and WinLDAP on Microsoft Windows. LDAP entries are mapped to a special Python case-insensitive dictionary, tracking the changes of the dictionary to modify the entry on the server easily. * Supports only Python 3.6 or newer, and LDAPv3. * Uses LDAP libraries (OpenLDAP and WinLDAP) written in C for faster processing. * Simple pythonic design. * Implements an own dictionary-like object for mapping LDAP entries that makes easier to add and modify them. * Works with various asynchronous libraries (like asyncio, gevent). I intend on maintaining this package in the context of the Debian Python Team. I will need a sponsor for uploads after pushing on Salsa. This package is not yet in Debian but I like the design and I think it would be a great addition in the archive. Also, it is the only asyncio compatible LDAP library that I know of. Thanks!
Bug#1001417: ITP: bonsai -- Python3 asyncio-compatible LDAP library
Package: wnpp Severity: wishlist Owner: Robin Jarry X-Debbugs-Cc: debian-de...@lists.debian.org * Package name: bonsai Version : 1.3.0 Upstream Author : noirello * URL : https://github.com/noirello/bonsai * License : MIT Programming Lang: Python 3 Description : Python3 asyncio-compatible LDAP library Bonsai is library for handling LDAP operations in Python. Uses libldap2 on UNIX platforms and WinLDAP on Microsoft Windows. LDAP entries are mapped to a special Python case-insensitive dictionary, tracking the changes of the dictionary to modify the entry on the server easily. * Supports only Python 3.6 or newer, and LDAPv3. * Uses LDAP libraries (OpenLDAP and WinLDAP) written in C for faster processing. * Simple pythonic design. * Implements an own dictionary-like object for mapping LDAP entries that makes easier to add and modify them. * Works with various asynchronous libraries (like asyncio, gevent). I intend on maintaining this package in the context of the Debian Python Team. I will need a sponsor for uploads after pushing on Salsa. This package is not yet in Debian but I like the design and I think it would be a great addition in the archive. Also, it is the only asyncio compatible LDAP library that I know of. Thanks!
Bug#1001417: ITP: bonsai -- Python3 asyncio-compatible LDAP library
Package: wnpp Severity: wishlist Owner: Robin Jarry X-Debbugs-Cc: debian-de...@lists.debian.org * Package name: bonsai Version : 1.3.0 Upstream Author : noirello * URL : https://github.com/noirello/bonsai * License : MIT Programming Lang: Python 3 Description : Python3 asyncio-compatible LDAP library Bonsai is library for handling LDAP operations in Python. Uses libldap2 on UNIX platforms and WinLDAP on Microsoft Windows. LDAP entries are mapped to a special Python case-insensitive dictionary, tracking the changes of the dictionary to modify the entry on the server easily. * Supports only Python 3.6 or newer, and LDAPv3. * Uses LDAP libraries (OpenLDAP and WinLDAP) written in C for faster processing. * Simple pythonic design. * Implements an own dictionary-like object for mapping LDAP entries that makes easier to add and modify them. * Works with various asynchronous libraries (like asyncio, gevent). I intend on maintaining this package in the context of the Debian Python Team. I will need a sponsor for uploads after pushing on Salsa. This package is not yet in Debian but I like the design and I think it would be a great addition in the archive. Also, it is the only asyncio compatible LDAP library that I know of. Thanks!
Bug#993522: buildbot-worker: Setting WORKER_OPTIONS to any non-empty value doesn't work
Hi Vadim,
2021-09-02, Vadim Zeitlin:
> Package: buildbot-worker
> Version: 2.10.1-1
> Severity: normal
>
> Dear Maintainer,
>
> Setting WORKER_OPTIONS in /etc/default/buildbot-worker to e.g. "--verbose"
> doesn't work because its value is used in a wrong place in the init.d
> script: it does
>
> "$WORKER_RUNNER $op ${WORKER_OPTIONS[$mi]} ${WORKER_BASEDIR[$mi]}
>
> when the correct syntax would be
>
> "$WORKER_RUNNER ${WORKER_OPTIONS[$mi]} $op ${WORKER_BASEDIR[$mi]}
>
> i.e. the options must come _before_ the operation for buildbot-worker, not
> after it.
Actually, this is only true for the --verbose option. Other options must
be passed after $op.
> -- System Information:
> Debian Release: 11.0
> APT prefers stable-security
> APT policy: (500, 'stable-security'), (500, 'stable')
> Architecture: amd64 (x86_64)
>
> Kernel: Linux 5.10.0-8-amd64 (SMP w/12 CPU threads)
> Locale: LANG=C.UTF-8, LC_CTYPE=C.UTF-8 (charmap=UTF-8) (ignored: LC_ALL set
> to C.UTF-8), LANGUAGE=en_US:en
> Shell: /bin/sh linked to /bin/dash
> Init: systemd (via /run/systemd/system)
I see that you are using systemd. You should not use the init.d script
but the systemd unit template which is provided with the package. There
are examples in the man page to tweak the unit parameters:
https://manpages.debian.org/bullseye/buildbot-worker/buildbot-worker.7.en.html#systemd
In your case, you should override ExecStart= in a drop-in file.
Also, it looks like this is a duplicate of bug #993521. Should I close
the first one?
--
Robin
Bug#984970: buildbot 2.10.2 in Debian/bullseye
Unfortunately, I don't know if 3.0.0 is stable. It deprecates a lot of things and only has been out for a couple of days. I'll submit 2.10.2 first and then I'll check with the python team what they think.
Bug#984970: buildbot 2.10.2 in Debian/bullseye
Hi Hans, I was considering uploading 3.0.0 now that has been released. https://github.com/buildbot/buildbot/releases/tag/v3.0.0 Would that be ok?
Bug#953966: buildbot: autopkgtest failure with Python 3.8 as default
Control: severity -1 important The autopkgtests seem to work now: https://ci.debian.net/user/britney/jobs?package=buildbot[]=testing[]=amd64 The bug may have been fixed by: python3-defaults 3.8.2-2 Lets leave it open for a while to see if the tests are stable now. -- Robin
Bug#953966: buildbot: autopkgtest failure with Python 3.8 as default
Control: severity -1 important The autopkgtests seem to work now: https://ci.debian.net/user/britney/jobs?package=buildbot[]=testing[]=amd64 The bug may have been fixed by: python3-defaults 3.8.2-2 Lets leave it open for a while to see if the tests are stable now. -- Robin
Accepted buildbot 2.4.1-1 (source) into unstable
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Format: 1.8 Date: Tue, 15 Oct 2019 12:10:59 +0200 Source: buildbot Architecture: source Version: 2.4.1-1 Distribution: unstable Urgency: medium Maintainer: Python Applications Packaging Team Changed-By: Robin Jarry Changes: buildbot (2.4.1-1) unstable; urgency=medium . [ Ondřej Nový ] * Bump Standards-Version to 4.4.0. . [ Robin Jarry ] * Convert blockdiag diagrams to ascii drawings * d/control: remove dependency to python3-sphinxcontrib.blockdiag * Bump Standards-Version to 4.4.1 * New upstream version 2.4.1 * d/patches: adapt for new version * d/control: add build dependency to git Checksums-Sha1: bcd2819a395e6b9241ae215ac2c083842d683f16 3203 buildbot_2.4.1-1.dsc 91302fcb6609e6093edb3c67bd03f3ceb77e2dab 481 buildbot_2.4.1.orig.tar.gz f3c052e770bdd06cae0995fbadd64c584655c4f2 488 buildbot_2.4.1.orig.tar.gz.asc 1ef7515d3fa5ee8b58b328725d87d8e9ea7687f1 32568 buildbot_2.4.1-1.debian.tar.xz ddb974a3d07241eab9d0a223a0d20c2d68e4eced 8625 buildbot_2.4.1-1_source.buildinfo Checksums-Sha256: 59ab40a9f50f4a59e87be4541363ae0e47cdcce361d49782005d5c4010dd4e07 3203 buildbot_2.4.1-1.dsc 6237a4c61a6208324ca246b328ba01660fa40a40cce20430963b33901cd80672 481 buildbot_2.4.1.orig.tar.gz 286129ea0369171ad78ca75feeafaca1611199f1d097d2c8de34d72d0e94897e 488 buildbot_2.4.1.orig.tar.gz.asc 48c77c64449d050abc6efeaa5cf55e7c5c1da5baada5876341c8b04026544671 32568 buildbot_2.4.1-1.debian.tar.xz 525b37effb285f5e9f7a0e3b9d07a1d143b177e26c83ff8a1e05cec751852876 8625 buildbot_2.4.1-1_source.buildinfo Files: c45903e44e02716f556495c8dd1046a7 3203 devel optional buildbot_2.4.1-1.dsc 50d2aacbb1f8e11e9fe6ccb588bf89a4 481 devel optional buildbot_2.4.1.orig.tar.gz 20df0a63aa2413eb62e593846e7973c2 488 devel optional buildbot_2.4.1.orig.tar.gz.asc 3c40f75c68b66f53ab900350803cb6c2 32568 devel optional buildbot_2.4.1-1.debian.tar.xz 25ab1077ac5c954e9d8d7b6be6d50381 8625 devel optional buildbot_2.4.1-1_source.buildinfo -BEGIN PGP SIGNATURE- iQJDBAEBCgAtFiEExyA8CpIGcL+U8AuxsB0acqyNyaEFAl2m184PHGpjY0BkZWJp YW4ub3JnAAoJELAdGnKsjcmhSzAQAISXb09BCwLqdgtcvGuX6C4t1dcxgVcRyIHz 4zMYpSglFvePXzWjMIC+/s+Ys0Xn/Kb5tdBW5SXbZTW4b037249vGoTg3YZmWCqM 3yKxYQrpfs3BeT4tuHFbqwR82b+p23Cw+7GJ8R2e8Wmc91aQDhlhjYsPRLvrfui8 7YP4bPaZvv4tAYNb3QS355+Zw52/Htq36oLc2gtkqhxU4bXHA6ksJUp1es8IaJg/ 1huEBt+V+ZOql0VETBul6cjpW40hQwplawtrz7deI7j8/VtqWWLfZIngR5PdEf0Q uD1pa5bTCcbjsKZvsS/Ab5UJ0tSkfs1z1jNVB9PG1WVzptxFfSo+lACDcTEqmz/j U9i2fhiRqmR/d1o9o/XpGAqI/CFIQ+7ALMtQYjIT9gTNLFbQCk6w2RnF0cMAOUKf T0QGEDXe7fYaKAR9WMU+GuY1gmYepwHjr3J0Jztw9dna3tMYoYpC4xjtDJ2k4WwN NuBngECG104h/yiQmTOYg6pJyP5YQPLckkU7Wz1t2sPvc9KgSbpUp/SNy9uoJ2sW KyLg/CxehtRWIn7J7uwFq1Z745EdreAJANFX+OF0+tN+SpqYQhWrbJPq3YMlycfV iWJO/tfSLSfg+jzqCzZhr5iPOfp6E/raBGHUmzpkZETkjn0meDBZZsq1/biIzQW8 ZcOqLiwB =Qxmv -END PGP SIGNATURE-
Accepted buildbot 2.3.1-1 (source) into unstable
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Format: 1.8 Date: Tue, 09 Jul 2019 09:38:25 +0200 Source: buildbot Architecture: source Version: 2.3.1-1 Distribution: unstable Urgency: medium Maintainer: Python Applications Packaging Team Changed-By: Robin Jarry Changes: buildbot (2.3.1-1) unstable; urgency=medium . * d/upstream: add second signing key * New upstream version 2.3.1 * d/patches: remove merged by upstream * add parameterized dep * d/control: update dependencies Checksums-Sha1: b03d33fdf2fac577252add5f98f228d14c4e2223 3210 buildbot_2.3.1-1.dsc f1a8a58f22c9d5416da32b116582fc3d61f6c2ed 4700479 buildbot_2.3.1.orig.tar.gz 0c97eda7d37472365288a176823c1541c95bbb62 488 buildbot_2.3.1.orig.tar.gz.asc f5a5989e43af716bb9dd00a9dad65c1a66b0f59e 30420 buildbot_2.3.1-1.debian.tar.xz b431c86ce89c522fe0c5e19600b82a50770b0a6e 10543 buildbot_2.3.1-1_amd64.buildinfo Checksums-Sha256: 7b0e71e850884641c59c733133a9d5d26d0d297378d0f4c4797675706d4f7a5c 3210 buildbot_2.3.1-1.dsc 93241e1660cc8e0a37dba6c8c4bbb2e989f5da35006d6b5b523dfc3f37e8b927 4700479 buildbot_2.3.1.orig.tar.gz 10b9f486c3b261c278c909aaed272d1722e5dca0c7937e35be21d1ca2c261242 488 buildbot_2.3.1.orig.tar.gz.asc dd4e22f0bc4c66b2c7dc7e3ad27ca54d7f229eccee053a602ef8e67e42b95355 30420 buildbot_2.3.1-1.debian.tar.xz 1f5d9ba0394a0c93075ce1c0fae4e12f5a40854286ed22fd7494f71360122041 10543 buildbot_2.3.1-1_amd64.buildinfo Files: 31d3216a97dd288d27fe6251a61ae666 3210 devel optional buildbot_2.3.1-1.dsc e860765b1c9b592c7d964a6d934714f0 4700479 devel optional buildbot_2.3.1.orig.tar.gz 4576cd849c58c6e927dee45ad2db6c4b 488 devel optional buildbot_2.3.1.orig.tar.gz.asc 67ffef0a5b1232b33a7c45fbad96f85a 30420 devel optional buildbot_2.3.1-1.debian.tar.xz 5de96eb21d53829c91ecf74e93b39233 10543 devel optional buildbot_2.3.1-1_amd64.buildinfo -BEGIN PGP SIGNATURE- iQIzBAEBCgAdFiEEi3hoeGwz5cZMTQpICBa54Yx2K60FAl0w8GgACgkQCBa54Yx2 K61faA//RuV8AMu/lNXmpJ8tuao3FpDntXkHLZJpsn7e7XANPKcTGBGFrEseAqkN gU+ZRx2SRnjoE7Ckpj6xaGm9ZAOcLZy2Y+N0OTU1z0S2cXGzHj6dlmIWAYjBfDgV R4qLHGQ1tNnPewLyi/xyeU/88CaNXj/Tfq1gAWy2aXN8Qm7X8eefIpzLDlae2lLi nsE/3MoaMISGUWqh0H+LYcYhkKl6Ieh53kAPE/Dl+wTUW6ez7XwaYAVx8Fad6OnF kQnR4cjM1PnvdFGh1MhlaLla1NZj2H6ar3UwcfNT69xJ7ae1W7fmTKEIPW2TTqo7 5/fqtNTyhaJVJ0c2giyu0MtomkMuU4t5x6FHXbXXw4xG/PEKWvKidQAJ0oxvn4gR asxcObLHpxgJ14ElF2bciawEClrjzGjw7TIZZDgWnOeeHNbCPgR045QraXXI2tj8 yYW0kOQApkl4hy6f3x3KAf1bABB4Zu6xzC7MbUDScKJsi6oCD6VArJcIVLD73IYk RAxX+pbSt4xtp6ryALF6jhM/cvH4ESZuOF2m+Mc6um65ISvjN23iebW25vzGaNuc KQbnFc9OxG2zaNfnhH1rJ6fnA5KrNS0tjoTeOSD/4Ri15dkPG0e/hPjg9JnSzWEY xZlezHYOMoZfXGvkqTvXY/DhYrAZk8FYFuDAG+RCrTJND/a2cJo= =UWiE -END PGP SIGNATURE-
Bug#872864: gbp-buildpackage: orig signature file not exported with pristine-tar
Hi,
This problem causes a lintian warning:
W: buildbot source: orig-tarball-missing-upstream-signature
buildbot_2.3.1.orig.tar.gz
N:
N:The packaging includes an upstream signing key but the corresponding
N:.asc signature for one or more source tarballs are not included in your
N:.changes file.
N:
N:Please ensure a _.orig.tar..asc file exists in
N:the same directory as your _.orig.tar. tarball
N:prior to dpkg-source --build being called.
N:
N:If you are repackaging your source tarballs for Debian Free Software
N:Guidelines compliance reasons, ensure that your package version includes
N:dfsg or similar.
N:
N:Support for signatures was added to pristine-tar in version 1.41 and
N:support in git-buildpackage is being tracked in #872864.
N:
N:Severity: normal, Certainty: certain
N:
N:Check: control-file, Type: source
N:
I attached an inline patch that "fixes" the problem at the end of this
message but I am not entirely sure if this is the correct approach.
Please tell me if you need more info.
Thanks
--- /usr/lib/python3/dist-packages/gbp/deb/git.py.old 2019-07-09
13:01:18.913389409 +0200
+++ /usr/lib/python3/dist-packages/gbp/deb/git.py 2019-07-09
12:59:34.854238866 +0200
@@ -320,7 +320,7 @@ class DebianGitRepository(PkgGitReposito
output = source.upstream_tarball_name(comp.type, component=component)
try:
self.pristine_tar.checkout(source.name, source.upstream_version,
comp.type, output_dir,
- component=component, quiet=True)
+ component=component, quiet=True,
signature=True)
except Exception as e:
raise GitRepositoryError("Error creating %s: %s" % (output, e))
return True
--
Robin
Accepted buildbot 2.0.1-2 (source) into unstable
-BEGIN PGP SIGNED MESSAGE- Hash: SHA512 Format: 1.8 Date: Mon, 03 Jun 2019 14:47:25 +0200 Source: buildbot Architecture: source Version: 2.0.1-2 Distribution: unstable Urgency: high Maintainer: Python Applications Packaging Team Changed-By: Robin Jarry Closes: 929849 Changes: buildbot (2.0.1-2) unstable; urgency=high . * Fix OAuth module security bypass [CVE-2019-12300] (Closes: #929849) Checksums-Sha1: fd5d53656fe2b5f8b9f113b7bceef79e293ba2f6 2940 buildbot_2.0.1-2.dsc ac60fc782403d2b33a8af618733e645da8b3471d 27264 buildbot_2.0.1-2.debian.tar.xz 34364721210542b644e05ec07274b84526654dc2 10342 buildbot_2.0.1-2_amd64.buildinfo Checksums-Sha256: 847ce062f7d2aec73dfd836b69a7e5f529f7b5bfd720585822527e2386bffefc 2940 buildbot_2.0.1-2.dsc 5f5cf29f009a1368f0799d5fb2f451047526c57b9f141043517b399f93dd1b13 27264 buildbot_2.0.1-2.debian.tar.xz 9ba1194fac15fd9e16f21dfb62a940f4fb2212ce94e2b8efedec1c67f9c7ac4e 10342 buildbot_2.0.1-2_amd64.buildinfo Files: 91800be6d3435f56cb4d126d44e165c6 2940 devel optional buildbot_2.0.1-2.dsc 0296ef4ae3e68ed695253040e5eb8718 27264 devel optional buildbot_2.0.1-2.debian.tar.xz def7ae3898eb0b043c5a7ea1d6b15754 10342 devel optional buildbot_2.0.1-2_amd64.buildinfo -BEGIN PGP SIGNATURE- iQIzBAEBCgAdFiEEolIP6gqGcKZh3YxVM2L3AxpJkuEFAlz2Y78ACgkQM2L3AxpJ kuFEgg//ako9B5+N4ctX3ggA74xFbJPY4ivbTAnYWUfGOXfQ5fvXOVKJruHyns3H lH1v5dyDmOs8DQFT1Ceu7A2y4N0LR3Qgmf5kZ6p3eg7aI8qJ/caHlybh6UflBC8x sfmShzMbYAXgwN/nncF+oNQDQ6sGZZsO4mmizpVS+qZJ2Za/8xHEWcnPyqxJsmNH rG1Ntb/FXCor82dJmwWcrfTB2e/Sd+52rLPybb41hKak4zfA0xcagfP9eQue/bF9 P63+SEUHB5msogCZS/iMzk4hGIgof+HMEiS34/m2T/30ZcT4LQaX1tivDxAY/bbV rP01ZaxaNcuDJQetnSwCWXqYHbqzPG61KgfZlgCuwnXAZWL6ibVeDrAoKVlzLso5 XtKLlma8W1r1f6LvxmdZtPNJozAxuAQvrI0ZXJOz4rgd+Nwu0O/CD++GRLuUBq0b 1WzYynoXU2dHDCs1zDwAS/lv32cakInhC/Fi7FpHHoRSihS6Zja1XhwpJKgyYOPR xd9tUzyEc4r3LFEc0228OFmVfhA8ZkV5KsnLR/J2iCQLhqwuOKX4j0hYoeRjojnu OufZNbrauSncbIZlEcMXY355ggDV69hXMYqhzJdsuZYVzdusxsRlMKnq7oBeywrZ tJoDKl5d7IMJzLf71BKZ6fNLG0GAOlYfsklAsPGgNyTJ9+Gmih0= =xCkJ -END PGP SIGNATURE-
Bug#929965: unblock: buildbot/2.0.1-2
Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: unblock
Tags: security
Control: affects -1 src:buildbot
Please unblock package buildbot.
Version 2.0.1-2 resolves a grave security bug in buildbot (#929849).
A source debdiff against 2.0.1-1 follows.
Thank you!
Robin
diff -Nru buildbot-2.0.1/debian/changelog buildbot-2.0.1/debian/changelog
--- buildbot-2.0.1/debian/changelog 2019-02-11 21:26:20.0 +0100
+++ buildbot-2.0.1/debian/changelog 2019-06-03 14:47:25.0 +0200
@@ -1,3 +1,9 @@
+buildbot (2.0.1-2) unstable; urgency=high
+
+ * Fix OAuth module security bypass [CVE-2019-12300] (Closes: #929849)
+
+ -- Robin Jarry Mon, 03 Jun 2019 14:47:25 +0200
+
buildbot (2.0.1-1) unstable; urgency=medium
* Use scdoc for man pages
diff -Nru
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
---
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
1970-01-01 01:00:00.0 +0100
+++
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
2019-06-03 14:47:25.0 +0200
@@ -0,0 +1,153 @@
+From: Robin Jarry
+Date: Mon, 3 Jun 2019 14:43:12 +0200
+Subject: Revert "master: Accept GitHub `access_token` directly from user"
+
+This is a backport of upstream commit e1dcfce4388bfb: ("Revert "master:
+Accept GitHub `access_token` directly from user"").
+
+Fixes: CVE-2019-12300
+Signed-off-by: Pierre Tardy
+Signed-off-by: Robin Jarry
+---
+ master/buildbot/test/unit/test_www_oauth.py | 37 +-
+ master/buildbot/www/oauth2.py | 41 +++--
+ 2 files changed, 21 insertions(+), 57 deletions(-)
+
+diff --git a/master/buildbot/test/unit/test_www_oauth.py
b/master/buildbot/test/unit/test_www_oauth.py
+index 551f221..fd3b0de 100644
+--- a/master/buildbot/test/unit/test_www_oauth.py
b/master/buildbot/test/unit/test_www_oauth.py
+@@ -191,30 +191,7 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ 'full_name': 'foo bar'}, res)
+
+ @defer.inlineCallbacks
+-def test_GithubAcceptToken(self):
+-requests.get.side_effect = []
+-requests.post.side_effect = [
+-FakeResponse(dict(access_token="TOK3N"))]
+-self.githubAuth.get = mock.Mock(side_effect=[
+-dict( # /user
+-login="bar",
+-name="foo bar",
+-email="buzz@bar"),
+-[ # /user/emails
+-{'email': 'buzz@bar', 'verified': True, 'primary': False},
+-{'email': 'bar@foo', 'verified': True, 'primary': True}],
+-[ # /user/orgs
+-dict(login="hello"),
+-dict(login="grp"),
+-]])
+-res = yield self.githubAuth.acceptToken("TOK3N")
+-self.assertEqual({'email': 'bar@foo',
+- 'username': 'bar',
+- 'groups': ["hello", "grp"],
+- 'full_name': 'foo bar'}, res)
+-
+-@defer.inlineCallbacks
+-def test_GithubAcceptToken_v4(self):
++def test_GithubVerifyCode_v4(self):
+ requests.get.side_effect = []
+ requests.post.side_effect = [
+ FakeResponse(dict(access_token="TOK3N"))]
+@@ -243,14 +220,14 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ }
+ }
+ ])
+-res = yield self.githubAuth_v4.acceptToken("TOK3N")
++res = yield self.githubAuth_v4.verifyCode("code!")
+ self.assertEqual({'email': 'bar@foo',
+ 'username': 'bar',
+ 'groups': ["hello", "grp"],
+ 'full_name': 'foo bar'}, res)
+
+ @defer.inlineCallbacks
+-def test_GithubAcceptToken_v4_teams(self):
++def test_GithubVerifyCode_v4_teams(self):
+ requests.get.side_effect = []
+ requests.post.side_effect = [
+ FakeResponse(dict(access_token="TOK3N"))]
+@@ -331,7 +308,7 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ }
+ }
+ ])
+-res = yield self.githubAuth_v4_teams.acceptToken("TOK3N")
++res = yield self.githubAuth_v4_teams.verifyCode("code!")
+ self.assertEqual({'email': 'bar@foo',
+ 'username': 'bar',
+ 'groups': [
+@@ -434,11 +411,9 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ rsrc.auth.verifyCode.assert_cal
Bug#929965: unblock: buildbot/2.0.1-2
Package: release.debian.org
Severity: normal
User: release.debian@packages.debian.org
Usertags: unblock
Tags: security
Control: affects -1 src:buildbot
Please unblock package buildbot.
Version 2.0.1-2 resolves a grave security bug in buildbot (#929849).
A source debdiff against 2.0.1-1 follows.
Thank you!
Robin
diff -Nru buildbot-2.0.1/debian/changelog buildbot-2.0.1/debian/changelog
--- buildbot-2.0.1/debian/changelog 2019-02-11 21:26:20.0 +0100
+++ buildbot-2.0.1/debian/changelog 2019-06-03 14:47:25.0 +0200
@@ -1,3 +1,9 @@
+buildbot (2.0.1-2) unstable; urgency=high
+
+ * Fix OAuth module security bypass [CVE-2019-12300] (Closes: #929849)
+
+ -- Robin Jarry Mon, 03 Jun 2019 14:47:25 +0200
+
buildbot (2.0.1-1) unstable; urgency=medium
* Use scdoc for man pages
diff -Nru
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
---
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
1970-01-01 01:00:00.0 +0100
+++
buildbot-2.0.1/debian/patches/0005-Revert-master-Accept-GitHub-access_token-directly-fr.patch
2019-06-03 14:47:25.0 +0200
@@ -0,0 +1,153 @@
+From: Robin Jarry
+Date: Mon, 3 Jun 2019 14:43:12 +0200
+Subject: Revert "master: Accept GitHub `access_token` directly from user"
+
+This is a backport of upstream commit e1dcfce4388bfb: ("Revert "master:
+Accept GitHub `access_token` directly from user"").
+
+Fixes: CVE-2019-12300
+Signed-off-by: Pierre Tardy
+Signed-off-by: Robin Jarry
+---
+ master/buildbot/test/unit/test_www_oauth.py | 37 +-
+ master/buildbot/www/oauth2.py | 41 +++--
+ 2 files changed, 21 insertions(+), 57 deletions(-)
+
+diff --git a/master/buildbot/test/unit/test_www_oauth.py
b/master/buildbot/test/unit/test_www_oauth.py
+index 551f221..fd3b0de 100644
+--- a/master/buildbot/test/unit/test_www_oauth.py
b/master/buildbot/test/unit/test_www_oauth.py
+@@ -191,30 +191,7 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ 'full_name': 'foo bar'}, res)
+
+ @defer.inlineCallbacks
+-def test_GithubAcceptToken(self):
+-requests.get.side_effect = []
+-requests.post.side_effect = [
+-FakeResponse(dict(access_token="TOK3N"))]
+-self.githubAuth.get = mock.Mock(side_effect=[
+-dict( # /user
+-login="bar",
+-name="foo bar",
+-email="buzz@bar"),
+-[ # /user/emails
+-{'email': 'buzz@bar', 'verified': True, 'primary': False},
+-{'email': 'bar@foo', 'verified': True, 'primary': True}],
+-[ # /user/orgs
+-dict(login="hello"),
+-dict(login="grp"),
+-]])
+-res = yield self.githubAuth.acceptToken("TOK3N")
+-self.assertEqual({'email': 'bar@foo',
+- 'username': 'bar',
+- 'groups': ["hello", "grp"],
+- 'full_name': 'foo bar'}, res)
+-
+-@defer.inlineCallbacks
+-def test_GithubAcceptToken_v4(self):
++def test_GithubVerifyCode_v4(self):
+ requests.get.side_effect = []
+ requests.post.side_effect = [
+ FakeResponse(dict(access_token="TOK3N"))]
+@@ -243,14 +220,14 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ }
+ }
+ ])
+-res = yield self.githubAuth_v4.acceptToken("TOK3N")
++res = yield self.githubAuth_v4.verifyCode("code!")
+ self.assertEqual({'email': 'bar@foo',
+ 'username': 'bar',
+ 'groups': ["hello", "grp"],
+ 'full_name': 'foo bar'}, res)
+
+ @defer.inlineCallbacks
+-def test_GithubAcceptToken_v4_teams(self):
++def test_GithubVerifyCode_v4_teams(self):
+ requests.get.side_effect = []
+ requests.post.side_effect = [
+ FakeResponse(dict(access_token="TOK3N"))]
+@@ -331,7 +308,7 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ }
+ }
+ ])
+-res = yield self.githubAuth_v4_teams.acceptToken("TOK3N")
++res = yield self.githubAuth_v4_teams.verifyCode("code!")
+ self.assertEqual({'email': 'bar@foo',
+ 'username': 'bar',
+ 'groups': [
+@@ -434,11 +411,9 @@ class OAuth2Auth(www.WwwTestMixin, ConfigErrorsMixin,
unittest.TestCase):
+ rsrc.auth.verifyCode.assert_cal
Re: [Qemu-devel] -device ipmi-bmc-sim attached to -netdev vde
Hi Corey, Thanks a lot for your detailed answer! 2019-02-19, Corey Minyard: > If you really wanted to do this, you would need to implement the IPMI > LAN protocol inside QEMU and sit it on top of a UDP chardev. It could > then plug into the standard IPMI infrastructure in QEMU. The power > management functions are already there. > > The openipmi lanserv code is something you can steal from, it's at > https://github.com/cminyard/openipmi/tree/master/lanserv > > My suggestion, though, would be to implement something that ran over > TLS with two-way authentication. It doesn't look too hard to do > in qemu (though I haven't tried it) but you could have a qemu console > running over TLS that would allow you control from another qemu session. > Plus it would give you authorization and encryption on your qemu > console logins, which is probably a good thing. > > I have been working on a library that makes it easy > (easier? The pain is always in the key management) to make TLS > connections. It's at https://github.com/cminyard/gensio and you > can use it from C or Python. > But there are many tools to accomplish this. Ugh, I feared this would not be a walk in the park :) This is unfortunately too much work for me at the moment. I don't like it but I think I will have to live with vms communicating through a SLIRP device to access the ipmi lanserv daemon. I just need to figure out how this works, because vbmc-qemu does not cut it for me. It does not allow fine-tuning the QEMU command line. -- Robin
