Hi David,
On Fri, Sep 4, 2026 at 11:20 AM David Marchand
<[email protected]> wrote:
>
> On Mon, 20 Jul 2026 at 10:01, Maxime Leroy <[email protected]> wrote:
> >
> > After arming the Rx interrupts rx_interrupt_wait() sleeps in
> > rte_epoll_wait(). A packet that arrived between the last poll and the arm
> > can be missed on PMDs whose interrupt is edge-triggered on an empty to
> > non-empty transition, such as dpaa2: arming a queue that is already
> > non-empty raises no notification, so the lcore sleeps until the next
> > packet.
> >
> > Before sleeping, check whether any queue already holds traffic and, if
> > so, skip the wait and go back to polling. NICs with level-triggered
> > interrupts are unaffected: a pending packet keeps the interrupt asserted,
> > so rte_epoll_wait() would return immediately anyway.
> >
> > Signed-off-by: Maxime Leroy <[email protected]>
> > ---
> > examples/l3fwd-power/main.c | 27 ++++++++++++++++++++++++++-
> > 1 file changed, 26 insertions(+), 1 deletion(-)
> >
> > diff --git a/examples/l3fwd-power/main.c b/examples/l3fwd-power/main.c
> > index fa64e28e5c..c423b0dba5 100644
> > --- a/examples/l3fwd-power/main.c
> > +++ b/examples/l3fwd-power/main.c
> > @@ -949,6 +949,26 @@ static int event_register(struct lcore_conf *qconf)
> > return 0;
> > }
> >
> > +static bool
> > +rx_queue_pending(struct lcore_conf *qconf)
> > +{
> > + struct lcore_rx_queue *rx_queue;
> > + uint16_t queue_id;
> > + uint16_t port_id;
> > + int i;
> > +
> > + for (i = 0; i < qconf->n_rx_queue; ++i) {
> > + rx_queue = &(qconf->rx_queue_list[i]);
> > + port_id = rx_queue->port_id;
> > + queue_id = rx_queue->queue_id;
> > +
> > + if (rte_eth_rx_queue_count(port_id, queue_id) > 0)
> > + return true;
> > + }
> > +
> > + return false;
> > +}
> > +
> > static int
> > rx_interrupt_wait(struct lcore_conf *qconf, int *intr_registered)
> > {
> > @@ -968,7 +988,12 @@ rx_interrupt_wait(struct lcore_conf *qconf, int
> > *intr_registered)
> > *intr_registered = 1;
> > }
> >
> > - sleep_until_rx_interrupt(qconf->n_rx_queue, rte_lcore_id());
> > + /*
> > + * A packet that arrived during the arm window raises no new wakeup
> > on
> > + * edge-triggered PMDs, so skip the sleep if a queue already has
> > traffic.
> > + */
> > + if (!rx_queue_pending(qconf))
> > + sleep_until_rx_interrupt(qconf->n_rx_queue, rte_lcore_id());
> > rx_intr_disable_all(qconf);
> > return 0;
> > }
>
> I don't see how this solves the race.
> There is stil a window between checking the queue count and sleeping.
The check runs after the arm, not before:
rx_intr_enable_all(qconf); /* arm */
if (!rx_queue_pending(qconf)) /* check */
sleep_until_rx_interrupt(); /* sleep */
A packet arriving in the window you point at arrives with the interrupt
armed: the event is raised and latched in the event fd, so rte_epoll_wait()
returns immediately. Only a packet that arrived before the arm can be lost,
and that is what the check catches.
>
> Looks like a bug that should be solved in the driver.
This is not a dpaa2 workaround. The loop currently sleeps with a 10 ms
timeout, so a missed wakeup costs one poll round; patch 6/6 makes the wait
blocking, and from then on it is a permanent stall. This patch is what makes
that safe.
rte_ethdev.h does not define whether rx_queue_intr_enable() must raise when
the queue is already non-empty, and PMDs differ: mlx5 arms the CQ for the
next completion (mlx5_arm_cq(), the model whose documentation tells you to
re-poll after arming), ice sets GLINT_DYN_CTL_CLEARPBA_M while re-enabling,
and virtio just clears VRING_AVAIL_F_NO_INTERRUPT so the backend notifies
only on the next used buffer. I have only tested dpaa2, so correct me on the
first two if the hardware does re-raise.
Fixing this in drivers means defining that contract in the API first and
auditing every PMD that implements rx_queue_intr_enable.
>
>
> --
> David Marchand
>
--
Maxime Leroy
