This issue is firstly observed with an ixgbe NIC in VPP project, which is
software switching application based on DPDK.
There's a daemon thread running in background keeping polling hardware link
status, using ixgbe_dev_link_update_share().
Once flag IXGBE_FLAG_NEED_LINK_CONFIG is set, ixgbe_dev_link_update_share()
will just return link down status without actually polling hardware status.
In the issue, flag IXGBE_FLAG_NEED_LINK_CONFIG is always set, and never be
cleared, meaning ixgbe_dev_link_update_share() cannot get hardware status, but
always get link down status.
The condition causing IXGBE_FLAG_NEED_LINK_CONFIG always set is as below.
The ixgbe_dev_link_update_share() is always running in the background.
1. In the beginning, IXGBE_FLAG_NEED_LINK_CONFIG is 0 and it is link down
status.
2. ixgbe_dev_link_update_share() will set IXGBE_FLAG_NEED_LINK_CONFIG to 1
3. Then it triggers ixgbe_dev_setup_link_thread_handler() thread to configure
the interface.
4. At the end of configuring thread, ixgbe_dev_setup_link_thread_handler() will
clear the flag IXGBE_FLAG_NEED_LINK_CONFIG.
5. With IXGBE_FLAG_NEED_LINK_CONFIG being cleared,
ixgbe_dev_link_update_share() can poll hardware link status in the next round.
But when the user is setting interface link up or down in the CLI, it will call
ixgbe_dev_start() or ixgbe_dev_stop(). In both function, they will call
ixgbe_dev_cancel_link_thread() to interrupt any running configuring thread
(which is running in above step 3 and step 4), without clearing the flag
IXGBE_FLAG_NEED_LINK_CONFIG. This will leave IXGBE_FLAG_NEED_LINK_CONFIG always
set, and ixgbe_dev_link_update_share() cannot get hardware status.
Thanks.
> -----Original Message-----
> From: Phil Yang <phil.y...@arm.com>
> Sent: 2020年3月19日 14:42
> To: dev@dpdk.org; konstantin.anan...@intel.com; wenzhuo...@intel.com
> Cc: qi.z.zh...@intel.com; Lijian Zhang <lijian.zh...@arm.com>; Gavin Hu
> <gavin...@arm.com>; Honnappa Nagarahalli
> <honnappa.nagaraha...@arm.com>; nd <n...@arm.com>; sta...@dpdk.org
> Subject: [PATCH] net/ixgbe: fix link state timing issue on fiber ports
>
> With some models of fiber ports (e.g. X520-2 device ID 0x10fb), it is possible
> when a port is started to experience a timing issue which prevents the link
> from ever being fully set up.
>
> In ixgbe_dev_link_update_share(), if the media type is fiber and the link is
> down, a flag (IXGBE_FLAG_NEED_LINK_CONFIG) is set. A callback to
> ixgbe_dev_setup_link_thread_handler() is scheduled which should try to set up
> the link and clear the flag afterwards.
>
> If the device is started before the flag is cleared, the scheduled callback is
> cancelled. This causes the flag to remain set and subsequent calls to
> ixgbe_dev_link_update_share() return without trying to retrieve the link state
> because the flag is set.
>
> In ixgbe_dev_cancel_link_thread(), after cancelling the callback, unset the
> flag
> on the device to avoid this condition.
>
> Fixes: 819d0d1d57f1 ("net/ixgbe: fix blocking system events")
> Cc: sta...@dpdk.org
>
> Bugzilla ID: 388
>
> Signed-off-by: Phil Yang <phil.y...@arm.com>
> Signed-off-by: Lijian Zhang <lijian.zh...@arm.com>
> Reviewed-by: Gavin Hu <gavin...@arm.com>
> ---
> drivers/net/ixgbe/ixgbe_ethdev.c | 14 +++++++++++++-
> 1 file changed, 13 insertions(+), 1 deletion(-)
>
> diff --git a/drivers/net/ixgbe/ixgbe_ethdev.c
> b/drivers/net/ixgbe/ixgbe_ethdev.c
> index 23b3f5b..2b65750 100644
> --- a/drivers/net/ixgbe/ixgbe_ethdev.c
> +++ b/drivers/net/ixgbe/ixgbe_ethdev.c
> @@ -4147,11 +4147,19 @@ static void
> ixgbe_dev_cancel_link_thread(struct rte_eth_dev *dev) {
> struct ixgbe_adapter *ad = dev->data->dev_private;
> + struct ixgbe_interrupt *intr =
> + IXGBE_DEV_PRIVATE_TO_INTR(dev->data->dev_private);
> void *retval;
>
> if (rte_atomic32_read(&ad->link_thread_running)) {
> pthread_cancel(ad->link_thread_tid);
> pthread_join(ad->link_thread_tid, &retval);
> + /* clear this flag once the thread has been
> + * cancelled, to avoid link status error in
> + * case unfinished threads cannot clean up
> + * this flag.
> + */
> + intr->flags &= ~IXGBE_FLAG_NEED_LINK_CONFIG;
> rte_atomic32_clear(&ad->link_thread_running);
> }
> }
> @@ -4262,8 +4270,12 @@ ixgbe_dev_link_update_share(struct rte_eth_dev
> *dev,
>
> if (link_up == 0) {
> if (ixgbe_get_media_type(hw) == ixgbe_media_type_fiber) {
> - intr->flags |= IXGBE_FLAG_NEED_LINK_CONFIG;
> if (rte_atomic32_test_and_set(&ad-
> >link_thread_running)) {
> + /* To avoid race condition between threads,
> set
> + * the IXGBE_FLAG_NEED_LINK_CONFIG flag
> only
> + * when there is no link thread running.
> + */
> + intr->flags |=
> IXGBE_FLAG_NEED_LINK_CONFIG;
> if (rte_ctrl_thread_create(&ad-
> >link_thread_tid,
> "ixgbe-link-handler",
> NULL,
> --
> 2.7.4
