On Thu, Oct 08, 2015 at 01:20 +0200, Hrvoje Popovski wrote:
> Hi all,
>
> i have fairly simple setup with receiver connected to em2 and sender
> connected to em3. Both em are Intel I350. Setup is without pf with these
> sysctls:
>
> kern.pool_debug=1
> net.inet.ip.forwarding=1
> net.inet.ip.ifq.maxlen=8192
> ddb.console=1
>
> with if_em.c revisions 1.307 and 1.306 i can trigger
> em2: unable to fill any rx descriptors
> when doing ifconfig em2 down/up (receiver side) while generating
> traffic. i can't trigger this with ifconfig em3 down/up (sender side) or
> destroying bridge and enabling it. this is reproducible.
>
> with bridged setup when doing ifconfig em2 down/up i'm getting rx
> descriptors log and bridge stops bridging traffic until doing this:
> stop generating traffic
> ifconfig em2 down
> ifconfig em3 down
> ifconfig bridge0 destroy
> ifconfig em2 up
> ifconfig em3 up
> sh netstart bridge0
> start generating traffic
>
>
> with routed setup when doing ifconfig em2 down/up traffic is not
> forwarded until
> stop generating traffic
> ifconfig em2 down
> ifconfig em2 up
> start generating traffic
>
>
> with if_em.c revisions 1.305 and if_em.h revision 1.57 i can't trigger
> rx descriptors log and bridge starts to bridge traffic almost instantly
> when doing ifconfig em2 up
>
>
> i am willing to debug this further but i don't know how...
>
Is it possible that em_reset_hw is called a bit too early
while we don't have full control of the driver?
Also it appears to me that while we're mi_switch'ing in the
sched_barrier/sleep_finish another process can start running
em_init since mi_switch will unlock all KERNEL_LOCKs that the
process doing em_stop might hold. Now of course KASSERT would
have fired if IFF_RUNNING would have been set, but perhaps we
just haven't hit this yet. Shouldn't we do rwlock protection
to make sure that while em_stop is not finished no other
process can do em_stop and/or em_init?
diff --git sys/dev/pci/if_em.c sys/dev/pci/if_em.c
index f2ddb32..7b24dce 100644
--- sys/dev/pci/if_em.c
+++ sys/dev/pci/if_em.c
@@ -1559,13 +1559,14 @@ em_stop(void *arg, int softonly)
timeout_del(&sc->timer_handle);
timeout_del(&sc->tx_fifo_timer_handle);
- if (!softonly) {
+ if (!softonly)
em_disable_intr(sc);
- em_reset_hw(&sc->hw);
- }
intr_barrier(sc->sc_intrhand);
+ if (!softonly)
+ em_reset_hw(&sc->hw);
+
KASSERT((ifp->if_flags & IFF_RUNNING) == 0);
em_free_transmit_structures(sc);
