> Date: Tue, 30 Aug 2016 07:48:09 +0200
> From: Mike Belopuhov <[email protected]>
>
> On Tue, Aug 30, 2016 at 09:58 +1000, David Gwynne wrote:
> > On Mon, Aug 29, 2016 at 08:30:37PM +0200, Alexander Bluhm wrote:
> > > On Mon, Aug 29, 2016 at 07:10:48PM +0200, Mike Belopuhov wrote:
> > > > Due to a recent change in -current the socket sending routine
> > > > has started producing small data packets crossing memory page
> > > > boundary. This is not supported by Xen and kernels with this
> > > > change will experience broken bulk TCP transmit behaviour.
> > > > We're working on fixing it.
> > >
> > > For the same reason some old i386 machines from 2006 and 2005 have
> > > performance problems when sending data with tcpbench.
> > >
> > > em 82573E drops to 200 MBit/sec output, 82546GB and 82540EM do only
> > > 10 MBit anymore.
> > >
> > > With the patch below I get 946, 642, 422 MBit/sec output performance
> > > over these chips respectively.
> > >
> > > Don't know wether PAGE_SIZE is the correct fix as I think the problem
> > > is more related to the network chip than to the processor's page
> > > size.
> >
> > does this diff help those chips?
> >
>
> This diff defeats the purpose of the sosend change by punishing
> every other chip not suffering from the aforementioned problem.
> Lots of packets from the bulk TCP transfer will have to be
> defragmented for no good reason.
No, this em diff will still do proper scatter/gather. It might
consume more descriptors as it will use two descriptors for packets
crossing a page boundary. But the fact that we collect more data into
an mbuf will actually reduce the number of descriptors in other cases.
Regarding the xnf(4) issue; I think any driver that can't properly
deal with an mbuf crossing a page boundary is broken. I can't think
of any modern dma engine that can't handle that properly, or doesn't
at least support scatter/gather of some sort. There may be old crufty
stuff though that can't deal with it, but those probably already have
"bcopy" drivers. Now there may be drivers that don't enforce the
boundary properly. Those will mysteriously stop working. Will we be
able to fix all of those before 6.1 gets released?
> > Index: if_em.c
> > ===================================================================
> > RCS file: /cvs/src/sys/dev/pci/if_em.c,v
> > retrieving revision 1.331
> > diff -u -p -r1.331 if_em.c
> > --- if_em.c 13 Apr 2016 10:34:32 -0000 1.331
> > +++ if_em.c 29 Aug 2016 23:52:07 -0000
> > @@ -2134,7 +2134,7 @@ em_setup_transmit_structures(struct em_s
> > pkt = &sc->sc_tx_pkts_ring[i];
> > error = bus_dmamap_create(sc->sc_dmat, MAX_JUMBO_FRAME_SIZE,
> > EM_MAX_SCATTER / (sc->pcix_82544 ? 2 : 1),
> > - MAX_JUMBO_FRAME_SIZE, 0, BUS_DMA_NOWAIT, &pkt->pkt_map);
> > + MAX_JUMBO_FRAME_SIZE, 4096, BUS_DMA_NOWAIT, &pkt->pkt_map);
> > if (error != 0) {
> > printf("%s: Unable to create TX DMA map\n",
> > DEVNAME(sc));
>
>
