Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Quoting Parks Fields <[EMAIL PROTECTED]>: > Subject: Re: [openib-general] [PATCHv4] IPoIB CM Experimental support > > > Hi, > > Is there a doc that can explain more about how this work. Also if I > download ( git ) the current trunk can I build it and test ?? > > thanks Can you take a look at the original mail you reply to please? It included detailed answers to you questions. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
Hi, Is there a doc that can explain more about how this work. Also if I download ( git ) the current trunk can I build it and test ?? thanks * Correspondence * This email contains no programmatic content that requires independent ADC review ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
OK, I pushed out a new tree with both IPoIB NAPI and CM in the for-mm branch. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCHv4] IPoIB CM Experimental support
- Message from "Roland Dreier" <[EMAIL PROTECTED]> on Wed, 10 > Jan 2007 07:15:12 -0800 - > > To: > > "Michael S. Tsirkin" <[EMAIL PROTECTED]> > > cc: > > openib-general@openib.org > > Subject: > > Re: [openib-general] [PATCHv4] IPoIB CM Experimental support > > > - Using path MTU discovery, multicast and UDP traffic to UD mode now work, > > only a small number of packets is dropped. > > How does this work? What happens if I set my MTU to 8K and send a > 8000-byte UDP message to a node that doesn't implement CM? What > happens if I send an 8000-byte UDP multicast? If we support a large interface MTU using RC, then I would expect both UDP and TCP to both use RC mode. It doesn't make sense for UDP to use UD mode because of the higher CPU overhead and lower bandwdith, when TCP uses RC. Can we not support UDP multicast over RC? > > I didn't think UDP did path MTU discovery by default... > > - R. > > > Bernie King-Smith IBM Corporation Server Group Cluster System Performance [EMAIL PROTECTED](845)433-8483 Tie. 293-8483 or wombat2 on NOTES "We are not responsible for the world we are born into, only for the world we leave when we die. So we have to accept what has gone before us and work to change the only thing we can, -- The Future." William Shatner ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Maybe for-mm for now? Definitely, I just need to update all my branches. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Quoting Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCHv4] IPoIB CM Experimental support > > > Roland, can we queue this for 2.6.21? > > Yes, once I have a chance to really read it over. Maybe for-mm for now? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Quoting Roland Dreier <[EMAIL PROTECTED]>:
> Subject: Re: [PATCHv4] IPoIB CM Experimental support
>
> Also, I haven't really looked yet, but how does the connected mode
> patch interact with the NAPI patches?
The latest version uses prov->cq for all RX packets, so
it's trivial to merge it with NAPI if we want: the relevant part
of the patch is:
static void ipoib_ib_handle_wc(struct net_device *dev, struct ib_wc *wc)
{
- if (wc->wr_id & IPOIB_OP_RECV)
+ if (wc->wr_id & IPOIB_CM_OP_SRQ)
+ ipoib_cm_handle_rx_wc(dev, wc);
+ else if (wc->wr_id & IPOIB_OP_RECV)
ipoib_ib_handle_rx_wc(dev, wc);
else
ipoib_ib_handle_tx_wc(dev, wc);
I haven't tested this combination yet.
--
MST
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Quoting Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [PATCHv4] IPoIB CM Experimental support > > > - Using path MTU discovery, multicast and UDP traffic to UD mode now work, > > only a small number of packets is dropped. > > How does this work? What happens if I set my MTU to 8K and send a > 8000-byte UDP message to a node that doesn't implement CM? What > happens if I send an 8000-byte UDP multicast? > > I didn't think UDP did path MTU discovery by default... Before packet drop, we reduce the dst mtu, so linux will start splitting following packets to this destination. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> - Using path MTU discovery, multicast and UDP traffic to UD mode now work, > only a small number of packets is dropped. How does this work? What happens if I set my MTU to 8K and send a 8000-byte UDP message to a node that doesn't implement CM? What happens if I send an 8000-byte UDP multicast? I didn't think UDP did path MTU discovery by default... - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
Also, I haven't really looked yet, but how does the connected mode patch interact with the NAPI patches? - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Roland, can we queue this for 2.6.21? Yes, once I have a chance to really read it over. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> Subject: Re: [PATCHv4] IPoIB CM Experimental support Roland, can we queue this for 2.6.21? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
> > e. Some notes on code > > 1. SRQ is used for scalability to large cluster sizes > > I still want to support non-SRQ adapters with this code. IPoIB with and without patch performs the same on non-SRQ CAs. So let's see this code land upstream first, then we can look at uglifying it to wring a bit more performance from ehca, too, on some topologies. -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCHv4] IPoIB CM Experimental support
- Message from "Michael S. Tsirkin" <[EMAIL PROTECTED]> on Mon, > 8 Jan 2007 18:57:14 +0200 - > > To: > > openib-general@openib.org, "Roland Dreier" <[EMAIL PROTECTED]> > > Subject: > > [openib-general] [PATCHv4] IPoIB CM Experimental support > > The following patch adds experimental support for IPoIB connected mode. > The idea is to increase performance by increasing the MTU > from the maximum of 2K (theoretically 4K) supported by IPoIB on top of UD. > With this code, I'm able to get 800MByte/sec or more with netperf > without options on a Mellanox 4x back-to-back DDR system. > > Signed-off-by: Michael S. Tsirkin <[EMAIL PROTECTED]> > > --- > > Sorry about the churn, just fixed a bug in this code. [SNIP] > e. Some notes on code > 1. SRQ is used for scalability to large cluster sizes I still want to support non-SRQ adapters with this code. Not all systems have 100's or 1000's of endpoints and those smaller systems will benefit from IPoIB-CM. The larger systems tend to have larger memory per node so can support the additional memory requirements. At the November meeting one of the main themes from application developers and customers is we must have a well performing TCP/IP story across as much of the IB space as possible. If only one or two of the IB adapters perform well, then we haven't addressed the customer needs. Those adapters that can't support RC is one issue, but for those who do without SRQ, smaller configurations should be able to use IPoIB-CM. > 2. Only RC connections are used (UC does not support SRQ now) > 3. Retry count is set to 0 since spec draft warns against retries > 4. Each connection is used for data transfers in only 1 direction, >so each connection is either active(TX) or passive (RX). >2 sides that want to communicate create 2 connections. > 5. Each active (TX) connection has a separate CQ for send completions - >this keeps the code simple without CQ resize and other tricks > Bernie King-Smith IBM Corporation Server Group Cluster System Performance [EMAIL PROTECTED](845)433-8483 Tie. 293-8483 or wombat2 on NOTES "We are not responsible for the world we are born into, only for the world we leave when we die. So we have to accept what has gone before us and work to change the only thing we can, -- The Future." William Shatner___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
