Re: RDMA will be reverted

2006-07-25 Thread Tom Tucker
On Mon, 2006-07-24 at 15:23 -0700, David Miller wrote: > From: Tom Tucker <[EMAIL PROTECTED]> > Date: Wed, 05 Jul 2006 12:09:42 -0500 > > > "A TOE net stack is closed source firmware. Linux engineers have no way > > to fix security issues that arise. As a result, only non-TOE users will > > recei

Re: RDMA will be reverted

2006-07-25 Thread Andi Kleen
> It may be a hardware interpretation, but doesn't it have non-trivial system > implications - where one runs threads/processes etc? Only if you do process context RX processing. If you chose not to it doesn't have much influence. -Andi - To unsubscribe from this list: send the line "unsubscri

Re: RDMA will be reverted

2006-07-25 Thread Rick Jones
David Miller wrote: From: Rick Jones <[EMAIL PROTECTED]> Date: Mon, 24 Jul 2006 17:55:24 -0700 Even enough bits for 1024 or 2048 CPUs in the single system image? I have seen 1024 touted by SGI, and with things going so multi-core, perhaps 16384 while sounding initially bizzare would be in th

Re: RDMA will be reverted

2006-07-25 Thread Evgeniy Polyakov
On Tue, Jul 25, 2006 at 12:33:44AM -0700, David Miller ([EMAIL PROTECTED]) wrote: > From: Evgeniy Polyakov <[EMAIL PROTECTED]> > Date: Tue, 25 Jul 2006 10:59:21 +0400 > > > As a side completely unrelated to either my or others work note :) - > > I think it is a nanooptimisation - we get a bit of

Re: RDMA will be reverted

2006-07-25 Thread David Miller
From: Evgeniy Polyakov <[EMAIL PROTECTED]> Date: Tue, 25 Jul 2006 10:59:21 +0400 > As a side completely unrelated to either my or others work note :) - > I think it is a nanooptimisation - we get a bit of performance here, > and lose those bit in other place. > When bag is filled, there is no mu

Re: RDMA will be reverted

2006-07-25 Thread Evgeniy Polyakov
On Mon, Jul 24, 2006 at 11:48:53PM -0700, David Miller ([EMAIL PROTECTED]) wrote: > > And if that CPU is very busy? > > Linux should somehow tell NIC that some CPUs are valid and some are not > > right now, but not in a second, so scheduler must be tightly bound with > > network internals. > > Ye

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Evgeniy Polyakov <[EMAIL PROTECTED]> Date: Tue, 25 Jul 2006 09:51:28 +0400 > On Mon, Jul 24, 2006 at 03:06:13PM -0700, David Miller ([EMAIL PROTECTED]) > wrote: > > Furthermore, the VJ netchannel gains can be partially obtained from > > generic stateless facilities that we are going to get

Re: RDMA will be reverted

2006-07-24 Thread Evgeniy Polyakov
On Mon, Jul 24, 2006 at 03:06:13PM -0700, David Miller ([EMAIL PROTECTED]) wrote: > Don't get too excited about VJ netchannels, more and more roadblocks > to their practicality are being found every day. > > For example, my idea to allow ESTABLISHED TCP socket demux to be done > before netfilter

Re: RDMA will be reverted

2006-07-24 Thread Andi Kleen
On Tuesday 25 July 2006 02:29, Rick Jones wrote: > This all sounds like the discussions we had within HP-UX between 10.20 and > 11.0 > concerning Inbound Packet Scheduling vs Thread Optimized Packet Scheduling. We've also talking about this for many years, just no code so far. Or rather Linux

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Rick Jones <[EMAIL PROTECTED]> Date: Mon, 24 Jul 2006 17:55:24 -0700 > Even enough bits for 1024 or 2048 CPUs in the single system image? I have > seen > 1024 touted by SGI, and with things going so multi-core, perhaps 16384 while > sounding initially bizzare would be in the realm of the

Re: RDMA will be reverted

2006-07-24 Thread Andi Kleen
> Even enough bits for 1024 or 2048 CPUs in the single system image? MSI-X supports 255 sub interrupts max, most hardware probably much less (e.g. 8 seems to be a popular number). It can be always hashed to the existing CPUs. It's a nice idea but I think standard hashing + processing in softi

Re: RDMA will be reverted

2006-07-24 Thread Rick Jones
It would have to be done in such a way as to not make the PAWS tests fail by accident. But I think it's doable. CPU ID and higher-order generation number such that whenever the process migrates to a lower-numbered CPU, the generation number is bumped to make the timestamp larger than before?

Re: RDMA will be reverted

2006-07-24 Thread Rick Jones
David Miller wrote: From: Rick Jones <[EMAIL PROTECTED]> Date: Mon, 24 Jul 2006 17:29:05 -0700 Nirvana I suppose would be the addition of a field in the header which could be used for the determination of where to process. A Transport Protocol option I suppose, maybe the IPv6 flow id, but knut

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Rick Jones <[EMAIL PROTECTED]> Date: Mon, 24 Jul 2006 17:29:05 -0700 > Nirvana I suppose would be the addition of a field in the header > which could be used for the determination of where to process. A > Transport Protocol option I suppose, maybe the IPv6 flow id, but > knuth only knows if

Re: RDMA will be reverted

2006-07-24 Thread Rick Jones
This all sounds like the discussions we had within HP-UX between 10.20 and 11.0 concerning Inbound Packet Scheduling vs Thread Optimized Packet Scheduling. IPS was done by the 10.20 stack at the handoff between the driver and netisr. If the packet was not an IP datagram fragment, parts of the

Re: RDMA will be reverted

2006-07-24 Thread Andi Kleen
On Tuesday 25 July 2006 01:22, David Miller wrote: > From: Andi Kleen <[EMAIL PROTECTED]> > Date: Tue, 25 Jul 2006 01:10:25 +0200 > > > > All the original costs of route, netfilter, TCP socket lookup all > > > reappear as we make VJ netchannels fit all the rules of real practical > > > systems, el

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Andi Kleen <[EMAIL PROTECTED]> Date: Tue, 25 Jul 2006 01:10:25 +0200 > > All the original costs of route, netfilter, TCP socket lookup all > > reappear as we make VJ netchannels fit all the rules of real practical > > systems, eliminating their gains entirely. > > At least most of the optim

Re: RDMA will be reverted

2006-07-24 Thread Andi Kleen
> For example, my idea to allow ESTABLISHED TCP socket demux to be done > before netfilter is flawed. Connection tracking and NAT can change > the packet ID and loop it back to us to hit exactly an ESTABLISHED TCP > socket, therefore we must always hit netfilter first. Hmm, how does this happen?

RE: RDMA will be reverted

2006-07-24 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > From: Tom Tucker <[EMAIL PROTECTED]> > Date: Wed, 05 Jul 2006 12:09:42 -0500 > >> "A TOE net stack is closed source firmware. Linux engineers have no >> way to fix security issues that arise. As a result, only non-TOE >> users will receive security updates, leaving rando

RE: RDMA will be reverted

2006-07-24 Thread Caitlin Bestler
[EMAIL PROTECTED] wrote: > From: Steve Wise <[EMAIL PROTECTED]> > Date: Wed, 05 Jul 2006 12:50:34 -0500 > >> However, iWARP devices _could_ integrate with netfilter. For most >> devices the connection request event (SYN) gets passed up to the host >> driver. So the driver can enforce filter rule

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Steve Wise <[EMAIL PROTECTED]> Date: Wed, 05 Jul 2006 12:50:34 -0500 > However, iWARP devices _could_ integrate with netfilter. For most > devices the connection request event (SYN) gets passed up to the host > driver. So the driver can enforce filter rules then. This doesn't work. In or

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Tom Tucker <[EMAIL PROTECTED]> Date: Wed, 05 Jul 2006 12:09:42 -0500 > "A TOE net stack is closed source firmware. Linux engineers have no way > to fix security issues that arise. As a result, only non-TOE users will > receive security updates, leaving random windows of vulnerability for > e

Re: RDMA will be reverted

2006-07-24 Thread David Miller
From: Roland Dreier <[EMAIL PROTECTED]> Date: Tue, 04 Jul 2006 13:34:27 -0700 > Well, here's a quick overview, leaving out some of the details. The > difference between TOE and iWARP/RDMA is really the interface that > they present. Thanks for the description Roland. It helps me understand the

Re: RDMA will be reverted

2006-07-07 Thread Tom Tucker
On Thu, 2006-07-06 at 23:53 -0700, David Miller wrote: > From: Tom Tucker <[EMAIL PROTECTED]> > Date: Thu, 06 Jul 2006 00:25:03 -0500 > > > This patch is about dotting I's and crossing T's, it's not about > > foundations. > > You assume that I've flat out rejected RDMA, in fact I haven't. I > re

Re: RDMA will be reverted

2006-07-06 Thread David Miller
From: Tom Tucker <[EMAIL PROTECTED]> Date: Thu, 06 Jul 2006 00:25:03 -0500 > This patch is about dotting I's and crossing T's, it's not about > foundations. You assume that I've flat out rejected RDMA, in fact I haven't. I really don't have enough information to form a final opinion yet. There's

Re: RDMA will be reverted

2006-07-06 Thread Tom Tucker
On Fri, 2006-07-07 at 10:03 +1000, Herbert Xu wrote: > On Thu, Jul 06, 2006 at 12:36:24PM -0500, Tom Tucker wrote: > > > > The RDMA CMA uses IP addresses and port numbers to create a uniform > > addressing scheme across all transport types. For IB, it is necessary to > > resolve IP addresses to IB

Re: RDMA will be reverted

2006-07-06 Thread Herbert Xu
On Thu, Jul 06, 2006 at 12:36:24PM -0500, Tom Tucker wrote: > > The RDMA CMA uses IP addresses and port numbers to create a uniform > addressing scheme across all transport types. For IB, it is necessary to > resolve IP addresses to IB GIDs. The ARP protocol is used to do this and > a netfilter ru

Re: RDMA will be reverted

2006-07-06 Thread Tom Tucker
On Fri, 2006-07-07 at 00:08 +1000, Herbert Xu wrote: > Tom Tucker <[EMAIL PROTECTED]> wrote: > > > > All that said, the proposed patch helps not only iWARP, but other > > transports (iSCSI, IB) as well. It is not large, invasive, > > Care to explain on how it helps those other technologies? The

Re: RDMA will be reverted

2006-07-06 Thread Herbert Xu
Tom Tucker <[EMAIL PROTECTED]> wrote: > > All that said, the proposed patch helps not only iWARP, but other > transports (iSCSI, IB) as well. It is not large, invasive, Care to explain on how it helps those other technologies? Cheers, -- Visit Openswan at http://www.openswan.org/ Email: Herbert

RE: RDMA will be reverted

2006-07-06 Thread Caitlin Bestler
Andi Kleen wrote: > >> We're focusing on netfilter here. Is breaking netfilter really the >> only issue with this stuff? > > Another concern is that it will just not be able to keep > up with a high rate of new connections or a high number of them > (because the hardware has too limited state) >

Re: RDMA will be reverted

2006-07-05 Thread Tom Tucker
On Wed, 2006-07-05 at 20:03 -0700, David Miller wrote: > From: Roland Dreier <[EMAIL PROTECTED]> > Date: Wed, 05 Jul 2006 13:29:35 -0700 > > > The way forward seems to be to merge basic iWARP support that lives in > > drivers/infiniband, and then you can accept or reject things for > > better inte

Re: RDMA will be reverted

2006-07-05 Thread David Miller
From: Roland Dreier <[EMAIL PROTECTED]> Date: Wed, 05 Jul 2006 13:29:35 -0700 > The way forward seems to be to merge basic iWARP support that lives in > drivers/infiniband, and then you can accept or reject things for > better integration, like notifiers for routing changes. Let's merge in drive

Re: RDMA will be reverted

2006-07-05 Thread Roland Dreier
> Then why in the world would we put up explicit web pages that > say "TOE is bad, here's a list of reasons why" if we had any > intention of ever adding support for these kinds of devices? I think there's a little bit of leap of logic there. Everyone agrees that winmodems are bad and yet ther

Re: RDMA will be reverted

2006-07-05 Thread David Miller
From: Roland Dreier <[EMAIL PROTECTED]> Date: Tue, 04 Jul 2006 13:34:30 -0700 > Well, in the past it's seemed more like patches have been rejected not > because of a blanket refusal to consider support for certain hardware, Then why in the world would we put up explicit web pages that say "TOE is

Re: RDMA will be reverted

2006-07-05 Thread Steve Wise
On Wed, 2006-07-05 at 12:09 -0500, Tom Tucker wrote: > On Sat, 2006-07-01 at 16:26 +0200, Andi Kleen wrote: > > On Saturday 01 July 2006 01:01, Tom Tucker wrote: > > > On Fri, 2006-06-30 at 14:16 -0700, David Miller wrote: > > > > > > > The TOE folks have tried to submit their hooks and drivers >

Re: RDMA will be reverted

2006-07-05 Thread Tom Tucker
On Sat, 2006-07-01 at 16:26 +0200, Andi Kleen wrote: > On Saturday 01 July 2006 01:01, Tom Tucker wrote: > > On Fri, 2006-06-30 at 14:16 -0700, David Miller wrote: > > > > > The TOE folks have tried to submit their hooks and drivers > > > on several occaisions, and we've rejected it every time. >

Re: RDMA will be reverted

2006-07-04 Thread Andi Kleen
> > Think of the ordinary single linux box somewhere at a rackspace provider > > which > > represents the majority of Linux boxes around. > How many of those need 10G nics? Most of them already have gigabit. At some point they will have 10G too. Admittedly the iThingy under discussion here se

Re: RDMA will be reverted

2006-07-04 Thread Andy Gay
On Wed, 2006-07-05 at 01:01 +0200, Andi Kleen wrote: > > My point wasn't really about performance here, more that systems needing > > this level of performance (server farm is just an example) will probably > > be on an 'inside' network with firewalling being done elsewhere (at the > > access layer

Re: RDMA will be reverted

2006-07-04 Thread Andi Kleen
> My point wasn't really about performance here, more that systems needing > this level of performance (server farm is just an example) will probably > be on an 'inside' network with firewalling being done elsewhere (at the > access layer, to use the Cisco paradigm). It's just not good design to >

Re: RDMA will be reverted

2006-07-04 Thread Andy Gay
On Tue, 2006-07-04 at 22:47 +0200, Andi Kleen wrote: > > So perhaps there's a good argument to make that a Linux system with the > > right hardware could be considered a core device. Likely any place you > > have such a system it would be dedicated to just moving data as well as > > possible, and l

Re: RDMA will be reverted

2006-07-04 Thread Andi Kleen
> So perhaps there's a good argument to make that a Linux system with the > right hardware could be considered a core device. Likely any place you > have such a system it would be dedicated to just moving data as well as > possible, and let other systems do the other stuff. You wouldn't want > you

Re: RDMA will be reverted

2006-07-04 Thread Roland Dreier
Andi> Perhaps a good start of that discussion David asked for Andi> would be if you could give us an overview of the differences Andi> and how you avoid the TOE problems. Well, here's a quick overview, leaving out some of the details. The difference between TOE and iWARP/RDMA is reall

Re: RDMA will be reverted

2006-07-04 Thread Roland Dreier
> Roland stated that it has never been the case that we have > rejected adding support for a certain class of devices on the > kinds of merits being discussed in this thread. And I'm saying > that TOE is such a case where we have emphatically done so. Well, in the past it's seemed more like p

Re: RDMA will be reverted

2006-07-04 Thread Andy Gay
On Sat, 2006-07-01 at 16:26 +0200, Andi Kleen wrote: > On Saturday 01 July 2006 01:01, Tom Tucker wrote: > > On Fri, 2006-06-30 at 14:16 -0700, David Miller wrote: > > > > > The TOE folks have tried to submit their hooks and drivers > > > on several occaisions, and we've rejected it every time. >

Re: RDMA will be reverted

2006-07-01 Thread David Miller
From: Tom Tucker <[EMAIL PROTECTED]> Date: Fri, 30 Jun 2006 18:01:43 -0500 > On Fri, 2006-06-30 at 14:16 -0700, David Miller wrote: > > > The TOE folks have tried to submit their hooks and drivers > > on several occaisions, and we've rejected it every time. > > iWARP != TOE You are taking my co

Re: RDMA will be reverted

2006-07-01 Thread Andi Kleen
On Saturday 01 July 2006 01:01, Tom Tucker wrote: > On Fri, 2006-06-30 at 14:16 -0700, David Miller wrote: > > > The TOE folks have tried to submit their hooks and drivers > > on several occaisions, and we've rejected it every time. > > iWARP != TOE Perhaps a good start of that discussion David

Re: RDMA will be reverted

2006-06-30 Thread Tom Tucker
On Fri, 2006-06-30 at 14:16 -0700, David Miller wrote: > The TOE folks have tried to submit their hooks and drivers > on several occaisions, and we've rejected it every time. iWARP != TOE > - > To unsubscribe from this list: send the line "unsubscribe netdev" in > the body of a message to [EMAIL

Re: RDMA will be reverted

2006-06-30 Thread David Miller
From: Roland Dreier <[EMAIL PROTECTED]> Date: Fri, 30 Jun 2006 13:51:19 -0700 > And I can't think of a single other example of a case where we > refused to merge a driver, not because of any issues with the driver > code, but because we don't like the hardware it drives and think > that people sho

Re: RDMA will be reverted

2006-06-30 Thread Roland Dreier
You snipped my question about what specifically you wanted reverted, so I'm going to assume that after cooling down and understanding the situation, you're OK with everything that's in Linus's tree... > The integration of iWARP with the Linux networking, while much better > than TOE, is still he

Re: RDMA will be reverted

2006-06-29 Thread Tom Tucker
On Thu, 2006-06-29 at 13:53 -0700, David Miller wrote: > From: Tom Tucker <[EMAIL PROTECTED]> > Date: Thu, 29 Jun 2006 15:47:13 -0500 > > > I concede that features have been lost, but some applications benefit > > greatly from RDMA. For these applications and these customers, > > TOE folks give t

Re: RDMA will be reverted

2006-06-29 Thread Andi Kleen
> They do not bypass netfilter, they do not bypass > packet scheduling, and yet they provide a hardware assist performance > improvement for receive. Not that I'm a TOE advocate, but as long as the adapter leaves SYN/SYN-ACK to the stack and only turns on RDMA in ESTABLISHED it could at least do n

Re: RDMA will be reverted

2006-06-29 Thread Tom Tucker
On Thu, 2006-06-29 at 13:19 -0700, David Miller wrote: > From: Tom Tucker <[EMAIL PROTECTED]> > Date: Thu, 29 Jun 2006 15:11:06 -0500 > > > Doesn't this position force vendors to build deeper adapters, not > > shallower adapters? Isn't this exactly the opposite of what is intended? > > Nope. > >

Re: RDMA will be reverted

2006-06-29 Thread David Miller
From: Tom Tucker <[EMAIL PROTECTED]> Date: Thu, 29 Jun 2006 15:47:13 -0500 > I concede that features have been lost, but some applications benefit > greatly from RDMA. For these applications and these customers, TOE folks give the same story... it's a broken record, really. Let us know when you

Re: RDMA will be reverted

2006-06-29 Thread James Morris
On Thu, 29 Jun 2006, Tom Tucker wrote: > On Thu, 2006-06-29 at 12:46 -0700, David Miller wrote: > > From: Roland Dreier <[EMAIL PROTECTED]> > > Date: Thu, 29 Jun 2006 09:54:37 -0700 > > > > > In any case I think we need to find a way for Linux to support iWARP > > > hardware, since there are user

Re: RDMA will be reverted

2006-06-29 Thread Tom Tucker
[...snip...] > community at large, Linux and Andrew, you? I'm not trying to be Linus sorry... spell checker... - To unsubscribe from this list: send the line "unsubscribe netdev" in the body of a message to [EMAIL PROTECTED] More majordomo info at http://vger.kernel.org/majordo

Re: RDMA will be reverted

2006-06-29 Thread David Miller
From: Tom Tucker <[EMAIL PROTECTED]> Date: Thu, 29 Jun 2006 15:11:06 -0500 > Doesn't this position force vendors to build deeper adapters, not > shallower adapters? Isn't this exactly the opposite of what is intended? Nope. Look at what the networking developers give a lot of attention and effor

Re: RDMA will be reverted

2006-06-29 Thread Tom Tucker
On Thu, 2006-06-29 at 12:46 -0700, David Miller wrote: > From: Roland Dreier <[EMAIL PROTECTED]> > Date: Thu, 29 Jun 2006 09:54:37 -0700 > > > In any case I think we need to find a way for Linux to support iWARP > > hardware, since there are users that want this, and (some of) the > > vendors are

Re: RDMA will be reverted

2006-06-29 Thread David Miller
From: Roland Dreier <[EMAIL PROTECTED]> Date: Thu, 29 Jun 2006 09:54:37 -0700 > In any case I think we need to find a way for Linux to support iWARP > hardware, since there are users that want this, and (some of) the > vendors are working hard to do things the right way (including cc'ing > netdev

Re: RDMA will be reverted

2006-06-29 Thread YOSHIFUJI Hideaki / 吉藤英明
In article <[EMAIL PROTECTED]> (at Thu, 29 Jun 2006 10:35:56 -0700), Roland Dreier <[EMAIL PROTECTED]> says: > > > Dave, you're going to have to be more specific. What do you mean by > > > RDMA? The whole drivers/infiniband infrastructure, which handles RDMA > > > over IB, has been upstream

Re: RDMA will be reverted

2006-06-29 Thread Roland Dreier
> > Dave, you're going to have to be more specific. What do you mean by > > RDMA? The whole drivers/infiniband infrastructure, which handles RDMA > > over IB, has been upstream for a year and a half, and was in fact > > originally merged by you, so I'm guessing that's not what you mean. >

Re: RDMA will be reverted

2006-06-29 Thread YOSHIFUJI Hideaki / 吉藤英明
Hello. In article <[EMAIL PROTECTED]> (at Thu, 29 Jun 2006 09:54:37 -0700), Roland Dreier <[EMAIL PROTECTED]> says: > David> Roland, there is no way in the world we would have let > David> support for RDMA into the kernel tree had we seen and > David> reviewed it on netdev. I've dis

Re: RDMA will be reverted

2006-06-29 Thread Roland Dreier
David> Roland, there is no way in the world we would have let David> support for RDMA into the kernel tree had we seen and David> reviewed it on netdev. I've discussed this with Andrew David> Morton, and we'd like you to please revert all of the RDMA David> code from Linus's tr

Re: RDMA will be reverted

2006-06-28 Thread Steve Wise
On Wed, 2006-06-28 at 00:07 -0700, David Miller wrote: > Roland, there is no way in the world we would have let support for > RDMA into the kernel tree had we seen and reviewed it on netdev. I've > discussed this with Andrew Morton, and we'd like you to please revert > all of the RDMA code from Li

Re: RDMA will be reverted

2006-06-28 Thread Tom Tucker
On Wed, 2006-06-28 at 00:07 -0700, David Miller wrote: > Roland, there is no way in the world we would have let support for > RDMA into the kernel tree had we seen and reviewed it on netdev. I've > discussed this with Andrew Morton, and we'd like you to please revert > all of the RDMA code from Li

Re: RDMA will be reverted

2006-06-28 Thread Evgeniy Polyakov
On Wed, Jun 28, 2006 at 12:07:15AM -0700, David Miller ([EMAIL PROTECTED]) wrote: > Roland, there is no way in the world we would have let support for > RDMA into the kernel tree had we seen and reviewed it on netdev. I've > discussed this with Andrew Morton, and we'd like you to please revert >