Re: [openib-general] [PATCH] FMR support in mthca

2005-04-01 Thread Hal Rosenstock
On Fri, 2005-04-01 at 18:07, Libor Michalek wrote: > You need to increase the amount of memory that the user is allowed > to lock. The following command in each shell from which you are running > ttcp: > > limit memorylocked unlimited > > In 2.6 mlock() looks at the rlimits for the process

Re: [openib-general] [PATCH] FMR support in mthca

2005-04-01 Thread Libor Michalek
On Fri, Apr 01, 2005 at 05:56:00PM -0500, Hal Rosenstock wrote: > On Thu, 2005-03-31 at 16:41, Libor Michalek wrote: > > > > ./ttcp.aio.x -r -l 98304 -a 20 -f M > > ./ttcp.aio.x -t -l 98304 -n 20 -a 20 -f M 192.168.0.100 > > > > > > For the FMR pool cache miss async results I used

Re: [openib-general] [PATCH] FMR support in mthca

2005-04-01 Thread Hal Rosenstock
On Thu, 2005-03-31 at 16:41, Libor Michalek wrote: > On Thu, Mar 31, 2005 at 04:25:28PM -0500, Hal Rosenstock wrote: > > On Wed, 2005-03-30 at 19:43, Libor Michalek wrote: > > > The program has a decent help for available parameters, but here are > > > some reasonable defaults: > > > > > > serv

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-31 Thread Libor Michalek
On Thu, Mar 31, 2005 at 04:25:28PM -0500, Hal Rosenstock wrote: > On Wed, 2005-03-30 at 19:43, Libor Michalek wrote: > > The program has a decent help for available parameters, but here are > > some reasonable defaults: > > > > server: > > > > ./ttcp.aio.x -r -l 65536 -a 20 > > > > clien

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-31 Thread Hal Rosenstock
On Wed, 2005-03-30 at 19:43, Libor Michalek wrote: > The program has a decent help for available parameters, but here are > some reasonable defaults: > > server: > > ./ttcp.aio.x -r -l 65536 -a 20 > > client: > > ./ttcp.aio.x -t -l 65536 -n 10 -a 20 192.168.0.100 Are these the

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-30 Thread Libor Michalek
On Wed, Mar 30, 2005 at 05:40:33PM -0800, Peter Buckingham wrote: > Libor Michalek wrote: > > The recompile modifications to use SDP protocol familt for regular > > ttcp are pretty straight forward: > > > > #include > > #undef AF_INET > > #define AF_INET AF_INET_SDP > > if i want to do

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-30 Thread Peter Buckingham
Libor Michalek wrote: The recompile modifications to use SDP protocol familt for regular ttcp are pretty straight forward: #include #undef AF_INET #define AF_INET AF_INET_SDP if i want to do in-kernel SDP can i just change my in-kernel sockets address family too, or is it more complicat

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-30 Thread Libor Michalek
On Tue, Mar 29, 2005 at 07:29:04PM -0800, Grant Grundler wrote: > On Tue, Mar 29, 2005 at 06:12:28PM -0800, Libor Michalek wrote: > > > I could checkin the source for the modified ttcp's but I'm not sure > > exactly where... (gen2/users/libor ???) > > I wouldn't bother for the "#define AF_INET

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-30 Thread Libor Michalek
On Wed, Mar 30, 2005 at 10:09:24AM -0500, Hal Rosenstock wrote: > On Mon, 2005-03-28 at 20:03, Libor Michalek wrote: > > I haven't looked closely at the code yet, but I did try it out > > with SDP/AIO on a pair of x86 systems with Tavors and a pair of > > x86_64 systems with Arbels. With a small

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-30 Thread Michael S. Tsirkin
Quoting r. Hal Rosenstock <[EMAIL PROTECTED]>: > Subject: Re: [openib-general] [PATCH] FMR support in mthca > > On Mon, 2005-03-28 at 20:03, Libor Michalek wrote: > > I haven't looked closely at the code yet, but I did try it out > > with SDP/AIO on a pair of x8

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-30 Thread Hal Rosenstock
On Mon, 2005-03-28 at 20:03, Libor Michalek wrote: > I haven't looked closely at the code yet, but I did try it out > with SDP/AIO on a pair of x86 systems with Tavors and a pair of > x86_64 systems with Arbels. With a small change to core/fmr_pool.c > and enabling pool creation in SDP it worke

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-29 Thread Grant Grundler
On Wed, Mar 30, 2005 at 07:44:08AM +0200, Michael S. Tsirkin wrote: > > As Libor reported, SDP is working and getting the expected good > > AIO performance with this code. > What benchmark do you use to measure AIO performance? I just asked libor as well. It was a hacked version of ttcp:

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-29 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [openib-general] [PATCH] FMR support in mthca > > Thanks, I applied this in a few pieces with some minor cleanups. > > As Libor reported, SDP is working and getting the expected good > AIO performance wit

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-29 Thread Roland Dreier
Thanks, I applied this in a few pieces with some minor cleanups. As Libor reported, SDP is working and getting the expected good AIO performance with this code. Thanks, Roland ___ openib-general mailing list openib-general@openib.org http://openib.org

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-29 Thread Grant Grundler
On Tue, Mar 29, 2005 at 06:12:28PM -0800, Libor Michalek wrote: > The recompile modifications to use SDP protocol familt for regular > ttcp are pretty straight forward: > > #include > #undef AF_INET > #define AF_INET AF_INET_SDP Yes, that's pretty easy. > I could checkin the source f

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-29 Thread Libor Michalek
On Tue, Mar 29, 2005 at 05:08:14PM -0800, Grant Grundler wrote: > On Mon, Mar 28, 2005 at 05:03:51PM -0800, Libor Michalek wrote: > > I haven't looked closely at the code yet, but I did try it out > > with SDP/AIO on a pair of x86 systems with Tavors and a pair of > > x86_64 systems with Arbels.

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-29 Thread Grant Grundler
On Mon, Mar 28, 2005 at 05:03:51PM -0800, Libor Michalek wrote: > I haven't looked closely at the code yet, but I did try it out > with SDP/AIO on a pair of x86 systems with Tavors and a pair of > x86_64 systems with Arbels. With a small change to core/fmr_pool.c > and enabling pool creation in

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-28 Thread Roland Dreier
Libor> Here is the change I made to core/fmr_pool.c: BTW I already committed this fix. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-28 Thread Libor Michalek
On Sun, Mar 27, 2005 at 05:31:13PM +0200, Michael S. Tsirkin wrote: > OK, here's an updated version of the patch. This passed basic > tests: allocate/free, map/remap/unmap. > > For Tavor, MTTs for FMR are separate from regular MTTs, and are reserved > at driver initialization. This is done to limi

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-28 Thread Grant Grundler
On Sun, Mar 27, 2005 at 05:31:13PM +0200, Michael S. Tsirkin wrote: > For Tavor, MTTs for FMR are separate from regular MTTs, and are reserved > at driver initialization. This is done to limit the amount of > virtual memory needed to map the MTTs. > For Arbel, there's no such limitation, and all MT

Re: [openib-general] [PATCH] FMR support in mthca

2005-03-28 Thread Roland Dreier
Excellent! I'll review this ASAP, and see if Libor and I can get SDP/AIO going with it. - R. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mai

[openib-general] [PATCH] FMR support in mthca

2005-03-27 Thread Michael S. Tsirkin
OK, here's an updated version of the patch. This passed basic tests: allocate/free, map/remap/unmap. For Tavor, MTTs for FMR are separate from regular MTTs, and are reserved at driver initialization. This is done to limit the amount of virtual memory needed to map the MTTs. For Arbel, there's no s

Re: [openib-general] [PATCH] fmr support in mthca

2005-03-18 Thread Roland Dreier
Michael> Good, glad to help. I will try to address your comments Michael> next week (its already weekend here). No problem. Libor won't be back until Monday so I won't even try SDP until then. Roland> What if we just reserve something like 64K MPTs and MTTs Roland> for FMRs and i

Re: [openib-general] [PATCH] fmr support in mthca

2005-03-18 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: [openib-general] [PATCH] fmr support in mthca > > Thanks for implementing this. You saved me a lot of work and Libor > will be very happy when he gets back next week. Good, glad to help. I will try to address your

Re: [openib-general] [PATCH] fmr support in mthca

2005-03-17 Thread Roland Dreier
Thanks for implementing this. You saved me a lot of work and Libor will be very happy when he gets back next week. Some comments from my first read through: > This patch implements FMR support. I also rolled into it two fixes > for regular mrs that I posed previously, let me know if its a prob

[openib-general] [PATCH] fmr support in mthca

2005-03-17 Thread Michael S. Tsirkin
This patch implements FMR support. I also rolled into it two fixes for regular mrs that I posed previously, let me know if its a problem. This seems to be working fine for me, although I only did relatively basic tests. Both Tavor and Arbel Native modes are supported. I made some tradeoffs for sim