RE: [openib-general] How about ib_send_page() ?

2005-06-09 Thread Vivek Kashyap
On Tue, 7 Jun 2005, Michael Krause wrote: At 09:28 AM 6/7/2005, Fab Tillier wrote: From: Roland Dreier [mailto:[EMAIL PROTECTED] Sent: Tuesday, June 07, 2005 8:38 AM Michael Why not just use the IETF draft for RC / UC based IP over Michael IB and not worry about creating

Re: [openib-general] Re: [openib-commits] r2480 - gen2/trunk/src/userspace/perftest

2005-06-09 Thread Bernhard Fischer
On Wed, Jun 08, 2005 at 04:46:37PM -0700, Grant Grundler wrote: On Wed, Jun 08, 2005 at 04:40:57PM -0700, Grant Grundler wrote: Index: rdma_bw.c === --- rdma_bw.c(revision 2570) +++ rdma_bw.c(working copy) @@

RE: [openib-general] [PATCH][RFC] kDAPL: remove dat wrapper funct ion dat_ia_query()

2005-06-09 Thread Itamar Rabenstein
Hi Tom, It looks very good to me but i want to ask why do we need the dat_common struct ? as we can see in dat.h dat_common is defined as : struct dat_common { struct dat_provider *provider; union dat_context context; }; and is being use as data member of any dat_* struct

[openib-general] RE: [patch][kdapl] enable kdapltest -T P

2005-06-09 Thread James Lentini
On Wed, 8 Jun 2005, Itamar Rabenstein wrote: Current openib gen2 code is not reporting the max cq size and i dont think that we should put a fix number . if we want to get the number we need Roland to fill this number in mthca but as Roland said before what real App will meed this number?

[openib-general] Re: [patch][kdapl] enable kdapltest -T P

2005-06-09 Thread James Lentini
Committed in revision 2574. On Thu, 2 Jun 2005, Itamar wrote: itamar in order to enable kdapltest -T P i needed to remark attr that are not set by openib gen2 itamar 1) ia_attr.max_evd_qlen itamar 2) ia_attr.max_rdma_read_per_ep_in itamar 3) ia_attr.max_rdma_read_per_ep_out itamar itamar also

RE: [openib-general] kdaptest wedges server

2005-06-09 Thread James Lentini
Itamar, Were you able to get to the bottom of this? james On Thu, 2 Jun 2005, Itamar Rabenstein wrote: itamarHi Josh, itamar itamarCan you run it with debug option ( -d ) itamarand tell me what is the last debug print on the server side ? itamarwhen you say kill can you hit ^c or is it

[openib-general] Re: IPoIB MTU Changing

2005-06-09 Thread Roland Dreier
Hal Shouldn't the new MTU be compared to the link MTU (minus the Hal encapsulation header length) rather than a hardcoded 2K ? Yes, that seems right. - R. ___ openib-general mailing list openib-general@openib.org

Re: [openib-general] Re: IPoIB MTU Changing

2005-06-09 Thread Roland Dreier
I take that back. I believe the code is correct as it stands. There are two MTU values we keep track of: admin_mtu, requested by the user, and mcast_mtu, calculated from the link MTU. We use the lower of the two values. This allows the user to set an MTU and not have the setting lost if the

Re: [openib-general] [PATCH][RFC] kDAPL: remove dat wrapper funct ion dat_ia_query()

2005-06-09 Thread Christoph Hellwig
Btw, it seems the openib list manager software has a bug in that it doesn't send you a copy of a mail you received a personal copy of. Any chance the list admin could look into this? ___ openib-general mailing list openib-general@openib.org

Re: [openib-general] Re: IPoIB MTU Changing

2005-06-09 Thread Hal Rosenstock
On Thu, 2005-06-09 at 12:13, Roland Dreier wrote: I take that back. I believe the code is correct as it stands. There are two MTU values we keep track of: admin_mtu, requested by the user, and mcast_mtu, calculated from the link MTU. We use the lower of the two values. This allows the

Re: [openib-general] shutdown issue with SDP

2005-06-09 Thread Libor Michalek
On Wed, Jun 08, 2005 at 05:53:10PM -0400, William Jordan wrote: On 6/8/05, Libor Michalek [EMAIL PROTECTED] wrote: On Wed, Jun 08, 2005 at 01:29:42PM -0400, William Jordan wrote: I'm trying to run netperf over SDP using libsdp. I'm having intermittent success, but usually, I get a

Re: [openib-general] Re: IPoIB MTU Changing

2005-06-09 Thread Roland Dreier
Hal Yes, the multicast MTU could further trim this down but the Hal admin MTU appears to be currently capped at IPOIB_PACKET_SIZE Hal - IPOIB_ENCAP_LEN. That's OK for a 2K link MTU but if the Hal link were negotiated at 4K MTU, shouldn't this be allowed to Hal be larger if

[openib-general] RE: [patch][kdapl] enable kdapltest -T P

2005-06-09 Thread James Lentini
On Thu, 9 Jun 2005, James Lentini wrote: On Wed, 8 Jun 2005, Itamar Rabenstein wrote: Current openib gen2 code is not reporting the max cq size and i dont think that we should put a fix number . if we want to get the number we need Roland to fill this number in mthca but as Roland said

Re: [openib-general] Re: IPoIB MTU Changing

2005-06-09 Thread Hal Rosenstock
On Thu, 2005-06-09 at 12:48, Roland Dreier wrote: Hal Yes, the multicast MTU could further trim this down but the Hal admin MTU appears to be currently capped at IPOIB_PACKET_SIZE Hal - IPOIB_ENCAP_LEN. That's OK for a 2K link MTU but if the Hal link were negotiated at 4K

Re: [openib-general] Re: IPoIB MTU Changing

2005-06-09 Thread Roland Dreier
Hal OK. The changes go a little deeper than I thought. Would it Hal be worthwhile to accomodate this for HCAs which support 4K Hal MTU ? Sure. - R. ___ openib-general mailing list openib-general@openib.org

RE: [openib-general] kdaptest wedges server

2005-06-09 Thread Itamar Rabenstein
No , I dont see the problem at my system . Itamar -Original Message- From: James Lentini [mailto:[EMAIL PROTECTED] Sent: Thursday, June 09, 2005 6:03 PM To: Itamar Rabenstein Cc: Josh England; openib-general Subject: RE: [openib-general] kdaptest wedges server Itamar,

[openib-general] RE: [patch][kdapl] enable kdapltest -T P

2005-06-09 Thread Itamar Rabenstein
In preparation for asking Roland about this, I added a print statement as you suggested. I actually see the ib_device_attr structure's max_qp_wr and max_cqe values as being initialized to 65535. After taking a second look at the initialization code in mthca_query_device(), I see that

[openib-general] RE: [patch][kdapl] enable kdapltest -T P

2005-06-09 Thread James Lentini
Can you confirm that removing the #if 0 on line 84 of dapl_performance_util.c works with the current DAT provider and IB stack? james On Thu, 9 Jun 2005, Itamar Rabenstein wrote: In preparation for asking Roland about this, I added a print statement as you suggested. I actually see the

[openib-general] [PATCHv3] kDAPL: remove dat wrapper function dat_ia_query()

2005-06-09 Thread Tom Duffy
On Thu, 2005-06-09 at 11:37 +0300, Itamar Rabenstein wrote: and i agree that we dont need the dat_ before the functions name Ok, this patch renames the method ia_query (instead of dat_ia_query). James, can you please apply? If you do, I will write a patch to do the rest. We can do the

RE: [openib-general] kdaptest wedges server

2005-06-09 Thread James Lentini
Josh, Was there any information about what occurred? Loading the the DAT modules with modprobe dat dbg_mask=0xff modprobe ib_dat_provider dbg_mask=0x will produce a lot of debug information that might help. james On Thu, 9 Jun 2005, Itamar Rabenstein wrote: No , I dont see the

[openib-general] Re: uverbs_main cleanup

2005-06-09 Thread Roland Dreier
Hal Hi Roland, Should the order of unregistering the uverbs event Hal fs and the unmounting be reversed or does this not matter ? I don't think this makes any practical difference but in the interest of consistency between initialization and cleanup order, I made this change.

[openib-general] [PATCHv6] SDP: Allow SDP to compile on 2.6.12-rc6

2005-06-09 Thread Tom Duffy
This updates the 2.6.12 SDP patch against the state change patch Libor checked into r2577. It also fixes a bug I was seeing regarding freeing of resources when tearing down a connection. I am still trying to track down a bug with this patch that occurs when you try to initiate an active

Re: [openib-general] [PATCHv6] SDP: Allow SDP to compile on 2.6.12-rc6

2005-06-09 Thread Libor Michalek
On Thu, Jun 09, 2005 at 11:51:32AM -0700, Tom Duffy wrote: I am still trying to track down a bug with this patch that occurs when you try to initiate an active connection and there is no passive listener ready. This can cause a panic on the passive side if ib_sdp is loaded. Do you want

[openib-general] [PATCH] [TRIVIAL] [kdapl] CM: Message and commentary changes

2005-06-09 Thread Hal Rosenstock
Message and commentary changes to make consistent with ongoing code changes Signed-off-by: Hal Rosenstock [EMAIL PROTECTED] Index: dapl_openib_cm.c === --- dapl_openib_cm.c(revision 2577) +++ dapl_openib_cm.c(working copy)

Re: [openib-general] Re: [PATCHv3] kDAPL: remove use of HANDLE's (vs. r2564)

2005-06-09 Thread Hal Rosenstock
On Wed, 2005-06-08 at 16:44, Tom Duffy wrote: This just in! Don't know if was caused by your patches or mine, but I ran the server on 192.168.0.26, and tried to connect with the client, it didn't connect right away, so I did a control-c. Then, the kernel panic'ed: [EMAIL PROTECTED] ~]#

[openib-general] Re: [PATCH] [TRIVIAL] [kdapl] CM: Message and commentary changes

2005-06-09 Thread James Lentini
Committed in revision 2578. On Thu, 9 Jun 2005, Hal Rosenstock wrote: halr Message and commentary changes to make consistent with ongoing code halr changes halr halr Signed-off-by: Hal Rosenstock [EMAIL PROTECTED] halr halr Index: dapl_openib_cm.c halr

RE: [openib-general] [CM] possible problem with crossing DREQs.

2005-06-09 Thread Sean Hefty
I'm seeing an unusual problem when both halves of a connection actively disconnect at the same time. Each connection peer issues a DREQ at the same time, next each receive the DREQ and responds with a DREP, and finally each connection gets a callback for the transition to the idle state.

[openib-general] opensm

2005-06-09 Thread Joe Damato
Hello, I recently started to check out infiniband -- unfortunately I am unable to find documentation explaining how to use the different tools. Most notably, I have read lots of things that mention opensm and/or osmtest, but I have been unable to find documentation that explains how to use

Re: [openib-general] opensm

2005-06-09 Thread Hal Rosenstock
On Thu, 2005-06-09 at 16:21, Joe Damato wrote: Hello, I recently started to check out infiniband -- unfortunately I am unable to find documentation explaining how to use the different tools. Most notably, I have read lots of things that mention opensm and/or osmtest, I'm curious as

Re: [openib-general] [PATCH][RFC] kDAPL: remove dat wrapper funct ion dat_ia_query()

2005-06-09 Thread Caitlin Bestler
The reason the method table is pointed at in each object is to avoid having to run long chains of memory references to get the method table. If you have the ep pointer you should be able to get the method table directly, not have to fetch the ia then the hca then the method table. And as you

Re: [openib-general] [PATCH][RFC] kDAPL: remove dat wrapper funct ion dat_ia_query()

2005-06-09 Thread Tom Duffy
On Thu, 2005-06-09 at 14:27 -0400, James Lentini wrote: Tom, I think this change has a lot of merits, but I would prefer to defer this sort of change until we have a more stable provider. Once we have removed the bugs and finished our update of the provider code (removing the custom DAPL

[openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Shirley Ma
I used cyclets_to_units() get Bandwidth peak (#0 to #972): 1852.72 MByte/sec Bandwidth average: 1852.67 MByte/sec Which is far away from the 1X (2.5Gbit/s) throughtput. This per secs cycles value is incorrect. Either change the measurment to gettimeofday() to measure MB/s or measure the how

Re: [openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Roland Dreier
Shirley I used cyclets_to_units() get Bandwidth peak (#0 to Shirley #972): 1852.72 MByte/sec Shirley Which is far away from the 1X (2.5Gbit/s) throughtput. Actually it's not that far away -- the 2.5 Gbit/sec is after 8b/10b coding, so the raw data rate is really 2 Gbit/sec. Getting

Re: [openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Shirley Ma
BTW I'm assuming MByte/sec is a typo for Mbit/sec. No, that's the reason I said wrong. Using gettimeofday(), the throughtput is around 230MB/sec. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll Parkway Beaverton, OR 97006-6063 Phone(Fax): (503) 578-7638

Re: [openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Shirley Ma
So are you sure MByte/sec is not just a typo? Yes. It's easy to test it. I tested on both intel and PPC platforms. The get_cycles() doesn't return the value as we expect. Need to read the architecture to find the register definitions. Thanks Shirley Ma IBM Linux Technology Center 15300 SW Koll

Re: [openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Grant Grundler
On Thu, Jun 09, 2005 at 04:14:17PM -0700, Shirley Ma wrote: I used cyclets_to_units() get cycles_to_units is a FP value, not a function. get_cpu_mhz() is probably what you meant. What does cat /proc/cpuinfo say on the boxes you see this on? Bandwidth peak (#0 to #972): 1852.72 MByte/sec

Re: [openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Shirley Ma
Does rdma_lat have the same issue? What does rdma_lat report? Latency typical: 0.717812 usec Latency best : 0.705625 usec Latency typical: 1149 cycles Latency best : 1129.5 cycles after modifying the per usec cycles the results are: Latency typical: 5.76884 usec Latency best : 5.66834

Re: [openib-general] cycles_to_units is incorrect in rdma_lat, rdma_bw.

2005-06-09 Thread Shirley Ma
I think adding such a calibration to get_cpu_mhz() so we can warn if /proc/cpuinfo data doesn't agree with gettimeofday() and get_cycles(). using below to calculate the cycles_to_unit is more accurate. clock_gettime(); get_cycles(); sleep(5); clock_gettime(); get_cycles();