Re: [openib-general] question - mapping QPIDs back to ptrs

2006-11-17 Thread Roland Dreier
> The Chelsio driver is hogging lots of memory right now for mapping > PDIDs, QPIDs, CQIDs, and STAG IDs back to their respective kernel > structures. This is done via an array of pointers, indexed by the ID. > The critical performance mapping is finding a QP struct from the QPID in > the pol

[openib-general] question - mapping QPIDs back to ptrs

2006-11-17 Thread Steve Wise
The Chelsio driver is hogging lots of memory right now for mapping PDIDs, QPIDs, CQIDs, and STAG IDs back to their respective kernel structures. This is done via an array of pointers, indexed by the ID. The critical performance mapping is finding a QP struct from the QPID in the poll path. Arrays

Re: [openib-general] Question about the query QP mask

2006-11-16 Thread Roland Dreier
> What should be the expected behavior? > Should this description should be changed or should the low level drivers > of mthca and ipath need to be changed? The mask is used as a hint to the low-level driver about which attributes the consumer cares about. The driver may fill in more fields, b

Re: [openib-general] Question about multicast GIDs

2006-11-16 Thread Robert Walsh
Robert Walsh wrote: > Roland Dreier wrote: >> > Is there are registration authority for multicast GIDs? Or at >> least a > safe way of assigning a range of GIDs to a vendor? >> >> I don't think so. Perhaps RFC 3307 would be of some use... > > Ah - looks exactly like what I was looking for. T

[openib-general] Question about the query QP mask

2006-11-16 Thread dotanb
Hi. in the file ib_verbs, in the description of the verb ib_query_qp it is written: "The qp_attr_mask may be used to limit the query o gathering only the selected attributes.". I checked the low level drivers of all of the HCAs and only the eHCA is actually behave like this (and set ONLY the mask

Re: [openib-general] Question about multicast GIDs

2006-11-15 Thread Robert Walsh
Roland Dreier wrote: > > Is there are registration authority for multicast GIDs? Or at least a > > safe way of assigning a range of GIDs to a vendor? > > I don't think so. Perhaps RFC 3307 would be of some use... Ah - looks exactly like what I was looking for. Thanks. _

Re: [openib-general] Question about multicast GIDs

2006-11-15 Thread Roland Dreier
> Is there are registration authority for multicast GIDs? Or at least a > safe way of assigning a range of GIDs to a vendor? I don't think so. Perhaps RFC 3307 would be of some use... - R. ___ openib-general mailing list openib-general@openib.org

[openib-general] Question about multicast GIDs

2006-11-15 Thread Robert Walsh
Hi all, Is there are registration authority for multicast GIDs? Or at least a safe way of assigning a range of GIDs to a vendor? Regards, Robert. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-ge

Re: [openib-general] question on QoS support

2006-11-06 Thread Hal Rosenstock
hysErr:8 > OverrunErr:..8 > MaxCreditHint:...0 > RoundTrip:...0 Do you have an IB analyzer ? -- Hal > Feiyi > > -Original Message- > From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > Sent:

Re: [openib-general] question on QoS support

2006-11-06 Thread Wang, Feiyi
:...0 RoundTrip:...0 Feiyi -Original Message- From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Friday, November 03, 2006 3:58 PM To: Wang, Feiyi Cc: openib-general@openib.org Subject: RE: [openib-general] question on QoS support On Fri, 2006-11-03 at

Re: [openib-general] question on QoS support

2006-11-03 Thread Hal Rosenstock
o:[EMAIL PROTECTED] > Sent: Friday, November 03, 2006 3:51 PM > To: Wang, Feiyi > Cc: openib-general@openib.org > Subject: RE: [openib-general] question on QoS support > > On Fri, 2006-11-03 at 15:43, Wang, Feiyi wrote: > > The test is done on two hosts, say A and B. A has 4x S

Re: [openib-general] question on QoS support

2006-11-03 Thread Wang, Feiyi
er 03, 2006 3:51 PM To: Wang, Feiyi Cc: openib-general@openib.org Subject: RE: [openib-general] question on QoS support On Fri, 2006-11-03 at 15:43, Wang, Feiyi wrote: > The test is done on two hosts, say A and B. A has 4x SDR (run ib_rdam_bw > as server), B has 4x DDR (run more than one thread of

Re: [openib-general] question on QoS support

2006-11-03 Thread Hal Rosenstock
On Fri, 2006-11-03 at 15:12, Feiyi Wang wrote: > In our test at the ORNL - it appears you can "turn off" the traffic by > giving every VL weight 0. A weight of 0 indicates to skip that entry. > As soon as you assign non-zero VL weight, > the traffic starts to flow, however, VL with more weight d

Re: [openib-general] question on QoS support

2006-11-03 Thread Feiyi Wang
In our test at the ORNL - it appears you can "turn off" the traffic by giving every VL weight 0. As soon as you assign non-zero VL weight, the traffic starts to flow, however, VL with more weight doesn't have expected preference treatment. In other words, traffic shaping didn't take place. smpquery

Re: [openib-general] Question on ucma

2006-11-03 Thread Steve Wise
Sean posted 7 patches that include the ucma support. You'll need those + the one librdmacm patch he posted. Steve. On Fri, 2006-11-03 at 13:59 +0530, Krishna Kumar2 wrote: > Hi, > > I installed the 2.6.19-rc3 bits, and when I try to run > perftest/rdma_bw (with '-c' option), I get the error :

[openib-general] Question on ucma

2006-11-03 Thread Krishna Kumar2
Hi, I installed the 2.6.19-rc3 bits, and when I try to run perftest/rdma_bw (with '-c' option), I get the error : "librdmacm: Couldnt open rdma_cm ABI version". I found that this is due to ucma not being present in mainline kernel bits (which creates /sys/class/misc/rdma_cm). So how can I resolve

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
Hi Oliver, On Thu, 2006-11-02 at 10:20, Oliver wrote: > Hi, Hal - > > > How is this being observed/measured ? > > Host A, B, with 4x DDR both connected to Flextronic switch. > A single process of ibv_read_bw gives about 1415MB /s average > bandwidth. Two concurrent process report 714.45 MB/s eac

Re: [openib-general] question on QoS support

2006-11-02 Thread Oliver
Hi, Hal - > How is this being observed/measured ? Host A, B, with 4x DDR both connected to Flextronic switch. A single process of ibv_read_bw gives about 1415MB /s average bandwidth. Two concurrent process report 714.45 MB/s each, dead even. Now if I bump up one process with a different SL, then

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
On Thu, 2006-11-02 at 09:15, Makia Minich wrote: > Hal Rosenstock wrote: > > Makia, > > > > On Wed, 2006-11-01 at 17:42, Makia Minich wrote: > >> It just so happens that we've started looking at this here at ORNL as > >> well. I had a question about the options. The manpage makes it seem > >> th

Re: [openib-general] question on QoS support

2006-11-02 Thread Makia Minich
Hal Rosenstock wrote: > Makia, > > On Wed, 2006-11-01 at 17:42, Makia Minich wrote: >> It just so happens that we've started looking at this here at ORNL as >> well. I had a question about the options. The manpage makes it seem >> that you can set these qos options (e.g. qos_high_limit) from the

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
Makia, On Wed, 2006-11-01 at 17:42, Makia Minich wrote: > It just so happens that we've started looking at this here at ORNL as > well. I had a question about the options. The manpage makes it seem > that you can set these qos options (e.g. qos_high_limit) from the > command line, but I haven't

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
Hi Oliver, On Wed, 2006-11-01 at 16:52, Oliver wrote: > Hi, folks - > > I am trying to verify and evaluate IB QoS support, running openSM as > subnet manager. The perftest program is extended to set SL as command > line options instead of default 0, and by modifying VL arbitration > tables, I am

Re: [openib-general] question on QoS support

2006-11-01 Thread Sasha Khapyorsky
On 17:42 Wed 01 Nov , Makia Minich wrote: > It just so happens that we've started looking at this here at ORNL as > well. I had a question about the options. The manpage makes it seem > that you can set these qos options (e.g. qos_high_limit) from the > command line, AFAIK there is option -Q

Re: [openib-general] question on QoS support

2006-11-01 Thread Makia Minich
It just so happens that we've started looking at this here at ORNL as well. I had a question about the options. The manpage makes it seem that you can set these qos options (e.g. qos_high_limit) from the command line, but I haven't been overly successful. Is there an example of this being done?

Re: [openib-general] question on QoS support

2006-11-01 Thread Sasha Khapyorsky
On 16:52 Wed 01 Nov , Oliver wrote: > Hi, folks - > > I am trying to verify and evaluate IB QoS support, running openSM as > subnet manager. The perftest program is extended to set SL as command > line options instead of default 0, and by modifying VL arbitration > tables, I am expecting to se

[openib-general] question on QoS support

2006-11-01 Thread Oliver
Hi, folks - I am trying to verify and evaluate IB QoS support, running openSM as subnet manager. The perftest program is extended to set SL as command line options instead of default 0, and by modifying VL arbitration tables, I am expecting to see the traffic shaping can actually take place, but i

[openib-general] Question about gen2_basic test

2006-10-13 Thread Robert Walsh
Hi all, I've a question about one of the gen2_basic tests. The test is test 3 of the QP test collection. There's a piece of code in this test that does a modify_qp, followed by a query_qp. The query QP bit checks that the modify_qp did what was expected of it. One check looks like this:

Re: [openib-general] Question about ehca CQ handling

2006-10-02 Thread Christoph Raisch
> While looking over the ehca driver from the perspective of adding a > "peek CQ" operation, I noticed some code that looked funny. > > In hipz_set_cqx_n0() and hipz_set_cqx_n1(), what is the point of the > calls to hipz_galpa_load_cq()? The return value is discarded. I see > that hipz_galpa_load

[openib-general] Question about ehca CQ handling

2006-09-25 Thread Roland Dreier
While looking over the ehca driver from the perspective of adding a "peek CQ" operation, I noticed some code that looked funny. In hipz_set_cqx_n0() and hipz_set_cqx_n1(), what is the point of the calls to hipz_galpa_load_cq()? The return value is discarded. I see that hipz_galpa_load_cq() deref

Re: [openib-general] Question about interrupt generation

2006-09-05 Thread harish
Hi,One more question. What kind of event mask helps mask the interrupts?thanksharishOn 9/5/06, harish < [EMAIL PROTECTED]> wrote:Hi All,I tried the following simple experiment and am not able to understand the results: Calcualted the number of interrupts  generated by the infiniband [with little or

[openib-general] Question about interrupt generation

2006-09-05 Thread harish
Hi All,I tried the following simple experiment and am not able to understand the results:Calcualted the number of interrupts  generated by the infiniband [with little or no traffic to the NIC] over a period of 10seconds and saw around 10-20 interrupts/sec. Then ran a netperf test and saw around 100

Re: [openib-general] question: ib_umem page_size

2006-08-23 Thread Roland Dreier
> > It gives the page size for the user memory described by the struct. > > The idea was that if/when someone tries to optimize for huge pages, > > then the low-level driver can know that a region is using huge pages > > without having to walk through the page list and search for the > > minim

Re: [openib-general] question: ib_umem page_size

2006-08-23 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: question: ib_umem page_size > > Michael> Roland, could you please clarify what does the page_size > Michael> field in struct ib_mem do? > > It gives the page size for the user memory described by the struct. > The idea was that

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-22 Thread Sean Hefty
>Cool, I would go for XOR-ing a random value with the **local id** . > >Sean, my understanding it can be narrowed for doing so in: > >1) cm_alloc_id() after calling idr_get_new_above() >2) cm_free_id() before calling idr_remove() >3) cm_get_id() before calling idr_find() > >and initializing the ran

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-22 Thread Or Gerlitz
Sean Hefty wrote: > When a new REQ is received, we enter its timewait structure into two trees: one > sorted by remote ID, one sorted by remote QPN. If the REQ is new, both would > succeed, and timewait_info would be NULL. Since timewait_info is not NULL, we > are dealing with a REQ that re-us

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-22 Thread Or Gerlitz
Roland Dreier wrote: > Sean> If we record a base offset, we can start at any random > Sean> number. We just need to always add/subtract the base when > Sean> getting a value from the IDR. > > Good point -- or better still, we could XOR in a random bit pattern. > That way we don't have

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Roland Dreier
Sean> If we record a base offset, we can start at any random Sean> number. We just need to always add/subtract the base when Sean> getting a value from the IDR. Good point -- or better still, we could XOR in a random bit pattern. That way we don't have to keep straight when to add and

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Sean Hefty
>> If we get here, this means that the REQ was a new REQ and not a >> duplicate, but the remote_id or remote_qpn is already in use. We need >> to reject the new REQ as containing stale data. > >I don't follow, if we get to the else case its as of cm_get_id() >returning NULL. This holds when idr_fi

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Sean Hefty
>> Just to emphasize what Sean has pointed out, you are asking how can a CM >> consumer know that a **local** QPN is not in the timewait state >> according to the **remote** CM. Since the issue is with the remote CM, >> it seems to me that pushing down timewait into verbs is not the correct >> dire

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Sean Hefty
>How about (for the meantime, till this rework is designed && done) going >to projecting the initial random local id into the range of (say) >[0-1022] (i think 1023 is prime, if not choose a prime near it) this way >with very good probability and with very little overhead on memory >consumption a c

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Roland Dreier
Or> How about (for the meantime, till this rework is designed && Or> done) going to projecting the initial random local id into the Or> range of (say) [0-1022] (i think 1023 is prime, if not choose Or> a prime near it) this way with very good probability and with Or> very little

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
This email appear in the archive, but seems not to be distributed to the subscribers so i am reposting it. Or Gerlitz wrote: > Sean Hefty wrote: >> Even if we pushed timewait handling under verbs, a user could always >> get a QP that the remote side thinks is connected. The original >> connec

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
This email appear in the archive, but seems not to be distributed to the subscribers so i am reposting it. Or Gerlitz wrote: > Arlin Davis wrote: >> We are running into connection reject issues (IB_CM_REJ_STALE_CONN) >> with our application under heavy load and lots of connections. >> >> We occa

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
>>> + } else >>> + cm_issue_rej(work->port, work->mad_recv_wc, >>> +IB_CM_REJ_STALE_CONN, >>> CM_MSG_RESPONSE_REQ, >>> +NULL, 0); >> >> >> what is this case? there is no entry but there is

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
Sean Hefty wrote: > Or Gerlitz wrote: >> If you don't mind (also related to the patch you have sent Eric of >> randomizing the initial local cm id) to get into this deeper, can we do > There's an issue trying to randomize the initial local CM ID. The way > the IDR works, if you start at a hig

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-17 Thread Sean Hefty
Or Gerlitz wrote: > If you don't mind (also related to the patch you have sent Eric of > randomizing the initial local cm id) to get into this deeper, can we do There's an issue trying to randomize the initial local CM ID. The way the IDR works, if you start at a high value, then the IDR size

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-16 Thread Sean Hefty
Arlin Davis wrote: > How can a consumer know for sure that the new QP will not be in a > timewait state according to the CM? Given that the QP may have been in use by another process, I don't think that there's any way for the new owner to know. > Does it make sense to push the timewait functio

[openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-16 Thread Arlin Davis
We are running into connection reject issues (IB_CM_REJ_STALE_CONN) with our application under heavy load and lots of connections. We occassionally get a reject based on the QP being in timewait state leftover from a prior connection. It appears that the CM keeps track of the QP's in timewait

Re: [openib-general] question: ib_umem page_size

2006-08-15 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: question: ib_umem page_size > > Michael> Roland, could you please clarify what does the page_size > Michael> field in struct ib_mem do? > > It gives the page size for the user memory described by the struct. > The idea was that

Re: [openib-general] question: ib_umem page_size

2006-08-15 Thread Roland Dreier
Michael> Roland, could you please clarify what does the page_size Michael> field in struct ib_mem do? It gives the page size for the user memory described by the struct. The idea was that if/when someone tries to optimize for huge pages, then the low-level driver can know that a region is

[openib-general] question: ib_umem page_size

2006-08-15 Thread Michael S. Tsirkin
Roland, could you please clarify what does the page_size field in struct ib_mem do? -- MST ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailm

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Parks Fields
At 12:36 PM 6/5/2006, Talpey, Thomas wrote: Thanks Parks, this is a very interesting perspective. I will avoid going into my rant about edge devices for now, however. :-) Cool, you can send it direct if you want. I am not sure what you mean about using SDP "end to end". I assume you would pe

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
Thanks Parks, this is a very interesting perspective. I will avoid going into my rant about edge devices for now, however. :-) I am not sure what you mean about using SDP "end to end". I assume you would perhaps use SDP to these edge nodes, but this would require terminating the SDP connection and

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Parks Fields
I consider IPoIB to be Ethernet emulation. As for apples and oranges, my point exactly. It is not really about comparisons. Here at LANL we have an environment where all our new Clusters have to mount our global parallel file system Panasas. It is ethernet and will be for a while. Clust

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
bout the IPoIB bandwidth performance ? > (Talpey, Thomas) > 8. Re: Question about the IPoIB bandwidth performance ? (hbchen) > >----- Message from "hbchen" <[EMAIL PROTECTED]> on Mon, 05 Jun 2006 09:38:24 >-0600 - > >

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hycsw
Tom, We are in the process of measuring the CPU utilization on our NFS/RDMA experiments in contrast with regular the NFS, we also intend to include netperf numbers and will keep you posted with our results as soon as possible. Helen - original Message - >From [EMAIL PROTECTED] Mon Jun

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Bernard King-Smith
IL PROTECTED]> on Mon, 05 Jun 2006 09:38:24 -0600 - To: "Hal Rosenstock" <[EMAIL PROTECTED]> cc: "OPENIB"

RE: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Felix Marti
    From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of hbchen Sent: Monday, June 05, 2006 9:12 AM To: Talpey, Thomas Cc: openib-general@openib.org Subject: Re: [openib-general] Question about the IPoIB bandwidth performance ?   Talpey, Thomas wrote: At 11:38

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
At 12:11 PM 6/5/2006, hbchen wrote: >>Perhaps you are out of CPU. >> >> >Tom, >I am HB Chen from LANL not the Helen Chen from SNL. Oops, sorry! I have too many email messages going by. :-) HB, then. >I didn't run out of CPU. It is about 70-80 % of CPU utilization. But, is one CPU at 100%? In

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hbchen
Talpey, Thomas wrote: At 11:38 AM 6/5/2006, hbchen wrote: Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth utilization is still very low. IPoIB=420MB/sec bandwidth utilization= 420/1024 = 41.01% Helen, have you

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
At 11:38 AM 6/5/2006, hbchen wrote: >Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth utilization >is still very low. >>> IPoIB=420MB/sec >>> bandwidth utilization= 420/1024 = 41.01% Helen, have you measured the CPU utilizations during these runs? Perhaps you are out of CPU. O

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Bernard King-Smith
Hal Rosenstock wrote: > On Mon, 2006-06-05 at 11:12, hbchen wrote: > > Hi, > > I have a question about the IPoIB bandwidth performance. > > I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G > > ethernet card, > > and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). > > >

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hbchen
Hal Rosenstock wrote: On Mon, 2006-06-05 at 11:12, hbchen wrote: Hi, I have a question about the IPoIB bandwidth performance. I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G ethernet card, and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). NIC (Jumb

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Hal Rosenstock
On Mon, 2006-06-05 at 11:12, hbchen wrote: > Hi, > I have a question about the IPoIB bandwidth performance. > I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G > ethernet card, > and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). > > > NIC (Jumbo enabled) Line bandwid

[openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hbchen
Hi, I have a question about the IPoIB bandwidth performance. I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G ethernet card, and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). NIC (Jumbo enabled) Line bandwidth(LB) IPoverNIC bandwidth utilization (IPoNIC/LB)

[openib-general] [Question]Fast Registration Work Request and Fast Memory Regions

2006-05-24 Thread Ian Jiang
Hi all!In IBA spec. 1.2 defines a Fast Registration Work Request and it should be used together with the Allocate L Key verb to get a fast registration in my opinion. Fast Memory Regions is used in the ULPs of IB Stack, such as kDAPL, iSER. The two comfuse me very much. I don't think they are the s

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Hal Rosenstock
On Sun, 2006-05-14 at 15:30, Jason Gunthorpe wrote: > On Sun, May 14, 2006 at 07:40:25AM -0400, Hal Rosenstock wrote: > > > > Not always true in terms of local subnet (multicast and management MAD > > > > response exceptions). > > > > > > Yes, but these are well specified. Multicast must always ha

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Jason Gunthorpe
On Sun, May 14, 2006 at 07:40:25AM -0400, Hal Rosenstock wrote: > > > Not always true in terms of local subnet (multicast and management MAD > > > response exceptions). > > > > Yes, but these are well specified. Multicast must always have a GRH. > > MAD requests are covered under my scenario above

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Hal Rosenstock
On Fri, 2006-05-12 at 13:55, Sean Hefty wrote: > Jason Gunthorpe wrote: > > How about this, how do you see this scenario: > > > > 1) Client gets a DGID from 'someplace' > > 2) Client sends a SA query to resolve the DGID to a Path Record > > 3) Client configures a QP based on the Path Record > > >

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Hal Rosenstock
On Fri, 2006-05-12 at 13:10, Jason Gunthorpe wrote: > On Fri, May 12, 2006 at 08:11:17AM -0400, Hal Rosenstock wrote: > > > > To allow what Roland is talking about you need an unambiguous > > > mechanism where the SA can signal to the client that the path > > > needs a GRH. > > > > Ah, you are re

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Sean Hefty
Jason Gunthorpe wrote: How about this, how do you see this scenario: 1) Client gets a DGID from 'someplace' 2) Client sends a SA query to resolve the DGID to a Path Record 3) Client configures a QP based on the Path Record Now, the question I'm interested in is this: During step #3 what test

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Jason Gunthorpe
On Fri, May 12, 2006 at 08:11:17AM -0400, Hal Rosenstock wrote: > > To allow what Roland is talking about you need an unambiguous > > mechanism where the SA can signal to the client that the path > > needs a GRH. > > Ah, you are referring to the SA path record response not the request. Yes.. Tho

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Hal Rosenstock
On Thu, 2006-05-11 at 13:12, Jason Gunthorpe wrote: > On Thu, May 11, 2006 at 07:20:19AM -0400, Hal Rosenstock wrote: > > > That would be a simpler check but HopLimit is not a required component > > of PathRecord but I think this may not be sufficient as just because a > > HopLimit >= 2 doesn't me

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Jason Gunthorpe
On Thu, May 11, 2006 at 10:21:08AM -0700, Sean Hefty wrote: > Hal Rosenstock wrote: > >Anytime the send is off the local subnet (as well as multicast), a GRH > >is required. Also, there is a management response rule for responding > >when the request contained a GRH that require a GRH (13.5.4.4 p.

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Hal Rosenstock
On Thu, 2006-05-11 at 13:29, Roland Dreier wrote: > Sean> We currently use ARP to resolve an IP address to a DGID, > Sean> which I don't believe will work across a router. Does an > Sean> app even know enough to be able to get a path record? > > I think you're fine. The IB router jus

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Roland Dreier
Sean> We currently use ARP to resolve an IP address to a DGID, Sean> which I don't believe will work across a router. Does an Sean> app even know enough to be able to get a path record? I think you're fine. The IB router just has to handle forwarding multicasts between two IB subnets

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Sean Hefty
Hal Rosenstock wrote: Anytime the send is off the local subnet (as well as multicast), a GRH is required. Also, there is a management response rule for responding when the request contained a GRH that require a GRH (13.5.4.4 p. 769). Reading through the responses, I think my problems are worse.

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Jason Gunthorpe
On Thu, May 11, 2006 at 07:20:19AM -0400, Hal Rosenstock wrote: > That would be a simpler check but HopLimit is not a required component > of PathRecord but I think this may not be sufficient as just because a > HopLimit >= 2 doesn't mean that a packet would be forwarded off subnet. I was thinkin

RE: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Eitan Zahavi
; From: [EMAIL PROTECTED] [mailto:openib-general- > [EMAIL PROTECTED] On Behalf Of Hal Rosenstock > Sent: Thursday, May 11, 2006 2:20 PM > To: Jason Gunthorpe > Cc: Roland Dreier; openib-general@openib.org > Subject: Re: [openib-general] question regarding GRH flag in ib_ah_attr >

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Hal Rosenstock
On Thu, 2006-05-11 at 01:48, Jason Gunthorpe wrote: > On Wed, May 10, 2006 at 09:56:58PM -0700, Roland Dreier wrote: > > Hal> Huh ? In this case, aren't the subnet prefixes are required > > Hal> to be different ? > > > > It's kind of a crazy thing to do but I don't see anything in the IB >

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Hal Rosenstock
On Thu, 2006-05-11 at 00:56, Roland Dreier wrote: > Hal> Huh ? In this case, aren't the subnet prefixes are required > Hal> to be different ? > > It's kind of a crazy thing to do but I don't see anything in the IB > spec that forbids two subnets with the same subnet prefix, There's errata

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Jason Gunthorpe
On Wed, May 10, 2006 at 09:56:58PM -0700, Roland Dreier wrote: > Hal> Huh ? In this case, aren't the subnet prefixes are required > Hal> to be different ? > > It's kind of a crazy thing to do but I don't see anything in the IB > spec that forbids two subnets with the same subnet prefix, or

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Roland Dreier
Hal> What you are describing is similar to a NAT function for IB Hal> which would need to be supported in the IB edge router to Hal> that private network. Why does there have to be any NAT? The router would just have to replace the DLID the same as it usually does. I don't see why th

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Roland Dreier
Hal> Huh ? In this case, aren't the subnet prefixes are required Hal> to be different ? It's kind of a crazy thing to do but I don't see anything in the IB spec that forbids two subnets with the same subnet prefix, or any reason why a router couldn't route between them. The SMs would just

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Hal Rosenstock
On Wed, 2006-05-10 at 21:26, Hal Rosenstock wrote: > On Wed, 2006-05-10 at 19:44, Roland Dreier wrote: > > Sean> Does anyone know how the user determines if the grh flag > > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > > Sean> Do they do this by examining the GIDs i

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Hal Rosenstock
On Wed, 2006-05-10 at 19:35, Sean Hefty wrote: > For context, I'm trying to work backwards from send a message on a UD QP to > determine what information is needed and how it is obtained. > > Does anyone know how the user determines if the grh flag should be set in the > ib_ah_attr when allocating

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Hal Rosenstock
On Wed, 2006-05-10 at 19:44, Roland Dreier wrote: > Sean> Does anyone know how the user determines if the grh flag > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > Sean> Do they do this by examining the GIDs in a path record? > > Good question. It's always needed fo

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Jason Gunthorpe
On Wed, May 10, 2006 at 04:44:42PM -0700, Roland Dreier wrote: > Sean> Does anyone know how the user determines if the grh flag > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > Sean> Do they do this by examining the GIDs in a path record? > > Good question. It's alw

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Roland Dreier
Sean> Does anyone know how the user determines if the grh flag Sean> should be set in the ib_ah_attr when allocating an ib_ah? Sean> Do they do this by examining the GIDs in a path record? Good question. It's always needed for multicast, of course. For unicast, I guess one could look

[openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Sean Hefty
For context, I'm trying to work backwards from send a message on a UD QP to determine what information is needed and how it is obtained. Does anyone know how the user determines if the grh flag should be set in the ib_ah_attr when allocating an ib_ah? Do they do this by examining the GIDs in a pa

[openib-general] question on rdma_disconnect

2006-05-08 Thread Or Gerlitz
Resending, this time with a subject line. Or. -- Forwarded message -- Date: Mon, 8 May 2006 18:01:51 +0300 (IDT) From: Or Gerlitz <[EMAIL PROTECTED]> To: [EMAIL PROTECTED] Cc: openib-general@openib.org Hi Sean, Looking in the code i have realized that it is a must for the CMA c

Re: [openib-general] Question on : ib_reg_phys_mr()

2006-04-10 Thread James Lentini
On Sat, 8 Apr 2006, Devesh Sharma wrote: > In your nfs-rdma context what this function is supposed to do? It should create a memory region for the specified address range. For the exact semantics, see the IBTA spec's description of the REGISTER PHYSICAL MEMORY REGION verb (section 11.2.8.3 of

Re: [openib-general] Question on : ib_reg_phys_mr()

2006-04-07 Thread Devesh Sharma
Thanks James for quick reply,   In your nfs-rdma context what this function is supposed to do? I know that this function returns memory region, but what is the difference from other mr returning functions?why get_dma_mr can't be used?   Devesh  On 4/7/06, James Lentini <[EMAIL PROTECTED]> wrote: On

Re: [openib-general] Question on : ib_reg_phys_mr()

2006-04-07 Thread James Lentini
On Fri, 7 Apr 2006, Devesh Sharma wrote: > Hello list, > In Ib kernel verbs there is a function ib_reg_phys_mr(). > I am not able to trace the call of this verb by any ulp or uverb. > Who calls this function? NFS-RDMA uses this function: http://sourceforge.net/projects/nfs-rdma > Is this func

[openib-general] Question on : ib_reg_phys_mr()

2006-04-07 Thread Devesh Sharma
Hello list, In Ib kernel verbs there is a function ib_reg_phys_mr(). I am not able to trace the call of this verb by any ulp or uverb. Who calls this function? Is this function mendatory to be supported by the HCA driver provider? please guide me. Devesh

Re: [openib-general] Question on get_dma_mr()

2006-04-04 Thread Devesh Sharma
Hi list and Roland, Is this verb (ib_get_dma_mr) is equivalent to the verb explained in the section 11.2.8.1 Allocate L_key?On 3/30/06, Steve Wise <[EMAIL PROTECTED]> wrote: On Wed, 2006-03-29 at 20:35 -0800, Roland Dreier wrote:> Devesh> Here I am saying that assigning Key is sufficient Or th

Re: [openib-general] Question on get_dma_mr()

2006-03-30 Thread Steve Wise
On Wed, 2006-03-29 at 20:35 -0800, Roland Dreier wrote: > Devesh> Here I am saying that assigning Key is sufficient Or there > Devesh> are some other specific setps to be taken? > > It would depend on the device. You can look at the mthca, ipath and ehca > drivers' implementation of get_d

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Devesh Sharma
yha Ok Thanks for replying Once again. DeveshOn 3/30/06, Roland Dreier <[EMAIL PROTECTED]> wrote: Devesh> Here I am saying that assigning Key is sufficient Or thereDevesh> are some other specific setps to be taken?It would depend on the device.  You can look at the mthca, ipath and ehcadri

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Roland Dreier
Devesh> Here I am saying that assigning Key is sufficient Or there Devesh> are some other specific setps to be taken? It would depend on the device. You can look at the mthca, ipath and ehca drivers' implementation of get_dma_mr() for examples. - R. _

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Devesh Sharma
On 3/29/06, Roland Dreier <[EMAIL PROTECTED]> wrote: Devesh> S/G entry ?scatter gather entryDevesh> What is the size of this region ? is there any limitationDevesh> in providing this size?It must be large enough to cover all DMA (bus) addresses for the device. Devesh> Finally you me

  1   2   3   >