Re: [openib-general] question - mapping QPIDs back to ptrs

2006-11-17 Thread Roland Dreier
> The Chelsio driver is hogging lots of memory right now for mapping > PDIDs, QPIDs, CQIDs, and STAG IDs back to their respective kernel > structures. This is done via an array of pointers, indexed by the ID. > The critical performance mapping is finding a QP struct from the QPID in > the pol

Re: [openib-general] Question about the query QP mask

2006-11-16 Thread Roland Dreier
> What should be the expected behavior? > Should this description should be changed or should the low level drivers > of mthca and ipath need to be changed? The mask is used as a hint to the low-level driver about which attributes the consumer cares about. The driver may fill in more fields, b

Re: [openib-general] Question about multicast GIDs

2006-11-16 Thread Robert Walsh
Robert Walsh wrote: > Roland Dreier wrote: >> > Is there are registration authority for multicast GIDs? Or at >> least a > safe way of assigning a range of GIDs to a vendor? >> >> I don't think so. Perhaps RFC 3307 would be of some use... > > Ah - looks exactly like what I was looking for. T

Re: [openib-general] Question about multicast GIDs

2006-11-15 Thread Robert Walsh
Roland Dreier wrote: > > Is there are registration authority for multicast GIDs? Or at least a > > safe way of assigning a range of GIDs to a vendor? > > I don't think so. Perhaps RFC 3307 would be of some use... Ah - looks exactly like what I was looking for. Thanks. _

Re: [openib-general] Question about multicast GIDs

2006-11-15 Thread Roland Dreier
> Is there are registration authority for multicast GIDs? Or at least a > safe way of assigning a range of GIDs to a vendor? I don't think so. Perhaps RFC 3307 would be of some use... - R. ___ openib-general mailing list openib-general@openib.org

Re: [openib-general] question on QoS support

2006-11-06 Thread Hal Rosenstock
hysErr:8 > OverrunErr:..8 > MaxCreditHint:...0 > RoundTrip:...0 Do you have an IB analyzer ? -- Hal > Feiyi > > -Original Message- > From: Hal Rosenstock [mailto:[EMAIL PROTECTED] > Sent:

Re: [openib-general] question on QoS support

2006-11-06 Thread Wang, Feiyi
:...0 RoundTrip:...0 Feiyi -Original Message- From: Hal Rosenstock [mailto:[EMAIL PROTECTED] Sent: Friday, November 03, 2006 3:58 PM To: Wang, Feiyi Cc: openib-general@openib.org Subject: RE: [openib-general] question on QoS support On Fri, 2006-11-03 at

Re: [openib-general] question on QoS support

2006-11-03 Thread Hal Rosenstock
o:[EMAIL PROTECTED] > Sent: Friday, November 03, 2006 3:51 PM > To: Wang, Feiyi > Cc: openib-general@openib.org > Subject: RE: [openib-general] question on QoS support > > On Fri, 2006-11-03 at 15:43, Wang, Feiyi wrote: > > The test is done on two hosts, say A and B. A has 4x S

Re: [openib-general] question on QoS support

2006-11-03 Thread Wang, Feiyi
er 03, 2006 3:51 PM To: Wang, Feiyi Cc: openib-general@openib.org Subject: RE: [openib-general] question on QoS support On Fri, 2006-11-03 at 15:43, Wang, Feiyi wrote: > The test is done on two hosts, say A and B. A has 4x SDR (run ib_rdam_bw > as server), B has 4x DDR (run more than one thread of

Re: [openib-general] question on QoS support

2006-11-03 Thread Hal Rosenstock
On Fri, 2006-11-03 at 15:12, Feiyi Wang wrote: > In our test at the ORNL - it appears you can "turn off" the traffic by > giving every VL weight 0. A weight of 0 indicates to skip that entry. > As soon as you assign non-zero VL weight, > the traffic starts to flow, however, VL with more weight d

Re: [openib-general] question on QoS support

2006-11-03 Thread Feiyi Wang
In our test at the ORNL - it appears you can "turn off" the traffic by giving every VL weight 0. As soon as you assign non-zero VL weight, the traffic starts to flow, however, VL with more weight doesn't have expected preference treatment. In other words, traffic shaping didn't take place. smpquery

Re: [openib-general] Question on ucma

2006-11-03 Thread Steve Wise
Sean posted 7 patches that include the ucma support. You'll need those + the one librdmacm patch he posted. Steve. On Fri, 2006-11-03 at 13:59 +0530, Krishna Kumar2 wrote: > Hi, > > I installed the 2.6.19-rc3 bits, and when I try to run > perftest/rdma_bw (with '-c' option), I get the error :

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
Hi Oliver, On Thu, 2006-11-02 at 10:20, Oliver wrote: > Hi, Hal - > > > How is this being observed/measured ? > > Host A, B, with 4x DDR both connected to Flextronic switch. > A single process of ibv_read_bw gives about 1415MB /s average > bandwidth. Two concurrent process report 714.45 MB/s eac

Re: [openib-general] question on QoS support

2006-11-02 Thread Oliver
Hi, Hal - > How is this being observed/measured ? Host A, B, with 4x DDR both connected to Flextronic switch. A single process of ibv_read_bw gives about 1415MB /s average bandwidth. Two concurrent process report 714.45 MB/s each, dead even. Now if I bump up one process with a different SL, then

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
On Thu, 2006-11-02 at 09:15, Makia Minich wrote: > Hal Rosenstock wrote: > > Makia, > > > > On Wed, 2006-11-01 at 17:42, Makia Minich wrote: > >> It just so happens that we've started looking at this here at ORNL as > >> well. I had a question about the options. The manpage makes it seem > >> th

Re: [openib-general] question on QoS support

2006-11-02 Thread Makia Minich
Hal Rosenstock wrote: > Makia, > > On Wed, 2006-11-01 at 17:42, Makia Minich wrote: >> It just so happens that we've started looking at this here at ORNL as >> well. I had a question about the options. The manpage makes it seem >> that you can set these qos options (e.g. qos_high_limit) from the

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
Makia, On Wed, 2006-11-01 at 17:42, Makia Minich wrote: > It just so happens that we've started looking at this here at ORNL as > well. I had a question about the options. The manpage makes it seem > that you can set these qos options (e.g. qos_high_limit) from the > command line, but I haven't

Re: [openib-general] question on QoS support

2006-11-02 Thread Hal Rosenstock
Hi Oliver, On Wed, 2006-11-01 at 16:52, Oliver wrote: > Hi, folks - > > I am trying to verify and evaluate IB QoS support, running openSM as > subnet manager. The perftest program is extended to set SL as command > line options instead of default 0, and by modifying VL arbitration > tables, I am

Re: [openib-general] question on QoS support

2006-11-01 Thread Sasha Khapyorsky
On 17:42 Wed 01 Nov , Makia Minich wrote: > It just so happens that we've started looking at this here at ORNL as > well. I had a question about the options. The manpage makes it seem > that you can set these qos options (e.g. qos_high_limit) from the > command line, AFAIK there is option -Q

Re: [openib-general] question on QoS support

2006-11-01 Thread Makia Minich
It just so happens that we've started looking at this here at ORNL as well. I had a question about the options. The manpage makes it seem that you can set these qos options (e.g. qos_high_limit) from the command line, but I haven't been overly successful. Is there an example of this being done?

Re: [openib-general] question on QoS support

2006-11-01 Thread Sasha Khapyorsky
On 16:52 Wed 01 Nov , Oliver wrote: > Hi, folks - > > I am trying to verify and evaluate IB QoS support, running openSM as > subnet manager. The perftest program is extended to set SL as command > line options instead of default 0, and by modifying VL arbitration > tables, I am expecting to se

Re: [openib-general] Question about ehca CQ handling

2006-10-02 Thread Christoph Raisch
> While looking over the ehca driver from the perspective of adding a > "peek CQ" operation, I noticed some code that looked funny. > > In hipz_set_cqx_n0() and hipz_set_cqx_n1(), what is the point of the > calls to hipz_galpa_load_cq()? The return value is discarded. I see > that hipz_galpa_load

Re: [openib-general] Question about interrupt generation

2006-09-05 Thread harish
Hi,One more question. What kind of event mask helps mask the interrupts?thanksharishOn 9/5/06, harish < [EMAIL PROTECTED]> wrote:Hi All,I tried the following simple experiment and am not able to understand the results: Calcualted the number of interrupts  generated by the infiniband [with little or

Re: [openib-general] question: ib_umem page_size

2006-08-23 Thread Roland Dreier
> > It gives the page size for the user memory described by the struct. > > The idea was that if/when someone tries to optimize for huge pages, > > then the low-level driver can know that a region is using huge pages > > without having to walk through the page list and search for the > > minim

Re: [openib-general] question: ib_umem page_size

2006-08-23 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: question: ib_umem page_size > > Michael> Roland, could you please clarify what does the page_size > Michael> field in struct ib_mem do? > > It gives the page size for the user memory described by the struct. > The idea was that

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-22 Thread Sean Hefty
>Cool, I would go for XOR-ing a random value with the **local id** . > >Sean, my understanding it can be narrowed for doing so in: > >1) cm_alloc_id() after calling idr_get_new_above() >2) cm_free_id() before calling idr_remove() >3) cm_get_id() before calling idr_find() > >and initializing the ran

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-22 Thread Or Gerlitz
Sean Hefty wrote: > When a new REQ is received, we enter its timewait structure into two trees: one > sorted by remote ID, one sorted by remote QPN. If the REQ is new, both would > succeed, and timewait_info would be NULL. Since timewait_info is not NULL, we > are dealing with a REQ that re-us

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-22 Thread Or Gerlitz
Roland Dreier wrote: > Sean> If we record a base offset, we can start at any random > Sean> number. We just need to always add/subtract the base when > Sean> getting a value from the IDR. > > Good point -- or better still, we could XOR in a random bit pattern. > That way we don't have

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Roland Dreier
Sean> If we record a base offset, we can start at any random Sean> number. We just need to always add/subtract the base when Sean> getting a value from the IDR. Good point -- or better still, we could XOR in a random bit pattern. That way we don't have to keep straight when to add and

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Sean Hefty
>> If we get here, this means that the REQ was a new REQ and not a >> duplicate, but the remote_id or remote_qpn is already in use. We need >> to reject the new REQ as containing stale data. > >I don't follow, if we get to the else case its as of cm_get_id() >returning NULL. This holds when idr_fi

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Sean Hefty
>> Just to emphasize what Sean has pointed out, you are asking how can a CM >> consumer know that a **local** QPN is not in the timewait state >> according to the **remote** CM. Since the issue is with the remote CM, >> it seems to me that pushing down timewait into verbs is not the correct >> dire

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Sean Hefty
>How about (for the meantime, till this rework is designed && done) going >to projecting the initial random local id into the range of (say) >[0-1022] (i think 1023 is prime, if not choose a prime near it) this way >with very good probability and with very little overhead on memory >consumption a c

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Roland Dreier
Or> How about (for the meantime, till this rework is designed && Or> done) going to projecting the initial random local id into the Or> range of (say) [0-1022] (i think 1023 is prime, if not choose Or> a prime near it) this way with very good probability and with Or> very little

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
This email appear in the archive, but seems not to be distributed to the subscribers so i am reposting it. Or Gerlitz wrote: > Sean Hefty wrote: >> Even if we pushed timewait handling under verbs, a user could always >> get a QP that the remote side thinks is connected. The original >> connec

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
This email appear in the archive, but seems not to be distributed to the subscribers so i am reposting it. Or Gerlitz wrote: > Arlin Davis wrote: >> We are running into connection reject issues (IB_CM_REJ_STALE_CONN) >> with our application under heavy load and lots of connections. >> >> We occa

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
>>> + } else >>> + cm_issue_rej(work->port, work->mad_recv_wc, >>> +IB_CM_REJ_STALE_CONN, >>> CM_MSG_RESPONSE_REQ, >>> +NULL, 0); >> >> >> what is this case? there is no entry but there is

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-20 Thread Or Gerlitz
Sean Hefty wrote: > Or Gerlitz wrote: >> If you don't mind (also related to the patch you have sent Eric of >> randomizing the initial local cm id) to get into this deeper, can we do > There's an issue trying to randomize the initial local CM ID. The way > the IDR works, if you start at a hig

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-17 Thread Sean Hefty
Or Gerlitz wrote: > If you don't mind (also related to the patch you have sent Eric of > randomizing the initial local cm id) to get into this deeper, can we do There's an issue trying to randomize the initial local CM ID. The way the IDR works, if you start at a high value, then the IDR size

Re: [openib-general] Question about QP's in timewait state and CM stale conn rejects

2006-08-16 Thread Sean Hefty
Arlin Davis wrote: > How can a consumer know for sure that the new QP will not be in a > timewait state according to the CM? Given that the QP may have been in use by another process, I don't think that there's any way for the new owner to know. > Does it make sense to push the timewait functio

Re: [openib-general] question: ib_umem page_size

2006-08-15 Thread Michael S. Tsirkin
Quoting r. Roland Dreier <[EMAIL PROTECTED]>: > Subject: Re: question: ib_umem page_size > > Michael> Roland, could you please clarify what does the page_size > Michael> field in struct ib_mem do? > > It gives the page size for the user memory described by the struct. > The idea was that

Re: [openib-general] question: ib_umem page_size

2006-08-15 Thread Roland Dreier
Michael> Roland, could you please clarify what does the page_size Michael> field in struct ib_mem do? It gives the page size for the user memory described by the struct. The idea was that if/when someone tries to optimize for huge pages, then the low-level driver can know that a region is

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Parks Fields
At 12:36 PM 6/5/2006, Talpey, Thomas wrote: Thanks Parks, this is a very interesting perspective. I will avoid going into my rant about edge devices for now, however. :-) Cool, you can send it direct if you want. I am not sure what you mean about using SDP "end to end". I assume you would pe

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
Thanks Parks, this is a very interesting perspective. I will avoid going into my rant about edge devices for now, however. :-) I am not sure what you mean about using SDP "end to end". I assume you would perhaps use SDP to these edge nodes, but this would require terminating the SDP connection and

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Parks Fields
I consider IPoIB to be Ethernet emulation. As for apples and oranges, my point exactly. It is not really about comparisons. Here at LANL we have an environment where all our new Clusters have to mount our global parallel file system Panasas. It is ethernet and will be for a while. Clust

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
bout the IPoIB bandwidth performance ? > (Talpey, Thomas) > 8. Re: Question about the IPoIB bandwidth performance ? (hbchen) > >----- Message from "hbchen" <[EMAIL PROTECTED]> on Mon, 05 Jun 2006 09:38:24 >-0600 - > >

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hycsw
Tom, We are in the process of measuring the CPU utilization on our NFS/RDMA experiments in contrast with regular the NFS, we also intend to include netperf numbers and will keep you posted with our results as soon as possible. Helen - original Message - >From [EMAIL PROTECTED] Mon Jun

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Bernard King-Smith
IL PROTECTED]> on Mon, 05 Jun 2006 09:38:24 -0600 - To: "Hal Rosenstock" <[EMAIL PROTECTED]> cc: "OPENIB"

RE: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Felix Marti
    From: [EMAIL PROTECTED] [mailto:[EMAIL PROTECTED] On Behalf Of hbchen Sent: Monday, June 05, 2006 9:12 AM To: Talpey, Thomas Cc: openib-general@openib.org Subject: Re: [openib-general] Question about the IPoIB bandwidth performance ?   Talpey, Thomas wrote: At 11:38

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
At 12:11 PM 6/5/2006, hbchen wrote: >>Perhaps you are out of CPU. >> >> >Tom, >I am HB Chen from LANL not the Helen Chen from SNL. Oops, sorry! I have too many email messages going by. :-) HB, then. >I didn't run out of CPU. It is about 70-80 % of CPU utilization. But, is one CPU at 100%? In

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hbchen
Talpey, Thomas wrote: At 11:38 AM 6/5/2006, hbchen wrote: Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth utilization is still very low. IPoIB=420MB/sec bandwidth utilization= 420/1024 = 41.01% Helen, have you

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Talpey, Thomas
At 11:38 AM 6/5/2006, hbchen wrote: >Even with this IB-4X = 8Gb/sec = 1024 MB/sec the IPoIB bandwidth utilization >is still very low. >>> IPoIB=420MB/sec >>> bandwidth utilization= 420/1024 = 41.01% Helen, have you measured the CPU utilizations during these runs? Perhaps you are out of CPU. O

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Bernard King-Smith
Hal Rosenstock wrote: > On Mon, 2006-06-05 at 11:12, hbchen wrote: > > Hi, > > I have a question about the IPoIB bandwidth performance. > > I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G > > ethernet card, > > and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). > > >

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread hbchen
Hal Rosenstock wrote: On Mon, 2006-06-05 at 11:12, hbchen wrote: Hi, I have a question about the IPoIB bandwidth performance. I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G ethernet card, and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). NIC (Jumb

Re: [openib-general] Question about the IPoIB bandwidth performance ?

2006-06-05 Thread Hal Rosenstock
On Mon, 2006-06-05 at 11:12, hbchen wrote: > Hi, > I have a question about the IPoIB bandwidth performance. > I did netperf testing using Single GiGE, Myrinet D card, Myrinet 10G > ethernet card, > and Voltaire Infiniband 4X HCA400Ex (PCI-Express interface). > > > NIC (Jumbo enabled) Line bandwid

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Hal Rosenstock
On Sun, 2006-05-14 at 15:30, Jason Gunthorpe wrote: > On Sun, May 14, 2006 at 07:40:25AM -0400, Hal Rosenstock wrote: > > > > Not always true in terms of local subnet (multicast and management MAD > > > > response exceptions). > > > > > > Yes, but these are well specified. Multicast must always ha

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Jason Gunthorpe
On Sun, May 14, 2006 at 07:40:25AM -0400, Hal Rosenstock wrote: > > > Not always true in terms of local subnet (multicast and management MAD > > > response exceptions). > > > > Yes, but these are well specified. Multicast must always have a GRH. > > MAD requests are covered under my scenario above

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Hal Rosenstock
On Fri, 2006-05-12 at 13:55, Sean Hefty wrote: > Jason Gunthorpe wrote: > > How about this, how do you see this scenario: > > > > 1) Client gets a DGID from 'someplace' > > 2) Client sends a SA query to resolve the DGID to a Path Record > > 3) Client configures a QP based on the Path Record > > >

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Hal Rosenstock
On Fri, 2006-05-12 at 13:10, Jason Gunthorpe wrote: > On Fri, May 12, 2006 at 08:11:17AM -0400, Hal Rosenstock wrote: > > > > To allow what Roland is talking about you need an unambiguous > > > mechanism where the SA can signal to the client that the path > > > needs a GRH. > > > > Ah, you are re

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Sean Hefty
Jason Gunthorpe wrote: How about this, how do you see this scenario: 1) Client gets a DGID from 'someplace' 2) Client sends a SA query to resolve the DGID to a Path Record 3) Client configures a QP based on the Path Record Now, the question I'm interested in is this: During step #3 what test

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Jason Gunthorpe
On Fri, May 12, 2006 at 08:11:17AM -0400, Hal Rosenstock wrote: > > To allow what Roland is talking about you need an unambiguous > > mechanism where the SA can signal to the client that the path > > needs a GRH. > > Ah, you are referring to the SA path record response not the request. Yes.. Tho

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Hal Rosenstock
On Thu, 2006-05-11 at 13:12, Jason Gunthorpe wrote: > On Thu, May 11, 2006 at 07:20:19AM -0400, Hal Rosenstock wrote: > > > That would be a simpler check but HopLimit is not a required component > > of PathRecord but I think this may not be sufficient as just because a > > HopLimit >= 2 doesn't me

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Jason Gunthorpe
On Thu, May 11, 2006 at 10:21:08AM -0700, Sean Hefty wrote: > Hal Rosenstock wrote: > >Anytime the send is off the local subnet (as well as multicast), a GRH > >is required. Also, there is a management response rule for responding > >when the request contained a GRH that require a GRH (13.5.4.4 p.

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Hal Rosenstock
On Thu, 2006-05-11 at 13:29, Roland Dreier wrote: > Sean> We currently use ARP to resolve an IP address to a DGID, > Sean> which I don't believe will work across a router. Does an > Sean> app even know enough to be able to get a path record? > > I think you're fine. The IB router jus

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Roland Dreier
Sean> We currently use ARP to resolve an IP address to a DGID, Sean> which I don't believe will work across a router. Does an Sean> app even know enough to be able to get a path record? I think you're fine. The IB router just has to handle forwarding multicasts between two IB subnets

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Sean Hefty
Hal Rosenstock wrote: Anytime the send is off the local subnet (as well as multicast), a GRH is required. Also, there is a management response rule for responding when the request contained a GRH that require a GRH (13.5.4.4 p. 769). Reading through the responses, I think my problems are worse.

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Jason Gunthorpe
On Thu, May 11, 2006 at 07:20:19AM -0400, Hal Rosenstock wrote: > That would be a simpler check but HopLimit is not a required component > of PathRecord but I think this may not be sufficient as just because a > HopLimit >= 2 doesn't mean that a packet would be forwarded off subnet. I was thinkin

RE: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Eitan Zahavi
; From: [EMAIL PROTECTED] [mailto:openib-general- > [EMAIL PROTECTED] On Behalf Of Hal Rosenstock > Sent: Thursday, May 11, 2006 2:20 PM > To: Jason Gunthorpe > Cc: Roland Dreier; openib-general@openib.org > Subject: Re: [openib-general] question regarding GRH flag in ib_ah_attr >

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Hal Rosenstock
On Thu, 2006-05-11 at 01:48, Jason Gunthorpe wrote: > On Wed, May 10, 2006 at 09:56:58PM -0700, Roland Dreier wrote: > > Hal> Huh ? In this case, aren't the subnet prefixes are required > > Hal> to be different ? > > > > It's kind of a crazy thing to do but I don't see anything in the IB >

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Hal Rosenstock
On Thu, 2006-05-11 at 00:56, Roland Dreier wrote: > Hal> Huh ? In this case, aren't the subnet prefixes are required > Hal> to be different ? > > It's kind of a crazy thing to do but I don't see anything in the IB > spec that forbids two subnets with the same subnet prefix, There's errata

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Jason Gunthorpe
On Wed, May 10, 2006 at 09:56:58PM -0700, Roland Dreier wrote: > Hal> Huh ? In this case, aren't the subnet prefixes are required > Hal> to be different ? > > It's kind of a crazy thing to do but I don't see anything in the IB > spec that forbids two subnets with the same subnet prefix, or

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Roland Dreier
Hal> What you are describing is similar to a NAT function for IB Hal> which would need to be supported in the IB edge router to Hal> that private network. Why does there have to be any NAT? The router would just have to replace the DLID the same as it usually does. I don't see why th

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Roland Dreier
Hal> Huh ? In this case, aren't the subnet prefixes are required Hal> to be different ? It's kind of a crazy thing to do but I don't see anything in the IB spec that forbids two subnets with the same subnet prefix, or any reason why a router couldn't route between them. The SMs would just

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Hal Rosenstock
On Wed, 2006-05-10 at 21:26, Hal Rosenstock wrote: > On Wed, 2006-05-10 at 19:44, Roland Dreier wrote: > > Sean> Does anyone know how the user determines if the grh flag > > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > > Sean> Do they do this by examining the GIDs i

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Hal Rosenstock
On Wed, 2006-05-10 at 19:35, Sean Hefty wrote: > For context, I'm trying to work backwards from send a message on a UD QP to > determine what information is needed and how it is obtained. > > Does anyone know how the user determines if the grh flag should be set in the > ib_ah_attr when allocating

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Hal Rosenstock
On Wed, 2006-05-10 at 19:44, Roland Dreier wrote: > Sean> Does anyone know how the user determines if the grh flag > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > Sean> Do they do this by examining the GIDs in a path record? > > Good question. It's always needed fo

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Jason Gunthorpe
On Wed, May 10, 2006 at 04:44:42PM -0700, Roland Dreier wrote: > Sean> Does anyone know how the user determines if the grh flag > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > Sean> Do they do this by examining the GIDs in a path record? > > Good question. It's alw

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Roland Dreier
Sean> Does anyone know how the user determines if the grh flag Sean> should be set in the ib_ah_attr when allocating an ib_ah? Sean> Do they do this by examining the GIDs in a path record? Good question. It's always needed for multicast, of course. For unicast, I guess one could look

Re: [openib-general] Question on : ib_reg_phys_mr()

2006-04-10 Thread James Lentini
On Sat, 8 Apr 2006, Devesh Sharma wrote: > In your nfs-rdma context what this function is supposed to do? It should create a memory region for the specified address range. For the exact semantics, see the IBTA spec's description of the REGISTER PHYSICAL MEMORY REGION verb (section 11.2.8.3 of

Re: [openib-general] Question on : ib_reg_phys_mr()

2006-04-07 Thread Devesh Sharma
Thanks James for quick reply,   In your nfs-rdma context what this function is supposed to do? I know that this function returns memory region, but what is the difference from other mr returning functions?why get_dma_mr can't be used?   Devesh  On 4/7/06, James Lentini <[EMAIL PROTECTED]> wrote: On

Re: [openib-general] Question on : ib_reg_phys_mr()

2006-04-07 Thread James Lentini
On Fri, 7 Apr 2006, Devesh Sharma wrote: > Hello list, > In Ib kernel verbs there is a function ib_reg_phys_mr(). > I am not able to trace the call of this verb by any ulp or uverb. > Who calls this function? NFS-RDMA uses this function: http://sourceforge.net/projects/nfs-rdma > Is this func

Re: [openib-general] Question on get_dma_mr()

2006-04-04 Thread Devesh Sharma
Hi list and Roland, Is this verb (ib_get_dma_mr) is equivalent to the verb explained in the section 11.2.8.1 Allocate L_key?On 3/30/06, Steve Wise <[EMAIL PROTECTED]> wrote: On Wed, 2006-03-29 at 20:35 -0800, Roland Dreier wrote:> Devesh> Here I am saying that assigning Key is sufficient Or th

Re: [openib-general] Question on get_dma_mr()

2006-03-30 Thread Steve Wise
On Wed, 2006-03-29 at 20:35 -0800, Roland Dreier wrote: > Devesh> Here I am saying that assigning Key is sufficient Or there > Devesh> are some other specific setps to be taken? > > It would depend on the device. You can look at the mthca, ipath and ehca > drivers' implementation of get_d

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Devesh Sharma
yha Ok Thanks for replying Once again. DeveshOn 3/30/06, Roland Dreier <[EMAIL PROTECTED]> wrote: Devesh> Here I am saying that assigning Key is sufficient Or thereDevesh> are some other specific setps to be taken?It would depend on the device.  You can look at the mthca, ipath and ehcadri

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Roland Dreier
Devesh> Here I am saying that assigning Key is sufficient Or there Devesh> are some other specific setps to be taken? It would depend on the device. You can look at the mthca, ipath and ehca drivers' implementation of get_dma_mr() for examples. - R. _

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Devesh Sharma
On 3/29/06, Roland Dreier <[EMAIL PROTECTED]> wrote: Devesh> S/G entry ?scatter gather entryDevesh> What is the size of this region ? is there any limitationDevesh> in providing this size?It must be large enough to cover all DMA (bus) addresses for the device. Devesh> Finally you me

Re: [openib-general] Question on get_dma_mr()

2006-03-29 Thread Roland Dreier
Devesh> S/G entry ? scatter gather entry Devesh> What is the size of this region ? is there any limitation Devesh> in providing this size? It must be large enough to cover all DMA (bus) addresses for the device. Devesh> Finally you mean to say in the implementation of this D

Re: [openib-general] Question on get_dma_mr()

2006-03-28 Thread Devesh Sharma
Thanks to all  of you On 3/27/06, Roland Dreier <[EMAIL PROTECTED]> wrote: Devesh> Hello all, Please any body explain me about theDevesh> functionality of verbs ib_get_dma_mr()?Actually, the responses you've gotten are not quite right.ib_get_dma_mr() returns a memory region that can be used

Re: [openib-general] question related to rdma_bind_addr

2006-03-27 Thread James Lentini
On Sun, 26 Mar 2006, Or Gerlitz wrote: > > I would find calling it rdma_bind_device() confusing. > > why? I find it very much unconfusing I associate the word bind with bind(2). For that reason, rdma_bind_addr() is a good name because it is the CMA's analog for bind(2). Since it isn't relat

Re: [openib-general] Question on get_dma_mr()

2006-03-26 Thread Roland Dreier
Devesh> Hello all, Please any body explain me about the Devesh> functionality of verbs ib_get_dma_mr()? Actually, the responses you've gotten are not quite right. ib_get_dma_mr() returns a memory region that can be used for any _bus_ addresses. In other words, if an S/G entry is passed to

Re: [openib-general] question related to rdma_bind_addr

2006-03-26 Thread Or Gerlitz
James Lentini wrote: On Thu, 23 Mar 2006, Sean Hefty wrote: I think that Or is just exploring the idea of synchronously binding to a local *device* based on a remote address. This would allow an application to bind, then allocate PDs, CQs, QPs, etc. up front, rather than deferring resource al

Re: [openib-general] Question on get_dma_mr()

2006-03-24 Thread Sean Hefty
Devesh Sharma wrote: Please any body explain me about the functionality of verbs ib_get_dma_mr()? What is the need of this function? what a driver implementer is supposed to implement in this function? This function returns a memory region for all of system memory. See mthca_provider.c for a

RE: [openib-general] question related to rdma_bind_addr

2006-03-24 Thread James Lentini
On Thu, 23 Mar 2006, Sean Hefty wrote: > >What does it mean to bind to a remote address? What functionality > >would that enable? Spoofing? > > I think that Or is just exploring the idea of synchronously binding > to a local *device* based on a remote address. > > This would allow an applica

Re: [openib-general] Question on get_dma_mr()

2006-03-24 Thread Steve Wise
On Fri, 2006-03-24 at 12:51 +0530, Devesh Sharma wrote: > Hello all, > > Please any body explain me about the functionality of verbs > ib_get_dma_mr()? > What is the need of this function? > what a driver implementer is supposed to implement in this function? It returns a MR that maps all of phy

RE: [openib-general] question related to rdma_bind_addr

2006-03-23 Thread Sean Hefty
>What does it mean to bind to a remote address? What functionality >would that enable? Spoofing? I think that Or is just exploring the idea of synchronously binding to a local *device* based on a remote address. This would allow an application to bind, then allocate PDs, CQs, QPs, etc. up front,

RE: [openib-general] question related to rdma_bind_addr

2006-03-23 Thread James Lentini
On Thu, 23 Mar 2006, Sean Hefty wrote: > >I could not approve my assumptions from looking on the cma/addr > >code, but if i am correct this opens the door for future > >enhancement of rdma_bind_addr() to work on non local addresses. > > I believe that could be the case. What does it mean to b

Re: [openib-general] question related to rdma_bind_addr

2006-03-23 Thread Or Gerlitz
Sean Hefty wrote: If my understanding is correct, the current code of rdma_bind_addr assumes you supply it one of three (all are **src** address) +1 ANY (0.0.0.0) addr +2 local loopback addr +3 other local addr Correct. - Note that currently a valid port number needs to be provided, but this

Re: [openib-general] question related to rdma_bind_addr

2006-03-22 Thread Or Gerlitz
Or Gerlitz wrote: At this point i see an actual need, it just related to some change we discuss in the open scsi model for iser integration, and i wanted to make sure that currently creating the IB resources in synchronous manner is impossible. Sorry, my fingers are broken today... I meant to

Re: [openib-general] Question On mad.c

2006-01-20 Thread Sean Hefty
Devesh Sharma wrote: In mad.c while calling ib_post_receive() operation spin_lock_irqsave(&recv_queue->lock, flags); post = (++recv_queue->count < recv_queue->max_active); list_add_tail(&mad_priv->header.mad_list.list, &recv_queue->list); spin_unlock_irqrestore(&recv

Re: [openib-general] Question about QP access flags (struct ib_qp_attr.qp_access_flags)

2005-11-08 Thread Sean Hefty
Ralph Campbell wrote: Ralph> When ib_modify_qp() is called with the IB_QP_ACCESS_FLAGS Ralph> set in the mask, what values should be used in struct Ralph> ib_qp_attr.qp_access_flags? The IB spec. seems to indicate Ralph> that RDMA and atomic operations are all enabled or disabled

Re: [openib-general] Question about locked pages

2005-10-31 Thread Roland Dreier
Jeff> Ditto (I thought those were shmem values / didn't think they Jeff> had any effect on Open IB). The information that I got was Jeff> third-hand, which is why I posted here to ask about it. :-) Jeff> I'll remove them from the FAQ entry -- any other comments? Well, a normal u

  1   2   >