Re: [openib-general] [PATCH] libmthca: optimize calls to htonl with constant parameter

2007-02-22 Thread Jason Gunthorpe
On Thu, Feb 22, 2007 at 10:09:28PM -0800, Roland Dreier wrote: > (BTW, one thing I did notice while looking at the i386 assembly is > that one micro-optimization that might make sense to use something > like __attribute__((regparm(3))) for internal function calls within > libibverbs and libmthca o

Re: [openib-general] IB routing discussion summary

2007-02-15 Thread Jason Gunthorpe
On Thu, Feb 15, 2007 at 11:39:49AM -0800, Sean Hefty wrote: > I think this may allow establishing inter-subnet connections. As an > example of its usage: I think you are right, this does contain enough information. > a. Active side issues a PathRecord query to the local SA with SGID=local, > D

Re: [openib-general] Problem is routing CM REQ

2007-02-12 Thread Jason Gunthorpe
On Mon, Feb 12, 2007 at 04:45:33PM -0800, Sean Hefty wrote: > >>4. A PR from the local SA with reversible=1 indicates that data sent from > >>the remote GID to the local GID using the PR TC and FL will route locally > >>using the specified LID pair. This holds whether the PR SGID is local or >

Re: [openib-general] Problem is routing CM REQ

2007-02-12 Thread Jason Gunthorpe
On Mon, Feb 12, 2007 at 03:31:15PM -0800, Michael Krause wrote: > TClass is intended to communicate the end-to-end QoS desired. TClass is > then mapped to a SL that is local to each subnet. A flow label is > intended to much the same as in the IP world and is left, in essence, to > routers

Re: [openib-general] Problem is routing CM REQ

2007-02-12 Thread Jason Gunthorpe
On Mon, Feb 12, 2007 at 02:47:42PM -0800, Sean Hefty wrote: > Maybe it would help if we can agree on a set of expectations. These are > what I am thinking: > > 1. An SA should be able to respond to a valid PR query if at least one of > the GIDs in the path record is local. > > 2. The LIDs in

Re: [openib-general] Problem is routing CM REQ

2007-02-12 Thread Jason Gunthorpe
On Mon, Feb 12, 2007 at 09:23:06AM -0800, Sean Hefty wrote: > >Ah, I think I missed the key step in your scheme.. You plan to query > >the local SM for SGID=remote DGID=local? (ie reversed from 'normal'. I > >was thinking only about the SGID=local DGID=remote query direction) > > I'm not sure that

Re: [openib-general] Problem is routing CM REQ

2007-02-11 Thread Jason Gunthorpe
On Fri, Feb 09, 2007 at 06:08:34PM -0800, Sean Hefty wrote: > >So basically what you are saying is that the TClass and FlowLabel act > >as some kind of global dis-ambiguation that lets all SAs know that the > >tuple MUST be matched with > >on each side. > > Sort of... My reasoning is that if yo

Re: [openib-general] Problem is routing CM REQ

2007-02-09 Thread Jason Gunthorpe
On Fri, Feb 09, 2007 at 03:08:12PM -0800, Sean Hefty wrote: > The route itself is determined using the SGID, DGID, TClass, FlowLabel. > So, as long as the two queries match on these fields, I would think that it > would work. So basically what you are saying is that the TClass and FlowLabel ac

Re: [openib-general] Problem is routing CM REQ

2007-02-09 Thread Jason Gunthorpe
On Fri, Feb 09, 2007 at 04:45:29PM -0500, Hal Rosenstock wrote: > >Off hand I don't see that the existing path record query structure > >has enough information to do this.. Particularly, in cases > >where each subnet has more than 1 router port there is no real > >guarentee that qu

Re: [openib-general] Problem is routing CM REQ

2007-02-09 Thread Jason Gunthorpe
On Fri, Feb 09, 2007 at 12:58:51PM -0500, Hal Rosenstock wrote: > > For simplicity, assume a single path. My assumption in this case was that > > the > > SLID/DLID values would be reversed. That is, the LIDs are relative to the > > local > > subnet, not the SGID. But if I set the SGID = DGID

Re: [openib-general] Problem is routing CM REQ

2007-02-08 Thread Jason Gunthorpe
On Thu, Feb 08, 2007 at 03:43:24PM -0800, Sean Hefty wrote: > > Looking at the problem more, I think that the issue extends to the remote > > port > > LID as well. My expectation with a local path record query is that the > > SLID is > > the local port, and the DLID is the local router. This

Re: [openib-general] Problem is routing CM REQ was: Use a GRH when appropriate for unicast packets

2007-02-08 Thread Jason Gunthorpe
On Thu, Feb 08, 2007 at 10:23:11AM -0800, Sean Hefty wrote: > >>The active side clearly cannot learn what the SLID of the passive > >>side's router should be. > >> > >>We don't want to have the routers snoop and alter CM GMPs. > >> > >>The passive side cannot use information from the LRH to get the

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-02-07 Thread Jason Gunthorpe
On Wed, Feb 07, 2007 at 07:23:47PM -0500, Hal Rosenstock wrote: > > One feature I've thought has been underused in IBA is the raw IPv6 > > packet feature. > > I thought raw support (including IPv6 header) although still in the spec > was largely deprecated as the CRC protection was deemed too weak

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-02-07 Thread Jason Gunthorpe
On Wed, Feb 07, 2007 at 02:31:10PM -0800, Sean Hefty wrote: > >I agree that special casing some IPv6 addresses is a bad idea. It > >needs to be integrated correctly with NET and the routing table/etc > I haven't given this more than a few minutes of thought, but I was thinking > more along the

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-02-07 Thread Jason Gunthorpe
On Wed, Feb 07, 2007 at 02:40:51PM -0800, Sean Hefty wrote: > Are you referring to the SLID in the CM REQ? If so, I've been looking at > this issue as well. I simply cannot think of any way to come up with this > LID, and my current solution is to punt this problem over to the passive > side,

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-02-07 Thread Jason Gunthorpe
On Wed, Feb 07, 2007 at 12:24:08PM -0800, Sean Hefty wrote: > >I didn't get too far on getting CMA to work. Beyond the bad HopLimit > >feild I was seeing Hal pointed out a number of problems in IBA that > >would prevent it from working as is :< > > I've started thinking about what it would take to

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-02-07 Thread Jason Gunthorpe
On Wed, Feb 07, 2007 at 01:35:25PM -0800, Roland Dreier wrote: > Hmm, why is that? Shouldn't IPoIB work through a router, and > correctly get the GID of the final destination via ARP just fine? Basically, if IB routers are used, and the IPoIB feature of *not* spanning a subnet is used (for scalab

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-02-07 Thread Jason Gunthorpe
On Wed, Feb 07, 2007 at 10:55:00AM -0800, Sean Hefty wrote: > >Oops, I'll fix these style things and send a new patch. > > Jason, what's the status of this patch? (I ask because I'm starting to > look at router support in the stack.) I was going to resend it after Roland's earlier patch to clea

Re: [openib-general] ipath and current git woes

2007-02-01 Thread Jason Gunthorpe
On Wed, Jan 31, 2007 at 04:42:25PM -0800, Robert Walsh wrote: > Jason Gunthorpe wrote: > >Has anyone been able to use ipath with the current latest git > >everything? > > We're working on getting this up to date right now. Give us a couple of > days and we'l

[openib-general] ipath and current git woes

2007-01-31 Thread Jason Gunthorpe
Has anyone been able to use ipath with the current latest git everything? The libipathverbs git repository seems to be missing a patch from Roland to make it work with libibverbs.2 in an email titled: [openib-general] [PATCH 3/7] libipathverbs: Update libipathverbs for new libibverbs driver handl

Re: [openib-general] [PATCH 0/10] osm: QoS in OpenSM

2007-01-31 Thread Jason Gunthorpe
On Wed, Jan 31, 2007 at 12:41:43PM -0500, Hal Rosenstock wrote: > IMO, the XML syntax needs to be explained, discussed, and vetted on the > list. I am hopping this can occur reasonably quickly. If we are doing > XML for this, we need to get to a stable agreed syntax. I didn't see a DTD or schema

Re: [openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-01-26 Thread Jason Gunthorpe
On Thu, Jan 25, 2007 at 09:15:31PM -0800, Roland Dreier wrote: > > + if ((ret = > ib_init_ah_from_path(priv->ca,priv->port,pathrec,&av))) > > kernel style is spaces after commas, like Oops, I'll fix these style things and send a new patch. > > + ipoib_dbg(priv, "Path

[openib-general] [PATCH] IPOIB: Use a GRH when appropriate for unicast packets

2007-01-25 Thread Jason Gunthorpe
ib_init_ah_from_path contains the logic to decide when to use a GRH so call ib_init_ah_from_path instead of the hand coded version in IPOIB. This change along with recent opensm changes allows unicast IPOIB traffic to traverse a router. Signed-off-by: Jason Gunthorpe <[EMAIL PROTEC

Re: [openib-general] [PATCH] IB/core - ib_umad can cause address alignment fault on ia64

2007-01-19 Thread Jason Gunthorpe
this patch. --- Make the untyped data region in ib_user_mad u64 aligned so that casting ib_user_mad to structs with u64s in them works on ia64. Signed-off-by: Jason Gunthorpe <[EMAIL PROTECTED]> --- include/rdma/ib_user_mad.h |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) dif

[openib-general] [PATCH] IB/core - ib_umad can cause address alignment fault on ia64

2007-01-18 Thread Jason Gunthorpe
Make the untyped data region in ib_user_mad u64 aligned so that casting ib_user_mad to structs with u64s in them works on ia64. --- include/rdma/ib_user_mad.h |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/include/rdma/ib_user_mad.h b/include/rdma/ib_user_mad.h index 445

Re: [openib-general] [PATCH] IB/core - ib_umad can cause address alignment fault onia64

2007-01-18 Thread Jason Gunthorpe
On Thu, Jan 18, 2007 at 12:17:35PM -0600, Tom Tucker wrote: > Does changing the size of the structure break the ABI? ib_user_mad_hdr is 56 bytes long, that is already a multiple of 8 so .data is already aligned on 8. Thus the size of ib_user_mad does not change and there is no ABI concern here.

Re: [openib-general] [PATCH] IB/core - ib_umad can cause address alignment fault onia64

2007-01-18 Thread Jason Gunthorpe
On Thu, Jan 18, 2007 at 07:50:35PM +0200, Michael S. Tsirkin wrote: > > The int -> long change is needed because struct ib_umad_packet > > includes struct ib_user_mad (which has 4 byte alignment) but > > is then cast to struct ib_mad_hdr which has 8 byte alignment. > But I thought it is the data

Re: [openib-general] [PATCH] IB/core - ib_umad can cause address alignment fault on ia64

2007-01-18 Thread Jason Gunthorpe
On Thu, Jan 18, 2007 at 06:14:31PM +0200, Michael S. Tsirkin wrote: > So the issue is that we are casting char *data which has no > alignment guarantees to 64 bit number. We really must find a way to > force 64 bit alignment for struct ib_user_mad all over. Would not > something like the following

Re: [openib-general] ib_gid_is_link_local

2007-01-08 Thread Jason Gunthorpe
On Tue, Jan 09, 2007 at 01:17:47AM -0500, Hal Rosenstock wrote: > > I would like it if routers did not have to worry about joins in order > > to send a multicast packet. > > Send-only members are not supposed to receive. How do they receive then > ? Send-only members do not receive. Don't routers

Re: [openib-general] ib_gid_is_link_local

2007-01-08 Thread Jason Gunthorpe
On Fri, Jan 05, 2007 at 11:44:49AM -0500, Hal Rosenstock wrote: > > However, it might be smart to have opensm consider the routers to be a > > send-only member for every MLID.. > > Do you mean non-member rather than send-only member ? Routers need to > receive as well as send, right ? Or are you

Re: [openib-general] [PATCH] diags/saquery: fix node_desc.description as string usages

2006-10-30 Thread Jason Gunthorpe
On Tue, Oct 31, 2006 at 06:59:02AM +0200, Sasha Khapyorsky wrote: > > > This would be simpler. However some web searching shows that not all > > > printf() implementation permits not null terminated arrays even when > > > precision is specified (some issues were reported even with glibc-2.3.2). >

Re: [openib-general] Ordering between PCI config space writes and MMIO reads?

2006-10-25 Thread Jason Gunthorpe
On Wed, Oct 25, 2006 at 08:18:59AM -0600, Matthew Wilcox wrote: > PCI-PCI bridges are allowed to do it. If you look in table E-1 of PCI > 2.3, or table 8-3 of PCI-X 2.0, you'll see that a Posted Memory Write > can pass a Delayed Write Request (or in PCI-X, a Memory Write can pass a > Split Write

Re: [openib-general] Ordering between PCI config space writes and MMIO reads?

2006-10-24 Thread Jason Gunthorpe
On Tue, Oct 24, 2006 at 04:36:32PM -0600, Matthew Wilcox wrote: > On Tue, Oct 24, 2006 at 02:51:30PM -0700, Roland Dreier wrote: > > > I think the right way to fix this is to ensure mmio write ordering in > > > the pci_write_config_*() implementations. Like this. > > > > I'm happy to fix this i

Re: [openib-general] Race in mthca_cmd_post()

2006-10-23 Thread Jason Gunthorpe
On Mon, Oct 23, 2006 at 08:00:45PM -0700, Roland Dreier wrote: > > I just took a quick look at asm-ia64/io.h and there is __ia64_mf_a > > barriers after all non-posted IO operations (ib/outb). config write and > > config read transcations have identical rules to IO transactions at > > the PCI b

Re: [openib-general] Race in mthca_cmd_post()

2006-10-20 Thread Jason Gunthorpe
On Fri, Oct 20, 2006 at 02:14:43PM -0700, Roland Dreier wrote: > I just saw "the master cannot proceed" but the first few times I read > this, I didn't see the "(if a dependency exists)." Since no Yes, the languange in PCI-X is actually a bit clearer (pg 80): ``As in conventional PCI, if a reque

Re: [openib-general] Race in mthca_cmd_post()

2006-10-20 Thread Jason Gunthorpe
On Thu, Oct 19, 2006 at 08:48:44PM -0700, Roland Dreier wrote: > OK, this is the crux of my confusion. I always thought (and the PCI > spec seems to say this too) that config writes are non-posted, which > means that the Config Write cycle in your trace should block > everything until it is comple

Re: [openib-general] [PATCH] use mmiowb after doorbell ring

2006-10-18 Thread Jason Gunthorpe
On Wed, Oct 18, 2006 at 04:25:21PM -0700, Roland Dreier wrote: > > AC_DEFUN(rc_LIBSTDCPP_VER, > Thanks -- this actually solves the easiest part of my problem, and > does it in a way that's not really useful for me (libibverbs needs to > know what extra bits are getting added to plugin names, and

Re: [openib-general] [PATCH] use mmiowb after doorbell ring

2006-10-18 Thread Jason Gunthorpe
On Wed, Oct 18, 2006 at 01:43:03PM -0700, Roland Dreier wrote: > The only two things I need to figure out, I hope with help from > smarter people: I'm by no means an expert, but this might be helpfull to someone who is: AC_DEFUN(rc_LIBSTDCPP_VER, [AC_MSG_CHECKING([libstdc++ version])

Re: [openib-general] [PATCH] use mmiowb after doorbell ring

2006-10-17 Thread Jason Gunthorpe
On Wed, Oct 18, 2006 at 06:43:54AM +0200, Michael S. Tsirkin wrote: > The difference here is that libibverbs insists on putting all plugins > in a separate directory and passing full path to dlopen, which of course > breaks this. Yeah, plugins in a seperate dir are not well supported by all the fa

Re: [openib-general] [PATCH] use mmiowb after doorbell ring

2006-10-17 Thread Jason Gunthorpe
On Tue, Oct 17, 2006 at 08:44:34PM -0700, Roland Dreier wrote: > Jason> I think the typical way this is done would be to use > Jason> ld.so's 'hwcap' handling and stick an optimized library in > Jason> /usr/lib/sse2. > It's a good suggestion, but the problem is that the CPU-dependent

Re: [openib-general] [PATCH] use mmiowb after doorbell ring

2006-10-17 Thread Jason Gunthorpe
On Tue, Oct 17, 2006 at 04:31:00PM -0700, Roland Dreier wrote: > For now I just used lock; addl %0 to implement rmb on i386. I'm > really not comfortable making libmthca depend on sse2, and I don't see > a good way to detect and use sse2 at runtime. I think the typical way this is done would be

Re: [openib-general] [PATCH] IB/ipoib: use appropriate path selector

2006-09-13 Thread Jason Gunthorpe
On Thu, Sep 14, 2006 at 01:08:59AM +0300, Michael S. Tsirkin wrote: > > Umm -- why does it need a 2K MTU? As far as I know it should work > > fine with any MTU, assuming the SA sets the MTU of the broadcast > > multicast group correctly. > > Hmm, you are right, it is just that existing implement

Re: [openib-general] How to support IOMMUs for ipath driver

2006-09-12 Thread Jason Gunthorpe
On Tue, Sep 12, 2006 at 05:40:10PM -0700, Ralph Campbell wrote: > The ib_ipath driver needs kernel virtual addresses in order to be able > to copy data to/from the posted work requests since it does not > use HW DMA. It currently relies on the mapping being one-to-one > and cannot reasonably rever

Re: [openib-general] basic IB doubt

2006-08-28 Thread Jason Gunthorpe
On Mon, Aug 28, 2006 at 01:49:27PM -0400, Talpey, Thomas wrote: > Okay, that's good. However, doesn't it delay reads and writes until the > necessary table walk / mapping is resolved? Because it passes all other > cycles through, it seems to me that an interrupt may pass data, meaning > that order

Re: [openib-general] basic IB doubt

2006-08-28 Thread Jason Gunthorpe
On Mon, Aug 28, 2006 at 08:23:00AM -0700, Felix Marti wrote: > Might be a typo: dcbf does flush & invalidate and is a non-privileged > instruction. dcbi does invalidate only but it is privileged. Er yes, this is correct, sorry about the typo. Jason

Re: [openib-general] basic IB doubt

2006-08-28 Thread Jason Gunthorpe
On Mon, Aug 28, 2006 at 10:38:43AM -0400, Talpey, Thomas wrote: > Will turning on the Opteron's IOMMU introduce some of these > issues to x86? No, definately not. The Opteron IOMMU (the GART) is a pure address translation mechanism and doesn't change the operation of the caches. If Sun has a pro

Re: [openib-general] [PATCH 22 of 23] IB/ipath - print warning if LID not acquired within one minute

2006-08-27 Thread Jason Gunthorpe
On Sun, Aug 27, 2006 at 06:41:08PM -0700, Robert Walsh wrote: > Roland Dreier wrote: > > Michael> Looks like your devices are all single-port. With a multi > > Michael> port device it is quite common to have one port down. > > > > My reading of the patch is that it warns if the link is up

Re: [openib-general] basic IB doubt

2006-08-27 Thread Jason Gunthorpe
On Sun, Aug 27, 2006 at 03:30:56PM -0700, Roland Dreier wrote: > glebn> So, before touching the data that was RDMAed into the > glebn> buffer application should cache invalidate the buffer, is > glebn> this even possible from user space? (Not on x86, but it > glebn> isn't needed the

Re: [openib-general] basic IB doubt

2006-08-25 Thread Jason Gunthorpe
On Fri, Aug 25, 2006 at 09:33:31AM -0700, Roland Dreier wrote: > Thomas> How does an adapter guarantee that no bridges or other > Thomas> intervening devices reorder their writes, or for that > Thomas> matter flush them to memory at all!? > That's a good point. The HCA would have to

Re: [openib-general] [PATCH 2/3] SA: add call to initialize ib_ah_attr from a path record

2006-05-19 Thread Jason Gunthorpe
On Fri, May 19, 2006 at 12:15:29PM -0400, Hal Rosenstock wrote: > Are you aware of any currently that don't ? Nope > In this case, IMO the penalty of including the GRH should be paid (until > those SMs are fixed) rather than adding any additional prefix checking > into the end node. I'm fine wi

Re: [openib-general] [PATCH 2/3] SA: add call to initialize ib_ah_attr from a path record

2006-05-19 Thread Jason Gunthorpe
On Fri, May 19, 2006 at 10:19:50AM -0400, Hal Rosenstock wrote: > > I think it is also prudent to not use a GRH if the DGID's prefix is > > fe80::/64 (link local scope). > > Agreed. This saves on the overhead in the local subnet case. > > However, wouldn't the HopLimit returned from the SA in th

Re: [openib-general] [PATCH 2/3] SA: add call to initialize ib_ah_attr from a path record

2006-05-18 Thread Jason Gunthorpe
On Thu, May 18, 2006 at 04:24:16PM -0700, Sean Hefty wrote: > I should note that I compared the subnet prefixes to determine if the GRH > should be used. Reading back over the 'GRH flag in ib_ah_attr' thread, it > looks like there's consensus that hop_limit > 1 is the check that we want. > I

Re: [openib-general] opensm segfault?

2006-05-16 Thread Jason Gunthorpe
On Wed, May 17, 2006 at 09:10:11AM +0300, Eitan Zahavi wrote: > cl_memcpy should have some debug capabilities on top of memcpy ... > cl memory management provide means to track all memory allocations, etc. There are a huge number of canned solutions that provide a way to debug memory problems wit

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-14 Thread Jason Gunthorpe
On Sun, May 14, 2006 at 07:40:25AM -0400, Hal Rosenstock wrote: > > > Not always true in terms of local subnet (multicast and management MAD > > > response exceptions). > > > > Yes, but these are well specified. Multicast must always have a GRH. > > MAD requests are covered under my scenario above

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-12 Thread Jason Gunthorpe
On Fri, May 12, 2006 at 08:11:17AM -0400, Hal Rosenstock wrote: > > To allow what Roland is talking about you need an unambiguous > > mechanism where the SA can signal to the client that the path > > needs a GRH. > > Ah, you are referring to the SA path record response not the request. Yes.. Tho

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Jason Gunthorpe
On Thu, May 11, 2006 at 10:21:08AM -0700, Sean Hefty wrote: > Hal Rosenstock wrote: > >Anytime the send is off the local subnet (as well as multicast), a GRH > >is required. Also, there is a management response rule for responding > >when the request contained a GRH that require a GRH (13.5.4.4 p.

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-11 Thread Jason Gunthorpe
On Thu, May 11, 2006 at 07:20:19AM -0400, Hal Rosenstock wrote: > That would be a simpler check but HopLimit is not a required component > of PathRecord but I think this may not be sufficient as just because a > HopLimit >= 2 doesn't mean that a packet would be forwarded off subnet. I was thinkin

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Jason Gunthorpe
On Wed, May 10, 2006 at 09:56:58PM -0700, Roland Dreier wrote: > Hal> Huh ? In this case, aren't the subnet prefixes are required > Hal> to be different ? > > It's kind of a crazy thing to do but I don't see anything in the IB > spec that forbids two subnets with the same subnet prefix, or

Re: [openib-general] question regarding GRH flag in ib_ah_attr

2006-05-10 Thread Jason Gunthorpe
On Wed, May 10, 2006 at 04:44:42PM -0700, Roland Dreier wrote: > Sean> Does anyone know how the user determines if the grh flag > Sean> should be set in the ib_ah_attr when allocating an ib_ah? > Sean> Do they do this by examining the GIDs in a path record? > > Good question. It's alw

Re: [openib-general] Re: [PATCH 09/12] SRP: Changing ibsrpdm

2006-05-02 Thread Jason Gunthorpe
s via C99 runtime sized arrays, ie: void foo(int len) { char bar[len]; } GCC has supported this feature from C99 for a very long time as long as it isn't turned off by a compiler flag. alloca was never standardized by ISO or POSIX. -- Jason Gunthorpe <[EMAIL PROTECTED]>(

Re: [openib-general] Re: [RFC] [PATCH 1/2] multicast support for multiple users

2006-04-06 Thread Jason Gunthorpe
On Thu, Apr 06, 2006 at 12:51:23PM -0700, Roland Dreier wrote: > Maybe we should give up the ghost and stop trying to support IB switches? IMHO, the main reason to have the IB stack on a switch is to support ipoib for in band management with a stable and well tested code base. My company is looki

Re: [openib-general] Help with CONFIG_PCI_MSI in the kernel

2006-04-04 Thread Jason Gunthorpe
On Tue, Apr 04, 2006 at 06:29:24PM -0700, Roland Dreier wrote: > Jason> Just as a preface, I whipped this patch up because I have a > Jason> Asus with a broken BIOS and I wanted to test with MSI-X on > Jason> IB cards. It needs a little more thought before being > Jason> merged, but

Re: [openib-general] Help with CONFIG_PCI_MSI in the kernel

2006-04-04 Thread Jason Gunthorpe
On Tue, Apr 04, 2006 at 05:19:37PM -0700, Greg KH wrote: Just as a preface, I whipped this patch up because I have a Asus with a broken BIOS and I wanted to test with MSI-X on IB cards. It needs a little more thought before being merged, but the method seemed relevent given the discussion on Fedor

Re: [openib-general] Help with CONFIG_PCI_MSI in the kernel

2006-04-04 Thread Jason Gunthorpe
On Tue, Apr 04, 2006 at 04:52:32PM -0700, Grant Grundler wrote: > Yes, that's what I meant with "routing of the transaction". > "interrupt message transaction" is not "transformed" - just routed. > Page 19 uses the same language. Yes, I think we are saying exactly the same thing - I was saying 't

Re: [openib-general] APIC error

2006-04-04 Thread Jason Gunthorpe
On Tue, Apr 04, 2006 at 02:04:43PM -0700, Makia Minich wrote: > Using earlier BIOS versions, I don't see the issue at all, but on the latest > BIOS versions I'm dead in the water. So, I was hoping that someone else has > seen this issue and perhaphs might know of a workaround. You might want to

Re: [openib-general] Help with CONFIG_PCI_MSI in the kernel

2006-04-04 Thread Jason Gunthorpe
On Tue, Apr 04, 2006 at 09:45:09AM -0700, Grant Grundler wrote: > > MSI requires end device support and something in a bridge > > to transform the MSI message into an APIC message, but the kernel > > currently only looks for end device support. > > APIC Message? > MSI is just a DMA-write from the

Re: [openib-general] Help with CONFIG_PCI_MSI in the kernel

2006-04-04 Thread Jason Gunthorpe
On Mon, Apr 03, 2006 at 09:36:00PM -0700, Grant Grundler wrote: > On Mon, Apr 03, 2006 at 03:14:56PM -0700, Greg Lindahl wrote: > > Red Hat has started turning off CONFIG_PCI_MSI in their kernels (FC5 > > and the latest FC4 update). I remember a while back there was a > > discussion about how MSI m

Re: [openib-general] RDMA CM and loopback addresses

2006-03-30 Thread Jason Gunthorpe
On Thu, Mar 30, 2006 at 03:48:50PM -0800, Sean Hefty wrote: > On the client side, the client does need to know which device to use before > calling connect. The client obtains the device by calling bind() or > resolve_addr(). The latter call takes the source and destination addresses, > and > o

Re: [openib-general] RDMA CM and loopback addresses

2006-03-30 Thread Jason Gunthorpe
On Thu, Mar 30, 2006 at 01:06:21PM -0800, Sean Hefty wrote: > It will be difficult, if not impossible, to fully match IP semantics. > Before a connection can occur, hardware resources need to be allocated, > which requires a specific device. So, for the purpose of RDMA, we may need > to treat

Re: [openib-general] RDMA CM and loopback addresses

2006-03-30 Thread Jason Gunthorpe
On Thu, Mar 30, 2006 at 10:45:29AM -0800, Sean Hefty wrote: > Jason Gunthorpe wrote: > >If you disable all packet filtering and you have two hosts > >[10.0.0.1 and 10.0.0.2] doing the following on .2: > > > >ip route add 127.0.0.1 via 10.0.0.1 dev eth0 > >telnet -b

Re: [openib-general] RDMA CM and loopback addresses

2006-03-29 Thread Jason Gunthorpe
On Wed, Mar 29, 2006 at 04:50:27PM -0800, Sean Hefty wrote: > To obtain similar behavior with the RDMA CM, I propose the following: > > 1. Binding to the loopback address will no longer result in acquiring a local > RDMA device. (This will be deferred to rdma_resolve_addr().) > 2. Listening on a

Re: [openib-general] Re: RFC: kill CFLAGS in makefiles?

2006-03-22 Thread Jason Gunthorpe
On Wed, Mar 22, 2006 at 02:24:07PM +0200, Michael S. Tsirkin wrote: > Nothing. But that's not what we had for mutex backport. > We had > > #if LINUX_VERSION_CODE < KERNEL_VERSION(2,6,16) > #include > #else > #include > #endif /* XXX end of hack */ If you keep the -I you can do this: Source:

Re: [openib-general] Re: [iproute2] IPoIB link layer address bug

2006-03-21 Thread Jason Gunthorpe
On Tue, Mar 21, 2006 at 03:56:17PM -0800, Stephen Hemminger wrote: > Okay, but there are number of other places in iproute2 that call ll_addr_a2n() > with ifr.ifr_hwaddr.sa_data. And that is 14 bytes. If you want to fix those > it will be harder since it would increase the sizeof(struct sockaddr)

Re: [openib-general] [PATCH] IPoIB network interface "RUNNING" status with the cable disconnected - fix

2006-03-20 Thread Jason Gunthorpe
On Mon, Mar 20, 2006 at 04:12:44PM -0800, Roland Dreier wrote: > Leonid>the patch fixes a problem with the network interface > Leonid> "RUNNING" status. The problem was that the status stayed > Leonid> "RUNNING" with the cable disconnected, > > Hmm, is this really a bug? For exam

Re: [openib-general] Re: [PATCH 6/6] IB: userspace support for RDMA connection manager

2006-03-06 Thread Jason Gunthorpe
On Mon, Mar 06, 2006 at 04:05:30PM -0800, Bryan O'Sullivan wrote: > Depends on the driver. Ours needs the interrupt vector rather than the > number, which means we don't work without CONFIG_PCI_MSI. That is, > unless there's some other way to get the vector that I don't know about > (entirely li

[openib-general] [PATCH] Minor fix for ibnetdiscover

2006-02-28 Thread Jason Gunthorpe
Trivial, ibnetdiscover does not display some switches for some topologies. This seems to come up if you connect a CA to a switch but nothing else to the switch. Output before: # Topology file: generated on Tue Feb 28 18:21:30 2006 # # Max of 1 hops discovered # Initiated from node 0002c9020020d654

[openib-general] Re: Low cost MBs compatible with Mellanox PCIE Cards

2006-02-16 Thread Jason Gunthorpe
On Thu, Feb 16, 2006 at 09:52:19PM +0200, Michael S. Tsirkin wrote: > It could make sense to put this in openib Wiki together with any info > on patches needed to make it work. That sounds like a good idea, I have no wikki experience myself - can anyone add a new page? Any pointers? Thanks, Jaso

Re: [openib-general] Re: Plans for libibverbs 1.0, 1.1 and beyond

2006-02-16 Thread Jason Gunthorpe
On Thu, Feb 16, 2006 at 10:29:29PM +0200, Michael S. Tsirkin wrote: > Quoting r. James Lentini <[EMAIL PROTECTED]>: > > I wouldn't want to see the repository split up. Is moving all of the > > OpenIB code from svn to git an option? > > I dont think git supports multiple people working on the same

Re: [openib-general] Low cost MBs compatible with Mellanox PCIE Cards

2006-02-16 Thread Jason Gunthorpe
On Thu, Feb 16, 2006 at 09:21:24AM -0600, Steve Welch wrote: > We do not have interrupt assignment problems for the HCA that I'm aware of > using the A8N32-SLI Deluxe MB. We do have message signaled interrupts > enabled as well (Roland's suggestion to Jason). Output follows. Thanks, thats good

Re: [openib-general] Low cost MBs compatible with Mellanox PCIE Cards

2006-02-16 Thread Jason Gunthorpe
On Wed, Feb 15, 2006 at 04:52:17PM -0800, Roland Dreier wrote: > I've not seen any problems with MSI/MSI-X with nforce4 and PCIe HCAs > on both my Asus A8N-SLI and HP DL145G2 systems. I've managed to make it work here too. I found an earlier email from you about the HT MSI Mapping capability whic

Re: [openib-general] Low cost MBs compatible with Mellanox PCIE Cards

2006-02-15 Thread Jason Gunthorpe
On Wed, Feb 15, 2006 at 02:12:10PM +1300, Dave Watkins wrote: > I've tried a few Asus boards with IB cards in their graphics card slots > with no luck, the boards did post but the cards weren't visable to the > drivers. Same cards in other machines were fine. I just ran out an bought an Asus A8N-V

[openib-general] Low cost MBs compatible with Mellanox PCIE Cards

2006-02-14 Thread Jason Gunthorpe
Hi All, Does anyone have any idea if any lower end bare bones motherboards (ie the Asus/Gigabyte type vendors) work successfully with a Mellanox PCI-E x8 card (MHGS18-XT A2 specifically)? I'm looking for a cost-effective way to get a bunch of IB hosts for testing, so compute performance isn't a c