I see that the EWG list is now calling itself the Engineering Working
Group, has it been renamed from the Enterprise Working Group? If so,
did the nature of the list change? Or was it a typo?
-- greg
___
openib-general mailing list
openib-general@openi
On Mon, Jan 08, 2007 at 12:26:00PM -0600, Sean Hubbell wrote:
> I have had to make significant
> work arounds for our current, third party network API that we purchased
> and continue to watch if fall down and still not take advantage on the
> bandwidth that I need. With that said, does anyone
Or remove SVN code which is misleading, as it continues to mislead
people repeatedly.
-- greg
> We should put this type of warning in all the infiniband/core modules
> that have moved to the kernel...
>
>
> On Wed, 2006-11-15 at 11:29 -0800, Roland Dreier wrote:
> > > #warning The mthca driver
On Fri, Nov 10, 2006 at 03:32:34PM +0530, john t wrote:
> If I send data from mthca0-1 to mthca0-1 meaning from same port to the same
> port i.e. same port doing send/recv (also same cable doing send/recv) I get
> a BW of around 10 Gb/sec.
Note that the IB standard says in this case that the adap
On Thu, Oct 26, 2006 at 03:16:50PM +1300, vishal wrote:
> I am interested in moving data from memory in one PC to
> memory in the other using Infiniband hardware. Any suggestions on what
> would be the best way to do it ?
There are 2 main approaches, messages and RDMA. The best one depends
on t
On Tue, Oct 24, 2006 at 08:35:18AM -0500, Sean Hubbell wrote:
> We are currently looking at the new tickless kernel. Do you have one
> that you recommend?
The main one to less-recommend is 2.6.9-based kernels, those are the
slowest at TCP. Modern kernels, like the ones you see in Fedora 4 and
up
On Mon, Oct 23, 2006 at 07:53:06AM -0500, Hubbell, Sean C Contractor/Decibel
wrote:
> I currently have several applications that uses a legacy IPv4 protocol
> and I use IPoIB to utilize my infiniband network which works great. I
> have completed some timing and throughput analysis and noticed t
On Thu, Oct 19, 2006 at 11:37:55AM -0400, Doug Ledford wrote:
> and ISTR that it
> isn't even required by the MPI spec since that leaves behavior of an MPI
> app undefined after a fork() call and hence any application written to
> depend on undefined behavior is broken by design,
Doug,
There are
On Tue, Oct 17, 2006 at 05:18:34PM -0700, Scott Weitzenkamp (sweitzen) wrote:
> I agree the 32-bit byte and packet counters are useless as they get
> pegged in a few seconds on a busy IB networks. I thought there was an
> effort in IBTA to fix this.
Yes, it's in the management working group.
>
On Wed, Oct 11, 2006 at 04:21:41PM +0200, Or Gerlitz wrote:
> On 10/9/06, Michael S. Tsirkin <[EMAIL PROTECTED]> wrote:
>
> > I'm trying to build a network device driver supporting a very large MTU
> > (around 64K)
> > on top of an infiniband connection, and I've hit a couple of issues I'd
> > ap
On Fri, Sep 08, 2006 at 03:49:57PM +0530, john t wrote:
> What is the difference between HCAs with memory and without memory.
And to answer for QLogic InfiniPath HCAs, we don't sell HCAs with
memory. We don't need it. There's actually a small amount of memory
within the single chip that makes up
On Sun, Aug 27, 2006 at 06:28:06PM -0400, Doug Ledford wrote:
> I would definitely put the option in, and in fact would default it to
> *NOT* truncate.
I agree. I have never seen any other daemon with a logfile do this,
why are we out to surprise the admin? The admin might want the start
of the l
On Fri, Aug 25, 2006 at 03:21:20PM -0400, [EMAIL PROTECTED] wrote:
> I presume you meant invalidate the cache, not flush it, before accessing
> DMA'ed
> data.
Yes, this is what I meant. Sorry!
-- greg
___
openib-general mailing list
openib-general
On Fri, Aug 25, 2006 at 10:00:50AM -0400, Thomas Bachman wrote:
> Not that I have any stance on this issue, but is this is the text in the
> spec that is being debated?
>
> (page 269, section 9.5, Transaction Ordering):
> "An application shall not depend upon the order of data writes to
> memory
On Fri, Aug 25, 2006 at 10:13:01AM -0500, Tom Tucker wrote:
> He does say this, but his analysis does not support this conclusion. His
> analysis revolves around MPI send/recv, not the MPI 2.0 get/put
> services.
Nobody uses MPI put/get anyway, so leaving out analyzing that doesn't
change reality
On Fri, Aug 25, 2006 at 05:17:04PM +0300, Sasha Khapyorsky wrote:
> So more generic question: some application performs blocked read() from
> /dev/umadN, should this read() be interrupted and return error (with
> appropriate errno value), then the port state becomes DOWN?
Iif the SM gets a signal
On Thu, Aug 24, 2006 at 04:13:22PM -0700, Sean Hefty wrote:
> >Why don't you measure it, then?
>
> Why? Reading a memory location directly will be faster than calling a
> function
> to read from a memory location.
... sigh. This is not true, there are quite obvious implementations
where this i
On Thu, Aug 24, 2006 at 03:53:37PM -0700, Woodruff, Robert J wrote:
> If the overhead in polling the CQ rather than memory was not so
> high, they would have used it, but they found that it added > 2us to
> the latency and found they could get better performance if they
> polled memory,
I keep on
On Thu, Aug 24, 2006 at 03:43:38PM -0700, Sean Hefty wrote:
> >Actually, if a hardware implementation provided the same performance
> >(in this case latency) by polling on a CQ as one where polling on
> >memory was guaranteed to work, the customer may actually prefer the
> >"standard" implementatio
For those of you interested in this topic, there's an interesting
article by Patrick Geoffrey in HPCWire entitled "A Critique of RDMA".
http://www.hpcwire.com/hpc/815242.html
(you might have to be a subscriber, but I'm sure Patrick would send
you a copy if you ask.)
It's basically a critique of
On Thu, Aug 24, 2006 at 03:37:21PM -0700, Sean Hefty wrote:
> I'm missing the standard you're using to judge what's theoretically good and
> bad.
Having simpler programming to get good performance is a theoretical
good. Extra hacks for specific hardware is theoretically bad,
pratically good only
On Thu, Aug 24, 2006 at 03:28:44PM -0700, Woodruff, Robert J wrote:
> Yes, IMO, the iWarp folks and the IBTA should consider making this a
> requirement, but even if they do not the ISVs will still require it.
Ahah. So with all this head and light, we agree that it should be
added to the standar
On Thu, Aug 24, 2006 at 03:13:33PM -0700, Woodruff, Robert J wrote:
> If the feature gives them a huge advantage in performance (and it
> does) and all of the hardware vendors that they deal with already
> implement it, then yes, they will force, by defacto standard that
> all other newcomers impl
On Thu, Aug 24, 2006 at 02:58:18PM -0700, Sean Hefty wrote:
> >We're trying to create *inter-operable* hardware and
> >software in this community. So we follow the IB standard.
>
> Atomic operations and RDD are optional, yet still part of the IB "standard".
> An
> application that makes use of e
On Thu, Aug 24, 2006 at 02:10:38PM -0700, Woodruff, Robert J wrote:
> The other way to look at it is, the customer goes to the ISV and asks,
> what hardware should I buy, and the ISV says I support X version of MPI
> and vendor Y's hardware works with X version of MPI.
I thought the goal of Infi
On Wed, Aug 23, 2006 at 04:46:52PM -0700, Roland Dreier wrote:
> Yes, Mellanox documents that it is safe to rely on the last byte of an
> RDMA being written last.
OK, great. I'm fine with people using things which are supported, but
then we need the big, blinking "Warning! This program is non-sta
On Thu, Aug 24, 2006 at 01:57:39PM -0700, Sean Hefty wrote:
> >OK, great. I'm fine with people using things which are supported, but
> >then we need the big, blinking "Warning! This program is non-standard, and
> >won't work with many of the devices supported by Open Fabrics!" sign.
>
> If an appl
On Wed, Aug 23, 2006 at 09:29:18AM -0700, Sean Hefty wrote:
> I don't believe that there is any ordering guarantee by the architecture.
> However, specific adapters may behave this way, and I've seen applications
> make
> use of this by polling the last memory byte for a completion, for example
On Wed, Aug 23, 2006 at 06:01:32PM +0300, Michael S. Tsirkin wrote:
> So this seems to be ripping out chunks of upstream code (ipath_ht400)
> replacing them with something else (ipath_iba6110, ipath_iba6120.o)
To answer this piece of the question, we were acquired last April, and
of course we hav
On Tue, Aug 01, 2006 at 01:25:44PM +0200, Christoph Hellwig wrote:
> Exactly. Please don't even try to put brand names (especially if
> they're as stupid as this) in. We don't call our wireless stack
> centrino just because intel contributed to it either.
Centrino: Intel-only brand name
WiFi:
On Mon, Jul 31, 2006 at 12:04:21PM -0700, Roland Dreier wrote:
> Unless someone else has a problem with the rmdav_ name then I think we
> should let this die.
Sounds like a call for an open discssion on it, with a proper subject
line, even. And asking outside of openib-general. Which is what I am
On Mon, Jul 31, 2006 at 11:31:33AM -0700, Roland Dreier wrote:
> Greg> Hint: did you ever hold a discussion as to whether or not
> Greg> that was the right transport-neutral name?
>
> Jeeze, Sean posted the RDMA CM code to three mailing lists for review
> about 100 times. Did you ever com
On Mon, Jul 31, 2006 at 01:34:41PM -0500, Steve Wise wrote:
> You seem to be the only one objecting to rdma_ and/or rdmav_.
At Sonoma, I was not the only one. I forget, were you there?
> I've listened to your arguments for why you think rdma is a bad name,
> and I'm not convinced.
I'm not su
On Mon, Jul 31, 2006 at 11:18:16AM -0700, Roland Dreier wrote:
> My gut reaction is negative. The whole idea of "verbs" is a bit of
> technical jargon that makes no sense unless you've lived in the RDMA
> world for a while,
Given the way you are defining RDMA, I'm not surprised at the
conclusion
On Mon, Jul 31, 2006 at 01:03:16PM -0500, Steve Wise wrote:
> I agree. Plus we already have precedence for rdma_ with the RDMA CMA...
That's precedence about like "I used the term 'wimps' in a poster
paper once, so now you should allow me to use 'wimps' in my
Astrophysical Journal article."
Tru
On Mon, Jul 31, 2006 at 02:17:20PM -0400, James Lentini wrote:
> Dusting off my copy of vipl.h, circa 1996, I see that these operations
> were called RDMA READ/WRITE in VIA.
Yes, and that's the predecessor to IB, so that's no surprise that it
uses the same term. The IETF RDMA people also use it.
On Mon, Jul 31, 2006 at 10:54:55AM -0700, Caitlin Bestler wrote:
> Trying to characterize "RDMA" as consisting *solely* of
> messages that identify target buffers in the message is
> off target.
You're using circular arguments: "Because one particular subset of the
RDMA community defines RDMA in
On Mon, Jul 31, 2006 at 10:45:39AM -0700, Roland Dreier wrote:
> No other drivers have a brand name and it's pretty silly trying to
> brand IB/iWARP/RDMA/whatever drivers.
I don't see this as branding or marketing. I see it as trying to come
up with a name that's accurate.
What do you think of v
On Mon, Jul 31, 2006 at 10:39:49AM -0700, Sean Hefty wrote:
> Or maybe just "verb". Would that be better?
That's a good one.
> IMO, the underlying issue with using 'rdma' is that a software based
> solution doesn't actually do 'rdma'. I think this is Greg's complaint, and
> why he uses the t
On Mon, Jul 31, 2006 at 10:32:05AM -0700, Roland Dreier wrote:
> Greg, what would be your suggestion of a more generic (not
> IB-specific) replacement of the libibverbs name and ibv_ prefix?
Anything that makes it clear that it's an Open Fabrics call. Which is
what our organization and software s
On Mon, Jul 31, 2006 at 01:25:39PM -0400, James Lentini wrote:
> I agree that the term RDMA SEND is confusing. However, the data in an
> RDMA SEND is deposited directly (zero copy) into the users memory.
There are many mechanisms other than DMA or RDMA which have this
property. You're confusing
On Mon, Jul 31, 2006 at 09:01:16AM -0700, Caitlin Bestler wrote:
> That would imply that the purpose of the openfabrics stack
> is to replace netdev.
I don't think it implies that at all.
-- greg
___
openib-general mailing list
openib-general@openib.o
On Mon, Jul 31, 2006 at 10:24:11AM -0500, Steve Wise wrote:
> However, the IETF RDMA protocol defines SEND as well as READ, WRITE,
> etc. So in my mind, that's all RDMA, not just read and write.
Well, most people think RDMA means RDMA. The RDMA protocol undoubtedly
defines SEND/RECV because it's
On Mon, Jul 31, 2006 at 09:52:48AM -0500, Steve Wise wrote:
> rdma_* is more descriptive than something like ofv_* or of_* in my
> opinion. I would think the prefix should help describe the
> functionality being implemented: Transport Neutral RDMA.
Some functions are RDMA. Others are not. If a
> This patchset is a proposal to create new API's and data structures with
> transport neutral names.
We named ourselves OpenFabrics instead of OpenRDMA for a reason, did I
miss some point where we decided that we would use RDMA as a transport
neutral name in the source code?
-- greg
__
On Tue, Jun 13, 2006 at 05:11:47PM -0700, Ira Weiny wrote:
> After some tracking down he found that apparently if he used a "system" call
> [int system(const char *string)] the next MPI command will fail.
Are you sure MVAPICH supports fork()? It is not unusual for MPI
implementations to not suppo
So this is a _critical_ bugfix ?
> Auuuch it is there!
> My mistake. Sp please apply the patch to the OFED 1.0 branch too.
> BTW: Is the osmtest -f a excersizes this query on the OFED 1.0 ?
>
> > Huh ? What's
> >
> https://openfabrics.org/svn/gen2/branches/1.0/src/userspace/management/o
> sm/ope
On Thu, Jun 08, 2006 at 09:49:35AM -0700, Sean Hefty wrote:
> What if we started with something like the following compliance statement, and
> tried to add this to the spec?
>
> An SM, upon becoming the master, shall respect all existing communication in
> the
> fabric, where possible.
Isn't th
On Mon, May 22, 2006 at 07:56:03PM -0700, zhu shi song wrote:
> I won't wait sdp OK. I hope to use another method to
> port squid.
How about IPoIB?
-- greg
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/op
> >Do you really need this heavy debug logging in the first place? You
> >can use kprobes for arbitrary run-time inspection anyway, so logging
> >everything seems wasteful.
>
> The problem I see with kprobes is that you have to set several kernel
> configuration options (e.g. CONFIG_KPROBES, CONFI
On Fri, Apr 28, 2006 at 11:23:44PM -0700, Sean Hefty wrote:
> The proposal is to only discard duplicate requests while a response to the
> first
> request is being generated.
Ah. Does that happen that often?
-- greg
___
openib-general mailing list
ope
On Fri, Apr 28, 2006 at 03:20:13PM -0700, Sean Hefty wrote:
> I'd like to propose that the MAD layer detect duplicate requests.
Sean,
You can't add this kind of thing piecemeal to a protocol and have it
work. If the sender doesn't see a response (perhaps the response was
lost, or was slow coming
On Thu, Apr 27, 2006 at 04:22:40PM -0700, Grant Grundler wrote:
> Anything preventnig such a gateway from routing SDP to ethernet?
> Those gateways obviously will grok IB protocols.
> I'm asking becuase I don't understand/know if there is a real
> barrier to an IB -> ethernet gateway _without_ IPo
On Wed, Apr 26, 2006 at 11:13:24PM -0500, Troy Benjegerdes wrote:
> David is right. If you care about performance, you are already using SDP
> or verbs layer for the transport anyway. If I am going to be doing IPoIB,
> it's because eventually I expect the packet might get off the IB network
> and
On Thu, Apr 27, 2006 at 01:15:57AM +0300, Sasha Khapyorsky wrote:
> There is small patch for osm_log, this provide possibility to drop log
> output to stdout or stderr.
Isn't the Unix convention to use "--" to mean stdout? Or you can use
/dev/fd/{0,1}...
-- greg
On Thu, Apr 20, 2006 at 01:00:43PM -0400, Hal Rosenstock wrote:
> Also, is IPoIB always setup when running MPI ?
That's an easy one: No.
-- greg
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
On Mon, Apr 03, 2006 at 09:36:00PM -0700, Grant Grundler wrote:
> The only evidence I have is one AMD chipset is buggy WRT MSI.
Grant,
I know about that case, the kernel disables stuff if it sees an AMD
8131 due to a bug. What I am referring to was IPoIB performance on
Mellanox HCAs being improv
Red Hat has started turning off CONFIG_PCI_MSI in their kernels (FC5
and the latest FC4 update). I remember a while back there was a
discussion about how MSI made the Mellanox HCA run faster, can someone
please add some concrete details about this to the bug? Thanks.
http://bugzilla.redhat.com/bug
On Fri, Mar 31, 2006 at 08:37:58AM -0800, Roland Dreier wrote:
> If you want to forward the necessary patches to [EMAIL PROTECTED] for
> inclusion in 2.6.16.x releases, that would be great.
It would be nice to have a single person responsible for spotting
things that qualify for the stable kernel
On Mon, Mar 27, 2006 at 08:18:28AM -0800, Shirley Ma wrote:
> Is it a good idea to send such a big messages size using UD mode?
The usual reason for not doing this with UDP is that it turns a small
amount of packet loss into lots more lossage. This is a real issue
with ethernet when it drops pack
On Sat, Mar 25, 2006 at 10:36:10PM +0200, Michael S. Tsirkin wrote:
> In what sense is it more like an ethernet device than IPoIB? Since IP over IB
> just puts an IP packet inside a UD packet without overhead, what wire protocol
> changes can thinkably be done, while still using InfiniBand protoco
On Fri, Mar 24, 2006 at 10:38:35PM -0800, Matt Leininger wrote:
>That could be. I'll check. I thought ipath_ether was suppose to
> work when IPoIB was enabled.
In general, our non-IB protocols work together with OpenIB. This item
is the one thing that doesn't work, and it's due to an SMA b
On Thu, Mar 23, 2006 at 11:50:18PM -0800, Matt Leininger wrote:
> I have the ipath driver up, running, and working with IPoIB. I'm using
> 2.6.16 with svn 5938. The ipath_ether comes up as eth2. I can set the
> netmask and broadcast, but when I try to set the ip address for this
> device I get
On Tue, Feb 21, 2006 at 11:40:53PM -0800, Fabian Tillier wrote:
> You'd have to make the group 1X. Note that the group being 1X doesn't
> limit unicast traffic to 1X rates, since the rate for unicast traffic
> would be set based on the rate reported in the path records for the
> various endpoint
On Tue, Feb 21, 2006 at 08:17:02PM -0800, Fabian Tillier wrote:
> The node joining or creating the multicast group doesn't need to
> specify the rate - the SA can figure out the rate to use based on the
> requestor (for creation), or validate that the requestor supports the
> existing group's rate
On Tue, Feb 21, 2006 at 05:21:33PM -0800, Roland Dreier wrote:
> No, but an IB multicast group has a speed associated to it. This is
> to allow, say, a 4X port sending multicast packets to use the right
> static rate to avoid overrunning a 1X port that is also a member of
> the group.
Thanks for
Is this a correct summary of this thread?
* IPoIB uses an InfiniBand multicast group to fake ethernet broadcast
* This is optional, I'm not sure what functionality is lost without it
* MVAPICH uses a multicast group for some MPI collectives
* This can be turned off by setting env var DISABLE_
On Tue, Jan 31, 2006 at 03:53:22PM +0100, Xavier Grave wrote:
> I'm using Ada95 and I would like to implement a thin binding in order to
> use infiniband.
If MPI meets your needs -- do you need recovery from failures? -- then
you can use one of the existing MPI/ADA bindings.
-- greg
___
Since no one's really answered this yet:
Many sysadmins are not going to want to install a relational database
to run an SA cache. So I'd stick to Berkeley DB if I were you.
-- greg
___
openib-general mailing list
openib-general@openib.org
http://open
On Thu, Dec 22, 2005 at 12:14:44PM -0800, Sean Hefty wrote:
> Can MPI operate with unreliable multicast support? Does MPI plan on
> using IB multicast?
Given the large number of MPI implementations over IB, I don't think
there's a single answer.
-- greg
On Tue, Dec 20, 2005 at 09:11:12AM +0200, Yael Kalka wrote:
> > - if (&p_ur->signal)
> > + if (&p_ur->signal != NULL)
Aren't these 2 statements required to execute the same according to
the C standard?
I wrote a tiny test program and gcc4.0.0 as distributed with Fedora
Core 3 generat
On Mon, Dec 12, 2005 at 09:55:03AM -0800, Bill Boas wrote:
> Now I believe that when a Linux
> distribution, an IB company or a Tier One OEM decides that is a
> version of the code that they will support, then that is a "release".
Why don't we imitate the Linux kernel process? OpenIB has to fo
On Wed, Nov 09, 2005 at 01:57:06PM -0800, Michael Krause wrote:
> What you indicate above is that RDS
> will implement a resync of the two sides of the association to determine
> what has been successfully sent.
More accurate to say that it "could" implement that. I'm just
kibbutzing on someone
On Wed, Nov 09, 2005 at 12:18:28PM -0800, Michael Krause wrote:
> So, things like HCA failure are not transparent and one cannot simply
> replay the operations since you don't know what was really seen by the
> other side unless the application performs the resync itself.
I think you are over-s
On Tue, Nov 08, 2005 at 01:08:13PM -0800, Michael Krause wrote:
> If an application takes any action assuming that send complete means
> it is delivered, then it is subject to silent data corruption.
Right. That's the same as pretty much all other *transport* layers. I
don't think anyone's assert
Caitlin,
Can you please use the standard quoting style? I can't tell which
comments are yours. Thanks.
-- greg
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http
On Fri, Nov 04, 2005 at 11:54:27AM -0800, pandit ib wrote:
> Since a UDP application assumes the underlying transport is
> unrealiable it should not have any problems running on RDS.
> On getting EWOUDBLOCK it will simply retry.
Most existing UDP applications do not expect a return error code of
On Thu, Oct 27, 2005 at 09:29:57AM -0500, Troy Benjegerdes wrote:
> I guess the point of all this is find a end-user use-case for the SM
> MIB, and work back from there to decide if haveing a MIB actually helps
> solve the problem.
The end-use case is likely to be something like "an enterprise wh
> Windows compiler does not enable declaration not in the beginning of
> the function, so I would
> like to have it changed.
> We can either move the declaration to the beginning of the function,
> or add {} around the declaration.
Isn't there a gcc flag to convince it to give an error in this ca
On Thu, May 19, 2005 at 12:46:04PM -0700, Tom Duffy wrote:
> > The article states: 'InfiniPath will also support the OpenIB software stack
> > providing full InfiniBand compliance.'
>
> Also interesting that OpenIB is suddenly an "industry standard." ;-)
Tom,
There's a reason for that -- when
On Tue, May 03, 2005 at 11:43:25AM -0700, Andy Isaacson wrote:
> [1] You might want to allow the child to start a completely new RDMA
> context, but I don't see that as necessary.
Some people use a hybrid OpenMP+MPI model, so some MPI implementations
want to fork and have the child open a new
> Todd> But that implies the hardware has an MMU and it also puts an
> Todd> interrupt in the path per page sent.
>
> Well, there's one interrupt per non-resident page sent. But nearly
> all of the time the page will be present.
It doesn't imply that there's an MMU, either. I know that M
On Fri, Apr 29, 2005 at 12:33:54PM -0700, Grant Grundler wrote:
> Being mostly clueless about Quadrics implementation, I'm probably
> missing something that makes Quadrics a MMU but not the IB variants.
> Can someone clue me in please?
As far as I can tell it's mostly a marketing distinction. Man
On Fri, Mar 04, 2005 at 11:58:33AM -0800, Tom Duffy wrote:
> In any event, I think being able to plop an IB network in an Ethernet
> world will require things like RARP to work. If there is no spec now,
> it should be written.
Much more important is understanding the role of RARP in the ethernet
On Wed, Jan 05, 2005 at 05:08:31PM -0500, Hal Rosenstock wrote:
> What TZ ? PST ?
Ah, if only you knew how funny that question is! 2 beer penalty and
loss of down for failure to appreciate the uniqueness of Sandia's
geography.
-- greg
___
openib-gener
85 matches
Mail list logo
