On Tue, Oct 24, 2006 at 12:13:19PM -0700, Roland Dreier wrote:
> John Partridge found an interesting bug involving mthca (Mellanox
> InfiniBand HCA driver) on IA64/Altix systems. Basically, during
> initialization, mthca does:
>
> - do some config writes, including enabling BARs
> - then
>The problem is for the the remote node where RC QP listen was accepted
>this connection (Passive side). There is no way for this node to know
>that port failure has occurred on the Active side. So it requires some
>interfaces to get this notification. So ib_sa_serv_notice_hdlr()
>interface as desc
>but this function
>
>cm_init_qp_rts_attr(struct cm_id_private *cm_id_priv,
> struct ib_qp_attr *qp_attr,
> int *qp_attr_mask)
>
>does perform the state transition using
>if (cm_id_priv->alt_av.ah_attr.dlid) {
>*qp_attr_mask |= IB_QP_PATH_MIG_STATE;
>
> I have implemented this interface and proposing this in bug#172. You
>have to patch it manually to get it working.
Can you post the patch to the list? If the functionality is needed, I'd like to
queue it for 2.6.20. Although, I'd prefer if the existing ib_cm_init_qp_attr()
routine were used wi
I have implemented this interface and proposing this in bug#172. You
have to patch it manually to get it working.
VBabu
somenath wrote:
> I don't see that API( ib_cm_init_rearm_attr() the build I am using.
>
> so I I use a function like this:
>
> modifyqp_rearm(con_t *con) {
>struct ib_q
Sean Hefty wrote:
> somenath wrote:
>
>> any idea when this stuff will get done?
>
>
> As a rough estimate, Q1, though I will try to get to it after adding
> support for SA events, which likely would be near late November, early
> December. The framework in the ib_cm is there; I just need a w
I don't see that API( ib_cm_init_rearm_attr() the build I am using.
so I I use a function like this:
modifyqp_rearm(con_t *con) {
struct ib_qp_attr qp_attr;
int qp_attr_mask =0;
int ib_stat =0;
memset(&qp_attr, 0, sizeof(struct ib_qp_attr));
qp_attr.qp_state = IB_QPS_RTS;
1. No my application has to make the state trasition from RTS to RTS.
In case of failover set path_mig_state to IB_MIG_MIGRATED.
In case of rearming call the function ib_cm_init_rearm_attr(). This
function is defined in the bug#172.
2. No, I have my own ULP module which sits alongside to the SDP
Sean Hefty wrote:
> somenath wrote:
>
>> any idea when this stuff will get done?
>
>
> As a rough estimate, Q1, though I will try to get to it after adding
> support for SA events, which likely would be near late November, early
> December. The framework in the ib_cm is there; I just need a way
somenath wrote:
> any idea when this stuff will get done?
As a rough estimate, Q1, though I will try to get to it after adding support
for
SA events, which likely would be near late November, early December. The
framework in the ib_cm is there; I just need a way to signal that failover has
oc
Sean Hefty wrote:
>> 1. does re-enabling Migration (as defined in vol1 of ib spec in
>> 17.2.8.1.4) work for you?
>> (I mean after the 1st path failure, you do lap/apr packet transfer)
>
>
> I believe that there's other issues that need to be fixed for this to
> fully work. The ib_cm uses the p
On Tue, Oct 24, 2006 at 04:36:32PM -0600, Matthew Wilcox wrote:
> On Tue, Oct 24, 2006 at 02:51:30PM -0700, Roland Dreier wrote:
> > > I think the right way to fix this is to ensure mmio write ordering in
> > > the pci_write_config_*() implementations. Like this.
> >
> > I'm happy to fix this i
Is this with a combination of TCP and UDP or just TCP?
Sean
Scott Weitzenkamp (sweitzen) wrote:
> We see 3.6 Gb/sec with IPoIB using RHEL4U4 2.6.9-42 x86_64 kernel on
> Dell PE1950 Woodcrest systems.
>
> In my testing, faster hardware is more important than newer kernels, but
> I don't try newer
Quoting r. Matthew Wilcox <[EMAIL PROTECTED]>:
> Subject: Re: Ordering between PCI config space writes and MMIO reads?
>
> On Tue, Oct 24, 2006 at 02:51:30PM -0700, Roland Dreier wrote:
> > > I think the right way to fix this is to ensure mmio write ordering in
> > > the pci_write_config_*() imp
> 1. does re-enabling Migration (as defined in vol1 of ib spec in
> 17.2.8.1.4) work for you?
> (I mean after the 1st path failure, you do lap/apr packet transfer)
I believe that there's other issues that need to be fixed for this to fully
work. The ib_cm uses the primary_path specified during
On Tue, Oct 24, 2006 at 04:36:32PM -0600, Matthew Wilcox wrote:
> On Tue, Oct 24, 2006 at 02:51:30PM -0700, Roland Dreier wrote:
> > > I think the right way to fix this is to ensure mmio write ordering in
> > > the pci_write_config_*() implementations. Like this.
> >
> > I'm happy to fix this i
Export the rdma_cm capabilities to userspace.
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]>
---
diff --git a/drivers/infiniband/core/Makefile b/drivers/infiniband/core/Makefile
index 8873b63..189e5d4 100644
--- a/drivers/infiniband/core/Makefile
+++ b/drivers/infiniband/core/Makefile
@@ -1,9 +1,11
Venkatesh Babu wrote:
> 1. Yes, I can rearm the alternate path by sending LAP and APR messages.
does the qpair go to rearm state just by sending LAP and APR messages?
I mean, you don't have to change the QP state to REARM explicitely?
>
> 2. I was sending some network traffic (netperf) while do
Add multicast QP support to the rdma_cm.
- Users identify multicast groups by using a multicast IP address.
Normal IP address translation services are used to map the address to
a local RDMA port.
- IB multicast group parameters are based on the ipoib broadcast group.
The MGID is derived usi
Add missing support for RDMA_PS_UDP. This allows the use of UD QPs
through the rdma_cm, which provides address translation services
over IB.
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]>
---
diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
index 416fee8..9bfd427 100644
-
Allow ULPs to transition to RTS before sending a REP. This allows the
ULP to respond to a received message if it arrives before the RTU or
communication established event.
Modify the RDMA CM to transition to RTS when sending a REP over IB, and
expose a new rdma_establish interface that a user can
1. Yes, I can rearm the alternate path by sending LAP and APR messages.
2. I was sending some network traffic (netperf) while doing these failovers.
VBabu
somenath wrote:
> hi Venkatesh:
>
> Two questions:
>
> 1. does re-enabling Migration (as defined in vol1 of ib spec in
> 17.2.8.1.4) work
From: Matthew Wilcox <[EMAIL PROTECTED]>
Date: Tue, 24 Oct 2006 16:36:32 -0600
> This is only really a problem for setup (when we program the BARs), so
> it seems silly to enforce an ordering at any other time. Reluctantly, I
> must disagree with Jeff -- drivers need to fix this.
One thing is th
When establishing a connection, users of the rdma_cm provide connection
parameters during calls to rdma_connect() and rdma_accept(). These
parameters are not given to the remote side during connection establishment.
The result is that the remote side does not know parameters such as
initiator_dept
There is a 1:1 correspondence between the qp_type for a connection and
the port space associated with an rdma_cm_id. Remove the qp_type from
the rdma_cm interface.
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]>
---
Removal makes the resulting userspace interface a little cleaner.
diff --git a/dri
hi Venkatesh:
Two questions:
1. does re-enabling Migration (as defined in vol1 of ib spec in
17.2.8.1.4) work for you?
(I mean after the 1st path failure, you do lap/apr packet transfer)
2. What applications you are testing with?
thanks, som.
Venkatesh Babu wrote:
>
> I have added couple of
On Tue, Oct 24, 2006 at 02:51:30PM -0700, Roland Dreier wrote:
> > I think the right way to fix this is to ensure mmio write ordering in
> > the pci_write_config_*() implementations. Like this.
>
> I'm happy to fix this in the PCI core and not force drivers to worry
> about this.
>
> John, can
The IB SA tracks multicast join / leave requests on a per port basis.
In order to support multiple users of the same multicast group from
the same port, we need to perform local reference counting on each
of the nodes.
Modify the ib_sa module to perform reference counting of multicast
join / leave
The following set of patches expand the rdma_cm support to include
UD and multicast, and expose the rdma_cm to userspace. I would like to
target the 2.6.20 kernel, but at least getting them into one or more
branches would be helpful for other developers to test against these
changes.
As mentioned
Roland Dreier wrote:
> > I think the right way to fix this is to ensure mmio write ordering in
> > the pci_write_config_*() implementations. Like this.
>
> I'm happy to fix this in the PCI core and not force drivers to worry
> about this.
>
> John, can you confirm that this patch fixes the iss
> I think the right way to fix this is to ensure mmio write ordering in
> the pci_write_config_*() implementations. Like this.
I'm happy to fix this in the PCI core and not force drivers to worry
about this.
John, can you confirm that this patch fixes the issue for you?
Thanks,
Roland
On Tue, Oct 24, 2006 at 03:22:10PM -0400, Jeff Garzik wrote:
> The PCI config APIs have traditionally enforced very strong ordering.
> Heck, the PCI config APIs often take a spinlock on each read or write;
> so they are definitely not intended to be as fast as MMIO.
s/often/always/. It's implemen
On Tue, Oct 24, 2006 at 02:29:47PM -0700, Roland Dreier wrote:
> > It is good to be conservative in this area. Some AMD chipsets at least
> > had ordering problems with some configurations in the K7 era.
>
> Could you expand a little? Do you mean that the arch implementation
> of pci_write_conf
> It is good to be conservative in this area. Some AMD chipsets at least
> had ordering problems with some configurations in the K7 era.
Could you expand a little? Do you mean that the arch implementation
of pci_write_config_xxx() should have extra barriers, or that drivers
should do belt-and-s
Ar Maw, 2006-10-24 am 12:13 -0700, ysgrifennodd Roland Dreier:
> 1) Is this something that should be fixed in the driver? The PCI
> spec allows MMIO cycles to start before an earlier config cycle
> completed, but do we want to expose this fact to drivers? Would
> it be better for ia6
On IA64 there are two memory barriers mf and mf.a. To protect against
the scenario below mf.a (slower of course) would be required.
JW
- Original Message -
From: "Roland Dreier" <[EMAIL PROTECTED]>
To: <[EMAIL PROTECTED]>;
Cc: ;
Sent: Tuesday, October 24, 2006 2:13 PM
Subjec
Thanks, queued for 2.6.19
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Thanks, queued for 2.6.20
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
http://openib.org/bugzilla/show_bug.cgi?id=286
Summary: "ifconfig ib# down" hangs telnet connection-- NETDEV
WATCHDOG: ib0: transmit timed out
Product: OpenFabrics Linux
Version: 1.0rc6
Platform: X86
OS/Version: SLES 9
From: Dotan Barak <[EMAIL PROTECTED]>
The qp_access_flags are for remote access permissions only,
so local write is an invalid value.
Signed-off-by: Dotan Barak <[EMAIL PROTECTED]>
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]>
---
Current drivers appear to ignore this flag being set when the QP i
We see 3.6 Gb/sec with IPoIB using RHEL4U4 2.6.9-42 x86_64 kernel on
Dell PE1950 Woodcrest systems.
In my testing, faster hardware is more important than newer kernels, but
I don't try newer kernels much.
Scott Weitzenkamp
SQA and Release Manager
Server Virtualization Business Unit
Cisco Systems
From: Krishna Kumar <[EMAIL PROTECTED]>
rdma_bind_addr() leaks a cma_dev reference count in failure case.
Signed-off-by: Krishna Kumar <[EMAIL PROTECTED]>
Signed-off-by: Sean Hefty <[EMAIL PROTECTED]>
---
Modified from Krishna's patch to drop use of did_acquire_dev flag.
Because this bug is in er
On Tue, Oct 24, 2006 at 08:35:18AM -0500, Sean Hubbell wrote:
> We are currently looking at the new tickless kernel. Do you have one
> that you recommend?
The main one to less-recommend is 2.6.9-based kernels, those are the
slowest at TCP. Modern kernels, like the ones you see in Fedora 4 and
up
Quoting r. Parks Fields <[EMAIL PROTECTED]>:
> Subject: Kernel for IPoIB Question
>
>
> Sincethe to[pic of kernel affecting the performance, which kernel is
> now giving the best performance. We are trying to choose the next
> kernel for our clusters and wanted to choose one that does well wit
On Tue, Oct 24, 2006 at 12:13:19PM -0700, Roland Dreier wrote:
> 1) Is this something that should be fixed in the driver? The PCI
> spec allows MMIO cycles to start before an earlier config cycle
> completed, but do we want to expose this fact to drivers? Would
> it be better for ia6
John Partridge found an interesting bug involving mthca (Mellanox
InfiniBand HCA driver) on IA64/Altix systems. Basically, during
initialization, mthca does:
- do some config writes, including enabling BARs
- then start a firmware command
- read an MMIO register from a BAR (to check
Thanks -- just for the future:
> Signed-off-by: John Partridge [EMAIL PROTECTED]
The email address should really be in angle brackets like:
Signed-off-by: John Partridge <[EMAIL PROTECTED]>
No need to resend this time, but please fix it next time.
thanks,
Roland
___
Sincethe to[pic of kernel affecting the performance, which kernel is
now giving the best performance. We are trying to choose the next
kernel for our clusters and wanted to choose one that does well with
IPoIB. I know 2.6.16 was not doing well, even with 10G myrinet cards.
What is recommended
At 10:00 PM 10/23/2006, Greg Lindahl wrote:
>On Mon, Oct 23, 2006 at 07:53:06AM -0500, Hubbell, Sean C
>Contractor/Decibel wrote:
>
> > I currently have several applications that uses a legacy IPv4 protocol
> > and I use IPoIB to utilize my infiniband network which works great. I
> > have comple
Fixing signed/unsigned data types problem (discovered on Windows)
--
Yevgeny
Signed-off-by: Yevgeny Kliteynik <[EMAIL PROTECTED]>
Index: osmtest/osmt_multicast.c
===
--- osmtest/osmt_multicast.c(revision 9948)
+++ osmtest/osmt_
Roland Dreier wrote:
I thought I had everything I needed -- but actually could you send a
new copy of the patch along with a Signed-off-by: line?
OK here it is. A file copy is attached too in case my mailer messed up the
format :-
mthca_reset.c: Fix a non-blocking Config Write that needs flus
Thanks,
Applied.
Regards,
Vladimir
> -Original Message-
> From: Or Gerlitz [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 24, 2006 4:36 PM
> To: Vladimir Sokolovsky
> Cc: openib-general@openib.org
> Subject: [PATCH] fix two typos in ipoib HA documentation
>
> diff -up
> SOURCES/o
diff -up SOURCES/openib-1.1/src/userspace/ipoibtools-orig/README
SOURCES/openib-1.1/src/userspace/ipoibtools/README
--- SOURCES/openib-1.1/src/userspace/ipoibtools-orig/README 2006-10-19
16:21:08.0 +0200
+++ SOURCES/openib-1.1/src/userspace/ipoibtools/README 2006-10-24
17:35:29.
> -Original Message-
> From: Or Gerlitz [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 24, 2006 3:43 PM
> To: Vladimir Sokolovsky
> Cc: openib-general@openib.org
> Subject: Re: problems using the ipoib ha package of RH4 U3
>
> Vlad,
>
> On a related issue, i think it was mentioned a
Thanks Eli.So the switch is incrementing the rcvswrelay counter when it sends the broadcast back through the original port. This doesn't seem to be correct behavior, it makes that counter unreliable.
On 10/24/06, Eli Cohen <[EMAIL PROTECTED]> wrote:
Todd,This does not look like an error. The first
Greg Lindahl wrote:
> On Mon, Oct 23, 2006 at 07:53:06AM -0500, Hubbell, Sean C Contractor/Decibel
> wrote:
>
>
>> I currently have several applications that uses a legacy IPv4 protocol
>> and I use IPoIB to utilize my infiniband network which works great. I
>> have completed some timing and
Vlad,
On a related issue, i think it was mentioned at some point that the
ipoib ha package does not support devices which use the non standard
pkey, is it correct? why?
My thinking was that if the pkey 8001 is configured for both port 1 and
port2 of the hca, the user hacks /etc/init.d/openibd
Vladimir Sokolovsky wrote:
> The HA service (ipoib_ha.pl) is looking for "NO-CARRIER" in the "ip"
> utility output.
> Ipoib_ha.pl use an updated "ip" utility which is a part of ipoibtools
> RPM and located under 'prefix'/utils directory.
OK, when using the ipoib_ha.pl in the ipoibtools rpm framewo
Michael S. Tsirkin wrote:
> Quoting vishal <[EMAIL PROTECTED]>:
>> Thread model: posix
>> gcc version 4.1.0 (SUSE Linux)
>> configure:2471: $? = 0
>> configure:2473: gcc -V &5
>> gcc: '-V' option must have argument
>> configure:2476: $? = 1
>> configure:2499: checking for C compiler default output
The HA service (ipoib_ha.pl) is looking for "NO-CARRIER" in the "ip"
utility output.
Ipoib_ha.pl use an updated "ip" utility which is a part of ipoibtools
RPM and located under 'prefix'/utils directory.
The fail-over flow is the following:
On every event received from 'ip monitor link' check if t
Quoting vishal <[EMAIL PROTECTED]>:
> Thread model: posix
> gcc version 4.1.0 (SUSE Linux)
> configure:2471: $? = 0
> configure:2473: gcc -V &5
> gcc: '-V' option must have argument
> configure:2476: $? = 1
> configure:2499: checking for C compiler default output file name
> configure:2502: gcc -m
Vladimir Sokolovsky wrote:
> Try to disconnect the port 1 of HCA from IB subnet and then check if ib1
> became active.
OK, i will be able to do it later today, anyway, something bothers me
here: the script is based on parsing the output of
$ /sbin/ip monitor link
and this output should be the s
Hi Or,
Try to disconnect the port 1 of HCA from IB subnet and then check if ib1
became active.
Regards,
Vladimir
> -Original Message-
> From: Or Gerlitz [mailto:[EMAIL PROTECTED]
> Sent: Tuesday, October 24, 2006 10:36 AM
> To: Vladimir Sokolovsky
> Cc: openib-general@openib.org
> Subj
Hi Vlad,
I am trying to test the ipoib ha package on OFED 1.1 over RH4 U3 and does
not manage to get it working fine. I am running the following commands:
ifup ib0
ifup ib1
ipoib_ha_log="/var/log/ipoib_ha.log"
sleep 2
./ipoib_ha.pl -p ib0 -s ib1 --with-arping --with-multicast -v > ${ipoib_ha_log}
Todd,
This does not look like an error. The first arp is a broadcast
(qpn=ff) so it is received in at the sending interface and is
dropped. The second on is a unicast arp (qpn=0x000404) so it is not
received at the local interface.
On Mon, 2006-10-23 at 13:48 -0600, Todd Bowman wrote:
> Usin
Quoting r. Jason Gunthorpe <[EMAIL PROTECTED]>:
> After thinking about it some more, there are more cases in the kernel
> than just the one in mthca. For instance there are paths in the pci
> core than manipulate BAR registers without a barrier. It would be bad
> to change a bar via config write an
Hi Kyle!
> And, setting the debug_level flag definitely caused the server to not
> respond... I rebooted and tried it again, same thing, setting the
> debug_level flag causes the server to crash. (I can still login, but
> cannot execute anything, e.g. 'ls', it seems all the cpu's are spinning)
> p
67 matches
Mail list logo
