On Thu, Sep 29, 2005 at 09:42:27AM -0700, Roland Dreier wrote:
> Vu> Have you reviewed the FMR? What your take on Christoph's point
> Vu> about the high bit of dma_address_ts are used by some
> Vu> platforms IOMMU - I think that it's OK since FMR code only
> Vu> touch the lower bit
Hi All,
I am trying to use MAD services from a user-land application. Basically
I want to do few things like registering to traps/generating Multicast
Send requests from user-space using Userspace VAPI.
After reading through some code in osmtest.c, I couldn't figure out
whether it is using the
I'd like to get some feedback about the following design change to the CMA.
Currently, a user receives a device GUID and port number as part of address
resolution. The user matches the GUID to an existing device and creates a
cma_id associated with that device.
I'm considering the following alte
>I think the usage model is the following: you have some magic device
>that has an IB port on one side and "something else" on the other
>side. Think of something like a gateway that talks SDP on the IB side
>and TCP/IP on the other side.
>
>You configure your IPoIB routing so that this magic devi
Sean> Can you explain how RDMA works in this case? This is simply
Sean> performing IP routing, and not IB routing, correct? Are you
Sean> referring to a protocol running on top of IP or IB directly?
Sean> Is the router establishing a second reliable connection on
Sean> the bac
$B(B $B(#(!(!($(#(!(!($(B
$B(B $B("40A4("("L5NA("(B
$B(B $B(&(!(!(%(&(!(!(%(B
$B(#(!(!($(#(!(!($(#(!(!($(B
$B("M-L>("("?M!*("(">R2p("(B
Roland,
Vu> Since all the tuned parameter are target-centralized (passing
Vu> in when add new target) I think about moving FMR resources
Vu> (size, max_page...) ie. fmr_pool into srp_target_port
Vu> struct. Each newly added target will have their own customized
Vu> FMR po
ScanMail for Microsoft Exchange has blocked an attachment.
Sender = [EMAIL PROTECTED]
Recipient(s) = openib-general@openib.org
Subject = [openib-general] fnfetceueegbftqoi
Scanning time = 9/29/2005 3:49:02 PM
Action on file blocking:
The attachment email-details.zip matches the file blocking sett
ScanMail for Microsoft Exchange has detected virus-infected attachment(s).
Sender = [EMAIL PROTECTED]
Recipient(s) = openib-general@openib.org
Subject = [openib-general] fnfetceueegbftqoi
Scanning time = 9/29/2005 5:48:24 PM
Engine/Pattern = 7.510-1002/2.865.00
Action on virus found:
The attachme
Yaron Haviv wrote:
The RC connection is established with the DGID of the router (it's the
equivalent of a MAC address and its ok), the ServiceID + private data in
the case of SDP or iSER (or NFS-R assuming the IBTA proposal will pass)
also contains info on the REAL destination IP that can be used
On Thu, 2005-09-29 at 14:58, Sean Hefty wrote:
> As an aside, do the IPoIB subnets all fall into the same broadcast domain?
That would depend on the PKey, right ?
-- Hal
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailm
On Thu, 2005-09-29 at 16:18, Sean Hubbell wrote:
> Hal Rosenstock wrote:
>
> >Hi Sean,
> >
> >On Thu, 2005-09-29 at 14:03, Sean Hubbell wrote:
> >
> >
> >> We are having problems with loading the mthca module running Linux
> >>2.6.13 Kernel with the svn repository pulled yesterday afternoon wh
> -Original Message-
> From: Sean Hefty [mailto:[EMAIL PROTECTED]
> Sent: Thursday, September 29, 2005 5:16 PM
> To: Yaron Haviv
> Cc: Hal Rosenstock; Openib
> Subject: Re: [openib-general] [RFC] IB address translation using ARP
>
> Yaron Haviv wrote:
> > 4. send an arp on the net device f
On Thursday 29 September 2005 23:44, you wrote:
> I would try 2 nodes point to point. If that works, then
> I suspect the switch. I did see an issue with one of our MT2400 switches
> with IPoIB connectivity. We replaced the switch and it
> seemed to fix the problem, so we did not investigate furthe
Hal wrote,
>> > Also, what is your HCA firmware version ?
>>
>> $ cat /sys/class/infiniband/mthca0/fw_ver
>> 3.3.3
>That's the most recent.
>-- Hal
I would try 2 nodes point to point. If that works, then
I suspect the switch. I did see an issue with one of our MT2400 switches
with IPoIB connec
On Thu, 2005-09-29 at 17:01, Thomas Moschny wrote:
> On Thursday 29 September 2005 22:08, you wrote:
> > On Thu, 2005-09-29 at 16:01, Thomas Moschny wrote:
> > > Maybe a switch firmware problem? We once observed a complete switch
> > > lockup that shut down all communication.
> >
> > Could be. Do y
Yaron Haviv wrote:
4. send an arp on the net device find destination MAC
Note the destination IP in the ARP phase is either the REAL destination
IP in case of a local subnet, or the IP router IP address in case of a
gateway/router.
5. issue a path record between the source/dest GIDs (DGID taken
> -Original Message-
> From: [EMAIL PROTECTED] [mailto:openib-general-
> [EMAIL PROTECTED] On Behalf Of Sean Hefty
> Sent: Thursday, September 29, 2005 2:58 PM
> To: Hal Rosenstock
> Cc: Openib
> Subject: Re: [openib-general] [RFC] IB address translation using ARP
>
> Hal Rosenstock wrote:
On Thursday 29 September 2005 22:08, you wrote:
> On Thu, 2005-09-29 at 16:01, Thomas Moschny wrote:
> > Maybe a switch firmware problem? We once observed a complete switch
> > lockup that shut down all communication.
>
> Could be. Do you know what rev of firmware you are running ? Is it 0.7.0
> ?
>Also, what is your HCA firmware version ?
>-- Hal
Good point. I have seen IPoIB connectivity issues in the past
when dealing with down rev FW.
I just re-tested IPoIB on my IPF machines and they seem to
work OK for me. I suspect either the HCA FW rev or the switch.
[EMAIL PROTECTED] SPECS]# ca
Thomas wrote,
>Yes, it's a single MTS-2400 with 24 ports.
>Maybe a switch firmware problem? We once observed a complete switch
lockup
>that shut down all communication.
If you suspect a bad switch, do you have another one you could try ?
or you can try to direct connect a couple of nodes.
woo
Hal Rosenstock wrote:
Hi Sean,
On Thu, 2005-09-29 at 14:03, Sean Hubbell wrote:
We are having problems with loading the mthca module running Linux
2.6.13 Kernel with the svn repository pulled yesterday afternoon when we
are booting.
What problem is occuring with loading mthca duri
On Thu, 2005-09-29 at 16:01, Thomas Moschny wrote:
> Maybe a switch firmware problem? We once observed a complete switch lockup
> that shut down all communication.
Could be. Do you know what rev of firmware you are running ? Is it 0.7.0
? (MTS-2400 is Anafa-2 based).
Also, what is your HCA firmw
On Thursday 29 September 2005 21:25, Hal Rosenstock wrote:
> In the log, I do see several nodes successfully join the IPoIB broadcast
> group and the multicast tree for this got setup (I didn't actually
> validate the tree itself).
>
> PortGid.0xfe80 : 0x0002c9021575
On Thu, 2005-09-29 at 12:44 -0700, Roland Dreier wrote:
> Just so we don't forget, we need to add something like this to the
> uverbs code on your branch.
Yup. It's on my TODO list. I need to take care of some unrelated stuff
today, but I'll get back to working on all the feedback I've received
Just so we don't forget, we need to add something like this to the
uverbs code on your branch.
(I didn't even compile test but you get the idea...)
--- src/linux-kernel/infiniband/core/uverbs_main.c (revision 3620)
+++ src/linux-kernel/infiniband/core/uverbs_main.c (working copy)
@@ -11
On Thu, 2005-09-29 at 15:11, Thomas Moschny wrote:
> On Thursday 29 September 2005 20:32, you wrote:
> > Can you ping the subnet broadcast address (e.g. ping -b 192.168.0.255 if
> > the ib0 is 192.168.0.x) ?
>
> The only answer I get is from the sender itself:
>
> $ ping -b 192.168.204.255
> WAR
Pradeep Satyanarayana wrote:
I am not very clear about this -what happens when the ARP cache is empty
on the active side and IPoIB module has also been unloaded? Is the IP
translation information stored somewhere by the CMA, permitting the
application to continue?
If the route is not found in
I am not very clear about this -what happens when the ARP cache is empty on the active side and IPoIB module has also been unloaded? Is the IP translation information stored somewhere by the CMA, permitting the application to continue?
Pradeep
[EMAIL PROTECTED]
[EMAIL PROTECTED] wrote on 09/29/2
Hal Rosenstock wrote:
I'm struggling with understanding how translation can even occur in this case.
What DGID is used when querying for the path record, and how is it obtained?
Isn't it the DGID of the next hop IP router ? (I suppose in the case of
multiple IPoIB subnets on the same IB subnet,
On Thu, 2005-09-29 at 14:38, Sean Hefty wrote:
> Hal Rosenstock wrote:
> >>If we restrict IB to a single subnet, do we need to worry about IP routing?
> >> My
> >>assumption was no. Is this an invalid assumption?
> >
> > I think so. There is nothing that precludes having multiple IPoIB
> > subn
Hal Rosenstock wrote:
If we restrict IB to a single subnet, do we need to worry about IP routing? My
assumption was no. Is this an invalid assumption?
I think so. There is nothing that precludes having multiple IPoIB
subnets on the same IB subnet.
This seems similar to having multiple IP su
On Thu, 2005-09-29 at 14:00, Thomas Moschny wrote:
> Hi,
>
> Do I have to do something special in order to configure IPoverIB besides
> from loading the ib_ipoib kernel module (and it's dependencies), and calling
> ifconfig ib0 up?
No, that should be sufficient.
> On our machines, the module
Hi Sean,
On Thu, 2005-09-29 at 14:03, Sean Hubbell wrote:
> We are having problems with loading the mthca module running Linux
> 2.6.13 Kernel with the svn repository pulled yesterday afternoon when we
> are booting.
What problem is occuring with loading mthca during boot ?
> Once we boot,
On Thu, 2005-09-29 at 12:57, Sean Hefty wrote:
> Hal Rosenstock wrote:
> Would it require global routing,
> >>>
> >>>Yes.
> >>
> >>If it requires global routing of IB, then I think that we should defer it
> >>until
> >>global routing is available. At least this was my original thinking.
> >
This worked. On 9/29/05, Sean Hefty <[EMAIL PROTECTED]> wrote:
Todd Bowman wrote:> I've tracked the error to ib_cm_listen:...> These are the modules I have loaded:>> ib_sdp 93792 0Can you unload SDP and see if you see the same issue? (SDP reserves a range of
listen addresses that
Hal,
We are having problems with loading the mthca module running Linux
2.6.13 Kernel with the svn repository pulled yesterday afternoon when we
are booting. Once we boot, we get passed this but we have problems when
attempting to run ibping. The log files are 1.4M. Would you like me to
send
Hi,
Do I have to do something special in order to configure IPoverIB besides
from loading the ib_ipoib kernel module (and it's dependencies), and calling
ifconfig ib0 up?
On our machines, the modules load fine, opensm runs, ports are in active
state, no error messages from ifconfig. However,
On Thu, 29 Sep 2005, Arlin Davis wrote:
> Yes. I am not an autogen/configure expert so I will defer.
>
> In the meantime we need to change things around to build with the latest
> changes with no OS_VENDER.
Committed in revision 3619.
___
openib-gen
Todd Bowman wrote:
I am runing udapl on 32bit intel and running into this error:
setup_listener(conn=0x8060008 cm_id=134611368)
destroy_cm_id: conn 0x8060008 id 134611368
--> dapl_psp_create setup_conn_listener failed: 3
20664 Error dat_psp_create: DAT_INSUFFICIENT_RESOURCES
20664 Error co
Todd Bowman wrote:
I've tracked the error to ib_cm_listen:
...
These are the modules I have loaded:
ib_sdp 93792 0
Can you unload SDP and see if you see the same issue? (SDP reserves a range of
listen addresses that can cause other listen calls to fail. Not sure it's the
James Lentini wrote:
of the CMA API is fine.
However, ATS is needed for interoperability with existing kDAPL and
uDAPL installations.
Note that we lose interoperability between the CMA and kDAPL.
- Sean
___
openib-general mailing list
openib-genera
>
>Committed in revision 3606 except for this:
>
>> Index: dapl/udapl/Makefile
>> ===
>> --- dapl/udapl/Makefile (revision 3565)
>> +++ dapl/udapl/Makefile (working copy)
>> @@ -57,6 +57,13 @@
>> endif
>>
>> #
>> +# Set up
On Wed, 28 Sep 2005, Sean Hefty wrote:
> Roland Dreier wrote:
> > Guy> But the main question is: does openib wants to support ATS
> > Guy> arp ? do we also want to support ATS
> > Guy> registration/deregistration ? openib can support, for
> > Guy> example, only the ATS arp and r
Michael S. Tsirkin wrote:
I'd suggest you also take a look at sdp_link.c - I hear that's where
ib_at code came from, and I think it does some things in a better way -
such as only keeping device reference around for a short while only.
Thanks for pointing this out. I wasn't aware that SDP did
I am runing udapl on 32bit intel and running into this error:
setup_listener(conn=0x8060008 cm_id=134611368)
destroy_cm_id: conn 0x8060008 id 134611368
--> dapl_psp_create setup_conn_listener failed: 3
20664 Error dat_psp_create: DAT_INSUFFICIENT_RESOURCES
20664 Error connect_ep: DAT_INSUFFIC
Hal Rosenstock wrote:
Would it require global routing,
Yes.
If it requires global routing of IB, then I think that we should defer it until
global routing is available. At least this was my original thinking.
I was referring to IP not IB routing.
If we restrict IB to a single subnet, d
On Thu, 2005-09-29 at 11:48, Roland Dreier wrote:
> Hal> Hi, Does anyone know what the Linux routing code does in
> Hal> terms of selecting an outgoing interface when multiple net
> Hal> devices have been configured with the same IP address ?
>
> It follows the routing table. If you a
On Thu, 2005-09-29 at 12:40, Sean Hefty wrote:
> Hal Rosenstock wrote:
> >>How does that map to the IB subnet?
> >
> > or IP subnet in the case of iWARP, right ? It's still an outgoing
> > interface just more than 1 IP hop away.
>
> The intent of the module is only to deal with IB. Although, it
Vu> Since all the tuned parameter are target-centralized (passing
Vu> in when add new target) I think about moving FMR resources
Vu> (size, max_page...) ie. fmr_pool into srp_target_port
Vu> struct. Each newly added target will have their own customized
Vu> FMR pool.
That makes
Hal Rosenstock wrote:
How does that map to the IB subnet?
or IP subnet in the case of iWARP, right ? It's still an outgoing
interface just more than 1 IP hop away.
The intent of the module is only to deal with IB. Although, it seems generic
enough that it could return hardware addresses for
Roland Dreier wrote:
Vu, you pointed out that the current SRP code might look at an IU that
it sent after that IU has been reused for a different command. I
realized that a simple fix for this is just to keep the DMA address
(the only thing we look at in the IU) in the request structure.
Just
On Thu, 2005-09-29 at 12:05, Sean Hefty wrote:
> > What happens if the destination IP address is a local one ? I think
> > there is some missing code here.
>
> I think there's code in at.c to handle that case that could be re-used.
Yes. This is the code related to ip_dev_find which has been discu
Thomas wrote,
>Hi,
>while building a 2.6.9-11.EL kernel with the patches from
>gen2/branches/backport-to-2.6.9 applied, I found a minor glitch:
>CONFIG_INFINIBAND_IPOIB_DEBUG should not be there. The debug part
cannot be
>build, because debugfs is missing. It seems to be introduced in 2.6.11.
On Wed, Sep 28, 2005 at 06:44:55AM -0700, Roland Dreier wrote:
> Greg> I didn't think that git pulls were going to be allowed from
> Greg> subsystem maintainers after -rc1 came out. After that,
> Greg> patches by email were required to be sent, not git pulls.
> Greg> This does caus
Hi,
while building a 2.6.9-11.EL kernel with the patches from
gen2/branches/backport-to-2.6.9 applied, I found a minor glitch:
CONFIG_INFINIBAND_IPOIB_DEBUG should not be there. The debug part cannot be
build, because debugfs is missing. It seems to be introduced in 2.6.11.
- Thomas
pgpLw1cO
Michael S. Tsirkin wrote:
BTW, one idea of mine that some people apparently were interested in,
is adding an option to do ARP resolution on the passive side after REQ is
received, as a weak security measure.
Is there still interest in that?
I don't think that there's anything to restrict this.
Hal Rosenstock wrote:
(We should be able to pull that from ib_at.)
or sdp_link which has the more temporal netdev references currently :-)
I will look at both. Thanks.
The API is similar to the route
portion of ib_at, but corrects issues with canceling requests.
What are you referring to
Sean Hefty wrote:
struct ib_addr_svc* ib_addr_create_svc(void *context, ib_addr_handler handler);
void ib_addr_destroy_svc(struct ib_addr_svc *svc);
On second thought, I think this can be done without the need to create/destroy a
service without changing the functionality.
void ib_addr_can
Hal> Hi, Does anyone know what the Linux routing code does in
Hal> terms of selecting an outgoing interface when multiple net
Hal> devices have been configured with the same IP address ?
It follows the routing table. If you assign the same local IP address
to multiple interfaces then
On Thu, 29 Sep 2005, Michael S. Tsirkin wrote:
> BTW, one idea of mine that some people apparently were interested in,
> is adding an option to do ARP resolution on the passive side after REQ is
> received, as a weak security measure.
>
> Is there still interest in that?
Given that the ARP res
On Thu, 2005-09-29 at 09:59, Hal Rosenstock wrote:
> On Wed, 2005-09-28 at 21:26, Sean Hefty wrote:
> > Here's a first attempt at an API / implementation (that compiles only) for
> > an address translation module for IB using ARP. The code should check the
> > ARP cache for information, but is mis
Hi,
Does anyone know what the Linux routing code does in terms of selecting
an outgoing interface when multiple net devices have been configured
with the same IP address ?
Does it always select the first one or is there some other algorithm in
play here ?
Thanks.
-- Hal
___
Sean Hefty wrote:
I don't object to the name, just combining the current functionality
that ib_at tries to provide into a single abstraction. I think that the
disagreement is what functionality a core address translation module
should provide.
...
> If other functionality from ib_at is needed
On Wed, 2005-09-28 at 21:26, Sean Hefty wrote:
> Here's a first attempt at an API / implementation (that compiles only) for
> an address translation module for IB using ARP. The code should check the
> ARP cache for information, but is missing the actual ARP processing.
Where would the path recor
Hi
We are running here mvapich gen 2 downloaded from osu about Sept. 12.,
with 2.6.12.6 from kernel.org, Fedora core 4 (gcc 4.0.0) and the IB tree
from openib.org downloaded about 3 weeks ago, without any subsequent
patches added.
The hardware of the cluster is somewhat special: We use AMD dual
$B!|5U%G%j%X%k$C$F$4B8CN$G$9$+!)(Bhttp://1191.jp/lvhost/index.html
[EMAIL PROTECTED]@$7$?$7$^$9$H!"%[%F%k!"$b$7$/[EMAIL PROTECTED]<+Bp$K$F!"[EMAIL
PROTECTED],[EMAIL PROTECTED]&;[EMAIL PROTECTED](B
$B!|IaDL$N%G%j%X%k$H$O0[$J$k$N$O=P8~$/B&$b$"$kDxEY!"Ajhttp://1191.jp/lvhost/index.html
$B!|6b3[LL
I found my problem, which had to do with incorrect library loading
(LD_LIBRARY_PATH). There was a different mvapich (0.9.5) being loaded
instead of the new one. Perhaps a version check with a nice error
message could help in the future.
However, mvapich gen2 works just fine according to my prelimi
Hi!
In order to build the openib-userspace rpm on rhel4/ia64, I had to slightly
modify the specfile. Here's the patch:
--- openib-userspace.spec.orig 2005-09-29 15:02:42.961550300 +0200
+++ openib-userspace.spec 2005-09-29 15:03:09.500216310 +0200
@@ -124,7 +124,7 @@
%config /etc/modprob
On Wed, 2005-09-28 at 16:01, James Lentini wrote:
> On Wed, 28 Sep 2005, Hal Rosenstock wrote:
>
> > > What are the contents of /etc/cpuinfo and
> > > /etc/meminfo on this system?
> >
> > Those files don't exist on my machine.
>
> Sorry, I meant /proc/cpuinfo
processor : 0
vendor_id
Hi Eitan,
On Thu, 2005-09-29 at 06:40, Eitan Zahavi wrote:
> Hi Hal,
>
> I have updated IBIS :
>
> 1. Support sm_key, v_key
What's v_key ?
> and m_key (yes the last one is temporary until we
>
>will need to support m_key(lid), m_key(dr path), m_key(guid).
>
>to set them (anytime du
Title: IBIS modification
Hi Hal,
I have updated IBIS :
1. Support sm_key, v_key and m_key (yes the last one is temporary until we
will need to support m_key(lid), m_key(dr path), m_key(guid).
to set them (anytime during the flow) you can ibis_opts configure -sm_key
Example:
% ibi
Quoting r. Sean Hefty <[EMAIL PROTECTED]>:
> Subject: [RFC] IB address translation using ARP
> The code should check the
> ARP cache for information, but is missing the actual ARP processing.
> (We should be able to pull that from ib_at.)
I'd suggest you also take a look at sdp_link.c - I hear tha
73 matches
Mail list logo
