hello,
i'm writing sample kernel ulp driver to get me acquainted
with openib stack on linux kernel 2.6.16.2 (fedora 5) with
openib gen 2 stack checkout from openib.org website.
the setup is two nodes with point-to-point connection, viz.
primary & secondary node. the secondary node starts in a
listen mode, until primary node makes a connection and start
exchanging messages.
the problem that i am running into is that the ib_send_cm_req
api fails with errorno 22. i'm using local id to make the
connection on port 1. ib_send_cm_req() api calls function
cm_init_av_by_path(), which calls ib_find_cached_gid().
function ib_find_cached_gid() fails because it can't locate
cached gid in device's cache table. below is full control
flow from both primary & secondary node.
priamry node secondary node
-------------- -----------------
ib_register_client()
using active port = 1
ib_create_cm_id()
ib_cm_listen()
listening .... (waiting)
ib_register_client()
using active port = 1
ib_create_cm_id()
ib_alloc_pd()
ib_create_cq()
ib_req_notify_cq()
ib_get_dma_mr()
ib_create_qp()
ib_query_gid()
source.lid = 0x1
dest.lid = 0x2
ib_sa_path_rec_get()
sa_path_rec handler returned success
ib_send_cm_req()
ib_send_cm_req() failed with error -22
how would update device's cache to get cached gid? am i missing
any steps?
here is output from ib* commands:
from primary node:
[EMAIL PROTECTED]:~:23> sminfo
sminfo: sm lid 0x2 sm guid 0x5ad0000030655, activity count 1707
priority 1 state SMINFO_MASTER 3
[EMAIL PROTECTED]:~:26> ibhosts
Ca : 0x0005ad0000030654 ports 2 "Topspin HCA"
Ca : 0x0005ad0000030860 ports 2 "Topspin HCA"
[EMAIL PROTECTED]:~:28> ibstatus
Infiniband device 'mthca0' port 1 status:
default gid: fe80:0000:0000:0000:0005:ad00:0003:0861
base lid: 0x1
sm lid: 0x2
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 10 Gb/sec (4X)
Infiniband device 'mthca0' port 2 status:
default gid: fe80:0000:0000:0000:0005:ad00:0003:0862
base lid: 0x0
sm lid: 0x0
state: 1: DOWN
phys state: 2: Polling
rate: 2.5 Gb/sec (1X)
[EMAIL PROTECTED]:~:29> ibnetdiscover
#
# Topology file: generated on Tue Apr 18 12:36:39 2006
#
# Max of 1 hops discovered
# Initiated from node 0005ad0000030860 port 0005ad0000030861
vendid=0x5ad
devid=0x5a44
sysimgguid=0x5ad000100d050
caguid=0x5ad0000030654
Ca 2 "H-0005ad0000030654" # Topspin HCA
[1] "H-0005ad0000030860"[1] # lid 2 lmc 0
vendid=0x5ad
devid=0x5a44
sysimgguid=0x5ad000100d050
caguid=0x5ad0000030860
Ca 2 "H-0005ad0000030860" # Topspin HCA
[1] "H-0005ad0000030654"[1] # lid 1 lmc
from secondary node:
[EMAIL PROTECTED]:~:8> sminfo
sminfo: sm lid 0x2 sm guid 0x5ad0000030655, activity count 1738
priority 1 state SMINFO_MASTER 3
[EMAIL PROTECTED]:~:15> ibhosts
Ca : 0x0005ad0000030860 ports 2 "Topspin HCA"
Ca : 0x0005ad0000030654 ports 2 "Topspin HCA"
[EMAIL PROTECTED]:~:16> ibstatus
Infiniband device 'mthca0' port 1 status:
default gid: fe80:0000:0000:0000:0005:ad00:0003:0655
base lid: 0x2
sm lid: 0x2
state: 4: ACTIVE
phys state: 5: LinkUp
rate: 10 Gb/sec (4X)
Infiniband device 'mthca0' port 2 status:
default gid: fe80:0000:0000:0000:0005:ad00:0003:0656
base lid: 0x0
sm lid: 0x0
state: 1: DOWN
phys state: 2: Polling
rate: 2.5 Gb/sec (1X)
[EMAIL PROTECTED]:~:17> ibnetdiscover
#
# Topology file: generated on Tue Apr 18 12:36:15 2006
#
# Max of 1 hops discovered
# Initiated from node 0005ad0000030654 port 0005ad0000030655
vendid=0x5ad
devid=0x5a44
sysimgguid=0x5ad000100d050
caguid=0x5ad0000030860
Ca 2 "H-0005ad0000030860" # Topspin HCA
[1] "H-0005ad0000030654"[1] # lid 1 lmc 0
vendid=0x5ad
devid=0x5a44
sysimgguid=0x5ad000100d050
caguid=0x5ad0000030654
Ca 2 "H-0005ad0000030654" # Topspin HCA
[1] "H-0005ad0000030860"[1] # lid 2 lmc 0
do you know what's wrong?
thanks,
susan
_______________________________________________
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
