Re: [openib-general] cannot instal ofed-1.2 kernel rpm on 2.6.20.1
I opened bug 399 to track this. I also opened bug 398 because I got an error installing opensm with this same OFED-1.2 build. Steve. On Tue, 2007-02-27 at 16:43 -0600, Steve Wise wrote: > I built the ofed 1.2 rpms from the OFED-1.2-20070227-0602 build and the > kernel rpm fails to install on a 2.6.20.1 kernel: > > vic13:/usr/local/src/OFED-1.2-20070227-0602/RPMS/sles-release-10-15.2 # rpm > -U kernel-ib-1.2-2.6.20.1.x86_64.rpm > error: Failed dependencies: > ksym(schedule) = 1000e51 is needed by kernel-ib-1.2-2.6.20.1.x86_64 > ksym(__up_wakeup) = 1042cbb5 is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > ksym(pci_request_region) = 10cc2981 is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > ksym(skb_dequeue) = 10fc721b is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > ksym(mod_timer) = 14777d07 is needed by kernel-ib-1.2-2.6.20.1.x86_64 > ksym(remap_pfn_range) = 155834a8 is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > ksym(unregister_netevent_notifier) = 1598dc9d is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > ksym(bad_dma_address) = 1675606f is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > ksym(dev_get_by_name) = 16ab1a6b is needed by > kernel-ib-1.2-2.6.20.1.x86_64 > > ... > > > > Anybody seen this? > > > > > > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] cannot instal ofed-1.2 kernel rpm on 2.6.20.1
I built the ofed 1.2 rpms from the OFED-1.2-20070227-0602 build and the kernel rpm fails to install on a 2.6.20.1 kernel: vic13:/usr/local/src/OFED-1.2-20070227-0602/RPMS/sles-release-10-15.2 # rpm -U kernel-ib-1.2-2.6.20.1.x86_64.rpm error: Failed dependencies: ksym(schedule) = 1000e51 is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(__up_wakeup) = 1042cbb5 is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(pci_request_region) = 10cc2981 is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(skb_dequeue) = 10fc721b is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(mod_timer) = 14777d07 is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(remap_pfn_range) = 155834a8 is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(unregister_netevent_notifier) = 1598dc9d is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(bad_dma_address) = 1675606f is needed by kernel-ib-1.2-2.6.20.1.x86_64 ksym(dev_get_by_name) = 16ab1a6b is needed by kernel-ib-1.2-2.6.20.1.x86_64 ... Anybody seen this? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] for OFED 1.2
> > > > > > Sean, please install quilt and try using it for working with the system. > > > Adding new patch is usually done in this way > > > quilt new > > > quilt add > > > edit > > > quilt refresh > > > > > > cp patches/ kernel_patches/fixes/ > > > git add kernel_patches/fixes/ > > > git commit kernel_patches/fixes/ > > > > NOTE: The key to the above process is the assumption that the developer > > maintains _all_ of the existing patches from kernel_patches/ on top of > > the ofed_1_2 tree using quilt or stg. Otherwise quilt/stg isn't buying > > you anything. > > OFED will do this automatically. > uh, can you explain this? Given I have a freshly cloned ofed_1_2 git tree, and I want to change cma.c (a good one cuz there are patches). What do I do? There's no quilt stack at all at this point. Right? > > And this doesn't take into account backports. > > The process works with backport patches too: you just have to do this > > > quilt pop -a > > > > > > quilt new > > > > quilt add > > > > edit > > > > quilt refresh > > > > quilt push -a But you cannot keep a stack for more than one backport pushed, right? So you still need to be slapping the stacks of patches around for each backport. Or maybe I'm confused? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] for OFED 1.2
On Tue, 2007-02-27 at 19:44 +0200, Michael S. Tsirkin wrote: > > Quoting Sean Hefty <[EMAIL PROTECTED]>: > > Subject: Re: [PATCH] for OFED 1.2 > > > > >I think with stacked git or just git and rebasing at key times, you > > >could keep an ofed_1_2 tree that folks can easily apply patches to... > > > > > >Its too late to change this for 1.2, but you might want to reconsider > > >the design for 1.3. > > > > Can't we just create a new branch (ofed_1_2_patched) with these patches > > already > > applied and in the correct order? > > Then what we do when we want to update to new upstream? Throw this branch > away? > As it is, I just pull then build and remove patches that conflict. > > By the way, there are backport patches, etc - it is still incorrect > to say that you would be able to generate a patch out of git > and know it's a good one without test-build. > > > Maybe I'm just not understanding the work flow here... > > Sean, please install quilt and try using it for working with the system. > Adding new patch is usually done in this way > quilt new > quilt add > edit > quilt refresh > > cp patches/ kernel_patches/fixes/ > git add kernel_patches/fixes/ > git commit kernel_patches/fixes/ NOTE: The key to the above process is the assumption that the developer maintains _all_ of the existing patches from kernel_patches/ on top of the ofed_1_2 tree using quilt or stg. Otherwise quilt/stg isn't buying you anything. And this doesn't take into account backports. Regardless, you need to build, install and test any ofed patch on an ofed system, so you're gonna have extra work: 1) create ofed-specific patch build/test it on ofed post it to openib-general/ewg 2) create kernel.org patch build/test it on kernel.org post it to openib-gernel/lklm/netdev My .27 cents... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] for OFED 1.2
On Tue, 2007-02-27 at 18:55 +0200, Michael S. Tsirkin wrote: > > Quoting Sean Hefty <[EMAIL PROTECTED]>: > > Subject: Re: [PATCH] for OFED 1.2 > > > > >Please send patches that will be added to kernel_patches/fixes. > > > > > >Please update your git tree from > > >git://git.openfabrics.org/~vlad/ofed_1_2/.git ofed_1_2 > > > > You want me to create a patch that adds a file that contains the actual > > patches? > > > > Why not apply the patches directly? > > That's the ofed structure, this was discussed multiple times already. > The point is to keep all changes to upstream components separate, > to make updating to upstream kernel trivial in the future. > > Worked quite well for OFED 1.1 -> 1.2 transition. > Having these patches as files is painful for every developer because they cannot create a patch against ofed_1_2/drivers/infiniband/* nor the kernel.org upstream tree. They need to apply all the current patches and then create a patch on top of that. Or hope the patch applies fuzzily. I think with stacked git or just git and rebasing at key times, you could keep an ofed_1_2 tree that folks can easily apply patches to... Its too late to change this for 1.2, but you might want to reconsider the design for 1.3. my 2 cents... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 6/6] Populate Rx free list with pages.
Populate Rx free list with pages.
Signed-off-by: Divy Le Ray <[EMAIL PROTECTED]>
---
drivers/net/cxgb3/adapter.h |9 +
drivers/net/cxgb3/sge.c | 318 +++
2 files changed, 235 insertions(+), 92 deletions(-)
diff --git a/drivers/net/cxgb3/adapter.h b/drivers/net/cxgb3/adapter.h
index 01b99b9..80c3d8f 100644
--- a/drivers/net/cxgb3/adapter.h
+++ b/drivers/net/cxgb3/adapter.h
@@ -74,6 +74,11 @@ enum { /* adapter flags */
struct rx_desc;
struct rx_sw_desc;
+struct sge_fl_page {
+ struct skb_frag_struct frag;
+ unsigned char *va;
+};
+
struct sge_fl {/* SGE per free-buffer list state */
unsigned int buf_size; /* size of each Rx buffer */
unsigned int credits; /* # of available Rx buffers */
@@ -81,11 +86,13 @@ struct sge_fl { /* SGE per free-buffer
unsigned int cidx; /* consumer index */
unsigned int pidx; /* producer index */
unsigned int gen; /* free list generation */
+ unsigned int cntxt_id; /* SGE context id for the free list */
+ struct sge_fl_page page;
struct rx_desc *desc; /* address of HW Rx descriptor ring */
struct rx_sw_desc *sdesc; /* address of SW Rx descriptor ring */
dma_addr_t phys_addr; /* physical address of HW ring start */
- unsigned int cntxt_id; /* SGE context id for the free list */
unsigned long empty;/* # of times queue ran out of buffers */
+ unsigned long alloc_failed; /* # of times buffer allocation failed */
};
/*
diff --git a/drivers/net/cxgb3/sge.c b/drivers/net/cxgb3/sge.c
index 4ff0ab6..c237834 100644
--- a/drivers/net/cxgb3/sge.c
+++ b/drivers/net/cxgb3/sge.c
@@ -45,9 +45,25 @@ #include "firmware_exports.h"
#define USE_GTS 0
#define SGE_RX_SM_BUF_SIZE 1536
+
+/*
+ * If USE_RX_PAGE is defined, the small freelist populated with (partial)
+ * pages instead of skbs. Pages are carved up into RX_PAGE_SIZE chunks (must
+ * be a multiple of the host page size).
+ */
+#define USE_RX_PAGE
+#define RX_PAGE_SIZE 2048
+
+/*
+ * skb freelist packets are copied into a new skb (and the freelist one is
+ * reused) if their len is <=
+ */
#define SGE_RX_COPY_THRES 256
-# define SGE_RX_DROP_THRES 16
+/*
+ * Minimum number of freelist entries before we start dropping TUNNEL frames.
+ */
+#define SGE_RX_DROP_THRES 16
/*
* Period of the Tx buffer reclaim timer. This timer does not need to run
@@ -85,7 +101,10 @@ struct tx_sw_desc { /* SW state per Tx
};
struct rx_sw_desc {/* SW state per Rx descriptor */
- struct sk_buff *skb;
+ union {
+ struct sk_buff *skb;
+ struct sge_fl_page page;
+ } t;
DECLARE_PCI_UNMAP_ADDR(dma_addr);
};
@@ -332,16 +351,27 @@ static void free_rx_bufs(struct pci_dev
pci_unmap_single(pdev, pci_unmap_addr(d, dma_addr),
q->buf_size, PCI_DMA_FROMDEVICE);
- kfree_skb(d->skb);
- d->skb = NULL;
+
+ if (q->buf_size != RX_PAGE_SIZE) {
+ kfree_skb(d->t.skb);
+ d->t.skb = NULL;
+ } else {
+ if (d->t.page.frag.page)
+ put_page(d->t.page.frag.page);
+ d->t.page.frag.page = NULL;
+ }
if (++cidx == q->size)
cidx = 0;
}
+
+ if (q->page.frag.page)
+ put_page(q->page.frag.page);
+ q->page.frag.page = NULL;
}
/**
* add_one_rx_buf - add a packet buffer to a free-buffer list
- * @skb: the buffer to add
+ * @va: va of the buffer to add
* @len: the buffer length
* @d: the HW Rx descriptor to write
* @sd: the SW Rx descriptor to write
@@ -351,14 +381,13 @@ static void free_rx_bufs(struct pci_dev
* Add a buffer of the given length to the supplied HW and SW Rx
* descriptors.
*/
-static inline void add_one_rx_buf(struct sk_buff *skb, unsigned int len,
+static inline void add_one_rx_buf(unsigned char *va, unsigned int len,
struct rx_desc *d, struct rx_sw_desc *sd,
unsigned int gen, struct pci_dev *pdev)
{
dma_addr_t mapping;
- sd->skb = skb;
- mapping = pci_map_single(pdev, skb->data, len, PCI_DMA_FROMDEVICE);
+ mapping = pci_map_single(pdev, va, len, PCI_DMA_FROMDEVICE);
pci_unmap_addr_set(sd, dma_addr, mapping);
d->addr_lo = cpu_to_be32(mapping);
@@ -383,14 +412,47 @@ static void refill_fl(struct adapter *ad
{
struct rx_sw_desc *sd = &q->sdesc[q->pidx];
struct rx_desc *d = &q->desc[q->pidx];
+ struct sge_fl_page *p = &q->page;
while (n--) {
- struct sk_buff *skb = alloc_skb(q->buf_size, gfp);
+ uns
[openib-general] [PATCH 5/6] Improve the traffic recovery after the HW ran out of response queue entries.
Improve the traffic recovery after the HW ran out of response queue entries.
Signed-off-by: Divy Le Ray <[EMAIL PROTECTED]>
---
drivers/net/cxgb3/adapter.h |2 ++
drivers/net/cxgb3/sge.c | 15 ++-
2 files changed, 16 insertions(+), 1 deletions(-)
diff --git a/drivers/net/cxgb3/adapter.h b/drivers/net/cxgb3/adapter.h
old mode 100755
new mode 100644
index 5c97a64..01b99b9
--- a/drivers/net/cxgb3/adapter.h
+++ b/drivers/net/cxgb3/adapter.h
@@ -121,6 +121,8 @@ struct sge_rspq { /* state for an SGE r
unsigned long empty;/* # of times queue ran out of credits */
unsigned long nomem;/* # of responses deferred due to no mem */
unsigned long unhandled_irqs; /* # of spurious intrs */
+ unsigned long starved;
+ unsigned long restarted;
};
struct tx_desc;
diff --git a/drivers/net/cxgb3/sge.c b/drivers/net/cxgb3/sge.c
index 822a598..4ff0ab6 100644
--- a/drivers/net/cxgb3/sge.c
+++ b/drivers/net/cxgb3/sge.c
@@ -2376,13 +2376,26 @@ static void sge_timer_cb(unsigned long d
spin_unlock(&qs->txq[TXQ_OFLD].lock);
}
lock = (adap->flags & USING_MSIX) ? &qs->rspq.lock :
- &adap->sge.qs[0].rspq.lock;
+ &adap->sge.qs[0].rspq.lock;
if (spin_trylock_irq(lock)) {
if (!napi_is_scheduled(qs->netdev)) {
+ u32 status = t3_read_reg(adap, A_SG_RSPQ_FL_STATUS);
+
if (qs->fl[0].credits < qs->fl[0].size)
__refill_fl(adap, &qs->fl[0]);
if (qs->fl[1].credits < qs->fl[1].size)
__refill_fl(adap, &qs->fl[1]);
+
+ if (status & (1 << qs->rspq.cntxt_id)) {
+ qs->rspq.starved++;
+ if (qs->rspq.credits) {
+ refill_rspq(adap, &qs->rspq, 1);
+ qs->rspq.credits--;
+ qs->rspq.restarted++;
+ t3_write_reg(adap, A_SG_RSPQ_FL_STATUS,
+1 << qs->rspq.cntxt_id);
+ }
+ }
}
spin_unlock_irq(lock);
}
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 2/6] Clean up some private ioctls.
Clean up some private ioctls.
Signed-off-by: Divy Le Ray <[EMAIL PROTECTED]>
---
drivers/net/cxgb3/cxgb3_ioctl.h | 33 +--
drivers/net/cxgb3/cxgb3_main.c | 48 +++
2 files changed, 15 insertions(+), 66 deletions(-)
diff --git a/drivers/net/cxgb3/cxgb3_ioctl.h b/drivers/net/cxgb3/cxgb3_ioctl.h
old mode 100755
new mode 100644
index a942818..0a82fcd
--- a/drivers/net/cxgb3/cxgb3_ioctl.h
+++ b/drivers/net/cxgb3/cxgb3_ioctl.h
@@ -36,28 +36,17 @@ #define __CHIOCTL_H__
* Ioctl commands specific to this driver.
*/
enum {
- CHELSIO_SETREG = 1024,
- CHELSIO_GETREG,
- CHELSIO_SETTPI,
- CHELSIO_GETTPI,
- CHELSIO_GETMTUTAB,
- CHELSIO_SETMTUTAB,
- CHELSIO_GETMTU,
- CHELSIO_SET_PM,
- CHELSIO_GET_PM,
- CHELSIO_GET_TCAM,
- CHELSIO_SET_TCAM,
- CHELSIO_GET_TCB,
- CHELSIO_GET_MEM,
- CHELSIO_LOAD_FW,
- CHELSIO_GET_PROTO,
- CHELSIO_SET_PROTO,
- CHELSIO_SET_TRACE_FILTER,
- CHELSIO_SET_QSET_PARAMS,
- CHELSIO_GET_QSET_PARAMS,
- CHELSIO_SET_QSET_NUM,
- CHELSIO_GET_QSET_NUM,
- CHELSIO_SET_PKTSCHED,
+ CHELSIO_GETMTUTAB = 1029,
+ CHELSIO_SETMTUTAB = 1030,
+ CHELSIO_SET_PM = 1032,
+ CHELSIO_GET_PM = 1033,
+ CHELSIO_GET_MEM = 1038,
+ CHELSIO_LOAD_FW = 1041,
+ CHELSIO_SET_TRACE_FILTER= 1044,
+ CHELSIO_SET_QSET_PARAMS = 1045,
+ CHELSIO_GET_QSET_PARAMS = 1046,
+ CHELSIO_SET_QSET_NUM= 1047,
+ CHELSIO_GET_QSET_NUM= 1048,
};
struct ch_reg {
diff --git a/drivers/net/cxgb3/cxgb3_main.c b/drivers/net/cxgb3/cxgb3_main.c
old mode 100755
new mode 100644
index 638b0ab..0e84c4e
--- a/drivers/net/cxgb3/cxgb3_main.c
+++ b/drivers/net/cxgb3/cxgb3_main.c
@@ -1547,32 +1547,6 @@ static int cxgb_extension_ioctl(struct n
return -EFAULT;
switch (cmd) {
- case CHELSIO_SETREG:{
- struct ch_reg edata;
-
- if (!capable(CAP_NET_ADMIN))
- return -EPERM;
- if (copy_from_user(&edata, useraddr, sizeof(edata)))
- return -EFAULT;
- if ((edata.addr & 3) != 0
- || edata.addr >= adapter->mmio_len)
- return -EINVAL;
- writel(edata.val, adapter->regs + edata.addr);
- break;
- }
- case CHELSIO_GETREG:{
- struct ch_reg edata;
-
- if (copy_from_user(&edata, useraddr, sizeof(edata)))
- return -EFAULT;
- if ((edata.addr & 3) != 0
- || edata.addr >= adapter->mmio_len)
- return -EINVAL;
- edata.val = readl(adapter->regs + edata.addr);
- if (copy_to_user(useraddr, &edata, sizeof(edata)))
- return -EFAULT;
- break;
- }
case CHELSIO_SET_QSET_PARAMS:{
int i;
struct qset_params *q;
@@ -1836,10 +1810,10 @@ static int cxgb_extension_ioctl(struct n
return -EINVAL;
/*
- * Version scheme:
- * bits 0..9: chip version
- * bits 10..15: chip revision
- */
+* Version scheme:
+* bits 0..9: chip version
+* bits 10..15: chip revision
+*/
t.version = 3 | (adapter->params.rev << 10);
if (copy_to_user(useraddr, &t, sizeof(t)))
return -EFAULT;
@@ -1888,20 +1862,6 @@ static int cxgb_extension_ioctl(struct n
t.trace_rx);
break;
}
- case CHELSIO_SET_PKTSCHED:{
- struct ch_pktsched_params p;
-
- if (!capable(CAP_NET_ADMIN))
- return -EPERM;
- if (!adapter->open_device_map)
- return -EAGAIN; /* uP and SGE must be running */
- if (copy_from_user(&p, useraddr, sizeof(p)))
- return -EFAULT;
- send_pktsched_cmd(adapter, p.sched, p.idx, p.min, p.max,
- p.binding);
- break;
-
- }
default:
return -EOPNOTSUPP;
}
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 4/6] Offload packets may be DMAed long after their SGE Tx descriptors are done
Offload packets may be DMAed long after their SGE Tx descriptors are done
so they must remain mapped until they are freed rather than until their
descriptors are freed. Unmap such packets through an skb destructor.
Signed-off-by: Divy Le Ray <[EMAIL PROTECTED]>
---
drivers/net/cxgb3/sge.c | 63 ++-
1 files changed, 61 insertions(+), 2 deletions(-)
diff --git a/drivers/net/cxgb3/sge.c b/drivers/net/cxgb3/sge.c
old mode 100755
new mode 100644
index 3f2cf8a..822a598
--- a/drivers/net/cxgb3/sge.c
+++ b/drivers/net/cxgb3/sge.c
@@ -105,6 +105,15 @@ struct unmap_info {/* packet unmapping
};
/*
+ * Holds unmapping information for Tx packets that need deferred unmapping.
+ * This structure lives at skb->head and must be allocated by callers.
+ */
+struct deferred_unmap_info {
+ struct pci_dev *pdev;
+ dma_addr_t addr[MAX_SKB_FRAGS + 1];
+};
+
+/*
* Maps a number of flits to the number of Tx descriptors that can hold them.
* The formula is
*
@@ -252,10 +261,13 @@ static void free_tx_desc(struct adapter
struct pci_dev *pdev = adapter->pdev;
unsigned int cidx = q->cidx;
+ const int need_unmap = need_skb_unmap() &&
+ q->cntxt_id >= FW_TUNNEL_SGEEC_START;
+
d = &q->sdesc[cidx];
while (n--) {
if (d->skb) { /* an SGL is present */
- if (need_skb_unmap())
+ if (need_unmap)
unmap_skb(d->skb, q, cidx, pdev);
if (d->skb->priority == cidx)
kfree_skb(d->skb);
@@ -1227,6 +1239,50 @@ int t3_mgmt_tx(struct adapter *adap, str
}
/**
+ * deferred_unmap_destructor - unmap a packet when it is freed
+ * @skb: the packet
+ *
+ * This is the packet destructor used for Tx packets that need to remain
+ * mapped until they are freed rather than until their Tx descriptors are
+ * freed.
+ */
+static void deferred_unmap_destructor(struct sk_buff *skb)
+{
+ int i;
+ const dma_addr_t *p;
+ const struct skb_shared_info *si;
+ const struct deferred_unmap_info *dui;
+ const struct unmap_info *ui = (struct unmap_info *)skb->cb;
+
+ dui = (struct deferred_unmap_info *)skb->head;
+ p = dui->addr;
+
+ if (ui->len)
+ pci_unmap_single(dui->pdev, *p++, ui->len, PCI_DMA_TODEVICE);
+
+ si = skb_shinfo(skb);
+ for (i = 0; i < si->nr_frags; i++)
+ pci_unmap_page(dui->pdev, *p++, si->frags[i].size,
+ PCI_DMA_TODEVICE);
+}
+
+static void setup_deferred_unmapping(struct sk_buff *skb, struct pci_dev *pdev,
+const struct sg_ent *sgl, int sgl_flits)
+{
+ dma_addr_t *p;
+ struct deferred_unmap_info *dui;
+
+ dui = (struct deferred_unmap_info *)skb->head;
+ dui->pdev = pdev;
+ for (p = dui->addr; sgl_flits >= 3; sgl++, sgl_flits -= 3) {
+ *p++ = be64_to_cpu(sgl->addr[0]);
+ *p++ = be64_to_cpu(sgl->addr[1]);
+ }
+ if (sgl_flits)
+ *p = be64_to_cpu(sgl->addr[0]);
+}
+
+/**
* write_ofld_wr - write an offload work request
* @adap: the adapter
* @skb: the packet to send
@@ -1262,8 +1318,11 @@ static void write_ofld_wr(struct adapter
sgp = ndesc == 1 ? (struct sg_ent *)&d->flit[flits] : sgl;
sgl_flits = make_sgl(skb, sgp, skb->h.raw, skb->tail - skb->h.raw,
adap->pdev);
- if (need_skb_unmap())
+ if (need_skb_unmap()) {
+ setup_deferred_unmapping(skb, adap->pdev, sgp, sgl_flits);
+ skb->destructor = deferred_unmap_destructor;
((struct unmap_info *)skb->cb)->len = skb->tail - skb->h.raw;
+ }
write_wr_hdr_sgl(ndesc, skb, d, pidx, q, sgl, flits, sgl_flits,
gen, from->wr_hi, from->wr_lo);
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 3/6] Update FW version to 3.2
Update FW version to 3.2 Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/net/cxgb3/t3_hw.c |6 -- drivers/net/cxgb3/version.h |2 ++ 2 files changed, 6 insertions(+), 2 deletions(-) diff --git a/drivers/net/cxgb3/t3_hw.c b/drivers/net/cxgb3/t3_hw.c old mode 100755 new mode 100644 index 365a7f5..eaa7a2e --- a/drivers/net/cxgb3/t3_hw.c +++ b/drivers/net/cxgb3/t3_hw.c @@ -884,11 +884,13 @@ int t3_check_fw_version(struct adapter * major = G_FW_VERSION_MAJOR(vers); minor = G_FW_VERSION_MINOR(vers); - if (type == FW_VERSION_T3 && major == 3 && minor == 1) + if (type == FW_VERSION_T3 && major == FW_VERSION_MAJOR && + minor == FW_VERSION_MINOR) return 0; CH_ERR(adapter, "found wrong FW version(%u.%u), " - "driver needs version 3.1\n", major, minor); + "driver needs version %u.%u\n", major, minor, + FW_VERSION_MAJOR, FW_VERSION_MINOR); return -EINVAL; } diff --git a/drivers/net/cxgb3/version.h b/drivers/net/cxgb3/version.h old mode 100755 new mode 100644 index 2b67dd5..782a6cf --- a/drivers/net/cxgb3/version.h +++ b/drivers/net/cxgb3/version.h @@ -36,4 +36,6 @@ #define DRV_DESC "Chelsio T3 Network Dri #define DRV_NAME "cxgb3" /* Driver version */ #define DRV_VERSION "1.0" +#define FW_VERSION_MAJOR 3 +#define FW_VERSION_MINOR 2 #endif /* __CHELSIO_VERSION_H */ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 1/6] sysfs attributes are now managed per port, no longer per adapter.
sysfs attributes are now managed per port, no longer per adapter.
Signed-off-by: Divy Le Ray <[EMAIL PROTECTED]>
---
drivers/net/cxgb3/cxgb3_main.c | 21 -
1 files changed, 12 insertions(+), 9 deletions(-)
diff --git a/drivers/net/cxgb3/cxgb3_main.c b/drivers/net/cxgb3/cxgb3_main.c
index dfa035a..638b0ab 100755
--- a/drivers/net/cxgb3/cxgb3_main.c
+++ b/drivers/net/cxgb3/cxgb3_main.c
@@ -435,26 +435,24 @@ static int setup_sge_qsets(struct adapte
}
static ssize_t attr_show(struct class_device *cd, char *buf,
-ssize_t(*format) (struct adapter *, char *))
+ssize_t(*format) (struct net_device *, char *))
{
ssize_t len;
- struct adapter *adap = to_net_dev(cd)->priv;
/* Synchronize with ioctls that may shut down the device */
rtnl_lock();
- len = (*format) (adap, buf);
+ len = (*format) (to_net_dev(cd), buf);
rtnl_unlock();
return len;
}
static ssize_t attr_store(struct class_device *cd, const char *buf, size_t len,
- ssize_t(*set) (struct adapter *, unsigned int),
+ ssize_t(*set) (struct net_device *, unsigned int),
unsigned int min_val, unsigned int max_val)
{
char *endp;
ssize_t ret;
unsigned int val;
- struct adapter *adap = to_net_dev(cd)->priv;
if (!capable(CAP_NET_ADMIN))
return -EPERM;
@@ -464,7 +462,7 @@ static ssize_t attr_store(struct class_d
return -EINVAL;
rtnl_lock();
- ret = (*set) (adap, val);
+ ret = (*set) (to_net_dev(cd), val);
if (!ret)
ret = len;
rtnl_unlock();
@@ -472,8 +470,9 @@ static ssize_t attr_store(struct class_d
}
#define CXGB3_SHOW(name, val_expr) \
-static ssize_t format_##name(struct adapter *adap, char *buf) \
+static ssize_t format_##name(struct net_device *dev, char *buf) \
{ \
+ struct adapter *adap = dev->priv; \
return sprintf(buf, "%u\n", val_expr); \
} \
static ssize_t show_##name(struct class_device *cd, char *buf) \
@@ -481,8 +480,10 @@ static ssize_t show_##name(struct class_
return attr_show(cd, buf, format_##name); \
}
-static ssize_t set_nfilters(struct adapter *adap, unsigned int val)
+static ssize_t set_nfilters(struct net_device *dev, unsigned int val)
{
+ struct adapter *adap = dev->priv;
+
if (adap->flags & FULL_INIT_DONE)
return -EBUSY;
if (val && adap->params.rev == 0)
@@ -499,8 +500,10 @@ static ssize_t store_nfilters(struct cla
return attr_store(cd, buf, len, set_nfilters, 0, ~0);
}
-static ssize_t set_nservers(struct adapter *adap, unsigned int val)
+static ssize_t set_nservers(struct net_device *dev, unsigned int val)
{
+ struct adapter *adap = dev->priv;
+
if (adap->flags & FULL_INIT_DONE)
return -EBUSY;
if (val > t3_mc5_size(&adap->mc5) - adap->params.mc5.nfilters)
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 0/6] ofed_1_2: cxgb3 bug fixes
Hey Vlad, These fixes need to be pulled into ofed_1_2 for the Chelsio Ethernet driver. You can pull them directly from my ofa git tree: git://staging.openfabrics.org/~swise/ofed_1_2 cxgb3_fixes Thanks, Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] HOWTO check ofa_kernel build from your git tree
Where are all the kernel src trees on ssh. openfabrics.org? I would like to build against specific trees that are failing with cxgb3... Also: what RH distro ships: linux-2.6.9-22.ELsmp and linux-2.6.9-34.ELsmp Thanks, Steve. On Mon, 2007-02-26 at 17:07 +0200, Vladimir Sokolovsky wrote: > On ssh.openfabrics.org: > Run > env git_url=/home/mst/scm/ofed_1_2_devel.git git_branch=ofed_1_2 \ > CHECK_LOCAL=yes \ > CHECK_KERNEL_ORG=yes \ > CHECK_CROSS=yes /home/vlad/scripts/build_ofa_kernel.sh > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] librdmacm examples not working under OFED 1.2 alpha
What device? On Thu, 2007-02-22 at 17:07 +0200, Or Gerlitz wrote: > I have tested RH4 U4 and to some extent also RH5 beta and see the following: > > under RH4 U4 > > > - rping: addr and route resolution passing, client getting reject on conn req > > - udaddy: working fine on both UDP and IPOIB port spaces > > - mckey: not applicable on RH4 U4 till my patch with ip_ib_mc_map is merged > > under both udaddy and rping librdmacm report: > > librdmacm: couldn't read ABI version. > librdmacm: assuming: 4 > > under RH5 > = > > basically, the same: rping does not work, udaddy works on both port spaces. > Also was able to check mckey and it works fine on both port spaces. > The ABI error print is not seen. > > The rping client/server logs are below, > > Or. > > rping client > > > [EMAIL PROTECTED] ~]# rping -c -v -d -a 193.168.80.175 > ipaddr (193.168.80.175) > librdmacm: couldn't read ABI version. > librdmacm: assuming: 4 > created cm_id 0x505f10 > cma_event type 0 cma_id 0x505f10 (parent) > cma_event type 2 cma_id 0x505f10 (parent) > rdma_resolve_addr - rdma_resolve_route successful > created pd 0x507830 > created channel 0x506260 > created cq 0x507880 > created qp 0x507990 > rping_setup_buffers called on cb 0x505010 > allocated & registered buffers... > cq_thread started. > cq completion failed status 5 > wait for CONNECTED state 10 > connect error -1 > rping_free_buffers called on cb 0x505010 > cma_event type 8 cma_id 0x505f10 (parent) > cma event 8, error 0 > > rping server > === > [EMAIL PROTECTED] ~]# rping -s -d -v -S 100 -C 100 > verbose > size 100 > count 100 > librdmacm: couldn't read ABI version. > librdmacm: assuming: 4 > created cm_id 0x505f00 > rdma_bind_addr successful > rdma_listen > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 0/7] cxgb3 - Chelsio T3 1G/10G driver updates
Divy, Do these need to be pulled into OFED 1.2 as well? Steve. On Thu, 2007-02-22 at 03:58 -0800, Divy Le Ray wrote: > Jeff, > > I'm sending a series of incremental patches updating > the cxgb3 driver. These patches are built against Linus'git tree. > > Cheers, > Divy > - > To unsubscribe from this list: send the line "unsubscribe linux-kernel" in > the body of a message to [EMAIL PROTECTED] > More majordomo info at http://vger.kernel.org/majordomo-info.html > Please read the FAQ at http://www.tux.org/lkml/ ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 2/2] ofed_1_2 Fix copyrights in the iw_cxgb3 driver.
And this one too... Thanks, Steve. On Thu, 2007-02-15 at 14:00 -0600, Steve Wise wrote: > Fix copyrights in the iw_cxgb3 driver. > > Remove the Open Grid Computing copyright. It shouldn't be there. > > Signed-off-by: Steve Wise <[EMAIL PROTECTED]> > --- > > drivers/infiniband/hw/cxgb3/core/cxio_dbg.c |1 - > drivers/infiniband/hw/cxgb3/core/cxio_hal.c |1 - > drivers/infiniband/hw/cxgb3/core/cxio_hal.h |1 - > drivers/infiniband/hw/cxgb3/core/cxio_resource.c |1 - > drivers/infiniband/hw/cxgb3/core/cxio_resource.h |1 - > drivers/infiniband/hw/cxgb3/core/cxio_wr.h |1 - > drivers/infiniband/hw/cxgb3/iwch.c |1 - > drivers/infiniband/hw/cxgb3/iwch.h |1 - > drivers/infiniband/hw/cxgb3/iwch_cm.c|1 - > drivers/infiniband/hw/cxgb3/iwch_cm.h|1 - > drivers/infiniband/hw/cxgb3/iwch_cq.c|1 - > drivers/infiniband/hw/cxgb3/iwch_ev.c|1 - > drivers/infiniband/hw/cxgb3/iwch_mem.c |1 - > drivers/infiniband/hw/cxgb3/iwch_provider.c |1 - > drivers/infiniband/hw/cxgb3/iwch_provider.h |1 - > drivers/infiniband/hw/cxgb3/iwch_qp.c|1 - > drivers/infiniband/hw/cxgb3/iwch_user.h |1 - > 17 files changed, 0 insertions(+), 17 deletions(-) > > diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c > b/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c > index dfaa704..d6b6c97 100644 > --- a/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c > +++ b/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_hal.c > b/drivers/infiniband/hw/cxgb3/core/cxio_hal.c > index 5e31816..229edd5 100644 > --- a/drivers/infiniband/hw/cxgb3/core/cxio_hal.c > +++ b/drivers/infiniband/hw/cxgb3/core/cxio_hal.c > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_hal.h > b/drivers/infiniband/hw/cxgb3/core/cxio_hal.h > index e5e702d..1553bda 100644 > --- a/drivers/infiniband/hw/cxgb3/core/cxio_hal.h > +++ b/drivers/infiniband/hw/cxgb3/core/cxio_hal.h > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_resource.c > b/drivers/infiniband/hw/cxgb3/core/cxio_resource.c > index d1d8722..cf78050 100644 > --- a/drivers/infiniband/hw/cxgb3/core/cxio_resource.c > +++ b/drivers/infiniband/hw/cxgb3/core/cxio_resource.c > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_resource.h > b/drivers/infiniband/hw/cxgb3/core/cxio_resource.h > index a6bbe83..a2703a3 100644 > --- a/drivers/infiniband/hw/cxgb3/core/cxio_resource.h > +++ b/drivers/infiniband/hw/cxgb3/core/cxio_resource.h > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_wr.h > b/drivers/infiniband/hw/cxgb3/core/cxio_wr.h > index 234a084..6c7ac55 100644 > --- a/drivers/infiniband/hw/cxgb3/core/cxio_wr.h > +++ b/drivers/infiniband/hw/cxgb3/core/cxio_wr.h > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU >
Re: [openib-general] [PATCH 1/2] ofed_1_2 Fix copyrights in the cxgb3 driver.
Vlad, Please apply this to ofed_1_2. Thanks, Steve. On Thu, 2007-02-15 at 13:59 -0600, Steve Wise wrote: > Fix copyrights in the cxgb3 driver. > > Remove the Open Grid Computing copyright. It shouldn't be there. > > Signed-off-by: Steve Wise <[EMAIL PROTECTED]> > --- > > drivers/net/cxgb3/cxgb3_defs.h|1 - > drivers/net/cxgb3/cxgb3_offload.c |1 - > drivers/net/cxgb3/cxgb3_offload.h |1 - > drivers/net/cxgb3/l2t.c |1 - > drivers/net/cxgb3/l2t.h |1 - > drivers/net/cxgb3/t3cdev.h|1 - > 6 files changed, 0 insertions(+), 6 deletions(-) > > diff --git a/drivers/net/cxgb3/cxgb3_defs.h b/drivers/net/cxgb3/cxgb3_defs.h > old mode 100755 > new mode 100644 > index 16e0049..e14862b > --- a/drivers/net/cxgb3/cxgb3_defs.h > +++ b/drivers/net/cxgb3/cxgb3_defs.h > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006-2007 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/net/cxgb3/cxgb3_offload.c > b/drivers/net/cxgb3/cxgb3_offload.c > old mode 100755 > new mode 100644 > index c3a02d6..46e9068 > --- a/drivers/net/cxgb3/cxgb3_offload.c > +++ b/drivers/net/cxgb3/cxgb3_offload.c > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006-2007 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/net/cxgb3/cxgb3_offload.h > b/drivers/net/cxgb3/cxgb3_offload.h > old mode 100755 > new mode 100644 > index 0e6beb6..f15446a > --- a/drivers/net/cxgb3/cxgb3_offload.h > +++ b/drivers/net/cxgb3/cxgb3_offload.h > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2006-2007 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/net/cxgb3/l2t.c b/drivers/net/cxgb3/l2t.c > old mode 100755 > new mode 100644 > index 3c0cb85..d660af7 > --- a/drivers/net/cxgb3/l2t.c > +++ b/drivers/net/cxgb3/l2t.c > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2003-2007 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/net/cxgb3/l2t.h b/drivers/net/cxgb3/l2t.h > old mode 100755 > new mode 100644 > index ba5d2cb..d790013 > --- a/drivers/net/cxgb3/l2t.h > +++ b/drivers/net/cxgb3/l2t.h > @@ -1,6 +1,5 @@ > /* > * Copyright (c) 2003-2007 Chelsio, Inc. All rights reserved. > - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > diff --git a/drivers/net/cxgb3/t3cdev.h b/drivers/net/cxgb3/t3cdev.h > old mode 100755 > new mode 100644 > index 9af3bcd..fa4099b > --- a/drivers/net/cxgb3/t3cdev.h > +++ b/drivers/net/cxgb3/t3cdev.h > @@ -1,6 +1,5 @@ > /* > * Copyright (C) 2006-2007 Chelsio Communications. All rights reserved. > - * Copyright (C) 2006-2007 Open Grid Computing, Inc. All rights reserved. > * > * This software is available to you under a choice of one of two > * licenses. You may choose to be licensed under the terms of the GNU > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH ofed_1_2] iw_cxgb3: Stop the EP Timer on BAD CLOSE.
Stop the ep timer in ec_status() if the status indicates a bad close. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/infiniband/hw/cxgb3/iwch_cm.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index e5442e3..d00e5dd 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -1635,6 +1635,7 @@ static int ec_status(struct t3cdev *tdev printk(KERN_ERR MOD "%s BAD CLOSE - Aborting tid %u\n", __FUNCTION__, ep->hwtid); + stop_ep_timer(ep); attrs.next_state = IWCH_QP_STATE_ERROR; iwch_modify_qp(ep->com.qp->rhp, ep->com.qp, IWCH_QP_ATTR_NEXT_STATE, ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 2.6.21] iw_cxgb3: Stop the EP Timer on BAD CLOSE.
Stop the ep timer in ec_status() if the status indicates a bad close. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/infiniband/hw/cxgb3/iwch_cm.c |1 + 1 files changed, 1 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_cm.c b/drivers/infiniband/hw/cxgb3/iwch_cm.c index e5442e3..d00e5dd 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_cm.c +++ b/drivers/infiniband/hw/cxgb3/iwch_cm.c @@ -1635,6 +1635,7 @@ static int ec_status(struct t3cdev *tdev printk(KERN_ERR MOD "%s BAD CLOSE - Aborting tid %u\n", __FUNCTION__, ep->hwtid); + stop_ep_timer(ep); attrs.next_state = IWCH_QP_STATE_ERROR; iwch_modify_qp(ep->com.qp->rhp, ep->com.qp, IWCH_QP_ATTR_NEXT_STATE, ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [ewg] Re: OFED-1.2-20070221-1741.tgz package is available
On Wed, 2007-02-21 at 20:14 +0200, Michael S. Tsirkin wrote: > Steve, can't you post a patch for 357? > I could, but I'm not sure what is _not_ supported for SLES9SP3. The script currently only allows mthca, sdp, and ipoib. cxgb3 should be allowed. But probably most other drivers too... I can provide a patch to allow cxgb3 if that's what you want... Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [2.6 patch] drivers/infiniband/hw/cxgb3/: cleanups
Thanks Adrian!
Acked-by: Steve Wise <[EMAIL PROTECTED]>
On Wed, 2007-02-21 at 11:52 +0100, Adrian Bunk wrote:
> On Tue, Feb 20, 2007 at 08:43:06AM -0600, Steve Wise wrote:
> > On Tue, 2007-02-20 at 01:02 +0100, Adrian Bunk wrote:
> > > This patch contains the following possible cleanups:
> > > - don't mark static functions in C files as inline - gcc should know
> > > best whether inlining makes sense
> > > - never compile the unused cxio_dbg.c
> > > - make the following needlessly global functions static:
> > > - cxio_hal.c: cxio_hal_clear_qp_ctx()
> > > - iwch_provider.c: iwch_get_qp()
> > > - #if 0 the following unused global functions:
> > > - cxio_hal.c: cxio_allocate_stag()
> > > - cxio_resource.: cxio_hal_get_rhdl()
> > > - cxio_resource.: cxio_hal_put_rhdl()
> > >
> >
> > You could just remove the code instead of #if 0...
> >...
>
> Updated patch below.
>
> cu
> Adrian
>
>
> <-- snip -->
>
>
> This patch contains the following possible cleanups:
> - don't mark static functions in C files as inline - gcc should know
> best whether inlining makes sense
> - never compile the unused cxio_dbg.c
> - make the following needlessly global functions static:
> - cxio_hal.c: cxio_hal_clear_qp_ctx()
> - iwch_provider.c: iwch_get_qp()
> - remove the following unused global functions:
> - cxio_hal.c: cxio_allocate_stag()
> - cxio_resource.: cxio_hal_get_rhdl()
> - cxio_resource.: cxio_hal_put_rhdl()
>
> Signed-off-by: Adrian Bunk <[EMAIL PROTECTED]>
>
> ---
>
> drivers/infiniband/hw/cxgb3/Makefile|1
> drivers/infiniband/hw/cxgb3/cxio_hal.c | 31 +---
> drivers/infiniband/hw/cxgb3/cxio_hal.h |5 ---
> drivers/infiniband/hw/cxgb3/cxio_resource.c | 14 +
> drivers/infiniband/hw/cxgb3/iwch_cm.c |5 +--
> drivers/infiniband/hw/cxgb3/iwch_provider.c |2 -
> drivers/infiniband/hw/cxgb3/iwch_provider.h |1
> drivers/infiniband/hw/cxgb3/iwch_qp.c | 29 --
> 8 files changed, 27 insertions(+), 61 deletions(-)
>
> --- linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/Makefile.old 2007-02-17
> 17:21:03.0 +0100
> +++ linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/Makefile 2007-02-17
> 17:21:08.0 +0100
> @@ -8,5 +8,4 @@
>
> ifdef CONFIG_INFINIBAND_CXGB3_DEBUG
> EXTRA_CFLAGS += -DDEBUG
> -iw_cxgb3-y += cxio_dbg.o
> endif
> --- linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/cxio_hal.h.old
> 2007-02-17 17:22:53.0 +0100
> +++ linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/cxio_hal.h 2007-02-17
> 17:25:08.0 +0100
> @@ -144,7 +144,6 @@
> void cxio_rdev_close(struct cxio_rdev *rdev);
> int cxio_hal_cq_op(struct cxio_rdev *rdev, struct t3_cq *cq,
> enum t3_cq_opcode op, u32 credit);
> -int cxio_hal_clear_qp_ctx(struct cxio_rdev *rdev, u32 qpid);
> int cxio_create_cq(struct cxio_rdev *rdev, struct t3_cq *cq);
> int cxio_destroy_cq(struct cxio_rdev *rdev, struct t3_cq *cq);
> int cxio_resize_cq(struct cxio_rdev *rdev, struct t3_cq *cq);
> @@ -155,8 +154,6 @@
> int cxio_destroy_qp(struct cxio_rdev *rdev, struct t3_wq *wq,
> struct cxio_ucontext *uctx);
> int cxio_peek_cq(struct t3_wq *wr, struct t3_cq *cq, int opcode);
> -int cxio_allocate_stag(struct cxio_rdev *rdev, u32 * stag, u32 pdid,
> -enum tpt_mem_perm perm, u32 * pbl_size, u32 * pbl_addr);
> int cxio_register_phys_mem(struct cxio_rdev *rdev, u32 * stag, u32 pdid,
> enum tpt_mem_perm perm, u32 zbva, u64 to, u32 len,
> u8 page_size, __be64 *pbl, u32 *pbl_size,
> @@ -172,8 +169,6 @@
> int cxio_rdma_init(struct cxio_rdev *rdev, struct t3_rdma_init_attr *attr);
> void cxio_register_ev_cb(cxio_hal_ev_callback_func_t ev_cb);
> void cxio_unregister_ev_cb(cxio_hal_ev_callback_func_t ev_cb);
> -u32 cxio_hal_get_rhdl(void);
> -void cxio_hal_put_rhdl(u32 rhdl);
> u32 cxio_hal_get_pdid(struct cxio_hal_resource *rscp);
> void cxio_hal_put_pdid(struct cxio_hal_resource *rscp, u32 pdid);
> int __init cxio_hal_init(void);
> --- linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/iwch_provider.h.old
> 2007-02-17 17:25:35.0 +0100
> +++ linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/iwch_provider.h
> 2007-02-17 17:25:41.0 +0100
> @@ -179,7 +179,6 @@
>
> void iwch_qp_add_ref(struct ib_qp *qp);
> void iwch_qp_rem_ref(struct ib_qp *qp);
> -struct ib_qp *iwch_get_qp(struct ib_device *dev, int qpn);
>
> struct iwch_ucontext {
> struct ib_ucontext ibucontext;
>
Re: [openib-general] [2.6 patch] drivers/infiniband/hw/cxgb3/: possible cleanups
On Tue, 2007-02-20 at 01:02 +0100, Adrian Bunk wrote:
> This patch contains the following possible cleanups:
> - don't mark static functions in C files as inline - gcc should know
> best whether inlining makes sense
> - never compile the unused cxio_dbg.c
> - make the following needlessly global functions static:
> - cxio_hal.c: cxio_hal_clear_qp_ctx()
> - iwch_provider.c: iwch_get_qp()
> - #if 0 the following unused global functions:
> - cxio_hal.c: cxio_allocate_stag()
> - cxio_resource.: cxio_hal_get_rhdl()
> - cxio_resource.: cxio_hal_put_rhdl()
>
You could just remove the code instead of #if 0...
> Signed-off-by: Adrian Bunk <[EMAIL PROTECTED]>
>
> ---
>
> drivers/infiniband/hw/cxgb3/Makefile|1
> drivers/infiniband/hw/cxgb3/cxio_hal.c | 22 +++
> drivers/infiniband/hw/cxgb3/cxio_hal.h |5 ---
> drivers/infiniband/hw/cxgb3/cxio_resource.c |8 -
> drivers/infiniband/hw/cxgb3/iwch_cm.c |5 +--
> drivers/infiniband/hw/cxgb3/iwch_provider.c |2 -
> drivers/infiniband/hw/cxgb3/iwch_provider.h |1
> drivers/infiniband/hw/cxgb3/iwch_qp.c | 29
> 8 files changed, 33 insertions(+), 40 deletions(-)
>
> --- linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/Makefile.old 2007-02-17
> 17:21:03.0 +0100
> +++ linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/Makefile 2007-02-17
> 17:21:08.0 +0100
> @@ -8,5 +8,4 @@
>
> ifdef CONFIG_INFINIBAND_CXGB3_DEBUG
> EXTRA_CFLAGS += -DDEBUG
> -iw_cxgb3-y += cxio_dbg.o
> endif
> --- linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/cxio_hal.h.old
> 2007-02-17 17:22:53.0 +0100
> +++ linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/cxio_hal.h 2007-02-17
> 17:25:08.0 +0100
> @@ -144,7 +144,6 @@
> void cxio_rdev_close(struct cxio_rdev *rdev);
> int cxio_hal_cq_op(struct cxio_rdev *rdev, struct t3_cq *cq,
> enum t3_cq_opcode op, u32 credit);
> -int cxio_hal_clear_qp_ctx(struct cxio_rdev *rdev, u32 qpid);
> int cxio_create_cq(struct cxio_rdev *rdev, struct t3_cq *cq);
> int cxio_destroy_cq(struct cxio_rdev *rdev, struct t3_cq *cq);
> int cxio_resize_cq(struct cxio_rdev *rdev, struct t3_cq *cq);
> @@ -155,8 +154,6 @@
> int cxio_destroy_qp(struct cxio_rdev *rdev, struct t3_wq *wq,
> struct cxio_ucontext *uctx);
> int cxio_peek_cq(struct t3_wq *wr, struct t3_cq *cq, int opcode);
> -int cxio_allocate_stag(struct cxio_rdev *rdev, u32 * stag, u32 pdid,
> -enum tpt_mem_perm perm, u32 * pbl_size, u32 * pbl_addr);
> int cxio_register_phys_mem(struct cxio_rdev *rdev, u32 * stag, u32 pdid,
> enum tpt_mem_perm perm, u32 zbva, u64 to, u32 len,
> u8 page_size, __be64 *pbl, u32 *pbl_size,
> @@ -172,8 +169,6 @@
> int cxio_rdma_init(struct cxio_rdev *rdev, struct t3_rdma_init_attr *attr);
> void cxio_register_ev_cb(cxio_hal_ev_callback_func_t ev_cb);
> void cxio_unregister_ev_cb(cxio_hal_ev_callback_func_t ev_cb);
> -u32 cxio_hal_get_rhdl(void);
> -void cxio_hal_put_rhdl(u32 rhdl);
> u32 cxio_hal_get_pdid(struct cxio_hal_resource *rscp);
> void cxio_hal_put_pdid(struct cxio_hal_resource *rscp, u32 pdid);
> int __init cxio_hal_init(void);
> --- linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/cxio_hal.c.old
> 2007-02-17 17:23:11.0 +0100
> +++ linux-2.6.20-mm1/drivers/infiniband/hw/cxgb3/cxio_hal.c 2007-02-17
> 17:36:40.0 +0100
> @@ -46,7 +46,7 @@
> static LIST_HEAD(rdev_list);
> static cxio_hal_ev_callback_func_t cxio_ev_cb = NULL;
>
> -static inline struct cxio_rdev *cxio_hal_find_rdev_by_name(char *dev_name)
> +static struct cxio_rdev *cxio_hal_find_rdev_by_name(char *dev_name)
> {
> struct cxio_rdev *rdev;
>
> @@ -56,8 +56,7 @@
> return NULL;
> }
>
> -static inline struct cxio_rdev *cxio_hal_find_rdev_by_t3cdev(struct t3cdev
> - *tdev)
> +static struct cxio_rdev *cxio_hal_find_rdev_by_t3cdev(struct t3cdev *tdev)
> {
> struct cxio_rdev *rdev;
>
> @@ -119,7 +118,7 @@
> return 0;
> }
>
> -static inline int cxio_hal_clear_cq_ctx(struct cxio_rdev *rdev_p, u32 cqid)
> +static int cxio_hal_clear_cq_ctx(struct cxio_rdev *rdev_p, u32 cqid)
> {
> struct rdma_cq_setup setup;
> setup.id = cqid;
> @@ -131,7 +130,7 @@
> return (rdev_p->t3cdev_p->ctl(rdev_p->t3cdev_p, RDMA_CQ_SETUP, &setup));
> }
>
> -int cxio_hal_clear_qp_ctx(struct cxio_rdev *rdev_p, u32 qpid)
> +static int cxio_hal_clear_qp_ctx(struct cxio_rdev *rdev_p, u32 qpid)
> {
> u64 sge_cmd;
> struct t3_modify_qp_wr *wqe;
> @@ -426,7 +425,7 @@
> }
> }
>
> -static inline int cqe_completes_wr(struct t3_cqe *cqe, struct t3_wq *wq)
> +static int cqe_completes_wr(struct t3_cqe *cqe, struct t3_wq *wq)
> {
> if (CQE_OPCODE(*cqe) == T3_TERMINATE)
> return 0;
> @@ -761,6 +760,7 @@
> return err;
> }
>
> +#if 0
Re: [openib-general] OFA 1.2 tarball creation
On Mon, 2007-02-19 at 14:38 -0800, Sean Hefty wrote: > >How exactly is various developers' source code pulled together to create > >the nightly OFA tarballs at www.openfabrics.org/builds (could this be > >put on the wiki somewhere?)? I went looking to see if some of Sean's > >work on RDMA CM had made it into these tarballs, and am not seeing code > >with the patches I'm looking for. > > I do not know how OFED creates their tarballs or manages their source. > > >The exact patch I'm after was 'rdma_cm: allow joins to return a unique > >address'. I remember seeing this patch on the ofed_1_2 branch in Sean's > >rdma-dev git repository about two weeks ago, though I don't see the > >ofed_1_2 branch anymore (the patch does exist on the multicast branch). > > Sean, was this patch supposed to make it to the nightly 1.2 tarballs? > > Assuming that ~vlad/ofed_1_2.git is the OFED kernel tree, then this patch does > not appear to be included. I was asked by OFED to publish an ofed_1_2 branch, > which I did, but I do not know if it was used in constructing the OFED tree. > The patch was intended to go into OFED. > The ofed_1_2 tree has the 2.6.20 drivers/modules in drivers/infiniband. They are, I think, the stock 2.6.20 drivers and modules. If there are fixes to any driver post 2.6.20, then patches get created in kernel_patches/fixes directory. These are applied as part of the configuration process when the tree is being built. Look in there to see if your change is in the form of a patch file. So you can't necessarily look at ofed_1_2/drivers/infiniband/core for the exact code, because it may get modified/patched as part of the configure done on the kernel tree during build/installation. Note in addition to patches from kernel_patches/fixes, there are other patches applied based on the kernel version (backports). Also, Sean: If you have changes that you want in OFED 1.2, you need to explicitly post patches to vlad and cc the group. They don't, by process, go pull your kernel tree to get the latest. This is different from the user libs which they pull each time they create a user package. Vlad/Michael, correct me if I'm wrong here. This all took me some time to understand and perhaps documenting this would help folks... Steve. > - Sean > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
Good news! This SRPM works with alpha1. I'm able to run the IMB benchmarks on a 4 node iwarp cluster! Thanks, Steve. On Thu, 2007-02-15 at 22:50 -0500, Shaun Rowland wrote: > Steve Wise wrote: > > Shaun, > > > > Lemme know if you have an mvapich2 kit that I can test with iwarp... > > Hi Steve. I've updated our SRPM: > > https://www.openfabrics.org/~rowland/ofed_1_2/ > > The latest is mvapich2-0.9.8-4.src.rpm. This version should solve the > shared library linking issues. This can be built outside of the OFED 1.2 > alpha1 release with the information in the README file or can replace > the previous SRPM in the OFED-1.2-alpha1/SRPMS/ directory. To use iWARP, > use the OFA build of the SRPM and set MV2_ENABLE_IWARP_MODE=1 in your > environment. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
Ok I'll try it out today! Thanks, Steve. On Thu, 2007-02-15 at 22:50 -0500, Shaun Rowland wrote: > Steve Wise wrote: > > Shaun, > > > > Lemme know if you have an mvapich2 kit that I can test with iwarp... > > Hi Steve. I've updated our SRPM: > > https://www.openfabrics.org/~rowland/ofed_1_2/ > > The latest is mvapich2-0.9.8-4.src.rpm. This version should solve the > shared library linking issues. This can be built outside of the OFED 1.2 > alpha1 release with the information in the README file or can replace > the previous SRPM in the OFED-1.2-alpha1/SRPMS/ directory. To use iWARP, > use the OFA build of the SRPM and set MV2_ENABLE_IWARP_MODE=1 in your > environment. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] remap_page_range() in older kernels
On Thu, 2007-02-15 at 12:08 -0800, Roland Dreier wrote: > > Do you remember any issues with using remap_page_range() in older > > kernels for mapping memory allocated in the kernel back to a user > > process? > > No, I would have thought it should work just like remap_pfn_range() in > later kernels. > > - R. Me too. But it definitely isn't working for cxgb3. Sigh... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 2/2] ofed_1_2 Fix copyrights in the iw_cxgb3 driver.
Fix copyrights in the iw_cxgb3 driver. Remove the Open Grid Computing copyright. It shouldn't be there. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/infiniband/hw/cxgb3/core/cxio_dbg.c |1 - drivers/infiniband/hw/cxgb3/core/cxio_hal.c |1 - drivers/infiniband/hw/cxgb3/core/cxio_hal.h |1 - drivers/infiniband/hw/cxgb3/core/cxio_resource.c |1 - drivers/infiniband/hw/cxgb3/core/cxio_resource.h |1 - drivers/infiniband/hw/cxgb3/core/cxio_wr.h |1 - drivers/infiniband/hw/cxgb3/iwch.c |1 - drivers/infiniband/hw/cxgb3/iwch.h |1 - drivers/infiniband/hw/cxgb3/iwch_cm.c|1 - drivers/infiniband/hw/cxgb3/iwch_cm.h|1 - drivers/infiniband/hw/cxgb3/iwch_cq.c|1 - drivers/infiniband/hw/cxgb3/iwch_ev.c|1 - drivers/infiniband/hw/cxgb3/iwch_mem.c |1 - drivers/infiniband/hw/cxgb3/iwch_provider.c |1 - drivers/infiniband/hw/cxgb3/iwch_provider.h |1 - drivers/infiniband/hw/cxgb3/iwch_qp.c|1 - drivers/infiniband/hw/cxgb3/iwch_user.h |1 - 17 files changed, 0 insertions(+), 17 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c b/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c index dfaa704..d6b6c97 100644 --- a/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c +++ b/drivers/infiniband/hw/cxgb3/core/cxio_dbg.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_hal.c b/drivers/infiniband/hw/cxgb3/core/cxio_hal.c index 5e31816..229edd5 100644 --- a/drivers/infiniband/hw/cxgb3/core/cxio_hal.c +++ b/drivers/infiniband/hw/cxgb3/core/cxio_hal.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_hal.h b/drivers/infiniband/hw/cxgb3/core/cxio_hal.h index e5e702d..1553bda 100644 --- a/drivers/infiniband/hw/cxgb3/core/cxio_hal.h +++ b/drivers/infiniband/hw/cxgb3/core/cxio_hal.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_resource.c b/drivers/infiniband/hw/cxgb3/core/cxio_resource.c index d1d8722..cf78050 100644 --- a/drivers/infiniband/hw/cxgb3/core/cxio_resource.c +++ b/drivers/infiniband/hw/cxgb3/core/cxio_resource.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_resource.h b/drivers/infiniband/hw/cxgb3/core/cxio_resource.h index a6bbe83..a2703a3 100644 --- a/drivers/infiniband/hw/cxgb3/core/cxio_resource.h +++ b/drivers/infiniband/hw/cxgb3/core/cxio_resource.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_wr.h b/drivers/infiniband/hw/cxgb3/core/cxio_wr.h index 234a084..6c7ac55 100644 --- a/drivers/infiniband/hw/cxgb3/core/cxio_wr.h +++ b/drivers/infiniband/hw/cxgb3/core/cxio_wr.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c index 0c95f2c..de44c57 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.c +++ b/drivers/infiniband/hw/cxgb3/iwch.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/iwch.h b/drivers/infiniband/hw/cxgb3/iwch.h index 8b111
[openib-general] [PATCH 1/2] ofed_1_2 Fix copyrights in the cxgb3 driver.
Fix copyrights in the cxgb3 driver. Remove the Open Grid Computing copyright. It shouldn't be there. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/net/cxgb3/cxgb3_defs.h|1 - drivers/net/cxgb3/cxgb3_offload.c |1 - drivers/net/cxgb3/cxgb3_offload.h |1 - drivers/net/cxgb3/l2t.c |1 - drivers/net/cxgb3/l2t.h |1 - drivers/net/cxgb3/t3cdev.h|1 - 6 files changed, 0 insertions(+), 6 deletions(-) diff --git a/drivers/net/cxgb3/cxgb3_defs.h b/drivers/net/cxgb3/cxgb3_defs.h old mode 100755 new mode 100644 index 16e0049..e14862b --- a/drivers/net/cxgb3/cxgb3_defs.h +++ b/drivers/net/cxgb3/cxgb3_defs.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006-2007 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/cxgb3/cxgb3_offload.c b/drivers/net/cxgb3/cxgb3_offload.c old mode 100755 new mode 100644 index c3a02d6..46e9068 --- a/drivers/net/cxgb3/cxgb3_offload.c +++ b/drivers/net/cxgb3/cxgb3_offload.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006-2007 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/cxgb3/cxgb3_offload.h b/drivers/net/cxgb3/cxgb3_offload.h old mode 100755 new mode 100644 index 0e6beb6..f15446a --- a/drivers/net/cxgb3/cxgb3_offload.h +++ b/drivers/net/cxgb3/cxgb3_offload.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006-2007 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/cxgb3/l2t.c b/drivers/net/cxgb3/l2t.c old mode 100755 new mode 100644 index 3c0cb85..d660af7 --- a/drivers/net/cxgb3/l2t.c +++ b/drivers/net/cxgb3/l2t.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2003-2007 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/cxgb3/l2t.h b/drivers/net/cxgb3/l2t.h old mode 100755 new mode 100644 index ba5d2cb..d790013 --- a/drivers/net/cxgb3/l2t.h +++ b/drivers/net/cxgb3/l2t.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2003-2007 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006-2007 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/net/cxgb3/t3cdev.h b/drivers/net/cxgb3/t3cdev.h old mode 100755 new mode 100644 index 9af3bcd..fa4099b --- a/drivers/net/cxgb3/t3cdev.h +++ b/drivers/net/cxgb3/t3cdev.h @@ -1,6 +1,5 @@ /* * Copyright (C) 2006-2007 Chelsio Communications. All rights reserved. - * Copyright (C) 2006-2007 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] iw_cxgb3 Fix copyrights in the iw_cxgb3 driver.
Fix copyrights in the iw_cxgb3 driver. Remove the Open Grid Computing copyright. It shouldn't be there. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/infiniband/hw/cxgb3/cxio_dbg.c |1 - drivers/infiniband/hw/cxgb3/cxio_hal.c |1 - drivers/infiniband/hw/cxgb3/cxio_hal.h |1 - drivers/infiniband/hw/cxgb3/cxio_resource.c |1 - drivers/infiniband/hw/cxgb3/cxio_resource.h |1 - drivers/infiniband/hw/cxgb3/cxio_wr.h |1 - drivers/infiniband/hw/cxgb3/iwch.c |1 - drivers/infiniband/hw/cxgb3/iwch.h |1 - drivers/infiniband/hw/cxgb3/iwch_cm.c |1 - drivers/infiniband/hw/cxgb3/iwch_cm.h |1 - drivers/infiniband/hw/cxgb3/iwch_cq.c |1 - drivers/infiniband/hw/cxgb3/iwch_ev.c |1 - drivers/infiniband/hw/cxgb3/iwch_mem.c |1 - drivers/infiniband/hw/cxgb3/iwch_provider.c |1 - drivers/infiniband/hw/cxgb3/iwch_provider.h |1 - drivers/infiniband/hw/cxgb3/iwch_qp.c |1 - drivers/infiniband/hw/cxgb3/iwch_user.h |1 - 17 files changed, 0 insertions(+), 17 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/cxio_dbg.c b/drivers/infiniband/hw/cxgb3/cxio_dbg.c index 5a7306f..75f7b16 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_dbg.c +++ b/drivers/infiniband/hw/cxgb3/cxio_dbg.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.c b/drivers/infiniband/hw/cxgb3/cxio_hal.c index 82fa720..114ac3b 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_hal.c +++ b/drivers/infiniband/hw/cxgb3/cxio_hal.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.h b/drivers/infiniband/hw/cxgb3/cxio_hal.h index 1b97e80..8ab04a7 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_hal.h +++ b/drivers/infiniband/hw/cxgb3/cxio_hal.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/cxio_resource.c b/drivers/infiniband/hw/cxgb3/cxio_resource.c index 997aa32..65bf577 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_resource.c +++ b/drivers/infiniband/hw/cxgb3/cxio_resource.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/cxio_resource.h b/drivers/infiniband/hw/cxgb3/cxio_resource.h index a6bbe83..a2703a3 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_resource.h +++ b/drivers/infiniband/hw/cxgb3/cxio_resource.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/cxio_wr.h b/drivers/infiniband/hw/cxgb3/cxio_wr.h index 103fc42..90d7b89 100644 --- a/drivers/infiniband/hw/cxgb3/cxio_wr.h +++ b/drivers/infiniband/hw/cxgb3/cxio_wr.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/iwch.c b/drivers/infiniband/hw/cxgb3/iwch.c index 4611afa..0315c9d 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.c +++ b/drivers/infiniband/hw/cxgb3/iwch.c @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 2006 Open Grid Computing, Inc. All rights reserved. * * This software is available to you under a choice of one of two * licenses. You may choose to be licensed under the terms of the GNU diff --git a/drivers/infiniband/hw/cxgb3/iwch.h b/drivers/infiniband/hw/cxgb3/iwch.h index 6517ef8..caf4e60 100644 --- a/drivers/infiniband/hw/cxgb3/iwch.h +++ b/drivers/infiniband/hw/cxgb3/iwch.h @@ -1,6 +1,5 @@ /* * Copyright (c) 2006 Chelsio, Inc. All rights reserved. - * Copyright (c) 200
[openib-general] bug 355 - problems building modules that depend on the ofed 1.2 modules
All, I've run into the following problem. Bug 335 opened to track this... I install the alpha1 ofed 1.2 rpms on a RHEL5b2 system with its 2.6.18-1.2747.el5 kernel. Then I build a module outside of the kernel that uses the IB verbs and RDMA CM kernel interface. (krping). This module builds and loads ok on a stock 2.6.20 system with ofed1.2 installed, but it fails to load on the rhel5b2 system with a version symbol problem. Here is a snipit of the errors: rdma_krping: disagrees about version of symbol ib_create_cq rdma_krping: Unknown symbol ib_create_cq rdma_krping: disagrees about version of symbol rdma_resolve_addr rdma_krping: Unknown symbol rdma_resolve_addr rdma_krping: disagrees about version of symbol ib_dereg_mr rdma_krping: Unknown symbol ib_dereg_mr I'm wondering if maybe the ofed modules are _not_ being build with src versioning even if the kernel has it turned on? We see similar problems with NFS-RDMA trying to use OFED 1.2 modules. And the NFS-RDMA works with OFED 1.1 modules, so I _think_ something is whacked with the OFED 1.2 build process. Here is the Makefile I'm using to build rdma_krping (borrowed from Intel's e1000 kit): [EMAIL PROTECTED] krping]# cat Makefile KSRC=/lib/modules/`uname -r`/source KOBJ=/lib/modules/`uname -r`/build CFLAGS += -DLINUX -D__KERNEL__ -DMODULE -O2 -pipe -Wall CFLAGS += -I/usr/local/ofed/src/ofa_kernel-1.2/include -I$(KSRC)/include -I. CFLAGS += $(shell [ -f $(KSRC)/include/linux/modversions.h ] && \ echo "-DMODVERSIONS -DEXPORT_SYMTAB \ -include $(KSRC)/include/linux/modversions.h") CFLAGS += $(CFLAGS_EXTRA) obj-m += rdma_krping.o rdma_krping-y := getopt.o krping.o default: make -C $(KSRC) O=$(KOBJ) SUBDIRS=$(shell pwd) modules clean: rm -f *.o rm -f *.ko rm -f rdma_krping.mod.c rm -f Module.symvers [EMAIL PROTECTED] krping]# ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
Shaun, Lemme know if you have an mvapich2 kit that I can test with iwarp... Thanks, Steve. On Wed, 2007-02-14 at 23:31 -0500, Shaun Rowland wrote: > Roland Dreier wrote: > > > When I build using the OFED-1.2-20070208-1508, libibverbs 1.0 is what is > > > built, at least by looking at the .so file result: > > > > > > [EMAIL PROTECTED] ~]$ ls /usr/local/ofed/lib64/ |grep ibverbs > > libibverbs.a > > > libibverbs.so > > > libibverbs.so.1 > > > libibverbs.so.1.0.0 > > > > The soname hasn't changed because the library is still compatible. > > But (I hope at least) OFED has libibverbs 1.1. > > The soname is libibverbs.so.1, so I guess the longer name would not > matter anyway. Clearly, what I posted shows the IBVERBS 1.1 ABI is > there. I think I have figured out why our code has this problem. The > problem below is similar to the original one posted about. > > I did some experimentation with the srq_pingpong libibverbs example > code. First I built it directly with: > > > gcc -g -c pingpong.c -I/usr/local/ofed/include > > gcc -g -c -D_GNU_SOURCE srq_pingpong.c -I/usr/local/ofed/include > > gcc -g -o srq_pingpong srq_pingpong.o pingpong.o -L/usr/local/ofed/lib64 > -libverbs > > > This works. Next I copied srq_pingpong.c to two files: > > srq_pingpong_rowland.c > - just has a main function that calls lib_start(). > > srq_pingpong_lib_rowland.c > - main() changed to lib_start(). > > This moves all of the SRQ pingpong code into a shared library. If I > build this shared library in this way, it works: > > > gcc -g -fpic -c pingpong.c -I/usr/local/ofed/include > > gcc -g -fpic -c -D_GNU_SOURCE srq_pingpong_lib_rowland.c > -I/usr/local/ofed/include > > gcc -g -shared -Wl,-soname,libsrqtest.so -o libsrqtest.so > srq_pingpong_lib_rowland.o pingpong.o -L/usr/local/ofed/lib64 -libverbs > > gcc -g -o srq_pingpong_rowland srq_pingpong_rowland.c -L$PWD -lsrqtest > > > Above I am linking libibverbs directly into my libsrqtest.so > library. This works and the IBVERBS 1.1 ABI is clearly in the > libsrqtest.so file: > > [EMAIL PROTECTED] ibverbs-examples]$ nm libsrqtest.so |grep ibv |head > U ibv_ack_cq_events@@IBVERBS_1.1 > U ibv_alloc_pd@@IBVERBS_1.1 > U ibv_close_device@@IBVERBS_1.1 > U ibv_create_comp_channel@@IBVERBS_1.0 > U ibv_create_cq@@IBVERBS_1.1 > U ibv_create_qp@@IBVERBS_1.1 > U ibv_create_srq@@IBVERBS_1.1 > U ibv_dealloc_pd@@IBVERBS_1.1 > U ibv_dereg_mr@@IBVERBS_1.1 > U ibv_destroy_comp_channel@@IBVERBS_1.0 > > However, if I build in a similar way to MVAPICH2, the resulting program > fails: > > > gcc -g -fpic -c pingpong.c -I/usr/local/ofed/include > > gcc -g -fpic -c -D_GNU_SOURCE srq_pingpong_lib_rowland.c > -I/usr/local/ofed/include > > gcc -g -shared -Wl,-soname,libsrqtest.so -o libsrqtest.so > srq_pingpong_lib_rowland.o pingpong.o > > gcc -g -o srq_pingpong_rowland srq_pingpong_rowland.c -L$PWD > -L/usr/local/ofed/lib64 -lsrqtest -libverbs > > > Above I am not linking libibverbs into libsrqtest.so, thus it is > required on the last gcc line. This is how MVAPICH2's libmpich.so file > works, and from past experience, I've seen this before. Running shows: > > [EMAIL PROTECTED] ibverbs-examples]$ gdb ./srq_pingpong_rowland > GNU gdb Red Hat Linux (6.3.0.0-1.132.EL4rh) > Copyright 2004 Free Software Foundation, Inc. > GDB is free software, covered by the GNU General Public License, and you are > welcome to change it and/or distribute copies of it under certain > conditions. > Type "show copying" to see the conditions. > There is absolutely no warranty for GDB. Type "show warranty" for details. > This GDB was configured as "x86_64-redhat-linux-gnu"...Using host > libthread_db library "/lib64/tls/libthread_db.so.1". > > (gdb) r > Starting program: > /home/7/rowland/z1-test/ibverbs-examples/srq_pingpong_rowland > [Thread debugging using libthread_db enabled] > [New Thread 182896403968 (LWP 29858)] > > Program received signal SIGSEGV, Segmentation fault. > [Switching to Thread 182896403968 (LWP 29858)] > post_srq_recv_wrapper_1_0 (srq=0x5075b0, wr=0x7fbfff88d0, > bad_wr=0x7fbfff88c8) > at src/compat-1_0.c:312 > 312 src/compat-1_0.c: No such file or directory. > in src/compat-1_0.c > (gdb) bt > #0 post_srq_recv_wrapper_1_0 (srq=0x5075b0, wr=0x7fbfff88d0, > bad_wr=0x7fbfff88c8) at src/compat-1_0.c:312 > #1 0x002a95559e12 in ibv_post_srq_recv (srq=0x5075b0, > recv_wr=0x7fbfff88d0, bad_recv_wr=0x7fbfff88c8) > at /usr/local/ofed/include/infiniband/verbs.h:915 > #2 0x002a95559dcf in pp_post_recv (ctx=0x5023d0, n=500) > at srq_pingpong_lib_rowland.c:496 > #3 0x002a9555a614 in lib_start (argc=1, argv=0x7fb7f8) > at srq_pingpong_lib_rowland.c:696 > #4 0x00400608 in main (argc=1, argv=0x7fb7f8) > at srq_pi
[openib-general] remap_page_range() in older kernels
Roland,
Do you remember any issues with using remap_page_range() in older
kernels for mapping memory allocated in the kernel back to a user
process?
I'm testing cxgb3 in ofed 1.2 on rhel4u4 with uses a 2.6.9 based kernel.
And cxgb3 kernel-bypass isn't working because my WQ and CQ memory isn't
getting correctly mapped into the user process.
I've confirmed that the mapping is wrong by scribbling in the memory
just after its allocated in the kernel (via dma_alloc_coherent()), then
reading in the library after mapping it. The process isn't reading the
correct scribbles...
For the ofed 1.2 backport, we've redefined remap_pfn_range() to:
static inline int
remap_pfn_range(struct vm_area_struct *vma, unsigned long addr,
unsigned long pfn, unsigned long size, pgprot_t prot)
{
return remap_page_range(vma, addr, pfn << PAGE_SHIFT, size, prot);
}
Any of this ring a bell? Any ideas?
Thanks,
Steve.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ofed_1_2 iw_cxgb3 Fail posts synchronously when in TERMINATE state.
Fail posts synchronously when in TERMINATE state. For T3B devices, mark user qp in error once we transition to TERMINATE. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/infiniband/hw/cxgb3/iwch_qp.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c index ad044bd..9cc8b5e 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_qp.c +++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c @@ -846,6 +846,8 @@ int iwch_modify_qp(struct iwch_dev *rhp, break; case IWCH_QP_STATE_TERMINATE: qhp->attr.state = IWCH_QP_STATE_TERMINATE; + if (t3b_device(qhp->rhp)) + cxio_set_wq_in_error(&qhp->wq); if (!internal) terminate = 1; break; ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] 2.6.21 iw_cxgb3 Fail posts synchronously when in TERMINATE state.
From: Steve Wise <[EMAIL PROTECTED]> Fail posts synchronously when in TERMINATE state. For T3B devices, mark user qp in error once we transition to TERMINATE. Signed-off-by: Steve Wise <[EMAIL PROTECTED]> --- drivers/infiniband/hw/cxgb3/iwch_qp.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/drivers/infiniband/hw/cxgb3/iwch_qp.c b/drivers/infiniband/hw/cxgb3/iwch_qp.c index e066727..da13a38 100644 --- a/drivers/infiniband/hw/cxgb3/iwch_qp.c +++ b/drivers/infiniband/hw/cxgb3/iwch_qp.c @@ -846,6 +846,8 @@ int iwch_modify_qp(struct iwch_dev *rhp, break; case IWCH_QP_STATE_TERMINATE: qhp->attr.state = IWCH_QP_STATE_TERMINATE; + if (t3b_device(qhp->rhp)) + cxio_set_wq_in_error(&qhp->wq); if (!internal) terminate = 1; break; ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] 2.6.21 iwcm - iw_cm_id destruction race condition fixes.
From: Steve Wise <[EMAIL PROTECTED]>
iwcm iw_cm_id destruction race condition fixes.
Several changes:
- iwcm_deref_id() always wakes up if there's another reference.
- clean up race condition in cm_work_handler().
- create static void free_cm_id() which deallocs the work entries and then
kfrees the cm_id memory. This reduces code replication.
- rem_ref() if this is the last reference -and- the IWCM owns freeing the
cm_id, then free it.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
Signed-off-by: Tom Tucker <[EMAIL PROTECTED]>
Acked-by: Krishna Kumar <[EMAIL PROTECTED]>
---
drivers/infiniband/core/iwcm.c | 47 +---
1 files changed, 25 insertions(+), 22 deletions(-)
diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c
index 1039ad5..891d1fa 100644
--- a/drivers/infiniband/core/iwcm.c
+++ b/drivers/infiniband/core/iwcm.c
@@ -146,6 +146,12 @@ static int copy_private_data(struct iw_c
return 0;
}
+static void free_cm_id(struct iwcm_id_private *cm_id_priv)
+{
+ dealloc_work_entries(cm_id_priv);
+ kfree(cm_id_priv);
+}
+
/*
* Release a reference on cm_id. If the last reference is being
* released, enable the waiting thread (in iw_destroy_cm_id) to
@@ -153,21 +159,14 @@ static int copy_private_data(struct iw_c
*/
static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
{
- int ret = 0;
-
BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
if (atomic_dec_and_test(&cm_id_priv->refcount)) {
BUG_ON(!list_empty(&cm_id_priv->work_list));
- if (waitqueue_active(&cm_id_priv->destroy_comp.wait)) {
- BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
- BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
- &cm_id_priv->flags));
- ret = 1;
- }
complete(&cm_id_priv->destroy_comp);
+ return 1;
}
- return ret;
+ return 0;
}
static void add_ref(struct iw_cm_id *cm_id)
@@ -181,7 +180,11 @@ static void rem_ref(struct iw_cm_id *cm_
{
struct iwcm_id_private *cm_id_priv;
cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
- iwcm_deref_id(cm_id_priv);
+ if (iwcm_deref_id(cm_id_priv) &&
+ test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags)) {
+ BUG_ON(!list_empty(&cm_id_priv->work_list));
+ free_cm_id(cm_id_priv);
+ }
}
static int cm_event_handler(struct iw_cm_id *cm_id, struct iw_cm_event *event);
@@ -355,7 +358,9 @@ static void destroy_cm_id(struct iw_cm_i
case IW_CM_STATE_CONN_RECV:
/*
* App called destroy before/without calling accept after
-* receiving connection request event notification.
+* receiving connection request event notification or
+* returned non zero from the event callback function.
+* In either case, must tell the provider to reject.
*/
cm_id_priv->state = IW_CM_STATE_DESTROYING;
break;
@@ -391,9 +396,7 @@ void iw_destroy_cm_id(struct iw_cm_id *c
wait_for_completion(&cm_id_priv->destroy_comp);
- dealloc_work_entries(cm_id_priv);
-
- kfree(cm_id_priv);
+ free_cm_id(cm_id_priv);
}
EXPORT_SYMBOL(iw_destroy_cm_id);
@@ -647,10 +650,11 @@ static void cm_conn_req_handler(struct i
/* Call the client CM handler */
ret = cm_id->cm_handler(cm_id, iw_event);
if (ret) {
+ iw_cm_reject(cm_id, NULL, 0);
set_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags);
destroy_cm_id(cm_id);
if (atomic_read(&cm_id_priv->refcount)==0)
- kfree(cm_id);
+ free_cm_id(cm_id_priv);
}
out:
@@ -854,13 +858,12 @@ static void cm_work_handler(struct work_
destroy_cm_id(&cm_id_priv->id);
}
BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
- if (iwcm_deref_id(cm_id_priv))
- return;
-
- if (atomic_read(&cm_id_priv->refcount)==0 &&
- test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags)) {
- dealloc_work_entries(cm_id_priv);
- kfree(cm_id_priv);
+ if (iwcm_deref_id(cm_id_priv)) {
+ if (test_bit(IWCM_F_CALLBACK_DESTROY,
+&cm_id_priv->flags)) {
+ BUG_ON(!list_empty(&cm_id_priv->work_list));
+ free_cm_id(cm_id_priv);
+ }
return
Re: [openib-general] [Bug 325] RDMA_CM and address translation broken on sles9sp3
On Wed, 2007-02-14 at 17:05 +0200, Tziporet Koren wrote: > Steve Wise wrote: > > Tziporet, > > > > I didn't think we were going to apply this patch until Michael tested it > > with SDP/IPoIB on various distros. > > > > Michael, did you get a chance to test it (I'm guessing not since you > > were out sick)? > > > > The reason I'm concerned is that it changes the behavior of > > xxx_ip_dev_find() and _all_ backports, and we needed to test it out and > > make sure it doesn't regress anything. If it causes problems on other > > backports, the plan was to just fix the sles9sp3 backport and leave the > > others alone. > > > > With the test build vlad published yesterday which has this patch, > > rhel4u4 kernel wasn't working for me with iWARP and I'm afraid it might > > be due to this patch. I'm investigating this now. > > > > > > > We tested this patch with our regression on IB and its worked fine for > both SDP and IPoIB. > Then we applied it. > Please report ASAP if you think there is an issue. > I undid that change on RHEL4U4 and still see my iwarp rping problem, so its not related... Thanks, Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [Bug 325] RDMA_CM and address translation broken on sles9sp3
Tziporet, I didn't think we were going to apply this patch until Michael tested it with SDP/IPoIB on various distros. Michael, did you get a chance to test it (I'm guessing not since you were out sick)? The reason I'm concerned is that it changes the behavior of xxx_ip_dev_find() and _all_ backports, and we needed to test it out and make sure it doesn't regress anything. If it causes problems on other backports, the plan was to just fix the sles9sp3 backport and leave the others alone. With the test build vlad published yesterday which has this patch, rhel4u4 kernel wasn't working for me with iWARP and I'm afraid it might be due to this patch. I'm investigating this now. Steve. On Wed, 2007-02-14 at 03:54 -0800, [EMAIL PROTECTED] wrote: > https://bugs.openfabrics.org/show_bug.cgi?id=325 > > > > > > --- Comment #2 from [EMAIL PROTECTED] 2007-02-14 03:54 --- > Patch from Steve was applied. > Please check again on alpha1 package. > > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [openfabrics-ewg] new OFED 1.2 package
I installed this on RHEL5 beta 2 with that distro's kernel and RHEL4U4 with a kernel.org 2.6.20 kernel. I successfully configured cxgb3 and mthca and could icmp-ping over both interfaces. I successfully ran rping over both IB and IW. I successfully ran dapltest/regress.sh over both IB and IW. I could _not_ get ib_rdma_bw to run in either cma mode or non-cma mode. The server side exits immediately without an error. ??? I'm blocked on mvapich2/iwarp testing due to the known issues with that package. I tried rping over iwarp on the RHEL4U4 distro's kernel and had problems. I'm thinking the SLES9SP3 fix that was pulled in might have problems on other distros (it changed the behavior of xxx_ip_dev_find() on all backports). I don't think this is stop-ship for alpha1, however. That's it for today. Steve. On Tue, 2007-02-13 at 22:11 +0200, Tziporet Koren wrote: > Vladimir Sokolovsky wrote: > > New OFED package was uploaded to the OFA server: > > http://www.openfabrics.org/~vlad/builds/ofed-1.2/OFED-1.2-20070213-1646.tgz > > > > > > > > Known issues: > > mvapich2 RPM build fails (will be fixed in alpha1). > > sdpnetstat compilation fails in RHEL5 > > > > > > > Hi All, > > This is the pre-alpha package for your testing. > Please send us feedback today so we can build the first alpha OFED tomorrow. > If any show-stopper issue for the alpha is found please let us know. > > Note that components compilation is blocked on kernels that they do not > support. > > Thanks, > Tziporet > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] OFED 1.2 dapl and dat.conf
Currently, the dapl rpms don't install dat.conf. I think they probably should, eh? Maybe in /etc/dat.conf Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ofed_1_2/iw_cxgb3 - Free any pending mmaps in iwch_dealloc_ucontext().
Vlad/Michael,
This should be pushed into ofed_1_2. It can wait until after alpha1,
however, if you want.
Steve.
-
Free any pending mmaps in iwch_dealloc_ucontext().
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch_provider.c |4
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index dbb3f71..4a46771 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -98,7 +98,11 @@ static int iwch_dealloc_ucontext(struct
{
struct iwch_dev *rhp = to_iwch_dev(context->device);
struct iwch_ucontext *ucontext = to_iwch_ucontext(context);
+ struct iwch_mm_entry *mm, *tmp;
+
PDBG("%s context %p\n", __FUNCTION__, context);
+ list_for_each_entry_safe(mm, tmp, &ucontext->mmaps, entry)
+ kfree(mm);
cxio_release_ucontext(&rhp->rdev, &ucontext->uctx);
kfree(ucontext);
return 0;
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
On Tue, 2007-02-13 at 13:01 -0500, Shaun Rowland wrote: > Roland Dreier wrote: > > > How do I tell? Can I tell from the .so files? > > > > ldd on the .so and the app would probably give you good info. > > > > I'm pretty sure that mpicc must be linking against an libibverbs 1.0 > > from somewhere. > > > > - R. > > When I build using the OFED-1.2-20070208-1508, libibverbs 1.0 is what is > built, at least by looking at the .so file result: > > [EMAIL PROTECTED] ~]$ ls /usr/local/ofed/lib64/ |grep ibverbs libibverbs.a > libibverbs.so > libibverbs.so.1 > libibverbs.so.1.0.0 > > This seems odd. Is it correct? I have updated the MVAPICH2 SRPM and sent > a new patch for the OFED install scripts. This won't be reflected until > the alpha1 release. Still, does the above seem strange? I noticed this > recently. I see symbols for both versions though: > > 5a50 T [EMAIL PROTECTED] > 82c0 T ibv_detach_mcast@@IBVERBS_1.1 > A IBVERBS_1.0 > A IBVERBS_1.1 > > Our code links to these libraries, and by default mpicc > should use what's in /usr/local/ofed/lib[64] in the -L path itself > directly too. Is this an issue in the library? The libmpich.so file > should not be any different when built. We will investigate this. > > I can provide a patch against the latest OFED tar.gz to use the > mvapich2-0.9.8-3.src.rpm once I download the release if that would help, > as we have changed some things since the -2 SRPM release. Again, this > should be reflected in the alpha1 release. I was hoping to sniff-test mvapich2 over OFA/iWARP. So if you can get something that works I'll try it out. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
So this program doesn't work: > [EMAIL PROTECTED] ~]$ ldd IMB_2.3/src/IMB-MPI1 > libmpich.so => > /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so (0x2b0d7cefb000) > librdmacm.so => /usr/local/ofed/lib64/librdmacm.so > (0x2b0d7d1b3000) > libibverbs.so.1 => /usr/local/ofed/lib64/libibverbs.so.1 > (0x2b0d7d2b8000) > libibumad.so.1 => /usr/local/ofed/lib64/libibumad.so.1 > (0x2b0d7d3c3000) > libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x003e0700) > librt.so.1 => /lib64/tls/librt.so.1 (0x003e0ba0) > libc.so.6 => /lib64/tls/libc.so.6 (0x003e0650) > libsysfs.so.1 => /usr/lib64/libsysfs.so.1 (0x003e06a0) > libdl.so.2 => /lib64/libdl.so.2 (0x003e0630) > libibcommon.so.1 => /usr/local/ofed/lib64/libibcommon.so.1 > (0x2b0d7d4cf000) > /lib64/ld-linux-x86-64.so.2 (0x003e0610) > And this one does: [EMAIL PROTECTED] ~]# ldd /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/examples/cpi libm.so.6 => /lib64/tls/libm.so.6 (0x003e0680) librdmacm.so => /usr/local/ofed/lib64/librdmacm.so (0x2b1353b65000) libibverbs.so.1 => /usr/local/ofed/lib64/libibverbs.so.1 (0x2b1353c6a000) libibumad.so.1 => /usr/local/ofed/lib64/libibumad.so.1 (0x2b1353d75000) libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x003e0700) librt.so.1 => /lib64/tls/librt.so.1 (0x003e0ba0) libc.so.6 => /lib64/tls/libc.so.6 (0x003e0650) libdl.so.2 => /lib64/libdl.so.2 (0x003e0630) libsysfs.so.1 => /usr/lib64/libsysfs.so.1 (0x003e06a0) libibcommon.so.1 => /usr/local/ofed/lib64/libibcommon.so.1 (0x2b1353e81000) /lib64/ld-linux-x86-64.so.2 (0x003e0610) Note the cpi program doesn't dynamically link with libmpich.so. That appears to be the difference... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
On Tue, 2007-02-13 at 09:21 -0800, Roland Dreier wrote: > > How do I tell? Can I tell from the .so files? > > ldd on the .so and the app would probably give you good info. > > I'm pretty sure that mpicc must be linking against an libibverbs 1.0 > from somewhere. > > - R. By the way, the problem also happens running over mthca/IB with librdmacm. mpicc has '-libverbs' mpicc.conf has '-libverbs' too. ldd output. Looks like they are all linking to libibverbs.so.1. Is that correct? [EMAIL PROTECTED] ~]$ ldd IMB_2.3/src/IMB-MPI1 libmpich.so => /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so (0x2b0d7cefb000) librdmacm.so => /usr/local/ofed/lib64/librdmacm.so (0x2b0d7d1b3000) libibverbs.so.1 => /usr/local/ofed/lib64/libibverbs.so.1 (0x2b0d7d2b8000) libibumad.so.1 => /usr/local/ofed/lib64/libibumad.so.1 (0x2b0d7d3c3000) libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x003e0700) librt.so.1 => /lib64/tls/librt.so.1 (0x003e0ba0) libc.so.6 => /lib64/tls/libc.so.6 (0x003e0650) libsysfs.so.1 => /usr/lib64/libsysfs.so.1 (0x003e06a0) libdl.so.2 => /lib64/libdl.so.2 (0x003e0630) libibcommon.so.1 => /usr/local/ofed/lib64/libibcommon.so.1 (0x2b0d7d4cf000) /lib64/ld-linux-x86-64.so.2 (0x003e0610) [EMAIL PROTECTED] ~]$ ldd /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so libc.so.6 => /lib64/tls/libc.so.6 (0x2b6a6061d000) /lib64/ld-linux-x86-64.so.2 (0x4000) [EMAIL PROTECTED] ~]$ ldd /usr/local/ofed/lib64/librdmacm.so libibverbs.so.1 => /usr/local/ofed/lib64/libibverbs.so.1 (0x2b3ef50de000) libsysfs.so.1 => /usr/lib64/libsysfs.so.1 (0x2b3ef51ea000) libc.so.6 => /lib64/tls/libc.so.6 (0x2b3ef52f6000) libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x2b3ef552a000) libdl.so.2 => /lib64/libdl.so.2 (0x2b3ef564) /lib64/ld-linux-x86-64.so.2 (0x4000) [EMAIL PROTECTED] ~]$ ldd /usr/local/ofed/lib64/libcxgb3-rdmav2.so libibverbs.so.1 => /usr/local/ofed/lib64/libibverbs.so.1 (0x2b83c160e000) libc.so.6 => /lib64/tls/libc.so.6 (0x2b83c171a000) libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x2b83c194e000) libdl.so.2 => /lib64/libdl.so.2 (0x2b83c1a63000) /lib64/ld-linux-x86-64.so.2 (0x4000) [EMAIL PROTECTED] ~]$ ldd /usr/local/ofed/lib64/libcxgb3.so libibverbs.so.1 => /usr/local/ofed/lib64/libibverbs.so.1 (0x2ac8e492) libc.so.6 => /lib64/tls/libc.so.6 (0x2ac8e4a2c000) libpthread.so.0 => /lib64/tls/libpthread.so.0 (0x2ac8e4c6) libdl.so.2 => /lib64/libdl.so.2 (0x2ac8e4d75000) /lib64/ld-linux-x86-64.so.2 (0x4000) ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] mvapich2 ofed 1.2 problem
On Tue, 2007-02-13 at 09:07 -0800, Roland Dreier wrote: > > Does this stack indicate that libibverbs is accessing a 1.0 provider? > > cxgb3 shouldn't be 1.0 right? > > > #1 0x2b832d4d4381 in __ibv_alloc_pd_1_0 (context=0x617830) > > at src/compat-1_0.c:572 > > #2 0x2b832cfef04e in rdma_cm_init_pd_cq () > >from /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so > > This means that the app (or maybe the RDMA CM library?) is linked > against the 1.0 API -- which should work even with cxgb3 actually. > But maybe mvapich is built against the 1.1 API and the RDMA CM is > built against 1.0 or something? > How do I tell? Can I tell from the .so files? I can build a non-mpi app against the librdmacm and libibverbs that got installed and things work ok. So maybe libmpich is balled up somehow. Interestingly, the mpi example program, cpi, that gets built with the rpm works. Its just mpi programs that I build using the mpicc which links to the libmpich.so Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] mvapich2 ofed 1.2 problem
Hey Roland,
Does this stack indicate that libibverbs is accessing a 1.0 provider?
cxgb3 shouldn't be 1.0 right?
Core was generated by `IMB_2.3/src/IMB-MPI1'.
Program terminated with signal 11, Segmentation fault.
...
(gdb) bt
#0 __ibv_alloc_pd (context=0x1) at src/verbs.c:143
#1 0x2b832d4d4381 in __ibv_alloc_pd_1_0 (context=0x617830)
at src/compat-1_0.c:572
#2 0x2b832cfef04e in rdma_cm_init_pd_cq ()
from /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so
#3 0x2b832cfef415 in rdma_cm_create_qp ()
from /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so
#4 0x2b832cfefa37 in ib_cma_event_handler ()
from /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so
#5 0x2b832cfefcc0 in cm_thread ()
from /usr/local/ofed/mpi/gcc/mvapich2-0.9.8-3/lib/libmpich.so
#6 0x003cd9406305 in start_thread () from /lib64/libpthread.so.0
#7 0x003cd88cd66d in clone () from /lib64/libc.so.6
#8 0x in ?? ()
(gdb) p *context
Cannot access memory at address 0x1
(gdb) up
#1 0x2b832d4d4381 in __ibv_alloc_pd_1_0 (context=0x617830)
at src/compat-1_0.c:572
572 src/compat-1_0.c: No such file or directory.
in src/compat-1_0.c
(gdb) p *context
$1 = {device = 0x617100, ops = {
query_device = 0x2b832dcf2bc0 ,
query_port = 0x2b832dcf2ba0 ,
alloc_pd = 0x2b832dcf2b30 ,
dealloc_pd = 0x2b832dcf2af0 ,
reg_mr = 0x2b832dcf29b0 ,
dereg_mr = 0x2b832dcf2c30 ,
create_cq = 0x2b832dcf3050 ,
poll_cq = 0x2b832dcf1770 ,
req_notify_cq = 0x2b832dcf10c0 , cq_event = 0,
resize_cq = 0x2b832dcf2870 ,
destroy_cq = 0x2b832dcf2f50 ,
create_srq = 0x2b832dcf2880 ,
modify_srq = 0x2b832dcf2890 , query_srq = 0,
destroy_srq = 0x2b832dcf28a0 ,
post_srq_recv = 0x2b832dcf28b0 ,
create_qp = 0x2b832dcf2d30 , query_qp = 0,
modify_qp = 0x2b832dcf2900 ,
destroy_qp = 0x2b832dcf3200 ,
post_send = 0x2b832dcf1fa0 ,
post_recv = 0x2b832dcf2460 ,
create_ah = 0x2b832dcf28c0 ,
destroy_ah = 0x2b832dcf28d0 ,
attach_mcast = 0x2b832dcf28e0 ,
detach_mcast = 0x2b832dcf28f0 }, cmd_fd = 768552128,
async_fd = 11139, num_comp_vectors = 8, real_context = 0x1}
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] issues with compilation of ofed 1.2
Ok, I hacked around this by changing the build_env.sh. But I think build_env.sh will have to distinguish between rhel5 and all other redhat releases so it can correctly set the prerequisite rpms... Steve. On Mon, 2007-02-12 at 17:24 -0600, Steve Wise wrote: > I still get this error building on rhel5b2 with the latest from the ofa > git trees: > > ERROR: The sysfsutils-devel package is required to build libibverbs_devel RPM > [EMAIL PROTECTED] OFED-1.2-stevo]# rpm -qa|grep sysfs > libsysfs-2.0.0-6 > libsysfs-devel-2.0.0-6 > libsysfs-2.0.0-6 > sysfsutils-2.0.0-6 > libsysfs-devel-2.0.0-6 > > > I installed all the sysfs rpms I could find. So is there some > dependency problem here in the OFED build script that is looking for the > wrong rpm in rhel5? > > Is there a bug to track this issue? > > Steve. > > > On Thu, 2007-02-08 at 17:28 -0500, Doug Ledford wrote: > > On Thu, 2007-02-08 at 09:02 +0200, Moni Levy wrote: > > > Doug, > > > On 2/7/07, Yosef Etigin <[EMAIL PROTECTED]> wrote: > > > > 7. On RHAS5 beta 2, the setup requires sysfstuils-devel RPM which is > > > > not included in this distro. > > > > > > Can you please help us with that ? > > > > The value of the sysfsutils is far overshadowed by the value of libsysfs > > (and libsysfs is far more commonly used). So, in RHEL5, the rpm package > > names reflect this: > > > > libsysfs > > sysfsutils (I think, might be libsysfs-utils) > > libsysfs-devel > > > > It's all still there, just a different name. > > > > > -- Moni > > > > > > > > > > > -- > > > > Yosef Etigin > > > > Alex Tabachnik > > > > > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] issues with compilation of ofed 1.2
I still get this error building on rhel5b2 with the latest from the ofa git trees: ERROR: The sysfsutils-devel package is required to build libibverbs_devel RPM [EMAIL PROTECTED] OFED-1.2-stevo]# rpm -qa|grep sysfs libsysfs-2.0.0-6 libsysfs-devel-2.0.0-6 libsysfs-2.0.0-6 sysfsutils-2.0.0-6 libsysfs-devel-2.0.0-6 I installed all the sysfs rpms I could find. So is there some dependency problem here in the OFED build script that is looking for the wrong rpm in rhel5? Is there a bug to track this issue? Steve. On Thu, 2007-02-08 at 17:28 -0500, Doug Ledford wrote: > On Thu, 2007-02-08 at 09:02 +0200, Moni Levy wrote: > > Doug, > > On 2/7/07, Yosef Etigin <[EMAIL PROTECTED]> wrote: > > > 7. On RHAS5 beta 2, the setup requires sysfstuils-devel RPM which is not > > > included in this distro. > > > > Can you please help us with that ? > > The value of the sysfsutils is far overshadowed by the value of libsysfs > (and libsysfs is far more commonly used). So, in RHEL5, the rpm package > names reflect this: > > libsysfs > sysfsutils (I think, might be libsysfs-utils) > libsysfs-devel > > It's all still there, just a different name. > > > -- Moni > > > > > > > > -- > > > Yosef Etigin > > > Alex Tabachnik > > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH][RFC] iw_cxgb3/2.6.21 - Don't use the physical address for mapping memory into userspace.
On Mon, 2007-02-12 at 12:23 -0800, Roland Dreier wrote:
> Actually, that patch doesn't apply because of the "%llx" warning fixes
> I pushed out. And git-apply also complains about trailing
> whitespace. Can you resend a version that applies to the my
> for-2.6.21 branch?
>
> Thanks
Here it is...
Don't use the physical address for mapping memory into userspace.
From: Steve Wise <[EMAIL PROTECTED]>
Currently iw_cxgb3 uses the physical address as the key/offset to return
to the user process for maping kernel memory into userspace. The user
process then calls mmap() using this key as the offset. Because the
physical address is 64 bits, this introduces a problem with 32-bit
userspace, which might not be able to pass an arbitrary 64-bit address
back into the kernel (since mmap2() is limited to a 32-bit number of
pages for the offset, which limits it to 44-bit addresses).
Change the mmap logic to use a u32 counter as the offset for mapping.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch_provider.c | 66 +--
drivers/infiniband/hw/cxgb3/iwch_provider.h | 14 +++---
drivers/infiniband/hw/cxgb3/iwch_user.h |6 +-
3 files changed, 52 insertions(+), 34 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index 549de0a..2e05e94 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -115,7 +115,7 @@ static struct ib_ucontext *iwch_alloc_uc
struct iwch_dev *rhp = to_iwch_dev(ibdev);
PDBG("%s ibdev %p\n", __FUNCTION__, ibdev);
- context = kmalloc(sizeof(*context), GFP_KERNEL);
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
if (!context)
return ERR_PTR(-ENOMEM);
cxio_init_ucontext(&rhp->rdev, &context->uctx);
@@ -141,13 +141,14 @@ static int iwch_destroy_cq(struct ib_cq
}
static struct ib_cq *iwch_create_cq(struct ib_device *ibdev, int entries,
-struct ib_ucontext *context,
+struct ib_ucontext *ib_context,
struct ib_udata *udata)
{
struct iwch_dev *rhp;
struct iwch_cq *chp;
struct iwch_create_cq_resp uresp;
struct iwch_create_cq_req ureq;
+ struct iwch_ucontext *ucontext = NULL;
PDBG("%s ib_dev %p entries %d\n", __FUNCTION__, ibdev, entries);
rhp = to_iwch_dev(ibdev);
@@ -155,12 +156,15 @@ static struct ib_cq *iwch_create_cq(stru
if (!chp)
return ERR_PTR(-ENOMEM);
- if (context && !t3a_device(rhp)) {
- if (ib_copy_from_udata(&ureq, udata, sizeof (ureq))) {
- kfree(chp);
- return ERR_PTR(-EFAULT);
+ if (ib_context) {
+ ucontext = to_iwch_ucontext(ib_context);
+ if (!t3a_device(rhp)) {
+ if (ib_copy_from_udata(&ureq, udata, sizeof (ureq))) {
+ kfree(chp);
+ return ERR_PTR(-EFAULT);
+ }
+ chp->user_rptr_addr = (u32 __user *)(unsigned
long)ureq.user_rptr_addr;
}
- chp->user_rptr_addr = (u32 __user *)(unsigned
long)ureq.user_rptr_addr;
}
if (t3a_device(rhp)) {
@@ -190,7 +194,7 @@ static struct ib_cq *iwch_create_cq(stru
init_waitqueue_head(&chp->wait);
insert_handle(rhp, &rhp->cqidr, chp, chp->cq.cqid);
- if (context) {
+ if (ucontext) {
struct iwch_mm_entry *mm;
mm = kmalloc(sizeof *mm, GFP_KERNEL);
@@ -200,16 +204,20 @@ static struct ib_cq *iwch_create_cq(stru
}
uresp.cqid = chp->cq.cqid;
uresp.size_log2 = chp->cq.size_log2;
- uresp.physaddr = virt_to_phys(chp->cq.queue);
+ spin_lock(&ucontext->mmap_lock);
+ uresp.key = ucontext->key;
+ ucontext->key += PAGE_SIZE;
+ spin_unlock(&ucontext->mmap_lock);
if (ib_copy_to_udata(udata, &uresp, sizeof (uresp))) {
kfree(mm);
iwch_destroy_cq(&chp->ibcq);
return ERR_PTR(-EFAULT);
}
- mm->addr = uresp.physaddr;
+ mm->key = uresp.key;
+ mm->addr = virt_to_phys(chp->cq.queue);
mm->len = PAGE_ALIGN((1UL << uresp.size_log2) *
sizeof (struct t3_cqe));
- insert_mmap(to_iwch_ucontext(context), mm);
+ insert_mmap(ucontext, mm);
}
PDBG("created cqid 0x%0x chp %p size 0x%0x, dm
Re: [openib-general] [PATCH][RFC] iw_cxgb3/2.6.21 - Don't use the physical address for mapping memory into userspace.
On Mon, 2007-02-12 at 12:08 -0800, Roland Dreier wrote: > > Because the key generator u32 is in the context now, and the kzalloc() > > initializes it. I could have done: > > > > context->key = 0; > > > > But km -> kz was less typing. ;-) > > OK, got it. Anyway as I said, from a quick read the changes look > sane, with the assumption that they work. I tested and it works. Do you want to pull this in before you push the driver upstream? Do I need to repost it? Thanks, Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH][RFC] iw_cxgb3/2.6.21 - Don't use the physical address for mapping memory into userspace.
On Mon, 2007-02-12 at 11:58 -0800, Roland Dreier wrote: > Looks mostly sane (assuming it works on 32-bit userspace on 64-bit > kernel now), but: > > > - context = kmalloc(sizeof(*context), GFP_KERNEL); > > + context = kzalloc(sizeof(*context), GFP_KERNEL); > > Why do you need this? Is this an unrelated change? > Because the key generator u32 is in the context now, and the kzalloc() initializes it. I could have done: context->key = 0; But km -> kz was less typing. ;-) Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH][RFC] iw_cxgb3/2.6.21 - Don't use the physical address for mapping memory into userspace.
Roland, can you review this?
-
From: Steve Wise <[EMAIL PROTECTED]>
Currently iw_cxgb3 uses the physical address as the key/offset to return
to the user process for maping kernel memory into userspace. The user
process then calls mmap() using this key as the offset. Because the
physical address is 64 bits, this introduces a problem with 32-bit
userspace, which might not be able to pass an arbitrary 64-bit address
back into the kernel (since mmap2() is limited to a 32-bit number of
pages for the offset, which limits it to 44-bit addresses).
Change the mmap logic to use a u32 counter as the offset for mapping.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch_provider.c | 66 +--
drivers/infiniband/hw/cxgb3/iwch_provider.h | 13 +++--
drivers/infiniband/hw/cxgb3/iwch_user.h |6 +-
3 files changed, 52 insertions(+), 33 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index d02cd72..b2c88d6 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -115,7 +115,7 @@ static struct ib_ucontext *iwch_alloc_uc
struct iwch_dev *rhp = to_iwch_dev(ibdev);
PDBG("%s ibdev %p\n", __FUNCTION__, ibdev);
- context = kmalloc(sizeof(*context), GFP_KERNEL);
+ context = kzalloc(sizeof(*context), GFP_KERNEL);
if (!context)
return ERR_PTR(-ENOMEM);
cxio_init_ucontext(&rhp->rdev, &context->uctx);
@@ -141,13 +141,14 @@ static int iwch_destroy_cq(struct ib_cq
}
static struct ib_cq *iwch_create_cq(struct ib_device *ibdev, int entries,
-struct ib_ucontext *context,
+struct ib_ucontext *ib_context,
struct ib_udata *udata)
{
struct iwch_dev *rhp;
struct iwch_cq *chp;
struct iwch_create_cq_resp uresp;
struct iwch_create_cq_req ureq;
+ struct iwch_ucontext *ucontext = NULL;
PDBG("%s ib_dev %p entries %d\n", __FUNCTION__, ibdev, entries);
rhp = to_iwch_dev(ibdev);
@@ -155,12 +156,15 @@ static struct ib_cq *iwch_create_cq(stru
if (!chp)
return ERR_PTR(-ENOMEM);
- if (context && !t3a_device(rhp)) {
- if (ib_copy_from_udata(&ureq, udata, sizeof (ureq))) {
- kfree(chp);
- return ERR_PTR(-EFAULT);
+ if (ib_context) {
+ ucontext = to_iwch_ucontext(ib_context);
+ if (!t3a_device(rhp)) {
+ if (ib_copy_from_udata(&ureq, udata, sizeof (ureq))) {
+ kfree(chp);
+ return ERR_PTR(-EFAULT);
+ }
+ chp->user_rptr_addr = (u32 __user *)(unsigned
long)ureq.user_rptr_addr;
}
- chp->user_rptr_addr = (u32 __user *)(unsigned
long)ureq.user_rptr_addr;
}
if (t3a_device(rhp)) {
@@ -190,7 +194,7 @@ static struct ib_cq *iwch_create_cq(stru
init_waitqueue_head(&chp->wait);
insert_handle(rhp, &rhp->cqidr, chp, chp->cq.cqid);
- if (context) {
+ if (ucontext) {
struct iwch_mm_entry *mm;
mm = kmalloc(sizeof *mm, GFP_KERNEL);
@@ -200,16 +204,20 @@ static struct ib_cq *iwch_create_cq(stru
}
uresp.cqid = chp->cq.cqid;
uresp.size_log2 = chp->cq.size_log2;
- uresp.physaddr = virt_to_phys(chp->cq.queue);
+ spin_lock(&ucontext->mmap_lock);
+ uresp.key = ucontext->key;
+ ucontext->key += PAGE_SIZE;
+ spin_unlock(&ucontext->mmap_lock);
if (ib_copy_to_udata(udata, &uresp, sizeof (uresp))) {
kfree(mm);
iwch_destroy_cq(&chp->ibcq);
return ERR_PTR(-EFAULT);
}
- mm->addr = uresp.physaddr;
+ mm->key = uresp.key;
+ mm->addr = virt_to_phys(chp->cq.queue);
mm->len = PAGE_ALIGN((1UL << uresp.size_log2) *
sizeof (struct t3_cqe));
- insert_mmap(to_iwch_ucontext(context), mm);
+ insert_mmap(ucontext, mm);
}
PDBG("created cqid 0x%0x chp %p size 0x%0x, dma_addr 0x%0llx\n",
chp->cq.cqid, chp, (1 << chp->cq.size_log2),
@@ -316,14 +324,14 @@ static int iwch_arm_cq(struct ib_cq *ibc
static int iwch_mmap(struct ib_ucontext *context, struct vm_area_struct *vma)
{
int len = vma->vm_end - vma->vm_start;
- u64 pgaddr = vma->vm_pgoff << PAGE_SHIFT;
+ u32 key = vma
Re: [openib-general] cxgb3 compilation fails on RHEL4.0U3
On Mon, 2007-02-12 at 19:56 +0200, Vladimir Sokolovsky wrote: > On Mon, 2007-02-12 at 10:55 -0600, Steve Wise wrote: > > I only backported to RHEL4U4 since that was the supported platform. > > > > Is OFED 1.2 supporting U3 too? > > > > I can add the backport if needed. > > > > > RHEL4U3 is not officially supported but there are some patches for cxgb3 > under kernel_patches/backport/2.6.9_U3: > > kernel_patches/backport/2.6.9_U3/cxgb3_main_to_2_6_13.patch > kernel_patches/backport/2.6.9_U3/cxgb3_makefile_to_2_6_19.patch Looks like Michael added this with commit: ea110866d640317fe889abdc3aaba317ae20da65 For alpha1, please just don't build cxgb3/libcxb3 for RHEL4U3. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] cxgb3 compilation fails on RHEL4.0U3
I only backported to RHEL4U4 since that was the supported platform. Is OFED 1.2 supporting U3 too? I can add the backport if needed. On Mon, 2007-02-12 at 18:06 +0200, Vladimir Sokolovsky wrote: > Hi Steve, > I got the following compilation failure on RHEL4.0U3 (2.6.9-34.ELsmp): > > gcc > -Wp,-MD,/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/.cxgb3_offload.o.d > -nostdinc -iwithprefix include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/kernel_addons/backport/2.6.9_U3/include/ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/include -Iinclude >-include include/linux/autoconf.h -include > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include/linux/autoconf.h -D__KERNEL__ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/kernel_addons/backport/2.6.9_U3/include/ > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/include -Iinclude >-include include/linux/autoconf.h -include > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include/linux/autoconf.h -Wall > -Wstrict-prototypes -Wno-trigraphs -fno-strict-aliasing -fno-common -Os > -fomit-frame-pointer -g -Wdeclaration-after-statement -mno-red-zone > -mcmodel=kernel -pipe -fno-reorder-blocks -Wno-sign-compare - f! > unit-at-a-time -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/include > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/ulp/ipoib > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/debug > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/cxgb3/core > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3 > -I/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/rds -DMODULE > -DKBUILD_BASENAME=cxgb3_offload -DKBUILD_MODNAME=cxgb3 -c -o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/.tmp_cxgb3_offload.o > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:57: > error: syntax error before "adapter_list_lock" > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:57: > warning: type defaults to `int' in declaration of `adapter_list_lock' > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:57: > error: incompatible types in initialization > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:57: > error: initializer element is not constant > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:57: > warning: data definition has no type or storage class > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c: In > function `is_offloading': > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:885: > warning: passing arg 1 of `_read_lock_bh' from incompatible pointer type > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:889: > warning: passing arg 1 of `_read_unlock_bh' from incompatible pointer type > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:894: > warning: passing arg 1 of `_read_unlock_bh' from incompatible pointer type > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c: In > function `add_adapter': > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:1062: > warning: passing arg 1 of `_write_lock_bh' from incompatible pointer type > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:1064: > warning: passing arg 1 of `_write_unlock_bh' from incompatible pointer type > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c: In > function `remove_adapter': > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:1069: > warning: passing arg 1 of `_write_lock_bh' from incompatible pointer type > /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.c:1071: > warning: passing arg 1 of `_write_unlock_bh' from incompatible pointer type > make[3]: *** > [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3/cxgb3_offload.o] > Error 1 > make[2]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/net/cxgb3] Error 2 > make[1]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2] Error 2 > make[1]: Leaving directory `/usr/src/kernels/2.6.9-34.EL-smp-x86_64' > make: *** [kernel] Error 2 > > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] ofed_1-2 IWCM - Set iniator depth and responder resources to device max values.
BTW: We need this for the alpha1 build or DAPL applications won't work
over iWARP devices.
Steve.
On Sun, 2007-02-11 at 13:58 -0600, Steve WIse wrote:
> IWCM - Set initiator depth and responder resources to device max values.
>
> For OFED 1.2, the IWCM will set the initiator depth and responder
> resources to the device max values for new connect request events.
>
>
> Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
> ---
>
> kernel_patches/fixes/iwcm_ordird.patch | 43
>
> 1 files changed, 43 insertions(+), 0 deletions(-)
>
> diff --git a/kernel_patches/fixes/iwcm_ordird.patch
> b/kernel_patches/fixes/iwcm_ordird.patch
> new file mode 100644
> index 000..3a9f643
> --- /dev/null
> +++ b/kernel_patches/fixes/iwcm_ordird.patch
> @@ -0,0 +1,43 @@
> +commit 7175034c7adf6b5fb5ba311929376af7501387a1
> +Author: Steve Wise <[EMAIL PROTECTED]>
> +Date: Sat Feb 10 14:16:35 2007 -0600
> +
> +IWCM - Set iniator depth and responder resources to device max values.
> +
> +For OFED 1.2, the IWCM will set the initiator depth and responder
> +resources to the device max values for new connect request events.
> +
> +Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
> +
> +diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
> +index 9e0ab04..e3afdf8 100644
> +--- a/drivers/infiniband/core/cma.c
> b/drivers/infiniband/core/cma.c
> +@@ -1137,6 +1137,7 @@ static int iw_conn_req_handler(struct iw
> + struct net_device *dev = NULL;
> + struct rdma_cm_event event;
> + int ret;
> ++struct ib_device_attr attr;
> +
> + listen_id = cm_id->context;
> + atomic_inc(&listen_id->dev_remove);
> +@@ -1189,10 +1190,19 @@ static int iw_conn_req_handler(struct iw
> + sin = (struct sockaddr_in *) &new_cm_id->route.addr.dst_addr;
> + *sin = iw_event->remote_addr;
> +
> ++ret = ib_query_device(conn_id->id.device, &attr);
> ++if (ret) {
> ++cma_release_remove(conn_id);
> ++rdma_destroy_id(new_cm_id);
> ++goto out;
> ++}
> ++
> + memset(&event, 0, sizeof event);
> + event.event = RDMA_CM_EVENT_CONNECT_REQUEST;
> + event.param.conn.private_data = iw_event->private_data;
> + event.param.conn.private_data_len = iw_event->private_data_len;
> ++event.param.conn.initiator_depth = attr.max_qp_init_rd_atom;
> ++event.param.conn.responder_resources = attr.max_qp_rd_atom;
> + ret = conn_id->id.event_handler(&conn_id->id, &event);
> + if (ret) {
> + /* User wants to destroy the CM ID */
>
>
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] OFED 1.2 build problem
Dunno if this has already been resolved? Building the 20070208-1508 OFED 1.2 kit. RHEL3U4 with that distro's kernel. Ran build.sh and selected "all". It fails building ipath: /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.c:44:22: linux/io.h: No such file or directory /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.c: In function `ipath_diag_open': /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.c:283: warning: implicit declaration of function `mutex_lock' /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.c:308: warning: implicit declaration of function `mutex_unlock' /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.c: In function `ipath_diagpkt_write': /var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.c:429: warning: implicit declaration of function `__iowrite32_copy' make[4]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath/ipath_diag.o] Error 1 make[3]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband/hw/ipath] Error 2 make[2]: *** [/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2/drivers/infiniband] Error 2 make[1]: *** [_module_/var/tmp/OFEDRPM/BUILD/ofa_kernel-1.2] Error 2 make[1]: Leaving directory `/usr/src/kernels/2.6.9-42.EL-smp-x86_64' make: *** [kernel] Error 2 error: Bad exit status from /var/tmp/rpm-tmp.66790 (%install) ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] iw_cxgb3 Change cxio semaphore to mutex.
From: Steve Wise <[EMAIL PROTECTED]>
Change cxio semaphore to mutex.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/cxio_hal.c | 10 +-
drivers/infiniband/hw/cxgb3/cxio_hal.h |4 ++--
2 files changed, 7 insertions(+), 7 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.c
b/drivers/infiniband/hw/cxgb3/cxio_hal.c
index 19553b3..de3cb15 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_hal.c
+++ b/drivers/infiniband/hw/cxgb3/cxio_hal.c
@@ -30,9 +30,9 @@
* CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
* SOFTWARE.
*/
-#include
#include
+#include
#include
#include
#include
@@ -527,7 +527,7 @@ static int cxio_hal_init_ctrl_qp(struct
memset(rdev_p->ctrl_qp.workq, 0,
(1 << T3_CTRL_QP_SIZE_LOG2) * sizeof(union t3_wr));
- init_MUTEX(&rdev_p->ctrl_qp.sem);
+ mutex_init(&rdev_p->ctrl_qp.lock);
init_waitqueue_head(&rdev_p->ctrl_qp.waitq);
/* update HW Ctrl QP context */
@@ -570,7 +570,7 @@ static int cxio_hal_destroy_ctrl_qp(stru
/* write len bytes of data into addr (32B aligned address)
* If data is NULL, clear len byte of memory to zero.
- * caller aquires the sem before the call
+ * caller aquires the ctrl_qp lock before the call
*/
static int cxio_hal_ctrl_qp_write_mem(struct cxio_rdev *rdev_p, u32 addr,
u32 len, void *data, int completion)
@@ -705,7 +705,7 @@ static int __cxio_tpt_op(struct cxio_rde
}
}
- down_interruptible(&rdev_p->ctrl_qp.sem);
+ mutex_lock(&rdev_p->ctrl_qp.lock);
/* write PBL first if any - update pbl only if pbl list exist */
if (pbl) {
@@ -752,7 +752,7 @@ static int __cxio_tpt_op(struct cxio_rde
cxio_hal_put_stag(rdev_p->rscp, stag_idx);
ret:
wptr = rdev_p->ctrl_qp.wptr;
- up(&rdev_p->ctrl_qp.sem);
+ mutex_unlock(&rdev_p->ctrl_qp.lock);
if (!err)
if (wait_event_interruptible(rdev_p->ctrl_qp.waitq,
SEQ32_GE(rdev_p->ctrl_qp.rptr,
diff --git a/drivers/infiniband/hw/cxgb3/cxio_hal.h
b/drivers/infiniband/hw/cxgb3/cxio_hal.h
index 8fb2999..1b97e80 100644
--- a/drivers/infiniband/hw/cxgb3/cxio_hal.h
+++ b/drivers/infiniband/hw/cxgb3/cxio_hal.h
@@ -62,8 +62,8 @@ #define T3_MAX_DEV_NAME_LEN 32
struct cxio_hal_ctrl_qp {
u32 wptr;
u32 rptr;
- struct semaphore sem; /* for the wtpr, can sleep */
- wait_queue_head_t waitq;/* wait for RspQ/CQE msg */
+ struct mutex lock; /* for the wtpr, can sleep */
+ wait_queue_head_t waitq;/* wait for RspQ/CQE msg */
union t3_wr *workq; /* the work request queue */
dma_addr_t dma_addr;/* pci bus address of the workq */
DECLARE_PCI_UNMAP_ADDR(mapping)
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] ofed_1-2 IWCM - Set iniator depth and responder resources to device max values.
IWCM - Set initiator depth and responder resources to device max values.
For OFED 1.2, the IWCM will set the initiator depth and responder
resources to the device max values for new connect request events.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
kernel_patches/fixes/iwcm_ordird.patch | 43
1 files changed, 43 insertions(+), 0 deletions(-)
diff --git a/kernel_patches/fixes/iwcm_ordird.patch
b/kernel_patches/fixes/iwcm_ordird.patch
new file mode 100644
index 000..3a9f643
--- /dev/null
+++ b/kernel_patches/fixes/iwcm_ordird.patch
@@ -0,0 +1,43 @@
+commit 7175034c7adf6b5fb5ba311929376af7501387a1
+Author: Steve Wise <[EMAIL PROTECTED]>
+Date: Sat Feb 10 14:16:35 2007 -0600
+
+IWCM - Set iniator depth and responder resources to device max values.
+
+For OFED 1.2, the IWCM will set the initiator depth and responder
+resources to the device max values for new connect request events.
+
+Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
+
+diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
+index 9e0ab04..e3afdf8 100644
+--- a/drivers/infiniband/core/cma.c
b/drivers/infiniband/core/cma.c
+@@ -1137,6 +1137,7 @@ static int iw_conn_req_handler(struct iw
+ struct net_device *dev = NULL;
+ struct rdma_cm_event event;
+ int ret;
++ struct ib_device_attr attr;
+
+ listen_id = cm_id->context;
+ atomic_inc(&listen_id->dev_remove);
+@@ -1189,10 +1190,19 @@ static int iw_conn_req_handler(struct iw
+ sin = (struct sockaddr_in *) &new_cm_id->route.addr.dst_addr;
+ *sin = iw_event->remote_addr;
+
++ ret = ib_query_device(conn_id->id.device, &attr);
++ if (ret) {
++ cma_release_remove(conn_id);
++ rdma_destroy_id(new_cm_id);
++ goto out;
++ }
++
+ memset(&event, 0, sizeof event);
+ event.event = RDMA_CM_EVENT_CONNECT_REQUEST;
+ event.param.conn.private_data = iw_event->private_data;
+ event.param.conn.private_data_len = iw_event->private_data_len;
++ event.param.conn.initiator_depth = attr.max_qp_init_rd_atom;
++ event.param.conn.responder_resources = attr.max_qp_rd_atom;
+ ret = conn_id->id.event_handler(&conn_id->id, &event);
+ if (ret) {
+ /* User wants to destroy the CM ID */
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] RDMA/iwcm: Bugs in cm_conn_req_handler()
On Sat, 2007-02-10 at 14:36 -0600, Steve Wise wrote:
> ugh.
>
> There is at least one bug in this patch. I cannot call iw_cm_reject()
> inside destroy_cm_id() because both functions grab the iw_cm lock...
>
>
This patch puts the iw_cm_reject() calls back in
cm_conn_req_handler()...
---
iw_cm_id destruction race condition fixes.
From: Steve Wise <[EMAIL PROTECTED]>
Several changes:
- iwcm_deref_id() always wakes up if there's another reference.
- clean up race condition in cm_work_handler().
- create static void free_cm_id() which deallocs the work entries and then
kfrees the cm_id memory. This reduces code replication.
- rem_ref() if this is the last reference -and- the IWCM owns freeing the
cm_id, then free it.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
Signed-off-by: Tom Tucker <[EMAIL PROTECTED]>
---
drivers/infiniband/core/iwcm.c | 47 +---
1 files changed, 25 insertions(+), 22 deletions(-)
diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c
index 1039ad5..891d1fa 100644
--- a/drivers/infiniband/core/iwcm.c
+++ b/drivers/infiniband/core/iwcm.c
@@ -146,6 +146,12 @@ static int copy_private_data(struct iw_c
return 0;
}
+static void free_cm_id(struct iwcm_id_private *cm_id_priv)
+{
+ dealloc_work_entries(cm_id_priv);
+ kfree(cm_id_priv);
+}
+
/*
* Release a reference on cm_id. If the last reference is being
* released, enable the waiting thread (in iw_destroy_cm_id) to
@@ -153,21 +159,14 @@ static int copy_private_data(struct iw_c
*/
static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
{
- int ret = 0;
-
BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
if (atomic_dec_and_test(&cm_id_priv->refcount)) {
BUG_ON(!list_empty(&cm_id_priv->work_list));
- if (waitqueue_active(&cm_id_priv->destroy_comp.wait)) {
- BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
- BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
- &cm_id_priv->flags));
- ret = 1;
- }
complete(&cm_id_priv->destroy_comp);
+ return 1;
}
- return ret;
+ return 0;
}
static void add_ref(struct iw_cm_id *cm_id)
@@ -181,7 +180,11 @@ static void rem_ref(struct iw_cm_id *cm_
{
struct iwcm_id_private *cm_id_priv;
cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
- iwcm_deref_id(cm_id_priv);
+ if (iwcm_deref_id(cm_id_priv) &&
+ test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags)) {
+ BUG_ON(!list_empty(&cm_id_priv->work_list));
+ free_cm_id(cm_id_priv);
+ }
}
static int cm_event_handler(struct iw_cm_id *cm_id, struct iw_cm_event *event);
@@ -355,7 +358,9 @@ static void destroy_cm_id(struct iw_cm_i
case IW_CM_STATE_CONN_RECV:
/*
* App called destroy before/without calling accept after
-* receiving connection request event notification.
+* receiving connection request event notification or
+* returned non zero from the event callback function.
+* In either case, must tell the provider to reject.
*/
cm_id_priv->state = IW_CM_STATE_DESTROYING;
break;
@@ -391,9 +396,7 @@ void iw_destroy_cm_id(struct iw_cm_id *c
wait_for_completion(&cm_id_priv->destroy_comp);
- dealloc_work_entries(cm_id_priv);
-
- kfree(cm_id_priv);
+ free_cm_id(cm_id_priv);
}
EXPORT_SYMBOL(iw_destroy_cm_id);
@@ -647,10 +650,11 @@ static void cm_conn_req_handler(struct i
/* Call the client CM handler */
ret = cm_id->cm_handler(cm_id, iw_event);
if (ret) {
+ iw_cm_reject(cm_id, NULL, 0);
set_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags);
destroy_cm_id(cm_id);
if (atomic_read(&cm_id_priv->refcount)==0)
- kfree(cm_id);
+ free_cm_id(cm_id_priv);
}
out:
@@ -854,13 +858,12 @@ static void cm_work_handler(struct work_
destroy_cm_id(&cm_id_priv->id);
}
BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
- if (iwcm_deref_id(cm_id_priv))
- return;
-
- if (atomic_read(&cm_id_priv->refcount)==0 &&
- test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags)) {
- dealloc_work_entries(cm_id_priv);
- kfree(cm_id_priv);
+ if (iwcm_deref_id(cm_id_priv)) {
+
Re: [openib-general] [PATCH] RDMA/iwcm: Bugs in cm_conn_req_handler()
ugh. There is at least one bug in this patch. I cannot call iw_cm_reject() inside destroy_cm_id() because both functions grab the iw_cm lock... On Sat, 2007-02-10 at 13:23 -0600, Steve Wise wrote: > Here is a patch that Tom and I think fixes the race condition Roland > discovered, plus cleans up the issues Krishna attempted to fix in his > first patch. I'm testing it now with a series of rping tests looping > with random sizes and counts and it seems to work, but the patch needs > more testing and review. > > Krishna, can you review this carefully and also test it and let us know > if you think its good? The patch is against for-2.6.21 from Roland's > tree. > > Roland, can you review this too and verify that it fixes the race > condition? > > > Krishna: here are comments to your original patch description: > > > cm_conn_req_handler() : > > 1. Calling destroy_cm_id leaks 3 work 'free' list entries. > > I don't think your original patch fixed all places this memory was > leaked. > > This has been address in the patch below by creating a function > free_cm_id() that frees the list entries -and- frees the cm_id memory. > It is then called from the 3 places where the cm_id can be freed. > > > 2. cm_id is freed up wrongly and not cm_id_priv (though the > >effect is the same since cm_id is the first element of > >cm_id_priv, but still a bug if the top level cm_id > > changes). > > > > This is addressed in the patch below since we have a prototyped function > for freeing the cm_id_priv. > > > 3. Reject message has to be sent on failure. Tested this > >without the fix and found the client hangs, waited for > > about > >20 mins and then did Ctrl-C but the process is unkillable. > > > > The call to iw_cm_reject() is now in destroy_cm_id() and is called based > on the cm_id state. So whenever a connection is destroyed, if it needs > a rejection sent, it will be sent. > > > 4. Setting IWCM_F_CALLBACK_DESTROY on cm_id (child handle) > >doesn't achieve anything, since checking for > >IWCM_F_CALLBACK_DESTROY in the parent's flag (in > >cm_work_handler) means that this will never be true. > > > > It does achieve something. I think you are failing to consider the fact > that the iWARP provider can have a reference on the cm_id even in the > case where the callback function returns an error thus giving > destruction ownership to the IWCM. Perhaps the ammasso provider never > does this, but cxgb3 can. And we haven't put any restrictions on > exactly when the provider _must_ release its reference. If the provider > _does_ have a reference at this point in the code, then the cm_id will > not be freed, and must be freed when the refcnt finally reaches zero > when the provider removes its reference. > > I wish to clarify this for everyone (and we need this text in > Documentation/infiniband/iwcm.txt IMO): > > This design is based on the RDMA_CM and IB_CM behavior. If the app > issues the destroy via rdma_destroy_cm_id, then we block that thread > until all references are gone. If the app returns non-zero in a > callback for a given cm_id, then the CM owns destroying the cm_id and > the application is done with it. That's the short of it. Here's the > long of it: > > There are 2 paths for freeing iw_cm memory. > > 1) the application issues a rdma_destroy_cm_id() which calls > iw_destroy_cm_id(). In this case (and this case only), the thread is > blocked until the refcnt reaches 0, then the thread continues and frees > the memory. > > 2) the application returns non zero from a callback function. In this > case, the IWCM is responsible to destroy the cm_id. However, the IWCM > _cannot_ block in its event handler thread because this can cause a > deadlock. A deadlock can occur if the provider has a reference to the > cm_id and needs to post some event before removing the reference. If > the IWCM were to block awaiting the refcnt to go to zero, it would > deadlock with the provider trying to post the last event before derefing > the cm_id. So the IWCM_F_CALLBACK_DESTROY bit is used to indicate that > the IWCM owns destroying this. If, in cm_work_handler(), the refcnt > goes to zero -and- the DESTROY bit is set, then the cm_id can be freed. > If the refcnt doesn't go to zero in that function, then either the > provider still has a reference, or subsequent queued work items have > additional references. In either case, the cm_id is not freed and > cm_work_handler() keeps chunking through the
Re: [openib-general] [PATCH] RDMA/iwcm: Bugs in cm_conn_req_handler()
r has the last reference, then the
cm_id is freed in rem_ref().
I hope this clarifies things.
Here's the proposed patch:
iw_cm_id destruction race condition fixes.
From: Steve Wise <[EMAIL PROTECTED]>
Several changes:
- iwcm_deref_id() always wakes up if there's another reference.
- move iw_cm_reject() into destroy_cm_id() to reduce code replication.
- clean up race condition in cm_work_handler().
- create static void free_cm_id() which deallocs the work entries and then
kfrees the cm_id memory. This reduces code replication.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
Signed-off-by: Tom Tucker <[EMAIL PROTECTED]>
---
drivers/infiniband/core/iwcm.c | 48 +---
1 files changed, 25 insertions(+), 23 deletions(-)
diff --git a/drivers/infiniband/core/iwcm.c b/drivers/infiniband/core/iwcm.c
index 1039ad5..403daed 100644
--- a/drivers/infiniband/core/iwcm.c
+++ b/drivers/infiniband/core/iwcm.c
@@ -146,6 +146,12 @@ static int copy_private_data(struct iw_c
return 0;
}
+static void free_cm_id(struct iwcm_id_private *cm_id_priv)
+{
+ dealloc_work_entries(cm_id_priv);
+ kfree(cm_id_priv);
+}
+
/*
* Release a reference on cm_id. If the last reference is being
* released, enable the waiting thread (in iw_destroy_cm_id) to
@@ -153,21 +159,14 @@ static int copy_private_data(struct iw_c
*/
static int iwcm_deref_id(struct iwcm_id_private *cm_id_priv)
{
- int ret = 0;
-
BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
if (atomic_dec_and_test(&cm_id_priv->refcount)) {
BUG_ON(!list_empty(&cm_id_priv->work_list));
- if (waitqueue_active(&cm_id_priv->destroy_comp.wait)) {
- BUG_ON(cm_id_priv->state != IW_CM_STATE_DESTROYING);
- BUG_ON(test_bit(IWCM_F_CALLBACK_DESTROY,
- &cm_id_priv->flags));
- ret = 1;
- }
complete(&cm_id_priv->destroy_comp);
+ return 1;
}
- return ret;
+ return 0;
}
static void add_ref(struct iw_cm_id *cm_id)
@@ -181,7 +180,11 @@ static void rem_ref(struct iw_cm_id *cm_
{
struct iwcm_id_private *cm_id_priv;
cm_id_priv = container_of(cm_id, struct iwcm_id_private, id);
- iwcm_deref_id(cm_id_priv);
+ if (iwcm_deref_id(cm_id_priv) &&
+ test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags)) {
+ BUG_ON(!list_empty(&cm_id_priv->work_list));
+ free_cm_id(cm_id_priv);
+ }
}
static int cm_event_handler(struct iw_cm_id *cm_id, struct iw_cm_event *event);
@@ -355,8 +358,11 @@ static void destroy_cm_id(struct iw_cm_i
case IW_CM_STATE_CONN_RECV:
/*
* App called destroy before/without calling accept after
-* receiving connection request event notification.
+* receiving connection request event notification or
+* returned non zero from the event callback function.
+* In either case, must tell the provider to reject.
*/
+ iw_cm_reject(cm_id, NULL, 0);
cm_id_priv->state = IW_CM_STATE_DESTROYING;
break;
case IW_CM_STATE_CONN_SENT:
@@ -391,9 +397,7 @@ void iw_destroy_cm_id(struct iw_cm_id *c
wait_for_completion(&cm_id_priv->destroy_comp);
- dealloc_work_entries(cm_id_priv);
-
- kfree(cm_id_priv);
+ free_cm_id(cm_id_priv);
}
EXPORT_SYMBOL(iw_destroy_cm_id);
@@ -639,7 +643,6 @@ static void cm_conn_req_handler(struct i
ret = alloc_work_entries(cm_id_priv, 3);
if (ret) {
- iw_cm_reject(cm_id, NULL, 0);
iw_destroy_cm_id(cm_id);
goto out;
}
@@ -650,7 +653,7 @@ static void cm_conn_req_handler(struct i
set_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags);
destroy_cm_id(cm_id);
if (atomic_read(&cm_id_priv->refcount)==0)
- kfree(cm_id);
+ free_cm_id(cm_id_priv);
}
out:
@@ -854,13 +857,12 @@ static void cm_work_handler(struct work_
destroy_cm_id(&cm_id_priv->id);
}
BUG_ON(atomic_read(&cm_id_priv->refcount)==0);
- if (iwcm_deref_id(cm_id_priv))
- return;
-
- if (atomic_read(&cm_id_priv->refcount)==0 &&
- test_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags)) {
- dealloc_work_entries(cm_id_priv);
- kfree(cm_id_priv);
+ if (iwcm_deref_id(cm_id_priv)) {
+ if (test_bit(IWCM_F_CALLBACK_DESTROY,
+&cm
Re: [openib-general] [PATCH] RDMA/iwcm: Bugs in cm_conn_req_handler()
> All 4 above cases were tested by injecting random error in > iw_conn_req_handler() and running rdma_bw/krping, they were > confirmed. I added the BUG_ON() to confirm the earlier check > for id_priv->refcount==0 should always be true (and could be > removed). Can you post the test case you're using for this? Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] for-2.6.21 Declare iwch_ev_dispatch in iwch.h
Declare iwch_ev_dispatch in iwch.h
Remove the extern declaration from iwch.c and put it in iwch.h
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch.c |2 --
drivers/infiniband/hw/cxgb3/iwch.h |2 ++
2 files changed, 2 insertions(+), 2 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch.c
b/drivers/infiniband/hw/cxgb3/iwch.c
index c353a9b..4611afa 100644
--- a/drivers/infiniband/hw/cxgb3/iwch.c
+++ b/drivers/infiniband/hw/cxgb3/iwch.c
@@ -162,8 +162,6 @@ static void close_rnic_dev(struct t3cdev
mutex_unlock(&dev_mutex);
}
-extern void iwch_ev_dispatch(struct cxio_rdev *rdev_p, struct sk_buff *skb);
-
static int __init iwch_init_module(void)
{
int err;
diff --git a/drivers/infiniband/hw/cxgb3/iwch.h
b/drivers/infiniband/hw/cxgb3/iwch.h
index 29cf2e8..6517ef8 100644
--- a/drivers/infiniband/hw/cxgb3/iwch.h
+++ b/drivers/infiniband/hw/cxgb3/iwch.h
@@ -172,4 +172,6 @@ static inline void remove_handle(struct
extern struct cxgb3_client t3c_client;
extern cxgb3_cpl_handler_func t3c_handlers[NUM_CPL_CMDS];
+extern void iwch_ev_dispatch(struct cxio_rdev *rdev_p, struct sk_buff *skb);
+
#endif
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] dapl broken for iWARP
On Fri, 2007-02-09 at 10:15 -0500, Kanevsky, Arkady wrote: > Steve, > what is an issue of using > max_qp_rd_atom and max_qp_init_rd_atom > beside the bad name? its a hack. But Bob already asked to do this, so I guess I will. We still don't ensure interoperability with DAPL consumers. A global value would. Using device max's wont. > Thanks, > > Arkady Kanevsky email: [EMAIL PROTECTED] > Network Appliance Inc. phone: 781-768-5395 > 1601 Trapelo Rd. - Suite 16.Fax: 781-895-1195 > Waltham, MA 02451 central phone: 781-768-5300 > > > > -Original Message- > > From: Steve Wise [mailto:[EMAIL PROTECTED] > > Sent: Thursday, February 08, 2007 6:11 PM > > To: Arlin Davis > > Cc: openib-general > > Subject: Re: [openib-general] dapl broken for iWARP > > > > On Wed, 2007-02-07 at 15:57 -0600, Steve Wise wrote: > > > On Wed, 2007-02-07 at 14:02 -0600, Steve Wise wrote: > > > > Arlin, > > > > > > > > The OFED dapl code is assuming the responder_resources and > > > > initiator_depth passed up on a connection request event > > are from the > > > > remote peer. This doesn't happen for iWARP. In the > > current iWARP > > > > specifications, its up to the application to exchange this > > > > information somehow. So these are defaulting to 0 on the > > server side > > > > of any dapl connection over iWARP. > > > > > > > > This is a fairly recent change, I think. We need to come up with > > > > some way to deal with this for OFED 1.2 IMO. > > > > > > > > > > The IWCM could set these to the device max values for instance. > > > > > > Steve. > > > > > > > There is a slight problem with all this. There are no device > > attributes currently for ORD and IRD. The ammasso driver > > maps these to max_qp_rd_atom (IRD) and > > max_qp_init_rd_atom(ORD). But this is screwy. > > We need new attribute for these. > > > > For OFED 1.2, I think I should just have the IWCM set them to > > 8. The only RNIC in ofed is cxgb3 and it supports 8... > > > > > > Steve. > > > > > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 0/5] iw_cxgb3 - misc cleanup and fixes
> I understand, I did not get that. > > But for example create_read_req_cqe builds it in software. > It could build ib_wc instead. > Reads are handled in a slightly different manner. This is due to the fact that the T3 HW can complete a read out of order. For example: POST READ POST WRITE The post read trigger the HW to send an RDMA_READ_REQUEST. Immediately after that the HW can (and will) send the RDMA_WRITE. Once the peer TCP ACKs the WRITE, the HW will post a CQE for the WRITE. That completion might happen before the peer sends back the READ_RESPONSE. Since the RDMAC verbs spec sez WRs must be completed in order, the T3 driver has to deal with this. (and its painful :) In addition, I have to maintain other state about a read. 1) the consumer wr_id. For non reads, the wr_id is actually reflected back by the HW from the WQE to the CQE. For reads, this doesn't happen. 2) the CQE for a read completion doesn't contain the original length. I need to pull that from the associated original WQE. So all this means the driver needs to construct a proper read cqe from several parts. That's why it creates it locally on the stack. BUT: You're right though: All WQEs get copied out of the HWCQ and into an on-stack variable in iwch_poll_cq_one(). Removing this, however, requires rethinking all the READ logic which assumes the WQE is copied out of the HWCQ. Can cannot make this change right now because of stability concerns (it took me long enough to understand how to correctly handle the read case as it stands :-) > ... > > > > Having to wade through 3 driver-specific layers of abstractions just > > > because I want to > > > for example change API in ib_verbs.h and need to update all drivers will > > > be > > > very taxing. I understand your design calls for 2 layers, but at least > > > the API exposed > > > by code in drivers/net is fairly small, while cxio_wr.h declares 27 > > > structures > > > which seem to just duplicate ib_verbs.h. > > > > cxio_wr.h is hw format. You want me to change ib_verbs.h to make WRs > > and CQEs align with Chelsio hardware? > > No, but it need not be part of interface. The reason I was confused is > because > you seem to create an extra copy e.g. for t3_cqe. cxio_poll_cq currently > creates an intermediate copy of the completion on the stack, I think it could > format ib_wc directly instead. > I'll log this as a performance optimization that we can do later. Thanks for helping review this stuff!! Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 0/5] iw_cxgb3 - misc cleanup and fixes
On Fri, 2007-02-09 at 08:23 -0600, Steve Wise wrote: > On Fri, 2007-02-09 at 08:51 +0200, Michael S. Tsirkin wrote: > > > > Also I agree with MST, I would like to see the core/ subdirectory die > > > > completely. > > > > > > > > > > ok ok...I'll kill the subdir... > > > > It's not just the directory BTW. Stuff like building completions in > > t3_cqe format and then reformatting to ib_wc seems to be much more confusing > > (and some of it is actually on datapath). > > The t3_cqe format is built BY THE HW. > > > > Same goes for t3_wq and I suspect everything else defined in cxio_wr.h - > > please, use the native types from include/rdma/. > > > > Ditto. t3_wq is the HW format. > To be more precise: struct t3_wq is the struct used to describe the T3 HW WQ, which is both the SQ and RQ for the QP. struct t3_cq is the struct used to describe the T3 HW CQ -and- a SW CQ used to maintain proper completion ordering that isn't maintained by the HW for some operations. union t3_wr defines the union of all the HW-specific WR structs. struct t3_cqe defines the HW CQE format. All of the is very tightly integrated with the HW. These HW-specific structs are included in a high-level struct that defines the object and has all the stuff needed to integrate into the OFA stack. Example: struct iwch_qp defines the top-level QP structure that maintains both the T3 HW struct (struct t3_wq) and the OFA struct (struct ib_qp) as well as attributes, wait objects, locks, etc to correctly implement the OFA QP object. This is similar to what other providers do: hw/mthca/mthca_provider.h: mthca_qp includes 2 mthca_wq structs for the SQ and RQ. hw/amso/c2_provider.h: c2_qp includes 2 c2_mq structs for the SQ and RQ message queues. hw/ipath/ipath_verbs.h: ipath_qp include ipath_swqe and ipath_rq for their work queues. WRT data path operations, consider iwch_poll_cq_one(). The CQE is in T3 FW format and must be converted into a OFA struct ib_wc. There's no way around this, right? mthca_poll_one() does the same thing. Ditto for c2_poll_one(). Hope this helps. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH 0/5] iw_cxgb3 - misc cleanup and fixes
On Fri, 2007-02-09 at 08:51 +0200, Michael S. Tsirkin wrote: > > > Also I agree with MST, I would like to see the core/ subdirectory die > > > completely. > > > > > > > ok ok...I'll kill the subdir... > > It's not just the directory BTW. Stuff like building completions in > t3_cqe format and then reformatting to ib_wc seems to be much more confusing > (and some of it is actually on datapath). The t3_cqe format is built BY THE HW. > Same goes for t3_wq and I suspect everything else defined in cxio_wr.h - > please, use the native types from include/rdma/. > Ditto. t3_wq is the HW format. > Having to wade through 3 driver-specific layers of abstractions just because > I want to > for example change API in ib_verbs.h and need to update all drivers will be > very taxing. I understand your design calls for 2 layers, but at least the > API exposed > by code in drivers/net is fairly small, while cxio_wr.h declares 27 structures > which seem to just duplicate ib_verbs.h. cxio_wr.h is hw format. You want me to change ib_verbs.h to make WRs and CQEs align with Chelsio hardware? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] dapl broken for iWARP
On Thu, 2007-02-08 at 19:19 -0600, Bob Sharp wrote: > > For OFED 1.2, I think I should just have the IWCM set them to 8. The > > only RNIC in ofed is cxgb3 and it supports 8... > > > Steve, > > If we can create the new attributes for RNICs, it seems like would be > better to agree on the mapping of IRD/ORD to IB parameters than it would > be to limit these parameters to 8. That number seems a bit low. > Hey Bob, This is for the OFED 1.2 release only and its too late to be adding new features methinks since we're at feature freeze. For the upstream kernel (ie 2.6.21) we can define the attributes. > Bob ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaroundfor ip_dev_find() bug.
Michael,
>From your email, it sounded like you would regression test this. Is it
ready to pull in?
Thanks!
Steve.
On Tue, 2007-02-06 at 17:39 -0600, Steve Wise wrote:
> Here it is (only tested with rping over iWARP on sles9sp3):
>
>
>
>
> xxx_ip_dev_find() must use scope HOST.
>
> From: Steve Wise <[EMAIL PROTECTED]>
>
> Function xxx_ip_dev_find(RT_SCOPE_LINK) returns the wrong interface on
> some kernels. The correct scope is RT_SCOPE_HOST.
>
> Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
> ---
>
> .../backport/2.6.11/include/linux/inetdevice.h |2 +-
> .../backport/2.6.11_FC4/include/linux/inetdevice.h |2 +-
> .../backport/2.6.12/include/linux/inetdevice.h |2 +-
> .../backport/2.6.13/include/linux/inetdevice.h |2 +-
> .../2.6.13_suse10_0_u/include/linux/inetdevice.h |2 +-
> .../backport/2.6.14/include/linux/inetdevice.h |2 +-
> .../backport/2.6.15/include/linux/inetdevice.h |2 +-
> .../2.6.15_ubuntu606/include/linux/inetdevice.h|2 +-
> .../backport/2.6.16/include/linux/inetdevice.h |2 +-
> .../backport/2.6.17/include/linux/inetdevice.h |2 +-
> .../2.6.5_sles9_sp3/include/linux/inetdevice.h |2 +-
> .../backport/2.6.9_U2/include/linux/inetdevice.h |2 +-
> .../backport/2.6.9_U3/include/linux/inetdevice.h |2 +-
> .../backport/2.6.9_U4/include/linux/inetdevice.h |2 +-
> 14 files changed, 14 insertions(+), 14 deletions(-)
>
> diff --git a/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
> b/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
> index 7244487..2d3c50f 100644
> --- a/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
> +++ b/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
> @@ -13,7 +13,7 @@ static inline struct net_device *xxx_ip_
>
> read_lock(&dev_base_lock);
> for (dev = dev_base; dev; dev = dev->next) {
> - ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
> + ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
> if (ip == addr) {
> dev_hold(dev);
> break;
> diff --git a/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
> b/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
> index 7244487..2d3c50f 100644
> --- a/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
> +++ b/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
> @@ -13,7 +13,7 @@ static inline struct net_device *xxx_ip_
>
> read_lock(&dev_base_lock);
> for (dev = dev_base; dev; dev = dev->next) {
> - ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
> + ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
> if (ip == addr) {
> dev_hold(dev);
> break;
> diff --git a/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
> b/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
> index 7244487..2d3c50f 100644
> --- a/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
> +++ b/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
> @@ -13,7 +13,7 @@ static inline struct net_device *xxx_ip_
>
> read_lock(&dev_base_lock);
> for (dev = dev_base; dev; dev = dev->next) {
> - ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
> + ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
> if (ip == addr) {
> dev_hold(dev);
> break;
> diff --git a/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
> b/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
> index 7a32313..fd0aa36 100644
> --- a/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
> +++ b/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
> @@ -11,7 +11,7 @@ static inline struct net_device *xxx_ip_
>
> read_lock(&dev_base_lock);
> for (dev = dev_base; dev; dev = dev->next) {
> - ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
> + ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
> if (ip == addr) {
> dev_hold(dev);
> break;
> diff --git
> a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
> b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
> index 7a32313..fd0aa36 100644
> --- a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
> +++ b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
> @@ -11,7 +11,7 @@ static inline struct net_device *xxx_ip_
>
> read_lock(&dev_base_lock);
>
Re: [openib-general] [PATCH 0/5] iw_cxgb3 - misc cleanup and fixes
On Thu, 2007-02-08 at 16:26 -0800, Roland Dreier wrote: > OK, I've pulled the cxgb3 stuff into a single commit in my for-2.6.21 > branch. I took the liberty of cleaning up some sparse warnings, etc. > There's still a few other obvious things to fix up: > > drivers/infiniband/hw/cxgb3/iwch_ev.c:102:6: warning: symbol 'iwch_ev_disp > atch' was not declared. Should it be static? > > Rather than putting an extern in iwch.c, please put a proper > definition in an appropriate header file included from iwch.c. > ok. > Also I agree with MST, I would like to see the core/ subdirectory die > completely. > ok ok...I'll kill the subdir... ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] dapl broken for iWARP
On Wed, 2007-02-07 at 15:57 -0600, Steve Wise wrote: > On Wed, 2007-02-07 at 14:02 -0600, Steve Wise wrote: > > Arlin, > > > > The OFED dapl code is assuming the responder_resources and > > initiator_depth passed up on a connection request event are from the > > remote peer. This doesn't happen for iWARP. In the current iWARP > > specifications, its up to the application to exchange this information > > somehow. So these are defaulting to 0 on the server side of any dapl > > connection over iWARP. > > > > This is a fairly recent change, I think. We need to come up with some > > way to deal with this for OFED 1.2 IMO. > > > > The IWCM could set these to the device max values for instance. > > Steve. > There is a slight problem with all this. There are no device attributes currently for ORD and IRD. The ammasso driver maps these to max_qp_rd_atom (IRD) and max_qp_init_rd_atom(ORD). But this is screwy. We need new attribute for these. For OFED 1.2, I think I should just have the IWCM set them to 8. The only RNIC in ofed is cxgb3 and it supports 8... Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 4/5] Get rid of static rdev table.
From: Steve Wise <[EMAIL PROTECTED]>
Use a liked list.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/core/cxio_hal.c | 57 +--
drivers/infiniband/hw/cxgb3/core/cxio_hal.h |2 -
2 files changed, 19 insertions(+), 40 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_hal.c
b/drivers/infiniband/hw/cxgb3/core/cxio_hal.c
index 2c4e351..acffe16 100644
--- a/drivers/infiniband/hw/cxgb3/core/cxio_hal.c
+++ b/drivers/infiniband/hw/cxgb3/core/cxio_hal.c
@@ -43,49 +43,28 @@ #include "cxio_hal.h"
#include "cxgb3_offload.h"
#include "sge_defs.h"
-static struct cxio_rdev *rdev_tbl[T3_MAX_NUM_RNIC];
+static LIST_HEAD(rdev_list);
static cxio_hal_ev_callback_func_t cxio_ev_cb = NULL;
static inline struct cxio_rdev *cxio_hal_find_rdev_by_name(char *dev_name)
{
- int i;
- for (i = 0; i < T3_MAX_NUM_RNIC; i++)
- if (rdev_tbl[i])
- if (!strcmp(rdev_tbl[i]->dev_name, dev_name))
- return rdev_tbl[i];
+ struct cxio_rdev *rdev;
+
+ list_for_each_entry(rdev, &rdev_list, entry)
+ if (!strcmp(rdev->dev_name, dev_name))
+ return rdev;
return NULL;
}
static inline struct cxio_rdev *cxio_hal_find_rdev_by_t3cdev(struct t3cdev
*tdev)
{
- int i;
- for (i = 0; i < T3_MAX_NUM_RNIC; i++)
- if (rdev_tbl[i])
- if (rdev_tbl[i]->t3cdev_p == tdev)
- return rdev_tbl[i];
- return NULL;
-}
-
-static inline int cxio_hal_add_rdev(struct cxio_rdev *rdev_p)
-{
- int i;
- for (i = 0; i < T3_MAX_NUM_RNIC; i++)
- if (!rdev_tbl[i]) {
- rdev_tbl[i] = rdev_p;
- break;
- }
- return (i == T3_MAX_NUM_RNIC);
-}
+ struct cxio_rdev *rdev;
-static inline void cxio_hal_delete_rdev(struct cxio_rdev *rdev_p)
-{
- int i;
- for (i = 0; i < T3_MAX_NUM_RNIC; i++)
- if (rdev_tbl[i] == rdev_p) {
- rdev_tbl[i] = NULL;
- break;
- }
+ list_for_each_entry(rdev, &rdev_list, entry)
+ if (rdev->t3cdev_p == tdev)
+ return rdev;
+ return NULL;
}
int cxio_hal_cq_op(struct cxio_rdev *rdev_p, struct t3_cq *cq,
@@ -937,8 +916,7 @@ int cxio_rdev_open(struct cxio_rdev *rde
return -EINVAL;
}
- if (cxio_hal_add_rdev(rdev_p))
- return -ENOMEM;
+ list_add_tail(&rdev_p->entry, &rdev_list);
PDBG("%s opening rnic dev %s\n", __FUNCTION__, rdev_p->dev_name);
memset(&rdev_p->ctrl_qp, 0, sizeof(rdev_p->ctrl_qp));
@@ -1018,7 +996,7 @@ err3:
err2:
cxio_hal_destroy_ctrl_qp(rdev_p);
err1:
- cxio_hal_delete_rdev(rdev_p);
+ list_del(&rdev_p->entry);
return err;
}
@@ -1027,7 +1005,7 @@ void cxio_rdev_close(struct cxio_rdev *r
if (rdev_p) {
cxio_hal_pblpool_destroy(rdev_p);
cxio_hal_rqtpool_destroy(rdev_p);
- cxio_hal_delete_rdev(rdev_p);
+ list_del(&rdev_p->entry);
rdev_p->t3cdev_p->ulp = NULL;
cxio_hal_destroy_ctrl_qp(rdev_p);
cxio_hal_destroy_resource(rdev_p->rscp);
@@ -1038,7 +1016,6 @@ int __init cxio_hal_init(void)
{
if (cxio_hal_init_rhdl_resource(T3_MAX_NUM_RI))
return -ENOMEM;
- memset(rdev_tbl, 0, T3_MAX_NUM_RNIC * sizeof(void *));
t3_register_cpl_handler(CPL_ASYNC_NOTIF, cxio_hal_ev_handler);
return 0;
}
@@ -1046,9 +1023,11 @@ int __init cxio_hal_init(void)
void __exit cxio_hal_exit(void)
{
int i;
+ struct cxio_rdev *rdev, *tmp;
+
t3_register_cpl_handler(CPL_ASYNC_NOTIF, NULL);
- for (i = 0; i < T3_MAX_NUM_RNIC; i++)
- cxio_rdev_close(rdev_tbl[i]);
+ list_for_each_entry_safe(rdev, tmp, &rdev_list, entry)
+ cxio_rdev_close(rdev);
cxio_hal_destroy_rhdl_resource();
}
diff --git a/drivers/infiniband/hw/cxgb3/core/cxio_hal.h
b/drivers/infiniband/hw/cxgb3/core/cxio_hal.h
index d5ae282..8fb2999 100644
--- a/drivers/infiniband/hw/cxgb3/core/cxio_hal.h
+++ b/drivers/infiniband/hw/cxgb3/core/cxio_hal.h
@@ -47,7 +47,6 @@ #define T3_CTRL_QP_SIZE_LOG2 8
#define T3_CTRL_CQ_ID0
/* TBD */
-#define T3_MAX_NUM_RNIC 8
#define T3_MAX_NUM_RI (1<<15)
#define T3_MAX_NUM_QP (1<<15)
#define T3_MAX_NUM_CQ (1<<15)
@@ -106,6 +105,7 @@ struct cxio_rdev {
struct cxio_ucontext uctx;
struct gen_pool *pbl_pool;
struct gen_pool *rqt_pool;
+ struct list_head entry;
};
static inline int cxio_num_stag
[openib-general] [PATCH 5/5] Hold the iwch device mutex around cxio_rdev_open().
From: Steve Wise <[EMAIL PROTECTED]>
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch.c |4 +++-
1 files changed, 3 insertions(+), 1 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch.c
b/drivers/infiniband/hw/cxgb3/iwch.c
index 0c95f2c..c353a9b 100644
--- a/drivers/infiniband/hw/cxgb3/iwch.c
+++ b/drivers/infiniband/hw/cxgb3/iwch.c
@@ -119,7 +119,10 @@ static void open_rnic_dev(struct t3cdev
rnicp->rdev.ulp = rnicp;
rnicp->rdev.t3cdev_p = tdev;
+ mutex_lock(&dev_mutex);
+
if (cxio_rdev_open(&rnicp->rdev)) {
+ mutex_unlock(&dev_mutex);
printk(KERN_ERR MOD "Unable to open CXIO rdev\n");
ib_dealloc_device(&rnicp->ibdev);
return;
@@ -127,7 +130,6 @@ static void open_rnic_dev(struct t3cdev
rnic_init(rnicp);
- mutex_lock(&dev_mutex);
list_add_tail(&rnicp->entry, &dev_list);
mutex_unlock(&dev_mutex);
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 3/5] Clean up pending mmaps on ucontext deallocation.
From: Steve Wise <[EMAIL PROTECTED]>
Free all pending mmap structs when the ucontext is deallocated.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch_provider.c |4
1 files changed, 4 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index db2b0a8..85484ac 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -98,7 +98,11 @@ static int iwch_dealloc_ucontext(struct
{
struct iwch_dev *rhp = to_iwch_dev(context->device);
struct iwch_ucontext *ucontext = to_iwch_ucontext(context);
+ struct iwch_mm_entry *mm, *tmp;
+
PDBG("%s context %p\n", __FUNCTION__, context);
+ list_for_each_entry_safe(mm, tmp, &ucontext->mmaps, entry)
+ kfree(mm);
cxio_release_ucontext(&rhp->rdev, &ucontext->uctx);
kfree(ucontext);
return 0;
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 0/5] iw_cxgb3 - misc cleanup and fixes
Here are some fixes to address various comments from Michael and Roland. This is _not_ for ofed_1_2, but rather for merging into 2.6.21. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH 2/5] No need to disable interrupts for mmap locking.
From: Steve Wise <[EMAIL PROTECTED]>
Lock mmap_lock is never taken from non-process context, so just use
bare spin_lock()/spin_unlock().
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch_provider.h | 10 +-
1 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.h
b/drivers/infiniband/hw/cxgb3/iwch_provider.h
index a8cfeaf..1ede8a7 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.h
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.h
@@ -205,29 +205,29 @@ static inline struct iwch_mm_entry *remo
struct list_head *pos, *nxt;
struct iwch_mm_entry *mm;
- spin_lock_irq(&ucontext->mmap_lock);
+ spin_lock(&ucontext->mmap_lock);
list_for_each_safe(pos, nxt, &ucontext->mmaps) {
mm = list_entry(pos, struct iwch_mm_entry, entry);
if (mm->addr == addr && mm->len == len) {
list_del_init(&mm->entry);
- spin_unlock_irq(&ucontext->mmap_lock);
+ spin_unlock(&ucontext->mmap_lock);
PDBG("%s addr 0x%llx len %d\n", __FUNCTION__, mm->addr,
mm->len);
return mm;
}
}
- spin_unlock_irq(&ucontext->mmap_lock);
+ spin_unlock(&ucontext->mmap_lock);
return NULL;
}
static inline void insert_mmap(struct iwch_ucontext *ucontext,
struct iwch_mm_entry *mm)
{
- spin_lock_irq(&ucontext->mmap_lock);
+ spin_lock(&ucontext->mmap_lock);
PDBG("%s addr 0x%llx len %d\n", __FUNCTION__, mm->addr, mm->len);
list_add_tail(&mm->entry, &ucontext->mmaps);
- spin_unlock_irq(&ucontext->mmap_lock);
+ spin_unlock(&ucontext->mmap_lock);
}
enum iwch_qp_attr_mask {
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] more comments on cxgb3
> - Consider a user that does e.g. create QP, but never calls mmap.
> Is there some code that will clean out the unclamed mmap object?
> I couldn't find it, and iwch_dealloc_ucontext does not seem to
> do anything with it.
BTW: Here is my fix for this.
-
Clean up pending mmaps on ucontext deallocation.
From: Steve Wise <[EMAIL PROTECTED]>
Free all pending mmap structs when the ucontext is deallocated.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/cxgb3/iwch_provider.c |1 +
drivers/infiniband/hw/cxgb3/iwch_provider.h | 15 +++
2 files changed, 16 insertions(+), 0 deletions(-)
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.c
b/drivers/infiniband/hw/cxgb3/iwch_provider.c
index db2b0a8..98568ee 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.c
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.c
@@ -99,6 +99,7 @@ static int iwch_dealloc_ucontext(struct
struct iwch_dev *rhp = to_iwch_dev(context->device);
struct iwch_ucontext *ucontext = to_iwch_ucontext(context);
PDBG("%s context %p\n", __FUNCTION__, context);
+ free_mmaps(ucontext);
cxio_release_ucontext(&rhp->rdev, &ucontext->uctx);
kfree(ucontext);
return 0;
diff --git a/drivers/infiniband/hw/cxgb3/iwch_provider.h
b/drivers/infiniband/hw/cxgb3/iwch_provider.h
index 1ede8a7..c8c07ee 100644
--- a/drivers/infiniband/hw/cxgb3/iwch_provider.h
+++ b/drivers/infiniband/hw/cxgb3/iwch_provider.h
@@ -199,6 +199,21 @@ struct iwch_mm_entry {
unsigned len;
};
+static inline void free_mmaps(struct iwch_ucontext *ucontext)
+{
+ struct list_head *pos, *nxt;
+ struct iwch_mm_entry *mm;
+
+ spin_lock(&ucontext->mmap_lock);
+ list_for_each_safe(pos, nxt, &ucontext->mmaps) {
+ mm = list_entry(pos, struct iwch_mm_entry, entry);
+ list_del(&mm->entry);
+ kfree(mm);
+ }
+ spin_unlock(&ucontext->mmap_lock);
+ return;
+}
+
static inline struct iwch_mm_entry *remove_mmap(struct iwch_ucontext *ucontext,
u64 addr, unsigned len)
{
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] sharing qp between user and kernel
On Thu, 2007-02-08 at 10:24 -0500, Pete Wyckoff wrote: > [EMAIL PROTECTED] wrote on Wed, 07 Feb 2007 15:50 -0800: > > Pete> Before I dig into this anymore, do you expect this to work? > > Pete> Are there fundamental problems with QP sharing between user > > Pete> and kernel? It would sure be nice not to have to stick the > > Pete> connection management aspects into the kernel. > > > > No, I wouldn't expect this to work. At first glance at least, yes, > > there are fundamental problems. Sharing a QP between user and > > kernelspace, where userspace is doing full kernel bypass (as eg mthca > > does -- there are NO system calls when doing post work request, poll > > CQ and request CQ notification operations), seems like a huge > > problem. I don't see any way that the kernel can keep a consistent > > view of the QP state unless userspace has to call into the kernel for > > every operation, which would kill performance. > > My hope was not to allow full QP sharing between user and kernel, > but just a limited interface to "send this kernel data now". It > requires that the kernel register some physical memory, and submit > the work requests. Perhaps the kernel can invoke the equivalent of > the userspace post function instead of trying to use the kernel API > for sending. > You could map the kernel data into the user process and then have the user process post the WR. But the user process would have to have that memory registered as part of a MR to post it. It could be done though. So basically instead of sharing QP, give your kernel module access memory from a registered MR. The kernel module can produce the data in that memory then tell the user process to post the WR... Steve ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] more comments on cxgb3
On Thu, 2007-02-08 at 08:40 +0200, Michael S. Tsirkin wrote:
> OK, so I looked at cxgb3 some more.
Thanks!
> To summarise my previous comments, I think the cxio hal layer needs to go to
> make the code readable - if I understand correctly it is there for historical
> reasons only.
>
I can do this but its low on the list of todos.
> I started looking at userspace/kernel interaction, and then
> went over to other code under cxgb3 (but not core/).
>
> - Consider a user that does e.g. create QP, but never calls mmap.
> Is there some code that will clean out the unclamed mmap object?
> I couldn't find it, and iwch_dealloc_ucontext does not seem to
> do anything with it.
>
This is a bug. I've got a fix for it too.
> - Passing physical address to userspace and back looks suspicios.
> Especially this:
> uresp.physaddr = virt_to_phys(chp->cq.queue);
> Could you elaborate on the design here? What are these phy addresses
> and how come userspace needs to know the phy address?
> You are not doing DMA by this address, by any chance?
>
No, Its not used for DMA by the HW. The physaddr is passed up to the
user, and the user then mmaps() using that as the offset.
I took this code from the ipath driver. It has been pointed out to me
that this is broken for 32b userspace on a 64 kernel because mmap2()
cannot pass down 64 bits. So I need to rework this code.
> - It seems that by passing in huge resource sizes, userspace will be able to
> drink up unlimited amounts of kernel memory.
> mthca handles this by using the mlock rlimit, should something be done here
> as well?
Hmm. That's a good point. I'll put this on the todo as well. So mthca
adds to process's rlimit value as things are allocated out of kernel
memory (or maybe even anything that gets pinned)?
>
> A couple of comments on PDBG macro.
> - I'd like to suggest following the practice of prefixing macro names with
> module name
> (same goes for functions like get_mhp really) - unless they are local to
> file.
>
> - You are using __FUNCTION__ a lot - it might be to just to kill it,
> messages are unique so you'll be able to locate the msg source anyway,
> save some kernel text and logs will be shorter. In any case I think
> __func__ is the recommended gcc way to get the name currently.
>
> - comment near pr_debug definition in include/linux/kernel.h says:
> /* If you are writing a driver, please use dev_dbg instead */
> so it might be a good idea for PDBG to follow this rule.
>
> - log messages do not look very informative to me.
> I also think they are a bit too many of them currently.
> For example, I do not think it is a good idea to print
> the kernel pointers out.
>
> For example, in code like the following:
> mhp = get_mhp(rhp, (sg_list[i].lkey) >> 8);
> if (!mhp) {
> PDBG("%s %d\n", __FUNCTION__, __LINE__);
> return -EIO;
> }
>
> might be better to say
> "MR key XXX does not exist. Exiting.".
> -EIO also looks like a strange error code to return here, does it not?
> Maybe something like EINVAL would be more appropriate?
>
I'll take a todo to clean up the debug stuff.
> - I wonder about the names like get_mhp - what does "mhp" mean?
> static inline struct iwch_mr *get_mhp(struct iwch_dev *rhp, u32 mmid)
> {
> return idr_find(&rhp->mmidr, mmid);
> }
>
Memory Handle Pointer.
> Looks like it looks up an mr. Is that right? Maybe the name shouldbe changed
> to convey this meaning.
>
> - In the following code, what does "missing pdid check" mean?
> /*
> * TBD: this is going to be moved to firmware. Missing pdid/qpid check for
> now.
> */
> This sounds interesting.
> Does this mean the code does not validate the PD currently?
>
I need firmware support for this. It will be done asap and I can remove
this code entirely.
> I have the same question for:
> /* TBD: check perms */
> in iwch_bind_mw.
>
> BTW, does TBD stand for "To Be Done" here?
Yes.
> Do you mean TODO really?
>
> - iwch_sgl2pbl_map is used in several places, and seems a bit too big to be
> inline
>
> Well, it's time to go do my day job now :)
>
> Hope this helps,
>
Thanks again Michael!
Steve.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] dapl broken for iWARP
On Wed, 2007-02-07 at 15:05 -0800, Arlin Davis wrote: > Steve Wise wrote: > > >On Wed, 2007-02-07 at 14:02 -0600, Steve Wise wrote: > > > > > >>Arlin, > >> > >>The OFED dapl code is assuming the responder_resources and > >>initiator_depth passed up on a connection request event are from the > >>remote peer. This doesn't happen for iWARP. In the current iWARP > >>specifications, its up to the application to exchange this information > >>somehow. So these are defaulting to 0 on the server side of any dapl > >>connection over iWARP. > >> > >>This is a fairly recent change, I think. We need to come up with some > >>way to deal with this for OFED 1.2 IMO. > >> > >> > Yes, this was changed recently to sync up with the rdma_cm changes that > exposed the values. > > >> > >> > > > >The IWCM could set these to the device max values for instance. > > > > > That would work fine as long as you know the remote settings will be > equal or better. The provider just sets the min of local device max > values and the remote values provided with the request. > I know Krishna Kumar is working on a solution for exchanging this info in private data so the IWCM can "do the right thing". Stay tuned for a patch series to review for this. But this functionality is definitely post OFED-1.2. So for the OFED-1.2, I will set these to the device max in the IWCM. Assuming the other side is OFED 1.2 DAPL, then it will work fine. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] sharing qp between user and kernel
On Wed, 2007-02-07 at 17:31 -0500, Pete Wyckoff wrote: > is an IB verbs consumer. The > plan was to connect up the QP in userspace and do some preliminary > communication, then hand the QP to the kernel and let it use the QP > directly to do some more communication. This works fine on ammasso, > but fails on mthca. I think the only reason it works on ammasso is because ammasso doesn't do any kernel bypass. For devices that _do_ kernel bypass, I'm not sure it will work. It will _not_ work for the Chelsio iWARP device as its implemented today. Once the decision is made to do kernel bypass, the kernel looses track of the state of the resources shared by HW and library. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] dapl broken for iWARP
On Wed, 2007-02-07 at 14:02 -0600, Steve Wise wrote: > Arlin, > > The OFED dapl code is assuming the responder_resources and > initiator_depth passed up on a connection request event are from the > remote peer. This doesn't happen for iWARP. In the current iWARP > specifications, its up to the application to exchange this information > somehow. So these are defaulting to 0 on the server side of any dapl > connection over iWARP. > > This is a fairly recent change, I think. We need to come up with some > way to deal with this for OFED 1.2 IMO. > The IWCM could set these to the device max values for instance. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] dapl broken for iWARP
Arlin, The OFED dapl code is assuming the responder_resources and initiator_depth passed up on a connection request event are from the remote peer. This doesn't happen for iWARP. In the current iWARP specifications, its up to the application to exchange this information somehow. So these are defaulting to 0 on the server side of any dapl connection over iWARP. This is a fairly recent change, I think. We need to come up with some way to deal with this for OFED 1.2 IMO. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] dapltest?
Hey Arlin, Shouldn't dapl/test be shipped with OFED? It appears not to be... Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] RDMA/iwcm: Bugs in cm_conn_req_handler()
This looks good for 2.6.21 IMO.
Acked-by: Steve Wise <[EMAIL PROTECTED]>
On Wed, 2007-02-07 at 12:26 +0530, Krishna Kumar wrote:
> (I had submitted this once earlier but got no response)
>
> cm_conn_req_handler() :
> 1. Calling destroy_cm_id leaks 3 work 'free' list entries.
> 2. cm_id is freed up wrongly and not cm_id_priv (though the
> effect is the same since cm_id is the first element of
> cm_id_priv, but still a bug if the top level cm_id changes).
> 3. Reject message has to be sent on failure. Tested this
> without the fix and found the client hangs, waited for about
> 20 mins and then did Ctrl-C but the process is unkillable.
> 4. Setting IWCM_F_CALLBACK_DESTROY on cm_id (child handle)
> doesn't achieve anything, since checking for
> IWCM_F_CALLBACK_DESTROY in the parent's flag (in
> cm_work_handler) means that this will never be true.
>
> All 4 above cases were tested by injecting random error in
> iw_conn_req_handler() and running rdma_bw/krping, they were
> confirmed. I added the BUG_ON() to confirm the earlier check
> for id_priv->refcount==0 should always be true (and could be
> removed).
>
> Patch against 2.6.20
>
> Signed-off-by: Krishna Kumar <[EMAIL PROTECTED]>
> ---
> diff -ruNp org/drivers/infiniband/core/iwcm.c
> new/drivers/infiniband/core/iwcm.c
> --- org/drivers/infiniband/core/iwcm.c2007-01-24 10:25:26.0
> +0530
> +++ new/drivers/infiniband/core/iwcm.c2007-01-24 10:25:31.0
> +0530
> @@ -647,10 +647,9 @@ static void cm_conn_req_handler(struct i
> /* Call the client CM handler */
> ret = cm_id->cm_handler(cm_id, iw_event);
> if (ret) {
> - set_bit(IWCM_F_CALLBACK_DESTROY, &cm_id_priv->flags);
> - destroy_cm_id(cm_id);
> - if (atomic_read(&cm_id_priv->refcount)==0)
> - kfree(cm_id);
> + BUG_ON(atomic_read(&cm_id_priv->refcount) != 1);
> + iw_cm_reject(cm_id, NULL, 0);
> + iw_destroy_cm_id(cm_id);
> }
>
> out:
>
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaroundfor ip_dev_find() bug.
Here it is (only tested with rping over iWARP on sles9sp3):
xxx_ip_dev_find() must use scope HOST.
From: Steve Wise <[EMAIL PROTECTED]>
Function xxx_ip_dev_find(RT_SCOPE_LINK) returns the wrong interface on
some kernels. The correct scope is RT_SCOPE_HOST.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
.../backport/2.6.11/include/linux/inetdevice.h |2 +-
.../backport/2.6.11_FC4/include/linux/inetdevice.h |2 +-
.../backport/2.6.12/include/linux/inetdevice.h |2 +-
.../backport/2.6.13/include/linux/inetdevice.h |2 +-
.../2.6.13_suse10_0_u/include/linux/inetdevice.h |2 +-
.../backport/2.6.14/include/linux/inetdevice.h |2 +-
.../backport/2.6.15/include/linux/inetdevice.h |2 +-
.../2.6.15_ubuntu606/include/linux/inetdevice.h|2 +-
.../backport/2.6.16/include/linux/inetdevice.h |2 +-
.../backport/2.6.17/include/linux/inetdevice.h |2 +-
.../2.6.5_sles9_sp3/include/linux/inetdevice.h |2 +-
.../backport/2.6.9_U2/include/linux/inetdevice.h |2 +-
.../backport/2.6.9_U3/include/linux/inetdevice.h |2 +-
.../backport/2.6.9_U4/include/linux/inetdevice.h |2 +-
14 files changed, 14 insertions(+), 14 deletions(-)
diff --git a/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
b/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
index 7244487..2d3c50f 100644
--- a/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
+++ b/kernel_addons/backport/2.6.11/include/linux/inetdevice.h
@@ -13,7 +13,7 @@ static inline struct net_device *xxx_ip_
read_lock(&dev_base_lock);
for (dev = dev_base; dev; dev = dev->next) {
- ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
+ ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
if (ip == addr) {
dev_hold(dev);
break;
diff --git a/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
b/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
index 7244487..2d3c50f 100644
--- a/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
+++ b/kernel_addons/backport/2.6.11_FC4/include/linux/inetdevice.h
@@ -13,7 +13,7 @@ static inline struct net_device *xxx_ip_
read_lock(&dev_base_lock);
for (dev = dev_base; dev; dev = dev->next) {
- ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
+ ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
if (ip == addr) {
dev_hold(dev);
break;
diff --git a/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
b/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
index 7244487..2d3c50f 100644
--- a/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
+++ b/kernel_addons/backport/2.6.12/include/linux/inetdevice.h
@@ -13,7 +13,7 @@ static inline struct net_device *xxx_ip_
read_lock(&dev_base_lock);
for (dev = dev_base; dev; dev = dev->next) {
- ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
+ ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
if (ip == addr) {
dev_hold(dev);
break;
diff --git a/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
b/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
index 7a32313..fd0aa36 100644
--- a/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
+++ b/kernel_addons/backport/2.6.13/include/linux/inetdevice.h
@@ -11,7 +11,7 @@ static inline struct net_device *xxx_ip_
read_lock(&dev_base_lock);
for (dev = dev_base; dev; dev = dev->next) {
- ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
+ ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
if (ip == addr) {
dev_hold(dev);
break;
diff --git
a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
index 7a32313..fd0aa36 100644
--- a/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
+++ b/kernel_addons/backport/2.6.13_suse10_0_u/include/linux/inetdevice.h
@@ -11,7 +11,7 @@ static inline struct net_device *xxx_ip_
read_lock(&dev_base_lock);
for (dev = dev_base; dev; dev = dev->next) {
- ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
+ ip = inet_select_addr(dev, 0, RT_SCOPE_HOST);
if (ip == addr) {
dev_hold(dev);
break;
diff --git a/kernel_addons/backport/2.6.14/include/linux/inetdevice.h
b/kernel_addons/backport/2.6.14/include/linux/inetdevice.h
index 7a32313..fd0aa36 100644
--- a/kernel_addons/backport/2.6.14/include/linux/inetdevice.h
+++ b/kernel_addons/backport/2.6.14/include/linux/inetdevice.h
@@ -11,7 +11,7 @@ static inline
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaroundfor ip_dev_find() bug.
On Wed, 2007-02-07 at 00:12 +0200, Michael S. Tsirkin wrote: > > Quoting Steve Wise <[EMAIL PROTECTED]>: > > Subject: Re: [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaroundfor > > ip_dev_find() bug. > > > > On Tue, 2007-02-06 at 23:36 +0200, Michael S. Tsirkin wrote: > > > > How shall I fix this? > > > > > > Patch? > > > > > > > Riiight. I'm afraid if I use HOST instead of LINK that I'll break some > > strange SDP loopback feature or some such thing. And I'm not in a > > position to test that. > > > > But I can post a patch. Shall I just change sles9sp3 since we don't see > > (yet) any problems with the other distros? > > If you post one that updates all kernels it will be easier to test. > I'm ok with this. Stay tuned. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaround for ip_dev_find() bug.
On Tue, 2007-02-06 at 23:36 +0200, Michael S. Tsirkin wrote: > > How shall I fix this? > > Patch? > Riiight. I'm afraid if I use HOST instead of LINK that I'll break some strange SDP loopback feature or some such thing. And I'm not in a position to test that. But I can post a patch. Shall I just change sles9sp3 since we don't see (yet) any problems with the other distros? ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaround for ip_dev_find() bug.
How shall I fix this?
I think the correct scope is RT_SCOPE_HOST.
Anyone know why RT_SCOPE_LINK was chosen?
On Tue, 2007-02-06 at 15:17 -0600, Steve Wise wrote:
> > Try copying inet_select_addr source in from some upstream kernel,
> > look at that.
> >
>
> It appears that xxx_ip_find_dev() should be calling inet_select_addr
> with RT_SCOPE_HOST and not RT_SCOPE_LINK. Everything works fine for me
> if I change xxx_ip_find_dev() to use RT_SCOPE_HOST.
>
>
> >From the header file linux/rtnetlink.h. Note the comment on HOST vs
> LINK:
>
>
> /* rtm_scope
>
>Really it is not scope, but sort of distance to the destination.
>NOWHERE are reserved for not existing destinations, HOST is our
>local addresses, LINK are destinations, located on directly attached
>link and UNIVERSE is everywhere in the Universe.
>
>Intermediate values are also possible f.e. interior routes
>could be assigned a value between UNIVERSE and LINK.
> */
>
> enum rt_scope_t
> {
> RT_SCOPE_UNIVERSE=0,
> /* User defined values */
> RT_SCOPE_SITE=200,
> RT_SCOPE_LINK=253,
> RT_SCOPE_HOST=254,
> RT_SCOPE_NOWHERE=255
> };
>
>
>
> ___
> openib-general mailing list
> openib-general@openib.org
> http://openib.org/mailman/listinfo/openib-general
>
> To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
>
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport -IWCM workaround for ip_dev_find() bug.
> Try copying inet_select_addr source in from some upstream kernel,
> look at that.
>
It appears that xxx_ip_find_dev() should be calling inet_select_addr
with RT_SCOPE_HOST and not RT_SCOPE_LINK. Everything works fine for me
if I change xxx_ip_find_dev() to use RT_SCOPE_HOST.
>From the header file linux/rtnetlink.h. Note the comment on HOST vs
LINK:
/* rtm_scope
Really it is not scope, but sort of distance to the destination.
NOWHERE are reserved for not existing destinations, HOST is our
local addresses, LINK are destinations, located on directly attached
link and UNIVERSE is everywhere in the Universe.
Intermediate values are also possible f.e. interior routes
could be assigned a value between UNIVERSE and LINK.
*/
enum rt_scope_t
{
RT_SCOPE_UNIVERSE=0,
/* User defined values */
RT_SCOPE_SITE=200,
RT_SCOPE_LINK=253,
RT_SCOPE_HOST=254,
RT_SCOPE_NOWHERE=255
};
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport - IWCM workaround for ip_dev_find() bug.
> > >
> > > Can we just backport our own version of ip_dev_find()? We had this once
> > > before
> > > in svn when they removed it from being exported from the kernel.
> >
> > Yes, this is in kernel_addons for 2.6.19 or something like that.
> > Just copy from there, much cleaner than the patch.
> >
>
> I just realized that ip_dev_find() is being redefined to xxx_ip_dev_find
> for sles9sp3. So maybe this function is causing the error. Stay tuned.
xxx_ip_dev_find() is returning the wrong interface (sometimes). I added
printks to xxx_ip_dev_find(). Then I ran rping -s -a
and it failed because xxx_ip_dev_find() returned loopback instead of my
eth device.
Here is the function with printks:
static inline struct net_device *xxx_ip_dev_find(u32 addr)
{
struct net_device *dev;
u32 ip;
read_lock(&dev_base_lock);
printk("%s looking for dev with addr %x\n", __FUNCTION__, addr);
for (dev = dev_base; dev; dev = dev->next) {
ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
printk("%s dev %p name %s ipaddr %x\n", __FUNCTION__,
dev, dev->name, ip);
if (ip == addr) {
dev_hold(dev);
break;
}
}
read_unlock(&dev_base_lock);
return dev;
}
Here is the printk log showing loopback being returned:
xxx_ip_dev_find looking for dev with addr 8846a8c0
xxx_ip_dev_find dev 804000e0 name lo ipaddr 8846a8c0
The address bound to eth3 is 192.168.70.136 (0xc0a84688). For some
reason, this line:
ip = inet_select_addr(dev, 0, RT_SCOPE_LINK);
Returns the 192.168.70.136 address for device->name == "lo".
Riddle me that!
Also, sometimes it works ok because the loopback interface gets some
other ip address that is assigned to the local system as opposed to my
rdma address. For example, I booted up the sles9sp3 system with a
rebuilt kernel and no ofed modules installed. The system gets
10.10.0.136 via DHCP for its "public" interface. I then built the ofed
modules and installed them. I then loaded them and configured my rnic
interface with 192.168.70.136. I ran rping and bound to the local
ipaddr and it worked. The log showed that inet_select_addr() returned
10.10.0.136 for loopback and thus xxx_ip_dev_find() continued walking
the list and found the correct ethernet interface. I then rebooted and
ran the test again and it failed. So somehow module load order affects
this, I think.
g.
Steve.
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport - IWCM workaround for ip_dev_find() bug.
On Tue, 2007-02-06 at 21:22 +0200, Michael S. Tsirkin wrote: > > Quoting Sean Hefty <[EMAIL PROTECTED]>: > > Subject: Re: [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport - IWCM workaround > > for ip_dev_find() bug. > > > > > Actually, yes it does. Here's one case (that I just tested :): > > > > > > If you rdma_bind() to an explicit address local address, it will fail. > > > > > > Foo! > > > > > > I guess I'll need to address the uses of ip_dev_find() in addr.c as well > > > before we commit this. > > > > Can we just backport our own version of ip_dev_find()? We had this once > > before > > in svn when they removed it from being exported from the kernel. > > Yes, this is in kernel_addons for 2.6.19 or something like that. > Just copy from there, much cleaner than the patch. > I just realized that ip_dev_find() is being redefined to xxx_ip_dev_find for sles9sp3. So maybe this function is causing the error. Stay tuned. Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] OFED-1.2 first release
I already opened one for the libibverbs.d problem. 339 On Tue, 2007-02-06 at 10:54 -0800, Scott Weitzenkamp (sweitzen) wrote: > libibverbs is not working. I have opened bugs 342-346 for the issues > I've found so far: > > # ibv_devices > libibverbs: Warning: couldn't open config directory > '/usr/local/ofed/etc/libibverbs.d'. > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/in > finiband_verbs/uverbs0 > device node GUID > -- > > > Scott Weitzenkamp > SQA and Release Manager > Server Virtualization Business Unit > Cisco Systems > > > > __ > From: Scott Weitzenkamp (sweitzen) > Sent: Tuesday, February 06, 2007 9:34 AM > To: Scott Weitzenkamp (sweitzen); 'Vladimir Sokolovsky'; > '[EMAIL PROTECTED]'; 'Tziporet Koren' > Cc: 'openib-general@openib.org' > Subject: RE: [openib-general] OFED-1.2 first release > > > > sdpnetstat is getting added to the dapl-devel RPM. > > # rpm -qlip dapl-devel-1.2.0-0.x86_64.rpm > Name: dapl-devel Relocations: (not > relocatable) > Version : 1.2.0 Vendor: > OpenFabrics > Release : 0 Build Date: Mon 05 > Feb 2007 09:48:50 > PM PST > Install Date: (not installed) Build Host: > svbu-qa1850-1.cisco.com > Group : System Environment/Libraries Source RPM: > ofa_user-1.2-alpha1.src > .rpm > Size: 692598 License: > GPL/BSD > Signature : (none) > URL : http://www.openfabrics.org/ > Summary : Development files for the libdat and libdapl > libraries > Description : > Static libraries and header files for the libdat and libdapl > library. > /usr/local/ofed/bin/sdpnetstat > /usr/local/ofed/include/dat/dat.h > /usr/local/ofed/include/dat/dat_error.h > /usr/local/ofed/include/dat/dat_platform_specific.h > /usr/local/ofed/include/dat/dat_redirection.h > /usr/local/ofed/include/dat/dat_registry.h > /usr/local/ofed/include/dat/dat_vendor_specific.h > /usr/local/ofed/include/dat/udat.h > /usr/local/ofed/include/dat/udat_config.h > /usr/local/ofed/include/dat/udat_redirection.h > /usr/local/ofed/include/dat/udat_vendor_specific.h > /usr/local/ofed/lib64/libdaplcma.a > /usr/local/ofed/lib64/libdaplcma.so > /usr/local/ofed/lib64/libdat.a > /usr/local/ofed/lib64/libdat.so > > > > __ > From: Scott Weitzenkamp (sweitzen) > Sent: Tuesday, February 06, 2007 12:07 AM > To: Scott Weitzenkamp (sweitzen); 'Vladimir > Sokolovsky'; '[EMAIL PROTECTED]'; 'Tziporet > Koren' > Cc: 'openib-general@openib.org' > Subject: RE: [openib-general] OFED-1.2 first release > > > > Not getting MPI RPMS for Intel compilers, either. > > Running /bin/rpm > -Uhv > /tmp/OFED-1.2-20070205-1823/RPMS/redhat-release-4AS-4.1/mp > itests_mvapich2_gcc-2.0-698.x86_64.rpm > > /tmp/OFED-1.2-20070205-1823/RPMS/redhat-release-4AS-4.1/mvapich2_intel-0.9.8-1.x > 86_64.rpm not found > Running /bin/rpm > -Uhv > /tmp/OFED-1.2-20070205-1823/RPMS/redhat-release-4AS-4.1/op > enmpi_gcc-1.2b4ofedr13470-1ofed.x86_64.rpm > Running /bin/rpm > -Uhv > /tmp/OFED-1.2-20070205-1823/RPMS/redhat-release-4AS-4.1/mp > itests_openmpi_gcc-2.0-698.x86_64.rpm > > /tmp/OFED-1.2-20070205-1823/RPMS/redhat-release-4AS-4.1/openmpi_intel-1.2b4ofedr > 13470-1ofed.x86_64.rpm not found > ERROR: -.x86_64.rpm not found > under /tmp/OFED-1.2-20070205-1823/RPMS/redhat-rele > ase-4AS-4.1. > Installation finished successfully... > > Scott > > > __ > From: Scott Weitzenkamp (sweitzen) > Sent: Monday, February 05, 2007 9:44 PM > To: Scott Weitzenkamp (sweitzen); 'Vladimir > Sokolovsky'; '[EMAIL PROTECTED]'; > 'Tziporet Koren' >
Re: [openib-general] OFED-1.2 first release
opened bug 340. On Mon, 2007-02-05 at 19:07 -0600, Steve Wise wrote: > I think there might be some dependency problem. I selected libibverbs, > libcxgb3, librdmacm, perftest, mvapich2/IWARP and mpitests. For some > reason it pulled in libibumad as a prereq, but not libibcommon... > > Also, I think mvapich2/IWARP links with libibumad or libibcommon and it > doesn't need to when using librdmacm. > > > > [EMAIL PROTECTED] redhat-release-4AS-5.5]# rpm -U * > error: Failed dependencies: > libibcommon.so.1()(64bit) is needed by libibumad-1.0.2-0.x86_64 > libibcommon.so.1(IBCOMMON_1.0)(64bit) is needed by > libibumad-1.0.2-0.x86_64 > Suggested resolutions: > libibcommon-1.0-1.x86_64.rpm > > > > On Tue, 2007-02-06 at 00:25 +0200, Vladimir Sokolovsky wrote: > > > Hi, > > > > > > OFED-1.2-20070205-1823.tgz can be downloaded from > > > > > > http://www.openfabrics.org/builds/ofed-1.2/ > > > > > > > > > > > > > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit > > http://openib.org/mailman/listinfo/openib-general > > > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport - IWCM workaround for ip_dev_find() bug.
On Tue, 2007-02-06 at 09:37 -0800, Sean Hefty wrote: > Steve Wise wrote: > > I propose the following fix for supporting iWARP on SLES9SP3. > > > > This fixes bug 325. > > > > Sean, can you please review this? > > The changes seem fine with me. > > Does this bug affect the ib_addr module as well? (addr_resolve_local and > rdma_translate_ip) > Actually, yes it does. Here's one case (that I just tested :): If you rdma_bind() to an explicit address local address, it will fail. Foo! I guess I'll need to address the uses of ip_dev_find() in addr.c as well before we commit this. What really bothers me is I cannot find the kernel code in the 2.6.5-7.244 kernel that is doing this (returning loopback for all local devices). ip_dev_find() does a FIB lookup to find this. I dug around the fib code but so far haven't found the culprit. I welcome any help from anyone out there interested in the rdma-cm working on sles9sp3. I would think if SDP does an rdma_bind() then SDP will also see this bug when run on sles9sp3. (Are SUSE folks listening?) Any thoughts? Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] [PATCH] [RFC] ofed_1_2 - SLES9SP3 Backport - IWCM workaround for ip_dev_find() bug.
I propose the following fix for supporting iWARP on SLES9SP3.
This fixes bug 325.
Sean, can you please review this?
Steve.
---
SLES9SP3 Backport - IWCM workaround for ip_dev_find() bug.
Acquire the cma_dev based on the ib device of the incoming
connect request.
This overcomes a sles9sp3 bug where ip_dev_find(local_ipaddr) always
returns the loopback net_device pointer instead of the actual local
interface pointer. Note: this workaround leaves the rdma_dev_addr in
the new connection request rdma_cm_id incomplete. But ULPs don't really
use this, so we'll have to live with it for SLES9SP3.
Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
---
.../iwcm_ip_dev_find_workaround.patch | 91
+++
1 files changed, 91 insertions(+), 0 deletions(-)
diff --git
a/kernel_patches/backport/2.6.5_sles9_sp3/iwcm_ip_dev_find_workaround.patch
b/kernel_patches/backport/2.6.5_sles9_sp3/iwcm_ip_dev_find_workaround.patch
new file mode 100644
index 000..a9d5bfe
--- /dev/null
+++ b/kernel_patches/backport/2.6.5_sles9_sp3/iwcm_ip_dev_find_workaround.patch
@@ -0,0 +1,91 @@
+SLES9SP3 Backport - IWCM workaround for ip_dev_find() bug.
+
+From: Steve Wise <[EMAIL PROTECTED]>
+
+Acquire the cma_dev based on the ib device of the incoming
+connect request.
+
+This overcomes a sles9sp3 bug where ip_dev_find(local_ipaddr) always
+returns the loopback net_device pointer instead of the actual local
+interface pointer. Note: this workaround leaves the rdma_dev_addr in
+the new connection request rdma_cm_id incomplete. But ULPs don't really
+use this, so we'll have to live with it for SLES9SP3.
+
+Signed-off-by: Steve Wise <[EMAIL PROTECTED]>
+---
+
+ drivers/infiniband/core/cma.c | 33 +++--
+ 1 files changed, 15 insertions(+), 18 deletions(-)
+
+diff --git a/drivers/infiniband/core/cma.c b/drivers/infiniband/core/cma.c
+index 9e0ab04..c89b611 100644
+--- a/drivers/infiniband/core/cma.c
b/drivers/infiniband/core/cma.c
+@@ -1128,13 +1128,25 @@ static int cma_iw_handler(struct iw_cm_i
+ return ret;
+ }
+
++static int iw_cma_acquire_dev(struct iw_cm_id *cm_id, struct rdma_id_private
*id_priv)
++{
++ struct cma_device *cma_dev;
++
++ list_for_each_entry(cma_dev, &dev_list, list) {
++ if (cma_dev->device == cm_id->device) {
++ cma_attach_to_dev(id_priv, cma_dev);
++ return 0;
++ }
++ }
++ return -ENODEV;
++}
++
+ static int iw_conn_req_handler(struct iw_cm_id *cm_id,
+ struct iw_cm_event *iw_event)
+ {
+ struct rdma_cm_id *new_cm_id;
+ struct rdma_id_private *listen_id, *conn_id;
+ struct sockaddr_in *sin;
+- struct net_device *dev = NULL;
+ struct rdma_cm_event event;
+ int ret;
+
+@@ -1157,22 +1169,8 @@ static int iw_conn_req_handler(struct iw
+ atomic_inc(&conn_id->dev_remove);
+ conn_id->state = CMA_CONNECT;
+
+- dev = ip_dev_find(iw_event->local_addr.sin_addr.s_addr);
+- if (!dev) {
+- ret = -EADDRNOTAVAIL;
+- cma_release_remove(conn_id);
+- rdma_destroy_id(new_cm_id);
+- goto out;
+- }
+- ret = rdma_copy_addr(&conn_id->id.route.addr.dev_addr, dev, NULL);
+- if (ret) {
+- cma_release_remove(conn_id);
+- rdma_destroy_id(new_cm_id);
+- goto out;
+- }
+-
+ mutex_lock(&lock);
+- ret = cma_acquire_dev(conn_id);
++ ret = iw_cma_acquire_dev(cm_id, conn_id);
+ mutex_unlock(&lock);
+ if (ret) {
+ cma_release_remove(conn_id);
+@@ -1184,6 +1182,7 @@ static int iw_conn_req_handler(struct iw
+ cm_id->context = conn_id;
+ cm_id->cm_handler = cma_iw_handler;
+
++ new_cm_id->route.addr.dev_addr.dev_type = RDMA_NODE_RNIC;
+ sin = (struct sockaddr_in *) &new_cm_id->route.addr.src_addr;
+ *sin = iw_event->local_addr;
+ sin = (struct sockaddr_in *) &new_cm_id->route.addr.dst_addr;
+@@ -1203,8 +1202,6 @@ static int iw_conn_req_handler(struct iw
+ }
+
+ out:
+- if (dev)
+- dev_put(dev);
+ cma_release_remove(listen_id);
+ return ret;
+ }
___
openib-general mailing list
openib-general@openib.org
http://openib.org/mailman/listinfo/openib-general
To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] build.sh not building libmthca
bug 338 opened. On Tue, 2007-02-06 at 12:05 -0500, Jeff Squyres wrote: > Yes, please file all bugs in bugzilla. > > Thanks! > > > On Feb 6, 2007, at 11:41 AM, Steve Wise wrote: > > > Do you want me to use bugzilla to track these issues? > > > > > > On Tue, 2007-02-06 at 10:06 -0600, Steve Wise wrote: > >> Another build problem with the alpha test package: > >> > >> If I run build.sh and _only_ select libmthca, it claims it builds > >> it ok, > >> but doesn't produce the .rpm file... > >> > >> Steve. > >> > >> > >> ___ > >> openib-general mailing list > >> openib-general@openib.org > >> http://openib.org/mailman/listinfo/openib-general > >> > >> To unsubscribe, please visit http://openib.org/mailman/listinfo/ > >> openib-general > >> > > > > > > ___ > > openib-general mailing list > > openib-general@openib.org > > http://openib.org/mailman/listinfo/openib-general > > > > To unsubscribe, please visit http://openib.org/mailman/listinfo/ > > openib-general > > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] OFED-1.2 first release - provider library install problem
bug 339 opened. On Tue, 2007-02-06 at 10:50 -0600, Steve Wise wrote: > provider library install problem ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] OFED-1.2 first release - provider library install problem
FYI: The libmthca rpm has the same issue... Steve. On Tue, 2007-02-06 at 09:38 -0600, Steve Wise wrote: > Vlad, > > After installing the test alpha1 build rpms on rhel4u4 with a > kernel.org 2.6.20 kernel, it appears that the provider library config > files didn't get installed for libcxgb3: > > [EMAIL PROTECTED] ~]# rping -s -a 0.0.0.0 -p > libibverbs: Warning: couldn't open config directory > '/usr/local/ofed/etc/libibverbs.d'. > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > libibverbs: Warning: couldn't open config directory > '/usr/local/ofed/etc/libibverbs.d'. > libibverbs: Warning: no userspace device-specific driver found for > /sys/class/infiniband_verbs/uverbs0 > libibverbs: Warning: no userspace device-specific driver found for uverbs0 > Segmentation fault > [EMAIL PROTECTED] ~]# ls /usr/local/ofed/etc > ls: /usr/local/ofed/etc: No such file or directory > [EMAIL PROTECTED] ~]# > > I'm running with the cxgb3 driver, so I guess libcxgb3 didn't install > itself correctly? This works when doing 'make install' from the > userspace tarballs. Is there some rpm magic missing? I'm not sure how > to debug this as I'm rpm-challenged (but willing to learn :). > > It appears libcxgb3 installed its v2 libs correctly: > > [EMAIL PROTECTED] ~]# ls /usr/local/ofed/lib64 > libcxgb3.a libdat.a libibcommon.solibibumad.a >libibverbs.so.1.0.0 > libcxgb3-rdmav2.so libdat.so libibcommon.so.1 libibumad.so >librdmacm.so > libcxgb3.so libdat.so.1 libibcommon.so.1.0.0 libibumad.so.1 >librdmacm.so.0.9.0 > libdaplcma.a libdat.so.1.0.2 libibmad.a > libibumad.so.1.0.0 > libdaplcma.solibibcm.solibibmad.so libibverbs.a > libdaplcma.so.1 libibcm.so.0.9.0 libibmad.so.1 libibverbs.so > libdaplcma.so.1.0.2 libibcommon.a libibmad.so.1.2.0 libibverbs.so.1 > [EMAIL PROTECTED] ~]# > > But /usr/local/ofed/etc/libibverbs.d didn't get created and the cxgb3.driver > file installed. > > > > Steve. > > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
Re: [openib-general] build.sh not building libmthca
Do you want me to use bugzilla to track these issues? On Tue, 2007-02-06 at 10:06 -0600, Steve Wise wrote: > Another build problem with the alpha test package: > > If I run build.sh and _only_ select libmthca, it claims it builds it ok, > but doesn't produce the .rpm file... > > Steve. > > > ___ > openib-general mailing list > openib-general@openib.org > http://openib.org/mailman/listinfo/openib-general > > To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general > ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
[openib-general] build.sh not building libmthca
Another build problem with the alpha test package: If I run build.sh and _only_ select libmthca, it claims it builds it ok, but doesn't produce the .rpm file... Steve. ___ openib-general mailing list openib-general@openib.org http://openib.org/mailman/listinfo/openib-general To unsubscribe, please visit http://openib.org/mailman/listinfo/openib-general
