Re: [PATCH 1/2]: Powerpc: Fix EHCA driver on relocatable kernel
On Thu, 19 Aug 2010 23:08:09 -0500
Sonny Rao wrote:
> Some modules (like eHCA) want to map all of kernel memory, for this to
> work with a relocated kernel, we need to export kernstart_addr so
> modules can use PHYSICAL_START and memstart_addr so they could use
> MEMORY_START. Note that the 32bit code already exports these symbols.
>
> Signed-off-By: Sonny Rao
Acked-by: Alexander Schmidt
> Index: common/arch/powerpc/mm/init_64.c
> ===
> --- common.orig/arch/powerpc/mm/init_64.c 2010-08-16 02:38:33.0
> -0500
> +++ common/arch/powerpc/mm/init_64.c 2010-08-16 02:39:25.0 -0500
> @@ -79,7 +79,9 @@
> #endif /* CONFIG_PPC_STD_MMU_64 */
>
> phys_addr_t memstart_addr = ~0;
> +EXPORT_SYMBOL(memstart_addr);
> phys_addr_t kernstart_addr;
> +EXPORT_SYMBOL(kernstart_addr);
>
> void free_initmem(void)
> {
> ___
> Linuxppc-dev mailing list
> Linuxppc-dev@lists.ozlabs.org
> https://lists.ozlabs.org/listinfo/linuxppc-dev
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 25/37] drivers/infiniband: use .dev.of_node instead of .node in struct of_device
On Thu, 11 Mar 2010 11:57:53 -0800 Roland Dreier wrote: > Seems fine... adding EHCA guys just to make sure. > > > .node is being removed > > > > Signed-off-by: Grant Likely Acked-by: Alexander Schmidt ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
[PATCH] ehca: use port autodetect mode as default
This patch sets the port autodetect mode as default for the ehca driver. The
autodetect code has been in the kernel for several releases now and has
proved to be stable.
---
Roland, please queue this change for 2.6.32 if you are okay with it.
drivers/infiniband/hw/ehca/ehca_main.c |8
1 file changed, 4 insertions(+), 4 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_main.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_main.c
@@ -52,7 +52,7 @@
#include "ehca_tools.h"
#include "hcp_if.h"
-#define HCAD_VERSION "0028"
+#define HCAD_VERSION "0029"
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Christoph Raisch ");
@@ -64,7 +64,7 @@ static int ehca_hw_level = 0;
static int ehca_poll_all_eqs = 1;
int ehca_debug_level = 0;
-int ehca_nr_ports = 2;
+int ehca_nr_ports = -1;
int ehca_use_hp_mr = 0;
int ehca_port_act_time = 30;
int ehca_static_rate = -1;
@@ -95,8 +95,8 @@ MODULE_PARM_DESC(hw_level,
"Hardware level (0: autosensing (default), "
"0x10..0x14: eHCA, 0x20..0x23: eHCA2)");
MODULE_PARM_DESC(nr_ports,
-"number of connected ports (-1: autodetect, 1: port one only, "
-"2: two ports (default)");
+"number of connected ports (-1: autodetect (default), "
+"1: port one only, 2: two ports)");
MODULE_PARM_DESC(use_hp_mr,
"Use high performance MRs (default: no)");
MODULE_PARM_DESC(port_act_time,
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [ewg] Re: [PATCH 2.6.31 try 2] ehca: Tolerate dynamic memory operations and huge pages
On Mon, 22 Jun 2009 22:19:21 -0700
Roland Dreier wrote:
> thanks, applied.
>
> > -#define HCAD_VERSION "0026"
> > +#define HCAD_VERSION "0027"
>
> the driver version was already 0027 (since bde2cfaf), so I dropped this chunk.
thank you for applying, we would like to increase the version number for this
patch, so please also apply the following:
ehca: Increment version number for DMEM toleration
Signed-off-by: Alexander Schmidt
---
drivers/infiniband/hw/ehca/ehca_main.c |2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_main.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_main.c
@@ -52,7 +52,7 @@
#include "ehca_tools.h"
#include "hcp_if.h"
-#define HCAD_VERSION "0027"
+#define HCAD_VERSION "0028"
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Christoph Raisch ");
___
Linuxppc-dev mailing list
Linuxppc-dev@lists.ozlabs.org
https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [ewg] Re: [PATCH 2.6.31] ehca: Tolerate dynamic memory operations and huge pages
On Tue, 16 Jun 2009 09:10:39 -0700 Roland Dreier wrote: > > > Yeah, the notifier code remains untouched as we still do not allow dynamic > > memory operations _while_ our module is loaded. The patch allows the > driver to > > cope with DMEM operations that happened before the module was loaded, which > > might result in a non-contiguous memory layout. When the driver registers > > its global memory region in the system, the memory layout must be > considered. > > > > We chose the term "toleration" instead of "support" to illustrate this. > > I see. So things just silently broke in some cases when the driver was > loaded after operations you didn't tolerate? > > Anyway, thanks for the explanation. Well, things did not break silently. The registration of the MR failed with an error code which was reported to userspace. Will you push the patch for .31 or .32? Thanks, Alex ___ Linuxppc-dev mailing list Linuxppc-dev@lists.ozlabs.org https://lists.ozlabs.org/listinfo/linuxppc-dev
Re: [PATCH 2.6.31 try 2] ehca: Tolerate dynamic memory operations and huge pages
From: Hannes Hering
This patch implements toleration of dynamic memory operations and 16 GB
gigantic pages. "Toleration" means that the driver can cope with dynamic
memory operations that happened before the driver was loaded. While using the
ehca driver, dynamic memory operations are still prohibited. On module load the
driver walks through available system memory, checks for available memory ranges
and then registers the kernel internal memory region accordingly. The
translation of address ranges is implemented via a 3-level busmap.
Signed-off-by: Hannes Hering
---
This patch is built and tested against infiniband.git. Please apply for 2.6.31.
drivers/infiniband/hw/ehca/ehca_main.c | 20 +
drivers/infiniband/hw/ehca/ehca_mrmw.c | 508 -
drivers/infiniband/hw/ehca/ehca_mrmw.h | 13
3 files changed, 523 insertions(+), 18 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_main.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_main.c
@@ -52,7 +52,7 @@
#include "ehca_tools.h"
#include "hcp_if.h"
-#define HCAD_VERSION "0026"
+#define HCAD_VERSION "0027"
MODULE_LICENSE("Dual BSD/GPL");
MODULE_AUTHOR("Christoph Raisch ");
@@ -506,6 +506,7 @@ static int ehca_init_device(struct ehca_
shca->ib_device.detach_mcast= ehca_detach_mcast;
shca->ib_device.process_mad = ehca_process_mad;
shca->ib_device.mmap= ehca_mmap;
+ shca->ib_device.dma_ops = &ehca_dma_mapping_ops;
if (EHCA_BMASK_GET(HCA_CAP_SRQ, shca->hca_cap)) {
shca->ib_device.uverbs_cmd_mask |=
@@ -1028,17 +1029,23 @@ static int __init ehca_module_init(void)
goto module_init1;
}
+ ret = ehca_create_busmap();
+ if (ret) {
+ ehca_gen_err("Cannot create busmap.");
+ goto module_init2;
+ }
+
ret = ibmebus_register_driver(&ehca_driver);
if (ret) {
ehca_gen_err("Cannot register eHCA device driver");
ret = -EINVAL;
- goto module_init2;
+ goto module_init3;
}
ret = register_memory_notifier(&ehca_mem_nb);
if (ret) {
ehca_gen_err("Failed registering memory add/remove notifier");
- goto module_init3;
+ goto module_init4;
}
if (ehca_poll_all_eqs != 1) {
@@ -1053,9 +1060,12 @@ static int __init ehca_module_init(void)
return 0;
-module_init3:
+module_init4:
ibmebus_unregister_driver(&ehca_driver);
+module_init3:
+ ehca_destroy_busmap();
+
module_init2:
ehca_destroy_slab_caches();
@@ -1073,6 +1083,8 @@ static void __exit ehca_module_exit(void
unregister_memory_notifier(&ehca_mem_nb);
+ ehca_destroy_busmap();
+
ehca_destroy_slab_caches();
ehca_destroy_comp_pool();
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_mrmw.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_mrmw.c
@@ -53,6 +53,38 @@
/* max number of rpages (per hcall register_rpages) */
#define MAX_RPAGES 512
+/* DMEM toleration management */
+#define EHCA_SECTSHIFTSECTION_SIZE_BITS
+#define EHCA_SECTSIZE (1UL << EHCA_SECTSHIFT)
+#define EHCA_HUGEPAGESHIFT 34
+#define EHCA_HUGEPAGE_SIZE (1UL << EHCA_HUGEPAGESHIFT)
+#define EHCA_HUGEPAGE_PFN_MASK ((EHCA_HUGEPAGE_SIZE - 1) >> PAGE_SHIFT)
+#define EHCA_INVAL_ADDR0xULL
+#define EHCA_DIR_INDEX_SHIFT 13 /* 8k Entries in 64k block */
+#define EHCA_TOP_INDEX_SHIFT (EHCA_DIR_INDEX_SHIFT * 2)
+#define EHCA_MAP_ENTRIES (1 << EHCA_DIR_INDEX_SHIFT)
+#define EHCA_TOP_MAP_SIZE (0x1) /* currently fixed map size
*/
+#define EHCA_DIR_MAP_SIZE (0x1)
+#define EHCA_ENT_MAP_SIZE (0x1)
+#define EHCA_INDEX_MASK (EHCA_MAP_ENTRIES - 1)
+
+static unsigned long ehca_mr_len;
+
+/*
+ * Memory map data structures
+ */
+struct ehca_dir_bmap {
+ u64 ent[EHCA_MAP_ENTRIES];
+};
+struct ehca_top_bmap {
+ struct ehca_dir_bmap *dir[EHCA_MAP_ENTRIES];
+};
+struct ehca_bmap {
+ struct ehca_top_bmap *top[EHCA_MAP_ENTRIES];
+};
+
+static struct ehca_bmap *ehca_bmap;
+
static struct kmem_cache *mr_cache;
static struct kmem_cache *mw_cache;
@@ -68,6 +100,8 @@ enum ehca_mr_pgsize {
#define EHCA_MR_PGSHIFT1M 20
#define EHCA_MR_PGSHIFT16M 24
+static u64 ehca_map_vaddr(void *caddr);
+
static u32 ehca_encode_hwpage_size(u32 pgsize)
{
int log = ilog2(pgsize);
@@ -135,7 +169,8 @@ struct ib_mr *ehca_get_dma_mr(struct ib_
goto get_dma_mr_exit0;
}
- ret = ehca_reg_maxmr(shca, e_maxmr, (u64 *)KERNELBASE,
+ ret = ehca_reg_maxmr(shca, e_maxmr,
+(void *)ehca_map_vaddr((void *)KERNELBASE),
mr_access_flags, e_pd,
&e_maxmr->ib.ib_mr.lk
Re: [PATCH 2.6.31] ehca: Tolerate dynamic memory operations and huge pages
Hi Roland,
thank you for taking a look at the code!
On Fri, 12 Jun 2009 21:50:58 -0700
Roland Dreier wrote:
> OK, one major issue with this patch and a few minor nits.
>
> First, the major issue is that I don't see anything in the patch that
> changes the code in ehca_mem_notifier() in ehca_main.c:
>
> case MEM_GOING_ONLINE:
> case MEM_GOING_OFFLINE:
> /* only ok if no hca is attached to the lpar */
> spin_lock_irqsave(&shca_list_lock, flags);
> if (list_empty(&shca_list)) {
> spin_unlock_irqrestore(&shca_list_lock, flags);
> return NOTIFY_OK;
> } else {
> spin_unlock_irqrestore(&shca_list_lock, flags);
> if (printk_timed_ratelimit(&ehca_dmem_warn_time,
> 30 * 1000))
> ehca_gen_err("DMEM operations are not allowed"
>"in conjunction with eHCA");
> return NOTIFY_BAD;
> }
>
> But your patch description says:
>
> > This patch implements toleration of dynamic memory operations
>
> But it seems you're still going to hit the same NOTIFY_BAD case above
> after your patch. So something doesn't compute for me. Could you
> explain more?
Yeah, the notifier code remains untouched as we still do not allow dynamic
memory operations _while_ our module is loaded. The patch allows the driver to
cope with DMEM operations that happened before the module was loaded, which
might result in a non-contiguous memory layout. When the driver registers
its global memory region in the system, the memory layout must be considered.
We chose the term "toleration" instead of "support" to illustrate this.
I'll put some more details into the changelog, incorporate the other comments
and send out a second version of the patch.
Thanks,
Alex
>
> Second, a nit:
>
> > +#define EHCA_REG_MR 0
> > +#define EHCA_REG_BUSMAP_MR (~0)
>
> and you pass these as the reg_busmap parm in:
>
> > int ehca_reg_mr(struct ehca_shca *shca,
> >struct ehca_mr *e_mr,
> >u64 *iova_start,
> > @@ -991,7 +1031,8 @@
> >struct ehca_pd *e_pd,
> >struct ehca_mr_pginfo *pginfo,
> >u32 *lkey, /*OUT*/
> > - u32 *rkey) /*OUT*/
> > + u32 *rkey, /*OUT*/
> > + int reg_busmap)
>
> and test it as:
>
> > + if (reg_busmap)
> > + ret = ehca_reg_bmap_mr_rpages(shca, e_mr, pginfo);
> > + else
> > + ret = ehca_reg_mr_rpages(shca, e_mr, pginfo);
>
> So the ~0 for true looks a bit odd. One option would be to make
> reg_busmap a bool, since that's how you're using it, but then you lose
> the nice self-documenting macro where you call things.
>
> So I think it would be cleaner to do something like
>
> enum ehca_reg_type {
> EHCA_REG_MR,
> EHCA_REG_BUSMAP_MR
> };
>
> and make the "int reg_busmap" parameter into "enum ehca_reg_type reg_type"
> and have the code become
>
> + if (reg_type == EHCA_REG_BUSMAP_MR)
> + ret = ehca_reg_bmap_mr_rpages(shca, e_mr, pginfo);
> + else if (reg_type == EHCA_REG_MR)
> + ret = ehca_reg_mr_rpages(shca, e_mr, pginfo);
> + else
> + ret = -EINVAL
>
> or something like that.
>
> > +struct ib_dma_mapping_ops ehca_dma_mapping_ops = {
> > + .mapping_error = ehca_dma_mapping_error,
> > + .map_single = ehca_dma_map_single,
> > + .unmap_single = ehca_dma_unmap_single,
> > + .map_page = ehca_dma_map_page,
> > + .unmap_page = ehca_dma_unmap_page,
> > + .map_sg = ehca_dma_map_sg,
> > + .unmap_sg = ehca_dma_unmap_sg,
> > + .dma_address = ehca_dma_address,
> > + .dma_len = ehca_dma_len,
> > + .sync_single_for_cpu = ehca_dma_sync_single_for_cpu,
> > + .sync_single_for_device = ehca_dma_sync_single_for_device,
> > + .alloc_coherent = ehca_dma_alloc_coherent,
> > + .free_coherent = ehca_dma_free_coherent,
> > +};
>
> I always think structures like this are easier to read if you align the
> '=' signs. But no big deal.
>
> > + ret = ehca_create_busmap();
> > + if (ret) {
> > + ehca_gen_err("Cannot create busmap.");
> > + goto module_init2;
> > + }
> > +
> >ret = ibmebus_register_driver(&ehca_driver);
> >if (ret) {
> >ehca_gen_err("Cannot register eHCA device driver");
> >ret = -EINVAL;
> > - goto module_init2;
> > + goto module_init3;
> >}
> >
> >ret = register_memory_notifier(&ehca_mem_nb);
> >if (ret) {
> >ehca_gen_err("Failed registering memory add/remove notifier");
> > - goto module_init3;
> > + goto module_init4;
>
> Having to renumber unrelated things is when something changes is why I
> don't like this style of error path labels. But I think it's well and
> truly too late to fix that in e
Re: [ewg] Re: [PATCH 1/3] IB/ehca: Replace vmalloc with kmalloc
On Tue, 28 Apr 2009 07:01:32 -0700 Roland Dreier wrote: > > did you have a chance to take a look at the patchset and will you apply > it, or > > are there any outstanding issues we need to address? > > I guess it's OK, but definitely 2.6.31 material. I guess I'll stick it > linux-next soon. > > - R. Okay with us, thank you very much! Alex ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
Re: [ewg] Re: [PATCH 1/3] IB/ehca: Replace vmalloc with kmalloc
Hi Roland,
did you have a chance to take a look at the patchset and will you apply it, or
are there any outstanding issues we need to address?
Regards,
Alex
On Wed, 22 Apr 2009 16:02:28 +0200
Stefan Roscher wrote:
> In case of large queue pairs there is the possibillity of allocation failures
> due to memory fragmentationo with kmalloc().To ensure the memory is allocated
> even
> if kmalloc() can not find chunks which are big enough, we try to allocate the
> memory
> with vmalloc().
>
> Signed-off-by: Stefan Roscher
> ---
>
> On Tuesday 21 April 2009 07:34:30 pm Roland Dreier wrote:
> > > +queue->queue_pages = kmalloc(nr_of_pages * sizeof(void *),
> > GFP_KERNEL);
> >
> > How big might this buffer be? Any chance of allocation failure due to
> > memory fragmentation?
> >
> > - R.
> Hey Roland,
> yes you are right and here is the patch to circumvent the described problem.
> It will apply on top of the patchset.
> regards Stefan
>
>
>
> drivers/infiniband/hw/ehca/ipz_pt_fn.c | 17 +
> 1 files changed, 13 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/infiniband/hw/ehca/ipz_pt_fn.c
> b/drivers/infiniband/hw/ehca/ipz_pt_fn.c
> index a260559..1227c59 100644
> --- a/drivers/infiniband/hw/ehca/ipz_pt_fn.c
> +++ b/drivers/infiniband/hw/ehca/ipz_pt_fn.c
> @@ -222,8 +222,11 @@ int ipz_queue_ctor(struct ehca_pd *pd, struct ipz_queue
> *queue,
> /* allocate queue page pointers */
> queue->queue_pages = kmalloc(nr_of_pages * sizeof(void *), GFP_KERNEL);
> if (!queue->queue_pages) {
> - ehca_gen_err("Couldn't allocate queue page list");
> - return 0;
> + queue->queue_pages = vmalloc(nr_of_pages * sizeof(void *));
> + if (!queue->queue_pages) {
> + ehca_gen_err("Couldn't allocate queue page list");
> + return 0;
> + }
> }
> memset(queue->queue_pages, 0, nr_of_pages * sizeof(void *));
>
> @@ -240,7 +243,10 @@ int ipz_queue_ctor(struct ehca_pd *pd, struct ipz_queue
> *queue,
> ipz_queue_ctor_exit0:
> ehca_gen_err("Couldn't alloc pages queue=%p "
>"nr_of_pages=%x", queue, nr_of_pages);
> - kfree(queue->queue_pages);
> + if (is_vmalloc_addr(queue->queue_pages))
> + vfree(queue->queue_pages);
> + else
> + kfree(queue->queue_pages);
>
> return 0;
> }
> @@ -262,7 +268,10 @@ int ipz_queue_dtor(struct ehca_pd *pd, struct ipz_queue
> *queue)
> free_page((unsigned long)queue->queue_pages[i]);
> }
>
> - kfree(queue->queue_pages);
> + if (is_vmalloc_addr(queue->queue_pages))
> + vfree(queue->queue_pages);
> + else
> + kfree(queue->queue_pages);
>
> return 1;
> }
___
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH] ib/ehca: add flush CQE generation
When a QP goes into error state, it is required that flush CQEs are
delivered to the application for any outstanding work requests. eHCA does not
do this in hardware, so this patch adds software flush CQE generation to the
ehca driver.
Whenever a QP gets into error state, it is added to the QP error list of its
respective CQ. If the error QP list of a CQ is not empty, poll_cq()
generates flush CQEs before polling the actual CQ.
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
Applies on top of 2.6.27-rc3, please consider this for 2.6.28.
drivers/infiniband/hw/ehca/ehca_classes.h | 14 +
drivers/infiniband/hw/ehca/ehca_cq.c |3
drivers/infiniband/hw/ehca/ehca_iverbs.h |2
drivers/infiniband/hw/ehca/ehca_qp.c | 225 --
drivers/infiniband/hw/ehca/ehca_reqs.c| 211
5 files changed, 412 insertions(+), 43 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_classes.h
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_classes.h
@@ -164,6 +164,13 @@ struct ehca_qmap_entry {
u16 reported;
};
+struct ehca_queue_map {
+ struct ehca_qmap_entry *map;
+ unsigned int entries;
+ unsigned int tail;
+ unsigned int left_to_poll;
+};
+
struct ehca_qp {
union {
struct ib_qp ib_qp;
@@ -173,8 +180,9 @@ struct ehca_qp {
enum ehca_ext_qp_type ext_type;
enum ib_qp_state state;
struct ipz_queue ipz_squeue;
- struct ehca_qmap_entry *sq_map;
+ struct ehca_queue_map sq_map;
struct ipz_queue ipz_rqueue;
+ struct ehca_queue_map rq_map;
struct h_galpas galpas;
u32 qkey;
u32 real_qp_num;
@@ -204,6 +212,8 @@ struct ehca_qp {
atomic_t nr_events; /* events seen */
wait_queue_head_t wait_completion;
int mig_armed;
+ struct list_head sq_err_node;
+ struct list_head rq_err_node;
};
#define IS_SRQ(qp) (qp->ext_type == EQPT_SRQ)
@@ -233,6 +243,8 @@ struct ehca_cq {
/* mmap counter for resources mapped into user space */
u32 mm_count_queue;
u32 mm_count_galpa;
+ struct list_head sqp_err_list;
+ struct list_head rqp_err_list;
};
enum ehca_mr_flag {
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -53,9 +53,25 @@
/* in RC traffic, insert an empty RDMA READ every this many packets */
#define ACK_CIRC_THRESHOLD 200
+static u64 replace_wr_id(u64 wr_id, u16 idx)
+{
+ u64 ret;
+
+ ret = wr_id & ~QMAP_IDX_MASK;
+ ret |= idx & QMAP_IDX_MASK;
+
+ return ret;
+}
+
+static u16 get_app_wr_id(u64 wr_id)
+{
+ return wr_id & QMAP_IDX_MASK;
+}
+
static inline int ehca_write_rwqe(struct ipz_queue *ipz_rqueue,
struct ehca_wqe *wqe_p,
- struct ib_recv_wr *recv_wr)
+ struct ib_recv_wr *recv_wr,
+ u32 rq_map_idx)
{
u8 cnt_ds;
if (unlikely((recv_wr->num_sge < 0) ||
@@ -69,7 +85,7 @@ static inline int ehca_write_rwqe(struct
/* clear wqe header until sglist */
memset(wqe_p, 0, offsetof(struct ehca_wqe, u.ud_av.sg_list));
- wqe_p->work_request_id = recv_wr->wr_id;
+ wqe_p->work_request_id = replace_wr_id(recv_wr->wr_id, rq_map_idx);
wqe_p->nr_of_data_seg = recv_wr->num_sge;
for (cnt_ds = 0; cnt_ds < recv_wr->num_sge; cnt_ds++) {
@@ -146,6 +162,7 @@ static inline int ehca_write_swqe(struct
u64 dma_length;
struct ehca_av *my_av;
u32 remote_qkey = send_wr->wr.ud.remote_qkey;
+ struct ehca_qmap_entry *qmap_entry = &qp->sq_map.map[sq_map_idx];
if (unlikely((send_wr->num_sge < 0) ||
(send_wr->num_sge > qp->ipz_squeue.act_nr_of_sg))) {
@@ -158,11 +175,10 @@ static inline int ehca_write_swqe(struct
/* clear wqe header until sglist */
memset(wqe_p, 0, offsetof(struct ehca_wqe, u.ud_av.sg_list));
- wqe_p->work_request_id = send_wr->wr_id & ~QMAP_IDX_MASK;
- wqe_p->work_request_id |= sq_map_idx & QMAP_IDX_MASK;
+ wqe_p->work_request_id = replace_wr_id(send_wr->wr_id, sq_map_idx);
- qp->sq_map[sq_map_idx].app_wr_id = send_wr->wr_id & QMAP_IDX_MASK;
- qp->sq_map[sq_map_idx].reported = 0;
+ qmap_entry->app_wr_id = get_app_wr_id(send_wr->wr_id);
+ qmap_entry->reported = 0;
switch (send_wr->opcode) {
case IB_WR_SEND:
@@ -496,7 +512,9 @@ static int internal_post_recv(struct ehc
struct ehca_wqe *wqe_p;
int wqe_cnt = 0;
int ret = 0;
+ u32 rq_map_idx;
unsigned long flags;
+ struct ehca_qmap_entry *qmap_entry;
if (unlikely(!HAS_RQ(my_qp))) {
[PATCH 2/5 try2] ib/ehca: rename goto label
Rename the "poll_cq_one_read_cqe" goto label to what it actually does,
"repoll".
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/ehca/ehca_reqs.c |6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -589,7 +589,7 @@ static inline int ehca_poll_cq_one(struc
struct ehca_qp *my_qp;
int cqe_count = 0, is_error;
-poll_cq_one_read_cqe:
+repoll:
cqe = (struct ehca_cqe *)
ipz_qeit_get_inc_valid(&my_cq->ipz_queue);
if (!cqe) {
@@ -617,7 +617,7 @@ poll_cq_one_read_cqe:
ehca_dmp(cqe, 64, "cq_num=%x qp_num=%x",
my_cq->cq_number, cqe->local_qp_number);
/* ignore this purged cqe */
- goto poll_cq_one_read_cqe;
+ goto repoll;
}
spin_lock_irqsave(&qp->spinlock_s, flags);
purgeflag = qp->sqerr_purgeflag;
@@ -636,7 +636,7 @@ poll_cq_one_read_cqe:
* that caused sqe and turn off purge flag
*/
qp->sqerr_purgeflag = 0;
- goto poll_cq_one_read_cqe;
+ goto repoll;
}
}
___
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 5/5 try2] ib/ehca: discard double CQE for one WR
Under rare circumstances, the ehca hardware might erroneously generate
two CQEs for the same WQE, which is not compliant to the IB spec and will
cause unpredictable errors like memory being freed twice. To avoid this
problem, the driver needs to detect the second CQE and discard it.
For this purpose, introduce an array holding as many elements as the SQ of
the QP, called sq_map. Each sq_map entry stores a "reported" flag for one
WQE in the SQ. When a work request is posted to the SQ, the respective
"reported" flag is set to zero. After the arrival of a CQE, the flag is set
to 1, which allows to detect the occurence of a second CQE.
The mapping between WQE / CQE and the corresponding sq_map element is
implemented by replacing the lowest 16 Bits of the wr_id with the index in
the queue map. The original 16 Bits are stored in the sq_map entry and are
restored when the CQE is passed to the application.
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/ehca/ehca_classes.h |9 +
drivers/infiniband/hw/ehca/ehca_qes.h |1
drivers/infiniband/hw/ehca/ehca_qp.c | 34 +-
drivers/infiniband/hw/ehca/ehca_reqs.c| 54 +++---
4 files changed, 78 insertions(+), 20 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -139,6 +139,7 @@ static void trace_send_wr_ud(const struc
static inline int ehca_write_swqe(struct ehca_qp *qp,
struct ehca_wqe *wqe_p,
const struct ib_send_wr *send_wr,
+ u32 sq_map_idx,
int hidden)
{
u32 idx;
@@ -157,7 +158,11 @@ static inline int ehca_write_swqe(struct
/* clear wqe header until sglist */
memset(wqe_p, 0, offsetof(struct ehca_wqe, u.ud_av.sg_list));
- wqe_p->work_request_id = send_wr->wr_id;
+ wqe_p->work_request_id = send_wr->wr_id & ~QMAP_IDX_MASK;
+ wqe_p->work_request_id |= sq_map_idx & QMAP_IDX_MASK;
+
+ qp->sq_map[sq_map_idx].app_wr_id = send_wr->wr_id & QMAP_IDX_MASK;
+ qp->sq_map[sq_map_idx].reported = 0;
switch (send_wr->opcode) {
case IB_WR_SEND:
@@ -381,6 +386,7 @@ static inline int post_one_send(struct e
{
struct ehca_wqe *wqe_p;
int ret;
+ u32 sq_map_idx;
u64 start_offset = my_qp->ipz_squeue.current_q_offset;
/* get pointer next to free WQE */
@@ -393,8 +399,15 @@ static inline int post_one_send(struct e
"qp_num=%x", my_qp->ib_qp.qp_num);
return -ENOMEM;
}
+
+ /*
+* Get the index of the WQE in the send queue. The same index is used
+* for writing into the sq_map.
+*/
+ sq_map_idx = start_offset / my_qp->ipz_squeue.qe_size;
+
/* write a SEND WQE into the QUEUE */
- ret = ehca_write_swqe(my_qp, wqe_p, cur_send_wr, hidden);
+ ret = ehca_write_swqe(my_qp, wqe_p, cur_send_wr, sq_map_idx, hidden);
/*
* if something failed,
* reset the free entry pointer to the start value
@@ -654,8 +667,34 @@ repoll:
my_cq, my_cq->cq_number);
}
- /* we got a completion! */
- wc->wr_id = cqe->work_request_id;
+ read_lock(&ehca_qp_idr_lock);
+ my_qp = idr_find(&ehca_qp_idr, cqe->qp_token);
+ read_unlock(&ehca_qp_idr_lock);
+ if (!my_qp)
+ goto repoll;
+ wc->qp = &my_qp->ib_qp;
+
+ if (!(cqe->w_completion_flags & WC_SEND_RECEIVE_BIT)) {
+ struct ehca_qmap_entry *qmap_entry;
+ /*
+* We got a send completion and need to restore the original
+* wr_id.
+*/
+ qmap_entry = &my_qp->sq_map[cqe->work_request_id &
+ QMAP_IDX_MASK];
+
+ if (qmap_entry->reported) {
+ ehca_warn(cq->device, "Double cqe on qp_num=%#x",
+ my_qp->real_qp_num);
+ /* found a double cqe, discard it and read next one */
+ goto repoll;
+ }
+ wc->wr_id = cqe->work_request_id & ~QMAP_IDX_MASK;
+ wc->wr_id |= qmap_entry->app_wr_id;
+ qmap_entry->reported = 1;
+ } else
+ /* We got a receive completion. */
+ wc->wr_id = cqe->work_request_id;
/* eval ib_wc_opcode */
wc->opcode = ib_wc_opcode[cqe->optype]-1;
@@ -678,13 +717,6 @@ repoll:
} else
wc->status = IB_WC_SUCCESS;
- read_lock(&ehca_qp_idr_lock);
-
[PATCH 3/5 try2] ib/ehca: repoll on invalid opcode
When the ehca driver detects an invalid opcode in a CQE, it currently passes the CQE to the application and returns with success. This patch changes the CQE handling to discard CQEs with invalid opcodes and to continue reading the next CQE from the CQ. Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]> --- drivers/infiniband/hw/ehca/ehca_reqs.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c +++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c @@ -667,7 +667,7 @@ repoll: ehca_dmp(cqe, 64, "ehca_cq=%p cq_num=%x", my_cq, my_cq->cq_number); /* update also queue adder to throw away this entry!!! */ - goto poll_cq_one_exit0; + goto repoll; } /* eval ib_wc_status */ ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 4/5 try2] ib/ehca: check idr_find() return value
The idr_find() function may fail when trying to get the QP that is associated with a CQE, e.g. when a QP has been destroyed between the generation of a CQE and the poll request for it. In consequence, the return value of idr_find() must be checked and the CQE must be discarded when the QP cannot be found. Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]> --- drivers/infiniband/hw/ehca/ehca_reqs.c |4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) --- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c +++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c @@ -680,8 +680,10 @@ repoll: read_lock(&ehca_qp_idr_lock); my_qp = idr_find(&ehca_qp_idr, cqe->qp_token); - wc->qp = &my_qp->ib_qp; read_unlock(&ehca_qp_idr_lock); + if (!my_qp) + goto repoll; + wc->qp = &my_qp->ib_qp; wc->byte_len = cqe->nr_bytes_transferred; wc->pkey_index = cqe->pkey_index; ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 0/5 try2] ib/ehca: Fix stability issues
Hi Roland, Sorry, the first set was mangled because of a broken mailer, so here it is again, double checked... the following patchset contains four small fixes and one bigger patch (5/5) for addressing some ehca issues we found during cluster test. [1/5] update qp_state on cached modify_qp() [2/5] rename goto label in ehca_poll_cq_one() [3/5] repoll on invalid opcode instead of returning success [4/5] check idr_find() return value [5/5] discard double CQE for one WR They all apply on top of 2.6.27-rc1. If possible, we would like to get them into 2.6.27. Regards, Alexander Schmidt ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 1/5 try2] ib/ehca: update qp_state on cached modify_qp()
Since the introduction of the port auto-detect mode for ehca, calls to
modify_qp() may be cached in the device driver when the ports are not
activated yet. When a modify_qp() call is cached, the qp state remains
untouched until the port is activated, which will leave the qp in the reset
state. In the reset state, however, it is not allowed to post SQ WQEs,
which confuses applications like ib_mad.
The solution for this problem is to immediately set the qp state as requested
by modify_qp(), even when the call is cached.
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/ehca/ehca_qp.c | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_qp.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -1534,8 +1534,6 @@ static int internal_modify_qp(struct ib_
if (attr_mask & IB_QP_QKEY)
my_qp->qkey = attr->qkey;
- my_qp->state = qp_new_state;
-
modify_qp_exit2:
if (squeue_locked) { /* this means: sqe -> rts */
spin_unlock_irqrestore(&my_qp->spinlock_s, flags);
@@ -1551,6 +1549,8 @@ modify_qp_exit1:
int ehca_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int attr_mask,
struct ib_udata *udata)
{
+ int ret = 0;
+
struct ehca_shca *shca = container_of(ibqp->device, struct ehca_shca,
ib_device);
struct ehca_qp *my_qp = container_of(ibqp, struct ehca_qp, ib_qp);
@@ -1597,12 +1597,18 @@ int ehca_modify_qp(struct ib_qp *ibqp, s
attr->qp_state, my_qp->init_attr.port_num,
ibqp->qp_type);
spin_unlock_irqrestore(&sport->mod_sqp_lock, flags);
- return 0;
+ goto out;
}
spin_unlock_irqrestore(&sport->mod_sqp_lock, flags);
}
- return internal_modify_qp(ibqp, attr, attr_mask, 0);
+ ret = internal_modify_qp(ibqp, attr, attr_mask, 0);
+
+out:
+ if ((ret == 0) && (attr_mask & IB_QP_STATE))
+ my_qp->state = attr->qp_state;
+
+ return ret;
}
void ehca_recover_sqp(struct ib_qp *sqp)
___
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 4/5] ib/ehca: check idr_find() return value
The idr_find() function may fail when trying to get the QP that is associated with a CQE, e.g. when a QP has been destroyed between the generation of a CQE and the poll request for it. In consequence, the return value of idr_find() must be checked and the CQE must be discarded when the QP cannot be found. Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]> --- drivers/infiniband/hw/ehca/ehca_reqs.c |4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) --- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c +++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c @@ -680,8 +680,10 @@ repoll: read_lock(&ehca_qp_idr_lock); my_qp = idr_find(&ehca_qp_idr, cqe->qp_token); - wc->qp = &my_qp->ib_qp; read_unlock(&ehca_qp_idr_lock); + if (!my_qp) + goto repoll; + wc->qp = &my_qp->ib_qp; wc->byte_len = cqe->nr_bytes_transferred; wc->pkey_index = cqe->pkey_index; ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 3/5] ib/ehca: repoll on invalid opcode
When the ehca driver detects an invalid opcode in a CQE, it currently passes the CQE to the application and returns with success. This patch changes the CQE handling to discard CQEs with invalid opcodes and to continue reading the next CQE from the CQ. Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]> --- drivers/infiniband/hw/ehca/ehca_reqs.c |2 +- 1 file changed, 1 insertion(+), 1 deletion(-) --- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c +++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c @@ -667,7 +667,7 @@ repoll: ehca_dmp(cqe, 64, "ehca_cq=%p cq_num=%x", my_cq, my_cq->cq_number); /* update also queue adder to throw away this entry!!! */ - goto poll_cq_one_exit0; + goto repoll; } /* eval ib_wc_status */ ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 5/5] ib/ehca: discard double CQE for one WR
Under rare circumstances, the ehca hardware might erroneously generate
two CQEs for the same WQE, which is not compliant to the IB spec and
will cause unpredictable errors like memory being freed twice. To avoid
this problem, the driver needs to detect the second CQE and discard it.
For this purpose, introduce an array holding as many elements as the SQ
of the QP, called sq_map. Each sq_map entry stores a "reported" flag
for one WQE in the SQ. When a work request is posted to the SQ, the
respective "reported" flag is set to zero. After the arrival of a CQE,
the flag is set to 1, which allows to detect the occurence of a second
CQE.
The mapping between WQE / CQE and the corresponding sq_map element is
implemented by replacing the lowest 16 Bits of the wr_id with the index
in the queue map. The original 16 Bits are stored in the sq_map entry
and are restored when the CQE is passed to the application.
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/ehca/ehca_classes.h |9 +
drivers/infiniband/hw/ehca/ehca_qes.h |1
drivers/infiniband/hw/ehca/ehca_qp.c | 34 +-
drivers/infiniband/hw/ehca/ehca_reqs.c| 54
+++--- 4 files changed, 78 insertions(+), 20
deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -139,6 +139,7 @@ static void trace_send_wr_ud(const struc
static inline int ehca_write_swqe(struct ehca_qp *qp,
struct ehca_wqe *wqe_p,
const struct ib_send_wr *send_wr,
+ u32 sq_map_idx,
int hidden)
{
u32 idx;
@@ -157,7 +158,11 @@ static inline int ehca_write_swqe(struct
/* clear wqe header until sglist */
memset(wqe_p, 0, offsetof(struct ehca_wqe, u.ud_av.sg_list));
- wqe_p->work_request_id = send_wr->wr_id;
+ wqe_p->work_request_id = send_wr->wr_id & ~QMAP_IDX_MASK;
+ wqe_p->work_request_id |= sq_map_idx & QMAP_IDX_MASK;
+
+ qp->sq_map[sq_map_idx].app_wr_id = send_wr->wr_id &
QMAP_IDX_MASK;
+ qp->sq_map[sq_map_idx].reported = 0;
switch (send_wr->opcode) {
case IB_WR_SEND:
@@ -381,6 +386,7 @@ static inline int post_one_send(struct e
{
struct ehca_wqe *wqe_p;
int ret;
+ u32 sq_map_idx;
u64 start_offset = my_qp->ipz_squeue.current_q_offset;
/* get pointer next to free WQE */
@@ -393,8 +399,15 @@ static inline int post_one_send(struct e
"qp_num=%x", my_qp->ib_qp.qp_num);
return -ENOMEM;
}
+
+ /*
+* Get the index of the WQE in the send queue. The same index
is used
+* for writing into the sq_map.
+*/
+ sq_map_idx = start_offset / my_qp->ipz_squeue.qe_size;
+
/* write a SEND WQE into the QUEUE */
- ret = ehca_write_swqe(my_qp, wqe_p, cur_send_wr, hidden);
+ ret = ehca_write_swqe(my_qp, wqe_p, cur_send_wr, sq_map_idx,
hidden); /*
* if something failed,
* reset the free entry pointer to the start value
@@ -654,8 +667,34 @@ repoll:
my_cq, my_cq->cq_number);
}
- /* we got a completion! */
- wc->wr_id = cqe->work_request_id;
+ read_lock(&ehca_qp_idr_lock);
+ my_qp = idr_find(&ehca_qp_idr, cqe->qp_token);
+ read_unlock(&ehca_qp_idr_lock);
+ if (!my_qp)
+ goto repoll;
+ wc->qp = &my_qp->ib_qp;
+
+ if (!(cqe->w_completion_flags & WC_SEND_RECEIVE_BIT)) {
+ struct ehca_qmap_entry *qmap_entry;
+ /*
+* We got a send completion and need to restore the
original
+* wr_id.
+*/
+ qmap_entry = &my_qp->sq_map[cqe->work_request_id &
+ QMAP_IDX_MASK];
+
+ if (qmap_entry->reported) {
+ ehca_warn(cq->device, "Double cqe on
qp_num=%#x",
+ my_qp->real_qp_num);
+ /* found a double cqe, discard it and read
next one */
+ goto repoll;
+ }
+ wc->wr_id = cqe->work_request_id & ~QMAP_IDX_MASK;
+ wc->wr_id |= qmap_entry->app_wr_id;
+ qmap_entry->reported = 1;
+ } else
+ /* We got a receive completion. */
+ wc->wr_id = cqe->work_request_id;
/* eval ib_wc_opcode */
wc->opcode = ib_wc_opcode[cqe->optype]-1;
@@ -678,13 +717,6 @@ repoll:
} else
wc->status = IB_WC_SUCCESS;
- read_lock(&ehca_qp_idr_lock);
- my_qp = idr_find(&
[PATCH 2/5] ib/ehca: rename goto label
Rename the "poll_cq_one_read_cqe" goto label to what it actually does,
"repoll".
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/ehca/ehca_reqs.c |6 +++---
1 file changed, 3 insertions(+), 3 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_reqs.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_reqs.c
@@ -589,7 +589,7 @@ static inline int ehca_poll_cq_one(struc
struct ehca_qp *my_qp;
int cqe_count = 0, is_error;
-poll_cq_one_read_cqe:
+repoll:
cqe = (struct ehca_cqe *)
ipz_qeit_get_inc_valid(&my_cq->ipz_queue);
if (!cqe) {
@@ -617,7 +617,7 @@ poll_cq_one_read_cqe:
ehca_dmp(cqe, 64, "cq_num=%x qp_num=%x",
my_cq->cq_number,
cqe->local_qp_number); /* ignore this purged cqe */
- goto poll_cq_one_read_cqe;
+ goto repoll;
}
spin_lock_irqsave(&qp->spinlock_s, flags);
purgeflag = qp->sqerr_purgeflag;
@@ -636,7 +636,7 @@ poll_cq_one_read_cqe:
* that caused sqe and turn off purge flag
*/
qp->sqerr_purgeflag = 0;
- goto poll_cq_one_read_cqe;
+ goto repoll;
}
}
___
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 1/5] ib/ehca: update qp_state on cached modify_qp()
Since the introduction of the port auto-detect mode for ehca, calls to
modify_qp() may be cached in the device driver when the ports are not
activated yet. When a modify_qp() call is cached, the qp state remains
untouched until the port is activated, which will leave the qp in the
reset state. In the reset state, however, it is not allowed to post SQ
WQEs, which confuses applications like ib_mad.
The solution for this problem is to immediately set the qp state as
requested by modify_qp(), even when the call is cached.
Signed-off-by: Alexander Schmidt <[EMAIL PROTECTED]>
---
drivers/infiniband/hw/ehca/ehca_qp.c | 14 ++
1 file changed, 10 insertions(+), 4 deletions(-)
--- infiniband.git.orig/drivers/infiniband/hw/ehca/ehca_qp.c
+++ infiniband.git/drivers/infiniband/hw/ehca/ehca_qp.c
@@ -1534,8 +1534,6 @@ static int internal_modify_qp(struct ib_
if (attr_mask & IB_QP_QKEY)
my_qp->qkey = attr->qkey;
- my_qp->state = qp_new_state;
-
modify_qp_exit2:
if (squeue_locked) { /* this means: sqe -> rts */
spin_unlock_irqrestore(&my_qp->spinlock_s, flags);
@@ -1551,6 +1549,8 @@ modify_qp_exit1:
int ehca_modify_qp(struct ib_qp *ibqp, struct ib_qp_attr *attr, int
attr_mask, struct ib_udata *udata)
{
+ int ret = 0;
+
struct ehca_shca *shca = container_of(ibqp->device, struct
ehca_shca, ib_device);
struct ehca_qp *my_qp = container_of(ibqp, struct ehca_qp,
ib_qp); @@ -1597,12 +1597,18 @@ int ehca_modify_qp(struct ib_qp *ibqp, s
attr->qp_state,
my_qp->init_attr.port_num, ibqp->qp_type);
spin_unlock_irqrestore(&sport->mod_sqp_lock,
flags);
- return 0;
+ goto out;
}
spin_unlock_irqrestore(&sport->mod_sqp_lock, flags);
}
- return internal_modify_qp(ibqp, attr, attr_mask, 0);
+ ret = internal_modify_qp(ibqp, attr, attr_mask, 0);
+
+out:
+ if ((ret == 0) && (attr_mask & IB_QP_STATE))
+ my_qp->state = attr->qp_state;
+
+ return ret;
}
void ehca_recover_sqp(struct ib_qp *sqp)
___
Linuxppc-dev mailing list
Linuxppc-dev@ozlabs.org
https://ozlabs.org/mailman/listinfo/linuxppc-dev
[PATCH 0/5] ib/ehca: Fix stability issues
Hi Roland, the following patchset contains four small fixes and one bigger patch (5/5) for addressing some ehca issues we found during cluster test. [1/5] update qp_state on cached modify_qp() [2/5] rename goto label in ehca_poll_cq_one() [3/5] repoll on invalid opcode instead of returning success [4/5] check idr_find() return value [5/5] discard double CQE for one WR They all apply on top of 2.6.27-rc1. If possible, we would like to get them into 2.6.27. Regards, Alexander Schmidt ___ Linuxppc-dev mailing list Linuxppc-dev@ozlabs.org https://ozlabs.org/mailman/listinfo/linuxppc-dev
