Re: [PATCH] infiniband-diags: add rdma-ndd daemon

2014-10-30 Thread Bart Van Assche
On 10/31/14 00:54, ira.we...@intel.com wrote: +int set_rdma_device_names(const char *hostname) +{ + DIR *class_dir; + struct dirent *dent; + + class_dir = opendir(SYS_INFINIBAND); + if (!class_dir) { + syslog(LOG_INFO, "Failed to open %s", SYS_INFINIBAND); +

[PATCH 0/1] infiniband-diags: add rdma-ndd daemon

2014-10-30 Thread ira . weiny
This is the follow on patch to the kernel series which changes the default Node Description to be " " This patch adds support for a "RDMA Node Description Daemon" (rdma-ndd) which uses the new poll support of the procfs hostname file to monitor for changes to the hostname and updates the Node Desc

[PATCH] infiniband-diags: add rdma-ndd daemon

2014-10-30 Thread ira . weiny
From: Ira Weiny rdma-ndd is a system daemon which watches the procfs hostname file for updates. Upon detecting an update it will update the Node Descriptions of the RDMA devices in the system. This deamon is intended to work with kernels which support polling of the procfs hostname file. If yo

[PATCH 3/6] ib/mlx4: change default node description processing

2014-10-30 Thread ira . weiny
From: Ira Weiny Change the default node description string to IB_DEFAULT_ND_FORMAT Use ib_build_node_desc to dynamically map this string when queried and to set the Node Description in FW Reviewed-by: John Fleck Reviewed-by: Michael Heinz Reviewed-by: Mike Marciniszyn Signed-off-by: Ira Weiny

[PATCH 4/6] ib/mthca: change default node description processing

2014-10-30 Thread ira . weiny
From: Ira Weiny Change the default node description string to IB_DEFAULT_ND_FORMAT Use ib_build_node_desc to dynamically map this string when queried Reviewed-by: John Fleck Reviewed-by: Michael Heinz Reviewed-by: Mike Marciniszyn Signed-off-by: Ira Weiny --- drivers/infiniband/hw/mthca/mt

[PATCH 6/6] ib/mlx5: change default node description processing

2014-10-30 Thread ira . weiny
From: Ira Weiny Change the default node description string to IB_DEFAULT_ND_FORMAT Use ib_build_node_desc to dynamically map this string to set Node Description in FW Reviewed-by: John Fleck Reviewed-by: Michael Heinz Reviewed-by: Mike Marciniszyn Signed-off-by: Ira Weiny --- drivers/infin

[PATCH 5/6] ib/qib: change default node description processing

2014-10-30 Thread ira . weiny
From: Ira Weiny Change the default node description string to IB_DEFAULT_ND_FORMAT Use ib_build_node_desc to dynamically map this string when queried Reviewed-by: John Fleck Reviewed-by: Michael Heinz Reviewed-by: Mike Marciniszyn Signed-off-by: Ira Weiny --- drivers/infiniband/hw/qib/qib_

[PATCH 1/6] ib/mad: Add function to support format specifiers for node description

2014-10-30 Thread ira . weiny
From: Ira Weiny ib_build_node_desc - prints src node description into dest while mapping format specifiers Specifiers supported: %h system hostname %d device name Define a default Node Description format to be "%h %d" Original work done by Mike Heinz. The function signature is generic to supp

[PATCH 2/6] ib/ipath: change default node description processing

2014-10-30 Thread ira . weiny
From: Ira Weiny Change the default node description string to IB_DEFAULT_ND_FORMAT Use ib_build_node_desc to dynamically map this string when queried Reviewed-by: John Fleck Reviewed-by: Michael Heinz Reviewed-by: Mike Marciniszyn Signed-off-by: Ira Weiny --- drivers/infiniband/hw/ipath/ip

[PATCH 0/6] Make Node Description default to " "

2014-10-30 Thread ira . weiny
The following series defines a new mad core function to support Node Description processing which supports format specifiers. The first 2 format specifiers are "%h" and "%d" for hostname and device respectively. It also changes the default Node Description of all the devices which support Node De

Re: [PATCH v2 12/12] IB/srp: Add multichannel support

2014-10-30 Thread Sagi Grimberg
On 10/30/2014 5:19 PM, Bart Van Assche wrote: On 10/30/14 16:06, Sagi Grimberg wrote: I'm not aware of any implicit interrupt coalescing effect... In case it was not clear what I was referring to: if multiple completion queue handling routines run on the same CPU then the average number of wor

Re: [PATCH v2 12/12] IB/srp: Add multichannel support

2014-10-30 Thread Bart Van Assche
On 10/30/14 16:06, Sagi Grimberg wrote: I'm not aware of any implicit interrupt coalescing effect... In case it was not clear what I was referring to: if multiple completion queue handling routines run on the same CPU then the average number of work completions processed by each completion ha

Re: [PATCH v3 11/11] IB/srp: Fix a race condition triggered by destroying a queue pair

2014-10-30 Thread Sagi Grimberg
On 10/30/2014 4:53 PM, Bart Van Assche wrote: On 10/30/14 15:26, Sagi Grimberg wrote: On 10/30/2014 3:50 PM, Bart Van Assche wrote: +/* Destroying a QP and reusing ch->done is only safe if not connected */ +WARN_ON_ONCE(target->connected); I thought we agreed that cannot happen. I gue

RE: Can someone help me understand the reason for this code in ib_isert.c?

2014-10-30 Thread Chris Moore
> On 10/29/2014 8:05 PM, Chris Moore wrote: > >> On Wed, 2014-10-22 at 12:02 +0300, Or Gerlitz wrote: > >>> On 10/22/2014 8:06 AM, Nicholas A. Bellinger wrote: > On Tue, 2014-10-21 at 00:13 +0300, Or Gerlitz wrote: > > On Mon, Oct 20, 2014 at 6:29 PM, Chris Moore > >> wrote: > >> The

Re: [PATCH v2 12/12] IB/srp: Add multichannel support

2014-10-30 Thread Sagi Grimberg
On 10/30/2014 4:36 PM, Bart Van Assche wrote: On 10/30/14 15:19, Sagi Grimberg wrote: My impression here that in the default settings, on a 1 NUMA node with 8 cores, 2 different srp connections (using 4 channels each) will be associated with comp vectors 0-3. while it could potentially use vecto

Re: [PATCH v3 11/11] IB/srp: Fix a race condition triggered by destroying a queue pair

2014-10-30 Thread Bart Van Assche
On 10/30/14 15:26, Sagi Grimberg wrote: On 10/30/2014 3:50 PM, Bart Van Assche wrote: +/* Destroying a QP and reusing ch->done is only safe if not connected */ +WARN_ON_ONCE(target->connected); I thought we agreed that cannot happen. I guess I don't mind keeping it... BTW, were you abl

Re: [PATCH v2 12/12] IB/srp: Add multichannel support

2014-10-30 Thread Bart Van Assche
On 10/30/14 15:19, Sagi Grimberg wrote: My impression here that in the default settings, on a 1 NUMA node with 8 cores, 2 different srp connections (using 4 channels each) will be associated with comp vectors 0-3. while it could potentially use vectors 4-7 and reduce possible mutual interference.

Re: [PATCH v3 09/11] IB/srp: Use block layer tags

2014-10-30 Thread Sagi Grimberg
On 10/30/2014 3:48 PM, Bart Van Assche wrote: Since the block layer already contains functionality to assign a tag to each request, use that functionality instead of reimplementing that functionality in the SRP initiator driver. This change makes the free_reqs list superfluous. Hence remove that

Re: [PATCH v3 06/11] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning

2014-10-30 Thread Sagi Grimberg
On 10/30/2014 3:47 PM, Bart Van Assche wrote: If a cable is pulled during LUN scanning it can happen that the SRP rport and the SCSI host have been created but no LUNs have been added to the SCSI host. Since multipathd only sends SCSI commands to a SCSI target if one or more SCSI devices are pres

Re: [PATCH v3 11/11] IB/srp: Fix a race condition triggered by destroying a queue pair

2014-10-30 Thread Sagi Grimberg
On 10/30/2014 3:50 PM, Bart Van Assche wrote: At least LID reassignment can trigger a race condition in the SRP initiator driver, namely the receive completion handler trying to post a request on a QP during or after QP destruction and before the CQ's have been destroyed. Avoid this race by modif

Re: [PATCH v2 12/12] IB/srp: Add multichannel support

2014-10-30 Thread Sagi Grimberg
On 10/29/2014 2:36 PM, Bart Van Assche wrote: On 10/21/14 11:14, Sagi Grimberg wrote: On 10/7/2014 4:07 PM, Bart Van Assche wrote: spin_lock_irqsave(&ch->lock, flags); ch->req_lim += be32_to_cpu(rsp->req_lim_delta); @@ -1906,7 +1970,7 @@ static int srp_queuecommand(s

Re: [PATCH v2 12/12] IB/srp: Add multichannel support

2014-10-30 Thread Sagi Grimberg
On 10/29/2014 12:52 PM, Bart Van Assche wrote: On 10/28/14 19:32, Sagi Grimberg wrote: On 10/21/2014 12:10 PM, Sagi Grimberg wrote: On 10/20/2014 3:56 PM, Bart Van Assche wrote: On 10/19/14 19:36, Sagi Grimberg wrote: On 10/7/2014 4:07 PM, Bart Van Assche wrote: * comp_vector, a nu

[PATCH v3 11/11] IB/srp: Fix a race condition triggered by destroying a queue pair

2014-10-30 Thread Bart Van Assche
At least LID reassignment can trigger a race condition in the SRP initiator driver, namely the receive completion handler trying to post a request on a QP during or after QP destruction and before the CQ's have been destroyed. Avoid this race by modifying a QP into the error state and by waiting un

[PATCH v3 10/11] IB/srp: Add multichannel support

2014-10-30 Thread Bart Van Assche
Improve performance by using multiple RDMA/RC channels per SCSI host for communication with an SRP target. About the implementation: - Introduce a loop over all channels in the code that uses target->ch. - Set the SRP_MULTICHAN_MULTI flag during login for the creation of the second and subseque

[PATCH v3 09/11] IB/srp: Use block layer tags

2014-10-30 Thread Bart Van Assche
Since the block layer already contains functionality to assign a tag to each request, use that functionality instead of reimplementing that functionality in the SRP initiator driver. This change makes the free_reqs list superfluous. Hence remove that list. Signed-off-by: Bart Van Assche Cc: Sagi

[PATCH v3 08/11] IB/srp: Separate target and channel variables

2014-10-30 Thread Bart Van Assche
Changes in this patch: - Move channel variables into a new structure (struct srp_rdma_ch). - Add an srp_target_port pointer, 'lock' and 'comp_vector' members in struct srp_rdma_ch. - Add code to initialize these three new member variables. - Many boring "target->" into "ch->" changes. - The cm_id

[PATCH v3 07/11] IB/srp: Introduce two new srp_target_port member variables

2014-10-30 Thread Bart Van Assche
Introduce the srp_target_port member variables 'sgid' and 'pkey'. Change the type of 'orig_dgid' from __be16[8] into union ib_gid. This patch does not change any functionality but makes the "Separate target and channel variables" patch easier to verify. Signed-off-by: Bart Van Assche Reviewed-by:

[PATCH v3 06/11] IB/srp: Avoid that I/O hangs due to a cable pull during LUN scanning

2014-10-30 Thread Bart Van Assche
If a cable is pulled during LUN scanning it can happen that the SRP rport and the SCSI host have been created but no LUNs have been added to the SCSI host. Since multipathd only sends SCSI commands to a SCSI target if one or more SCSI devices are present and since there is no keepalive mechanism fo

[PATCH v3 05/11] IB/srp: Remove stale connection retry mechanism

2014-10-30 Thread Bart Van Assche
Attempting to connect three times may be insufficient after an initiator system tries to relogin, especially if the relogin attempt occurs before the SRP target service ID has been registered. Since the srp_daemon retries a failed login attempt anyway, remove the stale connection retry mechanism.

[PATCH v3 04/11] IB/srp: Move ib_destroy_cm_id() call into srp_free_ch_ib()

2014-10-30 Thread Bart Van Assche
The patch that adds multichannel support into the SRP initiator driver introduces an additional call to srp_free_ch_ib(). This patch helps to keep that later patch simple. Signed-off-by: Bart Van Assche Reviewed-by: Sagi Grimberg Cc: Sebastian Parschauer --- drivers/infiniband/ulp/srp/ib_srp.c

[PATCH v3 03/11] scsi_tcq.h: Add support for multiple hardware queues

2014-10-30 Thread Bart Van Assche
Modify scsi_find_tag() and scsi_host_find_tag() such that these functions can translate a tag generated by blk_mq_unique_tag(). Signed-off-by: Bart Van Assche Reviewed-by: Martin K. Petersen Cc: Christoph Hellwig Cc: Sagi Grimberg --- include/scsi/scsi_tcq.h | 19 +++ 1 file c

[PATCH v3 02/11] scsi-mq: Add support for multiple hardware queues

2014-10-30 Thread Bart Van Assche
Allow a SCSI LLD to declare how many hardware queues it supports by setting Scsi_Host.nr_hw_queues before calling scsi_add_host(). Signed-off-by: Bart Van Assche Reviewed-by: Sagi Grimberg Reviewed-by: Martin K. Petersen --- drivers/scsi/scsi_lib.c | 2 +- include/scsi/scsi_host.h | 8 +++

[PATCH v3 01/11] blk-mq: Add blk_mq_unique_tag()

2014-10-30 Thread Bart Van Assche
The queuecommand() callback functions in SCSI low-level drivers need to know which hardware context has been selected by the block layer. Since this information is not available in the request structure, and since passing the hctx pointer directly to the queuecommand callback function would require

[PATCH v3 0/11] IB/srp: Add multichannel support

2014-10-30 Thread Bart Van Assche
Although the SRP protocol supports multichannel operation, although since considerable time RDMA HCA's are available that support multiple completion vectors and although multichannel operation yields better performance than using a single channel, the Linux SRP initiator does not yet support m

Re: Can someone help me understand the reason for this code in ib_isert.c?

2014-10-30 Thread Sagi Grimberg
On 10/29/2014 8:05 PM, Chris Moore wrote: On Wed, 2014-10-22 at 12:02 +0300, Or Gerlitz wrote: On 10/22/2014 8:06 AM, Nicholas A. Bellinger wrote: On Tue, 2014-10-21 at 00:13 +0300, Or Gerlitz wrote: On Mon, Oct 20, 2014 at 6:29 PM, Chris Moore wrote: The following code is in isert_conn_set