Re: [PATCH v3] cxlflash: Base support for IBM CXL Flash Adapter
Brian: Thanks for your review. Responses are inline below. - Manoj Kumar On 6/4/2015 9:38 AM, Brian King wrote: + + write_lock(cfg-tmf_lock); What is this lock protecting? The only thing it seems to be accomplishing is making sure one thread isn't sending a TMF and another thread is sending a normal I/O command at the exact same time, yet it looks like you still allow a TMF to be sent and a normal I/O to be sent immediately after, before receiving the TMF response. Originally this section was waiting for the TMF response. I see that is no longer the case. Will restore the original behavior, with adequate locking. + afu-room = readq_be(afu-host_map-cmd_room); Looks like you now have an MMIO load as part of sending every command, including commands coming from queuecommand. Won't that be a performance issue? Is there any way to avoid this? Could you perhaps decrement afu-room in this function and only re-read it from the AFU when the counter hits zero? Good point. Will revise to avoid MMIO in this performance path. -- To unsubscribe from this list: send the line unsubscribe linux-scsi in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v3] cxlflash: Base support for IBM CXL Flash Adapter
On 06/02/2015 06:55 PM, Matthew R. Ochs wrote:
+/**
+ * send_tmf() - sends a Task Management Function (TMF)
+ * @afu: AFU to checkout from.
+ * @scp: SCSI command from stack.
+ * @tmfcmd: TMF command to send.
+ *
+ * Return:
+ * 0 on success
+ * SCSI_MLQUEUE_HOST_BUSY when host is busy
+ */
+static int send_tmf(struct afu *afu, struct scsi_cmnd *scp, u64 tmfcmd)
+{
+ struct afu_cmd *cmd;
+
+ u32 port_sel = scp-device-channel + 1;
+ short lflag = 0;
+ struct Scsi_Host *host = scp-device-host;
+ struct cxlflash_cfg *cfg = (struct cxlflash_cfg *)host-hostdata;
+ int rc = 0;
+
+ write_lock(cfg-tmf_lock);
What is this lock protecting? The only thing it seems to be accomplishing is
making sure one thread isn't sending a TMF and another thread is sending
a normal I/O command at the exact same time, yet it looks like you still
allow a TMF to be sent and a normal I/O to be sent immediately after, before
receiving the TMF response.
+
+ cmd = cxlflash_cmd_checkout(afu);
+ if (unlikely(!cmd)) {
+ pr_err(%s: could not get a free command\n, __func__);
+ rc = SCSI_MLQUEUE_HOST_BUSY;
+ goto out;
+ }
+
+ cmd-rcb.ctx_id = afu-ctx_hndl;
+ cmd-rcb.port_sel = port_sel;
+ cmd-rcb.lun_id = lun_to_lunid(scp-device-lun);
+
+ lflag = SISL_REQ_FLAGS_TMF_CMD;
+
+ cmd-rcb.req_flags = (SISL_REQ_FLAGS_PORT_LUN_ID |
+ SISL_REQ_FLAGS_SUP_UNDERRUN | lflag);
+
+ /* Stash the scp in the reserved field, for reuse during interrupt */
+ cmd-rcb.scp = scp;
+
+ /* Copy the CDB from the cmd passed in */
+ memcpy(cmd-rcb.cdb, tmfcmd, sizeof(tmfcmd));
+
+ /* Send the command */
+ rc = cxlflash_send_cmd(afu, cmd);
+out:
+ write_unlock(cfg-tmf_lock);
+ return rc;
+
+}
+
+/**
+ * cxlflash_send_cmd() - sends an AFU command
+ * @afu: AFU associated with the host.
+ * @cmd: AFU command to send.
+ *
+ * Return:
+ * 0 on success
+ * -1 on failure
+ */
+int cxlflash_send_cmd(struct afu *afu, struct afu_cmd *cmd)
+{
+ int nretry = 0;
+ int rc = 0;
+
+ /* send_cmd is used by critical users such an AFU sync and to
+ * send a task management function (TMF). So we do want to retry
+ * a bit before returning an error.
+ */
+ do {
+ afu-room = readq_be(afu-host_map-cmd_room);
Looks like you now have an MMIO load as part of sending every command,
including commands coming from queuecommand. Won't that be a performance issue?
Is there any way to avoid this? Could you perhaps decrement afu-room
in this function and only re-read it from the AFU when the counter hits zero?
+ if (afu-room)
+ break;
+ udelay(nretry);
+ } while (nretry++ MC_ROOM_RETRY_CNT);
+
+ /* Write IOARRIN */
+ if (afu-room)
+ writeq_be((u64)cmd-rcb, afu-host_map-ioarrin);
+ else {
+ pr_err(%s: no cmd_room to send 0x%X\n,
+__func__, cmd-rcb.cdb[0]);
+ rc = SCSI_MLQUEUE_HOST_BUSY;
+ }
+
+ pr_debug(%s: cmd=%p len=%d ea=%p rc=%d\n, __func__, cmd,
+ cmd-rcb.data_len, (void *)cmd-rcb.data_ea, rc);
+
+ return rc;
+}
+
--
Brian King
Power Linux I/O
IBM Linux Technology Center
--
To unsubscribe from this list: send the line unsubscribe linux-scsi in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH v3] cxlflash: Base support for IBM CXL Flash Adapter
SCSI device driver to support filesystem access on the IBM CXL Flash adapter.
Signed-off-by: Matthew R. Ochs mro...@linux.vnet.ibm.com
Signed-off-by: Manoj N. Kumar ma...@linux.vnet.ibm.com
---
drivers/scsi/Kconfig|1 +
drivers/scsi/Makefile |1 +
drivers/scsi/cxlflash/Kconfig | 11 +
drivers/scsi/cxlflash/Makefile |2 +
drivers/scsi/cxlflash/common.h | 177
drivers/scsi/cxlflash/main.c| 2215 +++
drivers/scsi/cxlflash/main.h| 109 ++
drivers/scsi/cxlflash/sislite.h | 465
8 files changed, 2981 insertions(+)
create mode 100644 drivers/scsi/cxlflash/Kconfig
create mode 100644 drivers/scsi/cxlflash/Makefile
create mode 100644 drivers/scsi/cxlflash/common.h
create mode 100644 drivers/scsi/cxlflash/main.c
create mode 100644 drivers/scsi/cxlflash/main.h
create mode 100755 drivers/scsi/cxlflash/sislite.h
diff --git a/drivers/scsi/Kconfig b/drivers/scsi/Kconfig
index b021bcb..ebb12a7 100644
--- a/drivers/scsi/Kconfig
+++ b/drivers/scsi/Kconfig
@@ -345,6 +345,7 @@ source drivers/scsi/cxgbi/Kconfig
source drivers/scsi/bnx2i/Kconfig
source drivers/scsi/bnx2fc/Kconfig
source drivers/scsi/be2iscsi/Kconfig
+source drivers/scsi/cxlflash/Kconfig
config SGIWD93_SCSI
tristate SGI WD93C93 SCSI Driver
diff --git a/drivers/scsi/Makefile b/drivers/scsi/Makefile
index dee160a..619f8fb 100644
--- a/drivers/scsi/Makefile
+++ b/drivers/scsi/Makefile
@@ -101,6 +101,7 @@ obj-$(CONFIG_SCSI_7000FASST)+= wd7000.o
obj-$(CONFIG_SCSI_EATA)+= eata.o
obj-$(CONFIG_SCSI_DC395x) += dc395x.o
obj-$(CONFIG_SCSI_AM53C974)+= esp_scsi.o am53c974.o
+obj-$(CONFIG_CXLFLASH) += cxlflash/
obj-$(CONFIG_MEGARAID_LEGACY) += megaraid.o
obj-$(CONFIG_MEGARAID_NEWGEN) += megaraid/
obj-$(CONFIG_MEGARAID_SAS) += megaraid/
diff --git a/drivers/scsi/cxlflash/Kconfig b/drivers/scsi/cxlflash/Kconfig
new file mode 100644
index 000..e98c3f6
--- /dev/null
+++ b/drivers/scsi/cxlflash/Kconfig
@@ -0,0 +1,11 @@
+#
+# IBM CXL-attached Flash Accelerator SCSI Driver
+#
+
+config CXLFLASH
+ tristate Support for IBM CAPI Flash
+ depends on CXL
+ default m
+ help
+ Allows CAPI Accelerated IO to Flash
+ If unsure, say N.
diff --git a/drivers/scsi/cxlflash/Makefile b/drivers/scsi/cxlflash/Makefile
new file mode 100644
index 000..dc95e20
--- /dev/null
+++ b/drivers/scsi/cxlflash/Makefile
@@ -0,0 +1,2 @@
+obj-$(CONFIG_CXLFLASH) += cxlflash.o
+cxlflash-y += main.o
diff --git a/drivers/scsi/cxlflash/common.h b/drivers/scsi/cxlflash/common.h
new file mode 100644
index 000..1265558
--- /dev/null
+++ b/drivers/scsi/cxlflash/common.h
@@ -0,0 +1,177 @@
+/*
+ * CXL Flash Device Driver
+ *
+ * Written by: Manoj N. Kumar ma...@linux.vnet.ibm.com, IBM Corporation
+ * Matthew R. Ochs mro...@linux.vnet.ibm.com, IBM Corporation
+ *
+ * Copyright (C) 2015 IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _CXLFLASH_COMMON_H
+#define _CXLFLASH_COMMON_H
+
+#include linux/list.h
+#include linux/types.h
+#include scsi/scsi.h
+#include scsi/scsi_device.h
+
+
+#define MAX_CONTEXT CXLFLASH_MAX_CONTEXT /* num contexts per afu */
+
+#define CXLFLASH_BLOCK_SIZE4096/* 4K blocks */
+#define CXLFLASH_MAX_XFER_SIZE 16777216/* 16MB transfer */
+#define CXLFLASH_MAX_SECTORS (CXLFLASH_MAX_XFER_SIZE/512)/* SCSI wants
+ max_sectors
+ in units of
+ 512 byte
+ sectors
+ */
+
+#define NUM_RRQ_ENTRY16 /* for master issued cmds */
+#define MAX_RHT_PER_CONTEXT (PAGE_SIZE / sizeof(struct sisl_rht_entry))
+
+/* AFU command retry limit */
+#define MC_RETRY_CNT 5 /* sufficient for SCSI check and
+ certain AFU errors */
+
+/* Command management definitions */
+#define CXLFLASH_NUM_CMDS (2 * CXLFLASH_MAX_CMDS) /* Must be a pow2 for
+ alignment and more
+ efficient array
+ index derivation
+*/
+
+#define CXLFLASH_MAX_CMDS 16
+#define CXLFLASH_MAX_CMDS_PER_LUN CXLFLASH_MAX_CMDS
+
+
+static inline void check_sizes(void)
+{
+ BUILD_BUG_ON_NOT_POWER_OF_2(CXLFLASH_NUM_CMDS);
+}
+
+/* AFU defines a fixed size of 4K
