Re: [PATCH v3 02/27] mm/memory_hotplug: Allow check_hotplug_memory_addressable to be called from drivers
On 21/2/20 2:26 pm, Alastair D'Silva wrote: From: Alastair D'Silva When setting up OpenCAPI connected persistent memory, the range check may not be performed until quite late (or perhaps not at all, if the user does not establish a DAX device). This patch makes the range check callable so we can perform the check while probing the OpenCAPI SCM device. Signed-off-by: Alastair D'Silva Reviewed-by: Andrew Donnellan -- Andrew Donnellan OzLabs, ADL Canberra a...@linux.ibm.com IBM Australia Limited ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v3 04/27] ocxl: Remove unnecessary externs
On 21/2/20 2:26 pm, Alastair D'Silva wrote: From: Alastair D'Silva Function declarations don't need externs, remove the existing ones so they are consistent with newer code Signed-off-by: Alastair D'Silva Acked-by: Andrew Donnellan -- Andrew Donnellan OzLabs, ADL Canberra a...@linux.ibm.com IBM Australia Limited ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v3 27/27] MAINTAINERS: Add myself & nvdimm/ocxl to ocxl
On 21/2/20 2:27 pm, Alastair D'Silva wrote: From: Alastair D'Silva The OpenCAPI Persistent Memory driver will be maintained as part ofi the ppc tree. I'm also adding myself as an author of the driver & contributor to the generic ocxl driver. Signed-off-by: Alastair D'Silva You need to update the title of this patch :) --- MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index f8670989ec91..3fb9a9f576a7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12064,13 +12064,16 @@ F:tools/objtool/ OCXL (Open Coherent Accelerator Processor Interface OpenCAPI) DRIVER M:Frederic Barrat M:Andrew Donnellan +M: Alastair D'Silva L:linuxppc-...@lists.ozlabs.org S:Supported F:arch/powerpc/platforms/powernv/ocxl.c +F: arch/powerpc/platforms/powernv/pmem/* F:arch/powerpc/include/asm/pnv-ocxl.h F:drivers/misc/ocxl/ F:include/misc/ocxl* F:include/uapi/misc/ocxl.h +F: include/uapi/nvdimm/ocxl-pmem.h F:Documentation/userspace-api/accelerators/ocxl.rst Should this be part of the ocxl entry or a separate entry? I guess I don't care too much either way. -- Andrew Donnellan OzLabs, ADL Canberra a...@linux.ibm.com IBM Australia Limited ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 12/27] powerpc/powernv/pmem: Add register addresses & status values to the header
From: Alastair D'Silva
These values have been taken from the device specifications.
Signed-off-by: Alastair D'Silva
---
.../platforms/powernv/pmem/ocxl_internal.h| 72 +++
1 file changed, 72 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
index 0faf3740e9b8..9cf3e42750e7 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
@@ -8,6 +8,78 @@
#define LABEL_AREA_SIZE(1UL << PA_SECTION_SHIFT)
+#define GLOBAL_MMIO_CHI0x000
+#define GLOBAL_MMIO_CHIC 0x008
+#define GLOBAL_MMIO_CHIE 0x010
+#define GLOBAL_MMIO_CHIEC 0x018
+#define GLOBAL_MMIO_HCI0x020
+#define GLOBAL_MMIO_HCIC 0x028
+#define GLOBAL_MMIO_IMA0_OHP 0x040
+#define GLOBAL_MMIO_IMA0_CFP 0x048
+#define GLOBAL_MMIO_IMA1_OHP 0x050
+#define GLOBAL_MMIO_IMA1_CFP 0x058
+#define GLOBAL_MMIO_ACMA_CREQO 0x100
+#define GLOBAL_MMIO_ACMA_CRSPO 0x104
+#define GLOBAL_MMIO_ACMA_CDBO 0x108
+#define GLOBAL_MMIO_ACMA_CDBS 0x10c
+#define GLOBAL_MMIO_NSCMA_CREQO0x120
+#define GLOBAL_MMIO_NSCMA_CRSPO0x124
+#define GLOBAL_MMIO_NSCMA_CDBO 0x128
+#define GLOBAL_MMIO_NSCMA_CDBS 0x12c
+#define GLOBAL_MMIO_CSTS 0x140
+#define GLOBAL_MMIO_FWVER 0x148
+#define GLOBAL_MMIO_CCAP0 0x160
+#define GLOBAL_MMIO_CCAP1 0x168
+
+#define GLOBAL_MMIO_CHI_ACRA BIT_ULL(0)
+#define GLOBAL_MMIO_CHI_NSCRA BIT_ULL(1)
+#define GLOBAL_MMIO_CHI_CRDY BIT_ULL(4)
+#define GLOBAL_MMIO_CHI_CFFS BIT_ULL(5)
+#define GLOBAL_MMIO_CHI_MA BIT_ULL(6)
+#define GLOBAL_MMIO_CHI_ELABIT_ULL(7)
+#define GLOBAL_MMIO_CHI_CDABIT_ULL(8)
+#define GLOBAL_MMIO_CHI_CHFS BIT_ULL(9)
+
+#define GLOBAL_MMIO_CHI_ALL(GLOBAL_MMIO_CHI_ACRA | \
+GLOBAL_MMIO_CHI_NSCRA | \
+GLOBAL_MMIO_CHI_CRDY | \
+GLOBAL_MMIO_CHI_CFFS | \
+GLOBAL_MMIO_CHI_MA | \
+GLOBAL_MMIO_CHI_ELA | \
+GLOBAL_MMIO_CHI_CDA | \
+GLOBAL_MMIO_CHI_CHFS)
+
+#define GLOBAL_MMIO_HCI_ACRW BIT_ULL(0)
+#define GLOBAL_MMIO_HCI_NSCRW BIT_ULL(1)
+#define GLOBAL_MMIO_HCI_AFU_RESET BIT_ULL(2)
+#define GLOBAL_MMIO_HCI_FW_DEBUG BIT_ULL(3)
+#define GLOBAL_MMIO_HCI_CONTROLLER_DUMPBIT_ULL(4)
+#define GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED BIT_ULL(5)
+#define GLOBAL_MMIO_HCI_REQ_HEALTH_PERFBIT_ULL(6)
+
+#define ADMIN_COMMAND_HEARTBEAT0x00u
+#define ADMIN_COMMAND_SHUTDOWN 0x01u
+#define ADMIN_COMMAND_FW_UPDATE0x02u
+#define ADMIN_COMMAND_FW_DEBUG 0x03u
+#define ADMIN_COMMAND_ERRLOG 0x04u
+#define ADMIN_COMMAND_SMART0x05u
+#define ADMIN_COMMAND_CONTROLLER_STATS 0x06u
+#define ADMIN_COMMAND_CONTROLLER_DUMP 0x07u
+#define ADMIN_COMMAND_CMD_CAPS 0x08u
+#define ADMIN_COMMAND_MAX 0x08u
+
+#define STATUS_SUCCESS 0x00
+#define STATUS_MEM_UNAVAILABLE 0x20
+#define STATUS_BAD_OPCODE 0x50
+#define STATUS_BAD_REQUEST_PARM0x51
+#define STATUS_BAD_DATA_PARM 0x52
+#define STATUS_DEBUG_BLOCKED 0x70
+#define STATUS_FAIL0xFF
+
+#define STATUS_FW_UPDATE_BLOCKED 0x21
+#define STATUS_FW_ARG_INVALID 0x51
+#define STATUS_FW_INVALID 0x52
+
struct ocxlpmem_function0 {
struct pci_dev *pdev;
struct ocxl_fn *ocxl_fn;
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 21/27] powerpc/powernv/pmem: Add an IOCTL to request controller health & perf data
From: Alastair D'Silva
When health & performance data is requested from the controller,
it responds with an error log containing the requested information.
This patch allows the request to me issued via an IOCTL.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c | 16
include/uapi/nvdimm/ocxl-pmem.h| 1 +
2 files changed, 17 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index e46696d3cc36..081883a8247a 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -1000,6 +1000,18 @@ static int ioctl_event_check(struct ocxlpmem *ocxlpmem,
u64 __user *uarg)
return rc;
}
+/**
+ * req_controller_health_perf() - Request controller health & performance data
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success, negative on failure
+ */
+int req_controller_health_perf(struct ocxlpmem *ocxlpmem)
+{
+ return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI,
+ OCXL_LITTLE_ENDIAN,
+ GLOBAL_MMIO_HCI_REQ_HEALTH_PERF);
+}
+
static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args)
{
struct ocxlpmem *ocxlpmem = file->private_data;
@@ -1037,6 +1049,10 @@ static long file_ioctl(struct file *file, unsigned int
cmd, unsigned long args)
case IOCTL_OCXL_PMEM_EVENT_CHECK:
rc = ioctl_event_check(ocxlpmem, (u64 __user *)args);
break;
+
+ case IOCTL_OCXL_PMEM_REQUEST_HEALTH:
+ rc = req_controller_health_perf(ocxlpmem);
+ break;
}
return rc;
diff --git a/include/uapi/nvdimm/ocxl-pmem.h b/include/uapi/nvdimm/ocxl-pmem.h
index 988eb0bc413d..0d03abb44001 100644
--- a/include/uapi/nvdimm/ocxl-pmem.h
+++ b/include/uapi/nvdimm/ocxl-pmem.h
@@ -90,5 +90,6 @@ struct ioctl_ocxl_pmem_eventfd {
#define IOCTL_OCXL_PMEM_CONTROLLER_STATS _IO(OCXL_PMEM_MAGIC,
0x05)
#define IOCTL_OCXL_PMEM_EVENTFD
_IOW(OCXL_PMEM_MAGIC, 0x06, struct ioctl_ocxl_pmem_eventfd)
#define IOCTL_OCXL_PMEM_EVENT_CHECK_IOR(OCXL_PMEM_MAGIC,
0x07, __u64)
+#define IOCTL_OCXL_PMEM_REQUEST_HEALTH _IO(OCXL_PMEM_MAGIC,
0x08)
#endif /* _UAPI_OCXL_SCM_H */
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 16/27] powerpc/powernv/pmem: Register a character device for userspace to interact with
From: Alastair D'Silva
This patch introduces a character device (/dev/ocxl-scmX) which further
patches will use to interact with userspace.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c| 116 +-
.../platforms/powernv/pmem/ocxl_internal.h| 2 +
2 files changed, 116 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index b8bd7e703b19..63109a870d2c 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -10,6 +10,7 @@
#include
#include
#include
+#include
#include
#include
#include "ocxl_internal.h"
@@ -339,6 +340,9 @@ static void free_ocxlpmem(struct ocxlpmem *ocxlpmem)
free_minor(ocxlpmem);
+ if (ocxlpmem->cdev.owner)
+ cdev_del(&ocxlpmem->cdev);
+
if (ocxlpmem->metadata_addr)
devm_memunmap(&ocxlpmem->dev, ocxlpmem->metadata_addr);
@@ -396,6 +400,70 @@ static int ocxlpmem_register(struct ocxlpmem *ocxlpmem)
return device_register(&ocxlpmem->dev);
}
+static void ocxlpmem_put(struct ocxlpmem *ocxlpmem)
+{
+ put_device(&ocxlpmem->dev);
+}
+
+static struct ocxlpmem *ocxlpmem_get(struct ocxlpmem *ocxlpmem)
+{
+ return (get_device(&ocxlpmem->dev) == NULL) ? NULL : ocxlpmem;
+}
+
+static struct ocxlpmem *find_and_get_ocxlpmem(dev_t devno)
+{
+ struct ocxlpmem *ocxlpmem;
+ int minor = MINOR(devno);
+ /*
+* We don't declare an RCU critical section here, as our AFU
+* is protected by a reference counter on the device. By the time the
+* minor number of a device is removed from the idr, the ref count of
+* the device is already at 0, so no user API will access that AFU and
+* this function can't return it.
+*/
+ ocxlpmem = idr_find(&minors_idr, minor);
+ if (ocxlpmem)
+ ocxlpmem_get(ocxlpmem);
+ return ocxlpmem;
+}
+
+static int file_open(struct inode *inode, struct file *file)
+{
+ struct ocxlpmem *ocxlpmem;
+
+ ocxlpmem = find_and_get_ocxlpmem(inode->i_rdev);
+ if (!ocxlpmem)
+ return -ENODEV;
+
+ file->private_data = ocxlpmem;
+ return 0;
+}
+
+static int file_release(struct inode *inode, struct file *file)
+{
+ struct ocxlpmem *ocxlpmem = file->private_data;
+
+ ocxlpmem_put(ocxlpmem);
+ return 0;
+}
+
+static const struct file_operations fops = {
+ .owner = THIS_MODULE,
+ .open = file_open,
+ .release= file_release,
+};
+
+/**
+ * create_cdev() - Create the chardev in /dev for the device
+ * @ocxlpmem: the SCM metadata
+ * Return: 0 on success, negative on failure
+ */
+static int create_cdev(struct ocxlpmem *ocxlpmem)
+{
+ cdev_init(&ocxlpmem->cdev, &fops);
+ return cdev_add(&ocxlpmem->cdev, ocxlpmem->dev.devt, 1);
+}
+
/**
* ocxlpmem_remove() - Free an OpenCAPI persistent memory device
* @pdev: the PCI device information struct
@@ -572,6 +640,11 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
goto err;
}
+ if (create_cdev(ocxlpmem)) {
+ dev_err(&pdev->dev, "Could not create character device\n");
+ goto err;
+ }
+
elapsed = 0;
timeout = ocxlpmem->readiness_timeout +
ocxlpmem->memory_available_timeout;
while (!is_usable(ocxlpmem, false)) {
@@ -613,20 +686,59 @@ static struct pci_driver pci_driver = {
.shutdown = ocxlpmem_remove,
};
+static int file_init(void)
+{
+ int rc;
+
+ mutex_init(&minors_idr_lock);
+ idr_init(&minors_idr);
+
+ rc = alloc_chrdev_region(&ocxlpmem_dev, 0, NUM_MINORS, "ocxl-pmem");
+ if (rc) {
+ idr_destroy(&minors_idr);
+ pr_err("Unable to allocate OpenCAPI persistent memory major
number: %d\n", rc);
+ return rc;
+ }
+
+ ocxlpmem_class = class_create(THIS_MODULE, "ocxl-pmem");
+ if (IS_ERR(ocxlpmem_class)) {
+ idr_destroy(&minors_idr);
+ pr_err("Unable to create ocxl-pmem class\n");
+ unregister_chrdev_region(ocxlpmem_dev, NUM_MINORS);
+ return PTR_ERR(ocxlpmem_class);
+ }
+
+ return 0;
+}
+
+static void file_exit(void)
+{
+ class_destroy(ocxlpmem_class);
+ unregister_chrdev_region(ocxlpmem_dev, NUM_MINORS);
+ idr_destroy(&minors_idr);
+}
+
static int __init ocxlpmem_init(void)
{
- int rc = 0;
+ int rc;
- rc = pci_register_driver(&pci_driver);
+ rc = file_init();
if (rc)
return rc;
+ rc = pci_register_driver(&pci_driver);
+ if (rc) {
+ file_exit();
+ return rc;
+ }
+
return 0;
}
static void ocxlpmem_exit(void)
{
pci_unregister_driver(&pci_driver);
+ file_exit();
[PATCH v3 24/27] powerpc/powernv/pmem: Expose SMART data via ndctl
From: Alastair D'Silva
This patch retrieves proprietary formatted SMART data and makes it
available via ndctl. A later contribution will be made to ndctl to
parse this data.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c| 128 ++
.../platforms/powernv/pmem/ocxl_internal.h| 18 +++
include/uapi/linux/ndctl.h| 1 +
3 files changed, 147 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index d4ce5e9e0521..5cd1b6d78dd6 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -81,6 +81,129 @@ static int ndctl_config_size(struct nd_cmd_get_config_size
*command)
return 0;
}
+/**
+ * smart_header_parse() - Parse the first 64 bits of the SMART admin command
response
+ * @ocxlpmem: the device metadata
+ * @length: out, returns the number of bytes in the response (excluding the 64
bit header)
+ */
+static int smart_header_parse(struct ocxlpmem *ocxlpmem, u32 *length)
+{
+ int rc;
+ u64 val;
+
+ u16 data_identifier;
+ u32 data_length;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ data_identifier = val >> 48;
+ data_length = val & 0x;
+
+ if (data_identifier != 0x534D) { // 'SM'
+ dev_err(&ocxlpmem->dev,
+ "Bad data identifier for smart data, expected 'SM', got
'%-.*s'\n",
+ 2, (char *)&data_identifier);
+ return -EINVAL;
+ }
+
+ *length = data_length;
+ return 0;
+}
+
+static int ndctl_smart(struct ocxlpmem *ocxlpmem, struct nd_cmd_pkg *pkg)
+{
+ u32 length, i;
+ struct nd_ocxl_smart *out;
+ int rc;
+
+ mutex_lock(&ocxlpmem->admin_command.lock);
+
+ rc = admin_command_request(ocxlpmem, ADMIN_COMMAND_SMART);
+ if (rc)
+ goto out;
+
+ rc = admin_command_execute(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = admin_command_complete_timeout(ocxlpmem, ADMIN_COMMAND_SMART);
+ if (rc < 0) {
+ dev_err(&ocxlpmem->dev, "SMART timeout\n");
+ goto out;
+ }
+
+ rc = admin_response(ocxlpmem);
+ if (rc < 0)
+ goto out;
+ if (rc != STATUS_SUCCESS) {
+ warn_status(ocxlpmem, "Unexpected status from SMART", rc);
+ goto out;
+ }
+
+ rc = smart_header_parse(ocxlpmem, &length);
+ if (rc)
+ goto out;
+
+ pkg->nd_fw_size = length;
+
+ length = min(length, pkg->nd_size_out); // bytes
+ out = (struct nd_ocxl_smart *)pkg->nd_payload;
+ // Each SMART attribute is 2 * 64 bits
+ out->count = length / (2 * sizeof(u64)); // attributes
+
+ for (i = 0; i < length; i += sizeof(u64)) {
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+
ocxlpmem->admin_command.data_offset + sizeof(u64) + i,
+OCXL_LITTLE_ENDIAN,
+&out->attribs[i/sizeof(u64)]);
+ if (rc)
+ goto out;
+ }
+
+ rc = admin_response_handled(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = 0;
+ goto out;
+
+out:
+ mutex_unlock(&ocxlpmem->admin_command.lock);
+ return rc;
+}
+
+static int ndctl_call(struct ocxlpmem *ocxlpmem, void *buf, unsigned int
buf_len)
+{
+ struct nd_cmd_pkg *pkg = buf;
+
+ if (buf_len < sizeof(struct nd_cmd_pkg)) {
+ dev_err(&ocxlpmem->dev, "Invalid ND_CALL size=%u\n", buf_len);
+ return -EINVAL;
+ }
+
+ if (pkg->nd_family != NVDIMM_FAMILY_OCXL) {
+ dev_err(&ocxlpmem->dev, "Invalid ND_CALL family=0x%llx\n",
pkg->nd_family);
+ return -EINVAL;
+ }
+
+ switch (pkg->nd_command) {
+ case ND_CMD_OCXL_SMART:
+ ndctl_smart(ocxlpmem, pkg);
+ break;
+
+ default:
+ dev_err(&ocxlpmem->dev, "Invalid ND_CALL command=0x%llx\n",
pkg->nd_command);
+ return -EINVAL;
+ }
+
+
+ return 0;
+}
+
static int ndctl(struct nvdimm_bus_descriptor *nd_desc,
struct nvdimm *nvdimm,
unsigned int cmd, void *buf, unsigned int buf_len, int *cmd_rc)
@@ -88,6 +211,10 @@ static int ndctl(struct nvdimm_bus_descriptor *nd_desc,
struct ocxlpmem *ocxlpmem = container_of(nd_desc, struct ocxlpmem,
bus_desc);
switch (cmd) {
+ case ND_CMD_CALL:
+ *cmd_rc = ndctl_call(ocxlpmem, buf, buf_len);
+ return 0;
+
case ND_CMD_GET_CONFIG_SIZE:
[PATCH v3 19/27] powerpc/powernv/pmem: Add an IOCTL to report controller statistics
From: Alastair D'Silva
The controller can report a number of statistics that are useful
in evaluating the performance and reliability of the card.
This patch exposes this information via an IOCTL.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c | 185 +
include/uapi/nvdimm/ocxl-pmem.h| 17 ++
2 files changed, 202 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 2cabafe1fc58..009d4fd29e7d 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -758,6 +758,186 @@ static int ioctl_controller_dump_complete(struct ocxlpmem
*ocxlpmem)
GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED);
}
+/**
+ * controller_stats_header_parse() - Parse the first 64 bits of the controller
stats admin command response
+ * @ocxlpmem: the device metadata
+ * @length: out, returns the number of bytes in the response (excluding the 64
bit header)
+ */
+static int controller_stats_header_parse(struct ocxlpmem *ocxlpmem,
+ u32 *length)
+{
+ int rc;
+ u64 val;
+
+ u16 data_identifier;
+ u32 data_length;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ data_identifier = val >> 48;
+ data_length = val & 0x;
+
+ if (data_identifier != 0x4353) { // 'CS'
+ dev_err(&ocxlpmem->dev,
+ "Bad data identifier for controller stats, expected
'CS', got '%-.*s'\n",
+ 2, (char *)&data_identifier);
+ return -EINVAL;
+ }
+
+ *length = data_length;
+ return 0;
+}
+
+static int ioctl_controller_stats(struct ocxlpmem *ocxlpmem,
+ struct ioctl_ocxl_pmem_controller_stats
__user *uarg)
+{
+ struct ioctl_ocxl_pmem_controller_stats args;
+ u32 length;
+ int rc;
+ u64 val;
+
+ memset(&args, '\0', sizeof(args));
+
+ mutex_lock(&ocxlpmem->admin_command.lock);
+
+ rc = admin_command_request(ocxlpmem, ADMIN_COMMAND_CONTROLLER_STATS);
+ if (rc)
+ goto out;
+
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+ ocxlpmem->admin_command.request_offset +
0x08,
+ OCXL_LITTLE_ENDIAN, 0);
+ if (rc)
+ goto out;
+
+ rc = admin_command_execute(ocxlpmem);
+ if (rc)
+ goto out;
+
+
+ rc = admin_command_complete_timeout(ocxlpmem,
+ ADMIN_COMMAND_CONTROLLER_STATS);
+ if (rc < 0) {
+ dev_warn(&ocxlpmem->dev, "Controller stats timed out\n");
+ goto out;
+ }
+
+ rc = admin_response(ocxlpmem);
+ if (rc < 0)
+ goto out;
+ if (rc != STATUS_SUCCESS) {
+ warn_status(ocxlpmem,
+ "Unexpected status from controller stats", rc);
+ goto out;
+ }
+
+ rc = controller_stats_header_parse(ocxlpmem, &length);
+ if (rc)
+ goto out;
+
+ if (length != 0x140)
+ warn_status(ocxlpmem,
+ "Unexpected length for controller stats data,
expected 0x140, got 0x%x",
+ length);
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x08
+ 0x08,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ goto out;
+
+ args.reset_count = val >> 32;
+ args.reset_uptime = val & 0x;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x08
+ 0x10,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ goto out;
+
+ args.power_on_uptime = val >> 32;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x08
+ 0x40 + 0x08,
+OCXL_LITTLE_ENDIAN, &args.host_load_count);
+ if (rc)
+ goto out;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x08
+ 0x40 + 0x10,
+OCXL_LITTLE_ENDIAN,
&args.host_store_count);
+ if (rc)
+ goto out;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x08
+ 0x40 + 0x18,
+OCXL_LITTLE_ENDIAN,
&args.
[PATCH v3 23/27] powerpc/powernv/pmem: Add debug IOCTLs
From: Alastair D'Silva
These IOCTLs provide low level access to the card to aid in debugging
controller/FPGA firmware.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/Kconfig | 6 +
arch/powerpc/platforms/powernv/pmem/ocxl.c | 249
include/uapi/nvdimm/ocxl-pmem.h | 32 +++
3 files changed, 287 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/Kconfig
b/arch/powerpc/platforms/powernv/pmem/Kconfig
index c5d927520920..3f44429d70c9 100644
--- a/arch/powerpc/platforms/powernv/pmem/Kconfig
+++ b/arch/powerpc/platforms/powernv/pmem/Kconfig
@@ -12,4 +12,10 @@ config OCXL_PMEM
Select N if unsure.
+config OCXL_PMEM_DEBUG
+ bool "OpenCAPI Persistent Memory debugging"
+ depends on OCXL_PMEM
+ help
+ Enables low level IOCTLs for OpenCAPI Persistent Memory firmware
development
+
endif
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index e01f6f9fc180..d4ce5e9e0521 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -1050,6 +1050,235 @@ int req_controller_health_perf(struct ocxlpmem
*ocxlpmem)
GLOBAL_MMIO_HCI_REQ_HEALTH_PERF);
}
+#ifdef CONFIG_OCXL_PMEM_DEBUG
+/**
+ * enable_fwdebug() - Enable FW debug on the controller
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success, negative on failure
+ */
+static int enable_fwdebug(const struct ocxlpmem *ocxlpmem)
+{
+ return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI,
+ OCXL_LITTLE_ENDIAN,
+ GLOBAL_MMIO_HCI_FW_DEBUG);
+}
+
+/**
+ * disable_fwdebug() - Disable FW debug on the controller
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success, negative on failure
+ */
+static int disable_fwdebug(const struct ocxlpmem *ocxlpmem)
+{
+ return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCIC,
+ OCXL_LITTLE_ENDIAN,
+ GLOBAL_MMIO_HCI_FW_DEBUG);
+}
+
+static int ioctl_fwdebug(struct ocxlpmem *ocxlpmem,
+struct ioctl_ocxl_pmem_fwdebug __user *uarg)
+{
+ struct ioctl_ocxl_pmem_fwdebug args;
+ u64 val;
+ int i;
+ int rc;
+
+ if (copy_from_user(&args, uarg, sizeof(args)))
+ return -EFAULT;
+
+ // Buffer size must be a multiple of 8
+ if ((args.buf_size & 0x07))
+ return -EINVAL;
+
+ if (args.buf_size > ocxlpmem->admin_command.data_size)
+ return -EINVAL;
+
+ mutex_lock(&ocxlpmem->admin_command.lock);
+
+ rc = enable_fwdebug(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = admin_command_request(ocxlpmem, ADMIN_COMMAND_FW_DEBUG);
+ if (rc)
+ goto out;
+
+ // Write DebugAction & FunctionCode
+ val = ((u64)args.debug_action << 56) | ((u64)args.function_code << 40);
+
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+ ocxlpmem->admin_command.request_offset +
0x08,
+ OCXL_LITTLE_ENDIAN, val);
+ if (rc)
+ goto out;
+
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+ ocxlpmem->admin_command.request_offset +
0x10,
+ OCXL_LITTLE_ENDIAN,
args.debug_parameter_1);
+ if (rc)
+ goto out;
+
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+ ocxlpmem->admin_command.request_offset +
0x18,
+ OCXL_LITTLE_ENDIAN,
args.debug_parameter_2);
+ if (rc)
+ goto out;
+
+ for (i = 0x20; i < 0x38; i += 0x08)
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+
ocxlpmem->admin_command.request_offset + i,
+ OCXL_LITTLE_ENDIAN, 0);
+ if (rc)
+ goto out;
+
+
+ // Populate admin command buffer
+ if (args.buf_size) {
+ for (i = 0; i < args.buf_size; i += sizeof(u64)) {
+ u64 val;
+
+ if (copy_from_user(&val, &args.buf[i], sizeof(u64)))
+ return -EFAULT;
+
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+
ocxlpmem->admin_command.data_offset + i,
+ OCXL_HOST_ENDIAN, val);
+ if (rc)
+ goto out;
+ }
+ }
+
+ rc = admin_command_execute(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = admin_command_complete_timeout(ocxlpmem,
+
[PATCH v3 17/27] powerpc/powernv/pmem: Implement the Read Error Log command
From: Alastair D'Silva
The read error log command extracts information from the controller's
internal error log.
This patch exposes this information in 2 ways:
- During probe, if an error occurs & a log is available, print it to the
console
- After probe, make the error log available to userspace via an IOCTL.
Userspace is notified of pending error logs in a later patch
("powerpc/powernv/pmem: Forward events to userspace")
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c| 269 ++
.../platforms/powernv/pmem/ocxl_internal.h| 1 +
include/uapi/nvdimm/ocxl-pmem.h | 46 +++
3 files changed, 316 insertions(+)
create mode 100644 include/uapi/nvdimm/ocxl-pmem.h
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 63109a870d2c..2b64504f9129 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -447,10 +447,219 @@ static int file_release(struct inode *inode, struct file
*file)
return 0;
}
+/**
+ * error_log_header_parse() - Parse the first 64 bits of the error log command
response
+ * @ocxlpmem: the device metadata
+ * @length: out, returns the number of bytes in the response (excluding the 64
bit header)
+ */
+static int error_log_header_parse(struct ocxlpmem *ocxlpmem, u16 *length)
+{
+ int rc;
+ u64 val;
+
+ u16 data_identifier;
+ u32 data_length;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ data_identifier = val >> 48;
+ data_length = val & 0x;
+
+ if (data_identifier != 0x454C) { // 'EL'
+ dev_err(&ocxlpmem->dev,
+ "Bad data identifier for error log data, expected 'EL',
got '%2s' (%#x), data_length=%u\n",
+ (char *)&data_identifier,
+ (unsigned int)data_identifier, data_length);
+ return -EINVAL;
+ }
+
+ *length = data_length;
+ return 0;
+}
+
+static int error_log_offset_0x08(struct ocxlpmem *ocxlpmem,
+u32 *log_identifier, u32 *program_ref_code)
+{
+ int rc;
+ u64 val;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x08,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ *log_identifier = val >> 32;
+ *program_ref_code = val & 0x;
+
+ return 0;
+}
+
+static int read_error_log(struct ocxlpmem *ocxlpmem,
+ struct ioctl_ocxl_pmem_error_log *log, bool
buf_is_user)
+{
+ u64 val;
+ u16 user_buf_length;
+ u16 buf_length;
+ u16 i;
+ int rc;
+
+ if (log->buf_size % 8)
+ return -EINVAL;
+
+ rc = ocxlpmem_chi(ocxlpmem, &val);
+ if (rc)
+ goto out;
+
+ if (!(val & GLOBAL_MMIO_CHI_ELA))
+ return -EAGAIN;
+
+ user_buf_length = log->buf_size;
+
+ mutex_lock(&ocxlpmem->admin_command.lock);
+
+ rc = admin_command_request(ocxlpmem, ADMIN_COMMAND_ERRLOG);
+ if (rc)
+ goto out;
+
+ rc = admin_command_execute(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = admin_command_complete_timeout(ocxlpmem, ADMIN_COMMAND_ERRLOG);
+ if (rc < 0) {
+ dev_warn(&ocxlpmem->dev, "Read error log timed out\n");
+ goto out;
+ }
+
+ rc = admin_response(ocxlpmem);
+ if (rc < 0)
+ goto out;
+ if (rc != STATUS_SUCCESS) {
+ warn_status(ocxlpmem, "Unexpected status from retrieve error
log", rc);
+ goto out;
+ }
+
+
+ rc = error_log_header_parse(ocxlpmem, &log->buf_size);
+ if (rc)
+ goto out;
+ // log->buf_size now contains the returned buffer size, not the user
size
+
+ rc = error_log_offset_0x08(ocxlpmem, &log->log_identifier,
+ &log->program_reference_code);
+ if (rc)
+ goto out;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x10,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ goto out;
+
+ log->error_log_type = val >> 56;
+ log->action_flags = (log->error_log_type ==
OCXL_PMEM_ERROR_LOG_TYPE_GENERAL) ?
+ (val >> 32) & 0xFF : 0;
+ log->power_on_seconds = val & 0x;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+ocxlpmem->admin_command.data_offset + 0x18,
+
[PATCH v3 01/27] powerpc: Add OPAL calls for LPC memory alloc/release
From: Alastair D'Silva Add OPAL calls for LPC memory alloc/release Signed-off-by: Alastair D'Silva Acked-by: Andrew Donnellan Acked-by: Frederic Barrat --- arch/powerpc/include/asm/opal-api.h| 2 ++ arch/powerpc/include/asm/opal.h| 3 +++ arch/powerpc/platforms/powernv/opal-call.c | 2 ++ 3 files changed, 7 insertions(+) diff --git a/arch/powerpc/include/asm/opal-api.h b/arch/powerpc/include/asm/opal-api.h index c1f25a760eb1..9298e603001b 100644 --- a/arch/powerpc/include/asm/opal-api.h +++ b/arch/powerpc/include/asm/opal-api.h @@ -208,6 +208,8 @@ #define OPAL_HANDLE_HMI2 166 #defineOPAL_NX_COPROC_INIT 167 #define OPAL_XIVE_GET_VP_STATE 170 +#define OPAL_NPU_MEM_ALLOC 171 +#define OPAL_NPU_MEM_RELEASE 172 #define OPAL_MPIPL_UPDATE 173 #define OPAL_MPIPL_REGISTER_TAG174 #define OPAL_MPIPL_QUERY_TAG 175 diff --git a/arch/powerpc/include/asm/opal.h b/arch/powerpc/include/asm/opal.h index 9986ac34b8e2..8f7727e0f9ce 100644 --- a/arch/powerpc/include/asm/opal.h +++ b/arch/powerpc/include/asm/opal.h @@ -39,6 +39,9 @@ int64_t opal_npu_spa_clear_cache(uint64_t phb_id, uint32_t bdfn, uint64_t PE_handle); int64_t opal_npu_tl_set(uint64_t phb_id, uint32_t bdfn, long cap, uint64_t rate_phys, uint32_t size); +int64_t opal_npu_mem_alloc(uint64_t phb_id, uint32_t bdfn, + uint64_t size, uint64_t *bar); +int64_t opal_npu_mem_release(uint64_t phb_id, uint32_t bdfn); int64_t opal_console_write(int64_t term_number, __be64 *length, const uint8_t *buffer); diff --git a/arch/powerpc/platforms/powernv/opal-call.c b/arch/powerpc/platforms/powernv/opal-call.c index 5cd0f52d258f..f26e58b72c04 100644 --- a/arch/powerpc/platforms/powernv/opal-call.c +++ b/arch/powerpc/platforms/powernv/opal-call.c @@ -287,6 +287,8 @@ OPAL_CALL(opal_pci_set_pbcq_tunnel_bar, OPAL_PCI_SET_PBCQ_TUNNEL_BAR); OPAL_CALL(opal_sensor_read_u64,OPAL_SENSOR_READ_U64); OPAL_CALL(opal_sensor_group_enable,OPAL_SENSOR_GROUP_ENABLE); OPAL_CALL(opal_nx_coproc_init, OPAL_NX_COPROC_INIT); +OPAL_CALL(opal_npu_mem_alloc, OPAL_NPU_MEM_ALLOC); +OPAL_CALL(opal_npu_mem_release,OPAL_NPU_MEM_RELEASE); OPAL_CALL(opal_mpipl_update, OPAL_MPIPL_UPDATE); OPAL_CALL(opal_mpipl_register_tag, OPAL_MPIPL_REGISTER_TAG); OPAL_CALL(opal_mpipl_query_tag,OPAL_MPIPL_QUERY_TAG); -- 2.24.1 ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 04/27] ocxl: Remove unnecessary externs
From: Alastair D'Silva Function declarations don't need externs, remove the existing ones so they are consistent with newer code Signed-off-by: Alastair D'Silva --- arch/powerpc/include/asm/pnv-ocxl.h | 32 ++--- include/misc/ocxl.h | 6 +++--- 2 files changed, 18 insertions(+), 20 deletions(-) diff --git a/arch/powerpc/include/asm/pnv-ocxl.h b/arch/powerpc/include/asm/pnv-ocxl.h index 0b2a6707e555..b23c99bc0c84 100644 --- a/arch/powerpc/include/asm/pnv-ocxl.h +++ b/arch/powerpc/include/asm/pnv-ocxl.h @@ -9,29 +9,27 @@ #define PNV_OCXL_TL_BITS_PER_RATE 4 #define PNV_OCXL_TL_RATE_BUF_SIZE ((PNV_OCXL_TL_MAX_TEMPLATE+1) * PNV_OCXL_TL_BITS_PER_RATE / 8) -extern int pnv_ocxl_get_actag(struct pci_dev *dev, u16 *base, u16 *enabled, - u16 *supported); -extern int pnv_ocxl_get_pasid_count(struct pci_dev *dev, int *count); +int pnv_ocxl_get_actag(struct pci_dev *dev, u16 *base, u16 *enabled, u16 *supported); +int pnv_ocxl_get_pasid_count(struct pci_dev *dev, int *count); -extern int pnv_ocxl_get_tl_cap(struct pci_dev *dev, long *cap, +int pnv_ocxl_get_tl_cap(struct pci_dev *dev, long *cap, char *rate_buf, int rate_buf_size); -extern int pnv_ocxl_set_tl_conf(struct pci_dev *dev, long cap, +int pnv_ocxl_set_tl_conf(struct pci_dev *dev, long cap, uint64_t rate_buf_phys, int rate_buf_size); -extern int pnv_ocxl_get_xsl_irq(struct pci_dev *dev, int *hwirq); -extern void pnv_ocxl_unmap_xsl_regs(void __iomem *dsisr, void __iomem *dar, - void __iomem *tfc, void __iomem *pe_handle); -extern int pnv_ocxl_map_xsl_regs(struct pci_dev *dev, void __iomem **dsisr, - void __iomem **dar, void __iomem **tfc, - void __iomem **pe_handle); +int pnv_ocxl_get_xsl_irq(struct pci_dev *dev, int *hwirq); +void pnv_ocxl_unmap_xsl_regs(void __iomem *dsisr, void __iomem *dar, +void __iomem *tfc, void __iomem *pe_handle); +int pnv_ocxl_map_xsl_regs(struct pci_dev *dev, void __iomem **dsisr, + void __iomem **dar, void __iomem **tfc, + void __iomem **pe_handle); -extern int pnv_ocxl_spa_setup(struct pci_dev *dev, void *spa_mem, int PE_mask, - void **platform_data); -extern void pnv_ocxl_spa_release(void *platform_data); -extern int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle); +int pnv_ocxl_spa_setup(struct pci_dev *dev, void *spa_mem, int PE_mask, void **platform_data); +void pnv_ocxl_spa_release(void *platform_data); +int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle); -extern int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr); -extern void pnv_ocxl_free_xive_irq(u32 irq); +int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr); +void pnv_ocxl_free_xive_irq(u32 irq); #ifdef CONFIG_MEMORY_HOTPLUG_SPARSE u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size); void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev); diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h index 06dd5839e438..0a762e387418 100644 --- a/include/misc/ocxl.h +++ b/include/misc/ocxl.h @@ -173,7 +173,7 @@ int ocxl_context_detach(struct ocxl_context *ctx); * * Returns 0 on success, negative on failure */ -extern int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); +int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); /** * Frees an IRQ associated with an AFU context @@ -182,7 +182,7 @@ extern int ocxl_afu_irq_alloc(struct ocxl_context *ctx, int *irq_id); * * Returns 0 on success, negative on failure */ -extern int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); +int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); /** * Gets the address of the trigger page for an IRQ @@ -193,7 +193,7 @@ extern int ocxl_afu_irq_free(struct ocxl_context *ctx, int irq_id); * * returns the trigger page address, or 0 if the IRQ is not valid */ -extern u64 ocxl_afu_irq_get_addr(struct ocxl_context *ctx, int irq_id); +u64 ocxl_afu_irq_get_addr(struct ocxl_context *ctx, int irq_id); /** * Provide a callback to be called when an IRQ is triggered -- 2.24.1 ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 03/27] powerpc: Map & release OpenCAPI LPC memory
From: Alastair D'Silva
This patch adds platform support to map & release LPC memory.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/include/asm/pnv-ocxl.h | 4 +++
arch/powerpc/platforms/powernv/ocxl.c | 43 +++
2 files changed, 47 insertions(+)
diff --git a/arch/powerpc/include/asm/pnv-ocxl.h
b/arch/powerpc/include/asm/pnv-ocxl.h
index 7de82647e761..0b2a6707e555 100644
--- a/arch/powerpc/include/asm/pnv-ocxl.h
+++ b/arch/powerpc/include/asm/pnv-ocxl.h
@@ -32,5 +32,9 @@ extern int pnv_ocxl_spa_remove_pe_from_cache(void
*platform_data, int pe_handle)
extern int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr);
extern void pnv_ocxl_free_xive_irq(u32 irq);
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size);
+void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev);
+#endif
#endif /* _ASM_PNV_OCXL_H */
diff --git a/arch/powerpc/platforms/powernv/ocxl.c
b/arch/powerpc/platforms/powernv/ocxl.c
index 8c65aacda9c8..f2edbcc67361 100644
--- a/arch/powerpc/platforms/powernv/ocxl.c
+++ b/arch/powerpc/platforms/powernv/ocxl.c
@@ -475,6 +475,49 @@ void pnv_ocxl_spa_release(void *platform_data)
}
EXPORT_SYMBOL_GPL(pnv_ocxl_spa_release);
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+u64 pnv_ocxl_platform_lpc_setup(struct pci_dev *pdev, u64 size)
+{
+ struct pci_controller *hose = pci_bus_to_host(pdev->bus);
+ struct pnv_phb *phb = hose->private_data;
+ u32 bdfn = pci_dev_id(pdev);
+ __be64 base_addr_be64;
+ u64 base_addr;
+ int rc;
+
+ rc = opal_npu_mem_alloc(phb->opal_id, bdfn, size, &base_addr_be64);
+ if (rc) {
+ dev_warn(&pdev->dev,
+"OPAL could not allocate LPC memory, rc=%d\n", rc);
+ return 0;
+ }
+
+ base_addr = be64_to_cpu(base_addr_be64);
+
+ rc = check_hotplug_memory_addressable(base_addr >> PAGE_SHIFT,
+ size >> PAGE_SHIFT);
+ if (rc)
+ return 0;
+
+ return base_addr;
+}
+EXPORT_SYMBOL_GPL(pnv_ocxl_platform_lpc_setup);
+
+void pnv_ocxl_platform_lpc_release(struct pci_dev *pdev)
+{
+ struct pci_controller *hose = pci_bus_to_host(pdev->bus);
+ struct pnv_phb *phb = hose->private_data;
+ u32 bdfn = pci_dev_id(pdev);
+ int rc;
+
+ rc = opal_npu_mem_release(phb->opal_id, bdfn);
+ if (rc)
+ dev_warn(&pdev->dev,
+"OPAL reported rc=%d when releasing LPC memory\n", rc);
+}
+EXPORT_SYMBOL_GPL(pnv_ocxl_platform_lpc_release);
+#endif
+
int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data, int pe_handle)
{
struct spa_data *data = (struct spa_data *) platform_data;
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 05/27] ocxl: Address kernel doc errors & warnings
From: Alastair D'Silva
This patch addresses warnings and errors from the kernel doc scripts for
the OpenCAPI driver.
It also makes minor tweaks to make the docs more consistent.
Signed-off-by: Alastair D'Silva
---
drivers/misc/ocxl/config.c| 24
drivers/misc/ocxl/ocxl_internal.h | 9 +--
include/misc/ocxl.h | 96 ---
3 files changed, 55 insertions(+), 74 deletions(-)
diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c
index c8e19bfb5ef9..a62e3d7db2bf 100644
--- a/drivers/misc/ocxl/config.c
+++ b/drivers/misc/ocxl/config.c
@@ -273,16 +273,16 @@ static int read_afu_info(struct pci_dev *dev, struct
ocxl_fn_config *fn,
}
/**
- * Read the template version from the AFU
- * dev: the device for the AFU
- * fn: the AFU offsets
- * len: outputs the template length
- * version: outputs the major<<8,minor version
+ * read_template_version() - Read the template version from the AFU
+ * @dev: the device for the AFU
+ * @fn: the AFU offsets
+ * @len: outputs the template length
+ * @version: outputs the major<<8,minor version
*
* Returns 0 on success, negative on failure
*/
static int read_template_version(struct pci_dev *dev, struct ocxl_fn_config
*fn,
- u16 *len, u16 *version)
+u16 *len, u16 *version)
{
u32 val32;
u8 major, minor;
@@ -476,16 +476,16 @@ static int validate_afu(struct pci_dev *dev, struct
ocxl_afu_config *afu)
}
/**
- * Populate AFU metadata regarding LPC memory
- * dev: the device for the AFU
- * fn: the AFU offsets
- * afu: the AFU struct to populate the LPC metadata into
+ * read_afu_lpc_memory_info() - Populate AFU metadata regarding LPC memory
+ * @dev: the device for the AFU
+ * @fn: the AFU offsets
+ * @afu: the AFU struct to populate the LPC metadata into
*
* Returns 0 on success, negative on failure
*/
static int read_afu_lpc_memory_info(struct pci_dev *dev,
- struct ocxl_fn_config *fn,
- struct ocxl_afu_config *afu)
+ struct ocxl_fn_config *fn,
+ struct ocxl_afu_config *afu)
{
int rc;
u32 val32;
diff --git a/drivers/misc/ocxl/ocxl_internal.h
b/drivers/misc/ocxl/ocxl_internal.h
index 345bf843a38e..198e4e4bc51d 100644
--- a/drivers/misc/ocxl/ocxl_internal.h
+++ b/drivers/misc/ocxl/ocxl_internal.h
@@ -122,11 +122,12 @@ int ocxl_config_check_afu_index(struct pci_dev *dev,
struct ocxl_fn_config *fn, int afu_idx);
/**
- * Update values within a Process Element
+ * ocxl_link_update_pe() - Update values within a Process Element
+ * @link_handle: the link handle associated with the process element
+ * @pasid: the PASID for the AFU context
+ * @tid: the new thread id for the process element
*
- * link_handle: the link handle associated with the process element
- * pasid: the PASID for the AFU context
- * tid: the new thread id for the process element
+ * Returns 0 on success
*/
int ocxl_link_update_pe(void *link_handle, int pasid, __u16 tid);
diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h
index 0a762e387418..357ef1aadbc0 100644
--- a/include/misc/ocxl.h
+++ b/include/misc/ocxl.h
@@ -62,8 +62,7 @@ struct ocxl_context;
// Device detection & initialisation
/**
- * Open an OpenCAPI function on an OpenCAPI device
- *
+ * ocxl_function_open() - Open an OpenCAPI function on an OpenCAPI device
* @dev: The PCI device that contains the function
*
* Returns an opaque pointer to the function, or an error pointer (check with
IS_ERR)
@@ -71,8 +70,7 @@ struct ocxl_context;
struct ocxl_fn *ocxl_function_open(struct pci_dev *dev);
/**
- * Get the list of AFUs associated with a PCI function device
- *
+ * ocxl_function_afu_list() - Get the list of AFUs associated with a PCI
function device
* Returns a list of struct ocxl_afu *
*
* @fn: The OpenCAPI function containing the AFUs
@@ -80,8 +78,7 @@ struct ocxl_fn *ocxl_function_open(struct pci_dev *dev);
struct list_head *ocxl_function_afu_list(struct ocxl_fn *fn);
/**
- * Fetch an AFU instance from an OpenCAPI function
- *
+ * ocxl_function_fetch_afu() - Fetch an AFU instance from an OpenCAPI function
* @fn: The OpenCAPI function to get the AFU from
* @afu_idx: The index of the AFU to get
*
@@ -92,23 +89,20 @@ struct list_head *ocxl_function_afu_list(struct ocxl_fn
*fn);
struct ocxl_afu *ocxl_function_fetch_afu(struct ocxl_fn *fn, u8 afu_idx);
/**
- * Take a reference to an AFU
- *
+ * ocxl_afu_get() - Take a reference to an AFU
* @afu: The AFU to increment the reference count on
*/
void ocxl_afu_get(struct ocxl_afu *afu);
/**
- * Release a reference to an AFU
- *
+ * ocxl_afu_put() - Release a reference to an AFU
* @afu: The AFU to decrement the reference count on
*/
void ocxl_afu_put(struct ocxl_afu *afu);
/**
- * Get the configuration information for an Ope
[PATCH v3 10/27] powerpc: Add driver for OpenCAPI Persistent Memory
From: Alastair D'Silva
This driver exposes LPC memory on OpenCAPI pmem cards
as an NVDIMM, allowing the existing nvram infrastructure
to be used.
Namespace metadata is stored on the media itself, so
scm_reserve_metadata() maps 1 section's worth of PMEM storage
at the start to hold this. The rest of the PMEM range is registered
with libnvdimm as an nvdimm. scm_ndctl_config_read/write/size() provide
callbacks to libnvdimm to access the metadata.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/Kconfig| 3 +
arch/powerpc/platforms/powernv/Makefile | 1 +
arch/powerpc/platforms/powernv/pmem/Kconfig | 15 +
arch/powerpc/platforms/powernv/pmem/Makefile | 7 +
arch/powerpc/platforms/powernv/pmem/ocxl.c| 473 ++
.../platforms/powernv/pmem/ocxl_internal.h| 28 ++
6 files changed, 527 insertions(+)
create mode 100644 arch/powerpc/platforms/powernv/pmem/Kconfig
create mode 100644 arch/powerpc/platforms/powernv/pmem/Makefile
create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl.c
create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
diff --git a/arch/powerpc/platforms/powernv/Kconfig
b/arch/powerpc/platforms/powernv/Kconfig
index 938803eab0ad..fc8976af0e52 100644
--- a/arch/powerpc/platforms/powernv/Kconfig
+++ b/arch/powerpc/platforms/powernv/Kconfig
@@ -50,3 +50,6 @@ config PPC_VAS
config SCOM_DEBUGFS
bool "Expose SCOM controllers via debugfs"
depends on DEBUG_FS
+
+source "arch/powerpc/platforms/powernv/pmem/Kconfig"
+
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index c0f8120045c3..0bbd72988b6f 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -21,3 +21,4 @@ obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
+obj-$(CONFIG_LIBNVDIMM) += pmem/
diff --git a/arch/powerpc/platforms/powernv/pmem/Kconfig
b/arch/powerpc/platforms/powernv/pmem/Kconfig
new file mode 100644
index ..c5d927520920
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/pmem/Kconfig
@@ -0,0 +1,15 @@
+# SPDX-License-Identifier: GPL-2.0-only
+if LIBNVDIMM
+
+config OCXL_PMEM
+ tristate "OpenCAPI Persistent Memory"
+ depends on LIBNVDIMM && PPC_POWERNV && PCI && EEH && ZONE_DEVICE && OCXL
+ help
+ Exposes devices that implement the OpenCAPI Storage Class Memory
+ specification as persistent memory regions. You may also want
+ DEV_DAX, DEV_DAX_PMEM & FS_DAX if you plan on using DAX devices
+ stacked on top of this driver.
+
+ Select N if unsure.
+
+endif
diff --git a/arch/powerpc/platforms/powernv/pmem/Makefile
b/arch/powerpc/platforms/powernv/pmem/Makefile
new file mode 100644
index ..1c55c4193175
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/pmem/Makefile
@@ -0,0 +1,7 @@
+# SPDX-License-Identifier: GPL-2.0
+
+ccflags-$(CONFIG_PPC_WERROR) += -Werror
+
+obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o
+
+ocxlpmem-y := ocxl.o
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
new file mode 100644
index ..3c4eeb5dcc0f
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -0,0 +1,473 @@
+// SPDX-License-Id
+// Copyright 2019 IBM Corp.
+
+/*
+ * A driver for OpenCAPI devices that implement the Storage Class
+ * Memory specification.
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include "ocxl_internal.h"
+
+
+static const struct pci_device_id ocxlpmem_pci_tbl[] = {
+ { PCI_DEVICE(PCI_VENDOR_ID_IBM, 0x0625), },
+ { }
+};
+
+MODULE_DEVICE_TABLE(pci, ocxlpmem_pci_tbl);
+
+#define NUM_MINORS 256 // Total to reserve
+
+static dev_t ocxlpmem_dev;
+static struct class *ocxlpmem_class;
+static struct mutex minors_idr_lock;
+static struct idr minors_idr;
+
+/**
+ * ndctl_config_write() - Handle a ND_CMD_SET_CONFIG_DATA command from ndctl
+ * @ocxlpmem: the device metadata
+ * @command: the incoming data to write
+ * Return: 0 on success, negative on failure
+ */
+static int ndctl_config_write(struct ocxlpmem *ocxlpmem,
+ struct nd_cmd_set_config_hdr *command)
+{
+ if (command->in_offset + command->in_length > LABEL_AREA_SIZE)
+ return -EINVAL;
+
+ memcpy_flushcache(ocxlpmem->metadata_addr + command->in_offset,
command->in_buf,
+ command->in_length);
+
+ return 0;
+}
+
+/**
+ * ndctl_config_read() - Handle a ND_CMD_GET_CONFIG_DATA command from ndctl
+ * @ocxlpmem: the device metadata
+ * @command: the read request
+ * Return: 0 on success, negative on failure
+ */
+static int ndctl_config_read(struct ocxlpmem *ocxlpmem,
+struct nd_cmd_get_config_data_hdr *command)
+{
+ if (command->in_offset + command->in_length > LA
[PATCH v3 20/27] powerpc/powernv/pmem: Forward events to userspace
From: Alastair D'Silva
Some of the interrupts that the card generates are better handled
by the userspace daemon, in particular:
Controller Hardware/Firmware Fatal
Controller Dump Available
Error Log available
This patch allows a userspace application to register an eventfd with
the driver via SCM_IOCTL_EVENTFD to receive notifications of these
interrupts.
Userspace can then identify what events have occurred by calling
SCM_IOCTL_EVENT_CHECK and checking against the SCM_IOCTL_EVENT_FOO
masks.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c| 216 ++
.../platforms/powernv/pmem/ocxl_internal.h| 5 +
include/uapi/nvdimm/ocxl-pmem.h | 16 ++
3 files changed, 237 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 009d4fd29e7d..e46696d3cc36 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -10,6 +10,7 @@
#include
#include
#include
+#include
#include
#include
#include
@@ -335,11 +336,22 @@ static void free_ocxlpmem(struct ocxlpmem *ocxlpmem)
{
int rc;
+ // Disable doorbells
+ (void)ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIEC,
+OCXL_LITTLE_ENDIAN,
+GLOBAL_MMIO_CHI_ALL);
+
if (ocxlpmem->nvdimm_bus)
nvdimm_bus_unregister(ocxlpmem->nvdimm_bus);
free_minor(ocxlpmem);
+ if (ocxlpmem->irq_addr[1])
+ iounmap(ocxlpmem->irq_addr[1]);
+
+ if (ocxlpmem->irq_addr[0])
+ iounmap(ocxlpmem->irq_addr[0]);
+
if (ocxlpmem->cdev.owner)
cdev_del(&ocxlpmem->cdev);
@@ -443,6 +455,11 @@ static int file_release(struct inode *inode, struct file
*file)
{
struct ocxlpmem *ocxlpmem = file->private_data;
+ if (ocxlpmem->ev_ctx) {
+ eventfd_ctx_put(ocxlpmem->ev_ctx);
+ ocxlpmem->ev_ctx = NULL;
+ }
+
ocxlpmem_put(ocxlpmem);
return 0;
}
@@ -938,6 +955,51 @@ static int ioctl_controller_stats(struct ocxlpmem
*ocxlpmem,
return rc;
}
+static int ioctl_eventfd(struct ocxlpmem *ocxlpmem,
+struct ioctl_ocxl_pmem_eventfd __user *uarg)
+{
+ struct ioctl_ocxl_pmem_eventfd args;
+
+ if (copy_from_user(&args, uarg, sizeof(args)))
+ return -EFAULT;
+
+ if (ocxlpmem->ev_ctx)
+ return -EINVAL;
+
+ ocxlpmem->ev_ctx = eventfd_ctx_fdget(args.eventfd);
+ if (!ocxlpmem->ev_ctx)
+ return -EFAULT;
+
+ return 0;
+}
+
+static int ioctl_event_check(struct ocxlpmem *ocxlpmem, u64 __user *uarg)
+{
+ u64 val = 0;
+ int rc;
+ u64 chi = 0;
+
+ rc = ocxlpmem_chi(ocxlpmem, &chi);
+ if (rc < 0)
+ return rc;
+
+ if (chi & GLOBAL_MMIO_CHI_ELA)
+ val |= IOCTL_OCXL_PMEM_EVENT_ERROR_LOG_AVAILABLE;
+
+ if (chi & GLOBAL_MMIO_CHI_CDA)
+ val |= IOCTL_OCXL_PMEM_EVENT_CONTROLLER_DUMP_AVAILABLE;
+
+ if (chi & GLOBAL_MMIO_CHI_CFFS)
+ val |= IOCTL_OCXL_PMEM_EVENT_FIRMWARE_FATAL;
+
+ if (chi & GLOBAL_MMIO_CHI_CHFS)
+ val |= IOCTL_OCXL_PMEM_EVENT_HARDWARE_FATAL;
+
+ rc = copy_to_user((u64 __user *) uarg, &val, sizeof(val));
+
+ return rc;
+}
+
static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args)
{
struct ocxlpmem *ocxlpmem = file->private_data;
@@ -966,6 +1028,15 @@ static long file_ioctl(struct file *file, unsigned int
cmd, unsigned long args)
rc = ioctl_controller_stats(ocxlpmem,
(struct
ioctl_ocxl_pmem_controller_stats __user *)args);
break;
+
+ case IOCTL_OCXL_PMEM_EVENTFD:
+ rc = ioctl_eventfd(ocxlpmem,
+ (struct ioctl_ocxl_pmem_eventfd __user
*)args);
+ break;
+
+ case IOCTL_OCXL_PMEM_EVENT_CHECK:
+ rc = ioctl_event_check(ocxlpmem, (u64 __user *)args);
+ break;
}
return rc;
@@ -1107,6 +1178,146 @@ static void dump_error_log(struct ocxlpmem *ocxlpmem)
kfree(buf);
}
+static irqreturn_t imn0_handler(void *private)
+{
+ struct ocxlpmem *ocxlpmem = private;
+ u64 chi = 0;
+
+ (void)ocxlpmem_chi(ocxlpmem, &chi);
+
+ if (chi & GLOBAL_MMIO_CHI_ELA) {
+ dev_warn(&ocxlpmem->dev, "Error log is available\n");
+
+ if (ocxlpmem->ev_ctx)
+ eventfd_signal(ocxlpmem->ev_ctx, 1);
+ }
+
+ if (chi & GLOBAL_MMIO_CHI_CDA) {
+ dev_warn(&ocxlpmem->dev, "Controller dump is available\n");
+
+ if (ocxlpmem->ev_ctx)
+ eventfd_signal(ocxlpmem->ev_ctx, 1);
+ }
+
+
+ return IRQ_HANDLED;
+}
+
+static ir
[PATCH v3 15/27] powerpc/powernv/pmem: Add support for near storage commands
From: Alastair D'Silva
Similar to the previous patch, this adds support for near storage commands.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c| 6 +++
.../platforms/powernv/pmem/ocxl_internal.c| 41 +++
.../platforms/powernv/pmem/ocxl_internal.h| 37 +
3 files changed, 84 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 4e782d22605b..b8bd7e703b19 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -259,12 +259,18 @@ static int setup_command_metadata(struct ocxlpmem
*ocxlpmem)
int rc;
mutex_init(&ocxlpmem->admin_command.lock);
+ mutex_init(&ocxlpmem->ns_command.lock);
rc = extract_command_metadata(ocxlpmem, GLOBAL_MMIO_ACMA_CREQO,
&ocxlpmem->admin_command);
if (rc)
return rc;
+ rc = extract_command_metadata(ocxlpmem, GLOBAL_MMIO_NSCMA_CREQO,
+ &ocxlpmem->ns_command);
+ if (rc)
+ return rc;
+
return 0;
}
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
index 583f48023025..3e0b133feddf 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
@@ -133,6 +133,47 @@ int admin_response_handled(const struct ocxlpmem *ocxlpmem)
OCXL_LITTLE_ENDIAN, GLOBAL_MMIO_CHI_ACRA);
}
+int ns_command_request(struct ocxlpmem *ocxlpmem, u8 op_code)
+{
+ u64 val;
+ int rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHI,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ if (!(val & GLOBAL_MMIO_CHI_NSCRA))
+ return -EBUSY;
+
+ return scm_command_request(ocxlpmem, &ocxlpmem->ns_command, op_code);
+}
+
+int ns_response(const struct ocxlpmem *ocxlpmem)
+{
+ return command_response(ocxlpmem, &ocxlpmem->ns_command);
+}
+
+int ns_command_execute(const struct ocxlpmem *ocxlpmem)
+{
+ return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI,
+ OCXL_LITTLE_ENDIAN,
GLOBAL_MMIO_HCI_NSCRW);
+}
+
+bool ns_command_complete(const struct ocxlpmem *ocxlpmem)
+{
+ u64 val = 0;
+ int rc = ocxlpmem_chi(ocxlpmem, &val);
+
+ WARN_ON(rc);
+
+ return (val & GLOBAL_MMIO_CHI_NSCRA) != 0;
+}
+
+int ns_response_handled(const struct ocxlpmem *ocxlpmem)
+{
+ return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIC,
+ OCXL_LITTLE_ENDIAN,
GLOBAL_MMIO_CHI_NSCRA);
+}
+
void warn_status(const struct ocxlpmem *ocxlpmem, const char *message,
u8 status)
{
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
index 2fef68c71271..28e2020f6355 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
@@ -107,6 +107,7 @@ struct ocxlpmem {
struct ocxl_context *ocxl_context;
void *metadata_addr;
struct command_metadata admin_command;
+ struct command_metadata ns_command;
struct resource pmem_res;
struct nd_region *nd_region;
char fw_version[8+1];
@@ -175,6 +176,42 @@ int admin_command_complete_timeout(const struct ocxlpmem
*ocxlpmem,
*/
int admin_response_handled(const struct ocxlpmem *ocxlpmem);
+/**
+ * ns_command_request() - Issue a near storage command request
+ * @ocxlpmem: the device metadata
+ * @op_code: The op-code for the command
+ * Returns an identifier for the command, or negative on error
+ */
+int ns_command_request(struct ocxlpmem *ocxlpmem, u8 op_code);
+
+/**
+ * ns_response() - Validate a near storage response
+ * @ocxlpmem: the device metadata
+ * Returns the status code of the command, or negative on error
+ */
+int ns_response(const struct ocxlpmem *ocxlpmem);
+
+/**
+ * ns_command_execute() - Notify the controller to start processing a pending
near storage command
+ * @ocxlpmem: the device metadata
+ * Returns 0 on success, negative on error
+ */
+int ns_command_execute(const struct ocxlpmem *ocxlpmem);
+
+/**
+ * ns_command_complete() - Is a near storage command executing
+ * @ocxlpmem: the device metadata
+ * Returns true if the previous admin command has completed
+ */
+bool ns_command_complete(const struct ocxlpmem *ocxlpmem);
+
+/**
+ * ns_response_handled() - Notify the controller that the near storage
response has been handled
+ * @ocxlpmem: the device metadata
+ * Returns 0 on success, negative on failure
+ */
+int ns_response_handled(const struct ocxlpmem *ocxlpmem);
+
/**
* warn_status() - Emit a kernel warning showing a com
[PATCH v3 06/27] ocxl: Tally up the LPC memory on a link & allow it to be mapped
From: Alastair D'Silva
Tally up the LPC memory on an OpenCAPI link & allow it to be mapped
Signed-off-by: Alastair D'Silva
---
drivers/misc/ocxl/core.c | 10 ++
drivers/misc/ocxl/link.c | 53 +++
drivers/misc/ocxl/ocxl_internal.h | 33 +++
3 files changed, 96 insertions(+)
diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c
index b7a09b21ab36..2531c6cf19a0 100644
--- a/drivers/misc/ocxl/core.c
+++ b/drivers/misc/ocxl/core.c
@@ -230,8 +230,18 @@ static int configure_afu(struct ocxl_afu *afu, u8 afu_idx,
struct pci_dev *dev)
if (rc)
goto err_free_pasid;
+ if (afu->config.lpc_mem_size || afu->config.special_purpose_mem_size) {
+ rc = ocxl_link_add_lpc_mem(afu->fn->link,
afu->config.lpc_mem_offset,
+ afu->config.lpc_mem_size +
+
afu->config.special_purpose_mem_size);
+ if (rc)
+ goto err_free_mmio;
+ }
+
return 0;
+err_free_mmio:
+ unmap_mmio_areas(afu);
err_free_pasid:
reclaim_afu_pasid(afu);
err_free_actag:
diff --git a/drivers/misc/ocxl/link.c b/drivers/misc/ocxl/link.c
index 58d111afd9f6..1e039cc5ebe5 100644
--- a/drivers/misc/ocxl/link.c
+++ b/drivers/misc/ocxl/link.c
@@ -84,6 +84,11 @@ struct ocxl_link {
int dev;
atomic_t irq_available;
struct spa *spa;
+ struct mutex lpc_mem_lock; /* protects lpc_mem & lpc_mem_sz */
+ u64 lpc_mem_sz; /* Total amount of LPC memory presented on the link */
+ u64 lpc_mem;
+ int lpc_consumers;
+
void *platform_data;
};
static struct list_head links_list = LIST_HEAD_INIT(links_list);
@@ -396,6 +401,8 @@ static int alloc_link(struct pci_dev *dev, int PE_mask,
struct ocxl_link **out_l
if (rc)
goto err_spa;
+ mutex_init(&link->lpc_mem_lock);
+
/* platform specific hook */
rc = pnv_ocxl_spa_setup(dev, link->spa->spa_mem, PE_mask,
&link->platform_data);
@@ -711,3 +718,49 @@ void ocxl_link_free_irq(void *link_handle, int hw_irq)
atomic_inc(&link->irq_available);
}
EXPORT_SYMBOL_GPL(ocxl_link_free_irq);
+
+int ocxl_link_add_lpc_mem(void *link_handle, u64 offset, u64 size)
+{
+ struct ocxl_link *link = (struct ocxl_link *) link_handle;
+
+ // Check for overflow
+ if (offset > (offset + size))
+ return -EINVAL;
+
+ mutex_lock(&link->lpc_mem_lock);
+ link->lpc_mem_sz = max(link->lpc_mem_sz, offset + size);
+
+ mutex_unlock(&link->lpc_mem_lock);
+
+ return 0;
+}
+
+u64 ocxl_link_lpc_map(void *link_handle, struct pci_dev *pdev)
+{
+ struct ocxl_link *link = (struct ocxl_link *) link_handle;
+
+ mutex_lock(&link->lpc_mem_lock);
+
+ if(!link->lpc_mem)
+ link->lpc_mem = pnv_ocxl_platform_lpc_setup(pdev,
link->lpc_mem_sz);
+
+ if(link->lpc_mem)
+ link->lpc_consumers++;
+ mutex_unlock(&link->lpc_mem_lock);
+
+ return link->lpc_mem;
+}
+
+void ocxl_link_lpc_release(void *link_handle, struct pci_dev *pdev)
+{
+ struct ocxl_link *link = (struct ocxl_link *) link_handle;
+
+ mutex_lock(&link->lpc_mem_lock);
+ WARN_ON(--link->lpc_consumers < 0);
+ if (link->lpc_consumers == 0) {
+ pnv_ocxl_platform_lpc_release(pdev);
+ link->lpc_mem = 0;
+ }
+
+ mutex_unlock(&link->lpc_mem_lock);
+}
diff --git a/drivers/misc/ocxl/ocxl_internal.h
b/drivers/misc/ocxl/ocxl_internal.h
index 198e4e4bc51d..d0c8c4838f42 100644
--- a/drivers/misc/ocxl/ocxl_internal.h
+++ b/drivers/misc/ocxl/ocxl_internal.h
@@ -142,4 +142,37 @@ int ocxl_irq_offset_to_id(struct ocxl_context *ctx, u64
offset);
u64 ocxl_irq_id_to_offset(struct ocxl_context *ctx, int irq_id);
void ocxl_afu_irq_free_all(struct ocxl_context *ctx);
+/**
+ * ocxl_link_add_lpc_mem() - Increment the amount of memory required by an
OpenCAPI link
+ *
+ * @link_handle: The OpenCAPI link handle
+ * @offset: The offset of the memory to add
+ * @size: The amount of memory to increment by
+ *
+ * Returns 0 on success, negative on overflow
+ */
+int ocxl_link_add_lpc_mem(void *link_handle, u64 offset, u64 size);
+
+/**
+ * ocxl_link_lpc_map() - Map the LPC memory for an OpenCAPI device
+ * Since LPC memory belongs to a link, the whole LPC memory available
+ * on the link must be mapped in order to make it accessible to a device.
+ * @link_handle: The OpenCAPI link handle
+ * @pdev: A device that is on the link
+ *
+ * Returns the address of the mapped LPC memory, or 0 on error
+ */
+u64 ocxl_link_lpc_map(void *link_handle, struct pci_dev *pdev);
+
+/**
+ * ocxl_link_lpc_release() - Release the LPC memory device for an OpenCAPI
device
+ *
+ * Offlines LPC memory on an OpenCAPI link for a device. If this is the
+ * last device on the link to release the memory,
[PATCH v3 25/27] powerpc/powernv/pmem: Expose the serial number in sysfs
From: Alastair D'Silva
This information will be used by ndctl in userspace to help users identify
the device.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/Makefile | 2 +-
arch/powerpc/platforms/powernv/pmem/ocxl.c| 5 +++
.../platforms/powernv/pmem/ocxl_internal.h| 6 +++
.../platforms/powernv/pmem/ocxl_sysfs.c | 37 +++
4 files changed, 49 insertions(+), 1 deletion(-)
create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
diff --git a/arch/powerpc/platforms/powernv/pmem/Makefile
b/arch/powerpc/platforms/powernv/pmem/Makefile
index 4ceda25907d4..d02870806f30 100644
--- a/arch/powerpc/platforms/powernv/pmem/Makefile
+++ b/arch/powerpc/platforms/powernv/pmem/Makefile
@@ -4,4 +4,4 @@ ccflags-$(CONFIG_PPC_WERROR)+= -Werror
obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o
-ocxlpmem-y := ocxl.o ocxl_internal.o
+ocxlpmem-y := ocxl.o ocxl_internal.o ocxl_sysfs.o
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 5cd1b6d78dd6..ec73713d05ad 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -1878,6 +1878,11 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
goto err;
}
+ if (ocxlpmem_sysfs_add(ocxlpmem)) {
+ dev_err(&pdev->dev, "Could not create sysfs entries\n");
+ goto err;
+ }
+
elapsed = 0;
timeout = ocxlpmem->readiness_timeout +
ocxlpmem->memory_available_timeout;
while (!is_usable(ocxlpmem, false)) {
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
index 0eb7a35d24ae..12304ceace61 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.h
@@ -246,3 +246,9 @@ int ns_response_handled(const struct ocxlpmem *ocxlpmem);
*/
void warn_status(const struct ocxlpmem *ocxlpmem, const char *message,
u8 status);
+
+/**
+ * ocxlpmem_sysfs_add() - Create sysfs entries for an OpenCAPI persistent
memory device
+ * @ocxlpmem: the device metadata
+ */
+int ocxlpmem_sysfs_add(struct ocxlpmem *ocxlpmem);
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
b/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
new file mode 100644
index ..7829e4bc887d
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
@@ -0,0 +1,37 @@
+// SPDX-License-Identifier: GPL-2.0+
+// Copyright 2018 IBM Corp.
+
+#include
+#include
+#include
+#include
+#include "ocxl_internal.h"
+
+static ssize_t serial_show(struct device *device, struct device_attribute
*attr,
+ char *buf)
+{
+ struct ocxlpmem *ocxlpmem = container_of(device, struct ocxlpmem, dev);
+ const struct ocxl_fn_config *fn_config =
ocxl_function_config(ocxlpmem->ocxl_fn);
+
+ return scnprintf(buf, PAGE_SIZE, "%llu\n", fn_config->serial);
+}
+
+static struct device_attribute attrs[] = {
+ __ATTR_RO(serial),
+};
+
+int ocxlpmem_sysfs_add(struct ocxlpmem *ocxlpmem)
+{
+ int i, rc;
+
+ for (i = 0; i < ARRAY_SIZE(attrs); i++) {
+ rc = device_create_file(&ocxlpmem->dev, &attrs[i]);
+ if (rc) {
+ for (; --i >= 0;)
+ device_remove_file(&ocxlpmem->dev, &attrs[i]);
+
+ return rc;
+ }
+ }
+ return 0;
+}
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 22/27] powerpc/powernv/pmem: Implement the heartbeat command
From: Alastair D'Silva
The heartbeat admin command is a simple admin command that exercises
the communication mechanisms within the controller.
This patch issues a heartbeat command to the card during init to ensure
we can communicate with the card's controller.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c | 43 ++
1 file changed, 43 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 081883a8247a..e01f6f9fc180 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -306,6 +306,44 @@ static bool is_usable(const struct ocxlpmem *ocxlpmem,
bool verbose)
return true;
}
+/**
+ * heartbeat() - Issue a heartbeat command to the controller
+ * @ocxlpmem: the device metadata
+ * Return: 0 if the controller responded correctly, negative on error
+ */
+static int heartbeat(struct ocxlpmem *ocxlpmem)
+{
+ int rc;
+
+ mutex_lock(&ocxlpmem->admin_command.lock);
+
+ rc = admin_command_request(ocxlpmem, ADMIN_COMMAND_HEARTBEAT);
+ if (rc)
+ goto out;
+
+ rc = admin_command_execute(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = admin_command_complete_timeout(ocxlpmem, ADMIN_COMMAND_HEARTBEAT);
+ if (rc < 0) {
+ dev_err(&ocxlpmem->dev, "Heartbeat timeout\n");
+ goto out;
+ }
+
+ rc = admin_response(ocxlpmem);
+ if (rc < 0)
+ goto out;
+ if (rc != STATUS_SUCCESS)
+ warn_status(ocxlpmem, "Unexpected status from heartbeat", rc);
+
+ (void)admin_response_handled(ocxlpmem);
+
+out:
+ mutex_unlock(&ocxlpmem->admin_command.lock);
+ return rc;
+}
+
/**
* allocate_minor() - Allocate a minor number to use for an OpenCAPI pmem
device
* @ocxlpmem: the device metadata
@@ -1458,6 +1496,11 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
goto err;
}
+ if (heartbeat(ocxlpmem)) {
+ dev_err(&pdev->dev, "Heartbeat failed\n");
+ goto err;
+ }
+
elapsed = 0;
timeout = ocxlpmem->readiness_timeout +
ocxlpmem->memory_available_timeout;
while (!is_usable(ocxlpmem, false)) {
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 18/27] powerpc/powernv/pmem: Add controller dump IOCTLs
From: Alastair D'Silva
This patch adds IOCTLs to allow userspace to request & fetch dumps
of the internal controller state.
This is useful during debugging or when a fatal error on the controller
has occurred.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c | 132 +
include/uapi/nvdimm/ocxl-pmem.h| 15 +++
2 files changed, 147 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 2b64504f9129..2cabafe1fc58 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -640,6 +640,124 @@ static int ioctl_error_log(struct ocxlpmem *ocxlpmem,
return 0;
}
+static int ioctl_controller_dump_data(struct ocxlpmem *ocxlpmem,
+ struct ioctl_ocxl_pmem_controller_dump_data __user *uarg)
+{
+ struct ioctl_ocxl_pmem_controller_dump_data args;
+ u16 i;
+ u64 val;
+ int rc;
+
+ if (copy_from_user(&args, uarg, sizeof(args)))
+ return -EFAULT;
+
+ if (args.buf_size % 8)
+ return -EINVAL;
+
+ if (args.buf_size > ocxlpmem->admin_command.data_size)
+ return -EINVAL;
+
+ mutex_lock(&ocxlpmem->admin_command.lock);
+
+ rc = admin_command_request(ocxlpmem, ADMIN_COMMAND_CONTROLLER_DUMP);
+ if (rc)
+ goto out;
+
+ val = ((u64)args.offset) << 32;
+ val |= args.buf_size;
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+ ocxlpmem->admin_command.request_offset +
0x08,
+ OCXL_LITTLE_ENDIAN, val);
+ if (rc)
+ goto out;
+
+ rc = admin_command_execute(ocxlpmem);
+ if (rc)
+ goto out;
+
+ rc = admin_command_complete_timeout(ocxlpmem,
+ ADMIN_COMMAND_CONTROLLER_DUMP);
+ if (rc < 0) {
+ dev_warn(&ocxlpmem->dev, "Controller dump timed out\n");
+ goto out;
+ }
+
+ rc = admin_response(ocxlpmem);
+ if (rc < 0)
+ goto out;
+ if (rc != STATUS_SUCCESS) {
+ warn_status(ocxlpmem,
+ "Unexpected status from retrieve error log",
+ rc);
+ goto out;
+ }
+
+ for (i = 0; i < args.buf_size; i += 8) {
+ u64 val;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+
ocxlpmem->admin_command.data_offset + i,
+OCXL_HOST_ENDIAN, &val);
+ if (rc)
+ goto out;
+
+ if (copy_to_user(&args.buf[i], &val, sizeof(u64))) {
+ rc = -EFAULT;
+ goto out;
+ }
+ }
+
+ if (copy_to_user(uarg, &args, sizeof(args))) {
+ rc = -EFAULT;
+ goto out;
+ }
+
+ rc = admin_response_handled(ocxlpmem);
+ if (rc)
+ goto out;
+
+out:
+ mutex_unlock(&ocxlpmem->admin_command.lock);
+ return rc;
+}
+
+int request_controller_dump(struct ocxlpmem *ocxlpmem)
+{
+ int rc;
+ u64 busy = 1;
+
+ rc = ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CHIC,
+ OCXL_LITTLE_ENDIAN,
+ GLOBAL_MMIO_CHI_CDA);
+
+
+ rc = ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI,
+ OCXL_LITTLE_ENDIAN,
+ GLOBAL_MMIO_HCI_CONTROLLER_DUMP);
+ if (rc)
+ return rc;
+
+ while (busy) {
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu,
+GLOBAL_MMIO_HCI,
+OCXL_LITTLE_ENDIAN, &busy);
+ if (rc)
+ return rc;
+
+ busy &= GLOBAL_MMIO_HCI_CONTROLLER_DUMP;
+ cond_resched();
+ }
+
+ return 0;
+}
+
+static int ioctl_controller_dump_complete(struct ocxlpmem *ocxlpmem)
+{
+ return ocxl_global_mmio_set64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_HCI,
+ OCXL_LITTLE_ENDIAN,
+ GLOBAL_MMIO_HCI_CONTROLLER_DUMP_COLLECTED);
+}
+
static long file_ioctl(struct file *file, unsigned int cmd, unsigned long args)
{
struct ocxlpmem *ocxlpmem = file->private_data;
@@ -650,7 +768,21 @@ static long file_ioctl(struct file *file, unsigned int
cmd, unsigned long args)
rc = ioctl_error_log(ocxlpmem,
(struct ioctl_ocxl_pmem_error_log __user
*)args);
break;
+
+ case IOCTL_OCXL_PMEM_CONTROLLER_DUMP:
+ rc = request_controller_dump(ocxlpmem);
+ break;
+
+
[PATCH v3 27/27] MAINTAINERS: Add myself & nvdimm/ocxl to ocxl
From: Alastair D'Silva The OpenCAPI Persistent Memory driver will be maintained as part ofi the ppc tree. I'm also adding myself as an author of the driver & contributor to the generic ocxl driver. Signed-off-by: Alastair D'Silva --- MAINTAINERS | 3 +++ 1 file changed, 3 insertions(+) diff --git a/MAINTAINERS b/MAINTAINERS index f8670989ec91..3fb9a9f576a7 100644 --- a/MAINTAINERS +++ b/MAINTAINERS @@ -12064,13 +12064,16 @@ F:tools/objtool/ OCXL (Open Coherent Accelerator Processor Interface OpenCAPI) DRIVER M: Frederic Barrat M: Andrew Donnellan +M: Alastair D'Silva L: linuxppc-...@lists.ozlabs.org S: Supported F: arch/powerpc/platforms/powernv/ocxl.c +F: arch/powerpc/platforms/powernv/pmem/* F: arch/powerpc/include/asm/pnv-ocxl.h F: drivers/misc/ocxl/ F: include/misc/ocxl* F: include/uapi/misc/ocxl.h +F: include/uapi/nvdimm/ocxl-pmem.h F: Documentation/userspace-api/accelerators/ocxl.rst OMAP AUDIO SUPPORT -- 2.24.1 ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 13/27] powerpc/powernv/pmem: Read the capability registers & wait for device ready
From: Alastair D'Silva
This patch reads timeouts & firmware version from the controller, and
uses those timeouts to wait for the controller to report that it is ready
before handing the memory over to libnvdimm.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/Makefile | 2 +-
arch/powerpc/platforms/powernv/pmem/ocxl.c| 92 +++
.../platforms/powernv/pmem/ocxl_internal.c| 19
.../platforms/powernv/pmem/ocxl_internal.h| 24 +
4 files changed, 136 insertions(+), 1 deletion(-)
create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
diff --git a/arch/powerpc/platforms/powernv/pmem/Makefile
b/arch/powerpc/platforms/powernv/pmem/Makefile
index 1c55c4193175..4ceda25907d4 100644
--- a/arch/powerpc/platforms/powernv/pmem/Makefile
+++ b/arch/powerpc/platforms/powernv/pmem/Makefile
@@ -4,4 +4,4 @@ ccflags-$(CONFIG_PPC_WERROR)+= -Werror
obj-$(CONFIG_OCXL_PMEM) += ocxlpmem.o
-ocxlpmem-y := ocxl.o
+ocxlpmem-y := ocxl.o ocxl_internal.o
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 3c4eeb5dcc0f..431212c9f0cc 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -8,6 +8,7 @@
#include
#include
+#include
#include
#include
#include
@@ -215,6 +216,36 @@ static int register_lpc_mem(struct ocxlpmem *ocxlpmem)
return 0;
}
+/**
+ * is_usable() - Is a controller usable?
+ * @ocxlpmem: the device metadata
+ * @verbose: True to log errors
+ * Return: true if the controller is usable
+ */
+static bool is_usable(const struct ocxlpmem *ocxlpmem, bool verbose)
+{
+ u64 chi = 0;
+ int rc = ocxlpmem_chi(ocxlpmem, &chi);
+
+ if (rc < 0)
+ return false;
+
+ if (!(chi & GLOBAL_MMIO_CHI_CRDY)) {
+ if (verbose)
+ dev_err(&ocxlpmem->dev, "controller is not ready.\n");
+ return false;
+ }
+
+ if (!(chi & GLOBAL_MMIO_CHI_MA)) {
+ if (verbose)
+ dev_err(&ocxlpmem->dev,
+ "controller does not have memory available.\n");
+ return false;
+ }
+
+ return true;
+}
+
/**
* allocate_minor() - Allocate a minor number to use for an OpenCAPI pmem
device
* @ocxlpmem: the device metadata
@@ -328,6 +359,48 @@ static void ocxlpmem_remove(struct pci_dev *pdev)
}
}
+/**
+ * read_device_metadata() - Retrieve config information from the AFU and save
it for future use
+ * @ocxlpmem: the device metadata
+ * Return: 0 on success, negative on failure
+ */
+static int read_device_metadata(struct ocxlpmem *ocxlpmem)
+{
+ u64 val;
+ int rc;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CCAP0,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ ocxlpmem->scm_revision = val & 0x;
+ ocxlpmem->read_latency = (val >> 32) & 0xFF;
+ ocxlpmem->readiness_timeout = (val >> 48) & 0x0F;
+ ocxlpmem->memory_available_timeout = val >> 52;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_CCAP1,
+OCXL_LITTLE_ENDIAN, &val);
+ if (rc)
+ return rc;
+
+ ocxlpmem->max_controller_dump_size = val & 0x;
+
+ // Extract firmware version text
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, GLOBAL_MMIO_FWVER,
+OCXL_HOST_ENDIAN, (u64
*)ocxlpmem->fw_version);
+ if (rc)
+ return rc;
+
+ ocxlpmem->fw_version[8] = '\0';
+
+ dev_info(&ocxlpmem->dev,
+"Firmware version '%s' SCM revision %d:%d\n",
ocxlpmem->fw_version,
+ocxlpmem->scm_revision >> 4, ocxlpmem->scm_revision & 0x0F);
+
+ return 0;
+}
+
/**
* probe_function0() - Set up function 0 for an OpenCAPI persistent memory
device
* This is important as it enables templates higher than 0 across all other
functions,
@@ -368,6 +441,7 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
{
struct ocxlpmem *ocxlpmem;
int rc;
+ u16 elapsed, timeout;
if (PCI_FUNC(pdev->devfn) == 0)
return probe_function0(pdev);
@@ -422,6 +496,24 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
goto err;
}
+ if (read_device_metadata(ocxlpmem)) {
+ dev_err(&pdev->dev, "Could not read metadata\n");
+ goto err;
+ }
+
+ elapsed = 0;
+ timeout = ocxlpmem->readiness_timeout +
ocxlpmem->memory_available_timeout;
+ while (!is_usable(ocxlpmem, false)) {
+ if (elapsed++ > timeout) {
+ dev_warn(&ocxlpmem->dev, "OpenCAPI Persistent Memory
ready timeout.\n");
+ (void)is_usable(ocxlpme
[PATCH v3 11/27] powerpc: Enable the OpenCAPI Persistent Memory driver for powernv_defconfig
From: Alastair D'Silva This patch enables the OpenCAPI Persistent Memory driver, as well as DAX support, for the 'powernv' platform. DAX is not a strict requirement for the functioning of the driver, but it is likely that a user will want to create a DAX device on top of their persistent memory device. Signed-off-by: Alastair D'Silva --- arch/powerpc/configs/powernv_defconfig | 5 + 1 file changed, 5 insertions(+) diff --git a/arch/powerpc/configs/powernv_defconfig b/arch/powerpc/configs/powernv_defconfig index 71749377d164..921d77bbd3d2 100644 --- a/arch/powerpc/configs/powernv_defconfig +++ b/arch/powerpc/configs/powernv_defconfig @@ -348,3 +348,8 @@ CONFIG_KVM_BOOK3S_64=m CONFIG_KVM_BOOK3S_64_HV=m CONFIG_VHOST_NET=m CONFIG_PRINTK_TIME=y +CONFIG_ZONE_DEVICE=y +CONFIG_OCXL_PMEM=m +CONFIG_DEV_DAX=m +CONFIG_DEV_DAX_PMEM=m +CONFIG_FS_DAX=y -- 2.24.1 ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 00/27] Add support for OpenCAPI Persistent Memory devices
From: Alastair D'Silva This series adds support for OpenCAPI Persistent Memory devices, exposing them as nvdimms so that we can make use of the existing infrastructure. Alastair D'Silva (27): powerpc: Add OPAL calls for LPC memory alloc/release mm/memory_hotplug: Allow check_hotplug_memory_addressable to be called from drivers powerpc: Map & release OpenCAPI LPC memory ocxl: Remove unnecessary externs ocxl: Address kernel doc errors & warnings ocxl: Tally up the LPC memory on a link & allow it to be mapped ocxl: Add functions to map/unmap LPC memory ocxl: Emit a log message showing how much LPC memory was detected ocxl: Save the device serial number in ocxl_fn powerpc: Add driver for OpenCAPI Persistent Memory powerpc: Enable the OpenCAPI Persistent Memory driver for powernv_defconfig powerpc/powernv/pmem: Add register addresses & status values to the header powerpc/powernv/pmem: Read the capability registers & wait for device ready powerpc/powernv/pmem: Add support for Admin commands powerpc/powernv/pmem: Add support for near storage commands powerpc/powernv/pmem: Register a character device for userspace to interact with powerpc/powernv/pmem: Implement the Read Error Log command powerpc/powernv/pmem: Add controller dump IOCTLs powerpc/powernv/pmem: Add an IOCTL to report controller statistics powerpc/powernv/pmem: Forward events to userspace powerpc/powernv/pmem: Add an IOCTL to request controller health & perf data powerpc/powernv/pmem: Implement the heartbeat command powerpc/powernv/pmem: Add debug IOCTLs powerpc/powernv/pmem: Expose SMART data via ndctl powerpc/powernv/pmem: Expose the serial number in sysfs powerpc/powernv/pmem: Expose the firmware version in sysfs MAINTAINERS: Add myself & nvdimm/ocxl to ocxl MAINTAINERS |3 + arch/powerpc/configs/powernv_defconfig|5 + arch/powerpc/include/asm/opal-api.h |2 + arch/powerpc/include/asm/opal.h |3 + arch/powerpc/include/asm/pnv-ocxl.h | 40 +- arch/powerpc/platforms/powernv/Kconfig|3 + arch/powerpc/platforms/powernv/Makefile |1 + arch/powerpc/platforms/powernv/ocxl.c | 43 + arch/powerpc/platforms/powernv/opal-call.c|2 + arch/powerpc/platforms/powernv/pmem/Kconfig | 21 + arch/powerpc/platforms/powernv/pmem/Makefile |7 + arch/powerpc/platforms/powernv/pmem/ocxl.c| 1991 + .../platforms/powernv/pmem/ocxl_internal.c| 213 ++ .../platforms/powernv/pmem/ocxl_internal.h| 254 +++ .../platforms/powernv/pmem/ocxl_sysfs.c | 46 + drivers/misc/ocxl/config.c| 74 +- drivers/misc/ocxl/core.c | 61 + drivers/misc/ocxl/link.c | 53 + drivers/misc/ocxl/ocxl_internal.h | 45 +- include/linux/memory_hotplug.h|5 + include/misc/ocxl.h | 122 +- include/uapi/linux/ndctl.h|1 + include/uapi/nvdimm/ocxl-pmem.h | 127 ++ mm/memory_hotplug.c |4 +- 24 files changed, 3029 insertions(+), 97 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/pmem/Kconfig create mode 100644 arch/powerpc/platforms/powernv/pmem/Makefile create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl.c create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl_internal.c create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl_internal.h create mode 100644 arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c create mode 100644 include/uapi/nvdimm/ocxl-pmem.h -- 2.24.1 ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 09/27] ocxl: Save the device serial number in ocxl_fn
From: Alastair D'Silva
This patch retrieves the serial number of the card and makes it available
to consumers of the ocxl driver via the ocxl_fn struct.
Signed-off-by: Alastair D'Silva
Acked-by: Frederic Barrat
Acked-by: Andrew Donnellan
---
drivers/misc/ocxl/config.c | 46 ++
include/misc/ocxl.h| 1 +
2 files changed, 47 insertions(+)
diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c
index 701ae6216abf..ce33fafa7b50 100644
--- a/drivers/misc/ocxl/config.c
+++ b/drivers/misc/ocxl/config.c
@@ -71,6 +71,51 @@ static int find_dvsec_afu_ctrl(struct pci_dev *dev, u8
afu_idx)
return 0;
}
+/**
+ * get_function_0() - Find a related PCI device (function 0)
+ * @device: PCI device to match
+ *
+ * Returns a pointer to the related device, or null if not found
+ */
+static struct pci_dev *get_function_0(struct pci_dev *dev)
+{
+ unsigned int devfn = PCI_DEVFN(PCI_SLOT(dev->devfn), 0);
+
+ return pci_get_domain_bus_and_slot(pci_domain_nr(dev->bus),
+ dev->bus->number, devfn);
+}
+
+static void read_serial(struct pci_dev *dev, struct ocxl_fn_config *fn)
+{
+ u32 low, high;
+ int pos;
+
+ pos = pci_find_ext_capability(dev, PCI_EXT_CAP_ID_DSN);
+ if (pos) {
+ pci_read_config_dword(dev, pos + 0x04, &low);
+ pci_read_config_dword(dev, pos + 0x08, &high);
+
+ fn->serial = low | ((u64)high) << 32;
+
+ return;
+ }
+
+ if (PCI_FUNC(dev->devfn) != 0) {
+ struct pci_dev *related = get_function_0(dev);
+
+ if (!related) {
+ fn->serial = 0;
+ return;
+ }
+
+ read_serial(related, fn);
+ pci_dev_put(related);
+ return;
+ }
+
+ fn->serial = 0;
+}
+
static void read_pasid(struct pci_dev *dev, struct ocxl_fn_config *fn)
{
u16 val;
@@ -208,6 +253,7 @@ int ocxl_config_read_function(struct pci_dev *dev, struct
ocxl_fn_config *fn)
int rc;
read_pasid(dev, fn);
+ read_serial(dev, fn);
rc = read_dvsec_tl(dev, fn);
if (rc) {
diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h
index d8b0b4d46bfb..b8514dc64bd0 100644
--- a/include/misc/ocxl.h
+++ b/include/misc/ocxl.h
@@ -46,6 +46,7 @@ struct ocxl_fn_config {
int dvsec_afu_info_pos; /* offset of the AFU information DVSEC */
s8 max_pasid_log;
s8 max_afu_index;
+ u64 serial;
};
enum ocxl_endian {
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 26/27] powerpc/powernv/pmem: Expose the firmware version in sysfs
From: Alastair D'Silva
This information will be used by ndctl in userspace to help users identify
the device.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c | 9 +
1 file changed, 9 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
b/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
index 7829e4bc887d..84b23cc3e8b7 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_sysfs.c
@@ -16,8 +16,17 @@ static ssize_t serial_show(struct device *device, struct
device_attribute *attr,
return scnprintf(buf, PAGE_SIZE, "%llu\n", fn_config->serial);
}
+static ssize_t fw_version_show(struct device *device,
+ struct device_attribute *attr, char *buf)
+{
+ struct ocxlpmem *ocxlpmem = container_of(device, struct ocxlpmem, dev);
+
+ return scnprintf(buf, PAGE_SIZE, "%s\n", ocxlpmem->fw_version);
+}
+
static struct device_attribute attrs[] = {
__ATTR_RO(serial),
+ __ATTR_RO(fw_version),
};
int ocxlpmem_sysfs_add(struct ocxlpmem *ocxlpmem)
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 02/27] mm/memory_hotplug: Allow check_hotplug_memory_addressable to be called from drivers
From: Alastair D'Silva
When setting up OpenCAPI connected persistent memory, the range check may
not be performed until quite late (or perhaps not at all, if the user does
not establish a DAX device).
This patch makes the range check callable so we can perform the check while
probing the OpenCAPI SCM device.
Signed-off-by: Alastair D'Silva
---
include/linux/memory_hotplug.h | 5 +
mm/memory_hotplug.c| 4 ++--
2 files changed, 7 insertions(+), 2 deletions(-)
diff --git a/include/linux/memory_hotplug.h b/include/linux/memory_hotplug.h
index f4d59155f3d4..34a69aecc45e 100644
--- a/include/linux/memory_hotplug.h
+++ b/include/linux/memory_hotplug.h
@@ -337,6 +337,11 @@ static inline void __remove_memory(int nid, u64 start, u64
size) {}
extern void set_zone_contiguous(struct zone *zone);
extern void clear_zone_contiguous(struct zone *zone);
+#ifdef CONFIG_MEMORY_HOTPLUG_SPARSE
+int check_hotplug_memory_addressable(unsigned long pfn,
+ unsigned long nr_pages);
+#endif /* CONFIG_MEMORY_HOTPLUG_SPARSE */
+
extern void __ref free_area_init_core_hotplug(int nid);
extern int __add_memory(int nid, u64 start, u64 size);
extern int add_memory(int nid, u64 start, u64 size);
diff --git a/mm/memory_hotplug.c b/mm/memory_hotplug.c
index 0a54ffac8c68..14945f033594 100644
--- a/mm/memory_hotplug.c
+++ b/mm/memory_hotplug.c
@@ -276,8 +276,8 @@ static int check_pfn_span(unsigned long pfn, unsigned long
nr_pages,
return 0;
}
-static int check_hotplug_memory_addressable(unsigned long pfn,
- unsigned long nr_pages)
+int check_hotplug_memory_addressable(unsigned long pfn,
+unsigned long nr_pages)
{
const u64 max_addr = PFN_PHYS(pfn + nr_pages) - 1;
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 08/27] ocxl: Emit a log message showing how much LPC memory was detected
From: Alastair D'Silva This patch emits a message showing how much LPC memory & special purpose memory was detected on an OCXL device. Signed-off-by: Alastair D'Silva --- drivers/misc/ocxl/config.c | 4 1 file changed, 4 insertions(+) diff --git a/drivers/misc/ocxl/config.c b/drivers/misc/ocxl/config.c index a62e3d7db2bf..701ae6216abf 100644 --- a/drivers/misc/ocxl/config.c +++ b/drivers/misc/ocxl/config.c @@ -568,6 +568,10 @@ static int read_afu_lpc_memory_info(struct pci_dev *dev, afu->special_purpose_mem_size = total_mem_size - lpc_mem_size; } + + dev_info(&dev->dev, "Probed LPC memory of %#llx bytes and special purpose memory of %#llx bytes\n", + afu->lpc_mem_size, afu->special_purpose_mem_size); + return 0; } -- 2.24.1 ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH v3 14/27] powerpc/powernv/pmem: Add support for Admin commands
From: Alastair D'Silva
This patch requests the metadata required to issue admin commands, as well
as some helper functions to construct and check the completion of the
commands.
Signed-off-by: Alastair D'Silva
---
arch/powerpc/platforms/powernv/pmem/ocxl.c| 65
.../platforms/powernv/pmem/ocxl_internal.c| 153 ++
.../platforms/powernv/pmem/ocxl_internal.h| 61 +++
3 files changed, 279 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl.c
b/arch/powerpc/platforms/powernv/pmem/ocxl.c
index 431212c9f0cc..4e782d22605b 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl.c
@@ -216,6 +216,58 @@ static int register_lpc_mem(struct ocxlpmem *ocxlpmem)
return 0;
}
+/**
+ * extract_command_metadata() - Extract command data from MMIO & save it for
further use
+ * @ocxlpmem: the device metadata
+ * @offset: The base address of the command data structures (address of CREQO)
+ * @command_metadata: A pointer to the command metadata to populate
+ * Return: 0 on success, negative on failure
+ */
+static int extract_command_metadata(struct ocxlpmem *ocxlpmem, u32 offset,
+ struct command_metadata
*command_metadata)
+{
+ int rc;
+ u64 tmp;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, offset,
OCXL_LITTLE_ENDIAN,
+&tmp);
+ if (rc)
+ return rc;
+
+ command_metadata->request_offset = tmp >> 32;
+ command_metadata->response_offset = tmp & 0x;
+
+ rc = ocxl_global_mmio_read64(ocxlpmem->ocxl_afu, offset + 8,
OCXL_LITTLE_ENDIAN,
+&tmp);
+ if (rc)
+ return rc;
+
+ command_metadata->data_offset = tmp >> 32;
+ command_metadata->data_size = tmp & 0x;
+
+ command_metadata->id = 0;
+
+ return 0;
+}
+
+/**
+ * setup_command_metadata() - Set up the command metadata
+ * @ocxlpmem: the device metadata
+ */
+static int setup_command_metadata(struct ocxlpmem *ocxlpmem)
+{
+ int rc;
+
+ mutex_init(&ocxlpmem->admin_command.lock);
+
+ rc = extract_command_metadata(ocxlpmem, GLOBAL_MMIO_ACMA_CREQO,
+ &ocxlpmem->admin_command);
+ if (rc)
+ return rc;
+
+ return 0;
+}
+
/**
* is_usable() - Is a controller usable?
* @ocxlpmem: the device metadata
@@ -456,6 +508,14 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
}
ocxlpmem->pdev = pdev;
+ ocxlpmem->timeouts[ADMIN_COMMAND_ERRLOG] = 2000; // ms
+ ocxlpmem->timeouts[ADMIN_COMMAND_HEARTBEAT] = 100; // ms
+ ocxlpmem->timeouts[ADMIN_COMMAND_SMART] = 100; // ms
+ ocxlpmem->timeouts[ADMIN_COMMAND_CONTROLLER_DUMP] = 1000; // ms
+ ocxlpmem->timeouts[ADMIN_COMMAND_CONTROLLER_STATS] = 100; // ms
+ ocxlpmem->timeouts[ADMIN_COMMAND_SHUTDOWN] = 1000; // ms
+ ocxlpmem->timeouts[ADMIN_COMMAND_FW_UPDATE] = 16000; // ms
+
pci_set_drvdata(pdev, ocxlpmem);
ocxlpmem->ocxl_fn = ocxl_function_open(pdev);
@@ -501,6 +561,11 @@ static int probe(struct pci_dev *pdev, const struct
pci_device_id *ent)
goto err;
}
+ if (setup_command_metadata(ocxlpmem)) {
+ dev_err(&pdev->dev, "Could not read OCXL command matada\n");
+ goto err;
+ }
+
elapsed = 0;
timeout = ocxlpmem->readiness_timeout +
ocxlpmem->memory_available_timeout;
while (!is_usable(ocxlpmem, false)) {
diff --git a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
index 617ca943b1b8..583f48023025 100644
--- a/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
+++ b/arch/powerpc/platforms/powernv/pmem/ocxl_internal.c
@@ -17,3 +17,156 @@ int ocxlpmem_chi(const struct ocxlpmem *ocxlpmem, u64 *chi)
return 0;
}
+
+#define COMMAND_REQUEST_SIZE (8 * sizeof(u64))
+static int scm_command_request(const struct ocxlpmem *ocxlpmem,
+ struct command_metadata *cmd, u8 op_code)
+{
+ u64 val = op_code;
+ int rc;
+ u8 i;
+
+ cmd->op_code = op_code;
+ cmd->id++;
+
+ val |= ((u64)cmd->id) << 16;
+
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu, cmd->request_offset,
+ OCXL_LITTLE_ENDIAN, val);
+ if (rc)
+ return rc;
+
+ for (i = sizeof(u64); i < COMMAND_REQUEST_SIZE; i += sizeof(u64)) {
+ rc = ocxl_global_mmio_write64(ocxlpmem->ocxl_afu,
+ cmd->request_offset + i,
+ OCXL_LITTLE_ENDIAN, 0);
+ if (rc)
+ return rc;
+ }
+
+ return 0;
+}
+
+int admin_command_request(struct ocxlpmem *ocxlpmem, u8 op_code)
+{
+ u64
[PATCH v3 07/27] ocxl: Add functions to map/unmap LPC memory
From: Alastair D'Silva
Add functions to map/unmap LPC memory
Signed-off-by: Alastair D'Silva
---
drivers/misc/ocxl/core.c | 51 +++
drivers/misc/ocxl/ocxl_internal.h | 3 ++
include/misc/ocxl.h | 21 +
3 files changed, 75 insertions(+)
diff --git a/drivers/misc/ocxl/core.c b/drivers/misc/ocxl/core.c
index 2531c6cf19a0..75ff14e3882a 100644
--- a/drivers/misc/ocxl/core.c
+++ b/drivers/misc/ocxl/core.c
@@ -210,6 +210,56 @@ static void unmap_mmio_areas(struct ocxl_afu *afu)
release_fn_bar(afu->fn, afu->config.global_mmio_bar);
}
+int ocxl_afu_map_lpc_mem(struct ocxl_afu *afu)
+{
+ struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent);
+
+ if ((afu->config.lpc_mem_size + afu->config.special_purpose_mem_size)
== 0)
+ return 0;
+
+ afu->lpc_base_addr = ocxl_link_lpc_map(afu->fn->link, dev);
+ if (afu->lpc_base_addr == 0)
+ return -EINVAL;
+
+ if (afu->config.lpc_mem_size > 0) {
+ afu->lpc_res.start = afu->lpc_base_addr +
afu->config.lpc_mem_offset;
+ afu->lpc_res.end = afu->lpc_res.start +
afu->config.lpc_mem_size - 1;
+ }
+
+ if (afu->config.special_purpose_mem_size > 0) {
+ afu->special_purpose_res.start = afu->lpc_base_addr +
+
afu->config.special_purpose_mem_offset;
+ afu->special_purpose_res.end = afu->special_purpose_res.start +
+
afu->config.special_purpose_mem_size - 1;
+ }
+
+ return 0;
+}
+EXPORT_SYMBOL_GPL(ocxl_afu_map_lpc_mem);
+
+struct resource *ocxl_afu_lpc_mem(struct ocxl_afu *afu)
+{
+ return &afu->lpc_res;
+}
+EXPORT_SYMBOL_GPL(ocxl_afu_lpc_mem);
+
+static void unmap_lpc_mem(struct ocxl_afu *afu)
+{
+ struct pci_dev *dev = to_pci_dev(afu->fn->dev.parent);
+
+ if (afu->lpc_res.start || afu->special_purpose_res.start) {
+ void *link = afu->fn->link;
+
+ // only release the link when the the last consumer calls
release
+ ocxl_link_lpc_release(link, dev);
+
+ afu->lpc_res.start = 0;
+ afu->lpc_res.end = 0;
+ afu->special_purpose_res.start = 0;
+ afu->special_purpose_res.end = 0;
+ }
+}
+
static int configure_afu(struct ocxl_afu *afu, u8 afu_idx, struct pci_dev *dev)
{
int rc;
@@ -251,6 +301,7 @@ static int configure_afu(struct ocxl_afu *afu, u8 afu_idx,
struct pci_dev *dev)
static void deconfigure_afu(struct ocxl_afu *afu)
{
+ unmap_lpc_mem(afu);
unmap_mmio_areas(afu);
reclaim_afu_pasid(afu);
reclaim_afu_actag(afu);
diff --git a/drivers/misc/ocxl/ocxl_internal.h
b/drivers/misc/ocxl/ocxl_internal.h
index d0c8c4838f42..ce0cac1da416 100644
--- a/drivers/misc/ocxl/ocxl_internal.h
+++ b/drivers/misc/ocxl/ocxl_internal.h
@@ -52,6 +52,9 @@ struct ocxl_afu {
void __iomem *global_mmio_ptr;
u64 pp_mmio_start;
void *private;
+ u64 lpc_base_addr; /* Covers both LPC & special purpose memory */
+ struct resource lpc_res;
+ struct resource special_purpose_res;
};
enum ocxl_context_status {
diff --git a/include/misc/ocxl.h b/include/misc/ocxl.h
index 357ef1aadbc0..d8b0b4d46bfb 100644
--- a/include/misc/ocxl.h
+++ b/include/misc/ocxl.h
@@ -203,6 +203,27 @@ int ocxl_irq_set_handler(struct ocxl_context *ctx, int
irq_id,
// AFU Metadata
+/**
+ * ocxl_afu_map_lpc_mem() - Map the LPC system & special purpose memory for an
AFU
+ * Do not call this during device discovery, as there may me multiple
+ * devices on a link, and the memory is mapped for the whole link, not
+ * just one device. It should only be called after all devices have
+ * registered their memory on the link.
+ *
+ * @afu: The AFU that has the LPC memory to map
+ *
+ * Returns 0 on success, negative on failure
+ */
+int ocxl_afu_map_lpc_mem(struct ocxl_afu *afu);
+
+/**
+ * ocxl_afu_lpc_mem() - Get the physical address range of LPC memory for an AFU
+ * @afu: The AFU associated with the LPC memory
+ *
+ * Returns a pointer to the resource struct for the physical address range
+ */
+struct resource *ocxl_afu_lpc_mem(struct ocxl_afu *afu);
+
/**
* ocxl_afu_config() - Get a pointer to the config for an AFU
* @afu: a pointer to the AFU to get the config for
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v5 2/8] drivers/pmem: Allow pmem_clear_poison() to accept arbitrary offset and len
On Thu, Feb 20, 2020 at 04:35:17PM -0500, Jeff Moyer wrote:
> Vivek Goyal writes:
>
> > Currently pmem_clear_poison() expects offset and len to be sector aligned.
> > Atleast that seems to be the assumption with which code has been written.
> > It is called only from pmem_do_bvec() which is called only from
> > pmem_rw_page()
> > and pmem_make_request() which will only passe sector aligned offset and len.
> >
> > Soon we want use this function from dax_zero_page_range() code path which
> > can try to zero arbitrary range of memory with-in a page. So update this
> > function to assume that offset and length can be arbitrary and do the
> > necessary alignments as needed.
>
> What caller will try to zero a range that is smaller than a sector?
Hi Jeff,
New dax zeroing interface (dax_zero_page_range()) can technically pass
a range which is less than a sector. Or which is bigger than a sector
but start and end are not aligned on sector boundaries.
At this point of time, all I care about is that case of an arbitrary
range is handeled well. So if a caller passes a range in, we figure
out subrange which is sector aligned in terms of start and end, and
clear poison on those sectors and ignore rest of the range. And
this itself will be an improvement over current behavior where
nothing is cleared if I/O is not sector aligned.
>
> > nvdimm_clear_poison() seems to assume offset and len to be aligned to
> > clear_err_unit boundary. But this is currently internal detail and is
> > not exported for others to use. So for now, continue to align offset and
> > length to SECTOR_SIZE boundary. Improving it further and to align it
> > to clear_err_unit boundary is a TODO item for future.
>
> When there is a poisoned range of persistent memory, it is recorded by
> the badblocks infrastructure, which currently operates on sectors. So,
> no matter what the error unit is for the hardware, we currently can't
> record/report to userspace anything smaller than a sector, and so that
> is what we expect when clearing errors.
>
> Continuing on for completeness, we will currently not map a page with
> badblocks into a process' address space. So, let's say you have 256
> bytes of bad pmem, we will tell you we've lost 512 bytes, and even if
> you access a valid mmap()d address in the same page as the poisoned
> memory, you will get a segfault.
>
> Userspace can fix up the error by calling write(2) and friends to
> provide new data, or by punching a hole and writing new data to the hole
> (which may result in getting a new block, or reallocating the old block
> and zeroing it, which will clear the error).
Fair enough. I do not need poison clearing at finer granularity. It might
be needed once dev_dax path wants to clear poison. Not sure how exactly
that works.
>
> More comments below...
>
> > Signed-off-by: Vivek Goyal
> > ---
> > drivers/nvdimm/pmem.c | 22 ++
> > 1 file changed, 18 insertions(+), 4 deletions(-)
> >
> > diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
> > index 075b11682192..e72959203253 100644
> > --- a/drivers/nvdimm/pmem.c
> > +++ b/drivers/nvdimm/pmem.c
> > @@ -74,14 +74,28 @@ static blk_status_t pmem_clear_poison(struct
> > pmem_device *pmem,
> > sector_t sector;
> > long cleared;
> > blk_status_t rc = BLK_STS_OK;
> > + phys_addr_t start_aligned, end_aligned;
> > + unsigned int len_aligned;
> >
> > - sector = (offset - pmem->data_offset) / 512;
> > + /*
> > +* Callers can pass arbitrary offset and len. But nvdimm_clear_poison()
> > +* expects memory offset and length to meet certain alignment
> > +* restrction (clear_err_unit). Currently nvdimm does not export
> ^^
> > +* required alignment. So align offset and length to sector boundary
>
> What is "nvdimm" in that sentence? Because the nvdimm most certainly
> does export the required alignment. Perhaps you meant libnvdimm?
I meant nvdimm_clear_poison() function in drivers/nvdimm/bus.c. Whatever
it is called. It first queries alignement required (clear_err_unit) and
then makes sure range passed in meets that alignment requirement.
>
> > +* before passing it to nvdimm_clear_poison().
> > +*/
> > + start_aligned = ALIGN(offset, SECTOR_SIZE);
> > + end_aligned = ALIGN_DOWN((offset + len), SECTOR_SIZE) - 1;
> > + len_aligned = end_aligned - start_aligned + 1;
> > +
> > + sector = (start_aligned - pmem->data_offset) / 512;
> >
> > - cleared = nvdimm_clear_poison(dev, pmem->phys_addr + offset, len);
> > - if (cleared < len)
> > + cleared = nvdimm_clear_poison(dev, pmem->phys_addr + start_aligned,
> > + len_aligned);
> > + if (cleared < len_aligned)
> > rc = BLK_STS_IOERR;
> > if (cleared > 0 && cleared / 512) {
> > - hwpoison_clear(pmem, pmem->phys_addr + offset, cleared);
> > + hwpoison_clear(pmem, pmem->phys_addr + st
Re: [PATCH v5 2/8] drivers/pmem: Allow pmem_clear_poison() to accept arbitrary offset and len
Vivek Goyal writes:
> Currently pmem_clear_poison() expects offset and len to be sector aligned.
> Atleast that seems to be the assumption with which code has been written.
> It is called only from pmem_do_bvec() which is called only from pmem_rw_page()
> and pmem_make_request() which will only passe sector aligned offset and len.
>
> Soon we want use this function from dax_zero_page_range() code path which
> can try to zero arbitrary range of memory with-in a page. So update this
> function to assume that offset and length can be arbitrary and do the
> necessary alignments as needed.
What caller will try to zero a range that is smaller than a sector?
> nvdimm_clear_poison() seems to assume offset and len to be aligned to
> clear_err_unit boundary. But this is currently internal detail and is
> not exported for others to use. So for now, continue to align offset and
> length to SECTOR_SIZE boundary. Improving it further and to align it
> to clear_err_unit boundary is a TODO item for future.
When there is a poisoned range of persistent memory, it is recorded by
the badblocks infrastructure, which currently operates on sectors. So,
no matter what the error unit is for the hardware, we currently can't
record/report to userspace anything smaller than a sector, and so that
is what we expect when clearing errors.
Continuing on for completeness, we will currently not map a page with
badblocks into a process' address space. So, let's say you have 256
bytes of bad pmem, we will tell you we've lost 512 bytes, and even if
you access a valid mmap()d address in the same page as the poisoned
memory, you will get a segfault.
Userspace can fix up the error by calling write(2) and friends to
provide new data, or by punching a hole and writing new data to the hole
(which may result in getting a new block, or reallocating the old block
and zeroing it, which will clear the error).
More comments below...
> Signed-off-by: Vivek Goyal
> ---
> drivers/nvdimm/pmem.c | 22 ++
> 1 file changed, 18 insertions(+), 4 deletions(-)
>
> diff --git a/drivers/nvdimm/pmem.c b/drivers/nvdimm/pmem.c
> index 075b11682192..e72959203253 100644
> --- a/drivers/nvdimm/pmem.c
> +++ b/drivers/nvdimm/pmem.c
> @@ -74,14 +74,28 @@ static blk_status_t pmem_clear_poison(struct pmem_device
> *pmem,
> sector_t sector;
> long cleared;
> blk_status_t rc = BLK_STS_OK;
> + phys_addr_t start_aligned, end_aligned;
> + unsigned int len_aligned;
>
> - sector = (offset - pmem->data_offset) / 512;
> + /*
> + * Callers can pass arbitrary offset and len. But nvdimm_clear_poison()
> + * expects memory offset and length to meet certain alignment
> + * restrction (clear_err_unit). Currently nvdimm does not export
^^
> + * required alignment. So align offset and length to sector boundary
What is "nvdimm" in that sentence? Because the nvdimm most certainly
does export the required alignment. Perhaps you meant libnvdimm?
> + * before passing it to nvdimm_clear_poison().
> + */
> + start_aligned = ALIGN(offset, SECTOR_SIZE);
> + end_aligned = ALIGN_DOWN((offset + len), SECTOR_SIZE) - 1;
> + len_aligned = end_aligned - start_aligned + 1;
> +
> + sector = (start_aligned - pmem->data_offset) / 512;
>
> - cleared = nvdimm_clear_poison(dev, pmem->phys_addr + offset, len);
> - if (cleared < len)
> + cleared = nvdimm_clear_poison(dev, pmem->phys_addr + start_aligned,
> + len_aligned);
> + if (cleared < len_aligned)
> rc = BLK_STS_IOERR;
> if (cleared > 0 && cleared / 512) {
> - hwpoison_clear(pmem, pmem->phys_addr + offset, cleared);
> + hwpoison_clear(pmem, pmem->phys_addr + start_aligned, cleared);
> cleared /= 512;
> dev_dbg(dev, "%#llx clear %ld sector%s\n",
> (unsigned long long) sector, cleared,
We could potentially support clearing less than a sector, but I'd have
to understand the use cases better before offerring implementation
suggestions.
-Jeff
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
RE: 最新广交会126届参展商出炉了
关键有负责人有手机号 广交会126届展商数据 有2.7万多家参展商 QQ : 2083925785 E-mail： sales0...@hotmail.com ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v5 2/8] drivers/pmem: Allow pmem_clear_poison() to accept arbitrary offset and len
On Tue, Feb 18, 2020 at 04:48:35PM -0500, Vivek Goyal wrote: > Currently pmem_clear_poison() expects offset and len to be sector aligned. > Atleast that seems to be the assumption with which code has been written. > It is called only from pmem_do_bvec() which is called only from pmem_rw_page() > and pmem_make_request() which will only passe sector aligned offset and len. > > Soon we want use this function from dax_zero_page_range() code path which > can try to zero arbitrary range of memory with-in a page. So update this > function to assume that offset and length can be arbitrary and do the > necessary alignments as needed. > > nvdimm_clear_poison() seems to assume offset and len to be aligned to > clear_err_unit boundary. But this is currently internal detail and is > not exported for others to use. So for now, continue to align offset and > length to SECTOR_SIZE boundary. Improving it further and to align it > to clear_err_unit boundary is a TODO item for future. > > Signed-off-by: Vivek Goyal This looks sensibel to me, but I'd really like to have Dan take at look. ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Re: [PATCH v5 3/8] pmem: Enable pmem_do_write() to deal with arbitrary ranges
On Tue, Feb 18, 2020 at 04:48:36PM -0500, Vivek Goyal wrote: > Currently pmem_do_write() is written with assumption that all I/O is > sector aligned. Soon I want to use this function in zero_page_range() > where range passed in does not have to be sector aligned. > > Modify this function to be able to deal with an arbitrary range. Which > is specified by pmem_off and len. > > Signed-off-by: Vivek Goyal Looks good, Reviewed-by: Christoph Hellwig ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
Su factura eletrónica.
Estimado cliente, linux-nvdimm@lists.01.org Se adjunta su factura electrónica para el mes de febrero. Factura 20.02.2020(652KB).PDF Saludos Cordiales! No respondas a este correo electrónico. ___ Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[ndctl PATCH 8/8] libndctl,papr_scm: Add support for reporting bad shutdown
Provide support for dimm-op 'smart_get_shutdown_state' implemented as
papr_smart_get_shutdown_state() to report a bad shutdown that couldn't
flush contents of nvdimm to flash memory properly.
Signed-off-by: Vaibhav Jain
---
ndctl/lib/papr_scm.c | 12 +++-
1 file changed, 11 insertions(+), 1 deletion(-)
diff --git a/ndctl/lib/papr_scm.c b/ndctl/lib/papr_scm.c
index a01649d3a9fe..8a6fec992aff 100644
--- a/ndctl/lib/papr_scm.c
+++ b/ndctl/lib/papr_scm.c
@@ -231,13 +231,22 @@ static unsigned int papr_smart_get_health(struct
ndctl_cmd *cmd)
return 1 << (p->health.dimm_health - 1);
}
+static unsigned int papr_smart_get_shutdown_state(struct ndctl_cmd *cmd)
+{
+ struct dimm_priv *p = cmd->dimm->dimm_user_data;
+
+ /* Update dimm state and return f_flush */
+ return update_dimm_stats(cmd->dimm, cmd) ?
+ 0 : p->health.dimm_bad_shutdown;
+}
+
static unsigned int papr_smart_get_flags(struct ndctl_cmd *cmd)
{
/* In case of error return empty flags * */
if (update_dimm_stats(cmd->dimm, cmd))
return 0;
- return ND_SMART_HEALTH_VALID;
+ return ND_SMART_HEALTH_VALID | ND_SMART_SHUTDOWN_VALID;
}
static int papr_dimm_init(struct ndctl_dimm *dimm)
@@ -279,4 +288,5 @@ struct ndctl_dimm_ops * const papr_scm_dimm_ops = &(struct
ndctl_dimm_ops) {
.smart_get_flags = papr_smart_get_flags,
.new_smart = papr_new_smart_health,
.smart_get_health = papr_smart_get_health,
+ .smart_get_shutdown_state = papr_smart_get_shutdown_state,
};
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[ndctl PATCH 4/8] libndctl: Add support for parsing of_pmem dimm flags and monitor mode
Add support for parsing the dimm flags for of_pmem supporting
dimms. The flag file is exported in sysfs as 'papr_flags' and its
contents are space separated flags indicating the current state of the
dimm. A newly introduced function parse_of_pmem_flags() reads the contents
of this flag file and sets appropriate flag bits in 'struct
ndctl_dimm.flags'. This function is called at the end of of_pmem probe
function add_of_pmem_dimm().
Also we advertise support for monitor mode by allocating a file
descriptor to the 'papr_flags' file and assigning it to 'struct
ndctl_dimm.health_event_fd'.
Signed-off-by: Vaibhav Jain
---
ndctl/lib/libndctl.c | 33 -
1 file changed, 32 insertions(+), 1 deletion(-)
diff --git a/ndctl/lib/libndctl.c b/ndctl/lib/libndctl.c
index 650406d27512..7cc50afac404 100644
--- a/ndctl/lib/libndctl.c
+++ b/ndctl/lib/libndctl.c
@@ -801,6 +801,28 @@ static void parse_nfit_mem_flags(struct ndctl_dimm *dimm,
char *flags)
ndctl_dimm_get_devname(dimm), flags);
}
+static void parse_of_pmem_flags(struct ndctl_dimm *dimm, char *flags)
+{
+ char *start, *end;
+
+ start = flags;
+ while ((end = strchr(start, ' '))) {
+ *end = '\0';
+ if (strcmp(start, "not_armed") == 0)
+ dimm->flags.f_arm = 1;
+ else if (strcmp(start, "save_fail") == 0)
+ dimm->flags.f_save = 1;
+ else if (strcmp(start, "flush_fail") == 0)
+ dimm->flags.f_flush = 1;
+ else if (strcmp(start, "smart_notify") == 0)
+ dimm->flags.f_notify = 1;
+ start = end + 1;
+ }
+ if (end != start)
+ dbg(ndctl_dimm_get_ctx(dimm), "%s: %s\n",
+ ndctl_dimm_get_devname(dimm), flags);
+}
+
static void parse_dimm_flags(struct ndctl_dimm *dimm, char *flags)
{
char *start, *end;
@@ -1436,8 +1458,17 @@ static int add_of_pmem_dimm(struct ndctl_dimm *dimm,
const char *dimm_base)
if (strcmp(buf, "ibm,pmemory") == 0) {
dimm->cmd_family = NVDIMM_FAMILY_PAPR_SCM;
rc = 0;
- goto out;
+ goto out_monitor;
}
+
+out_monitor:
+ /* read the flags and also allocate the monitor mode event_fd */
+ sprintf(path, "%s/papr_flags", dimm_base);
+ if (sysfs_read_attr(ctx, path, buf) == 0)
+ parse_of_pmem_flags(dimm, buf);
+
+ /* Allocate monitor mode fd */
+ dimm->health_eventfd = open(path, O_RDONLY|O_CLOEXEC);
out:
free(path);
return rc;
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[ndctl PATCH 3/8] libncdtl: Add initial support for NVDIMM_FAMILY_PAPR_SCM dimm family
The patch-set adds necessary scaffolding in libndctl for dimms that
support papr_scm specification[1]. Since there can be platforms that
support Open-Firmware[2] but not the papr_scm specification, hence the
changes proposed first add support for probing if the dimm bus
supports Open-Firmware. This is done via querying for sysfs attribute
'of_node' in dimm device sysfs directory. If available newly
introduced member 'struct ndctl_bus.has_of_node' is set.
During the probe of the dimm and execution of add_dimm(), the newly
introduced add_of_pmem_dimm() is called if dimm bus reports supports
Open-Firmware.
Function add_of_pmem_dimm() queries the 'compatible' device tree
attribute and based on its value assign NVDIMM_FAMILY_PAPR_SCM to the
dimm command family. In future based on the contents of 'compatible'
attribute more of_pmem dimm families can be queried.
Presently the dimm-ops implementation for NVDIMM_FAMILY_PAPR_SCM is
available in global variable 'papr_scm_dimm_ops' which is a NULL
pointer. Subsequent patches will provide a working dimm-ops
implementation pointed to by 'papr_scm_dimm_ops'.
References:
[1] :
https://git.kernel.org/pub/scm/linux/kernel/git/powerpc/linux.git/commit/?id=58b278f568f0509497e2df7310bfd719156a60d1
[2] : https://en.wikipedia.org/wiki/Open_Firmware
Signed-off-by: Vaibhav Jain
---
ndctl/lib/libndctl.c | 42 ++
ndctl/lib/private.h | 2 ++
ndctl/libndctl.h | 1 +
ndctl/ndctl.h| 1 +
4 files changed, 46 insertions(+)
diff --git a/ndctl/lib/libndctl.c b/ndctl/lib/libndctl.c
index a5f5fdac9f48..650406d27512 100644
--- a/ndctl/lib/libndctl.c
+++ b/ndctl/lib/libndctl.c
@@ -858,6 +858,12 @@ static void *add_bus(void *parent, int id, const char
*ctl_base)
bus->revision = strtoul(buf, NULL, 0);
}
+ sprintf(path, "%s/device/of_node/compatible", ctl_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ bus->has_of_node = 0;
+ else
+ bus->has_of_node = 1;
+
sprintf(path, "%s/device/nfit/dsm_mask", ctl_base);
if (sysfs_read_attr(ctx, path, buf) < 0)
bus->nfit_dsm_mask = 0;
@@ -966,6 +972,10 @@ NDCTL_EXPORT int ndctl_bus_has_nfit(struct ndctl_bus *bus)
return bus->has_nfit;
}
+NDCTL_EXPORT int ndctl_bus_has_of_node(struct ndctl_bus *bus)
+{
+ return bus->has_of_node;
+}
/**
* ndctl_bus_get_major - nd bus character device major number
* @bus: ndctl_bus instance returned from ndctl_bus_get_{first|next}
@@ -1405,6 +1415,34 @@ static int ndctl_bind(struct ndctl_ctx *ctx, struct
kmod_module *module,
static int ndctl_unbind(struct ndctl_ctx *ctx, const char *devpath);
static struct kmod_module *to_module(struct ndctl_ctx *ctx, const char *alias);
+static int add_of_pmem_dimm(struct ndctl_dimm *dimm, const char *dimm_base)
+{
+ int rc = -ENODEV;
+ char buf[SYSFS_ATTR_SIZE];
+ struct ndctl_ctx *ctx = dimm->bus->ctx;
+ char *path = calloc(1, strlen(dimm_base) + 100);
+
+ dbg(ctx, "Probing of_pmem dimm %d at %s\n", dimm->id, dimm_base);
+
+ if (!path)
+ return -ENOMEM;
+
+ sprintf(path, "%s/../of_node/compatible", dimm_base);
+ if (sysfs_read_attr(ctx, path, buf) < 0)
+ goto out;
+
+
+ dbg(ctx, "Compatible of_pmem dimm %d at %s\n", dimm->id, buf);
+ if (strcmp(buf, "ibm,pmemory") == 0) {
+ dimm->cmd_family = NVDIMM_FAMILY_PAPR_SCM;
+ rc = 0;
+ goto out;
+ }
+out:
+ free(path);
+ return rc;
+}
+
static int add_nfit_dimm(struct ndctl_dimm *dimm, const char *dimm_base)
{
int i, rc = -1;
@@ -1583,6 +1621,8 @@ static void *add_dimm(void *parent, int id, const char
*dimm_base)
if (ndctl_bus_has_nfit(bus)) {
dimm->formats = formats;
rc = add_nfit_dimm(dimm, dimm_base);
+ } else if (ndctl_bus_has_of_node(bus)) {
+ rc = add_of_pmem_dimm(dimm, dimm_base);
}
if (rc == -ENODEV) {
@@ -1600,6 +1640,8 @@ static void *add_dimm(void *parent, int id, const char
*dimm_base)
dimm->ops = msft_dimm_ops;
if (dimm->cmd_family == NVDIMM_FAMILY_HYPERV)
dimm->ops = hyperv_dimm_ops;
+ if (dimm->cmd_family == NVDIMM_FAMILY_PAPR_SCM)
+ dimm->ops = papr_scm_dimm_ops;
/* Call the dimm initialization function if needed */
if (!rc && dimm->ops && dimm->ops->dimm_init)
diff --git a/ndctl/lib/private.h b/ndctl/lib/private.h
index fb7fa47f1f37..16754eda7634 100644
--- a/ndctl/lib/private.h
+++ b/ndctl/lib/private.h
@@ -167,6 +167,7 @@ struct ndctl_bus {
int dimms_init;
int regions_init;
int has_nfit;
+ int has_of_node;
char *bus_path;
char *bus_buf;
size_t buf_len;
@@ -351,6 +352,7 @@ struct ndctl_dimm_ops * const intel_dimm_ops;
struct ndctl_dimm_ops * const hpe1_dimm_ops;
struct ndc
[ndctl PATCH 7/8] libndctl,papr_scm: Implement support for DSM_PAPR_SCM_HEALTH
Add support for reporting DIMM health by issuing DSM_PAPR_SCM_HEALTH
DSM. It returns an instance of '
struct nd_papr_scm_dimm_health_stat' as defined in
'papr_scm_dsm.h'. The patch provides support for dimm-ops 'new_smart' &
'smart_get_health' as papr_new_smart_health() &
papr_smart_get_health() respectively. This callbacks should enable
ndctl to report DIMM health.
Also a new member 'struct dimm_priv.health' is introduced which holds
the current health status of the dimm. This member is set inside newly
added function 'update_dimm_health_v1()' which parses the v1 payload
returned by the kernel after servicing DSM_PAPR_SCM_HEALTH. The
function will also update dimm-flags viz 'struct ndctl_dimm.flags.f_*'
based on the flags set in the returned payload.
Signed-off-by: Vaibhav Jain
---
ndctl/lib/papr_scm.c | 80 ++--
1 file changed, 77 insertions(+), 3 deletions(-)
diff --git a/ndctl/lib/papr_scm.c b/ndctl/lib/papr_scm.c
index 0a3857e2a4c4..a01649d3a9fe 100644
--- a/ndctl/lib/papr_scm.c
+++ b/ndctl/lib/papr_scm.c
@@ -40,7 +40,9 @@
/* Per dimm data. Holds per-dimm data parsed from the cmd_pkgs */
struct dimm_priv {
- /* Empty for now */
+
+ /* Cache the dimm health status */
+ struct nd_papr_scm_dimm_health_stat health;
};
static bool papr_cmd_is_supported(struct ndctl_dimm *dimm, int cmd)
@@ -88,6 +90,43 @@ static bool cmd_is_valid(struct ndctl_dimm *dimm, struct
ndctl_cmd *cmd)
return true;
}
+/*
+ * Parse the nd_papr_scm_dimm_health_stat_v1 payload embedded in ndctl_cmd and
+ * update dimm health/flags
+ */
+static int update_dimm_health_v1(struct ndctl_dimm *dimm, struct ndctl_cmd
*cmd)
+{
+ struct nd_papr_scm_cmd_pkg *pcmd = nd_to_papr_cmd_pkg(cmd->pkg);
+ struct dimm_priv *p = dimm->dimm_user_data;
+ const struct nd_papr_scm_dimm_health_stat_v1 *health =
+ papr_scm_pcmd_to_payload(pcmd);
+
+ /* Update the dimm flags */
+ dimm->flags.f_arm = health->dimm_unarmed;
+ dimm->flags.f_flush = health->dimm_bad_shutdown;
+ dimm->flags.f_restore = health->dimm_bad_restore;
+ dimm->flags.f_smart = (health->dimm_health != 0);
+
+ /* Cache the dimm health information */
+ memcpy(&p->health, health, sizeof(*health));
+ return 0;
+}
+
+/* Check payload version returned and pass the packet to appropriate handler */
+static int update_dimm_health(struct ndctl_dimm *dimm, struct ndctl_cmd *cmd)
+{
+ const struct nd_papr_scm_cmd_pkg *pcmd = nd_to_papr_cmd_pkg(cmd->pkg);
+
+ if (pcmd->payload_version == 1)
+ return update_dimm_health_v1(dimm, cmd);
+
+ /* unknown version */
+ PAPR_ERR(dimm, "Unknown payload version for dimm_health."
+"Ver=%d, Supported=%d\n", pcmd->payload_version,
+ND_PAPR_SCM_DIMM_HEALTH_VERSION);
+ return -EINVAL;
+}
+
/* Parse a command payload and update dimm flags/private data */
static int update_dimm_stats(struct ndctl_dimm *dimm, struct ndctl_cmd *cmd)
{
@@ -113,6 +152,8 @@ static int update_dimm_stats(struct ndctl_dimm *dimm,
struct ndctl_cmd *cmd)
/* Get the command dsm and handle it */
pcmd = nd_to_papr_cmd_pkg(cmd->pkg);
switch (pcmd_to_dsm(pcmd)) {
+ case DSM_PAPR_SCM_HEALTH:
+ return update_dimm_health(dimm, cmd);
default:
PAPR_ERR(dimm, "Unhandled dsm-command 0x%016llx\n",
pcmd_to_dsm(pcmd));
@@ -158,14 +199,45 @@ static struct ndctl_cmd *allocate_cmd(struct ndctl_dimm
*dimm,
return cmd;
}
+static struct ndctl_cmd *papr_new_smart_health(struct ndctl_dimm *dimm)
+{
+ struct ndctl_cmd *cmd_ret;
+
+ cmd_ret = allocate_cmd(dimm, DSM_PAPR_SCM_HEALTH,
+ sizeof(struct nd_papr_scm_dimm_health_stat),
+ ND_PAPR_SCM_DIMM_HEALTH_VERSION);
+ if (!cmd_ret) {
+ PAPR_ERR(dimm, "Unable to allocate smart_health command\n");
+ return NULL;
+ }
+
+ cmd_ret->pkg[0].nd_size_out = ND_PAPR_SCM_ENVELOPE_CONTENT_SIZE(
+ struct nd_papr_scm_dimm_health_stat);
+
+ return cmd_ret;
+}
+
+static unsigned int papr_smart_get_health(struct ndctl_cmd *cmd)
+{
+ struct dimm_priv *p = cmd->dimm->dimm_user_data;
+
+ /*
+* Update the dimm stats and use some math to return one of
+* defined ND_SMART_*_HEALTH values
+*/
+ if (update_dimm_stats(cmd->dimm, cmd) || !p->health.dimm_health)
+ return 0;
+ else
+ return 1 << (p->health.dimm_health - 1);
+}
+
static unsigned int papr_smart_get_flags(struct ndctl_cmd *cmd)
{
/* In case of error return empty flags * */
if (update_dimm_stats(cmd->dimm, cmd))
return 0;
- /* Return empty flags for now as no DSM support */
- return 0;
+ return ND_SMART_HEALTH_VALID;
}
static int papr_dimm_
[ndctl PATCH 6/8] libndctl,papr_scm: Implement scaffolding to issue and handle DSM cmds
This patch implement necessary infrastructure inside 'papr_scm.c' to
issue and handle DSM commands. Changed implemented are:
* Implement dimm initialization/un-initialization functions
papr_dimm_init()/unint() to allocate a per-dimm 'struct dimm_priv'
instance.
* New helper function allocate_cmd() to allocate command packages for
a specific DSM command and payload size.
* New function update_dimm_state() to parse a given command payload
and update per dimm 'struct dimm_priv'.
* Provide an implementation of 'dimm_ops->smart_get_flags' to send the
submitted instance of 'struct ndctl_cmd' to update_dimm_state().
* Logging helpers for papr_scm that use the underlying libndctl
provided logging.
Signed-off-by: Vaibhav Jain
---
ndctl/lib/papr_scm.c | 174 +++
1 file changed, 174 insertions(+)
diff --git a/ndctl/lib/papr_scm.c b/ndctl/lib/papr_scm.c
index 878698a5a8b4..0a3857e2a4c4 100644
--- a/ndctl/lib/papr_scm.c
+++ b/ndctl/lib/papr_scm.c
@@ -20,6 +20,29 @@
#include
#include
+/* Utility logging maros for simplify logging */
+#define PAPR_DBG(_dimm_, _format_str_, ...) dbg(_dimm_->bus->ctx, \
+ "papr_scm:"#_format_str_, \
+ ##__VA_ARGS__)
+#define PAPR_INFO(_dimm_, _format_str_, ...) info(_dimm_->bus->ctx,\
+ "papr_scm:"#_format_str_, \
+ ##__VA_ARGS__)
+#define PAPR_ERR(_dimm_, _format_str_, ...) err(_dimm_->bus->ctx, \
+ "papr_scm:"#_format_str_, \
+ ##__VA_ARGS__)
+#define PAPR_NOTICE(_dimm_, _format_str_, ...) notice(_dimm_->bus->ctx,
\
+ "papr_scm:"#_format_str_, \
+ ##__VA_ARGS__)
+
+/* Command flags to indicate if a given command is parsed of not */
+#define CMD_PKG_SUBMITTED 1
+#define CMD_PKG_PARSED 2
+
+/* Per dimm data. Holds per-dimm data parsed from the cmd_pkgs */
+struct dimm_priv {
+ /* Empty for now */
+};
+
static bool papr_cmd_is_supported(struct ndctl_dimm *dimm, int cmd)
{
/* Handle this separately to support monitor mode */
@@ -29,6 +52,157 @@ static bool papr_cmd_is_supported(struct ndctl_dimm *dimm,
int cmd)
return !!(dimm->cmd_mask & (1ULL << cmd));
}
+static __u64 pcmd_to_dsm(const struct nd_papr_scm_cmd_pkg *pcmd)
+{
+ return pcmd->hdr.nd_command;
+}
+
+/* Verify if the given command is supported and valid */
+static bool cmd_is_valid(struct ndctl_dimm *dimm, struct ndctl_cmd *cmd)
+{
+ const struct nd_papr_scm_cmd_pkg *pcmd = nd_to_papr_cmd_pkg(cmd->pkg);
+
+ if (dimm == NULL)
+ return false;
+
+ if (cmd == NULL) {
+ PAPR_ERR(dimm, "Invalid command\n");
+ return false;
+ }
+
+ /* Verify the command family */
+ if (pcmd->hdr.nd_family != NVDIMM_FAMILY_PAPR_SCM) {
+ PAPR_ERR(dimm, "Invalid command family:0x%016llx\n",
+pcmd->hdr.nd_family);
+ return false;
+ }
+
+ /* Verify the DSM */
+ if (pcmd_to_dsm(pcmd) <= DSM_PAPR_SCM_MIN ||
+ pcmd_to_dsm(pcmd) >= DSM_PAPR_SCM_MAX) {
+ PAPR_ERR(dimm, "Invalid command :0x%016llx\n",
+pcmd->hdr.nd_command);
+ return false;
+ }
+
+ return true;
+}
+
+/* Parse a command payload and update dimm flags/private data */
+static int update_dimm_stats(struct ndctl_dimm *dimm, struct ndctl_cmd *cmd)
+{
+ const struct nd_papr_scm_cmd_pkg *pcmd;
+
+ if (!cmd_is_valid(dimm, cmd))
+ return -EINVAL;
+
+ /*
+* Silently prevent parsing of an already parsed ndctl_cmd else
+* mark the command as parsed.
+*/
+ if (cmd->status >= CMD_PKG_PARSED) {
+ return 0;
+ } else if (cmd->status < 0) {
+ PAPR_ERR(dimm, "Command error %d\n", cmd->status);
+ return -ENXIO;
+ }
+
+ /* Mark the command as parsed */
+ cmd->status = CMD_PKG_PARSED;
+
+ /* Get the command dsm and handle it */
+ pcmd = nd_to_papr_cmd_pkg(cmd->pkg);
+ switch (pcmd_to_dsm(pcmd)) {
+ default:
+ PAPR_ERR(dimm, "Unhandled dsm-command 0x%016llx\n",
+pcmd_to_dsm(pcmd));
+ return -ENOENT;
+ }
+}
+
+/* Allocate a struct ndctl_cmd for given dsm command with payload size */
+static struct ndctl_cmd *allocate_cmd(struct ndctl_dimm *dimm,
+ __u64 dsm_cmd, size_t payload_size,
+ uint16_t payload_version)
+{
+ struct ndctl_cmd *cmd;
+ struct nd_papr_scm_cmd_pkg *pcmd;
+ size_t size;
+
+ size = sizeof(struct ndctl_cmd) +
+
[ndctl PATCH 2/8] libndctl: Introduce a new dimm-ops dimm_init() & dimm_uninit()
There are scenarios when a dimm-provider need to allocate some
per-dimm data that can be quickly retrieved. This data can be used to
cache data that spans multiple 'struct ndctl_cmd' submissions.
Unfortunately currently in libnvdimm there is no easiy way to implement
this. Even if this data is some how store in an overloaded field of
'struct ndctl_dimm', managing its lifetime is a challenge.
To solve this problem this patch proposes a new member 'struct
ndctl_dimm.dimm_user_data' to store per-dimm data thats specific to a
dimm-provider. Also two new dimm-ops namely dimm_init() &
dimm_uninit() are introduced that can be used to manage the lifetime
of this per-dimm data.
Semantics
=
int (*dimm_init)(struct ndctl_dimm *):
This callback will be called just after dimm-probe inside add_dimm()
is completed. Dimm-providers should use this callback to allocate
per-dimm data and assign it to 'struct ndctl_dimm.dimm_user_data'
member. In case this function returns an error, dimm initialization is
halted and errors out.
void (*dimm_uninit)(struct ndctl_dimm *):
This callback will be called during free_dimm() and is only called if
previous call to 'dimm_ops->dimm_init()' had reported no
error. Dimm-providers should use this callback to unallocate and
cleanup 'dimm_user_data'.
Signed-off-by: Vaibhav Jain
---
ndctl/lib/libndctl.c | 13 -
ndctl/lib/private.h | 5 +
2 files changed, 17 insertions(+), 1 deletion(-)
diff --git a/ndctl/lib/libndctl.c b/ndctl/lib/libndctl.c
index 38ddfea6dbc0..a5f5fdac9f48 100644
--- a/ndctl/lib/libndctl.c
+++ b/ndctl/lib/libndctl.c
@@ -596,6 +596,10 @@ static void free_dimm(struct ndctl_dimm *dimm)
{
if (!dimm)
return;
+ /* If needed call the dimm uninitialization function */
+ if (dimm->ops && dimm->ops->dimm_uninit)
+ dimm->ops->dimm_uninit(dimm);
+
free(dimm->unique_id);
free(dimm->dimm_buf);
free(dimm->dimm_path);
@@ -1596,8 +1600,15 @@ static void *add_dimm(void *parent, int id, const char
*dimm_base)
dimm->ops = msft_dimm_ops;
if (dimm->cmd_family == NVDIMM_FAMILY_HYPERV)
dimm->ops = hyperv_dimm_ops;
- out:
+
+ /* Call the dimm initialization function if needed */
+ if (!rc && dimm->ops && dimm->ops->dimm_init)
+ rc = dimm->ops->dimm_init(dimm);
+
+out:
if (rc) {
+ /* Ensure dimm_uninit() is not called during free_dimm() */
+ dimm->ops = NULL;
err(ctx, "Unable to probe dimm:%d. Err:%d\n", id, rc);
goto err_read;
}
diff --git a/ndctl/lib/private.h b/ndctl/lib/private.h
index 1f6a01c55377..fb7fa47f1f37 100644
--- a/ndctl/lib/private.h
+++ b/ndctl/lib/private.h
@@ -98,6 +98,7 @@ struct ndctl_dimm {
} flags;
int locked;
int aliased;
+ void *dimm_user_data;
struct list_node list;
int formats;
int format[0];
@@ -340,6 +341,10 @@ struct ndctl_dimm_ops {
struct ndctl_cmd *(*new_ack_shutdown_count)(struct ndctl_dimm *);
int (*fw_update_supported)(struct ndctl_dimm *);
int (*xlat_firmware_status)(struct ndctl_cmd *);
+ /* Called just after dimm is initialized and probed */
+ int (*dimm_init)(struct ndctl_dimm *);
+ /* Called just before struct ndctl_dimm is de-allocated */
+ void (*dimm_uninit)(struct ndctl_dimm *);
};
struct ndctl_dimm_ops * const intel_dimm_ops;
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[ndctl PATCH 5/8] libndctl,papr_scm: Add definitions for PAPR_SCM DSM commands
Pull the kernel definition of PAPR_SCM DSM command which is located in
the kernel tree at Ref[1]. Also add an implementation of
'papr_scm_dimm_ops' in a new file named 'papr_scm.c'. For now only an
implementation of 'dimm_ops.cmd_is_supported' is provided.
References:
[1]: arch/powerpc/include/uapi/asm/papr_scm_dsm.h
Signed-off-by: Vaibhav Jain
---
ndctl/lib/Makefile.am| 1 +
ndctl/lib/papr_scm.c | 34 ++
ndctl/lib/papr_scm_dsm.h | 143 +++
3 files changed, 178 insertions(+)
create mode 100644 ndctl/lib/papr_scm.c
create mode 100644 ndctl/lib/papr_scm_dsm.h
diff --git a/ndctl/lib/Makefile.am b/ndctl/lib/Makefile.am
index e4eb0060bca4..3943541e435d 100644
--- a/ndctl/lib/Makefile.am
+++ b/ndctl/lib/Makefile.am
@@ -21,6 +21,7 @@ libndctl_la_SOURCES =\
hpe1.c \
msft.c \
hyperv.c \
+ papr_scm.c \
ars.c \
firmware.c \
libndctl.c
diff --git a/ndctl/lib/papr_scm.c b/ndctl/lib/papr_scm.c
new file mode 100644
index ..878698a5a8b4
--- /dev/null
+++ b/ndctl/lib/papr_scm.c
@@ -0,0 +1,34 @@
+/*
+ * Copyright (C) 2020 IBM Corporation
+ *
+ * This program is free software; you can redistribute it and/or modify it
+ * under the terms and conditions of the GNU Lesser General Public License,
+ * version 2.1, as published by the Free Software Foundation.
+ *
+ * This program is distributed in the hope it will be useful, but WITHOUT ANY
+ * WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS
+ * FOR A PARTICULAR PURPOSE. See the GNU Lesser General Public License for
+ * more details.
+ */
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+static bool papr_cmd_is_supported(struct ndctl_dimm *dimm, int cmd)
+{
+ /* Handle this separately to support monitor mode */
+ if (cmd == ND_CMD_SMART)
+ return true;
+
+ return !!(dimm->cmd_mask & (1ULL << cmd));
+}
+
+struct ndctl_dimm_ops * const papr_scm_dimm_ops = &(struct ndctl_dimm_ops) {
+ .cmd_is_supported = papr_cmd_is_supported,
+};
diff --git a/ndctl/lib/papr_scm_dsm.h b/ndctl/lib/papr_scm_dsm.h
new file mode 100644
index ..aacced453579
--- /dev/null
+++ b/ndctl/lib/papr_scm_dsm.h
@@ -0,0 +1,143 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * PAPR SCM Device specific methods for libndctl and ndctl
+ *
+ * (C) Copyright IBM 2020
+ *
+ * Author: Vaibhav Jain
+ *
+ * This program is free software; you can redistribute it and/or modify
+ * it under the terms of the GNU General Public License as published by
+ * the Free Software Foundation; either version 2, or (at your option)
+ * any later version.
+ *
+ * This program is distributed in the hope that it will be useful,
+ * but WITHOUT ANY WARRANTY; without even the implied warranty of
+ * MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
+ * GNU General Public License for more details.
+ */
+
+#ifndef _UAPI_ASM_POWERPC_PAPR_SCM_DSM_H_
+#define _UAPI_ASM_POWERPC_PAPR_SCM_DSM_H_
+
+#include
+
+#ifdef __KERNEL__
+#include
+#else
+#include
+#endif
+
+/*
+ * Sub commands for ND_CMD_CALL. To prevent overlap from ND_CMD_*, values for
+ * these enums start at 0x1. These values are then returned from
+ * cmd_to_func() making it easy to implement the switch-case block in
+ * papr_scm_ndctl()
+ */
+enum dsm_papr_scm {
+ DSM_PAPR_SCM_MIN = 0x1,
+ DSM_PAPR_SCM_HEALTH,
+ DSM_PAPR_SCM_STATS,
+ DSM_PAPR_SCM_MAX,
+};
+
+enum dsm_papr_scm_dimm_health {
+ DSM_PAPR_SCM_DIMM_HEALTHY,
+ DSM_PAPR_SCM_DIMM_UNHEALTHY,
+ DSM_PAPR_SCM_DIMM_CRITICAL,
+ DSM_PAPR_SCM_DIMM_FATAL,
+};
+
+/* Papr-scm-header + payload expected with ND_CMD_CALL ioctl from libnvdimm */
+struct nd_papr_scm_cmd_pkg {
+ struct nd_cmd_pkg hdr; /* Package header containing sub-cmd */
+ int32_t cmd_status; /* Out: Sub-cmd status returned back */
+ uint16_t payload_offset;/* In: offset from start of struct */
+ uint16_t payload_version; /* In/Out: version of the payload */
+ uint8_t payload[]; /* In/Out: Sub-cmd data buffer */
+};
+
+/* Helpers to evaluate the size of PAPR_SCM envelope */
+/* Calculate the papr_scm-header size */
+#define ND_PAPR_SCM_ENVELOPE_CONTENT_HDR_SIZE \
+ (sizeof(struct nd_papr_scm_cmd_pkg) - sizeof(struct nd_cmd_pkg))
+/*
+ * Given a type calculate the envelope size
+ * (nd-header + papr_scm-header + payload)
+ */
+#define ND_PAPR_SCM_ENVELOPE_SIZE(_type_) \
+ (sizeof(_type_) + sizeof(struct nd_papr_scm_cmd_pkg))
+
+/* Given a type envelope-content size (papr_scm-header + payload) */
+#define ND_PAPR_SCM_ENVELOPE_CONTENT_SIZE(_type_) \
+ (sizeof(_type_) + ND_PAPR_SCM_ENVELOPE_CONTENT_HDR_SIZE)
+
+/*
+ * Struct exchanged between kernel & ndctl in for PAPR_DSM_PAPR_SMART_HEALTH
+ * Various bitflags indicate the heal
[ndctl PATCH 1/8] libndctl: Refactor out add_dimm() to handle NFIT specific init
Presently add_dimm() only probes dimms that support NFIT/ACPI. Hence
this patch refactors this functionality into two functions namely
add_dimm() and add_nfit_dimm(). Function add_dimm() performs
allocation and common 'struct ndctl_dimm' initialization and depending
on whether the dimm-bus supports NIFT, calls add_nfit_dimm(). Once
the probe is completed based on the value of 'ndctl_dimm.cmd_family'
appropriate dimm-ops are assigned to the dimm.
In case dimm-bus is of unknown type or doesn't support NFIT the
initialization still continues, with no dimm-ops assigned to the
'struct ndctl_dimm' there-by limiting the functionality available.
This patch shouldn't introduce any behavioral change.
Signed-off-by: Vaibhav Jain
---
ndctl/lib/libndctl.c | 193 +--
1 file changed, 112 insertions(+), 81 deletions(-)
diff --git a/ndctl/lib/libndctl.c b/ndctl/lib/libndctl.c
index d6a28002e7d6..38ddfea6dbc0 100644
--- a/ndctl/lib/libndctl.c
+++ b/ndctl/lib/libndctl.c
@@ -1401,82 +1401,15 @@ static int ndctl_bind(struct ndctl_ctx *ctx, struct
kmod_module *module,
static int ndctl_unbind(struct ndctl_ctx *ctx, const char *devpath);
static struct kmod_module *to_module(struct ndctl_ctx *ctx, const char *alias);
-static void *add_dimm(void *parent, int id, const char *dimm_base)
+static int add_nfit_dimm(struct ndctl_dimm *dimm, const char *dimm_base)
{
- int formats, i;
- struct ndctl_dimm *dimm;
+ int i, rc = -1;
char buf[SYSFS_ATTR_SIZE];
- struct ndctl_bus *bus = parent;
- struct ndctl_ctx *ctx = bus->ctx;
+ struct ndctl_ctx *ctx = dimm->bus->ctx;
char *path = calloc(1, strlen(dimm_base) + 100);
if (!path)
- return NULL;
-
- sprintf(path, "%s/nfit/formats", dimm_base);
- if (sysfs_read_attr(ctx, path, buf) < 0)
- formats = 1;
- else
- formats = clamp(strtoul(buf, NULL, 0), 1UL, 2UL);
-
- dimm = calloc(1, sizeof(*dimm) + sizeof(int) * formats);
- if (!dimm)
- goto err_dimm;
- dimm->bus = bus;
- dimm->id = id;
-
- sprintf(path, "%s/dev", dimm_base);
- if (sysfs_read_attr(ctx, path, buf) < 0)
- goto err_read;
- if (sscanf(buf, "%d:%d", &dimm->major, &dimm->minor) != 2)
- goto err_read;
-
- sprintf(path, "%s/commands", dimm_base);
- if (sysfs_read_attr(ctx, path, buf) < 0)
- goto err_read;
- dimm->cmd_mask = parse_commands(buf, 1);
-
- dimm->dimm_buf = calloc(1, strlen(dimm_base) + 50);
- if (!dimm->dimm_buf)
- goto err_read;
- dimm->buf_len = strlen(dimm_base) + 50;
-
- dimm->dimm_path = strdup(dimm_base);
- if (!dimm->dimm_path)
- goto err_read;
-
- sprintf(path, "%s/modalias", dimm_base);
- if (sysfs_read_attr(ctx, path, buf) < 0)
- goto err_read;
- dimm->module = to_module(ctx, buf);
-
- dimm->handle = -1;
- dimm->phys_id = -1;
- dimm->serial = -1;
- dimm->vendor_id = -1;
- dimm->device_id = -1;
- dimm->revision_id = -1;
- dimm->health_eventfd = -1;
- dimm->dirty_shutdown = -ENOENT;
- dimm->subsystem_vendor_id = -1;
- dimm->subsystem_device_id = -1;
- dimm->subsystem_revision_id = -1;
- dimm->manufacturing_date = -1;
- dimm->manufacturing_location = -1;
- dimm->cmd_family = -1;
- dimm->nfit_dsm_mask = ULONG_MAX;
- for (i = 0; i < formats; i++)
- dimm->format[i] = -1;
-
- sprintf(path, "%s/flags", dimm_base);
- if (sysfs_read_attr(ctx, path, buf) < 0) {
- dimm->locked = -1;
- dimm->aliased = -1;
- } else
- parse_dimm_flags(dimm, buf);
-
- if (!ndctl_bus_has_nfit(bus))
- goto out;
+ return -1;
/*
* 'unique_id' may not be available on older kernels, so don't
@@ -1542,24 +1475,15 @@ static void *add_dimm(void *parent, int id, const char
*dimm_base)
sprintf(path, "%s/nfit/family", dimm_base);
if (sysfs_read_attr(ctx, path, buf) == 0)
dimm->cmd_family = strtoul(buf, NULL, 0);
- if (dimm->cmd_family == NVDIMM_FAMILY_INTEL)
- dimm->ops = intel_dimm_ops;
- if (dimm->cmd_family == NVDIMM_FAMILY_HPE1)
- dimm->ops = hpe1_dimm_ops;
- if (dimm->cmd_family == NVDIMM_FAMILY_MSFT)
- dimm->ops = msft_dimm_ops;
- if (dimm->cmd_family == NVDIMM_FAMILY_HYPERV)
- dimm->ops = hyperv_dimm_ops;
sprintf(path, "%s/nfit/dsm_mask", dimm_base);
if (sysfs_read_attr(ctx, path, buf) == 0)
dimm->nfit_dsm_mask = strtoul(buf, NULL, 0);
- dimm->formats = formats;
sprintf(path, "%s/nfit/format", dimm_base);
if (sysfs_read_attr(ctx, path, buf) == 0)
dimm->format[0] = strtoul(buf, NULL,
[ndctl PATCH 0/8] Add support for reporting papr-scm nvdimm health
This patch-set proposes changes to libndctl to add support for reporting
health for nvdimms that support the PAPR standard[1]. The standard defines
heathenism (HCALL) through which a guest kernel can query and fetch health
and performance stats of an nvdimm attached to the hypervisor[2]. Until
now 'ndctl' was unable to report these stats for papr_scm dimms on PPC64
guests due to absence of ACPI/NFIT, a limitation which this patch-set tries
to address.
The patch-set introduces support for the new PAPR-SCM DSM family defined at
[3] via a new dimm-ops named 'papr_scm_dimm_ops'. Infrastructure to probe
and distinguish papr-scm dimms from other dimm families that may support
ACPI/NFIT is implemented by updating the 'struct ndctl_dimm' initialization
routines to bifurcate based on the nvdimm type. We also introduce two new
dimm-ops for handling initialization of dimm specific data for specific DSM
families.
These changes coupled with proposed ndtcl changes located at Ref[4] should
provide a way for the user to retrieve NVDIMM health status using ndtcl for
papr_scm guests. Below is a sample output using proposed kernel + ndctl
changes:
# ndctl list -DH
[
{
"dev":"nmem0",
"health":{
"health_state":"fatal",
"shutdown_state":"dirty"
}
}
]
Structure of the patchset
=
We start with a re-factoring patch that splits the 'add_dimm()' function
into two functions one that take care of allocating and initializing
'struct ndctl_dimm' and another that takes care of initializing nfit
specific dimm attributes.
Patch-2 introduces new dimm ops 'dimm_init()' & 'dimm_uninit()' to handle
DSM family specific initialization of 'struct ndctl_dimm'.
Patch-3 introduces probe function of papr_scm nvdimms and assigning
'papr_scm_dimm_ops' to 'dimm->ops' if needed. Subsequent patch-4 adds
support for parsing papr_scm nvdimm flags and updating 'dimm->flags' with
them.
Patches-5,6 implements scaffolding to add support for PAPR_SCM DSM commands
and pull in their definitions from the kernel. This implementation resides
in newly added 'papr_scm.c' file.
Finally Patched-7,8 add support for issuing and handling the result of
'struct ndctl_cmd' to request dimm health stats from papr_scm kernel module
and returning appropriate health status to libndctl for reporting.
References:
[1]: "Power Architecture Platform Reference"
https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference
[2]: "[DOC,v2] powerpc: Provide initial documentation for PAPR hcalls"
https://patchwork.ozlabs.org/patch/1154292/
[3]: "[PATCH 5/8] powerpc/uapi: Introduce uapi header 'papr_scm_dsm.h' for
papr_scm DSMs"
https://patchwork.ozlabs.org/patch/1241352/
[4]: "powerpc/papr_scm: Add support for reporting nvdimm health"
https://patchwork.ozlabs.org/project/linuxppc-dev/list/?series=159701
Vaibhav Jain (8):
libndctl: Refactor out add_dimm() to handle NFIT specific init
libndctl: Introduce a new dimm-ops dimm_init() & dimm_uninit()
libncdtl: Add initial support for NVDIMM_FAMILY_PAPR_SCM dimm family
libndctl: Add support for parsing of_pmem dimm flags and monitor mode
libndctl,papr_scm: Add definitions for PAPR_SCM DSM commands
libndctl,papr_scm: Implement scaffolding to issue and handle DSM cmds
libndctl,papr_scm: Implement support for DSM_PAPR_SCM_HEALTH
libndctl,papr_scm: Add support for reporting bad shutdown
ndctl/lib/Makefile.am| 1 +
ndctl/lib/libndctl.c | 263 +--
ndctl/lib/papr_scm.c | 292 +++
ndctl/lib/papr_scm_dsm.h | 143 +++
ndctl/lib/private.h | 7 +
ndctl/libndctl.h | 1 +
ndctl/ndctl.h| 1 +
7 files changed, 634 insertions(+), 74 deletions(-)
create mode 100644 ndctl/lib/papr_scm.c
create mode 100644 ndctl/lib/papr_scm_dsm.h
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH 4/8] UAPI: ndctl: Introduce NVDIMM_FAMILY_PAPR_SCM as a new NVDIMM DSM family
Add PAPR-scm family of DSM command-set to the white list of NVDIMM
command sets.
Signed-off-by: Vaibhav Jain
---
include/uapi/linux/ndctl.h | 1 +
1 file changed, 1 insertion(+)
diff --git a/include/uapi/linux/ndctl.h b/include/uapi/linux/ndctl.h
index de5d90212409..99fb60600ef8 100644
--- a/include/uapi/linux/ndctl.h
+++ b/include/uapi/linux/ndctl.h
@@ -244,6 +244,7 @@ struct nd_cmd_pkg {
#define NVDIMM_FAMILY_HPE2 2
#define NVDIMM_FAMILY_MSFT 3
#define NVDIMM_FAMILY_HYPERV 4
+#define NVDIMM_FAMILY_PAPR_SCM 5
#define ND_IOCTL_CALL _IOWR(ND_IOCTL, ND_CMD_CALL,\
struct nd_cmd_pkg)
--
2.24.1
___
Linux-nvdimm mailing list -- linux-nvdimm@lists.01.org
To unsubscribe send an email to linux-nvdimm-le...@lists.01.org
[PATCH 0/7] powerpc/papr_scm: Add support for reporting nvdimm health
The PAPR standard[1][3] provides suitable mechanisms to query the health and
performance stats of an NVDIMM via various hcalls as described in Ref[2]. Until
now these stats were never available nor exposed to the user-space tools like
'ndctl'. This is partly due to PAPR platform not having support for ACPI and
NFIT. Hence 'ndctl' is unable to query and report the dimm health status and a
user had no way to determine the current health status of a NDVIMM.
To overcome this limitation this patch-set updates papr_scm kernel module to
query and fetch nvdimm health and performance stats using hcalls described in
Ref[2]. This health and performance stats are then exposed to userspace via
syfs and Dimm-Specific-Methods(DSM) issued by libndctl.
These changes coupled with proposed ndtcl changes located at Ref[4] should
provide a way for the user to retrieve NVDIMM health status using ndtcl. Below
is a sample output using proposed kernel + ndctl for PAPR NVDIMM in an
emulation environment:
# ndctl list -DH
[
{
"dev":"nmem0",
"health":{
"health_state":"fatal",
"shutdown_state":"dirty"
}
}
]
PAPR Dimm-Specific-Methods(DSM)
As the name suggests DSMs are used by vendor specific code in libndctl to
execute certain operations or fetch certain information for NVDIMMS. DSMs
can be sent to papr_scm module via libndctl (userspace) and libnvdimm(kernel)
using the ND_CMD_CALL ioctl which can be handled in the dimm control function
papr_scm_ndctl(). For PAPR this patchset proposes two DSMs defined in the newly
introduced uapi header named 'papr_scm_dsm.h', that directly map to hcalls
provided by PHYP to query NVDIMM health and stats. These DSMs are:
* DSM_PAPR_SCM_HEALTH: Which map to hcall H_SCM_HEALTH and returns dimm health.
* DSM_PAPR_SCM_STATS: Which map to hcall H_SCM_PERFORMANCE_STATS and returns
dimm performance stats.
P.S: The current patch-set only provides an implementation for servicing
DSM_PAPR_SCM_HEALTH and a future patch will add support for DSM_PAPR_SCM_STATS.
The ioctl ND_CMD_CALL can also transfer data between user-space and kernel via
'envelopes'. The envelop is part of a 'struct nd_cmd_pkg' which in return is
wrapped in a user defined struct which in our case is a newly introduced
'struct nd_papr_scm_cmd_pkg'. Apart from 'envelope header' this struct holds
'payload', 'payload offset', 'payload version' and 'command status'.
The 'payload' field of the envelop holds a struct depending on the DSM method
used and should be one of the structs defined in newly introduced uapi header
'papr_scm_dsm.h'. This makes it possible for libndctl/kernel to share the same
definitions for these DSM structs.
Earlier Work
An earlier RFC patch set titled "powerpc/papr_scm: Implement support for
reporting DIMM health and stats" [5] was proposed which tried to achieve
same functionality albeit with a different approach i.e papr_scm module
acted as a pass-through for the DSM calls from libndctl.
This patch-set however departs from that design by decoupling the
libndctl <--> papr_scm and papr_scm <--> phyp interfaces. This provides
more flexibility compared to earlier approach were these two interfaces were
coupled with each other.
Structure of the patch-set
==
The initial 3 patches of the patch-set add functionality of issuing necessary
HCALLs to PHYP to retrieve the dimm health/performance stats information and
exposing them to user-space via sysfs attributes.
Subsequent patches deal with defining and implementing support for
NVDIMM_FAMILY_PAPR_SCM DSM command family and implementing the payload
versioning scheme as mentioned above.
References:
[1]: "Power Architecture Platform Reference"
https://en.wikipedia.org/wiki/Power_Architecture_Platform_Reference
[2]: "[DOC,v2] powerpc: Provide initial documentation for PAPR hcalls"
https://patchwork.ozlabs.org/patch/1154292/
[3]: "Linux on Power Architecture Platform Reference"
https://members.openpowerfoundation.org/document/dl/469
[4]: https://github.com/vaibhav92/ndctl/tree/papr_scm_health_v1
[5]:
https://lore.kernel.org/linuxppc-dev/20200129152844.71286-1-vaib...@linux.ibm.com/
Vaibhav Jain (8):
powerpc: Add asm header 'papr_scm.h' describing the papr-scm interface
powerpc/papr_scm: Provide support for fetching dimm health information
powerpc/papr_scm: Fetch dimm performance stats from PHYP
UAPI: ndctl: Introduce NVDIMM_FAMILY_PAPR_SCM as a new NVDIMM DSM
family
powerpc/uapi: Introduce uapi header 'papr_scm_dsm.h' for papr_scm DSMs
powerpc/papr_scm: Add support for handling PAPR DSM commands
powerpc/papr_scm: Re-implement 'papr_flags' using
'nd_papr_scm_dimm_health_stat'
powerpc/papr_scm: Implement support for DSM_PAPR_SCM_HEALTH
arch/powerpc/include/asm/papr_scm.h | 68
arch/powerpc/include/uapi/asm/papr_scm_dsm.h | 143 +++
arch/powerpc/platforms/pseries/papr_scm.c| 399
