[PATCH v3 9/9] Documentation/powerpc: VAS API
Power9 introduced Virtual Accelerator Switchboard (VAS) which allows userspace to communicate with Nest Accelerator (NX) directly. But kernel has to establish channel to NX for userspace. This document describes user space API that application can use to establish communication channel. Signed-off-by: Sukadev Bhattiprolu Signed-off-by: Haren Myneni --- Documentation/powerpc/index.rst | 1 + Documentation/powerpc/vas-api.rst | 246 ++ 2 files changed, 247 insertions(+) create mode 100644 Documentation/powerpc/vas-api.rst diff --git a/Documentation/powerpc/index.rst b/Documentation/powerpc/index.rst index 0d45f0f..afe2d5e 100644 --- a/Documentation/powerpc/index.rst +++ b/Documentation/powerpc/index.rst @@ -30,6 +30,7 @@ powerpc syscall64-abi transactional_memory ultravisor +vas-api .. only:: subproject and html diff --git a/Documentation/powerpc/vas-api.rst b/Documentation/powerpc/vas-api.rst new file mode 100644 index 000..13ce4e7 --- /dev/null +++ b/Documentation/powerpc/vas-api.rst @@ -0,0 +1,246 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. _VAS-API: + +=== +Virtual Accelerator Switchboard (VAS) userspace API +=== + +Introduction + + +Power9 processor introduced Virtual Accelerator Switchboard (VAS) which +allows both userspace and kernel communicate to co-processor +(hardware accelerator) referred to as the Nest Accelerator (NX). The NX +unit comprises of one or more hardware engines or co-processor types +such as 842 compression, GZIP compression and encryption. On power9, +userspace applications will have access to only GZIP Compression engine +which supports ZLIB and GZIP compression algorithms in the hardware. + +To communicate with NX, kernel has to establish a channel or window and +then requests can be submitted directly without kernel involvement. +Requests to the GZIP engine must be formatted as a co-processor Request +Block (CRB) and these CRBs must be submitted to the NX using COPY/PASTE +instructions to paste the CRB to hardware address that is associated with +the engine's request queue. + +The GZIP engine provides two priority levels of requests: Normal and +High. Only Normal requests are supported from userspace right now. + +This document explains userspace API that is used to interact with +kernel to setup channel / window which can be used to send compression +requests directly to NX accelerator. + + +Overview + + +Application access to the GZIP engine is provided through +/dev/crypto/nx-gzip device node implemented by the VAS/NX device driver. +An application must open the /dev/crypto/nx-gzip device to obtain a file +descriptor (fd). Then should issue VAS_TX_WIN_OPEN ioctl with this fd to +establish connection to the engine. It means send window is opened on GZIP +engine for this process. Once a connection is established, the application +should use the mmap() system call to map the hardware address of engine's +request queue into the application's virtual address space. + +The application can then submit one or more requests to the the engine by +using copy/paste instructions and pasting the CRBs to the virtual address +(aka paste_address) returned by mmap(). User space can close the +established connection or send window by closing the file descriptior +(close(fd)) or upon the process exit. + +Note that applications can send several requests with the same window or +can establish multiple windows, but one window for each file descriptor. + +Following sections provide additional details and references about the +individual steps. + +NX-GZIP Device Node +=== + +There is one /dev/crypto/nx-gzip node in the system and it provides +access to all GZIP engines in the system. The only valid operations on +/dev/crypto/nx-gzip are: + + * open() the device for read and write. + * issue VAS_TX_WIN_OPEN ioctl + * mmap() the engine's request queue into application's virtual + address space (i.e. get a paste_address for the co-processor + engine). + * close the device node. + +Other file operations on this device node are undefined. + +Note that the copy and paste operations go directly to the hardware and +do not go through this device. Refer COPY/PASTE document for more +details. + +Although a system may have several instances of the NX co-processor +engines (typically, one per P9 chip) there is just one +/dev/crypto/nx-gzip device node in the system. When the nx-gzip device +node is opened, Kernel opens send window on a suitable instance of NX +accelerator. It finds CPU on which the user process is executing and +determine the NX instance for the corresponding chip on which this CPU +belongs. + +Applications may chose a specific instance of the NX co-processor using +the vas_id field in the VAS_TX_WIN_OPEN ioctl as detailed below. + +A userspace
[PATCH v3 8/9] crypto/nx: Remove 'pid' in vas_tx_win_attr struct
When window is opened, pid reference is taken for user space
windows. Not needed for kernel windows. So remove 'pid' in
vas_tx_win_attr struct.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/vas.h| 1 -
drivers/crypto/nx/nx-common-powernv.c | 1 -
2 files changed, 2 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index e064953..994db6f 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -86,7 +86,6 @@ struct vas_tx_win_attr {
int wcreds_max;
int lpid;
int pidr; /* hardware PID (from SPRN_PID) */
- int pid;/* linux process id */
int pswid;
int rsvd_txbuf_count;
int tc_mode;
diff --git a/drivers/crypto/nx/nx-common-powernv.c
b/drivers/crypto/nx/nx-common-powernv.c
index f570691..38333e4 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -692,7 +692,6 @@ static struct vas_window *nx_alloc_txwin(struct nx_coproc
*coproc)
*/
vas_init_tx_win_attr(, coproc->ct);
txattr.lpid = 0;/* lpid is 0 for kernel requests */
- txattr.pid = 0; /* pid is 0 for kernel requests */
/*
* Open a VAS send window which is used to send request to NX.
--
1.8.3.1
[PATCH v3 7/9] crypto/nx: Enable and setup GZIP compresstion type
Changes to probe GZIP device-tree nodes, open RX windows and setup
GZIP compression type. No plans to provide GZIP usage in kernel right
now, but this patch enables GZIP for user space usage.
Signed-off-by: Haren Myneni
---
drivers/crypto/nx/nx-common-powernv.c | 43 ++-
1 file changed, 37 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/nx/nx-common-powernv.c
b/drivers/crypto/nx/nx-common-powernv.c
index 82dfa60..f570691 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -65,6 +65,7 @@ struct nx_coproc {
* Using same values as in skiboot or coprocessor type representing
* in NX workbook.
*/
+#define NX_CT_GZIP (2) /* on P9 and later */
#define NX_CT_842 (3)
static int (*nx842_powernv_exec)(const unsigned char *in,
@@ -819,6 +820,9 @@ static int __init vas_cfg_coproc_info(struct device_node
*dn, int chip_id,
if (type == NX_CT_842)
ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_842_HIPRI,
VAS_COP_TYPE_842);
+ else if (type == NX_CT_GZIP)
+ ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_GZIP_HIPRI,
+ VAS_COP_TYPE_GZIP);
if (ret)
goto err_out;
@@ -867,12 +871,16 @@ static int __init vas_cfg_coproc_info(struct device_node
*dn, int chip_id,
return ret;
}
-static int __init nx_coproc_init(int chip_id, int ct_842)
+static int __init nx_coproc_init(int chip_id, int ct_842, int ct_gzip)
{
int ret = 0;
if (opal_check_token(OPAL_NX_COPROC_INIT)) {
ret = opal_nx_coproc_init(chip_id, ct_842);
+
+ if (!ret)
+ ret = opal_nx_coproc_init(chip_id, ct_gzip);
+
if (ret) {
ret = opal_error_code(ret);
pr_err("Failed to initialize NX for chip(%d): %d\n",
@@ -902,8 +910,8 @@ static int __init find_nx_device_tree(struct device_node
*dn, int chip_id,
static int __init nx_powernv_probe_vas(struct device_node *pn)
{
int chip_id, vasid, ret = 0;
+ int ct_842 = 0, ct_gzip = 0;
struct device_node *dn;
- int ct_842 = 0;
chip_id = of_get_ibm_chip_id(pn);
if (chip_id < 0) {
@@ -920,19 +928,24 @@ static int __init nx_powernv_probe_vas(struct device_node
*pn)
for_each_child_of_node(pn, dn) {
ret = find_nx_device_tree(dn, chip_id, vasid, NX_CT_842,
"ibm,p9-nx-842", _842);
+
+ if (!ret)
+ ret = find_nx_device_tree(dn, chip_id, vasid,
+ NX_CT_GZIP, "ibm,p9-nx-gzip", _gzip);
+
if (ret)
return ret;
}
- if (!ct_842) {
- pr_err("NX842 FIFO nodes are missing\n");
+ if (!ct_842 || !ct_gzip) {
+ pr_err("NX FIFO nodes are missing\n");
return -EINVAL;
}
/*
* Initialize NX instance for both high and normal priority FIFOs.
*/
- ret = nx_coproc_init(chip_id, ct_842);
+ ret = nx_coproc_init(chip_id, ct_842, ct_gzip);
return ret;
}
@@ -1072,10 +1085,19 @@ static __init int nx_compress_powernv_init(void)
nx842_powernv_exec = nx842_exec_icswx;
} else {
/*
+* Register VAS user space API for NX GZIP so
+* that user space can use GZIP engine.
+* 842 compression is supported only in kernel.
+*/
+ ret = vas_register_coproc_api(THIS_MODULE);
+
+ /*
* GZIP is not supported in kernel right now.
* So open tx windows only for 842.
*/
- ret = nx_open_percpu_txwins();
+ if (!ret)
+ ret = nx_open_percpu_txwins();
+
if (ret) {
nx_delete_coprocs();
return ret;
@@ -1096,6 +1118,15 @@ static __init int nx_compress_powernv_init(void)
static void __exit nx_compress_powernv_exit(void)
{
+ /*
+* GZIP engine is supported only in power9 or later and nx842_ct
+* is used on power8 (icswx).
+* VAS API for NX GZIP is registered during init for user space
+* use. So delete this API use for GZIP engine.
+*/
+ if (!nx842_ct)
+ vas_unregister_coproc_api();
+
crypto_unregister_alg(_powernv_alg);
nx_delete_coprocs();
--
1.8.3.1
[PATCH v3 6/9] crypto/NX: Make enable code generic to add new GZIP compression type
Make setup and enable code generic to support new GZIP compression type.
Changed nx842 reference to nx and moved some code to new functions.
Functionality is not changed except sparse warning fix - setting NULL
instead of 0 for per_cpu send window in nx_delete_coprocs().
Signed-off-by: Haren Myneni
---
drivers/crypto/nx/nx-common-powernv.c | 161 +-
1 file changed, 101 insertions(+), 60 deletions(-)
diff --git a/drivers/crypto/nx/nx-common-powernv.c
b/drivers/crypto/nx/nx-common-powernv.c
index f42881f..82dfa60 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -40,9 +40,9 @@ struct nx842_workmem {
char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
} __packed __aligned(WORKMEM_ALIGN);
-struct nx842_coproc {
+struct nx_coproc {
unsigned int chip_id;
- unsigned int ct;
+ unsigned int ct;/* Can be 842 or GZIP high/normal*/
unsigned int ci;/* Coprocessor instance, used with icswx */
struct {
struct vas_window *rxwin;
@@ -58,9 +58,15 @@ struct nx842_coproc {
static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
/* no cpu hotplug on powernv, so this list never changes after init */
-static LIST_HEAD(nx842_coprocs);
+static LIST_HEAD(nx_coprocs);
static unsigned int nx842_ct; /* used in icswx function */
+/*
+ * Using same values as in skiboot or coprocessor type representing
+ * in NX workbook.
+ */
+#define NX_CT_842 (3)
+
static int (*nx842_powernv_exec)(const unsigned char *in,
unsigned int inlen, unsigned char *out,
unsigned int *outlenp, void *workmem, int fc);
@@ -666,15 +672,15 @@ static int nx842_powernv_decompress(const unsigned char
*in, unsigned int inlen,
wmem, CCW_FC_842_DECOMP_CRC);
}
-static inline void nx842_add_coprocs_list(struct nx842_coproc *coproc,
+static inline void nx_add_coprocs_list(struct nx_coproc *coproc,
int chipid)
{
coproc->chip_id = chipid;
INIT_LIST_HEAD(>list);
- list_add(>list, _coprocs);
+ list_add(>list, _coprocs);
}
-static struct vas_window *nx842_alloc_txwin(struct nx842_coproc *coproc)
+static struct vas_window *nx_alloc_txwin(struct nx_coproc *coproc)
{
struct vas_window *txwin = NULL;
struct vas_tx_win_attr txattr;
@@ -704,9 +710,9 @@ static struct vas_window *nx842_alloc_txwin(struct
nx842_coproc *coproc)
* cpu_txwin is used in copy/paste operation for each compression /
* decompression request.
*/
-static int nx842_open_percpu_txwins(void)
+static int nx_open_percpu_txwins(void)
{
- struct nx842_coproc *coproc, *n;
+ struct nx_coproc *coproc, *n;
unsigned int i, chip_id;
for_each_possible_cpu(i) {
@@ -714,17 +720,18 @@ static int nx842_open_percpu_txwins(void)
chip_id = cpu_to_chip_id(i);
- list_for_each_entry_safe(coproc, n, _coprocs, list) {
+ list_for_each_entry_safe(coproc, n, _coprocs, list) {
/*
* Kernel requests use only high priority FIFOs. So
* open send windows for these FIFOs.
+* GZIP is not supported in kernel right now.
*/
if (coproc->ct != VAS_COP_TYPE_842_HIPRI)
continue;
if (coproc->chip_id == chip_id) {
- txwin = nx842_alloc_txwin(coproc);
+ txwin = nx_alloc_txwin(coproc);
if (IS_ERR(txwin))
return PTR_ERR(txwin);
@@ -743,13 +750,28 @@ static int nx842_open_percpu_txwins(void)
return 0;
}
+static int __init nx_set_ct(struct nx_coproc *coproc, const char *priority,
+ int high, int normal)
+{
+ if (!strcmp(priority, "High"))
+ coproc->ct = high;
+ else if (!strcmp(priority, "Normal"))
+ coproc->ct = normal;
+ else {
+ pr_err("Invalid RxFIFO priority value\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
- int vasid, int *ct)
+ int vasid, int type, int *ct)
{
struct vas_window *rxwin = NULL;
struct vas_rx_win_attr rxattr;
- struct nx842_coproc *coproc;
u32 lpid, pid, tid, fifo_size;
+ struct nx_coproc *coproc;
u64 rx_fifo;
const char *priority;
int ret;
@@ -794,15 +816,12 @@ static int __init vas_cfg_coproc_info(struct device_node
*dn, int chip_id,
if (!coproc)
[PATCH v3 5/9] crypto/nx: Rename nx-842-powernv file name to nx-common-powernv
Rename nx-842-powernv.c to nx-common-powernv.c to add code for setup
and enable new GZIP compression type. The actual functionality is not
changed in this patch.
Signed-off-by: Haren Myneni
---
drivers/crypto/nx/Makefile|2 +-
drivers/crypto/nx/nx-842-powernv.c| 1062 -
drivers/crypto/nx/nx-common-powernv.c | 1062 +
3 files changed, 1063 insertions(+), 1063 deletions(-)
delete mode 100644 drivers/crypto/nx/nx-842-powernv.c
create mode 100644 drivers/crypto/nx/nx-common-powernv.c
diff --git a/drivers/crypto/nx/Makefile b/drivers/crypto/nx/Makefile
index 015155d..bc89a20 100644
--- a/drivers/crypto/nx/Makefile
+++ b/drivers/crypto/nx/Makefile
@@ -15,4 +15,4 @@ obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES) +=
nx-compress-pseries.o nx-compres
obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV) += nx-compress-powernv.o
nx-compress.o
nx-compress-objs := nx-842.o
nx-compress-pseries-objs := nx-842-pseries.o
-nx-compress-powernv-objs := nx-842-powernv.o
+nx-compress-powernv-objs := nx-common-powernv.o
diff --git a/drivers/crypto/nx/nx-842-powernv.c
b/drivers/crypto/nx/nx-842-powernv.c
deleted file mode 100644
index 8e63326..000
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ /dev/null
@@ -1,1062 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Driver for IBM PowerNV 842 compression accelerator
- *
- * Copyright (C) 2015 Dan Streetman, IBM Corp
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include "nx-842.h"
-
-#include
-
-#include
-#include
-#include
-#include
-#include
-#include
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Dan Streetman ");
-MODULE_DESCRIPTION("842 H/W Compression driver for IBM PowerNV processors");
-MODULE_ALIAS_CRYPTO("842");
-MODULE_ALIAS_CRYPTO("842-nx");
-
-#define WORKMEM_ALIGN (CRB_ALIGN)
-#define CSB_WAIT_MAX (5000) /* ms */
-#define VAS_RETRIES(10)
-
-struct nx842_workmem {
- /* Below fields must be properly aligned */
- struct coprocessor_request_block crb; /* CRB_ALIGN align */
- struct data_descriptor_entry ddl_in[DDL_LEN_MAX]; /* DDE_ALIGN align */
- struct data_descriptor_entry ddl_out[DDL_LEN_MAX]; /* DDE_ALIGN align */
- /* Above fields must be properly aligned */
-
- ktime_t start;
-
- char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
-} __packed __aligned(WORKMEM_ALIGN);
-
-struct nx842_coproc {
- unsigned int chip_id;
- unsigned int ct;
- unsigned int ci;/* Coprocessor instance, used with icswx */
- struct {
- struct vas_window *rxwin;
- int id;
- } vas;
- struct list_head list;
-};
-
-/*
- * Send the request to NX engine on the chip for the corresponding CPU
- * where the process is executing. Use with VAS function.
- */
-static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
-
-/* no cpu hotplug on powernv, so this list never changes after init */
-static LIST_HEAD(nx842_coprocs);
-static unsigned int nx842_ct; /* used in icswx function */
-
-static int (*nx842_powernv_exec)(const unsigned char *in,
- unsigned int inlen, unsigned char *out,
- unsigned int *outlenp, void *workmem, int fc);
-
-/**
- * setup_indirect_dde - Setup an indirect DDE
- *
- * The DDE is setup with the the DDE count, byte count, and address of
- * first direct DDE in the list.
- */
-static void setup_indirect_dde(struct data_descriptor_entry *dde,
- struct data_descriptor_entry *ddl,
- unsigned int dde_count, unsigned int byte_count)
-{
- dde->flags = 0;
- dde->count = dde_count;
- dde->index = 0;
- dde->length = cpu_to_be32(byte_count);
- dde->address = cpu_to_be64(nx842_get_pa(ddl));
-}
-
-/**
- * setup_direct_dde - Setup single DDE from buffer
- *
- * The DDE is setup with the buffer and length. The buffer must be properly
- * aligned. The used length is returned.
- * Returns:
- * NSuccessfully set up DDE with N bytes
- */
-static unsigned int setup_direct_dde(struct data_descriptor_entry *dde,
-unsigned long pa, unsigned int len)
-{
- unsigned int l = min_t(unsigned int, len, LEN_ON_PAGE(pa));
-
- dde->flags = 0;
- dde->count = 0;
- dde->index = 0;
- dde->length = cpu_to_be32(l);
- dde->address = cpu_to_be64(pa);
-
- return l;
-}
-
-/**
- * setup_ddl - Setup DDL from buffer
- *
- * Returns:
- * 0 Successfully set up DDL
- */
-static int setup_ddl(struct data_descriptor_entry *dde,
-struct data_descriptor_entry *ddl,
-unsigned char *buf, unsigned int len,
-bool in)
-{
- unsigned long pa = nx842_get_pa(buf);
- int i, ret, total_len = len;
-
-
[PATCH v3 4/9] crypto/nx: Initialize coproc entry with kzalloc
coproc entry is initialized during NX probe on power9, but not on P8. nx842_delete_coprocs() is used for both and frees receive window if it is allocated. Getting crash for rmmod on P8 since coproc->vas.rxwin is not initialized. This patch replaces kmalloc with kzalloc in nx842_powernv_probe() Signed-off-by: Haren Myneni --- drivers/crypto/nx/nx-842-powernv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/nx/nx-842-powernv.c b/drivers/crypto/nx/nx-842-powernv.c index c037a24..8e63326 100644 --- a/drivers/crypto/nx/nx-842-powernv.c +++ b/drivers/crypto/nx/nx-842-powernv.c @@ -922,7 +922,7 @@ static int __init nx842_powernv_probe(struct device_node *dn) return -EINVAL; } - coproc = kmalloc(sizeof(*coproc), GFP_KERNEL); + coproc = kzalloc(sizeof(*coproc), GFP_KERNEL); if (!coproc) return -ENOMEM; -- 1.8.3.1
[PATCH v3 3/9] powerpc/vas: Add VAS user space API
On power9, userspace can send GZIP compression requests directly to NX
once kernel establishes NX channel / window with VAS. This patch provides
user space API which allows user space to establish channel using open
VAS_TX_WIN_OPEN ioctl, mmap and close operations.
Each window corresponds to file descriptor and application can open
multiple windows. After the window is opened, VAS_TX_WIN_OPEN icoctl to
open a window on specific VAS instance, mmap() system call to map
the hardware address of engine's request queue into the application's
virtual address space.
Then the application can then submit one or more requests to the the
engine by using the copy/paste instructions and pasting the CRBs to
the virtual address (aka paste_address) returned by mmap().
Only NX GZIP coprocessor type is supported right now and allow GZIP
engine access via /dev/crypto/nx-gzip device node.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/vas.h | 11 ++
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-api.c| 290
arch/powerpc/platforms/powernv/vas-window.c | 6 +-
arch/powerpc/platforms/powernv/vas.h| 2 +
5 files changed, 307 insertions(+), 4 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-api.c
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index f93e6b0..e064953 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -163,4 +163,15 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
*/
int vas_paste_crb(struct vas_window *win, int offset, bool re);
+/*
+ * Register / unregister coprocessor type to VAS API which will be exported
+ * to user space. Applications can use this API to open / close window
+ * which can be used to send / receive requests directly to cooprcessor.
+ *
+ * Only NX GZIP coprocessor type is supported now, but this API can be
+ * used for others in future.
+ */
+int vas_register_coproc_api(struct module *mod);
+void vas_unregister_coproc_api(void);
+
#endif /* __ASM_POWERPC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index 395789f..fe3f0fb 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o vas-api.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-api.c
b/arch/powerpc/platforms/powernv/vas-api.c
new file mode 100644
index 000..3473a4a
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-api.c
@@ -0,0 +1,290 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * VAS user space API for its accelerators (Only NX-GZIP is supported now)
+ * Copyright (C) 2019 Haren Myneni, IBM Corp
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include "vas.h"
+
+/*
+ * The driver creates the device node that can be used as follows:
+ * For NX-GZIP
+ *
+ * fd = open("/dev/crypto/nx-gzip", O_RDWR);
+ * rc = ioctl(fd, VAS_TX_WIN_OPEN, );
+ * paste_addr = mmap(NULL, PAGE_SIZE, prot, MAP_SHARED, fd, 0ULL).
+ * vas_copy(, 0, 1);
+ * vas_paste(paste_addr, 0, 1);
+ * close(fd) or exit process to close window.
+ *
+ * where "vas_copy" and "vas_paste" are defined in copy-paste.h.
+ * copy/paste returns to the user space directly. So refer NX hardware
+ * documententation for excat copy/paste usage and completion / error
+ * conditions.
+ */
+
+static char*coproc_dev_name = "nx-gzip";
+static atomic_tcoproc_instid = ATOMIC_INIT(0);
+
+/*
+ * Wrapper object for the nx-gzip device - there is just one instance of
+ * this node for the whole system.
+ */
+static struct coproc_dev {
+ struct cdev cdev;
+ struct device *device;
+ char *name;
+ dev_t devt;
+ struct class *class;
+} coproc_device;
+
+/*
+ * One instance per open of a nx-gzip device. Each coproc_instance is
+ * associated with a VAS window after the caller issues
+ * VAS_GZIP_TX_WIN_OPEN ioctl.
+ */
+struct coproc_instance {
+ int id;
+ struct vas_window *txwin;
+};
+
+static char *coproc_devnode(struct device *dev, umode_t *mode)
+{
+ return kasprintf(GFP_KERNEL, "crypto/%s", dev_name(dev));
+}
+
+static int coproc_open(struct inode *inode, struct file *fp)
+{
+ struct coproc_instance *instance;
+
+ instance = kzalloc(sizeof(*instance), GFP_KE
[PATCH v3 2/9] powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API
Define the VAS_TX_WIN_OPEN ioctl interface for NX GZIP access
from user space. This interface is used to open GZIP send window and
mmap region which can be used by userspace to send requests to NX
directly with copy/paste instructions.
Signed-off-by: Haren Myneni
---
Documentation/userspace-api/ioctl/ioctl-number.rst | 1 +
arch/powerpc/include/uapi/asm/vas-api.h| 22 ++
2 files changed, 23 insertions(+)
create mode 100644 arch/powerpc/include/uapi/asm/vas-api.h
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst
b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 2e91370..deabc73 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -287,6 +287,7 @@ Code Seq#Include File
Comments
'v' 00-1F linux/fs.h conflict!
'v' 00-0F linux/sonypi.h conflict!
'v' 00-0F media/v4l2-subdev.h conflict!
+'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.hVAS API
'v' C0-FF linux/meye.hconflict!
'w' allCERN SCI
driver
'y' 00-1F packet
based user level communications
diff --git a/arch/powerpc/include/uapi/asm/vas-api.h
b/arch/powerpc/include/uapi/asm/vas-api.h
new file mode 100644
index 000..fe95d67
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/vas-api.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Copyright 2019 IBM Corp.
+ */
+
+#ifndef _UAPI_MISC_VAS_H
+#define _UAPI_MISC_VAS_H
+
+#include
+
+#define VAS_MAGIC 'v'
+#define VAS_TX_WIN_OPEN_IOW(VAS_MAGIC, 0x20, struct
vas_tx_win_open_attr)
+
+struct vas_tx_win_open_attr {
+ __u32 version;
+ __s16 vas_id; /* specific instance of vas or -1 for default */
+ __u16 reserved1;
+ __u64 flags; /* Future use */
+ __u64 reserved2[6];
+};
+
+#endif /* _UAPI_MISC_VAS_H */
--
1.8.3.1
[PATCH v3 1/9] powerpc/vas: Initialize window attributes for GZIP coprocessor type
Initialize send and receive window attributes for GZIP high and
normal priority types.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 17 -
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index c60accd..e9ab851 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -817,7 +817,8 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
{
memset(rxattr, 0, sizeof(*rxattr));
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI ||
+ cop == VAS_COP_TYPE_GZIP || cop == VAS_COP_TYPE_GZIP_HIPRI) {
rxattr->pin_win = true;
rxattr->nx_win = true;
rxattr->fault_win = false;
@@ -892,7 +893,8 @@ void vas_init_tx_win_attr(struct vas_tx_win_attr *txattr,
enum vas_cop_type cop)
{
memset(txattr, 0, sizeof(*txattr));
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI ||
+ cop == VAS_COP_TYPE_GZIP || cop == VAS_COP_TYPE_GZIP_HIPRI) {
txattr->rej_no_credit = false;
txattr->rx_wcred_mode = true;
txattr->tx_wcred_mode = true;
@@ -976,9 +978,14 @@ static bool tx_win_args_valid(enum vas_cop_type cop,
if (attr->wcreds_max > VAS_TX_WCREDS_MAX)
return false;
- if (attr->user_win &&
- (cop != VAS_COP_TYPE_FTW || attr->rsvd_txbuf_count))
- return false;
+ if (attr->user_win) {
+ if (attr->rsvd_txbuf_count)
+ return false;
+
+ if (cop != VAS_COP_TYPE_FTW && cop != VAS_COP_TYPE_GZIP &&
+ cop != VAS_COP_TYPE_GZIP_HIPRI)
+ return false;
+ }
return true;
}
--
1.8.3.1
[PATCH v3 0/9] crypto/nx: Enable GZIP engine and provide userpace API
Power9 processor supports Virtual Accelerator Switchboard (VAS) which allows kernel and userspace to send compression requests to Nest Accelerator (NX) directly. The NX unit comprises of 2 842 compression engines and 1 GZIP engine. Linux kernel already has 842 compression support on kernel. This patch series adds GZIP compression support from user space. The GZIP Compression engine implements the ZLIB and GZIP compression algorithms. No plans of adding NX-GZIP compression support in kernel right now. Applications can send requests to NX directly with COPY/PASTE instructions. But kernel has to establish channel / window on NX-GZIP device for the userspace. So userspace access to the GZIP engine is provided through /dev/crypto/nx-gzip device with several operations. An application must open the this device to obtain a file descriptor (fd). Using the fd, application should issue the VAS_TX_WIN_OPEN ioctl to establish a connection to the engine. Once window is opened, should use mmap() system call to map the hardware address of engine's request queue into the application's virtual address space. Then user space forms the request as co-processor Request Block (CRB) and paste this CRB on the mapped HW address using COPY/PASTE instructions. Application can poll on status flags (part of CRB) with timeout for request completion. For VAS_TX_WIN_OPEN ioctl, if user space passes vas_id = -1 (struct vas_tx_win_open_attr), kernel determines the VAS instance on the corresponding chip based on the CPU on which the process is executing. Otherwise, the specified VAS instance is used if application passes the proper VAS instance (vas_id listed in /proc/device-tree/vas@*/ibm,vas_id). Process can open multiple windows with different FDs or can send several requests to NX on the same window at the same time. A userspace library libnxz is available: https://github.com/abalib/power-gzip Applications that use inflate/deflate calls can link with libNXz and use NX GZIP compression without any modification. Tested the available 842 compression on power8 and power9 system to make sure no regression and tested GZIP compression on power9 with tests available in the above link. Thanks to Bulent Abali for nxz library and tests development. Changelog: V2: - Move user space API code to powerpc as suggested. Also this API can be extended to any other coprocessor type that VAS can support in future. Example: Fast thread wakeup feature from VAS - Rebased to 5.6-rc3 V3: - Fix sparse warnings (patches 3&6) Haren Myneni (9): powerpc/vas: Initialize window attributes for GZIP coprocessor type powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API powerpc/vas: Add VAS user space API crypto/nx: Initialize coproc entry with kzalloc crypto/nx: Rename nx-842-powernv file name to nx-common-powernv crypto/NX: Make enable code generic to add new GZIP compression type crypto/nx: Enable and setup GZIP compresstion type crypto/nx: Remove 'pid' in vas_tx_win_attr struct Documentation/powerpc: VAS API Documentation/powerpc/index.rst|1 + Documentation/powerpc/vas-api.rst | 246 + Documentation/userspace-api/ioctl/ioctl-number.rst |1 + arch/powerpc/include/asm/vas.h | 12 +- arch/powerpc/include/uapi/asm/vas-api.h| 22 + arch/powerpc/platforms/powernv/Makefile|2 +- arch/powerpc/platforms/powernv/vas-api.c | 290 + arch/powerpc/platforms/powernv/vas-window.c| 23 +- arch/powerpc/platforms/powernv/vas.h |2 + drivers/crypto/nx/Makefile |2 +- drivers/crypto/nx/nx-842-powernv.c | 1062 -- drivers/crypto/nx/nx-common-powernv.c | 1133 12 files changed, 1723 insertions(+), 1073 deletions(-) create mode 100644 Documentation/powerpc/vas-api.rst create mode 100644 arch/powerpc/include/uapi/asm/vas-api.h create mode 100644 arch/powerpc/platforms/powernv/vas-api.c delete mode 100644 drivers/crypto/nx/nx-842-powernv.c create mode 100644 drivers/crypto/nx/nx-common-powernv.c -- 1.8.3.1
[PATCH V7 13/14] powerpc/vas: Display process stuck message
Process can not close send window until all requests are processed.
Means wait until window state is not busy and send credits are
returned. Display debug messages in case taking longer to close the
window.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 28
1 file changed, 28 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1439a6f..40c303a 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1182,6 +1182,7 @@ static void poll_window_credits(struct vas_window *window)
{
u64 val;
int creds, mode;
+ int count = 0;
val = read_hvwc_reg(window, VREG(WINCTL));
if (window->tx_win)
@@ -1200,10 +1201,27 @@ static void poll_window_credits(struct vas_window
*window)
creds = GET_FIELD(VAS_LRX_WCRED, val);
}
+ /*
+* Takes around few milliseconds to complete all pending requests
+* and return credits.
+* TODO: Scan fault FIFO and invalidate CRBs points to this window
+* and issue CRB Kill to stop all pending requests. Need only
+* if there is a bug in NX or fault handling in kernel.
+*/
if (creds < window->wcreds_max) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(10));
+ count++;
+ /*
+* Process can not close send window until all credits are
+* returned.
+*/
+ if (!(count % 1))
+ pr_debug("VAS: pid %d stuck. Waiting for credits
returned for Window(%d). creds %d, Retries %d\n",
+ vas_window_pid(window), window->winid,
+ creds, count);
+
goto retry;
}
}
@@ -1217,6 +1235,7 @@ static void poll_window_busy_state(struct vas_window
*window)
{
int busy;
u64 val;
+ int count = 0;
retry:
val = read_hvwc_reg(window, VREG(WIN_STATUS));
@@ -1225,6 +1244,15 @@ static void poll_window_busy_state(struct vas_window
*window)
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(5));
+ count++;
+ /*
+* Takes around few milliseconds to process all pending
+* requests.
+*/
+ if (!(count % 1))
+ pr_debug("VAS: pid %d stuck. Window (ID=%d) is in busy
state. Retries %d\n",
+ vas_window_pid(window), window->winid, count);
+
goto retry;
}
}
--
1.8.3.1
[PATCH V7 14/14] powerpc/vas: Free send window in VAS instance after credits returned
NX may be processing requests while trying to close window. Wait until all credits are returned and then free send window from VAS instance. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/powernv/vas-window.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c index 40c303a..c60accd 100644 --- a/arch/powerpc/platforms/powernv/vas-window.c +++ b/arch/powerpc/platforms/powernv/vas-window.c @@ -1317,14 +1317,14 @@ int vas_win_close(struct vas_window *window) unmap_paste_region(window); - clear_vinst_win(window); - poll_window_busy_state(window); unpin_close_window(window); poll_window_credits(window); + clear_vinst_win(window); + poll_window_castout(window); /* if send window, drop reference to matching receive window */ -- 1.8.3.1
[PATCH V7 12/14] powerpc/vas: Return credits after handling fault
NX expects OS to return credit for send window after processing each
fault. Also credit has to be returned even for fault window.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 9 +
arch/powerpc/platforms/powernv/vas-window.c | 17 +
arch/powerpc/platforms/powernv/vas.h| 1 +
3 files changed, 27 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 940adc5..cb8c51d 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -237,6 +237,10 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
memcpy(crb, fifo, CRB_SIZE);
entry->stamp.nx.pswid = cpu_to_be32(FIFO_INVALID_ENTRY);
entry->ccw |= cpu_to_be32(CCW0_INVALID);
+ /*
+* Return credit for the fault window.
+*/
+ vas_return_credit(vinst->fault_win, 0);
mutex_unlock(>mutex);
pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
@@ -266,6 +270,11 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
}
update_csb(window, crb);
+ /*
+* Return credit for send window after processing
+* fault CRB.
+*/
+ vas_return_credit(window, 1);
} while (true);
}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 427a884..1439a6f 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1318,6 +1318,23 @@ int vas_win_close(struct vas_window *window)
}
EXPORT_SYMBOL_GPL(vas_win_close);
+/*
+ * Return credit for the given window.
+ */
+void vas_return_credit(struct vas_window *window, bool tx)
+{
+ uint64_t val;
+
+ val = 0ULL;
+ if (tx) { /* send window */
+ val = SET_FIELD(VAS_TX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), val);
+ } else {
+ val = SET_FIELD(VAS_LRX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), val);
+ }
+}
+
struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid)
{
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index bc728d7..8c39a7d 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -428,6 +428,7 @@ struct vas_winctx {
extern void vas_window_free_dbgdir(struct vas_window *win);
extern int vas_setup_fault_window(struct vas_instance *vinst);
extern irqreturn_t vas_fault_thread_fn(int irq, void *data);
+extern void vas_return_credit(struct vas_window *window, bool tx);
extern struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid);
--
1.8.3.1
[PATCH V7 11/14] powerpc/vas: Do not use default credits for receive window
System checkstops if RxFIFO overruns with more requests than the
maximum possible number of CRBs allowed in FIFO at any time. So
max credits value (rxattr.wcreds_max) is set and is passed to
vas_rx_win_open() by the the driver.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 4 ++--
arch/powerpc/platforms/powernv/vas.h| 2 --
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 7587258..427a884 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -772,7 +772,7 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
- if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ if (!attr->wcreds_max)
return false;
if (attr->nx_win) {
@@ -877,7 +877,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
- rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ rxwin->wcreds_max = rxattr->wcreds_max;
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 16aa8ec..bc728d7 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -101,11 +101,9 @@
/*
* Initial per-process credits.
* Max send window credits:4K-1 (12-bits in VAS_TX_WCRED)
- * Max receive window credits: 64K-1 (16 bits in VAS_LRX_WCRED)
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
--
1.8.3.1
[PATCH V7 10/14] powerpc/vas: Print CRB and FIFO values
Dump FIFO entries if could not find send window and print CRB
for debugging.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 41 ++
1 file changed, 41 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 751ce48..940adc5 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -26,6 +26,28 @@
*/
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+static void dump_crb(struct coprocessor_request_block *crb)
+{
+ struct data_descriptor_entry *dde;
+ struct nx_fault_stamp *nx;
+
+ dde = >source;
+ pr_devel("SrcDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ dde = >target;
+ pr_devel("TgtDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ nx = >stamp.nx;
+ pr_devel("NX Stamp: PSWID 0x%x, FSA 0x%llx, flags 0x%x, FS 0x%x\n",
+ be32_to_cpu(nx->pswid),
+ be64_to_cpu(crb->stamp.nx.fault_storage_addr),
+ nx->flags, nx->fault_status);
+}
+
/*
* Update the CSB to indicate a translation error.
*
@@ -138,6 +160,23 @@ static void update_csb(struct vas_window *window,
pid_vnr(pid), rc);
}
+static void dump_fifo(struct vas_instance *vinst, void *entry)
+{
+ int i;
+ unsigned long *fifo = entry;
+ unsigned long *end = vinst->fault_fifo + vinst->fault_fifo_size;
+
+ pr_err("Fault fifo size %d, Max crbs %d\n", vinst->fault_fifo_size,
+ vinst->fault_fifo_size / CRB_SIZE);
+
+ /* Dump 10 CRB entries or until end of FIFO */
+ pr_err("Fault FIFO Dump:\n");
+ for (i = 0; i < 10*(CRB_SIZE/8) && fifo < end; i += 4, fifo += 4) {
+ pr_err("[%.3d, %p]: 0x%.16lx 0x%.16lx 0x%.16lx 0x%.16lx\n",
+ i, fifo, *fifo, *(fifo+1), *(fifo+2), *(fifo+3));
+ }
+}
+
/*
* Process valid CRBs in fault FIFO.
*/
@@ -204,6 +243,7 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
+ dump_crb(crb);
window = vas_pswid_to_window(vinst,
be32_to_cpu(crb->stamp.nx.pswid));
@@ -214,6 +254,7 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
* even clean it up (return credit).
* But we should not get here.
*/
+ dump_fifo(vinst, (void *)entry);
pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
vinst->vas_id, vinst->fault_fifo, fifo,
be32_to_cpu(crb->stamp.nx.pswid),
--
1.8.3.1
[PATCH V7 09/14] powerpc/vas: Update CSB and notify process for fault CRBs
For each fault CRB, update fault address in CRB (fault_storage_addr)
and translation error status in CSB so that user space can touch the
fault address and resend the request. If the user space passed invalid
CSB address send signal to process with SIGSEGV.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 114 +
1 file changed, 114 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 1c6d5cc..751ce48 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,6 +27,118 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Update the CSB to indicate a translation error.
+ *
+ * If we are unable to update the CSB means copy_to_user failed due to
+ * invalid csb_addr, send a signal to the process.
+ *
+ * Remaining settings in the CSB are based on wait_for_csb() of
+ * NX-GZIP.
+ */
+static void update_csb(struct vas_window *window,
+ struct coprocessor_request_block *crb)
+{
+ int rc;
+ struct pid *pid;
+ void __user *csb_addr;
+ struct task_struct *tsk;
+ struct kernel_siginfo info;
+ struct coprocessor_status_block csb;
+
+ /*
+* NX user space windows can not be opened for task->mm=NULL
+* and faults will not be generated for kernel requests.
+*/
+ if (!window->mm || !window->user_win)
+ return;
+
+ csb_addr = (void __user *)be64_to_cpu(crb->csb_addr);
+
+ csb.cc = CSB_CC_TRANSLATION;
+ csb.ce = CSB_CE_TERMINATION;
+ csb.cs = 0;
+ csb.count = 0;
+
+ /*
+* NX operates and returns in BE format as defined CRB struct.
+* So return fault_storage_addr in BE as NX pastes in FIFO and
+* expects user space to convert to CPU format.
+*/
+ csb.address = crb->stamp.nx.fault_storage_addr;
+ csb.flags = 0;
+
+ pid = window->pid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Send window will be closed after processing all NX requests
+* and process exits after closing all windows. In multi-thread
+* applications, thread may not exists, but does not close FD
+* (means send window) upon exit. Parent thread (tgid) can use
+* and close the window later.
+* pid and mm references are taken when window is opened by
+* process (pid). So tgid is used only when child thread opens
+* a window and exits without closing it in multithread tasks.
+*/
+ if (!tsk) {
+ pid = window->tgid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Parent thread will be closing window during its exit.
+* So should not get here.
+*/
+ if (!tsk)
+ return;
+ }
+
+ /* Return if the task is exiting. */
+ if (tsk->flags & PF_EXITING) {
+ put_task_struct(tsk);
+ return;
+ }
+
+ use_mm(window->mm);
+ rc = copy_to_user(csb_addr, , sizeof(csb));
+ /*
+* User space polls on csb.flags (first byte). So add barrier
+* then copy first byte with csb flags update.
+*/
+ smp_mb();
+ if (!rc) {
+ csb.flags = CSB_V;
+ rc = copy_to_user(csb_addr, , sizeof(u8));
+ }
+ unuse_mm(window->mm);
+ put_task_struct(tsk);
+
+ /* Success */
+ if (!rc)
+ return;
+
+ pr_debug("Invalid CSB address 0x%p signalling pid(%d)\n",
+ csb_addr, pid_vnr(pid));
+
+ clear_siginfo();
+ info.si_signo = SIGSEGV;
+ info.si_errno = EFAULT;
+ info.si_code = SEGV_MAPERR;
+ info.si_addr = csb_addr;
+
+ /*
+* process will be polling on csb.flags after request is sent to
+* NX. So generally CSB update should not fail except when an
+* application does not follow the process properly. So an error
+* message will be displayed and leave it to user space whether
+* to ignore or handle this signal.
+*/
+ rcu_read_lock();
+ rc = kill_pid_info(SIGSEGV, , pid);
+ rcu_read_unlock();
+
+ pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
+ pid_vnr(pid), rc);
+}
+
+/*
* Process valid CRBs in fault FIFO.
*/
irqreturn_t vas_fault_thread_fn(int irq, void *data)
@@ -111,6 +224,7 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
return IRQ_HANDLED;
}
+ update_csb(window, crb);
} while (true);
}
--
1.8.3.1
[PATCH V7 08/14] powerpc/vas: Take reference to PID and mm for user space windows
Process close windows after its requests are completed. In multi-thread
applications, child can open a window but release FD will not be called
upon its exit. Parent thread will be closing it later upon its exit.
The parent can also send NX requests with this window and NX can
generate page faults. After kernel handles the page fault, send
signal to process by using PID if CSB address is invalid. Parent
thread will not receive signal since its PID is different from the one
saved in vas_window. So use tgid in case if the task for the pid saved
in window is not running and send signal to its parent.
To prevent reusing the pid until the window closed, take reference to
pid and task mm.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-debug.c | 2 +-
arch/powerpc/platforms/powernv/vas-window.c | 53 ++---
arch/powerpc/platforms/powernv/vas.h| 9 -
3 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index 09e63df..ef9a717 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -38,7 +38,7 @@ static int info_show(struct seq_file *s, void *private)
seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
window->tx_win ? "Send" : "Receive");
- seq_printf(s, "Pid : %d\n", window->pid);
+ seq_printf(s, "Pid : %d\n", vas_window_pid(window));
unlock:
mutex_unlock(_mutex);
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a45d81d..7587258 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -12,6 +12,8 @@
#include
#include
#include
+#include
+#include
#include
#include
#include "vas.h"
@@ -876,8 +878,6 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
- if (rxattr->user_win)
- rxwin->pid = task_pid_vnr(current);
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
@@ -1027,7 +1027,6 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->tx_win = 1;
txwin->rxwin = rxwin;
txwin->nx_win = txwin->rxwin->nx_win;
- txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
@@ -1068,8 +1067,43 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
goto free_window;
}
- set_vinst_win(vinst, txwin);
+ if (txwin->user_win) {
+ /*
+* Window opened by child thread may not be closed when
+* it exits. So take reference to its pid and release it
+* when the window is free by parent thread.
+* Acquire a reference to the task's pid to make sure
+* pid will not be re-used - needed only for multithread
+* applications.
+*/
+ txwin->pid = get_task_pid(current, PIDTYPE_PID);
+ /*
+* Acquire a reference to the task's mm.
+*/
+ txwin->mm = get_task_mm(current);
+ if (!txwin->mm) {
+ put_pid(txwin->pid);
+ pr_err("VAS: pid(%d): mm_struct is not found\n",
+ current->pid);
+ rc = -EPERM;
+ goto free_window;
+ }
+
+ mmgrab(txwin->mm);
+ mmput(txwin->mm);
+ mm_context_add_copro(txwin->mm);
+ /*
+* Process closes window during exit. In the case of
+* multithread application, child can open window and
+* can exit without closing it. Expects parent thread
+* to use and close the window. So do not need to take
+* pid reference for parent thread.
+*/
+ txwin->tgid = find_get_pid(task_tgid_vnr(current));
+ }
+
+ set_vinst_win(vinst, txwin);
return txwin;
free_window:
@@ -1266,8 +1300,17 @@ int vas_win_close(struct vas_window *window)
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
- if (window->tx_win)
+ if (window->tx_win) {
+ if (window->user_win) {
+ /* Drop references to pid and mm */
+ put_pid(wi
[PATCH V7 07/14] powerpc/vas: Register NX with fault window ID and IRQ port value
For each user space send window, register NX with fault window ID
and port value so that NX paste CRBs in this fault FIFO when it
sees fault on the request buffer.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 15 +--
arch/powerpc/platforms/powernv/vas.h| 15 +++
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 7c6f55f..a45d81d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -373,7 +373,7 @@ int init_winctx_regs(struct vas_window *window, struct
vas_winctx *winctx)
init_xlate_regs(window, winctx->user_win);
val = 0ULL;
- val = SET_FIELD(VAS_FAULT_TX_WIN, val, 0);
+ val = SET_FIELD(VAS_FAULT_TX_WIN, val, winctx->fault_win_id);
write_hvwc_reg(window, VREG(FAULT_TX_WIN), val);
/* In PowerNV, interrupts go to HV. */
@@ -748,6 +748,8 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (rxwin->vinst->virq)
+ winctx->irq_port = rxwin->vinst->irq_port;
}
static bool rx_win_args_valid(enum vas_cop_type cop,
@@ -944,13 +946,22 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
winctx->rx_win_id = txwin->rxwin->winid;
+ /*
+* IRQ and fault window setup is successful. Set fault window
+* for the send window so that ready to handle faults.
+*/
+ if (txwin->vinst->virq)
+ winctx->fault_win_id = txwin->vinst->fault_win->winid;
winctx->dma_type = VAS_DMA_TYPE_INJECT;
winctx->tc_mode = txattr->tc_mode;
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (txwin->vinst->virq)
+ winctx->irq_port = txwin->vinst->irq_port;
- winctx->pswid = 0;
+ winctx->pswid = txattr->pswid ? txattr->pswid :
+ encode_pswid(txwin->vinst->vas_id, txwin->winid);
}
static bool tx_win_args_valid(enum vas_cop_type cop,
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index ecae7cd..310b8a0 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -468,6 +468,21 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ return ((u32)winid | (vasid << (31 - 7)));
+}
+
static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
{
if (vasid)
--
1.8.3.1
[PATCH V7 06/14] powerpc/vas: Setup thread IRQ handler per VAS instance
Setup thread IRQ handler per each VAS instance. When NX sees a fault
on CRB, kernel gets an interrupt and vas_fault_handler will be
executed to process fault CRBs. Read all valid CRBs from fault FIFO,
determine the corresponding send window from CRB and process fault
requests.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 90 +
arch/powerpc/platforms/powernv/vas-window.c | 60 +++
arch/powerpc/platforms/powernv/vas.c| 49 +++-
arch/powerpc/platforms/powernv/vas.h| 6 ++
4 files changed, 204 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 4044998..1c6d5cc 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include "vas.h"
@@ -25,6 +26,95 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Process valid CRBs in fault FIFO.
+ */
+irqreturn_t vas_fault_thread_fn(int irq, void *data)
+{
+ struct vas_instance *vinst = data;
+ struct coprocessor_request_block *crb, *entry;
+ struct coprocessor_request_block buf;
+ struct vas_window *window;
+ unsigned long flags;
+ void *fifo;
+
+ crb =
+
+ /*
+* VAS can interrupt with multiple page faults. So process all
+* valid CRBs within fault FIFO until reaches invalid CRB.
+* NX updates nx_fault_stamp in CRB and pastes in fault FIFO.
+* kernel retrives send window from parition send window ID
+* (pswid) in nx_fault_stamp. So pswid should be valid and
+* ccw[0] (in be) should be zero since this bit is reserved.
+* If user space touches this bit, NX returns with "CRB format
+* error".
+*
+* After reading CRB entry, invalidate it with pswid (set
+* 0x) and ccw[0] (set to 1).
+*
+* In case kernel receives another interrupt with different page
+* fault, CRBs are already processed by the previous handling. So
+* will be returned from this function when it sees invalid CRB.
+*/
+ do {
+ mutex_lock(>mutex);
+
+ spin_lock_irqsave(>fault_lock, flags);
+ /*
+* Advance the fault fifo pointer to next CRB.
+* Use CRB_SIZE rather than sizeof(*crb) since the latter is
+* aligned to CRB_ALIGN (256) but the CRB written to by VAS is
+* only CRB_SIZE in len.
+*/
+ fifo = vinst->fault_fifo + (vinst->fault_crbs * CRB_SIZE);
+ entry = fifo;
+
+ if ((entry->stamp.nx.pswid == cpu_to_be32(FIFO_INVALID_ENTRY))
+ || (entry->ccw & cpu_to_be32(CCW0_INVALID))) {
+ atomic_set(>faults_in_progress, 0);
+ spin_unlock_irqrestore(>fault_lock, flags);
+ mutex_unlock(>mutex);
+ return IRQ_HANDLED;
+ }
+
+ spin_unlock_irqrestore(>fault_lock, flags);
+ vinst->fault_crbs++;
+ if (vinst->fault_crbs == (vinst->fault_fifo_size / CRB_SIZE))
+ vinst->fault_crbs = 0;
+
+ memcpy(crb, fifo, CRB_SIZE);
+ entry->stamp.nx.pswid = cpu_to_be32(FIFO_INVALID_ENTRY);
+ entry->ccw |= cpu_to_be32(CCW0_INVALID);
+ mutex_unlock(>mutex);
+
+ pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ vinst->fault_crbs);
+
+ window = vas_pswid_to_window(vinst,
+ be32_to_cpu(crb->stamp.nx.pswid));
+
+ if (IS_ERR(window)) {
+ /*
+* We got an interrupt about a specific send
+* window but we can't find that window and we can't
+* even clean it up (return credit).
+* But we should not get here.
+*/
+ pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ be32_to_cpu(crb->stamp.nx.pswid),
+ vinst->fault_crbs);
+
+ WARN_ON_ONCE(1);
+ atomic_set(>faults_in_progress, 0);
+ return IRQ_HANDLED;
+ }
+
+ } while (true);
+}
+
+/*
* Fault window is opened per VAS instance. NX pastes fault
[PATCH V7 05/14] powerpc/vas: Setup fault window per VAS instance
Setup fault window for each VAS instance. When NX gets a fault on
request buffer, write fault CRBs in the corresponding fault FIFO and
then sends an interrupt to the OS.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-fault.c | 77 +
arch/powerpc/platforms/powernv/vas-window.c | 4 +-
arch/powerpc/platforms/powernv/vas.c| 20
arch/powerpc/platforms/powernv/vas.h| 16 ++
5 files changed, 116 insertions(+), 3 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index c0f8120..395789f 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
new file mode 100644
index 000..4044998
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -0,0 +1,77 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * VAS Fault handling.
+ * Copyright 2019, IBM Corporation
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "vas.h"
+
+/*
+ * The maximum FIFO size for fault window can be 8MB
+ * (VAS_RX_FIFO_SIZE_MAX). Using 4MB FIFO since each VAS
+ * instance will be having fault window.
+ * 8MB FIFO can be used if expects more faults for each VAS
+ * instance.
+ */
+#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+
+/*
+ * Fault window is opened per VAS instance. NX pastes fault CRB in fault
+ * FIFO upon page faults.
+ */
+int vas_setup_fault_window(struct vas_instance *vinst)
+{
+ struct vas_rx_win_attr attr;
+
+ vinst->fault_fifo_size = VAS_FAULT_WIN_FIFO_SIZE;
+ vinst->fault_fifo = kzalloc(vinst->fault_fifo_size, GFP_KERNEL);
+ if (!vinst->fault_fifo) {
+ pr_err("Unable to alloc %d bytes for fault_fifo\n",
+ vinst->fault_fifo_size);
+ return -ENOMEM;
+ }
+
+ /*
+* Invalidate all CRB entries. NX pastes valid entry for each fault.
+*/
+ memset(vinst->fault_fifo, FIFO_INVALID_ENTRY, vinst->fault_fifo_size);
+ vas_init_rx_win_attr(, VAS_COP_TYPE_FAULT);
+
+ attr.rx_fifo_size = vinst->fault_fifo_size;
+ attr.rx_fifo = vinst->fault_fifo;
+
+ /*
+* Max creds is based on number of CRBs can fit in the FIFO.
+* (fault_fifo_size/CRB_SIZE). If 8MB FIFO is used, max creds
+* will be 0x since the receive creds field is 16bits wide.
+*/
+ attr.wcreds_max = vinst->fault_fifo_size / CRB_SIZE;
+ attr.lnotify_lpid = 0;
+ attr.lnotify_pid = mfspr(SPRN_PID);
+ attr.lnotify_tid = mfspr(SPRN_PID);
+
+ vinst->fault_win = vas_rx_win_open(vinst->vas_id, VAS_COP_TYPE_FAULT,
+ );
+
+ if (IS_ERR(vinst->fault_win)) {
+ pr_err("VAS: Error %ld opening FaultWin\n",
+ PTR_ERR(vinst->fault_win));
+ kfree(vinst->fault_fifo);
+ return PTR_ERR(vinst->fault_win);
+ }
+
+ pr_devel("VAS: Created FaultWin %d, LPID/PID/TID [%d/%d/%d]\n",
+ vinst->fault_win->winid, attr.lnotify_lpid,
+ attr.lnotify_pid, attr.lnotify_tid);
+
+ return 0;
+}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0c0d27d..1783fa9 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -827,9 +827,9 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
rxattr->fault_win = true;
rxattr->notify_disable = true;
rxattr->rx_wcred_mode = true;
- rxattr->tx_wcred_mode = true;
rxattr->rx_win_ord_mode = true;
- rxattr->tx_win_ord_mode = true;
+ rxattr->rej_no_credit = true;
+ rxattr->tc_mode = VAS_THRESH_DISABLED;
} else if (cop == VAS_COP_TYPE_FTW) {
rxattr->user_win = true;
rxattr->intr_disable = true;
diff --git a/arch/powerpc/platfor
[PATCH V7 04/14] powerpc/vas: Alloc and setup IRQ and trigger port address
Alloc IRQ and get trigger port address for each VAS instance. Kernel
register this IRQ per VAS instance and sets this port for each send
window. NX interrupts the kernel when it sees page fault.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas.c | 34 --
arch/powerpc/platforms/powernv/vas.h | 2 ++
2 files changed, 30 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index ed9cc6d..168ab68 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -15,6 +15,7 @@
#include
#include
#include
+#include
#include "vas.h"
@@ -25,10 +26,12 @@
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, cpu, vasid;
- struct resource *res;
- struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
+ struct vas_instance *vinst;
+ uint32_t chipid, irq;
+ struct resource *res;
+ int rc, cpu, vasid;
+ uint64_t port;
rc = of_property_read_u32(dn, "ibm,vas-id", );
if (rc) {
@@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev)
return -ENODEV;
}
+ rc = of_property_read_u32(dn, "ibm,chip-id", );
+ if (rc) {
+ pr_err("No ibm,chip-id property for %s?\n", pdev->name);
+ return -ENODEV;
+ }
+
if (pdev->num_resources != 4) {
pr_err("Unexpected DT configuration for [%s, %d]\n",
pdev->name, vasid);
@@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev)
vinst->paste_win_id_shift = 63 - res->end;
- pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, "
- "paste_win_id_shift 0x%llx\n", pdev->name, vasid,
- vinst->paste_base_addr, vinst->paste_win_id_shift);
+ rc = xive_native_alloc_get_irq_info(chipid, , );
+ if (rc)
+ return rc;
+
+ vinst->virq = irq_create_mapping(NULL, irq);
+ if (!vinst->virq) {
+ pr_err("Inst%d: Unable to map global irq %d\n",
+ vinst->vas_id, irq);
+ return -EINVAL;
+ }
+
+ vinst->irq_port = port;
+ pr_devel("Initialized instance [%s, %d] paste_base 0x%llx
paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n",
+ pdev->name, vasid, vinst->paste_base_addr,
+ vinst->paste_win_id_shift, vinst->virq,
+ vinst->irq_port);
for_each_possible_cpu(cpu) {
if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 5574aec..598608b 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -313,6 +313,8 @@ struct vas_instance {
u64 paste_base_addr;
u64 paste_win_id_shift;
+ u64 irq_port;
+ int virq;
struct mutex mutex;
struct vas_window *rxwin[VAS_COP_TYPE_MAX];
struct vas_window *windows[VAS_WINDOWS_PER_CHIP];
--
1.8.3.1
[PATCH V7 03/14] powerpc/vas: Define nx_fault_stamp in coprocessor_request_block
Kernel sets fault address and status in CRB for NX page fault on user
space address after processing page fault. User space gets the signal
and handles the fault mentioned in CRB by bringing the page in to
memory and send NX request again.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/icswx.h | 18 +-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/icswx.h b/arch/powerpc/include/asm/icswx.h
index 9872f85..b233d1e 100644
--- a/arch/powerpc/include/asm/icswx.h
+++ b/arch/powerpc/include/asm/icswx.h
@@ -108,6 +108,17 @@ struct data_descriptor_entry {
__be64 address;
} __packed __aligned(DDE_ALIGN);
+/* 4.3.2 NX-stamped Fault CRB */
+
+#define NX_STAMP_ALIGN (0x10)
+
+struct nx_fault_stamp {
+ __be64 fault_storage_addr;
+ __be16 reserved;
+ __u8 flags;
+ __u8 fault_status;
+ __be32 pswid;
+} __packed __aligned(NX_STAMP_ALIGN);
/* Chapter 6.5.2 Coprocessor-Request Block (CRB) */
@@ -135,7 +146,12 @@ struct coprocessor_request_block {
struct coprocessor_completion_block ccb;
- u8 reserved[48];
+ union {
+ struct nx_fault_stamp nx;
+ u8 reserved[16];
+ } stamp;
+
+ u8 reserved[32];
struct coprocessor_status_block csb;
} __packed __aligned(CRB_ALIGN);
--
1.8.3.1
[PATCH V7 02/14] powerpc/xive: Define xive_native_alloc_get_irq_info()
pnv_ocxl_alloc_xive_irq() in ocxl.c allocates IRQ and gets trigger port
address. VAS also needs this function, but based on chip ID. So moved
this common function to xive/native.c.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 2 ++
arch/powerpc/platforms/powernv/ocxl.c | 20 ++--
arch/powerpc/sysdev/xive/native.c | 23 +++
3 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index d08ea11..fd337da 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -139,6 +139,8 @@ int xive_native_set_queue_state(u32 vp_id, uint32_t prio,
u32 qtoggle,
int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
bool xive_native_has_queue_state_support(void);
extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+extern int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq,
+ u64 *trigger_addr);
static inline u32 xive_native_alloc_irq(void)
{
diff --git a/arch/powerpc/platforms/powernv/ocxl.c
b/arch/powerpc/platforms/powernv/ocxl.c
index 8c65aac..fb8f99a 100644
--- a/arch/powerpc/platforms/powernv/ocxl.c
+++ b/arch/powerpc/platforms/powernv/ocxl.c
@@ -487,24 +487,8 @@ int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data,
int pe_handle)
int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr)
{
- __be64 flags, trigger_page;
- s64 rc;
- u32 hwirq;
-
- hwirq = xive_native_alloc_irq();
- if (!hwirq)
- return -ENOENT;
-
- rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
- NULL);
- if (rc || !trigger_page) {
- xive_native_free_irq(hwirq);
- return -ENOENT;
- }
- *irq = hwirq;
- *trigger_addr = be64_to_cpu(trigger_page);
- return 0;
-
+ return xive_native_alloc_get_irq_info(OPAL_XIVE_ANY_CHIP, irq,
+ trigger_addr);
}
EXPORT_SYMBOL_GPL(pnv_ocxl_alloc_xive_irq);
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 14d4406..abdd892 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -295,6 +295,29 @@ u32 xive_native_alloc_irq_on_chip(u32 chip_id)
}
EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
+int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq, u64 *trigger_addr)
+{
+ __be64 flags, trigger_page;
+ u32 hwirq;
+ s64 rc;
+
+ hwirq = xive_native_alloc_irq_on_chip(chip_id);
+ if (!hwirq)
+ return -ENOENT;
+
+ rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
+ NULL);
+ if (rc || !trigger_page) {
+ xive_native_free_irq(hwirq);
+ return -ENOENT;
+ }
+ *irq = hwirq;
+ *trigger_addr = be64_to_cpu(trigger_page);
+
+ return 0;
+}
+EXPORT_SYMBOL(xive_native_alloc_get_irq_info);
+
void xive_native_free_irq(u32 irq)
{
for (;;) {
--
1.8.3.1
[PATCH V7 01/14] powerpc/xive: Define xive_native_alloc_irq_on_chip()
This function allocates IRQ on a specific chip. VAS needs per chip
IRQ allocation and will have IRQ handler per VAS instance.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 9 -
arch/powerpc/sysdev/xive/native.c | 6 +++---
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 93f982db..d08ea11 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -5,6 +5,8 @@
#ifndef _ASM_POWERPC_XIVE_H
#define _ASM_POWERPC_XIVE_H
+#include
+
#define XIVE_INVALID_VP0x
#ifdef CONFIG_PPC_XIVE
@@ -108,7 +110,6 @@ struct xive_q {
int xive_native_populate_irq_data(u32 hw_irq,
struct xive_irq_data *data);
void xive_cleanup_irq_data(struct xive_irq_data *xd);
-u32 xive_native_alloc_irq(void);
void xive_native_free_irq(u32 irq);
int xive_native_configure_irq(u32 hw_irq, u32 target, u8 prio, u32 sw_irq);
@@ -137,6 +138,12 @@ int xive_native_set_queue_state(u32 vp_id, uint32_t prio,
u32 qtoggle,
u32 qindex);
int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
bool xive_native_has_queue_state_support(void);
+extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+
+static inline u32 xive_native_alloc_irq(void)
+{
+ return xive_native_alloc_irq_on_chip(OPAL_XIVE_ANY_CHIP);
+}
#else
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 0ff6b73..14d4406 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -279,12 +279,12 @@ static int xive_native_get_ipi(unsigned int cpu, struct
xive_cpu *xc)
}
#endif /* CONFIG_SMP */
-u32 xive_native_alloc_irq(void)
+u32 xive_native_alloc_irq_on_chip(u32 chip_id)
{
s64 rc;
for (;;) {
- rc = opal_xive_allocate_irq(OPAL_XIVE_ANY_CHIP);
+ rc = opal_xive_allocate_irq(chip_id);
if (rc != OPAL_BUSY)
break;
msleep(OPAL_BUSY_DELAY_MS);
@@ -293,7 +293,7 @@ u32 xive_native_alloc_irq(void)
return 0;
return rc;
}
-EXPORT_SYMBOL_GPL(xive_native_alloc_irq);
+EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
void xive_native_free_irq(u32 irq)
{
--
1.8.3.1
[PATCH V7 00/14] powerpc/vas: Page fault handling for user space NX requests
On power9, Virtual Accelerator Switchboard (VAS) allows user space or kernel to communicate with Nest Accelerator (NX) directly using COPY/PASTE instructions. NX provides various functionalities such as compression, encryption and etc. But only compression (842 and GZIP formats) is supported in Linux kernel on power9. 842 compression driver (drivers/crypto/nx/nx-842-powernv.c) is already included in Linux. Only GZIP support will be available from user space. Applications can issue GZIP compression / decompression requests to NX with COPY/PASTE instructions. When NX is processing these requests, can hit fault on the request buffer (not in memory). It issues an interrupt and pastes fault CRB in fault FIFO. Expects kernel to handle this fault and return credits for both send and fault windows after processing. This patch series adds IRQ and fault window setup, and NX fault handling: - Alloc IRQ and trigger port address, and configure IRQ per VAS instance. - Set port# for each window to generate an interrupt when noticed fault. - Set fault window and FIFO on which NX paste fault CRB. - Setup IRQ thread fault handler per VAS instance. - When receiving an interrupt, Read CRBs from fault FIFO and update coprocessor_status_block (CSB) in the corresponding CRB with translation failure (CSB_CC_TRANSLATION). After issuing NX requests, process polls on CSB address. When it sees translation error, can touch the request buffer to bring the page in to memory and reissue NX request. - If copy_to_user fails on user space CSB address, OS sends SEGV signal. Tested these patches with NX-GZIP support and will be posting this series soon. Patches 1 & 2: Define alloc IRQ and get port address per chip which are needed to alloc IRQ per VAS instance. Patch 3: Define nx_fault_stamp on which NX writes fault status for the fault CRB Patch 4: Alloc and setup IRQ and trigger port address for each VAS instance Patch 5: Setup fault window per each VAS instance. This window is used for NX to paste fault CRB in FIFO. Patches 6 & 7: Setup threaded IRQ per VAS and register NX with fault window ID and port number for each send window so that NX paste fault CRB in this window. Patch 8: Reference to pid and mm so that pid is not used until window closed. Needed for multi thread application where child can open a window and can be used by parent later. Patches 9 and 10: Process CRBs from fault FIFO and notify tasks by updating CSB or through signals. Patches 11 and 12: Return credits for send and fault windows after handling faults. Patch 14:Fix closing send window after all credits are returned. This issue happens only for user space requests. No page faults on kernel request buffer. Changelog: V2: - Use threaded IRQ instead of own kernel thread handler - Use pswid instead of user space CSB address to find valid CRB - Removed unused macros and other changes as suggested by Christoph Hellwig V3: - Rebased to 5.5-rc2 - Use struct pid * instead of pid_t for vas_window tgid - Code cleanup as suggested by Christoph Hellwig V4: - Define xive alloc and get IRQ info based on chip ID and use these functions for IRQ setup per VAS instance. It eliminates skiboot dependency as suggested by Oliver. V5: - Do not update CSB if the process is exiting (patch9) V6: - Add interrupt handler instead of default one and return IRQ_HANDLED if the fault handling thread is already in progress. (Patch6) - Use platform send window ID and CCW[0] bit to find valid CRB in fault FIFO (Patch6). - Return fault address to user space in BE and other changes as suggested by Michael Neuling. (patch9) - Rebased to 5.6-rc4 V7: - Fix sparse warnings (patches 6,9 and 10) Haren Myneni (14): powerpc/xive: Define xive_native_alloc_irq_on_chip() powerpc/xive: Define xive_native_alloc_get_irq_info() powerpc/vas: Define nx_fault_stamp in coprocessor_request_block powerpc/vas: Alloc and setup IRQ and trigger port address powerpc/vas: Setup fault window per VAS instance powerpc/vas: Setup thread IRQ handler per VAS instance powerpc/vas: Register NX with fault window ID and IRQ port value powerpc/vas: Take reference to PID and mm for user space windows powerpc/vas: Update CSB and notify process for fault CRBs powerpc/vas: Print CRB and FIFO values powerpc/vas: Do not use default credits for receive window powerpc/vas: Return credits after handling fault powerpc/vas: Display process stuck message powerpc/vas: Free send window in VAS instance after credits returned arch/powerpc/include/asm/icswx.h| 18 +- arch/powerpc/include/asm/xive.h | 11 +- arch/powerpc/platforms/powernv/Makefile | 2 +- arch/powerpc/platforms/powernv/ocxl.c | 20 +- arch/powerpc/platforms/powernv/vas-debug.c | 2 +- arch/powerpc/platforms/powernv/vas-fault.
[PATCH V6 14/14] powerpc/vas: Free send window in VAS instance after credits returned
NX may be processing requests while trying to close window. Wait until all credits are returned and then free send window from VAS instance. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/powernv/vas-window.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c index 40c303a..c60accd 100644 --- a/arch/powerpc/platforms/powernv/vas-window.c +++ b/arch/powerpc/platforms/powernv/vas-window.c @@ -1317,14 +1317,14 @@ int vas_win_close(struct vas_window *window) unmap_paste_region(window); - clear_vinst_win(window); - poll_window_busy_state(window); unpin_close_window(window); poll_window_credits(window); + clear_vinst_win(window); + poll_window_castout(window); /* if send window, drop reference to matching receive window */ -- 1.8.3.1
[PATCH V6 13/14] powerpc/vas: Display process stuck message
Process can not close send window until all requests are processed.
Means wait until window state is not busy and send credits are
returned. Display debug messages in case taking longer to close the
window.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 28
1 file changed, 28 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1439a6f..40c303a 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1182,6 +1182,7 @@ static void poll_window_credits(struct vas_window *window)
{
u64 val;
int creds, mode;
+ int count = 0;
val = read_hvwc_reg(window, VREG(WINCTL));
if (window->tx_win)
@@ -1200,10 +1201,27 @@ static void poll_window_credits(struct vas_window
*window)
creds = GET_FIELD(VAS_LRX_WCRED, val);
}
+ /*
+* Takes around few milliseconds to complete all pending requests
+* and return credits.
+* TODO: Scan fault FIFO and invalidate CRBs points to this window
+* and issue CRB Kill to stop all pending requests. Need only
+* if there is a bug in NX or fault handling in kernel.
+*/
if (creds < window->wcreds_max) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(10));
+ count++;
+ /*
+* Process can not close send window until all credits are
+* returned.
+*/
+ if (!(count % 1))
+ pr_debug("VAS: pid %d stuck. Waiting for credits
returned for Window(%d). creds %d, Retries %d\n",
+ vas_window_pid(window), window->winid,
+ creds, count);
+
goto retry;
}
}
@@ -1217,6 +1235,7 @@ static void poll_window_busy_state(struct vas_window
*window)
{
int busy;
u64 val;
+ int count = 0;
retry:
val = read_hvwc_reg(window, VREG(WIN_STATUS));
@@ -1225,6 +1244,15 @@ static void poll_window_busy_state(struct vas_window
*window)
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(5));
+ count++;
+ /*
+* Takes around few milliseconds to process all pending
+* requests.
+*/
+ if (!(count % 1))
+ pr_debug("VAS: pid %d stuck. Window (ID=%d) is in busy
state. Retries %d\n",
+ vas_window_pid(window), window->winid, count);
+
goto retry;
}
}
--
1.8.3.1
[PATCH V6 12/14] powerpc/vas: Return credits after handling fault
NX expects OS to return credit for send window after processing each
fault. Also credit has to be returned even for fault window.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 9 +
arch/powerpc/platforms/powernv/vas-window.c | 17 +
arch/powerpc/platforms/powernv/vas.h| 1 +
3 files changed, 27 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 3ab0831..614a27c 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -237,6 +237,10 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
memcpy(crb, fifo, CRB_SIZE);
entry->stamp.nx.pswid = FIFO_INVALID_ENTRY;
entry->ccw |= CCW0_INVALID;
+ /*
+* Return credit for the fault window.
+*/
+ vas_return_credit(vinst->fault_win, 0);
mutex_unlock(>mutex);
pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
@@ -266,6 +270,11 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
}
update_csb(window, crb);
+ /*
+* Return credit for send window after processing
+* fault CRB.
+*/
+ vas_return_credit(window, 1);
} while (true);
}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 427a884..1439a6f 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1318,6 +1318,23 @@ int vas_win_close(struct vas_window *window)
}
EXPORT_SYMBOL_GPL(vas_win_close);
+/*
+ * Return credit for the given window.
+ */
+void vas_return_credit(struct vas_window *window, bool tx)
+{
+ uint64_t val;
+
+ val = 0ULL;
+ if (tx) { /* send window */
+ val = SET_FIELD(VAS_TX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), val);
+ } else {
+ val = SET_FIELD(VAS_LRX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), val);
+ }
+}
+
struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid)
{
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 9ba4605..f0efb98 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -428,6 +428,7 @@ struct vas_winctx {
extern void vas_window_free_dbgdir(struct vas_window *win);
extern int vas_setup_fault_window(struct vas_instance *vinst);
extern irqreturn_t vas_fault_thread_fn(int irq, void *data);
+extern void vas_return_credit(struct vas_window *window, bool tx);
extern struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid);
--
1.8.3.1
[PATCH V6 09/14] powerpc/vas: Update CSB and notify process for fault CRBs
For each fault CRB, update fault address in CRB (fault_storage_addr)
and translation error status in CSB so that user space can touch the
fault address and resend the request. If the user space passed invalid
CSB address send signal to process with SIGSEGV.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 114 +
1 file changed, 114 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 85e4280..c79fdbd 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,6 +27,118 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Update the CSB to indicate a translation error.
+ *
+ * If we are unable to update the CSB means copy_to_user failed due to
+ * invalid csb_addr, send a signal to the process.
+ *
+ * Remaining settings in the CSB are based on wait_for_csb() of
+ * NX-GZIP.
+ */
+static void update_csb(struct vas_window *window,
+ struct coprocessor_request_block *crb)
+{
+ int rc;
+ struct pid *pid;
+ void __user *csb_addr;
+ struct task_struct *tsk;
+ struct kernel_siginfo info;
+ struct coprocessor_status_block csb;
+
+ /*
+* NX user space windows can not be opened for task->mm=NULL
+* and faults will not be generated for kernel requests.
+*/
+ if (!window->mm || !window->user_win)
+ return;
+
+ csb_addr = (void *)be64_to_cpu(crb->csb_addr);
+
+ csb.cc = CSB_CC_TRANSLATION;
+ csb.ce = CSB_CE_TERMINATION;
+ csb.cs = 0;
+ csb.count = 0;
+
+ /*
+* NX operates and returns in BE format as defined CRB struct.
+* So return fault_storage_addr in BE as NX pastes in FIFO and
+* expects user space to convert to CPU format.
+*/
+ csb.address = crb->stamp.nx.fault_storage_addr;
+ csb.flags = 0;
+
+ pid = window->pid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Send window will be closed after processing all NX requests
+* and process exits after closing all windows. In multi-thread
+* applications, thread may not exists, but does not close FD
+* (means send window) upon exit. Parent thread (tgid) can use
+* and close the window later.
+* pid and mm references are taken when window is opened by
+* process (pid). So tgid is used only when child thread opens
+* a window and exits without closing it in multithread tasks.
+*/
+ if (!tsk) {
+ pid = window->tgid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Parent thread will be closing window during its exit.
+* So should not get here.
+*/
+ if (!tsk)
+ return;
+ }
+
+ /* Return if the task is exiting. */
+ if (tsk->flags & PF_EXITING) {
+ put_task_struct(tsk);
+ return;
+ }
+
+ use_mm(window->mm);
+ rc = copy_to_user(csb_addr, , sizeof(csb));
+ /*
+* User space polls on csb.flags (first byte). So add barrier
+* then copy first byte with csb flags update.
+*/
+ smp_mb();
+ if (!rc) {
+ csb.flags = CSB_V;
+ rc = copy_to_user(csb_addr, , sizeof(u8));
+ }
+ unuse_mm(window->mm);
+ put_task_struct(tsk);
+
+ /* Success */
+ if (!rc)
+ return;
+
+ pr_debug("Invalid CSB address 0x%p signalling pid(%d)\n",
+ csb_addr, pid_vnr(pid));
+
+ clear_siginfo();
+ info.si_signo = SIGSEGV;
+ info.si_errno = EFAULT;
+ info.si_code = SEGV_MAPERR;
+ info.si_addr = csb_addr;
+
+ /*
+* process will be polling on csb.flags after request is sent to
+* NX. So generally CSB update should not fail except when an
+* application does not follow the process properly. So an error
+* message will be displayed and leave it to user space whether
+* to ignore or handle this signal.
+*/
+ rcu_read_lock();
+ rc = kill_pid_info(SIGSEGV, , pid);
+ rcu_read_unlock();
+
+ pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
+ pid_vnr(pid), rc);
+}
+
+/*
* Process valid CRBs in fault FIFO.
*/
irqreturn_t vas_fault_thread_fn(int irq, void *data)
@@ -111,6 +224,7 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
return IRQ_HANDLED;
}
+ update_csb(window, crb);
} while (true);
}
--
1.8.3.1
[PATCH V6 11/14] powerpc/vas: Do not use default credits for receive window
System checkstops if RxFIFO overruns with more requests than the
maximum possible number of CRBs allowed in FIFO at any time. So
max credits value (rxattr.wcreds_max) is set and is passed to
vas_rx_win_open() by the the driver.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 4 ++--
arch/powerpc/platforms/powernv/vas.h| 2 --
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 7587258..427a884 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -772,7 +772,7 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
- if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ if (!attr->wcreds_max)
return false;
if (attr->nx_win) {
@@ -877,7 +877,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
- rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ rxwin->wcreds_max = rxattr->wcreds_max;
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 7c8e0182..9ba4605 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -101,11 +101,9 @@
/*
* Initial per-process credits.
* Max send window credits:4K-1 (12-bits in VAS_TX_WCRED)
- * Max receive window credits: 64K-1 (16 bits in VAS_LRX_WCRED)
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
--
1.8.3.1
[PATCH V6 10/14] powerpc/vas: Print CRB and FIFO values
Dump FIFO entries if could not find send window and print CRB
for debugging.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 41 ++
1 file changed, 41 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index c79fdbd..3ab0831 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -26,6 +26,28 @@
*/
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+static void dump_crb(struct coprocessor_request_block *crb)
+{
+ struct data_descriptor_entry *dde;
+ struct nx_fault_stamp *nx;
+
+ dde = >source;
+ pr_devel("SrcDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ dde = >target;
+ pr_devel("TgtDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ nx = >stamp.nx;
+ pr_devel("NX Stamp: PSWID 0x%x, FSA 0x%llx, flags 0x%x, FS 0x%x\n",
+ be32_to_cpu(nx->pswid),
+ be64_to_cpu(crb->stamp.nx.fault_storage_addr),
+ nx->flags, be32_to_cpu(nx->fault_status));
+}
+
/*
* Update the CSB to indicate a translation error.
*
@@ -138,6 +160,23 @@ static void update_csb(struct vas_window *window,
pid_vnr(pid), rc);
}
+static void dump_fifo(struct vas_instance *vinst, void *entry)
+{
+ int i;
+ unsigned long *fifo = entry;
+ unsigned long *end = vinst->fault_fifo + vinst->fault_fifo_size;
+
+ pr_err("Fault fifo size %d, Max crbs %d\n", vinst->fault_fifo_size,
+ vinst->fault_fifo_size / CRB_SIZE);
+
+ /* Dump 10 CRB entries or until end of FIFO */
+ pr_err("Fault FIFO Dump:\n");
+ for (i = 0; i < 10*(CRB_SIZE/8) && fifo < end; i += 4, fifo += 4) {
+ pr_err("[%.3d, %p]: 0x%.16lx 0x%.16lx 0x%.16lx 0x%.16lx\n",
+ i, fifo, *fifo, *(fifo+1), *(fifo+2), *(fifo+3));
+ }
+}
+
/*
* Process valid CRBs in fault FIFO.
*/
@@ -204,6 +243,7 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
+ dump_crb(crb);
window = vas_pswid_to_window(vinst,
be32_to_cpu(crb->stamp.nx.pswid));
@@ -214,6 +254,7 @@ irqreturn_t vas_fault_thread_fn(int irq, void *data)
* even clean it up (return credit).
* But we should not get here.
*/
+ dump_fifo(vinst, (void *)entry);
pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
vinst->vas_id, vinst->fault_fifo, fifo,
be32_to_cpu(crb->stamp.nx.pswid),
--
1.8.3.1
[PATCH V6 08/14] powerpc/vas: Take reference to PID and mm for user space windows
Process close windows after its requests are completed. In multi-thread
applications, child can open a window but release FD will not be called
upon its exit. Parent thread will be closing it later upon its exit.
The parent can also send NX requests with this window and NX can
generate page faults. After kernel handles the page fault, send
signal to process by using PID if CSB address is invalid. Parent
thread will not receive signal since its PID is different from the one
saved in vas_window. So use tgid in case if the task for the pid saved
in window is not running and send signal to its parent.
To prevent reusing the pid until the window closed, take reference to
pid and task mm.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-debug.c | 2 +-
arch/powerpc/platforms/powernv/vas-window.c | 53 ++---
arch/powerpc/platforms/powernv/vas.h| 9 -
3 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index 09e63df..ef9a717 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -38,7 +38,7 @@ static int info_show(struct seq_file *s, void *private)
seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
window->tx_win ? "Send" : "Receive");
- seq_printf(s, "Pid : %d\n", window->pid);
+ seq_printf(s, "Pid : %d\n", vas_window_pid(window));
unlock:
mutex_unlock(_mutex);
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a45d81d..7587258 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -12,6 +12,8 @@
#include
#include
#include
+#include
+#include
#include
#include
#include "vas.h"
@@ -876,8 +878,6 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
- if (rxattr->user_win)
- rxwin->pid = task_pid_vnr(current);
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
@@ -1027,7 +1027,6 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->tx_win = 1;
txwin->rxwin = rxwin;
txwin->nx_win = txwin->rxwin->nx_win;
- txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
@@ -1068,8 +1067,43 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
goto free_window;
}
- set_vinst_win(vinst, txwin);
+ if (txwin->user_win) {
+ /*
+* Window opened by child thread may not be closed when
+* it exits. So take reference to its pid and release it
+* when the window is free by parent thread.
+* Acquire a reference to the task's pid to make sure
+* pid will not be re-used - needed only for multithread
+* applications.
+*/
+ txwin->pid = get_task_pid(current, PIDTYPE_PID);
+ /*
+* Acquire a reference to the task's mm.
+*/
+ txwin->mm = get_task_mm(current);
+ if (!txwin->mm) {
+ put_pid(txwin->pid);
+ pr_err("VAS: pid(%d): mm_struct is not found\n",
+ current->pid);
+ rc = -EPERM;
+ goto free_window;
+ }
+
+ mmgrab(txwin->mm);
+ mmput(txwin->mm);
+ mm_context_add_copro(txwin->mm);
+ /*
+* Process closes window during exit. In the case of
+* multithread application, child can open window and
+* can exit without closing it. Expects parent thread
+* to use and close the window. So do not need to take
+* pid reference for parent thread.
+*/
+ txwin->tgid = find_get_pid(task_tgid_vnr(current));
+ }
+
+ set_vinst_win(vinst, txwin);
return txwin;
free_window:
@@ -1266,8 +1300,17 @@ int vas_win_close(struct vas_window *window)
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
- if (window->tx_win)
+ if (window->tx_win) {
+ if (window->user_win) {
+ /* Drop references to pid and mm */
+ put_pid(wi
[PATCH V6 07/14] powerpc/vas: Register NX with fault window ID and IRQ port value
For each user space send window, register NX with fault window ID
and port value so that NX paste CRBs in this fault FIFO when it
sees fault on the request buffer.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 15 +--
arch/powerpc/platforms/powernv/vas.h| 15 +++
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 7c6f55f..a45d81d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -373,7 +373,7 @@ int init_winctx_regs(struct vas_window *window, struct
vas_winctx *winctx)
init_xlate_regs(window, winctx->user_win);
val = 0ULL;
- val = SET_FIELD(VAS_FAULT_TX_WIN, val, 0);
+ val = SET_FIELD(VAS_FAULT_TX_WIN, val, winctx->fault_win_id);
write_hvwc_reg(window, VREG(FAULT_TX_WIN), val);
/* In PowerNV, interrupts go to HV. */
@@ -748,6 +748,8 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (rxwin->vinst->virq)
+ winctx->irq_port = rxwin->vinst->irq_port;
}
static bool rx_win_args_valid(enum vas_cop_type cop,
@@ -944,13 +946,22 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
winctx->rx_win_id = txwin->rxwin->winid;
+ /*
+* IRQ and fault window setup is successful. Set fault window
+* for the send window so that ready to handle faults.
+*/
+ if (txwin->vinst->virq)
+ winctx->fault_win_id = txwin->vinst->fault_win->winid;
winctx->dma_type = VAS_DMA_TYPE_INJECT;
winctx->tc_mode = txattr->tc_mode;
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (txwin->vinst->virq)
+ winctx->irq_port = txwin->vinst->irq_port;
- winctx->pswid = 0;
+ winctx->pswid = txattr->pswid ? txattr->pswid :
+ encode_pswid(txwin->vinst->vas_id, txwin->winid);
}
static bool tx_win_args_valid(enum vas_cop_type cop,
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index b82d5da..ab41cd3 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -468,6 +468,21 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ return ((u32)winid | (vasid << (31 - 7)));
+}
+
static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
{
if (vasid)
--
1.8.3.1
[PATCH V6 06/14] powerpc/vas: Setup thread IRQ handler per VAS instance
Setup thread IRQ handler per each VAS instance. When NX sees a fault
on CRB, kernel gets an interrupt and vas_fault_handler will be
executed to process fault CRBs. Read all valid CRBs from fault FIFO,
determine the corresponding send window from CRB and process fault
requests.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 90 +
arch/powerpc/platforms/powernv/vas-window.c | 60 +++
arch/powerpc/platforms/powernv/vas.c| 49 +++-
arch/powerpc/platforms/powernv/vas.h| 6 ++
4 files changed, 204 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 4044998..85e4280 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include "vas.h"
@@ -25,6 +26,95 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Process valid CRBs in fault FIFO.
+ */
+irqreturn_t vas_fault_thread_fn(int irq, void *data)
+{
+ struct vas_instance *vinst = data;
+ struct coprocessor_request_block *crb, *entry;
+ struct coprocessor_request_block buf;
+ struct vas_window *window;
+ unsigned long flags;
+ void *fifo;
+
+ crb =
+
+ /*
+* VAS can interrupt with multiple page faults. So process all
+* valid CRBs within fault FIFO until reaches invalid CRB.
+* NX updates nx_fault_stamp in CRB and pastes in fault FIFO.
+* kernel retrives send window from parition send window ID
+* (pswid) in nx_fault_stamp. So pswid should be valid and
+* ccw[0] (in be) should be zero since this bit is reserved.
+* If user space touches this bit, NX returns with "CRB format
+* error".
+*
+* After reading CRB entry, invalidate it with pswid (set
+* 0x) and ccw[0] (set to 1).
+*
+* In case kernel receives another interrupt with different page
+* fault, CRBs are already processed by the previous handling. So
+* will be returned from this function when it sees invalid CRB.
+*/
+ do {
+ mutex_lock(>mutex);
+
+ spin_lock_irqsave(>fault_lock, flags);
+ /*
+* Advance the fault fifo pointer to next CRB.
+* Use CRB_SIZE rather than sizeof(*crb) since the latter is
+* aligned to CRB_ALIGN (256) but the CRB written to by VAS is
+* only CRB_SIZE in len.
+*/
+ fifo = vinst->fault_fifo + (vinst->fault_crbs * CRB_SIZE);
+ entry = fifo;
+
+ if ((entry->stamp.nx.pswid == FIFO_INVALID_ENTRY) ||
+ (entry->ccw & CCW0_INVALID)) {
+ atomic_set(>faults_in_progress, 0);
+ spin_unlock_irqrestore(>fault_lock, flags);
+ mutex_unlock(>mutex);
+ return IRQ_HANDLED;
+ }
+
+ spin_unlock_irqrestore(>fault_lock, flags);
+ vinst->fault_crbs++;
+ if (vinst->fault_crbs == (vinst->fault_fifo_size / CRB_SIZE))
+ vinst->fault_crbs = 0;
+
+ memcpy(crb, fifo, CRB_SIZE);
+ entry->stamp.nx.pswid = FIFO_INVALID_ENTRY;
+ entry->ccw |= CCW0_INVALID;
+ mutex_unlock(>mutex);
+
+ pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ vinst->fault_crbs);
+
+ window = vas_pswid_to_window(vinst,
+ be32_to_cpu(crb->stamp.nx.pswid));
+
+ if (IS_ERR(window)) {
+ /*
+* We got an interrupt about a specific send
+* window but we can't find that window and we can't
+* even clean it up (return credit).
+* But we should not get here.
+*/
+ pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ be32_to_cpu(crb->stamp.nx.pswid),
+ vinst->fault_crbs);
+
+ WARN_ON_ONCE(1);
+ atomic_set(>faults_in_progress, 0);
+ return IRQ_HANDLED;
+ }
+
+ } while (true);
+}
+
+/*
* Fault window is opened per VAS instance. NX pastes fault CRB in fault
* FIFO upon page faults.
*/
diff
[PATCH V6 05/14] powerpc/vas: Setup fault window per VAS instance
Setup fault window for each VAS instance. When NX gets a fault on
request buffer, write fault CRBs in the corresponding fault FIFO and
then sends an interrupt to the OS.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-fault.c | 77 +
arch/powerpc/platforms/powernv/vas-window.c | 4 +-
arch/powerpc/platforms/powernv/vas.c| 20
arch/powerpc/platforms/powernv/vas.h| 16 ++
5 files changed, 116 insertions(+), 3 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index c0f8120..395789f 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
new file mode 100644
index 000..4044998
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -0,0 +1,77 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * VAS Fault handling.
+ * Copyright 2019, IBM Corporation
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "vas.h"
+
+/*
+ * The maximum FIFO size for fault window can be 8MB
+ * (VAS_RX_FIFO_SIZE_MAX). Using 4MB FIFO since each VAS
+ * instance will be having fault window.
+ * 8MB FIFO can be used if expects more faults for each VAS
+ * instance.
+ */
+#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+
+/*
+ * Fault window is opened per VAS instance. NX pastes fault CRB in fault
+ * FIFO upon page faults.
+ */
+int vas_setup_fault_window(struct vas_instance *vinst)
+{
+ struct vas_rx_win_attr attr;
+
+ vinst->fault_fifo_size = VAS_FAULT_WIN_FIFO_SIZE;
+ vinst->fault_fifo = kzalloc(vinst->fault_fifo_size, GFP_KERNEL);
+ if (!vinst->fault_fifo) {
+ pr_err("Unable to alloc %d bytes for fault_fifo\n",
+ vinst->fault_fifo_size);
+ return -ENOMEM;
+ }
+
+ /*
+* Invalidate all CRB entries. NX pastes valid entry for each fault.
+*/
+ memset(vinst->fault_fifo, FIFO_INVALID_ENTRY, vinst->fault_fifo_size);
+ vas_init_rx_win_attr(, VAS_COP_TYPE_FAULT);
+
+ attr.rx_fifo_size = vinst->fault_fifo_size;
+ attr.rx_fifo = vinst->fault_fifo;
+
+ /*
+* Max creds is based on number of CRBs can fit in the FIFO.
+* (fault_fifo_size/CRB_SIZE). If 8MB FIFO is used, max creds
+* will be 0x since the receive creds field is 16bits wide.
+*/
+ attr.wcreds_max = vinst->fault_fifo_size / CRB_SIZE;
+ attr.lnotify_lpid = 0;
+ attr.lnotify_pid = mfspr(SPRN_PID);
+ attr.lnotify_tid = mfspr(SPRN_PID);
+
+ vinst->fault_win = vas_rx_win_open(vinst->vas_id, VAS_COP_TYPE_FAULT,
+ );
+
+ if (IS_ERR(vinst->fault_win)) {
+ pr_err("VAS: Error %ld opening FaultWin\n",
+ PTR_ERR(vinst->fault_win));
+ kfree(vinst->fault_fifo);
+ return PTR_ERR(vinst->fault_win);
+ }
+
+ pr_devel("VAS: Created FaultWin %d, LPID/PID/TID [%d/%d/%d]\n",
+ vinst->fault_win->winid, attr.lnotify_lpid,
+ attr.lnotify_pid, attr.lnotify_tid);
+
+ return 0;
+}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0c0d27d..1783fa9 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -827,9 +827,9 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
rxattr->fault_win = true;
rxattr->notify_disable = true;
rxattr->rx_wcred_mode = true;
- rxattr->tx_wcred_mode = true;
rxattr->rx_win_ord_mode = true;
- rxattr->tx_win_ord_mode = true;
+ rxattr->rej_no_credit = true;
+ rxattr->tc_mode = VAS_THRESH_DISABLED;
} else if (cop == VAS_COP_TYPE_FTW) {
rxattr->user_win = true;
rxattr->intr_disable = true;
diff --git a/arch/powerpc/platfor
[PATCH V6 04/14] powerpc/vas: Alloc and setup IRQ and trigger port address
Alloc IRQ and get trigger port address for each VAS instance. Kernel
register this IRQ per VAS instance and sets this port for each send
window. NX interrupts the kernel when it sees page fault.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas.c | 34 --
arch/powerpc/platforms/powernv/vas.h | 2 ++
2 files changed, 30 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index ed9cc6d..168ab68 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -15,6 +15,7 @@
#include
#include
#include
+#include
#include "vas.h"
@@ -25,10 +26,12 @@
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, cpu, vasid;
- struct resource *res;
- struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
+ struct vas_instance *vinst;
+ uint32_t chipid, irq;
+ struct resource *res;
+ int rc, cpu, vasid;
+ uint64_t port;
rc = of_property_read_u32(dn, "ibm,vas-id", );
if (rc) {
@@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev)
return -ENODEV;
}
+ rc = of_property_read_u32(dn, "ibm,chip-id", );
+ if (rc) {
+ pr_err("No ibm,chip-id property for %s?\n", pdev->name);
+ return -ENODEV;
+ }
+
if (pdev->num_resources != 4) {
pr_err("Unexpected DT configuration for [%s, %d]\n",
pdev->name, vasid);
@@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev)
vinst->paste_win_id_shift = 63 - res->end;
- pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, "
- "paste_win_id_shift 0x%llx\n", pdev->name, vasid,
- vinst->paste_base_addr, vinst->paste_win_id_shift);
+ rc = xive_native_alloc_get_irq_info(chipid, , );
+ if (rc)
+ return rc;
+
+ vinst->virq = irq_create_mapping(NULL, irq);
+ if (!vinst->virq) {
+ pr_err("Inst%d: Unable to map global irq %d\n",
+ vinst->vas_id, irq);
+ return -EINVAL;
+ }
+
+ vinst->irq_port = port;
+ pr_devel("Initialized instance [%s, %d] paste_base 0x%llx
paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n",
+ pdev->name, vasid, vinst->paste_base_addr,
+ vinst->paste_win_id_shift, vinst->virq,
+ vinst->irq_port);
for_each_possible_cpu(cpu) {
if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 5574aec..598608b 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -313,6 +313,8 @@ struct vas_instance {
u64 paste_base_addr;
u64 paste_win_id_shift;
+ u64 irq_port;
+ int virq;
struct mutex mutex;
struct vas_window *rxwin[VAS_COP_TYPE_MAX];
struct vas_window *windows[VAS_WINDOWS_PER_CHIP];
--
1.8.3.1
[PATCH V6 03/14] powerpc/vas: Define nx_fault_stamp in coprocessor_request_block
Kernel sets fault address and status in CRB for NX page fault on user
space address after processing page fault. User space gets the signal
and handles the fault mentioned in CRB by bringing the page in to
memory and send NX request again.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/icswx.h | 18 +-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/icswx.h b/arch/powerpc/include/asm/icswx.h
index 9872f85..b233d1e 100644
--- a/arch/powerpc/include/asm/icswx.h
+++ b/arch/powerpc/include/asm/icswx.h
@@ -108,6 +108,17 @@ struct data_descriptor_entry {
__be64 address;
} __packed __aligned(DDE_ALIGN);
+/* 4.3.2 NX-stamped Fault CRB */
+
+#define NX_STAMP_ALIGN (0x10)
+
+struct nx_fault_stamp {
+ __be64 fault_storage_addr;
+ __be16 reserved;
+ __u8 flags;
+ __u8 fault_status;
+ __be32 pswid;
+} __packed __aligned(NX_STAMP_ALIGN);
/* Chapter 6.5.2 Coprocessor-Request Block (CRB) */
@@ -135,7 +146,12 @@ struct coprocessor_request_block {
struct coprocessor_completion_block ccb;
- u8 reserved[48];
+ union {
+ struct nx_fault_stamp nx;
+ u8 reserved[16];
+ } stamp;
+
+ u8 reserved[32];
struct coprocessor_status_block csb;
} __packed __aligned(CRB_ALIGN);
--
1.8.3.1
[PATCH V6 02/14] powerpc/xive: Define xive_native_alloc_get_irq_info()
pnv_ocxl_alloc_xive_irq() in ocxl.c allocates IRQ and gets trigger port
address. VAS also needs this function, but based on chip ID. So moved
this common function to xive/native.c.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 2 ++
arch/powerpc/platforms/powernv/ocxl.c | 20 ++--
arch/powerpc/sysdev/xive/native.c | 23 +++
3 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index d08ea11..fd337da 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -139,6 +139,8 @@ int xive_native_set_queue_state(u32 vp_id, uint32_t prio,
u32 qtoggle,
int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
bool xive_native_has_queue_state_support(void);
extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+extern int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq,
+ u64 *trigger_addr);
static inline u32 xive_native_alloc_irq(void)
{
diff --git a/arch/powerpc/platforms/powernv/ocxl.c
b/arch/powerpc/platforms/powernv/ocxl.c
index 8c65aac..fb8f99a 100644
--- a/arch/powerpc/platforms/powernv/ocxl.c
+++ b/arch/powerpc/platforms/powernv/ocxl.c
@@ -487,24 +487,8 @@ int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data,
int pe_handle)
int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr)
{
- __be64 flags, trigger_page;
- s64 rc;
- u32 hwirq;
-
- hwirq = xive_native_alloc_irq();
- if (!hwirq)
- return -ENOENT;
-
- rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
- NULL);
- if (rc || !trigger_page) {
- xive_native_free_irq(hwirq);
- return -ENOENT;
- }
- *irq = hwirq;
- *trigger_addr = be64_to_cpu(trigger_page);
- return 0;
-
+ return xive_native_alloc_get_irq_info(OPAL_XIVE_ANY_CHIP, irq,
+ trigger_addr);
}
EXPORT_SYMBOL_GPL(pnv_ocxl_alloc_xive_irq);
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 14d4406..abdd892 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -295,6 +295,29 @@ u32 xive_native_alloc_irq_on_chip(u32 chip_id)
}
EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
+int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq, u64 *trigger_addr)
+{
+ __be64 flags, trigger_page;
+ u32 hwirq;
+ s64 rc;
+
+ hwirq = xive_native_alloc_irq_on_chip(chip_id);
+ if (!hwirq)
+ return -ENOENT;
+
+ rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
+ NULL);
+ if (rc || !trigger_page) {
+ xive_native_free_irq(hwirq);
+ return -ENOENT;
+ }
+ *irq = hwirq;
+ *trigger_addr = be64_to_cpu(trigger_page);
+
+ return 0;
+}
+EXPORT_SYMBOL(xive_native_alloc_get_irq_info);
+
void xive_native_free_irq(u32 irq)
{
for (;;) {
--
1.8.3.1
[PATCH V6 01/14] powerpc/xive: Define xive_native_alloc_irq_on_chip()
This function allocates IRQ on a specific chip. VAS needs per chip
IRQ allocation and will have IRQ handler per VAS instance.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 9 -
arch/powerpc/sysdev/xive/native.c | 6 +++---
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 93f982db..d08ea11 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -5,6 +5,8 @@
#ifndef _ASM_POWERPC_XIVE_H
#define _ASM_POWERPC_XIVE_H
+#include
+
#define XIVE_INVALID_VP0x
#ifdef CONFIG_PPC_XIVE
@@ -108,7 +110,6 @@ struct xive_q {
int xive_native_populate_irq_data(u32 hw_irq,
struct xive_irq_data *data);
void xive_cleanup_irq_data(struct xive_irq_data *xd);
-u32 xive_native_alloc_irq(void);
void xive_native_free_irq(u32 irq);
int xive_native_configure_irq(u32 hw_irq, u32 target, u8 prio, u32 sw_irq);
@@ -137,6 +138,12 @@ int xive_native_set_queue_state(u32 vp_id, uint32_t prio,
u32 qtoggle,
u32 qindex);
int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
bool xive_native_has_queue_state_support(void);
+extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+
+static inline u32 xive_native_alloc_irq(void)
+{
+ return xive_native_alloc_irq_on_chip(OPAL_XIVE_ANY_CHIP);
+}
#else
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 0ff6b73..14d4406 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -279,12 +279,12 @@ static int xive_native_get_ipi(unsigned int cpu, struct
xive_cpu *xc)
}
#endif /* CONFIG_SMP */
-u32 xive_native_alloc_irq(void)
+u32 xive_native_alloc_irq_on_chip(u32 chip_id)
{
s64 rc;
for (;;) {
- rc = opal_xive_allocate_irq(OPAL_XIVE_ANY_CHIP);
+ rc = opal_xive_allocate_irq(chip_id);
if (rc != OPAL_BUSY)
break;
msleep(OPAL_BUSY_DELAY_MS);
@@ -293,7 +293,7 @@ u32 xive_native_alloc_irq(void)
return 0;
return rc;
}
-EXPORT_SYMBOL_GPL(xive_native_alloc_irq);
+EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
void xive_native_free_irq(u32 irq)
{
--
1.8.3.1
[PATCH V6 00/14] powerpc/vas: Page fault handling for user space NX requests
On power9, Virtual Accelerator Switchboard (VAS) allows user space or kernel to communicate with Nest Accelerator (NX) directly using COPY/PASTE instructions. NX provides various functionalities such as compression, encryption and etc. But only compression (842 and GZIP formats) is supported in Linux kernel on power9. 842 compression driver (drivers/crypto/nx/nx-842-powernv.c) is already included in Linux. Only GZIP support will be available from user space. Applications can issue GZIP compression / decompression requests to NX with COPY/PASTE instructions. When NX is processing these requests, can hit fault on the request buffer (not in memory). It issues an interrupt and pastes fault CRB in fault FIFO. Expects kernel to handle this fault and return credits for both send and fault windows after processing. This patch series adds IRQ and fault window setup, and NX fault handling: - Alloc IRQ and trigger port address, and configure IRQ per VAS instance. - Set port# for each window to generate an interrupt when noticed fault. - Set fault window and FIFO on which NX paste fault CRB. - Setup IRQ thread fault handler per VAS instance. - When receiving an interrupt, Read CRBs from fault FIFO and update coprocessor_status_block (CSB) in the corresponding CRB with translation failure (CSB_CC_TRANSLATION). After issuing NX requests, process polls on CSB address. When it sees translation error, can touch the request buffer to bring the page in to memory and reissue NX request. - If copy_to_user fails on user space CSB address, OS sends SEGV signal. Tested these patches with NX-GZIP support and will be posting this series soon. Patches 1 & 2: Define alloc IRQ and get port address per chip which are needed to alloc IRQ per VAS instance. Patch 3: Define nx_fault_stamp on which NX writes fault status for the fault CRB Patch 4: Alloc and setup IRQ and trigger port address for each VAS instance Patch 5: Setup fault window per each VAS instance. This window is used for NX to paste fault CRB in FIFO. Patches 6 & 7: Setup threaded IRQ per VAS and register NX with fault window ID and port number for each send window so that NX paste fault CRB in this window. Patch 8: Reference to pid and mm so that pid is not used until window closed. Needed for multi thread application where child can open a window and can be used by parent later. Patches 9 and 10: Process CRBs from fault FIFO and notify tasks by updating CSB or through signals. Patches 11 and 12: Return credits for send and fault windows after handling faults. Patch 14:Fix closing send window after all credits are returned. This issue happens only for user space requests. No page faults on kernel request buffer. Changelog: V2: - Use threaded IRQ instead of own kernel thread handler - Use pswid instead of user space CSB address to find valid CRB - Removed unused macros and other changes as suggested by Christoph Hellwig V3: - Rebased to 5.5-rc2 - Use struct pid * instead of pid_t for vas_window tgid - Code cleanup as suggested by Christoph Hellwig V4: - Define xive alloc and get IRQ info based on chip ID and use these functions for IRQ setup per VAS instance. It eliminates skiboot dependency as suggested by Oliver. V5: - Do not update CSB if the process is exiting (patch9) V6: - Add interrupt handler instead of default one and return IRQ_HANDLED if the fault handling thread is already in progress. (Patch6) - Use platform send window ID and CCW[0] bit to find valid CRB in fault FIFO (Patch6). - Return fault address to user space in BE and other changes as suggested by Michael Neuling. (patch9) - Rebased to 5.6-rc4 Haren Myneni (14): powerpc/xive: Define xive_native_alloc_irq_on_chip() powerpc/xive: Define xive_native_alloc_get_irq_info() powerpc/vas: Define nx_fault_stamp in coprocessor_request_block powerpc/vas: Alloc and setup IRQ and trigger port address powerpc/vas: Setup fault window per VAS instance powerpc/vas: Setup thread IRQ handler per VAS instance powerpc/vas: Register NX with fault window ID and IRQ port value powerpc/vas: Take reference to PID and mm for user space windows powerpc/vas: Update CSB and notify process for fault CRBs powerpc/vas: Print CRB and FIFO values powerpc/vas: Do not use default credits for receive window powerpc/vas: Return credits after handling fault powerpc/vas: Display process stuck message powerpc/vas: Free send window in VAS instance after credits returned arch/powerpc/include/asm/icswx.h| 18 +- arch/powerpc/include/asm/xive.h | 11 +- arch/powerpc/platforms/powernv/Makefile | 2 +- arch/powerpc/platforms/powernv/ocxl.c | 20 +- arch/powerpc/platforms/powernv/vas-debug.c | 2 +- arch/powerpc/platforms/powernv/vas-fault.c | 331 arch/powerpc/platforms/pow
[PATCH V2 9/9] Documentation/powerpc: VAS API
Power9 introduced Virtual Accelerator Switchboard (VAS) which allows userspace to communicate with Nest Accelerator (NX) directly. But kernel has to establish channel to NX for userspace. This document describes user space API that application can use to establish communication channel. Signed-off-by: Sukadev Bhattiprolu Signed-off-by: Haren Myneni --- Documentation/powerpc/index.rst | 1 + Documentation/powerpc/vas-api.rst | 246 ++ 2 files changed, 247 insertions(+) create mode 100644 Documentation/powerpc/vas-api.rst diff --git a/Documentation/powerpc/index.rst b/Documentation/powerpc/index.rst index 0d45f0f..afe2d5e 100644 --- a/Documentation/powerpc/index.rst +++ b/Documentation/powerpc/index.rst @@ -30,6 +30,7 @@ powerpc syscall64-abi transactional_memory ultravisor +vas-api .. only:: subproject and html diff --git a/Documentation/powerpc/vas-api.rst b/Documentation/powerpc/vas-api.rst new file mode 100644 index 000..13ce4e7 --- /dev/null +++ b/Documentation/powerpc/vas-api.rst @@ -0,0 +1,246 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. _VAS-API: + +=== +Virtual Accelerator Switchboard (VAS) userspace API +=== + +Introduction + + +Power9 processor introduced Virtual Accelerator Switchboard (VAS) which +allows both userspace and kernel communicate to co-processor +(hardware accelerator) referred to as the Nest Accelerator (NX). The NX +unit comprises of one or more hardware engines or co-processor types +such as 842 compression, GZIP compression and encryption. On power9, +userspace applications will have access to only GZIP Compression engine +which supports ZLIB and GZIP compression algorithms in the hardware. + +To communicate with NX, kernel has to establish a channel or window and +then requests can be submitted directly without kernel involvement. +Requests to the GZIP engine must be formatted as a co-processor Request +Block (CRB) and these CRBs must be submitted to the NX using COPY/PASTE +instructions to paste the CRB to hardware address that is associated with +the engine's request queue. + +The GZIP engine provides two priority levels of requests: Normal and +High. Only Normal requests are supported from userspace right now. + +This document explains userspace API that is used to interact with +kernel to setup channel / window which can be used to send compression +requests directly to NX accelerator. + + +Overview + + +Application access to the GZIP engine is provided through +/dev/crypto/nx-gzip device node implemented by the VAS/NX device driver. +An application must open the /dev/crypto/nx-gzip device to obtain a file +descriptor (fd). Then should issue VAS_TX_WIN_OPEN ioctl with this fd to +establish connection to the engine. It means send window is opened on GZIP +engine for this process. Once a connection is established, the application +should use the mmap() system call to map the hardware address of engine's +request queue into the application's virtual address space. + +The application can then submit one or more requests to the the engine by +using copy/paste instructions and pasting the CRBs to the virtual address +(aka paste_address) returned by mmap(). User space can close the +established connection or send window by closing the file descriptior +(close(fd)) or upon the process exit. + +Note that applications can send several requests with the same window or +can establish multiple windows, but one window for each file descriptor. + +Following sections provide additional details and references about the +individual steps. + +NX-GZIP Device Node +=== + +There is one /dev/crypto/nx-gzip node in the system and it provides +access to all GZIP engines in the system. The only valid operations on +/dev/crypto/nx-gzip are: + + * open() the device for read and write. + * issue VAS_TX_WIN_OPEN ioctl + * mmap() the engine's request queue into application's virtual + address space (i.e. get a paste_address for the co-processor + engine). + * close the device node. + +Other file operations on this device node are undefined. + +Note that the copy and paste operations go directly to the hardware and +do not go through this device. Refer COPY/PASTE document for more +details. + +Although a system may have several instances of the NX co-processor +engines (typically, one per P9 chip) there is just one +/dev/crypto/nx-gzip device node in the system. When the nx-gzip device +node is opened, Kernel opens send window on a suitable instance of NX +accelerator. It finds CPU on which the user process is executing and +determine the NX instance for the corresponding chip on which this CPU +belongs. + +Applications may chose a specific instance of the NX co-processor using +the vas_id field in the VAS_TX_WIN_OPEN ioctl as detailed below. + +A userspace
[PATCH V2 8/9] crypto/nx: Remove 'pid' in vas_tx_win_attr struct
When window is opened, pid reference is taken for user space
windows. Not needed for kernel windows. So remove 'pid' in
vas_tx_win_attr struct.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/vas.h| 1 -
drivers/crypto/nx/nx-common-powernv.c | 1 -
2 files changed, 2 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index e064953..994db6f 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -86,7 +86,6 @@ struct vas_tx_win_attr {
int wcreds_max;
int lpid;
int pidr; /* hardware PID (from SPRN_PID) */
- int pid;/* linux process id */
int pswid;
int rsvd_txbuf_count;
int tc_mode;
diff --git a/drivers/crypto/nx/nx-common-powernv.c
b/drivers/crypto/nx/nx-common-powernv.c
index 6848aea1..c8b2e01 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -692,7 +692,6 @@ static struct vas_window *nx_alloc_txwin(struct nx_coproc
*coproc)
*/
vas_init_tx_win_attr(, coproc->ct);
txattr.lpid = 0;/* lpid is 0 for kernel requests */
- txattr.pid = 0; /* pid is 0 for kernel requests */
/*
* Open a VAS send window which is used to send request to NX.
--
1.8.3.1
[PATCH V2 7/9] crypto/nx: Enable and setup GZIP compression type
Changes to probe GZIP device-tree nodes, open RX windows and setup
GZIP compression type. No plans to provide GZIP usage in kernel right
now, but this patch enables GZIP for user space usage.
Signed-off-by: Haren Myneni
---
drivers/crypto/nx/nx-common-powernv.c | 43 ++-
1 file changed, 37 insertions(+), 6 deletions(-)
diff --git a/drivers/crypto/nx/nx-common-powernv.c
b/drivers/crypto/nx/nx-common-powernv.c
index 1cd4f40..6848aea1 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -65,6 +65,7 @@ struct nx_coproc {
* Using same values as in skiboot or coprocessor type representing
* in NX workbook.
*/
+#define NX_CT_GZIP (2) /* on P9 and later */
#define NX_CT_842 (3)
static int (*nx842_powernv_exec)(const unsigned char *in,
@@ -819,6 +820,9 @@ static int __init vas_cfg_coproc_info(struct device_node
*dn, int chip_id,
if (type == NX_CT_842)
ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_842_HIPRI,
VAS_COP_TYPE_842);
+ else if (type == NX_CT_GZIP)
+ ret = nx_set_ct(coproc, priority, VAS_COP_TYPE_GZIP_HIPRI,
+ VAS_COP_TYPE_GZIP);
if (ret)
goto err_out;
@@ -867,12 +871,16 @@ static int __init vas_cfg_coproc_info(struct device_node
*dn, int chip_id,
return ret;
}
-static int __init nx_coproc_init(int chip_id, int ct_842)
+static int __init nx_coproc_init(int chip_id, int ct_842, int ct_gzip)
{
int ret = 0;
if (opal_check_token(OPAL_NX_COPROC_INIT)) {
ret = opal_nx_coproc_init(chip_id, ct_842);
+
+ if (!ret)
+ ret = opal_nx_coproc_init(chip_id, ct_gzip);
+
if (ret) {
ret = opal_error_code(ret);
pr_err("Failed to initialize NX for chip(%d): %d\n",
@@ -902,8 +910,8 @@ static int __init find_nx_device_tree(struct device_node
*dn, int chip_id,
static int __init nx_powernv_probe_vas(struct device_node *pn)
{
int chip_id, vasid, ret = 0;
+ int ct_842 = 0, ct_gzip = 0;
struct device_node *dn;
- int ct_842 = 0;
chip_id = of_get_ibm_chip_id(pn);
if (chip_id < 0) {
@@ -920,19 +928,24 @@ static int __init nx_powernv_probe_vas(struct device_node
*pn)
for_each_child_of_node(pn, dn) {
ret = find_nx_device_tree(dn, chip_id, vasid, NX_CT_842,
"ibm,p9-nx-842", _842);
+
+ if (!ret)
+ ret = find_nx_device_tree(dn, chip_id, vasid,
+ NX_CT_GZIP, "ibm,p9-nx-gzip", _gzip);
+
if (ret)
return ret;
}
- if (!ct_842) {
- pr_err("NX842 FIFO nodes are missing\n");
+ if (!ct_842 || !ct_gzip) {
+ pr_err("NX FIFO nodes are missing\n");
return -EINVAL;
}
/*
* Initialize NX instance for both high and normal priority FIFOs.
*/
- ret = nx_coproc_init(chip_id, ct_842);
+ ret = nx_coproc_init(chip_id, ct_842, ct_gzip);
return ret;
}
@@ -1072,10 +1085,19 @@ static __init int nx_compress_powernv_init(void)
nx842_powernv_exec = nx842_exec_icswx;
} else {
/*
+* Register VAS user space API for NX GZIP so
+* that user space can use GZIP engine.
+* 842 compression is supported only in kernel.
+*/
+ ret = vas_register_coproc_api(THIS_MODULE);
+
+ /*
* GZIP is not supported in kernel right now.
* So open tx windows only for 842.
*/
- ret = nx_open_percpu_txwins();
+ if (!ret)
+ ret = nx_open_percpu_txwins();
+
if (ret) {
nx_delete_coprocs();
return ret;
@@ -1096,6 +1118,15 @@ static __init int nx_compress_powernv_init(void)
static void __exit nx_compress_powernv_exit(void)
{
+ /*
+* GZIP engine is supported only in power9 or later and nx842_ct
+* is used on power8 (icswx).
+* VAS API for NX GZIP is registered during init for user space
+* use. So delete this API use for GZIP engine.
+*/
+ if (!nx842_ct)
+ vas_unregister_coproc_api();
+
crypto_unregister_alg(_powernv_alg);
nx_delete_coprocs();
--
1.8.3.1
[PATCH V2 6/9] crypto/nx: Make enable code generic to add new GZIP compression type
Make setup and enable code generic to support new GZIP compression type.
Changed nx842 reference to nx and moved some code to new functions. The
actual functionality is not changed.
Signed-off-by: Haren Myneni
---
drivers/crypto/nx/nx-common-powernv.c | 159 +-
1 file changed, 100 insertions(+), 59 deletions(-)
diff --git a/drivers/crypto/nx/nx-common-powernv.c
b/drivers/crypto/nx/nx-common-powernv.c
index f42881f..1cd4f40 100644
--- a/drivers/crypto/nx/nx-common-powernv.c
+++ b/drivers/crypto/nx/nx-common-powernv.c
@@ -40,9 +40,9 @@ struct nx842_workmem {
char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
} __packed __aligned(WORKMEM_ALIGN);
-struct nx842_coproc {
+struct nx_coproc {
unsigned int chip_id;
- unsigned int ct;
+ unsigned int ct;/* Can be 842 or GZIP high/normal*/
unsigned int ci;/* Coprocessor instance, used with icswx */
struct {
struct vas_window *rxwin;
@@ -58,9 +58,15 @@ struct nx842_coproc {
static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
/* no cpu hotplug on powernv, so this list never changes after init */
-static LIST_HEAD(nx842_coprocs);
+static LIST_HEAD(nx_coprocs);
static unsigned int nx842_ct; /* used in icswx function */
+/*
+ * Using same values as in skiboot or coprocessor type representing
+ * in NX workbook.
+ */
+#define NX_CT_842 (3)
+
static int (*nx842_powernv_exec)(const unsigned char *in,
unsigned int inlen, unsigned char *out,
unsigned int *outlenp, void *workmem, int fc);
@@ -666,15 +672,15 @@ static int nx842_powernv_decompress(const unsigned char
*in, unsigned int inlen,
wmem, CCW_FC_842_DECOMP_CRC);
}
-static inline void nx842_add_coprocs_list(struct nx842_coproc *coproc,
+static inline void nx_add_coprocs_list(struct nx_coproc *coproc,
int chipid)
{
coproc->chip_id = chipid;
INIT_LIST_HEAD(>list);
- list_add(>list, _coprocs);
+ list_add(>list, _coprocs);
}
-static struct vas_window *nx842_alloc_txwin(struct nx842_coproc *coproc)
+static struct vas_window *nx_alloc_txwin(struct nx_coproc *coproc)
{
struct vas_window *txwin = NULL;
struct vas_tx_win_attr txattr;
@@ -704,9 +710,9 @@ static struct vas_window *nx842_alloc_txwin(struct
nx842_coproc *coproc)
* cpu_txwin is used in copy/paste operation for each compression /
* decompression request.
*/
-static int nx842_open_percpu_txwins(void)
+static int nx_open_percpu_txwins(void)
{
- struct nx842_coproc *coproc, *n;
+ struct nx_coproc *coproc, *n;
unsigned int i, chip_id;
for_each_possible_cpu(i) {
@@ -714,17 +720,18 @@ static int nx842_open_percpu_txwins(void)
chip_id = cpu_to_chip_id(i);
- list_for_each_entry_safe(coproc, n, _coprocs, list) {
+ list_for_each_entry_safe(coproc, n, _coprocs, list) {
/*
* Kernel requests use only high priority FIFOs. So
* open send windows for these FIFOs.
+* GZIP is not supported in kernel right now.
*/
if (coproc->ct != VAS_COP_TYPE_842_HIPRI)
continue;
if (coproc->chip_id == chip_id) {
- txwin = nx842_alloc_txwin(coproc);
+ txwin = nx_alloc_txwin(coproc);
if (IS_ERR(txwin))
return PTR_ERR(txwin);
@@ -743,13 +750,28 @@ static int nx842_open_percpu_txwins(void)
return 0;
}
+static int __init nx_set_ct(struct nx_coproc *coproc, const char *priority,
+ int high, int normal)
+{
+ if (!strcmp(priority, "High"))
+ coproc->ct = high;
+ else if (!strcmp(priority, "Normal"))
+ coproc->ct = normal;
+ else {
+ pr_err("Invalid RxFIFO priority value\n");
+ return -EINVAL;
+ }
+
+ return 0;
+}
+
static int __init vas_cfg_coproc_info(struct device_node *dn, int chip_id,
- int vasid, int *ct)
+ int vasid, int type, int *ct)
{
struct vas_window *rxwin = NULL;
struct vas_rx_win_attr rxattr;
- struct nx842_coproc *coproc;
u32 lpid, pid, tid, fifo_size;
+ struct nx_coproc *coproc;
u64 rx_fifo;
const char *priority;
int ret;
@@ -794,15 +816,12 @@ static int __init vas_cfg_coproc_info(struct device_node
*dn, int chip_id,
if (!coproc)
return -ENOMEM;
- if (!strcmp(priority, "High"))
-
[PATCH V2 5/9] crypto/nx: Rename nx-842-powernv file name to nx-common-powernv
Rename nx-842-powernv.c to nx-common-powernv.c to add code for setup
and enable new GZIP compression type. The actual functionality is not
changed in this patch.
Signed-off-by: Haren Myneni
---
drivers/crypto/nx/Makefile|2 +-
drivers/crypto/nx/nx-842-powernv.c| 1062 -
drivers/crypto/nx/nx-common-powernv.c | 1062 +
3 files changed, 1063 insertions(+), 1063 deletions(-)
delete mode 100644 drivers/crypto/nx/nx-842-powernv.c
create mode 100644 drivers/crypto/nx/nx-common-powernv.c
diff --git a/drivers/crypto/nx/Makefile b/drivers/crypto/nx/Makefile
index 015155d..bc89a20 100644
--- a/drivers/crypto/nx/Makefile
+++ b/drivers/crypto/nx/Makefile
@@ -15,4 +15,4 @@ obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_PSERIES) +=
nx-compress-pseries.o nx-compres
obj-$(CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV) += nx-compress-powernv.o
nx-compress.o
nx-compress-objs := nx-842.o
nx-compress-pseries-objs := nx-842-pseries.o
-nx-compress-powernv-objs := nx-842-powernv.o
+nx-compress-powernv-objs := nx-common-powernv.o
diff --git a/drivers/crypto/nx/nx-842-powernv.c
b/drivers/crypto/nx/nx-842-powernv.c
deleted file mode 100644
index 8e63326..000
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ /dev/null
@@ -1,1062 +0,0 @@
-// SPDX-License-Identifier: GPL-2.0-or-later
-/*
- * Driver for IBM PowerNV 842 compression accelerator
- *
- * Copyright (C) 2015 Dan Streetman, IBM Corp
- */
-
-#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
-
-#include "nx-842.h"
-
-#include
-
-#include
-#include
-#include
-#include
-#include
-#include
-
-MODULE_LICENSE("GPL");
-MODULE_AUTHOR("Dan Streetman ");
-MODULE_DESCRIPTION("842 H/W Compression driver for IBM PowerNV processors");
-MODULE_ALIAS_CRYPTO("842");
-MODULE_ALIAS_CRYPTO("842-nx");
-
-#define WORKMEM_ALIGN (CRB_ALIGN)
-#define CSB_WAIT_MAX (5000) /* ms */
-#define VAS_RETRIES(10)
-
-struct nx842_workmem {
- /* Below fields must be properly aligned */
- struct coprocessor_request_block crb; /* CRB_ALIGN align */
- struct data_descriptor_entry ddl_in[DDL_LEN_MAX]; /* DDE_ALIGN align */
- struct data_descriptor_entry ddl_out[DDL_LEN_MAX]; /* DDE_ALIGN align */
- /* Above fields must be properly aligned */
-
- ktime_t start;
-
- char padding[WORKMEM_ALIGN]; /* unused, to allow alignment */
-} __packed __aligned(WORKMEM_ALIGN);
-
-struct nx842_coproc {
- unsigned int chip_id;
- unsigned int ct;
- unsigned int ci;/* Coprocessor instance, used with icswx */
- struct {
- struct vas_window *rxwin;
- int id;
- } vas;
- struct list_head list;
-};
-
-/*
- * Send the request to NX engine on the chip for the corresponding CPU
- * where the process is executing. Use with VAS function.
- */
-static DEFINE_PER_CPU(struct vas_window *, cpu_txwin);
-
-/* no cpu hotplug on powernv, so this list never changes after init */
-static LIST_HEAD(nx842_coprocs);
-static unsigned int nx842_ct; /* used in icswx function */
-
-static int (*nx842_powernv_exec)(const unsigned char *in,
- unsigned int inlen, unsigned char *out,
- unsigned int *outlenp, void *workmem, int fc);
-
-/**
- * setup_indirect_dde - Setup an indirect DDE
- *
- * The DDE is setup with the the DDE count, byte count, and address of
- * first direct DDE in the list.
- */
-static void setup_indirect_dde(struct data_descriptor_entry *dde,
- struct data_descriptor_entry *ddl,
- unsigned int dde_count, unsigned int byte_count)
-{
- dde->flags = 0;
- dde->count = dde_count;
- dde->index = 0;
- dde->length = cpu_to_be32(byte_count);
- dde->address = cpu_to_be64(nx842_get_pa(ddl));
-}
-
-/**
- * setup_direct_dde - Setup single DDE from buffer
- *
- * The DDE is setup with the buffer and length. The buffer must be properly
- * aligned. The used length is returned.
- * Returns:
- * NSuccessfully set up DDE with N bytes
- */
-static unsigned int setup_direct_dde(struct data_descriptor_entry *dde,
-unsigned long pa, unsigned int len)
-{
- unsigned int l = min_t(unsigned int, len, LEN_ON_PAGE(pa));
-
- dde->flags = 0;
- dde->count = 0;
- dde->index = 0;
- dde->length = cpu_to_be32(l);
- dde->address = cpu_to_be64(pa);
-
- return l;
-}
-
-/**
- * setup_ddl - Setup DDL from buffer
- *
- * Returns:
- * 0 Successfully set up DDL
- */
-static int setup_ddl(struct data_descriptor_entry *dde,
-struct data_descriptor_entry *ddl,
-unsigned char *buf, unsigned int len,
-bool in)
-{
- unsigned long pa = nx842_get_pa(buf);
- int i, ret, total_len = len;
-
-
[PATCH V2 4/9] crypto/nx: Initialize coproc entry with kzalloc
coproc entry is initialized during NX probe on power9, but not on P8. nx842_delete_coprocs() is used for both and frees receive window if it is allocated. Getting crash for rmmod on P8 since coproc->vas.rxwin is not initialized. This patch replaces kmalloc with kzalloc in nx842_powernv_probe() Signed-off-by: Haren Myneni --- drivers/crypto/nx/nx-842-powernv.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/drivers/crypto/nx/nx-842-powernv.c b/drivers/crypto/nx/nx-842-powernv.c index c037a24..8e63326 100644 --- a/drivers/crypto/nx/nx-842-powernv.c +++ b/drivers/crypto/nx/nx-842-powernv.c @@ -922,7 +922,7 @@ static int __init nx842_powernv_probe(struct device_node *dn) return -EINVAL; } - coproc = kmalloc(sizeof(*coproc), GFP_KERNEL); + coproc = kzalloc(sizeof(*coproc), GFP_KERNEL); if (!coproc) return -ENOMEM; -- 1.8.3.1
[PATCH V3 3/9] powerpc/vas: Add VAS user space API
On power9, userspace can send GZIP compression requests directly to NX
once kernel establishes NX channel / window with VAS. This patch provides
user space API which allows user space to establish channel using open
VAS_TX_WIN_OPEN ioctl, mmap and close operations.
Each window corresponds to file descriptor and application can open
multiple windows. After the window is opened, VAS_TX_WIN_OPEN icoctl to
open a window on specific VAS instance, mmap() system call to map
the hardware address of engine's request queue into the application's
virtual address space.
Then the application can then submit one or more requests to the the
engine by using the copy/paste instructions and pasting the CRBs to
the virtual address (aka paste_address) returned by mmap().
Only NX GZIP coprocessor type is supported right now and allow GZIP
engine access via /dev/crypto/nx-gzip device node.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/vas.h | 11 ++
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-api.c| 290
arch/powerpc/platforms/powernv/vas-window.c | 6 +-
arch/powerpc/platforms/powernv/vas.h| 2 +
5 files changed, 307 insertions(+), 4 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-api.c
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index f93e6b0..e064953 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -163,4 +163,15 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
*/
int vas_paste_crb(struct vas_window *win, int offset, bool re);
+/*
+ * Register / unregister coprocessor type to VAS API which will be exported
+ * to user space. Applications can use this API to open / close window
+ * which can be used to send / receive requests directly to cooprcessor.
+ *
+ * Only NX GZIP coprocessor type is supported now, but this API can be
+ * used for others in future.
+ */
+int vas_register_coproc_api(struct module *mod);
+void vas_unregister_coproc_api(void);
+
#endif /* __ASM_POWERPC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index 395789f..fe3f0fb 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o vas-api.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-api.c
b/arch/powerpc/platforms/powernv/vas-api.c
new file mode 100644
index 000..d72755c
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-api.c
@@ -0,0 +1,290 @@
+// SPDX-License-Identifier: GPL-2.0-or-later
+/*
+ * VAS user space API for its accelerators (Only NX-GZIP is supported now)
+ * Copyright (C) 2019 Haren Myneni, IBM Corp
+ */
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include "vas.h"
+
+/*
+ * The driver creates the device node that can be used as follows:
+ * For NX-GZIP
+ *
+ * fd = open("/dev/crypto/nx-gzip", O_RDWR);
+ * rc = ioctl(fd, VAS_TX_WIN_OPEN, );
+ * paste_addr = mmap(NULL, PAGE_SIZE, prot, MAP_SHARED, fd, 0ULL).
+ * vas_copy(, 0, 1);
+ * vas_paste(paste_addr, 0, 1);
+ * close(fd) or exit process to close window.
+ *
+ * where "vas_copy" and "vas_paste" are defined in copy-paste.h.
+ * copy/paste returns to the user space directly. So refer NX hardware
+ * documententation for excat copy/paste usage and completion / error
+ * conditions.
+ */
+
+static char*coproc_dev_name = "nx-gzip";
+static atomic_tcoproc_instid = ATOMIC_INIT(0);
+
+/*
+ * Wrapper object for the nx-gzip device - there is just one instance of
+ * this node for the whole system.
+ */
+struct coproc_dev {
+ struct cdev cdev;
+ struct device *device;
+ char *name;
+ dev_t devt;
+ struct class *class;
+} coproc_device;
+
+/*
+ * One instance per open of a nx-gzip device. Each coproc_instance is
+ * associated with a VAS window after the caller issues
+ * VAS_GZIP_TX_WIN_OPEN ioctl.
+ */
+struct coproc_instance {
+ int id;
+ struct vas_window *txwin;
+};
+
+static char *coproc_devnode(struct device *dev, umode_t *mode)
+{
+ return kasprintf(GFP_KERNEL, "crypto/%s", dev_name(dev));
+}
+
+static int coproc_open(struct inode *inode, struct file *fp)
+{
+ struct coproc_instance *instance;
+
+ instance = kzalloc(sizeof(*instance), GFP_KE
[PATCH V2 2/9] powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API
Define the VAS_TX_WIN_OPEN ioctl interface for NX GZIP access
from user space. This interface is used to open GZIP send window and
mmap region which can be used by userspace to send requests to NX
directly with copy/paste instructions.
Signed-off-by: Haren Myneni
---
Documentation/userspace-api/ioctl/ioctl-number.rst | 1 +
arch/powerpc/include/uapi/asm/vas-api.h| 22 ++
2 files changed, 23 insertions(+)
create mode 100644 arch/powerpc/include/uapi/asm/vas-api.h
diff --git a/Documentation/userspace-api/ioctl/ioctl-number.rst
b/Documentation/userspace-api/ioctl/ioctl-number.rst
index 2e91370..deabc73 100644
--- a/Documentation/userspace-api/ioctl/ioctl-number.rst
+++ b/Documentation/userspace-api/ioctl/ioctl-number.rst
@@ -287,6 +287,7 @@ Code Seq#Include File
Comments
'v' 00-1F linux/fs.h conflict!
'v' 00-0F linux/sonypi.h conflict!
'v' 00-0F media/v4l2-subdev.h conflict!
+'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.hVAS API
'v' C0-FF linux/meye.hconflict!
'w' allCERN SCI
driver
'y' 00-1F packet
based user level communications
diff --git a/arch/powerpc/include/uapi/asm/vas-api.h
b/arch/powerpc/include/uapi/asm/vas-api.h
new file mode 100644
index 000..fe95d67
--- /dev/null
+++ b/arch/powerpc/include/uapi/asm/vas-api.h
@@ -0,0 +1,22 @@
+/* SPDX-License-Identifier: GPL-2.0+ WITH Linux-syscall-note */
+/*
+ * Copyright 2019 IBM Corp.
+ */
+
+#ifndef _UAPI_MISC_VAS_H
+#define _UAPI_MISC_VAS_H
+
+#include
+
+#define VAS_MAGIC 'v'
+#define VAS_TX_WIN_OPEN_IOW(VAS_MAGIC, 0x20, struct
vas_tx_win_open_attr)
+
+struct vas_tx_win_open_attr {
+ __u32 version;
+ __s16 vas_id; /* specific instance of vas or -1 for default */
+ __u16 reserved1;
+ __u64 flags; /* Future use */
+ __u64 reserved2[6];
+};
+
+#endif /* _UAPI_MISC_VAS_H */
--
1.8.3.1
[PATCH V2 1/9] powerpc/vas: Initialize window attributes for GZIP coprocessor type
Initialize send and receive window attributes for GZIP high and
normal priority types.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 17 -
1 file changed, 12 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 5f9c915..b6572af 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -817,7 +817,8 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
{
memset(rxattr, 0, sizeof(*rxattr));
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI ||
+ cop == VAS_COP_TYPE_GZIP || cop == VAS_COP_TYPE_GZIP_HIPRI) {
rxattr->pin_win = true;
rxattr->nx_win = true;
rxattr->fault_win = false;
@@ -892,7 +893,8 @@ void vas_init_tx_win_attr(struct vas_tx_win_attr *txattr,
enum vas_cop_type cop)
{
memset(txattr, 0, sizeof(*txattr));
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI ||
+ cop == VAS_COP_TYPE_GZIP || cop == VAS_COP_TYPE_GZIP_HIPRI) {
txattr->rej_no_credit = false;
txattr->rx_wcred_mode = true;
txattr->tx_wcred_mode = true;
@@ -976,9 +978,14 @@ static bool tx_win_args_valid(enum vas_cop_type cop,
if (attr->wcreds_max > VAS_TX_WCREDS_MAX)
return false;
- if (attr->user_win &&
- (cop != VAS_COP_TYPE_FTW || attr->rsvd_txbuf_count))
- return false;
+ if (attr->user_win) {
+ if (attr->rsvd_txbuf_count)
+ return false;
+
+ if (cop != VAS_COP_TYPE_FTW && cop != VAS_COP_TYPE_GZIP &&
+ cop != VAS_COP_TYPE_GZIP_HIPRI)
+ return false;
+ }
return true;
}
--
1.8.3.1
[PATCH V2 0/9] crypto/nx: Enable GZIP engine and provide userpace API
Power9 processor supports Virtual Accelerator Switchboard (VAS) which allows kernel and userspace to send compression requests to Nest Accelerator (NX) directly. The NX unit comprises of 2 842 compression engines and 1 GZIP engine. Linux kernel already has 842 compression support on kernel. This patch series adds GZIP compression support from user space. The GZIP Compression engine implements the ZLIB and GZIP compression algorithms. No plans of adding NX-GZIP compression support in kernel right now. Applications can send requests to NX directly with COPY/PASTE instructions. But kernel has to establish channel / window on NX-GZIP device for the userspace. So userspace access to the GZIP engine is provided through /dev/crypto/nx-gzip device with several operations. An application must open the this device to obtain a file descriptor (fd). Using the fd, application should issue the VAS_TX_WIN_OPEN ioctl to establish a connection to the engine. Once window is opened, should use mmap() system call to map the hardware address of engine's request queue into the application's virtual address space. Then user space forms the request as co-processor Request Block (CRB) and paste this CRB on the mapped HW address using COPY/PASTE instructions. Application can poll on status flags (part of CRB) with timeout for request completion. For VAS_TX_WIN_OPEN ioctl, if user space passes vas_id = -1 (struct vas_tx_win_open_attr), kernel determins the VAS instance on the corresponding chip based on the CPU on which the process is executing. Otherwise, the specified VAS instance is used if application passes the proper VAS instance (vas_id listed in /proc/device-tree/vas@*/ibm,vas_id). Process can open multiple windows with different FDs or can send several requests to NX on the same window at the same time. A userspace library libnxz is available: https://github.com/abalib/power-gzip Applications that use inflate/deflate calls can link with libNXz and use NX GZIP compression without any modification. Tested the available 842 compression on power8 and power9 system to make sure no regression and tested GZIP compression on power9 with tests available in the above link. Thanks to Bulent Abali for nxz library and tests development. Changelog: V2: - Move user space API code to powerpc as suggested. Also this API can be extended to any other coprocessor type that VAS can support in future. Example: Fast thread wakeup feature from VAS - Rebased to 5.6-rc3 Haren Myneni (9): powerpc/vas: Initialize window attributes for GZIP coprocessor type powerpc/vas: Define VAS_TX_WIN_OPEN ioctl API powerpc/vas: Add VAS user space API crypto/nx: Initialize coproc entry with kzalloc crypto/nx: Rename nx-842-powernv file name to nx-common-powernv crypto/NX: Make enable code generic to add new GZIP compression type crypto/nx: Enable and setup GZIP compresstion type crypto/nx: Remove 'pid' in vas_tx_win_attr struct Documentation/powerpc: VAS API Documentation/powerpc/index.rst|1 + Documentation/powerpc/vas-api.rst | 246 + Documentation/userspace-api/ioctl/ioctl-number.rst |1 + arch/powerpc/include/asm/vas.h | 12 +- arch/powerpc/include/uapi/asm/vas-api.h| 22 + arch/powerpc/platforms/powernv/Makefile|2 +- arch/powerpc/platforms/powernv/vas-api.c | 290 + arch/powerpc/platforms/powernv/vas-window.c| 23 +- arch/powerpc/platforms/powernv/vas.h |2 + drivers/crypto/nx/Makefile |2 +- drivers/crypto/nx/nx-842-powernv.c | 1062 -- drivers/crypto/nx/nx-common-powernv.c | 1133 12 files changed, 1723 insertions(+), 1073 deletions(-) create mode 100644 Documentation/powerpc/vas-api.rst create mode 100644 arch/powerpc/include/uapi/asm/vas-api.h create mode 100644 arch/powerpc/platforms/powernv/vas-api.c delete mode 100644 drivers/crypto/nx/nx-842-powernv.c create mode 100644 drivers/crypto/nx/nx-common-powernv.c -- 1.8.3.1
Re: [PATCH V5 06/14] powerpc/vas: Setup thread IRQ handler per VAS instance
On Fri, 2020-02-07 at 16:57 +1100, Michael Neuling wrote:
> > /*
> > + * Process CRBs that we receive on the fault window.
> > + */
> > +irqreturn_t vas_fault_handler(int irq, void *data)
> > +{
> > + struct vas_instance *vinst = data;
> > + struct coprocessor_request_block buf, *crb;
> > + struct vas_window *window;
> > + void *fifo;
> > +
> > + /*
> > +* VAS can interrupt with multiple page faults. So process all
> > +* valid CRBs within fault FIFO until reaches invalid CRB.
> > +* NX updates nx_fault_stamp in CRB and pastes in fault FIFO.
> > +* kernel retrives send window from parition send window ID
> > +* (pswid) in nx_fault_stamp. So pswid should be non-zero and
> > +* use this to check whether CRB is valid.
> > +* After reading CRB entry, it is reset with 0's in fault FIFO.
> > +*
> > +* In case kernel receives another interrupt with different page
> > +* fault and CRBs are processed by the previous handling, will be
> > +* returned from this function when it sees invalid CRB (means 0's).
> > +*/
> > + do {
> > + mutex_lock(>mutex);
>
> This isn't going to work.
>
> From Documentation/locking/mutex-design.rst
>
> - Mutexes may not be used in hardware or software interrupt
> contexts such as tasklets and timers.
Initially used kernel thread per VAS instance and later using IRQ
thread.
vas_fault_handler() is IRQ thread function, not IRQ handler. I thought
we can use mutex_lock() in thread function.
>
> Mikey
>
Re: [PATCH V5 09/14] powerpc/vas: Update CSB and notify process for fault CRBs
Mikey, Thanks for your review comments.
On Fri, 2020-02-07 at 16:46 +1100, Michael Neuling wrote:
> On Wed, 2020-01-22 at 00:17 -0800, Haren Myneni wrote:
> > For each fault CRB, update fault address in CRB (fault_storage_addr)
> > and translation error status in CSB so that user space can touch the
> > fault address and resend the request. If the user space passed invalid
> > CSB address send signal to process with SIGSEGV.
> >
> > Signed-off-by: Sukadev Bhattiprolu
> > Signed-off-by: Haren Myneni
> > ---
> > arch/powerpc/platforms/powernv/vas-fault.c | 116
> > +
> > 1 file changed, 116 insertions(+)
> >
> > diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
> > b/arch/powerpc/platforms/powernv/vas-fault.c
> > index 5c2cada..2cfab0c 100644
> > --- a/arch/powerpc/platforms/powernv/vas-fault.c
> > +++ b/arch/powerpc/platforms/powernv/vas-fault.c
> > @@ -11,6 +11,7 @@
> > #include
> > #include
> > #include
> > +#include
> > #include
> > #include
> >
> > @@ -26,6 +27,120 @@
> > #define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
> >
> > /*
> > + * Update the CSB to indicate a translation error.
> > + *
> > + * If the fault is in the CSB address itself or if we are unable to
> > + * update the CSB, send a signal to the process, because we have no
> > + * other way of notifying the user process.
> > + *
> > + * Remaining settings in the CSB are based on wait_for_csb() of
> > + * NX-GZIP.
> > + */
> > +static void update_csb(struct vas_window *window,
> > + struct coprocessor_request_block *crb)
> > +{
> > + int rc;
> > + struct pid *pid;
> > + void __user *csb_addr;
> > + struct task_struct *tsk;
> > + struct kernel_siginfo info;
> > + struct coprocessor_status_block csb;
> > +
> > + /*
> > +* NX user space windows can not be opened for task->mm=NULL
> > +* and faults will not be generated for kernel requests.
> > +*/
> > + if (!window->mm || !window->user_win)
> > + return;
> > +
> > + csb_addr = (void *)be64_to_cpu(crb->csb_addr);
> > +
> > + csb.cc = CSB_CC_TRANSLATION;
> > + csb.ce = CSB_CE_TERMINATION;
> > + csb.cs = 0;
> > + csb.count = 0;
> > +
> > + /*
> > +* Returns the fault address in CPU format since it is passed with
> > +* signal. But if the user space expects BE format, need changes.
> > +* i.e either kernel (here) or user should convert to CPU format.
> > +* Not both!
> > +*/
> > + csb.address = be64_to_cpu(crb->stamp.nx.fault_storage_addr);
>
> This looks wrong and I don't understand the comment. You need to convert this
> back to be64 to write it to csb.address. ie.
>
> csb.address = cpu_to_be64(be64_to_cpu(crb->stamp.nx.fault_storage_addr));
>
> Which I think you can just avoid the endian conversion all together.
NX pastes fault CRB in big-endian, so passing this address in CPU format
to user space, otherwise the library has to convert.
What is the standard way for passing to user space?
>
> > + csb.flags = 0;
> > +
> > + pid = window->pid;
> > + tsk = get_pid_task(pid, PIDTYPE_PID);
> > + /*
> > +* Send window will be closed after processing all NX requests
> > +* and process exits after closing all windows. In multi-thread
> > +* applications, thread may not exists, but does not close FD
> > +* (means send window) upon exit. Parent thread (tgid) can use
> > +* and close the window later.
> > +* pid and mm references are taken when window is opened by
> > +* process (pid). So tgid is used only when child thread opens
> > +* a window and exits without closing it in multithread tasks.
> > +*/
> > + if (!tsk) {
> > + pid = window->tgid;
> > + tsk = get_pid_task(pid, PIDTYPE_PID);
> > + /*
> > +* Parent thread will be closing window during its exit.
> > +* So should not get here.
> > +*/
> > + if (!tsk)
> > + return;
> > + }
> > +
> > + /* Return if the task is exiting. */
> > + if (tsk->flags & PF_EXITING) {
> > + put_task_struct(tsk);
> > + return;
> > + }
> > +
> > + use_mm(window->mm);
> > + rc = copy_to_user(csb_addr, , sizeof(csb));
> > + /*
> > +* User space polls on csb.flags (fir
[PATCH V5 14/14] powerpc/vas: Free send window in VAS instance after credits returned
NX may be processing requests while trying to close window. Wait until all credits are returned and then free send window from VAS instance. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/powernv/vas-window.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c index 88cecff..5f9c915 100644 --- a/arch/powerpc/platforms/powernv/vas-window.c +++ b/arch/powerpc/platforms/powernv/vas-window.c @@ -1316,14 +1316,14 @@ int vas_win_close(struct vas_window *window) unmap_paste_region(window); - clear_vinst_win(window); - poll_window_busy_state(window); unpin_close_window(window); poll_window_credits(window); + clear_vinst_win(window); + poll_window_castout(window); /* if send window, drop reference to matching receive window */ -- 1.8.3.1
[PATCH V5 13/14] powerpc/vas: Display process stuck message
Process can not close send window until all requests are processed.
Means wait until window state is not busy and send credits are
returned. Display debug message in case taking longer to close the
window.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 27 +++
1 file changed, 27 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1439a6f..88cecff 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1182,6 +1182,7 @@ static void poll_window_credits(struct vas_window *window)
{
u64 val;
int creds, mode;
+ int count = 0;
val = read_hvwc_reg(window, VREG(WINCTL));
if (window->tx_win)
@@ -1200,10 +1201,26 @@ static void poll_window_credits(struct vas_window
*window)
creds = GET_FIELD(VAS_LRX_WCRED, val);
}
+ /*
+* Takes around few microseconds to complete all pending requests
+* and return credits.
+* TODO: Scan fault FIFO and invalidate CRBs points to this window
+* and issue CRB Kill to stop all pending requests. Need only
+* if there is a bug in NX or fault handling in kernel.
+*/
if (creds < window->wcreds_max) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(10));
+ count++;
+ /*
+* Process can not close send window until all credits are
+* returned.
+*/
+ if (!(count % 1))
+ pr_debug("%s() pid %d stuck? retries %d\n", __func__,
+ vas_window_pid(window), count);
+
goto retry;
}
}
@@ -1217,6 +1234,7 @@ static void poll_window_busy_state(struct vas_window
*window)
{
int busy;
u64 val;
+ int count = 0;
retry:
val = read_hvwc_reg(window, VREG(WIN_STATUS));
@@ -1225,6 +1243,15 @@ static void poll_window_busy_state(struct vas_window
*window)
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(5));
+ count++;
+ /*
+* Takes around 5 microseconds to process all pending
+* requests.
+*/
+ if (!(count % 1))
+ pr_debug("%s() pid %d stuck? retries %d\n", __func__,
+ vas_window_pid(window), count);
+
goto retry;
}
}
--
1.8.3.1
[PATCH V5 12/14] powerpc/VAS: Return credits after handling fault
NX expects OS to return credit for send window after processing each
fault. Also credit has to be returned even for fault window.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 10 ++
arch/powerpc/platforms/powernv/vas-window.c | 17 +
arch/powerpc/platforms/powernv/vas.h| 1 +
3 files changed, 28 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 6431240..a993c5b 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -235,6 +235,11 @@ irqreturn_t vas_fault_handler(int irq, void *data)
memset(fifo, 0, CRB_SIZE);
mutex_unlock(>mutex);
+ /*
+* Return credit for the fault window.
+*/
+ vas_return_credit(vinst->fault_win, 0);
+
pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
@@ -261,6 +266,11 @@ irqreturn_t vas_fault_handler(int irq, void *data)
}
update_csb(window, crb);
+ /*
+* Return credit for send window after processing
+* fault CRB.
+*/
+ vas_return_credit(window, 1);
} while (true);
return IRQ_HANDLED;
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 427a884..1439a6f 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1318,6 +1318,23 @@ int vas_win_close(struct vas_window *window)
}
EXPORT_SYMBOL_GPL(vas_win_close);
+/*
+ * Return credit for the given window.
+ */
+void vas_return_credit(struct vas_window *window, bool tx)
+{
+ uint64_t val;
+
+ val = 0ULL;
+ if (tx) { /* send window */
+ val = SET_FIELD(VAS_TX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), val);
+ } else {
+ val = SET_FIELD(VAS_LRX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), val);
+ }
+}
+
struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid)
{
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index f5f45ea..495937a 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -415,6 +415,7 @@ struct vas_winctx {
extern void vas_window_free_dbgdir(struct vas_window *win);
extern int vas_setup_fault_window(struct vas_instance *vinst);
extern irqreturn_t vas_fault_handler(int irq, void *data);
+extern void vas_return_credit(struct vas_window *window, bool tx);
extern struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid);
--
1.8.3.1
[PATCH V5 11/14] powerpc/vas: Do not use default credits for receive window
System checkstops if RxFIFO overruns with more requests than the
maximum possible number of CRBs allowed in FIFO at any time. So
max credits value (rxattr.wcreds_max) is set and is passed to
vas_rx_win_open() by the the driver.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 4 ++--
arch/powerpc/platforms/powernv/vas.h| 2 --
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 7587258..427a884 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -772,7 +772,7 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
- if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ if (!attr->wcreds_max)
return false;
if (attr->nx_win) {
@@ -877,7 +877,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
- rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ rxwin->wcreds_max = rxattr->wcreds_max;
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index af03aa0..f5f45ea 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -101,11 +101,9 @@
/*
* Initial per-process credits.
* Max send window credits:4K-1 (12-bits in VAS_TX_WCRED)
- * Max receive window credits: 64K-1 (16 bits in VAS_LRX_WCRED)
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
--
1.8.3.1
[PATCH V5 10/14] powerpc/vas: Print CRB and FIFO values
Dump FIFO entry values if could not find send window and print CRB
for debugging.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 41 ++
1 file changed, 41 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 2cfab0c..6431240 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -26,6 +26,28 @@
*/
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+static void dump_crb(struct coprocessor_request_block *crb)
+{
+ struct data_descriptor_entry *dde;
+ struct nx_fault_stamp *nx;
+
+ dde = >source;
+ pr_devel("SrcDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ dde = >target;
+ pr_devel("TgtDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ nx = >stamp.nx;
+ pr_devel("NX Stamp: PSWID 0x%x, FSA 0x%llx, flags 0x%x, FS 0x%x\n",
+ be32_to_cpu(nx->pswid),
+ be64_to_cpu(crb->stamp.nx.fault_storage_addr),
+ nx->flags, be32_to_cpu(nx->fault_status));
+}
+
/*
* Update the CSB to indicate a translation error.
*
@@ -140,6 +162,23 @@ static void update_csb(struct vas_window *window,
pid_vnr(pid), rc);
}
+static void dump_fifo(struct vas_instance *vinst, void *entry)
+{
+ int i;
+ unsigned long *fifo = entry;
+
+ pr_err("Fault fifo size %d, max crbs %d, crb size %lu\n",
+ vinst->fault_fifo_size,
+ vinst->fault_fifo_size / CRB_SIZE,
+ sizeof(struct coprocessor_request_block));
+
+ pr_err("Fault FIFO Entry Dump:\n");
+ for (i = 0; i < CRB_SIZE; i += 4, fifo += 4) {
+ pr_err("[%.3d, %p]: 0x%.16lx 0x%.16lx 0x%.16lx 0x%.16lx\n",
+ i, fifo, *fifo, *(fifo+1), *(fifo+2), *(fifo+3));
+ }
+}
+
/*
* Process CRBs that we receive on the fault window.
*/
@@ -200,6 +239,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
+ dump_crb(crb);
window = vas_pswid_to_window(vinst,
be32_to_cpu(crb->stamp.nx.pswid));
@@ -210,6 +250,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
* even clean it up (return credit).
* But we should not get here.
*/
+ dump_fifo(vinst, (void *)crb);
pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
vinst->vas_id, vinst->fault_fifo, fifo,
be32_to_cpu(crb->stamp.nx.pswid),
--
1.8.3.1
[PATCH V5 09/14] powerpc/vas: Update CSB and notify process for fault CRBs
For each fault CRB, update fault address in CRB (fault_storage_addr)
and translation error status in CSB so that user space can touch the
fault address and resend the request. If the user space passed invalid
CSB address send signal to process with SIGSEGV.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 116 +
1 file changed, 116 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 5c2cada..2cfab0c 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,6 +27,120 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Update the CSB to indicate a translation error.
+ *
+ * If the fault is in the CSB address itself or if we are unable to
+ * update the CSB, send a signal to the process, because we have no
+ * other way of notifying the user process.
+ *
+ * Remaining settings in the CSB are based on wait_for_csb() of
+ * NX-GZIP.
+ */
+static void update_csb(struct vas_window *window,
+ struct coprocessor_request_block *crb)
+{
+ int rc;
+ struct pid *pid;
+ void __user *csb_addr;
+ struct task_struct *tsk;
+ struct kernel_siginfo info;
+ struct coprocessor_status_block csb;
+
+ /*
+* NX user space windows can not be opened for task->mm=NULL
+* and faults will not be generated for kernel requests.
+*/
+ if (!window->mm || !window->user_win)
+ return;
+
+ csb_addr = (void *)be64_to_cpu(crb->csb_addr);
+
+ csb.cc = CSB_CC_TRANSLATION;
+ csb.ce = CSB_CE_TERMINATION;
+ csb.cs = 0;
+ csb.count = 0;
+
+ /*
+* Returns the fault address in CPU format since it is passed with
+* signal. But if the user space expects BE format, need changes.
+* i.e either kernel (here) or user should convert to CPU format.
+* Not both!
+*/
+ csb.address = be64_to_cpu(crb->stamp.nx.fault_storage_addr);
+ csb.flags = 0;
+
+ pid = window->pid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Send window will be closed after processing all NX requests
+* and process exits after closing all windows. In multi-thread
+* applications, thread may not exists, but does not close FD
+* (means send window) upon exit. Parent thread (tgid) can use
+* and close the window later.
+* pid and mm references are taken when window is opened by
+* process (pid). So tgid is used only when child thread opens
+* a window and exits without closing it in multithread tasks.
+*/
+ if (!tsk) {
+ pid = window->tgid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Parent thread will be closing window during its exit.
+* So should not get here.
+*/
+ if (!tsk)
+ return;
+ }
+
+ /* Return if the task is exiting. */
+ if (tsk->flags & PF_EXITING) {
+ put_task_struct(tsk);
+ return;
+ }
+
+ use_mm(window->mm);
+ rc = copy_to_user(csb_addr, , sizeof(csb));
+ /*
+* User space polls on csb.flags (first byte). So add barrier
+* then copy first byte with csb flags update.
+*/
+ smp_mb();
+ if (!rc) {
+ csb.flags = CSB_V;
+ rc = copy_to_user(csb_addr, , sizeof(u8));
+ }
+ unuse_mm(window->mm);
+ put_task_struct(tsk);
+
+ /* Success */
+ if (!rc)
+ return;
+
+ pr_err("Invalid CSB address 0x%p signalling pid(%d)\n",
+ csb_addr, pid_vnr(pid));
+
+ clear_siginfo();
+ info.si_signo = SIGSEGV;
+ info.si_errno = EFAULT;
+ info.si_code = SEGV_MAPERR;
+ info.si_addr = csb_addr;
+
+ /*
+* process will be polling on csb.flags after request is sent to
+* NX. So generally CSB update should not fail except when an
+* application does not follow the process properly. So an error
+* message will be displayed and leave it to user space whether
+* to ignore or handle this signal.
+*/
+ rcu_read_lock();
+ rc = kill_pid_info(SIGSEGV, , pid);
+ rcu_read_unlock();
+
+ pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
+ pid_vnr(pid), rc);
+}
+
+/*
* Process CRBs that we receive on the fault window.
*/
irqreturn_t vas_fault_handler(int irq, void *data)
@@ -104,6 +219,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
return IRQ_HANDLED
[PATCH V5 08/14] powerpc/vas: Take reference to PID and mm for user space windows
Process close windows after its requests are completed. In multi-thread
applications, child can open a window but release FD will not be called
upon its exit. Parent thread will be closing it later upon its exit.
The parent can also send NX requests with this window and NX can
generate page faults. After kernel handles the page fault, send
signal to process by using PID if CSB address is invalid. Parent
thread will not receive signal since its PID is different from the one
saved in vas_window. So use tgid in case if the task for the pid saved
in window is not running and send signal to its parent.
To prevent reusing the pid until the window closed, take reference to
pid and task mm.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-debug.c | 2 +-
arch/powerpc/platforms/powernv/vas-window.c | 53 ++---
arch/powerpc/platforms/powernv/vas.h| 9 -
3 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index 09e63df..ef9a717 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -38,7 +38,7 @@ static int info_show(struct seq_file *s, void *private)
seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
window->tx_win ? "Send" : "Receive");
- seq_printf(s, "Pid : %d\n", window->pid);
+ seq_printf(s, "Pid : %d\n", vas_window_pid(window));
unlock:
mutex_unlock(_mutex);
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a45d81d..7587258 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -12,6 +12,8 @@
#include
#include
#include
+#include
+#include
#include
#include
#include "vas.h"
@@ -876,8 +878,6 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
- if (rxattr->user_win)
- rxwin->pid = task_pid_vnr(current);
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
@@ -1027,7 +1027,6 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->tx_win = 1;
txwin->rxwin = rxwin;
txwin->nx_win = txwin->rxwin->nx_win;
- txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
@@ -1068,8 +1067,43 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
goto free_window;
}
- set_vinst_win(vinst, txwin);
+ if (txwin->user_win) {
+ /*
+* Window opened by child thread may not be closed when
+* it exits. So take reference to its pid and release it
+* when the window is free by parent thread.
+* Acquire a reference to the task's pid to make sure
+* pid will not be re-used - needed only for multithread
+* applications.
+*/
+ txwin->pid = get_task_pid(current, PIDTYPE_PID);
+ /*
+* Acquire a reference to the task's mm.
+*/
+ txwin->mm = get_task_mm(current);
+ if (!txwin->mm) {
+ put_pid(txwin->pid);
+ pr_err("VAS: pid(%d): mm_struct is not found\n",
+ current->pid);
+ rc = -EPERM;
+ goto free_window;
+ }
+
+ mmgrab(txwin->mm);
+ mmput(txwin->mm);
+ mm_context_add_copro(txwin->mm);
+ /*
+* Process closes window during exit. In the case of
+* multithread application, child can open window and
+* can exit without closing it. Expects parent thread
+* to use and close the window. So do not need to take
+* pid reference for parent thread.
+*/
+ txwin->tgid = find_get_pid(task_tgid_vnr(current));
+ }
+
+ set_vinst_win(vinst, txwin);
return txwin;
free_window:
@@ -1266,8 +1300,17 @@ int vas_win_close(struct vas_window *window)
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
- if (window->tx_win)
+ if (window->tx_win) {
+ if (window->user_win) {
+ /* Drop references to pid and mm */
+ put_pid(wi
[PATCH V5 07/14] powerpc/vas: Register NX with fault window ID and IRQ port value
For each user space send window, register NX with fault window ID
and port value so that NX paste CRBs in this fault FIFO when it
sees fault on the request buffer.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 15 +--
arch/powerpc/platforms/powernv/vas.h| 15 +++
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 7c6f55f..a45d81d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -373,7 +373,7 @@ int init_winctx_regs(struct vas_window *window, struct
vas_winctx *winctx)
init_xlate_regs(window, winctx->user_win);
val = 0ULL;
- val = SET_FIELD(VAS_FAULT_TX_WIN, val, 0);
+ val = SET_FIELD(VAS_FAULT_TX_WIN, val, winctx->fault_win_id);
write_hvwc_reg(window, VREG(FAULT_TX_WIN), val);
/* In PowerNV, interrupts go to HV. */
@@ -748,6 +748,8 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (rxwin->vinst->virq)
+ winctx->irq_port = rxwin->vinst->irq_port;
}
static bool rx_win_args_valid(enum vas_cop_type cop,
@@ -944,13 +946,22 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
winctx->rx_win_id = txwin->rxwin->winid;
+ /*
+* IRQ and fault window setup is successful. Set fault window
+* for the send window so that ready to handle faults.
+*/
+ if (txwin->vinst->virq)
+ winctx->fault_win_id = txwin->vinst->fault_win->winid;
winctx->dma_type = VAS_DMA_TYPE_INJECT;
winctx->tc_mode = txattr->tc_mode;
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (txwin->vinst->virq)
+ winctx->irq_port = txwin->vinst->irq_port;
- winctx->pswid = 0;
+ winctx->pswid = txattr->pswid ? txattr->pswid :
+ encode_pswid(txwin->vinst->vas_id, txwin->winid);
}
static bool tx_win_args_valid(enum vas_cop_type cop,
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 879f5b4..2621df1 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -455,6 +455,21 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ return ((u32)winid | (vasid << (31 - 7)));
+}
+
static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
{
if (vasid)
--
1.8.3.1
[PATCH V5 06/14] powerpc/vas: Setup thread IRQ handler per VAS instance
Setup thread IRQ handler per each VAS instance. When NX sees a fault
on CRB, kernel gets an interrupt and vas_fault_handler will be
executed to process fault CRBs. Read all valid CRBs from fault FIFO,
determine the corresponding send window from CRB and process fault
requests.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 85 +
arch/powerpc/platforms/powernv/vas-window.c | 60
arch/powerpc/platforms/powernv/vas.c| 21 ++-
arch/powerpc/platforms/powernv/vas.h| 4 ++
4 files changed, 169 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index b0258ed..5c2cada 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include "vas.h"
@@ -25,6 +26,90 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Process CRBs that we receive on the fault window.
+ */
+irqreturn_t vas_fault_handler(int irq, void *data)
+{
+ struct vas_instance *vinst = data;
+ struct coprocessor_request_block buf, *crb;
+ struct vas_window *window;
+ void *fifo;
+
+ /*
+* VAS can interrupt with multiple page faults. So process all
+* valid CRBs within fault FIFO until reaches invalid CRB.
+* NX updates nx_fault_stamp in CRB and pastes in fault FIFO.
+* kernel retrives send window from parition send window ID
+* (pswid) in nx_fault_stamp. So pswid should be non-zero and
+* use this to check whether CRB is valid.
+* After reading CRB entry, it is reset with 0's in fault FIFO.
+*
+* In case kernel receives another interrupt with different page
+* fault and CRBs are processed by the previous handling, will be
+* returned from this function when it sees invalid CRB (means 0's).
+*/
+ do {
+ mutex_lock(>mutex);
+
+ /*
+* Advance the fault fifo pointer to next CRB.
+* Use CRB_SIZE rather than sizeof(*crb) since the latter is
+* aligned to CRB_ALIGN (256) but the CRB written to by VAS is
+* only CRB_SIZE in len.
+*/
+ fifo = vinst->fault_fifo + (vinst->fault_crbs * CRB_SIZE);
+ crb = fifo;
+
+ /*
+* NX pastes nx_fault_stamp in fault FIFO for each fault.
+* So use pswid to check whether fault CRB is valid.
+* pswid returned from NX will be in _be32, but just
+* checking non-zero value to make sure the CRB is valid.
+* Return if reached invalid CRB.
+*/
+ if (!crb->stamp.nx.pswid) {
+ mutex_unlock(>mutex);
+ return IRQ_HANDLED;
+ }
+
+ vinst->fault_crbs++;
+ if (vinst->fault_crbs == (vinst->fault_fifo_size / CRB_SIZE))
+ vinst->fault_crbs = 0;
+
+ crb =
+ memcpy(crb, fifo, CRB_SIZE);
+ memset(fifo, 0, CRB_SIZE);
+ mutex_unlock(>mutex);
+
+ pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ vinst->fault_crbs);
+
+ window = vas_pswid_to_window(vinst,
+ be32_to_cpu(crb->stamp.nx.pswid));
+
+ if (IS_ERR(window)) {
+ /*
+* We got an interrupt about a specific send
+* window but we can't find that window and we can't
+* even clean it up (return credit).
+* But we should not get here.
+*/
+ pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ be32_to_cpu(crb->stamp.nx.pswid),
+ vinst->fault_crbs);
+
+ WARN_ON_ONCE(1);
+ return IRQ_HANDLED;
+ }
+
+ } while (true);
+
+ return IRQ_HANDLED;
+}
+
+/*
* Fault window is opened per VAS instance. NX pastes fault CRB in fault
* FIFO upon page faults.
*/
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1783fa9..7c6f55f 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1040,6 +1040,15 @@ struct vas_w
[PATCH V5 05/14] powerpc/vas: Setup fault window per VAS instance
Setup fault window for each VAS instance. When NX gets fault on request
buffer, write fault CRBs in the corresponding fault FIFO and then sends
an interrupt to the OS.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-fault.c | 73 +
arch/powerpc/platforms/powernv/vas-window.c | 4 +-
arch/powerpc/platforms/powernv/vas.c| 20
arch/powerpc/platforms/powernv/vas.h| 5 ++
5 files changed, 101 insertions(+), 3 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index c0f8120..395789f 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
new file mode 100644
index 000..b0258ed
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * VAS Fault handling.
+ * Copyright 2019, IBM Corporation
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "vas.h"
+
+/*
+ * The maximum FIFO size for fault window can be 8MB
+ * (VAS_RX_FIFO_SIZE_MAX). Using 4MB FIFO since each VAS
+ * instance will be having fault window.
+ * 8MB FIFO can be used if expects more faults for each VAS
+ * instance.
+ */
+#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+
+/*
+ * Fault window is opened per VAS instance. NX pastes fault CRB in fault
+ * FIFO upon page faults.
+ */
+int vas_setup_fault_window(struct vas_instance *vinst)
+{
+ struct vas_rx_win_attr attr;
+
+ vinst->fault_fifo_size = VAS_FAULT_WIN_FIFO_SIZE;
+ vinst->fault_fifo = kzalloc(vinst->fault_fifo_size, GFP_KERNEL);
+ if (!vinst->fault_fifo) {
+ pr_err("Unable to alloc %d bytes for fault_fifo\n",
+ vinst->fault_fifo_size);
+ return -ENOMEM;
+ }
+
+ vas_init_rx_win_attr(, VAS_COP_TYPE_FAULT);
+
+ attr.rx_fifo_size = vinst->fault_fifo_size;
+ attr.rx_fifo = vinst->fault_fifo;
+
+ /*
+* Max creds is based on number of CRBs can fit in the FIFO.
+* (fault_fifo_size/CRB_SIZE). If 8MB FIFO is used, max creds
+* will be 0x since the receive creds field is 16bits wide.
+*/
+ attr.wcreds_max = vinst->fault_fifo_size / CRB_SIZE;
+ attr.lnotify_lpid = 0;
+ attr.lnotify_pid = mfspr(SPRN_PID);
+ attr.lnotify_tid = mfspr(SPRN_PID);
+
+ vinst->fault_win = vas_rx_win_open(vinst->vas_id, VAS_COP_TYPE_FAULT,
+ );
+
+ if (IS_ERR(vinst->fault_win)) {
+ pr_err("VAS: Error %ld opening FaultWin\n",
+ PTR_ERR(vinst->fault_win));
+ kfree(vinst->fault_fifo);
+ return PTR_ERR(vinst->fault_win);
+ }
+
+ pr_devel("VAS: Created FaultWin %d, LPID/PID/TID [%d/%d/%d]\n",
+ vinst->fault_win->winid, attr.lnotify_lpid,
+ attr.lnotify_pid, attr.lnotify_tid);
+
+ return 0;
+}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0c0d27d..1783fa9 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -827,9 +827,9 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
rxattr->fault_win = true;
rxattr->notify_disable = true;
rxattr->rx_wcred_mode = true;
- rxattr->tx_wcred_mode = true;
rxattr->rx_win_ord_mode = true;
- rxattr->tx_win_ord_mode = true;
+ rxattr->rej_no_credit = true;
+ rxattr->tc_mode = VAS_THRESH_DISABLED;
} else if (cop == VAS_COP_TYPE_FTW) {
rxattr->user_win = true;
rxattr->intr_disable = true;
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index 168ab68..557c8e4 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -24,6 +24,1
[PATCH V5 04/14] powerpc/vas: Alloc and setup IRQ and trigger port address
Alloc IRQ and get trigger port address for each VAS instance. Kernel
register this IRQ per VAS instance and sets this port for each send
window. NX interrupts the kernel when it sees page fault.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas.c | 34 --
arch/powerpc/platforms/powernv/vas.h | 2 ++
2 files changed, 30 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index ed9cc6d..168ab68 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -15,6 +15,7 @@
#include
#include
#include
+#include
#include "vas.h"
@@ -25,10 +26,12 @@
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, cpu, vasid;
- struct resource *res;
- struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
+ struct vas_instance *vinst;
+ uint32_t chipid, irq;
+ struct resource *res;
+ int rc, cpu, vasid;
+ uint64_t port;
rc = of_property_read_u32(dn, "ibm,vas-id", );
if (rc) {
@@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev)
return -ENODEV;
}
+ rc = of_property_read_u32(dn, "ibm,chip-id", );
+ if (rc) {
+ pr_err("No ibm,chip-id property for %s?\n", pdev->name);
+ return -ENODEV;
+ }
+
if (pdev->num_resources != 4) {
pr_err("Unexpected DT configuration for [%s, %d]\n",
pdev->name, vasid);
@@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev)
vinst->paste_win_id_shift = 63 - res->end;
- pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, "
- "paste_win_id_shift 0x%llx\n", pdev->name, vasid,
- vinst->paste_base_addr, vinst->paste_win_id_shift);
+ rc = xive_native_alloc_get_irq_info(chipid, , );
+ if (rc)
+ return rc;
+
+ vinst->virq = irq_create_mapping(NULL, irq);
+ if (!vinst->virq) {
+ pr_err("Inst%d: Unable to map global irq %d\n",
+ vinst->vas_id, irq);
+ return -EINVAL;
+ }
+
+ vinst->irq_port = port;
+ pr_devel("Initialized instance [%s, %d] paste_base 0x%llx
paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n",
+ pdev->name, vasid, vinst->paste_base_addr,
+ vinst->paste_win_id_shift, vinst->virq,
+ vinst->irq_port);
for_each_possible_cpu(cpu) {
if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 5574aec..598608b 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -313,6 +313,8 @@ struct vas_instance {
u64 paste_base_addr;
u64 paste_win_id_shift;
+ u64 irq_port;
+ int virq;
struct mutex mutex;
struct vas_window *rxwin[VAS_COP_TYPE_MAX];
struct vas_window *windows[VAS_WINDOWS_PER_CHIP];
--
1.8.3.1
[PATCH V5 03/14] powerpc/vas: Define nx_fault_stamp in coprocessor_request_block
Kernel sets fault address and status in CRB for NX page fault on user
space address after processing page fault. User space gets the signal
and handles the fault mentioned in CRB by bringing the page in to
memory and send NX request again.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/icswx.h | 18 +-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/icswx.h b/arch/powerpc/include/asm/icswx.h
index 9872f85..b233d1e 100644
--- a/arch/powerpc/include/asm/icswx.h
+++ b/arch/powerpc/include/asm/icswx.h
@@ -108,6 +108,17 @@ struct data_descriptor_entry {
__be64 address;
} __packed __aligned(DDE_ALIGN);
+/* 4.3.2 NX-stamped Fault CRB */
+
+#define NX_STAMP_ALIGN (0x10)
+
+struct nx_fault_stamp {
+ __be64 fault_storage_addr;
+ __be16 reserved;
+ __u8 flags;
+ __u8 fault_status;
+ __be32 pswid;
+} __packed __aligned(NX_STAMP_ALIGN);
/* Chapter 6.5.2 Coprocessor-Request Block (CRB) */
@@ -135,7 +146,12 @@ struct coprocessor_request_block {
struct coprocessor_completion_block ccb;
- u8 reserved[48];
+ union {
+ struct nx_fault_stamp nx;
+ u8 reserved[16];
+ } stamp;
+
+ u8 reserved[32];
struct coprocessor_status_block csb;
} __packed __aligned(CRB_ALIGN);
--
1.8.3.1
[PATCH V5 02/14] powerpc/xive: Define xive_native_alloc_get_irq_info()
pnv_ocxl_alloc_xive_irq() in ocxl.c allocates IRQ and gets trigger port
address. VAS also needs this function, but based on chip ID. So moved
this common function to xive/native.c.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 2 ++
arch/powerpc/platforms/powernv/ocxl.c | 20 ++--
arch/powerpc/sysdev/xive/native.c | 23 +++
3 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 7ba6a90..382f5ed 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -139,6 +139,8 @@ extern int xive_native_set_queue_state(u32 vp_id, uint32_t
prio, u32 qtoggle,
extern int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
extern bool xive_native_has_queue_state_support(void);
extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+extern int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq,
+ u64 *trigger_addr);
static inline u32 xive_native_alloc_irq(void)
{
diff --git a/arch/powerpc/platforms/powernv/ocxl.c
b/arch/powerpc/platforms/powernv/ocxl.c
index 8c65aac..fb8f99a 100644
--- a/arch/powerpc/platforms/powernv/ocxl.c
+++ b/arch/powerpc/platforms/powernv/ocxl.c
@@ -487,24 +487,8 @@ int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data,
int pe_handle)
int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr)
{
- __be64 flags, trigger_page;
- s64 rc;
- u32 hwirq;
-
- hwirq = xive_native_alloc_irq();
- if (!hwirq)
- return -ENOENT;
-
- rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
- NULL);
- if (rc || !trigger_page) {
- xive_native_free_irq(hwirq);
- return -ENOENT;
- }
- *irq = hwirq;
- *trigger_addr = be64_to_cpu(trigger_page);
- return 0;
-
+ return xive_native_alloc_get_irq_info(OPAL_XIVE_ANY_CHIP, irq,
+ trigger_addr);
}
EXPORT_SYMBOL_GPL(pnv_ocxl_alloc_xive_irq);
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 14d4406..abdd892 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -295,6 +295,29 @@ u32 xive_native_alloc_irq_on_chip(u32 chip_id)
}
EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
+int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq, u64 *trigger_addr)
+{
+ __be64 flags, trigger_page;
+ u32 hwirq;
+ s64 rc;
+
+ hwirq = xive_native_alloc_irq_on_chip(chip_id);
+ if (!hwirq)
+ return -ENOENT;
+
+ rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
+ NULL);
+ if (rc || !trigger_page) {
+ xive_native_free_irq(hwirq);
+ return -ENOENT;
+ }
+ *irq = hwirq;
+ *trigger_addr = be64_to_cpu(trigger_page);
+
+ return 0;
+}
+EXPORT_SYMBOL(xive_native_alloc_get_irq_info);
+
void xive_native_free_irq(u32 irq)
{
for (;;) {
--
1.8.3.1
[PATCH V5 01/14] powerpc/xive: Define xive_native_alloc_irq_on_chip()
This function allocates IRQ on a specific chip. VAS needs per chip
IRQ allocation and will have IRQ handler per VAS instance.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 9 -
arch/powerpc/sysdev/xive/native.c | 6 +++---
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 24cdf97..7ba6a90 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -5,6 +5,8 @@
#ifndef _ASM_POWERPC_XIVE_H
#define _ASM_POWERPC_XIVE_H
+#include
+
#define XIVE_INVALID_VP0x
#ifdef CONFIG_PPC_XIVE
@@ -108,7 +110,6 @@ struct xive_q {
extern int xive_native_populate_irq_data(u32 hw_irq,
struct xive_irq_data *data);
extern void xive_cleanup_irq_data(struct xive_irq_data *xd);
-extern u32 xive_native_alloc_irq(void);
extern void xive_native_free_irq(u32 irq);
extern int xive_native_configure_irq(u32 hw_irq, u32 target, u8 prio, u32
sw_irq);
@@ -137,6 +138,12 @@ extern int xive_native_set_queue_state(u32 vp_id, uint32_t
prio, u32 qtoggle,
u32 qindex);
extern int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
extern bool xive_native_has_queue_state_support(void);
+extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+
+static inline u32 xive_native_alloc_irq(void)
+{
+ return xive_native_alloc_irq_on_chip(OPAL_XIVE_ANY_CHIP);
+}
#else
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 0ff6b73..14d4406 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -279,12 +279,12 @@ static int xive_native_get_ipi(unsigned int cpu, struct
xive_cpu *xc)
}
#endif /* CONFIG_SMP */
-u32 xive_native_alloc_irq(void)
+u32 xive_native_alloc_irq_on_chip(u32 chip_id)
{
s64 rc;
for (;;) {
- rc = opal_xive_allocate_irq(OPAL_XIVE_ANY_CHIP);
+ rc = opal_xive_allocate_irq(chip_id);
if (rc != OPAL_BUSY)
break;
msleep(OPAL_BUSY_DELAY_MS);
@@ -293,7 +293,7 @@ u32 xive_native_alloc_irq(void)
return 0;
return rc;
}
-EXPORT_SYMBOL_GPL(xive_native_alloc_irq);
+EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
void xive_native_free_irq(u32 irq)
{
--
1.8.3.1
[PATCH V5 00/14] powerpc/vas: Page fault handling for user space NX requests
On power9, Virtual Accelerator Switchboard (VAS) allows user space or kernel to communicate with Nest Accelerator (NX) directly using COPY/PASTE instructions. NX provides verious functionalities such as compression, encryption and etc. But only compression (842 and GZIP formats) is supported in Linux kernel on power9. 842 compression driver (drivers/crypto/nx/nx-842-powernv.c) is already included in Linux. Only GZIP support will be available from user space. Applications can issue GZIP compression / decompression requests to NX with COPY/PASTE instructions. When NX is processing these requests, can hit fault on the request buffer (not in memory). It issues an interrupt and pastes fault CRB in fault FIFO. Expects kernel to handle this fault and return credits for both send and fault windows after processing. This patch series adds IRQ and fault window setup, and NX fault handling: - Alloc IRQ and trigger port address, and configure IRQ per VAS instance. - Set port# for each window to generate an interrupt when noticed fault. - Set fault window and FIFO on which NX paste fault CRB. - Setup IRQ thread fault handler per VAS instance. - When receiving an interrupt, Read CRBs from fault FIFO and update coprocessor_status_block (CSB) in the corresponding CRB with translation failure (CSB_CC_TRANSLATION). After issuing NX requests, process polls on CSB address. When it sees translation error, can touch the request buffer to bring the page in to memory and reissue NX request. - If copy_to_user fails on user space CSB address, OS sends SEGV signal. Tested these patches with NX-GZIP support and will be posting this series soon. Patches 1 & 2: Define alloc IRQ and get port address per chip which are needed to alloc IRQ per VAS instance. Patch 3: Define nx_fault_stamp on which NX writes fault status for the fault CRB Patch 4: Alloc and setup IRQ and trigger port address for each VAS instance Patch 5: Setup fault window per each VAS instance. This window is used for NX to paste fault CRB in FIFO. Patches 6 & 7: Setup threaded IRQ per VAS and register NX with fault window ID and port number for each send window so that NX paste fault CRB in this window. Patch 8: Reference to pid and mm so that pid is not used until window closed. Needed for multi thread application where child can open a window and can be used by parent later. Patches 9 and 10: Process CRBs from fault FIFO and notify tasks by updating CSB or through signals. Patches 11 and 12: Return credits for send and fault windows after handling faults. Patch 14:Fix closing send window after all credits are returned. This issue happens only for user space requests. No page faults on kernel request buffer. Changelog: V2: - Use threaded IRQ instead of own kernel thread handler - Use pswid insted of user space CSB address to find valid CRB - Removed unused macros and other changes as suggested by Christoph Hellwig V3: - Rebased to 5.5-rc2 - Use struct pid * instead of pid_t for vas_window tgid - Code cleanup as suggested by Christoph Hellwig V4: - Define xive alloc and get IRQ info based on chip ID and use these functions for IRQ setup per VAS instance. It eliminates skiboot dependency as suggested by Oliver. V5: - Do not update CSB if the process is exiting (patch9) Haren Myneni (14): powerpc/xive: Define xive_native_alloc_irq_on_chip() powerpc/xive: Define xive_native_alloc_get_irq_info() powerpc/vas: Define nx_fault_stamp in coprocessor_request_block powerpc/vas: Alloc and setup IRQ and trigger port address powerpc/vas: Setup fault window per VAS instance powerpc/vas: Setup thread IRQ handler per VAS instance powerpc/vas: Register NX with fault window ID and IRQ port value powerpc/vas: Take reference to PID and mm for user space windows powerpc/vas: Update CSB and notify process for fault CRBs powerpc/vas: Print CRB and FIFO values powerpc/vas: Do not use default credits for receive window powerpc/VAS: Return credits after handling fault powerpc/vas: Display process stuck message powerpc/vas: Free send window in VAS instance after credits returned arch/powerpc/include/asm/icswx.h| 18 +- arch/powerpc/include/asm/xive.h | 11 +- arch/powerpc/platforms/powernv/Makefile | 2 +- arch/powerpc/platforms/powernv/ocxl.c | 20 +- arch/powerpc/platforms/powernv/vas-debug.c | 2 +- arch/powerpc/platforms/powernv/vas-fault.c | 325 arch/powerpc/platforms/powernv/vas-window.c | 184 ++-- arch/powerpc/platforms/powernv/vas.c| 73 ++- arch/powerpc/platforms/powernv/vas.h| 38 +++- arch/powerpc/sysdev/xive/native.c | 29 ++- 10 files changed, 655 insertions(+), 47 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c -- 1.8.3.1
[PATCH V4 14/14] powerpc/vas: Free send window in VAS instance after credits returned
NX may be processing requests while trying to close window. Wait until all credits are returned and then free send window from VAS instance. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/powernv/vas-window.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c index d7d53e4..b612670 100644 --- a/arch/powerpc/platforms/powernv/vas-window.c +++ b/arch/powerpc/platforms/powernv/vas-window.c @@ -1317,14 +1317,14 @@ int vas_win_close(struct vas_window *window) unmap_paste_region(window); - clear_vinst_win(window); - poll_window_busy_state(window); unpin_close_window(window); poll_window_credits(window); + clear_vinst_win(window); + poll_window_castout(window); /* if send window, drop reference to matching receive window */ -- 1.8.3.1
[PATCH V4 13/14] powerpc/vas: Display process stuck message
Process can not close send window until all requests are processed.
Means wait until window state is not busy and send credits are
returned. Display debug message in case taking longer to close the
window.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 27 +++
1 file changed, 27 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8d90d25..d7d53e4 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1183,6 +1183,7 @@ static void poll_window_credits(struct vas_window *window)
{
u64 val;
int creds, mode;
+ int count = 0;
val = read_hvwc_reg(window, VREG(WINCTL));
if (window->tx_win)
@@ -1201,10 +1202,26 @@ static void poll_window_credits(struct vas_window
*window)
creds = GET_FIELD(VAS_LRX_WCRED, val);
}
+ /*
+* Takes around few microseconds to complete all pending requests
+* and return credits.
+* TODO: Scan fault FIFO and invalidate CRBs points to this window
+* and issue CRB Kill to stop all pending requests. Need only
+* if there is a bug in NX or fault handling in kernel.
+*/
if (creds < window->wcreds_max) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(10));
+ count++;
+ /*
+* Process can not close send window until all credits are
+* returned.
+*/
+ if (!(count % 1))
+ pr_debug("%s() pid %d stuck? retries %d\n", __func__,
+ vas_window_pid(window), count);
+
goto retry;
}
}
@@ -1218,6 +1235,7 @@ static void poll_window_busy_state(struct vas_window
*window)
{
int busy;
u64 val;
+ int count = 0;
retry:
val = read_hvwc_reg(window, VREG(WIN_STATUS));
@@ -1226,6 +1244,15 @@ static void poll_window_busy_state(struct vas_window
*window)
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(5));
+ count++;
+ /*
+* Takes around 5 microseconds to process all pending
+* requests.
+*/
+ if (!(count % 1))
+ pr_debug("%s() pid %d stuck? retries %d\n", __func__,
+ vas_window_pid(window), count);
+
goto retry;
}
}
--
1.8.3.1
[PATCH V4 12/14] powerpc/VAS: Return credits after handling fault
NX expects OS to return credit for send window after processing each
fault. Also credit has to be returned even for fault window.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 10 ++
arch/powerpc/platforms/powernv/vas-window.c | 17 +
arch/powerpc/platforms/powernv/vas.h| 1 +
3 files changed, 28 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index cf776d2..9c31568 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -239,6 +239,11 @@ irqreturn_t vas_fault_handler(int irq, void *data)
memset(fifo, 0, CRB_SIZE);
mutex_unlock(>mutex);
+ /*
+* Return credit for the fault window.
+*/
+ vas_return_credit(vinst->fault_win, 0);
+
pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
@@ -265,6 +270,11 @@ irqreturn_t vas_fault_handler(int irq, void *data)
}
update_csb(window, crb);
+ /*
+* Return credit for send window after processing
+* fault CRB.
+*/
+ vas_return_credit(window, 1);
} while (true);
return IRQ_HANDLED;
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8428970..8d90d25 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1319,6 +1319,23 @@ int vas_win_close(struct vas_window *window)
}
EXPORT_SYMBOL_GPL(vas_win_close);
+/*
+ * Return credit for the given window.
+ */
+void vas_return_credit(struct vas_window *window, bool tx)
+{
+ uint64_t val;
+
+ val = 0ULL;
+ if (tx) { /* send window */
+ val = SET_FIELD(VAS_TX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), val);
+ } else {
+ val = SET_FIELD(VAS_LRX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), val);
+ }
+}
+
struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid)
{
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index f5f45ea..495937a 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -415,6 +415,7 @@ struct vas_winctx {
extern void vas_window_free_dbgdir(struct vas_window *win);
extern int vas_setup_fault_window(struct vas_instance *vinst);
extern irqreturn_t vas_fault_handler(int irq, void *data);
+extern void vas_return_credit(struct vas_window *window, bool tx);
extern struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid);
--
1.8.3.1
[PATCH V4 11/14] powerpc/vas: Do not use default credits for receive window
System checkstops if RxFIFO overruns with more requests than the
maximum possible number of CRBs allowed in FIFO at any time. So
max credits value (rxattr.wcreds_max) is set and is passed to
vas_rx_win_open() by the the driver.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 4 ++--
arch/powerpc/platforms/powernv/vas.h| 2 --
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0f27ac5..8428970 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -772,7 +772,7 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
- if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ if (!attr->wcreds_max)
return false;
if (attr->nx_win) {
@@ -878,7 +878,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
- rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ rxwin->wcreds_max = rxattr->wcreds_max;
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index af03aa0..f5f45ea 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -101,11 +101,9 @@
/*
* Initial per-process credits.
* Max send window credits:4K-1 (12-bits in VAS_TX_WCRED)
- * Max receive window credits: 64K-1 (16 bits in VAS_LRX_WCRED)
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
--
1.8.3.1
[PATCH V4 10/14] powerpc/vas: Print CRB and FIFO values
Dump FIFO entry values if could not find send window and print CRB
for debugging.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 41 ++
1 file changed, 41 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 1d31d8d..cf776d2 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -26,6 +26,28 @@
*/
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+static void dump_crb(struct coprocessor_request_block *crb)
+{
+ struct data_descriptor_entry *dde;
+ struct nx_fault_stamp *nx;
+
+ dde = >source;
+ pr_devel("SrcDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ dde = >target;
+ pr_devel("TgtDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ nx = >stamp.nx;
+ pr_devel("NX Stamp: PSWID 0x%x, FSA 0x%llx, flags 0x%x, FS 0x%x\n",
+ be32_to_cpu(nx->pswid),
+ be64_to_cpu(crb->stamp.nx.fault_storage_addr),
+ nx->flags, be32_to_cpu(nx->fault_status));
+}
+
/*
* Update the CSB to indicate a translation error.
*
@@ -144,6 +166,23 @@ static void update_csb(struct vas_window *window,
pid_vnr(pid), rc);
}
+static void dump_fifo(struct vas_instance *vinst, void *entry)
+{
+ int i;
+ unsigned long *fifo = entry;
+
+ pr_err("Fault fifo size %d, max crbs %d, crb size %lu\n",
+ vinst->fault_fifo_size,
+ vinst->fault_fifo_size / CRB_SIZE,
+ sizeof(struct coprocessor_request_block));
+
+ pr_err("Fault FIFO Entry Dump:\n");
+ for (i = 0; i < CRB_SIZE; i += 4, fifo += 4) {
+ pr_err("[%.3d, %p]: 0x%.16lx 0x%.16lx 0x%.16lx 0x%.16lx\n",
+ i, fifo, *fifo, *(fifo+1), *(fifo+2), *(fifo+3));
+ }
+}
+
/*
* Process CRBs that we receive on the fault window.
*/
@@ -204,6 +243,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
+ dump_crb(crb);
window = vas_pswid_to_window(vinst,
be32_to_cpu(crb->stamp.nx.pswid));
@@ -214,6 +254,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
* even clean it up (return credit).
* But we should not get here.
*/
+ dump_fifo(vinst, (void *)crb);
pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
vinst->vas_id, vinst->fault_fifo, fifo,
be32_to_cpu(crb->stamp.nx.pswid),
--
1.8.3.1
[PATCH V4 09/14] powerpc/vas: Update CSB and notify process for fault CRBs
For each fault CRB, update fault address in CRB (fault_storage_addr)
and translation error status in CSB so that user space can touch the
fault address and resend the request. If the user space passed invalid
CSB address send signal to process with SIGSEGV.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 120
+
1 file changed, 120 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 5c2cada..1d31d8d 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,6 +27,124 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Update the CSB to indicate a translation error.
+ *
+ * If the fault is in the CSB address itself or if we are unable to
+ * update the CSB, send a signal to the process, because we have no
+ * other way of notifying the user process.
+ *
+ * Remaining settings in the CSB are based on wait_for_csb() of
+ * NX-GZIP.
+ */
+static void update_csb(struct vas_window *window,
+ struct coprocessor_request_block *crb)
+{
+ int rc;
+ struct pid *pid;
+ void __user *csb_addr;
+ struct task_struct *tsk;
+ struct kernel_siginfo info;
+ struct coprocessor_status_block csb;
+
+ /*
+* NX user space windows can not be opened for task->mm=NULL
+* and faults will not be generated for kernel requests.
+*/
+ if (!window->mm || !window->user_win)
+ return;
+
+ csb_addr = (void *)be64_to_cpu(crb->csb_addr);
+
+ csb.cc = CSB_CC_TRANSLATION;
+ csb.ce = CSB_CE_TERMINATION;
+ csb.cs = 0;
+ csb.count = 0;
+
+ /*
+* Returns the fault address in CPU format since it is passed with
+* signal. But if the user space expects BE format, need changes.
+* i.e either kernel (here) or user should convert to CPU format.
+* Not both!
+*/
+ csb.address = be64_to_cpu(crb->stamp.nx.fault_storage_addr);
+ csb.flags = 0;
+
+ use_mm(window->mm);
+ rc = copy_to_user(csb_addr, , sizeof(csb));
+ /*
+* User space polls on csb.flags (first byte). So add barrier
+* then copy first byte with csb flags update.
+*/
+ smp_mb();
+ if (!rc) {
+ csb.flags = CSB_V;
+ rc = copy_to_user(csb_addr, , sizeof(u8));
+ }
+ unuse_mm(window->mm);
+
+ /* Success */
+ if (!rc)
+ return;
+
+ /*
+* User space passed invalid CSB address, Notify process with
+* SEGV signal.
+*/
+ pid = window->pid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Send window will be closed after processing all NX requests
+* and process exits after closing all windows. In multi-thread
+* applications, thread may not exists, but does not close FD
+* (means send window) upon exit. Parent thread (tgid) can use
+* and close the window later.
+* pid and mm references are taken when window is opened by
+* process (pid). So tgid is used only when child thread opens
+* a window and exits without closing it in multithread tasks.
+*/
+ if (!tsk) {
+ pid = window->tgid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Parent thread will be closing window during its exit.
+* So should not get here.
+*/
+ if (!tsk)
+ return;
+ }
+
+ /* Do not notify if the task is exiting. */
+ if (tsk->flags & PF_EXITING) {
+ put_task_struct(tsk);
+ return;
+ }
+ put_task_struct(tsk);
+
+ pr_err("Invalid CSB address 0x%p signalling pid(%d)\n",
+ csb_addr, pid_vnr(pid));
+
+ clear_siginfo();
+ info.si_signo = SIGSEGV;
+ info.si_errno = EFAULT;
+ info.si_code = SEGV_MAPERR;
+ info.si_addr = csb_addr;
+
+ /*
+* process will be polling on csb.flags after request is sent to
+* NX. So generally CSB update should not fail except when an
+* application does not follow the process properly. So an error
+* message will be displayed and leave it to user space whether
+* to ignore or handle this signal.
+*/
+ rcu_read_lock();
+ rc = kill_pid_info(SIGSEGV, , pid);
+ rcu_read_unlock();
+
+ pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
+ pid_vnr(pid), rc);
+}
+
+/*
* Process CRBs that we receive on the fault window.
*/
irqreturn_t vas_fault_handler(int irq, void *dat
[PATCH V4 06/14] powerpc/vas: Setup thread IRQ handler per VAS instance
Setup thread IRQ handler per each VAS instance. When NX sees a fault
on CRB, kernel gets an interrupt and vas_fault_handler will be
executed to process fault CRBs. Read all valid CRBs from fault FIFO,
determine the corresponding send window from CRB and process fault
requests.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 85 +
arch/powerpc/platforms/powernv/vas-window.c | 60
arch/powerpc/platforms/powernv/vas.c| 21 ++-
arch/powerpc/platforms/powernv/vas.h| 4 ++
4 files changed, 169 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index b0258ed..5c2cada 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include "vas.h"
@@ -25,6 +26,90 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Process CRBs that we receive on the fault window.
+ */
+irqreturn_t vas_fault_handler(int irq, void *data)
+{
+ struct vas_instance *vinst = data;
+ struct coprocessor_request_block buf, *crb;
+ struct vas_window *window;
+ void *fifo;
+
+ /*
+* VAS can interrupt with multiple page faults. So process all
+* valid CRBs within fault FIFO until reaches invalid CRB.
+* NX updates nx_fault_stamp in CRB and pastes in fault FIFO.
+* kernel retrives send window from parition send window ID
+* (pswid) in nx_fault_stamp. So pswid should be non-zero and
+* use this to check whether CRB is valid.
+* After reading CRB entry, it is reset with 0's in fault FIFO.
+*
+* In case kernel receives another interrupt with different page
+* fault and CRBs are processed by the previous handling, will be
+* returned from this function when it sees invalid CRB (means 0's).
+*/
+ do {
+ mutex_lock(>mutex);
+
+ /*
+* Advance the fault fifo pointer to next CRB.
+* Use CRB_SIZE rather than sizeof(*crb) since the latter is
+* aligned to CRB_ALIGN (256) but the CRB written to by VAS is
+* only CRB_SIZE in len.
+*/
+ fifo = vinst->fault_fifo + (vinst->fault_crbs * CRB_SIZE);
+ crb = fifo;
+
+ /*
+* NX pastes nx_fault_stamp in fault FIFO for each fault.
+* So use pswid to check whether fault CRB is valid.
+* pswid returned from NX will be in _be32, but just
+* checking non-zero value to make sure the CRB is valid.
+* Return if reached invalid CRB.
+*/
+ if (!crb->stamp.nx.pswid) {
+ mutex_unlock(>mutex);
+ return IRQ_HANDLED;
+ }
+
+ vinst->fault_crbs++;
+ if (vinst->fault_crbs == (vinst->fault_fifo_size / CRB_SIZE))
+ vinst->fault_crbs = 0;
+
+ crb =
+ memcpy(crb, fifo, CRB_SIZE);
+ memset(fifo, 0, CRB_SIZE);
+ mutex_unlock(>mutex);
+
+ pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ vinst->fault_crbs);
+
+ window = vas_pswid_to_window(vinst,
+ be32_to_cpu(crb->stamp.nx.pswid));
+
+ if (IS_ERR(window)) {
+ /*
+* We got an interrupt about a specific send
+* window but we can't find that window and we can't
+* even clean it up (return credit).
+* But we should not get here.
+*/
+ pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ be32_to_cpu(crb->stamp.nx.pswid),
+ vinst->fault_crbs);
+
+ WARN_ON_ONCE(1);
+ return IRQ_HANDLED;
+ }
+
+ } while (true);
+
+ return IRQ_HANDLED;
+}
+
+/*
* Fault window is opened per VAS instance. NX pastes fault CRB in fault
* FIFO upon page faults.
*/
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index f07f49a..cec1b41 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1041,6 +1041,15 @@ struct vas_w
[PATCH V4 08/14] powerpc/vas: Take reference to PID and mm for userspace windows
Process close windows after its requests are completed. In multi-thread
applications, child can open a window but release FD will not be called
upon its exit. Parent thread will be closing it later upon its exit.
The parent can also send NX requests with this window and NX can
generate page faults. After kernel handles the page fault, send
signal to process by using PID if CSB address is invalid. Parent
thread will not receive signal since its PID is different from the one
saved in vas_window. So use tgid in case if the task for the pid saved
in window is not running and send signal to its parent.
To prevent reusing the pid until the window closed, take reference to
pid and task mm.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-debug.c | 2 +-
arch/powerpc/platforms/powernv/vas-window.c | 53 ++---
arch/powerpc/platforms/powernv/vas.h| 9 -
3 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index 09e63df..ef9a717 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -38,7 +38,7 @@ static int info_show(struct seq_file *s, void *private)
seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
window->tx_win ? "Send" : "Receive");
- seq_printf(s, "Pid : %d\n", window->pid);
+ seq_printf(s, "Pid : %d\n", vas_window_pid(window));
unlock:
mutex_unlock(_mutex);
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index e36c5d2..0f27ac5 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -12,6 +12,8 @@
#include
#include
#include
+#include
+#include
#include
#include
#include "vas.h"
@@ -877,8 +879,6 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
- if (rxattr->user_win)
- rxwin->pid = task_pid_vnr(current);
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
@@ -1028,7 +1028,6 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->tx_win = 1;
txwin->rxwin = rxwin;
txwin->nx_win = txwin->rxwin->nx_win;
- txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
@@ -1069,8 +1068,43 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
goto free_window;
}
- set_vinst_win(vinst, txwin);
+ if (txwin->user_win) {
+ /*
+* Window opened by child thread may not be closed when
+* it exits. So take reference to its pid and release it
+* when the window is free by parent thread.
+* Acquire a reference to the task's pid to make sure
+* pid will not be re-used - needed only for multithread
+* applications.
+*/
+ txwin->pid = get_task_pid(current, PIDTYPE_PID);
+ /*
+* Acquire a reference to the task's mm.
+*/
+ txwin->mm = get_task_mm(current);
+ if (!txwin->mm) {
+ put_pid(txwin->pid);
+ pr_err("VAS: pid(%d): mm_struct is not found\n",
+ current->pid);
+ rc = -EPERM;
+ goto free_window;
+ }
+
+ mmgrab(txwin->mm);
+ mmput(txwin->mm);
+ mm_context_add_copro(txwin->mm);
+ /*
+* Process closes window during exit. In the case of
+* multithread application, child can open window and
+* can exit without closing it. Expects parent thread
+* to use and close the window. So do not need to take
+* pid reference for parent thread.
+*/
+ txwin->tgid = find_get_pid(task_tgid_vnr(current));
+ }
+
+ set_vinst_win(vinst, txwin);
return txwin;
free_window:
@@ -1267,8 +1301,17 @@ int vas_win_close(struct vas_window *window)
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
- if (window->tx_win)
+ if (window->tx_win) {
+ if (window->user_win) {
+ /* Drop references to pid and mm */
+ put_pid(wi
[PATCH V4 07/14] powerpc/vas: Register NX with fault window ID and IRQ port value
For each user space send window, register NX with fault window ID
and port value so that NX paste CRBs in this fault FIFO when it
sees fault on the request buffer.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 15 +--
arch/powerpc/platforms/powernv/vas.h| 15 +++
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index cec1b41..e36c5d2 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -373,7 +373,7 @@ int init_winctx_regs(struct vas_window *window, struct
vas_winctx *winctx)
init_xlate_regs(window, winctx->user_win);
val = 0ULL;
- val = SET_FIELD(VAS_FAULT_TX_WIN, val, 0);
+ val = SET_FIELD(VAS_FAULT_TX_WIN, val, winctx->fault_win_id);
write_hvwc_reg(window, VREG(FAULT_TX_WIN), val);
/* In PowerNV, interrupts go to HV. */
@@ -748,6 +748,8 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (rxwin->vinst->virq)
+ winctx->irq_port = rxwin->vinst->irq_port;
}
static bool rx_win_args_valid(enum vas_cop_type cop,
@@ -945,13 +947,22 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
winctx->rx_win_id = txwin->rxwin->winid;
+ /*
+* IRQ and fault window setup is successful. Set fault window
+* for the send window so that ready to handle faults.
+*/
+ if (txwin->vinst->virq)
+ winctx->fault_win_id = txwin->vinst->fault_win->winid;
winctx->dma_type = VAS_DMA_TYPE_INJECT;
winctx->tc_mode = txattr->tc_mode;
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (txwin->vinst->virq)
+ winctx->irq_port = txwin->vinst->irq_port;
- winctx->pswid = 0;
+ winctx->pswid = txattr->pswid ? txattr->pswid :
+ encode_pswid(txwin->vinst->vas_id, txwin->winid);
}
static bool tx_win_args_valid(enum vas_cop_type cop,
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 879f5b4..2621df1 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -455,6 +455,21 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ return ((u32)winid | (vasid << (31 - 7)));
+}
+
static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
{
if (vasid)
--
1.8.3.1
[PATCH V4 05/14] powerpc/vas: Setup fault window per VAS instance
Setup fault window for each VAS instance. When NX gets fault on request
buffer, write fault CRBs in the corresponding fault FIFO and then sends
an interrupt to the OS.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-fault.c | 73
arch/powerpc/platforms/powernv/vas-window.c | 3 +-
arch/powerpc/platforms/powernv/vas.c| 20
arch/powerpc/platforms/powernv/vas.h| 5 ++
scripts/pnmtologo | Bin 0 -> 71976 bytes
6 files changed, 101 insertions(+), 2 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c
create mode 100755 scripts/pnmtologo
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index c0f8120..395789f 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
new file mode 100644
index 000..b0258ed
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * VAS Fault handling.
+ * Copyright 2019, IBM Corporation
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "vas.h"
+
+/*
+ * The maximum FIFO size for fault window can be 8MB
+ * (VAS_RX_FIFO_SIZE_MAX). Using 4MB FIFO since each VAS
+ * instance will be having fault window.
+ * 8MB FIFO can be used if expects more faults for each VAS
+ * instance.
+ */
+#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+
+/*
+ * Fault window is opened per VAS instance. NX pastes fault CRB in fault
+ * FIFO upon page faults.
+ */
+int vas_setup_fault_window(struct vas_instance *vinst)
+{
+ struct vas_rx_win_attr attr;
+
+ vinst->fault_fifo_size = VAS_FAULT_WIN_FIFO_SIZE;
+ vinst->fault_fifo = kzalloc(vinst->fault_fifo_size, GFP_KERNEL);
+ if (!vinst->fault_fifo) {
+ pr_err("Unable to alloc %d bytes for fault_fifo\n",
+ vinst->fault_fifo_size);
+ return -ENOMEM;
+ }
+
+ vas_init_rx_win_attr(, VAS_COP_TYPE_FAULT);
+
+ attr.rx_fifo_size = vinst->fault_fifo_size;
+ attr.rx_fifo = vinst->fault_fifo;
+
+ /*
+* Max creds is based on number of CRBs can fit in the FIFO.
+* (fault_fifo_size/CRB_SIZE). If 8MB FIFO is used, max creds
+* will be 0x since the receive creds field is 16bits wide.
+*/
+ attr.wcreds_max = vinst->fault_fifo_size / CRB_SIZE;
+ attr.lnotify_lpid = 0;
+ attr.lnotify_pid = mfspr(SPRN_PID);
+ attr.lnotify_tid = mfspr(SPRN_PID);
+
+ vinst->fault_win = vas_rx_win_open(vinst->vas_id, VAS_COP_TYPE_FAULT,
+ );
+
+ if (IS_ERR(vinst->fault_win)) {
+ pr_err("VAS: Error %ld opening FaultWin\n",
+ PTR_ERR(vinst->fault_win));
+ kfree(vinst->fault_fifo);
+ return PTR_ERR(vinst->fault_win);
+ }
+
+ pr_devel("VAS: Created FaultWin %d, LPID/PID/TID [%d/%d/%d]\n",
+ vinst->fault_win->winid, attr.lnotify_lpid,
+ attr.lnotify_pid, attr.lnotify_tid);
+
+ return 0;
+}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0c0d27d..f07f49a 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -827,9 +827,10 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
rxattr->fault_win = true;
rxattr->notify_disable = true;
rxattr->rx_wcred_mode = true;
- rxattr->tx_wcred_mode = true;
rxattr->rx_win_ord_mode = true;
rxattr->tx_win_ord_mode = true;
+ rxattr->rej_no_credit = true;
+ rxattr->tc_mode = VAS_THRESH_DISABLED;
} else if (cop == VAS_COP_TYPE_FTW) {
rxattr->user_win = true;
rxattr->intr_disable = true;
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index 168ab68..557c8e4
[PATCH V4 04/14] powerpc/vas: Alloc and setup IRQ and trigger port address
Alloc IRQ and get trigger port address for each VAS instance. Kernel
register this IRQ per VAS instance and sets this port for each send
window. NX interrupts the kernel when it sees page fault.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas.c | 34 --
arch/powerpc/platforms/powernv/vas.h | 2 ++
2 files changed, 30 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index ed9cc6d..168ab68 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -15,6 +15,7 @@
#include
#include
#include
+#include
#include "vas.h"
@@ -25,10 +26,12 @@
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, cpu, vasid;
- struct resource *res;
- struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
+ struct vas_instance *vinst;
+ uint32_t chipid, irq;
+ struct resource *res;
+ int rc, cpu, vasid;
+ uint64_t port;
rc = of_property_read_u32(dn, "ibm,vas-id", );
if (rc) {
@@ -36,6 +39,12 @@ static int init_vas_instance(struct platform_device *pdev)
return -ENODEV;
}
+ rc = of_property_read_u32(dn, "ibm,chip-id", );
+ if (rc) {
+ pr_err("No ibm,chip-id property for %s?\n", pdev->name);
+ return -ENODEV;
+ }
+
if (pdev->num_resources != 4) {
pr_err("Unexpected DT configuration for [%s, %d]\n",
pdev->name, vasid);
@@ -69,9 +78,22 @@ static int init_vas_instance(struct platform_device *pdev)
vinst->paste_win_id_shift = 63 - res->end;
- pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, "
- "paste_win_id_shift 0x%llx\n", pdev->name, vasid,
- vinst->paste_base_addr, vinst->paste_win_id_shift);
+ rc = xive_native_alloc_get_irq_info(chipid, , );
+ if (rc)
+ return rc;
+
+ vinst->virq = irq_create_mapping(NULL, irq);
+ if (!vinst->virq) {
+ pr_err("Inst%d: Unable to map global irq %d\n",
+ vinst->vas_id, irq);
+ return -EINVAL;
+ }
+
+ vinst->irq_port = port;
+ pr_devel("Initialized instance [%s, %d] paste_base 0x%llx
paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n",
+ pdev->name, vasid, vinst->paste_base_addr,
+ vinst->paste_win_id_shift, vinst->virq,
+ vinst->irq_port);
for_each_possible_cpu(cpu) {
if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 5574aec..598608b 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -313,6 +313,8 @@ struct vas_instance {
u64 paste_base_addr;
u64 paste_win_id_shift;
+ u64 irq_port;
+ int virq;
struct mutex mutex;
struct vas_window *rxwin[VAS_COP_TYPE_MAX];
struct vas_window *windows[VAS_WINDOWS_PER_CHIP];
--
1.8.3.1
[PATCH V4 03/14] powerpc/vas: Define nx_fault_stamp in coprocessor_request_block
Kernel sets fault address and status in CRB for NX page fault on user
space address after processing page fault. User space gets the signal
and handles the fault mentioned in CRB by bringing the page in to
memory and send NX request again.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/icswx.h | 18 +-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/icswx.h b/arch/powerpc/include/asm/icswx.h
index 9872f85..b233d1e 100644
--- a/arch/powerpc/include/asm/icswx.h
+++ b/arch/powerpc/include/asm/icswx.h
@@ -108,6 +108,17 @@ struct data_descriptor_entry {
__be64 address;
} __packed __aligned(DDE_ALIGN);
+/* 4.3.2 NX-stamped Fault CRB */
+
+#define NX_STAMP_ALIGN (0x10)
+
+struct nx_fault_stamp {
+ __be64 fault_storage_addr;
+ __be16 reserved;
+ __u8 flags;
+ __u8 fault_status;
+ __be32 pswid;
+} __packed __aligned(NX_STAMP_ALIGN);
/* Chapter 6.5.2 Coprocessor-Request Block (CRB) */
@@ -135,7 +146,12 @@ struct coprocessor_request_block {
struct coprocessor_completion_block ccb;
- u8 reserved[48];
+ union {
+ struct nx_fault_stamp nx;
+ u8 reserved[16];
+ } stamp;
+
+ u8 reserved[32];
struct coprocessor_status_block csb;
} __packed __aligned(CRB_ALIGN);
--
1.8.3.1
[PATCH V4 02/14] powerpc/xive: Define xive_native_alloc_get_irq_info()
pnv_ocxl_alloc_xive_irq() in ocxl.c allocates IRQ and gets trigger port
address. VAS also needs this function, but based on chip ID. So moved
this common function to xive/native.c.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 2 ++
arch/powerpc/platforms/powernv/ocxl.c | 20 ++--
arch/powerpc/sysdev/xive/native.c | 23 +++
3 files changed, 27 insertions(+), 18 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 7ba6a90..382f5ed 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -139,6 +139,8 @@ extern int xive_native_set_queue_state(u32 vp_id, uint32_t
prio, u32 qtoggle,
extern int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
extern bool xive_native_has_queue_state_support(void);
extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+extern int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq,
+ u64 *trigger_addr);
static inline u32 xive_native_alloc_irq(void)
{
diff --git a/arch/powerpc/platforms/powernv/ocxl.c
b/arch/powerpc/platforms/powernv/ocxl.c
index 8c65aac..fb8f99a 100644
--- a/arch/powerpc/platforms/powernv/ocxl.c
+++ b/arch/powerpc/platforms/powernv/ocxl.c
@@ -487,24 +487,8 @@ int pnv_ocxl_spa_remove_pe_from_cache(void *platform_data,
int pe_handle)
int pnv_ocxl_alloc_xive_irq(u32 *irq, u64 *trigger_addr)
{
- __be64 flags, trigger_page;
- s64 rc;
- u32 hwirq;
-
- hwirq = xive_native_alloc_irq();
- if (!hwirq)
- return -ENOENT;
-
- rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
- NULL);
- if (rc || !trigger_page) {
- xive_native_free_irq(hwirq);
- return -ENOENT;
- }
- *irq = hwirq;
- *trigger_addr = be64_to_cpu(trigger_page);
- return 0;
-
+ return xive_native_alloc_get_irq_info(OPAL_XIVE_ANY_CHIP, irq,
+ trigger_addr);
}
EXPORT_SYMBOL_GPL(pnv_ocxl_alloc_xive_irq);
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 14d4406..abdd892 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -295,6 +295,29 @@ u32 xive_native_alloc_irq_on_chip(u32 chip_id)
}
EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
+int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq, u64 *trigger_addr)
+{
+ __be64 flags, trigger_page;
+ u32 hwirq;
+ s64 rc;
+
+ hwirq = xive_native_alloc_irq_on_chip(chip_id);
+ if (!hwirq)
+ return -ENOENT;
+
+ rc = opal_xive_get_irq_info(hwirq, , NULL, _page, NULL,
+ NULL);
+ if (rc || !trigger_page) {
+ xive_native_free_irq(hwirq);
+ return -ENOENT;
+ }
+ *irq = hwirq;
+ *trigger_addr = be64_to_cpu(trigger_page);
+
+ return 0;
+}
+EXPORT_SYMBOL(xive_native_alloc_get_irq_info);
+
void xive_native_free_irq(u32 irq)
{
for (;;) {
--
1.8.3.1
[PATCH V4 01/14] powerpc/xive: Define xive_native_alloc_irq_on_chip()
This function allocates IRQ on a specific chip. VAS needs per chip
IRQ allocation and will have IRQ handler per VAS instance.
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/xive.h | 9 -
arch/powerpc/sysdev/xive/native.c | 6 +++---
2 files changed, 11 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 24cdf97..7ba6a90 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -5,6 +5,8 @@
#ifndef _ASM_POWERPC_XIVE_H
#define _ASM_POWERPC_XIVE_H
+#include
+
#define XIVE_INVALID_VP0x
#ifdef CONFIG_PPC_XIVE
@@ -108,7 +110,6 @@ struct xive_q {
extern int xive_native_populate_irq_data(u32 hw_irq,
struct xive_irq_data *data);
extern void xive_cleanup_irq_data(struct xive_irq_data *xd);
-extern u32 xive_native_alloc_irq(void);
extern void xive_native_free_irq(u32 irq);
extern int xive_native_configure_irq(u32 hw_irq, u32 target, u8 prio, u32
sw_irq);
@@ -137,6 +138,12 @@ extern int xive_native_set_queue_state(u32 vp_id, uint32_t
prio, u32 qtoggle,
u32 qindex);
extern int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
extern bool xive_native_has_queue_state_support(void);
+extern u32 xive_native_alloc_irq_on_chip(u32 chip_id);
+
+static inline u32 xive_native_alloc_irq(void)
+{
+ return xive_native_alloc_irq_on_chip(OPAL_XIVE_ANY_CHIP);
+}
#else
diff --git a/arch/powerpc/sysdev/xive/native.c
b/arch/powerpc/sysdev/xive/native.c
index 0ff6b73..14d4406 100644
--- a/arch/powerpc/sysdev/xive/native.c
+++ b/arch/powerpc/sysdev/xive/native.c
@@ -279,12 +279,12 @@ static int xive_native_get_ipi(unsigned int cpu, struct
xive_cpu *xc)
}
#endif /* CONFIG_SMP */
-u32 xive_native_alloc_irq(void)
+u32 xive_native_alloc_irq_on_chip(u32 chip_id)
{
s64 rc;
for (;;) {
- rc = opal_xive_allocate_irq(OPAL_XIVE_ANY_CHIP);
+ rc = opal_xive_allocate_irq(chip_id);
if (rc != OPAL_BUSY)
break;
msleep(OPAL_BUSY_DELAY_MS);
@@ -293,7 +293,7 @@ u32 xive_native_alloc_irq(void)
return 0;
return rc;
}
-EXPORT_SYMBOL_GPL(xive_native_alloc_irq);
+EXPORT_SYMBOL_GPL(xive_native_alloc_irq_on_chip);
void xive_native_free_irq(u32 irq)
{
--
1.8.3.1
[PATCH V4 00/14] powerpc/vas: Page fault handling for user space NX requests
On power9, Virtual Accelerator Switchboard (VAS) allows user space or kernel to communicate with Nest Accelerator (NX) directly using COPY/PASTE instructions. NX provides verious functionalities such as compression, encryption and etc. But only compression (842 and GZIP formats) is supported in Linux kernel on power9. 842 compression driver (drivers/crypto/nx/nx-842-powernv.c) is already included in Linux. Only GZIP support will be available from user space. Applications can issue GZIP compression / decompression requests to NX with COPY/PASTE instructions. When NX is processing these requests, can hit fault on the request buffer (not in memory). It issues an interrupt and pastes fault CRB in fault FIFO. Expects kernel to handle this fault and return credits for both send and fault windows after processing. This patch series adds IRQ and fault window setup, and NX fault handling: - Alloc IRQ and trigger port address, and configure IRQ per VAS instance. - Set port# for each window to generate an interrupt when noticed fault. - Set fault window and FIFO on which NX paste fault CRB. - Setup IRQ thread fault handler per VAS instance. - When receiving an interrupt, Read CRBs from fault FIFO and update coprocessor_status_block (CSB) in the corresponding CRB with translation failure (CSB_CC_TRANSLATION). After issuing NX requests, process polls on CSB address. When it sees translation error, can touch the request buffer to bring the page in to memory and reissue NX request. - If copy_to_user fails on user space CSB address, OS sends SEGV signal. Tested these patches with NX-GZIP support and will be posting this series soon. Patches 1 & 2: Define alloc IRQ and get port address per chip which are needed to alloc IRQ per VAS instance. Patch 3: Define nx_fault_stamp on which NX writes fault status for the fault CRB Patch 4: Alloc and setup IRQ and trigger port address for each VAS instance Patch 5: Setup fault window per each VAS instance. This window is used for NX to paste fault CRB in FIFO. Patches 6 & 7: Setup threaded IRQ per VAS and register NX with fault window ID and port number for each send window so that NX paste fault CRB in this window. Patch 8: Reference to pid and mm so that pid is not used until window closed. Needed for multi thread application where child can open a window and can be used by parent later. Patches 9 and 10: Process CRBs from fault FIFO and notify tasks by updating CSB or through signals. Patches 11 and 12: Return credits for send and fault windows after handling faults. Patch 14:Fix closing send window after all credits are returned. This issue happens only for user space requests. No page faults on kernel request buffer. Changelog: V2: - Use threaded IRQ instead of own kernel thread handler - Use pswid insted of user space CSB address to find valid CRB - Removed unused macros and other changes as suggested by Christoph Hellwig V3: - Rebased to 5.5-rc2 - Use struct pid * instead of pid_t for vas_window tgid - Code cleanup as suggested by Christoph Hellwig V4: - Define xive alloc and get IRQ info based on chip ID and use these functions for IRQ setup per VAS instance. It eliminates skiboot dependency as suggested by Oliver. Haren Myneni (14): powerpc/xive: Define xive_native_alloc_irq_on_chip() powerpc/xive: Define xive_native_alloc_get_irq_info() powerpc/vas: Define nx_fault_stamp in coprocessor_request_block powerpc/vas: Alloc and setup IRQ and trigger port address powerpc/vas: Setup fault window per VAS instance powerpc/vas: Setup thread IRQ handler per VAS instance powerpc/vas: Register NX with fault window ID and IRQ port value powerpc/vas: Take reference to PID and mm for user space windows powerpc/vas: Update CSB and notify process for fault CRBs powerpc/vas: Print CRB and FIFO values powerpc/vas: Do not use default credits for receive window powerpc/VAS: Return credits after handling fault powerpc/vas: Display process stuck message powerpc/vas: Free send window in VAS instance after credits returned arch/powerpc/include/asm/icswx.h| 18 +- arch/powerpc/include/asm/xive.h | 11 +- arch/powerpc/platforms/powernv/Makefile | 2 +- arch/powerpc/platforms/powernv/ocxl.c | 20 +- arch/powerpc/platforms/powernv/vas-debug.c | 2 +- arch/powerpc/platforms/powernv/vas-fault.c | 329 arch/powerpc/platforms/powernv/vas-window.c | 183 +++- arch/powerpc/platforms/powernv/vas.c| 73 +- arch/powerpc/platforms/powernv/vas.h| 38 +++- arch/powerpc/sysdev/xive/native.c | 29 ++- scripts/pnmtologo | Bin 0 -> 71976 bytes 11 files changed, 659 insertions(+), 46 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c create mode 100755 script
Re: [PATCH 04/14] powerpc/vas: Setup IRQ mapping and register port for each window
On Wed, 2019-12-18 at 15:13 -0800, Haren Myneni wrote:
> On Wed, 2019-12-18 at 18:18 +1100, Oliver O'Halloran wrote:
> > On Wed, Nov 27, 2019 at 12:07 PM Haren Myneni
> > wrote:
> > >
> > > *snip*
> > >
> > > @@ -36,7 +62,18 @@ static int init_vas_instance(struct platform_device
> > > *pdev)
> > > return -ENODEV;
> > > }
> > >
> > > - if (pdev->num_resources != 4) {
> > > + rc = of_property_read_u64(dn, "ibm,vas-port", );
> > > + if (rc) {
> > > + pr_err("No ibm,vas-port property for %s?\n", pdev->name);
> > > + /* No interrupts property */
> > > + nresources = 4;
> > > + }
> > > +
> > > + /*
> > > +* interrupts property is available with 'ibm,vas-port' property.
> > > +* 4 Resources and 1 IRQ if interrupts property is available.
> > > +*/
> > > + if (pdev->num_resources != nresources) {
> > > pr_err("Unexpected DT configuration for [%s, %d]\n",
> > > pdev->name, vasid);
> > > return -ENODEV;
> >
> > Right, so adding the IRQ in firmware will break the VAS driver in
> > existing kernels since it changes the resource count. This is IMO a
> > bug in the VAS driver that you should fix, but it does mean we need to
> > think twice about having firmware assign an interrupt at boot.
>
> Correct, Hence added vas-user-space nvram switch in skiboot.
>
> >
> > I had a closer look at this series and I'm not convinced that any
> > firmware changes are actually required either. We already have OPAL
> > calls for allocating an hwirq for the kernel to use and for getting
> > the IRQ's XIVE trigger port (see pnv_ocxl_alloc_xive_irq() for an
> > example). Why not use those here too? Doing so would allow us to
> > assign interrupts to individual windows too which might be useful for
> > the windows used by the kernel.
>
> Thanks for the pointer. like using pnv_ocxl_alloc_xive_irq(), we can
> disregard FW change. BTW, VAS fault handling is needed only for user
> space VAS windows.
>
> int vas_alloc_xive_irq(u32 chipid, u32 *irq, u64 *trigger_addr)
> {
> __be64 flags, trigger_page;
> u32 hwirq;
> s64 rc;
>
> hwirq = opal_xive_allocate_irq_raw(chipid);
> if (hwirq < 0)
> return -ENOENT;
>
> rc = opal_xive_get_irq_info(hwirq, , NULL, _page,
> NULL,
> NULL);
> if (rc || !trigger_page) {
> xive_native_free_irq(hwirq);
> return -ENOENT;
> }
>
> *irq = hwirq;
> *trigger_addr = be64_to_cpu(trigger_page);
> return 0;
> }
>
> We can have common function for VAS and cxl except per chip IRQ
> allocation is needed for each VAS instance. I will post patch-set with
> this change.
>
power9 will have only XIVE interrupt controller including on open-power
systems. Correct?
VAS need per chip IRQ allocation. The current interfaces (ex:
xive_native_alloc_irq(void)) allocates IRQ on any chip
(OPAL_XIVE_ANY_CHIP)
So to use these interfaces for VAS, any concerns with the following
patch:
Changes: passing chip_id to xive_native_alloc_irq() and define
xive_native_alloc_get_irq_info() in xive/native.c which can be used in
ocxl and VAS.
diff --git a/arch/powerpc/include/asm/xive.h b/arch/powerpc/include/asm/xive.h
index 24cdf97..b310062 100644
--- a/arch/powerpc/include/asm/xive.h
+++ b/arch/powerpc/include/asm/xive.h
@@ -108,7 +108,7 @@ struct xive_q {
extern int xive_native_populate_irq_data(u32 hw_irq,
struct xive_irq_data *data);
extern void xive_cleanup_irq_data(struct xive_irq_data *xd);
-extern u32 xive_native_alloc_irq(void);
+extern u32 xive_native_alloc_irq(u32 chip_id);
extern void xive_native_free_irq(u32 irq);
extern int xive_native_configure_irq(u32 hw_irq, u32 target, u8 prio, u32
sw_irq);
@@ -137,7 +137,8 @@ extern int xive_native_set_queue_state(u32 vp_id, uint32_t
prio, u32 qtoggle,
u32 qindex);
extern int xive_native_get_vp_state(u32 vp_id, u64 *out_state);
extern bool xive_native_has_queue_state_support(void);
-
+extern int xive_native_alloc_get_irq_info(u32 chip_id, u32 *irq,
+ u64 *trigger_addr);
#else
static inline bool xive_enabled(void) { return false; }
diff --git a/arch/powerpc/kvm/book3s_xive.c b/arch/powerpc/kvm/book3s_xive.c
index 66858b7..59009e1 100644
--- a/arch/powerp
Re: [PATCH 08/10] crypto/NX: Add NX GZIP user space API
On Tue, 2019-12-17 at 17:33 +0800, Herbert Xu wrote:
> On Sun, Dec 15, 2019 at 05:05:19AM -0800, Haren Myneni wrote:
> >
> > On power9, userspace can send GZIP compression requests directly to NX
> > once kernel establishes NX channel / window. This patch provides GZIP
> > engine access to user space via /dev/crypto/nx-gzip device node with
> > open, VAS_TX_WIN_OPEN ioctl, mmap and close operations.
> >
> > Each window corresponds to file descriptor and application can open
> > multiple windows. After the window is opened, mmap() system call to map
> > the hardware address of engine's request queue into the application's
> > virtual address space.
> >
> > Then the application can then submit one or more requests to the the
> > engine by using the copy/paste instructions and pasting the CRBs to
> > the virtual address (aka paste_address) returned by mmap().
> >
> > Signed-off-by: Sukadev Bhattiprolu
> > Signed-off-by: Haren Myneni
> > ---
> > drivers/crypto/nx/Makefile| 2 +-
> > drivers/crypto/nx/nx-842-powernv.h| 2 +
> > drivers/crypto/nx/nx-commom-powernv.c | 21 ++-
> > drivers/crypto/nx/nx-gzip-powernv.c | 282
> > ++
> > 4 files changed, 304 insertions(+), 3 deletions(-)
> > create mode 100644 drivers/crypto/nx/nx-gzip-powernv.c
>
> We already have a kernel compress API which could be exposed
> to user-space through af_alg. If every driver created their
> own user-space API it would be unmanageable.
Thanks.
Virtual Accelerator Switchboard (VAS) can provide support different
accelerators, Right now only NX is used, but possible to extend to
others in future. Or different functionalities such as fast thread
wakeup (VAS feature) with VAS windows.
So looking common VAS API for any its accelerators. Need open a window /
channel - open() and ioctl()) calls, and setup the communications with
mapping address to NX (mmap()) and close the window. Then user space
communicates to accelerator directly without kernel involvement.
Specific drivers should set window attributes such as how many requests
can be send at same time and etc. All other interfaces should be same
for any accelerator.
Also, since user space sends requests directly, should restrict
malicious users to prevent overload NX (security issue). Allowing
sysadmin to restrict /dev/crypto/nx-gzip usage.
As you suggested, SW crypto API (af_alg) can be used just for NX
compression like using API based on the accelerator functionalities. It
is socket based API with AF_ALG socket family. But is there a way for
sysadmin to restrict usage from user space? Need just few functions in
struct proto.
static struct proto_ops {
.family = PF_ALG,
.ioctl = nxgzip_ioctl,
.mmap = nxgzip_mmap,
.release = nxgzip_release,
};
Thanks
Haren
>
> Cheers,
Re: [PATCH V3 01/13] powerpc/vas: Describe vas-port and interrupts properties
On Wed, 2019-12-18 at 16:18 -0600, Rob Herring wrote: > On Mon, Dec 16, 2019 at 09:48:40PM -0800, Haren Myneni wrote: > > > > Commit message? > > > Signed-off-by: Haren Myneni > > Your author and S-o-b emails don't match. Thanks, Oliver suggested IRQ assign in the kernel instead of skiboot. In this case, we may not need this patch. > > > --- > > Documentation/devicetree/bindings/powerpc/ibm,vas.txt | 5 + > > 1 file changed, 5 insertions(+) > > > > diff --git a/Documentation/devicetree/bindings/powerpc/ibm,vas.txt > > b/Documentation/devicetree/bindings/powerpc/ibm,vas.txt > > index bf11d2f..12de08b 100644 > > --- a/Documentation/devicetree/bindings/powerpc/ibm,vas.txt > > +++ b/Documentation/devicetree/bindings/powerpc/ibm,vas.txt > > @@ -11,6 +11,8 @@ Required properties: > >window context start and length, OS/User window context start and length, > >"Paste address" start and length, "Paste window id" start bit and number > >of bits) > > +- ibm,vas-port : Port address for the interrupt. > > 64-bit? > > > +- interrupts: IRQ value for each VAS instance and level. > > > > Example: > > > > @@ -18,5 +20,8 @@ Example: > > compatible = "ibm,vas", "ibm,power9-vas"; > > reg = <0x60191 0x200 0x60190 0x1 > > 0x8 0x1 0x20 0x10>; > > name = "vas"; > > + interrupts = <0x1f 0>; > > + interrupt-parent = <>; > > ibm,vas-id = <0x1>; > > + ibm,vas-port = <0x601000100>; > > }; > > -- > > 1.8.3.1 > > > > > >
Re: [PATCH 04/14] powerpc/vas: Setup IRQ mapping and register port for each window
On Wed, 2019-12-18 at 18:18 +1100, Oliver O'Halloran wrote:
> On Wed, Nov 27, 2019 at 12:07 PM Haren Myneni
> wrote:
> >
> > *snip*
> >
> > @@ -36,7 +62,18 @@ static int init_vas_instance(struct platform_device
> > *pdev)
> > return -ENODEV;
> > }
> >
> > - if (pdev->num_resources != 4) {
> > + rc = of_property_read_u64(dn, "ibm,vas-port", );
> > + if (rc) {
> > + pr_err("No ibm,vas-port property for %s?\n", pdev->name);
> > + /* No interrupts property */
> > + nresources = 4;
> > + }
> > +
> > + /*
> > +* interrupts property is available with 'ibm,vas-port' property.
> > +* 4 Resources and 1 IRQ if interrupts property is available.
> > +*/
> > + if (pdev->num_resources != nresources) {
> > pr_err("Unexpected DT configuration for [%s, %d]\n",
> > pdev->name, vasid);
> > return -ENODEV;
>
> Right, so adding the IRQ in firmware will break the VAS driver in
> existing kernels since it changes the resource count. This is IMO a
> bug in the VAS driver that you should fix, but it does mean we need to
> think twice about having firmware assign an interrupt at boot.
Correct, Hence added vas-user-space nvram switch in skiboot.
>
> I had a closer look at this series and I'm not convinced that any
> firmware changes are actually required either. We already have OPAL
> calls for allocating an hwirq for the kernel to use and for getting
> the IRQ's XIVE trigger port (see pnv_ocxl_alloc_xive_irq() for an
> example). Why not use those here too? Doing so would allow us to
> assign interrupts to individual windows too which might be useful for
> the windows used by the kernel.
Thanks for the pointer. like using pnv_ocxl_alloc_xive_irq(), we can
disregard FW change. BTW, VAS fault handling is needed only for user
space VAS windows.
int vas_alloc_xive_irq(u32 chipid, u32 *irq, u64 *trigger_addr)
{
__be64 flags, trigger_page;
u32 hwirq;
s64 rc;
hwirq = opal_xive_allocate_irq_raw(chipid);
if (hwirq < 0)
return -ENOENT;
rc = opal_xive_get_irq_info(hwirq, , NULL, _page,
NULL,
NULL);
if (rc || !trigger_page) {
xive_native_free_irq(hwirq);
return -ENOENT;
}
*irq = hwirq;
*trigger_addr = be64_to_cpu(trigger_page);
return 0;
}
We can have common function for VAS and cxl except per chip IRQ
allocation is needed for each VAS instance. I will post patch-set with
this change.
Thanks
Haren
[PATCH V3 13/13] powerpc/vas: Free send window in VAS instance after credits returned
NX may be processing requests while trying to close window. Wait until all credits are returned and then free send window from VAS instance. Signed-off-by: Haren Myneni --- arch/powerpc/platforms/powernv/vas-window.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-window.c b/arch/powerpc/platforms/powernv/vas-window.c index e6ea7d3..aded7a5 100644 --- a/arch/powerpc/platforms/powernv/vas-window.c +++ b/arch/powerpc/platforms/powernv/vas-window.c @@ -1316,14 +1316,14 @@ int vas_win_close(struct vas_window *window) unmap_paste_region(window); - clear_vinst_win(window); - poll_window_busy_state(window); unpin_close_window(window); poll_window_credits(window); + clear_vinst_win(window); + poll_window_castout(window); /* if send window, drop reference to matching receive window */ -- 1.8.3.1
[PATCH V3 12/13] powerpc/vas: Display process stuck message
Process can not close send window until all requests are processed.
Means wait until window state is not busy and send credits are
returned. Display debug message in case taking longer to close the
window.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 26 ++
1 file changed, 26 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8d90d25..e6ea7d3 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1183,6 +1183,7 @@ static void poll_window_credits(struct vas_window *window)
{
u64 val;
int creds, mode;
+ int count = 0;
val = read_hvwc_reg(window, VREG(WINCTL));
if (window->tx_win)
@@ -1201,10 +1202,25 @@ static void poll_window_credits(struct vas_window
*window)
creds = GET_FIELD(VAS_LRX_WCRED, val);
}
+ /*
+* Takes around few microseconds to complete all pending requests
+* and return credits.
+* TODO: Issue CRB Kill to stop all pending requests. Need only
+* if there is a bug in NX or fault handling in kernel.
+*/
if (creds < window->wcreds_max) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(10));
+ count++;
+ /*
+* Process can not close send window until all credits are
+* returned.
+*/
+ if (!(count % 1))
+ pr_debug("%s() pid %d stuck? retries %d\n", __func__,
+ vas_window_pid(window), count);
+
goto retry;
}
}
@@ -1218,6 +1234,7 @@ static void poll_window_busy_state(struct vas_window
*window)
{
int busy;
u64 val;
+ int count = 0;
retry:
val = read_hvwc_reg(window, VREG(WIN_STATUS));
@@ -1226,6 +1243,15 @@ static void poll_window_busy_state(struct vas_window
*window)
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
schedule_timeout(msecs_to_jiffies(5));
+ count++;
+ /*
+* Takes around 5 microseconds to process all pending
+* requests.
+*/
+ if (!(count % 1))
+ pr_debug("%s() pid %d stuck? retries %d\n", __func__,
+ vas_window_pid(window), count);
+
goto retry;
}
}
--
1.8.3.1
[PATCH V3 11/13] powerpc/VAS: Return credits after handling fault
NX expects OS to return credit for send window after processing each
fault. Also credit has to be returned even for fault window.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 10 ++
arch/powerpc/platforms/powernv/vas-window.c | 17 +
arch/powerpc/platforms/powernv/vas.h| 1 +
3 files changed, 28 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 6e8b8c7..eadfee7 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -238,6 +238,11 @@ irqreturn_t vas_fault_handler(int irq, void *data)
memset(fifo, 0, CRB_SIZE);
mutex_unlock(>mutex);
+ /*
+* Return credit for the fault window.
+*/
+ vas_return_credit(vinst->fault_win, 0);
+
pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
@@ -264,6 +269,11 @@ irqreturn_t vas_fault_handler(int irq, void *data)
}
update_csb(window, crb);
+ /*
+* Return credit for send window after processing
+* fault CRB.
+*/
+ vas_return_credit(window, 1);
} while (true);
return IRQ_HANDLED;
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8428970..8d90d25 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1319,6 +1319,23 @@ int vas_win_close(struct vas_window *window)
}
EXPORT_SYMBOL_GPL(vas_win_close);
+/*
+ * Return credit for the given window.
+ */
+void vas_return_credit(struct vas_window *window, bool tx)
+{
+ uint64_t val;
+
+ val = 0ULL;
+ if (tx) { /* send window */
+ val = SET_FIELD(VAS_TX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), val);
+ } else {
+ val = SET_FIELD(VAS_LRX_WCRED, val, 1);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), val);
+ }
+}
+
struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid)
{
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index f5f45ea..495937a 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -415,6 +415,7 @@ struct vas_winctx {
extern void vas_window_free_dbgdir(struct vas_window *win);
extern int vas_setup_fault_window(struct vas_instance *vinst);
extern irqreturn_t vas_fault_handler(int irq, void *data);
+extern void vas_return_credit(struct vas_window *window, bool tx);
extern struct vas_window *vas_pswid_to_window(struct vas_instance *vinst,
uint32_t pswid);
--
1.8.3.1
[PATCH V3 10/13] powerpc/vas: Do not use default credits for receive window
System checkstops if RxFIFO overruns with more requests than the
maximum possible number of CRBs allowed in FIFO at any time. So
max credits value (rxattr.wcreds_max) is set and is passed to
vas_rx_win_open() by the the driver.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 4 ++--
arch/powerpc/platforms/powernv/vas.h| 2 --
2 files changed, 2 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0f27ac5..8428970 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -772,7 +772,7 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
- if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ if (!attr->wcreds_max)
return false;
if (attr->nx_win) {
@@ -878,7 +878,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
- rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ rxwin->wcreds_max = rxattr->wcreds_max;
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index af03aa0..f5f45ea 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -101,11 +101,9 @@
/*
* Initial per-process credits.
* Max send window credits:4K-1 (12-bits in VAS_TX_WCRED)
- * Max receive window credits: 64K-1 (16 bits in VAS_LRX_WCRED)
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
--
1.8.3.1
[PATCH V3 09/13] powerpc/vas: Print CRB and FIFO values
Dump FIFO entry values if could not find send window and print CRB
for debugging.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 41 ++
1 file changed, 41 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 45bea15c..6e8b8c7 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -26,6 +26,28 @@
*/
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+static void dump_crb(struct coprocessor_request_block *crb)
+{
+ struct data_descriptor_entry *dde;
+ struct nx_fault_stamp *nx;
+
+ dde = >source;
+ pr_devel("SrcDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ dde = >target;
+ pr_devel("TgtDDE: addr 0x%llx, len %d, count %d, idx %d, flags %d\n",
+ be64_to_cpu(dde->address), be32_to_cpu(dde->length),
+ dde->count, dde->index, dde->flags);
+
+ nx = >stamp.nx;
+ pr_devel("NX Stamp: PSWID 0x%x, FSA 0x%llx, flags 0x%x, FS 0x%x\n",
+ be32_to_cpu(nx->pswid),
+ be64_to_cpu(crb->stamp.nx.fault_storage_addr),
+ nx->flags, be32_to_cpu(nx->fault_status));
+}
+
/*
* Update the CSB to indicate a translation error.
*
@@ -145,6 +167,23 @@ static void update_csb(struct vas_window *window,
pid_vnr(pid), rc);
}
+static void dump_fifo(struct vas_instance *vinst, void *entry)
+{
+ int i;
+ unsigned long *fifo = entry;
+
+ pr_err("Fault fifo size %d, max crbs %d, crb size %lu\n",
+ vinst->fault_fifo_size,
+ vinst->fault_fifo_size / CRB_SIZE,
+ sizeof(struct coprocessor_request_block));
+
+ pr_err("Fault FIFO Entry Dump:\n");
+ for (i = 0; i < CRB_SIZE; i += 4, fifo += 4) {
+ pr_err("[%.3d, %p]: 0x%.16lx 0x%.16lx 0x%.16lx 0x%.16lx\n",
+ i, fifo, *fifo, *(fifo+1), *(fifo+2), *(fifo+3));
+ }
+}
+
/*
* Process CRBs that we receive on the fault window.
*/
@@ -203,6 +242,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
vinst->vas_id, vinst->fault_fifo, fifo,
vinst->fault_crbs);
+ dump_crb(crb);
window = vas_pswid_to_window(vinst,
be32_to_cpu(crb->stamp.nx.pswid));
@@ -213,6 +253,7 @@ irqreturn_t vas_fault_handler(int irq, void *data)
* even clean it up (return credit).
* But we should not get here.
*/
+ dump_fifo(vinst, (void *)crb);
pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
vinst->vas_id, vinst->fault_fifo, fifo,
be32_to_cpu(crb->stamp.nx.pswid),
--
1.8.3.1
[PATCH V3 08/13] powerpc/vas: Update CSB and notify process for fault CRBs
For each fault CRB, update fault address in CRB (fault_storage_addr)
and translation error status in CSB so that user space can touch the
fault address and resend the request. If the user space passed invalid
CSB address send signal to process with SIGSEGV.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 121 +
1 file changed, 121 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index 57f21ea..45bea15c 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include
@@ -26,6 +27,125 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Update the CSB to indicate a translation error.
+ *
+ * If the fault is in the CSB address itself or if we are unable to
+ * update the CSB, send a signal to the process, because we have no
+ * other way of notifying the user process.
+ *
+ * Remaining settings in the CSB are based on wait_for_csb() of
+ * NX-GZIP.
+ */
+static void update_csb(struct vas_window *window,
+ struct coprocessor_request_block *crb)
+{
+ int rc;
+ struct pid *pid;
+ void __user *csb_addr;
+ struct task_struct *tsk;
+ struct kernel_siginfo info;
+ struct coprocessor_status_block csb;
+
+ /*
+* NX user space windows can not be opened for task->mm=NULL
+* and faults will not be generated for kernel requests.
+*/
+ if (!window->mm || !window->user_win)
+ return;
+
+ csb_addr = (void *)be64_to_cpu(crb->csb_addr);
+
+ csb.cc = CSB_CC_TRANSLATION;
+ csb.ce = CSB_CE_TERMINATION;
+ csb.cs = 0;
+ csb.count = 0;
+
+ /*
+* Returns the fault address in CPU format since it is passed with
+* signal. But if the user space expects BE format, need changes.
+* i.e either kernel (here) or user should convert to CPU format.
+* Not both!
+*/
+ csb.address = be64_to_cpu(crb->stamp.nx.fault_storage_addr);
+ csb.flags = 0;
+
+ use_mm(window->mm);
+ rc = copy_to_user(csb_addr, , sizeof(csb));
+ /*
+* User space polls on csb.flags (first byte). So add barrier
+* then copy first byte with csb flags update.
+*/
+ smp_mb();
+ if (!rc) {
+ csb.flags = CSB_V;
+ rc = copy_to_user(csb_addr, , sizeof(u8));
+ }
+ unuse_mm(window->mm);
+
+ /* Success */
+ if (!rc)
+ return;
+
+ /*
+* User space passed invalid CSB address, Notify process with
+* SEGV signal.
+*/
+ pid = window->pid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Send window will be closed after processing all NX requests
+* and process exits after closing all windows. In multi-thread
+* applications, thread may not exists, but does not close FD
+* (means send window) upon exit. Parent thread (tgid) can use
+* and close the window later.
+* pid and mm references are taken when window is opened by
+* process (pid). So tgid is used only when child thread is not
+* available in multithread tasks.
+*
+*/
+ if (!tsk) {
+ pid = window->tgid;
+ tsk = get_pid_task(pid, PIDTYPE_PID);
+ /*
+* Parent thread will be closing window during its exit.
+* So should not get here.
+*/
+ if (!tsk)
+ return;
+ }
+
+ /* Do not notify if the task is exiting. */
+ if (tsk->flags & PF_EXITING) {
+ put_task_struct(tsk);
+ return;
+ }
+ put_task_struct(tsk);
+
+ pr_err("Invalid CSB address 0x%p signalling pid(%d)\n",
+ csb_addr, pid_vnr(pid));
+
+ clear_siginfo();
+ info.si_signo = SIGSEGV;
+ info.si_errno = EFAULT;
+ info.si_code = SEGV_MAPERR;
+ info.si_addr = csb_addr;
+
+ /*
+* process will be polling on csb.flags after request is sent to
+* NX. So generally CSB update should not fail except when an
+* application does not follow the process properly. So an error
+* message will be displayed and leave it to user space whether
+* to ignore or handle this signal.
+*/
+ rcu_read_lock();
+ rc = kill_pid_info(SIGSEGV, , pid);
+ rcu_read_unlock();
+
+ pr_devel("%s(): pid %d kill_proc_info() rc %d\n", __func__,
+ pid_vnr(pid), rc);
+}
+
+/*
* Process CRBs that we receive on the fault window.
*/
irqreturn_t vas_fault_handler(int irq, void *data)
@@ -102,6
[PATCH V3 07/13] powerpc/vas: Take reference to PID and mm for userspace windows
Process close windows after its requests are completed. In multi-thread
applications, child can open a window but release FD will not be called
upon its exit. Parent thread will be closing it later upon its exit.
The parent can also send NX requests with this window and NX can
generate page faults. After kernel handles the page fault, send
signal to process by using PID if CSB address is invalid. Parent
thread will not receive signal since its PID is different from the one
saved in vas_window. So use tgid in case if the task for the pid saved
in window is not running and send signal to its parent.
To prevent reusing the pid until the window closed, take reference to
pid and task mm.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-debug.c | 2 +-
arch/powerpc/platforms/powernv/vas-window.c | 53 ++---
arch/powerpc/platforms/powernv/vas.h| 9 -
3 files changed, 57 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index 09e63df..ef9a717 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -38,7 +38,7 @@ static int info_show(struct seq_file *s, void *private)
seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
window->tx_win ? "Send" : "Receive");
- seq_printf(s, "Pid : %d\n", window->pid);
+ seq_printf(s, "Pid : %d\n", vas_window_pid(window));
unlock:
mutex_unlock(_mutex);
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index e36c5d2..0f27ac5 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -12,6 +12,8 @@
#include
#include
#include
+#include
+#include
#include
#include
#include "vas.h"
@@ -877,8 +879,6 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
- if (rxattr->user_win)
- rxwin->pid = task_pid_vnr(current);
init_winctx_for_rxwin(rxwin, rxattr, );
init_winctx_regs(rxwin, );
@@ -1028,7 +1028,6 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->tx_win = 1;
txwin->rxwin = rxwin;
txwin->nx_win = txwin->rxwin->nx_win;
- txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
@@ -1069,8 +1068,43 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
goto free_window;
}
- set_vinst_win(vinst, txwin);
+ if (txwin->user_win) {
+ /*
+* Window opened by child thread may not be closed when
+* it exits. So take reference to its pid and release it
+* when the window is free by parent thread.
+* Acquire a reference to the task's pid to make sure
+* pid will not be re-used - needed only for multithread
+* applications.
+*/
+ txwin->pid = get_task_pid(current, PIDTYPE_PID);
+ /*
+* Acquire a reference to the task's mm.
+*/
+ txwin->mm = get_task_mm(current);
+ if (!txwin->mm) {
+ put_pid(txwin->pid);
+ pr_err("VAS: pid(%d): mm_struct is not found\n",
+ current->pid);
+ rc = -EPERM;
+ goto free_window;
+ }
+
+ mmgrab(txwin->mm);
+ mmput(txwin->mm);
+ mm_context_add_copro(txwin->mm);
+ /*
+* Process closes window during exit. In the case of
+* multithread application, child can open window and
+* can exit without closing it. Expects parent thread
+* to use and close the window. So do not need to take
+* pid reference for parent thread.
+*/
+ txwin->tgid = find_get_pid(task_tgid_vnr(current));
+ }
+
+ set_vinst_win(vinst, txwin);
return txwin;
free_window:
@@ -1267,8 +1301,17 @@ int vas_win_close(struct vas_window *window)
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
- if (window->tx_win)
+ if (window->tx_win) {
+ if (window->user_win) {
+ /* Drop references to pid and mm */
+ put_pid(wi
[PATCH V3 06/13] powerpc/vas: Register NX with fault window ID and IRQ port value
For each user space send window, register NX with fault window ID
and port value so that NX paste CRBs in this fault FIFO when it
sees fault on the request buffer.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-window.c | 15 +--
arch/powerpc/platforms/powernv/vas.h| 15 +++
2 files changed, 28 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index cec1b41..e36c5d2 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -373,7 +373,7 @@ int init_winctx_regs(struct vas_window *window, struct
vas_winctx *winctx)
init_xlate_regs(window, winctx->user_win);
val = 0ULL;
- val = SET_FIELD(VAS_FAULT_TX_WIN, val, 0);
+ val = SET_FIELD(VAS_FAULT_TX_WIN, val, winctx->fault_win_id);
write_hvwc_reg(window, VREG(FAULT_TX_WIN), val);
/* In PowerNV, interrupts go to HV. */
@@ -748,6 +748,8 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (rxwin->vinst->virq)
+ winctx->irq_port = rxwin->vinst->irq_port;
}
static bool rx_win_args_valid(enum vas_cop_type cop,
@@ -945,13 +947,22 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
winctx->rx_win_id = txwin->rxwin->winid;
+ /*
+* IRQ and fault window setup is successful. Set fault window
+* for the send window so that ready to handle faults.
+*/
+ if (txwin->vinst->virq)
+ winctx->fault_win_id = txwin->vinst->fault_win->winid;
winctx->dma_type = VAS_DMA_TYPE_INJECT;
winctx->tc_mode = txattr->tc_mode;
winctx->min_scope = VAS_SCOPE_LOCAL;
winctx->max_scope = VAS_SCOPE_VECTORED_GROUP;
+ if (txwin->vinst->virq)
+ winctx->irq_port = txwin->vinst->irq_port;
- winctx->pswid = 0;
+ winctx->pswid = txattr->pswid ? txattr->pswid :
+ encode_pswid(txwin->vinst->vas_id, txwin->winid);
}
static bool tx_win_args_valid(enum vas_cop_type cop,
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 879f5b4..2621df1 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -455,6 +455,21 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ return ((u32)winid | (vasid << (31 - 7)));
+}
+
static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
{
if (vasid)
--
1.8.3.1
[PATCH V3 05/13] powerpc/vas: Setup thread IRQ handler per VAS instance
Setup thread IRQ handler per each VAS instance. When NX sees a fault
on CRB, kernel gets an interrupt and vas_fault_handler will be
executed to process fault CRBs. Read all valid CRBs from fault FIFO,
determine the corresponding send window from CRB and process fault
requests.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas-fault.c | 83 +
arch/powerpc/platforms/powernv/vas-window.c | 60 +
arch/powerpc/platforms/powernv/vas.c| 21 +++-
arch/powerpc/platforms/powernv/vas.h| 4 ++
4 files changed, 167 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
index b0258ed..57f21ea 100644
--- a/arch/powerpc/platforms/powernv/vas-fault.c
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -11,6 +11,7 @@
#include
#include
#include
+#include
#include
#include "vas.h"
@@ -25,6 +26,88 @@
#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
/*
+ * Process CRBs that we receive on the fault window.
+ */
+irqreturn_t vas_fault_handler(int irq, void *data)
+{
+ struct vas_instance *vinst = data;
+ struct coprocessor_request_block buf, *crb;
+ struct vas_window *window;
+ void *fifo;
+
+ /*
+* VAS can interrupt with multiple page faults. So process all
+* valid CRBs within fault FIFO until reaches invalid CRB.
+* NX updates nx_fault_stamp in CRB and pastes in fault FIFO.
+* kernel retrives send window from parition send window ID
+* (pswid) in nx_fault_stamp. So pswid should be non-zero and
+* use this to check whether CRB is valid.
+* After reading CRB entry, it is reset with 0's in fault FIFO.
+*
+* In case kernel receives another interrupt with different page
+* fault and CRBs are processed by the previous handling, will be
+* returned from this function when it sees invalid CRB (means 0's).
+*/
+ do {
+ mutex_lock(>mutex);
+
+ /*
+* Advance the fault fifo pointer to next CRB.
+* Use CRB_SIZE rather than sizeof(*crb) since the latter is
+* aligned to CRB_ALIGN (256) but the CRB written to by VAS is
+* only CRB_SIZE in len.
+*/
+ fifo = vinst->fault_fifo + (vinst->fault_crbs * CRB_SIZE);
+ crb = fifo;
+
+ /*
+* pswid returned from NX will be in _be32, but just
+* checking non-zero value to make sure the CRB is valid.
+* Return if reached invalid CRB.
+*/
+ if (!crb->stamp.nx.pswid) {
+ mutex_unlock(>mutex);
+ return IRQ_HANDLED;
+ }
+
+ vinst->fault_crbs++;
+ if (vinst->fault_crbs == (vinst->fault_fifo_size / CRB_SIZE))
+ vinst->fault_crbs = 0;
+
+ crb =
+ memcpy(crb, fifo, CRB_SIZE);
+ memset(fifo, 0, CRB_SIZE);
+ mutex_unlock(>mutex);
+
+ pr_devel("VAS[%d] fault_fifo %p, fifo %p, fault_crbs %d\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ vinst->fault_crbs);
+
+ window = vas_pswid_to_window(vinst,
+ be32_to_cpu(crb->stamp.nx.pswid));
+
+ if (IS_ERR(window)) {
+ /*
+* We got an interrupt about a specific send
+* window but we can't find that window and we can't
+* even clean it up (return credit).
+* But we should not get here.
+*/
+ pr_err("VAS[%d] fault_fifo %p, fifo %p, pswid 0x%x,
fault_crbs %d bad CRB?\n",
+ vinst->vas_id, vinst->fault_fifo, fifo,
+ be32_to_cpu(crb->stamp.nx.pswid),
+ vinst->fault_crbs);
+
+ WARN_ON_ONCE(1);
+ return IRQ_HANDLED;
+ }
+
+ } while (true);
+
+ return IRQ_HANDLED;
+}
+
+/*
* Fault window is opened per VAS instance. NX pastes fault CRB in fault
* FIFO upon page faults.
*/
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index f07f49a..cec1b41 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1041,6 +1041,15 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
}
} else {
/*
+* Interrupt hanlder or fault
[PATCH V3 04/13] powerpc/vas: Setup fault window per VAS instance
Setup fault window for each VAS instance. When NX gets fault on request
buffer, write fault CRBs in the corresponding fault FIFO and then sends
an interrupt to the OS.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/Makefile | 2 +-
arch/powerpc/platforms/powernv/vas-fault.c | 73
arch/powerpc/platforms/powernv/vas-window.c | 3 +-
arch/powerpc/platforms/powernv/vas.c| 20
arch/powerpc/platforms/powernv/vas.h| 5 ++
scripts/pnmtologo | Bin 0 -> 71976 bytes
6 files changed, 101 insertions(+), 2 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c
create mode 100755 scripts/pnmtologo
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index c0f8120..395789f 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -17,7 +17,7 @@ obj-$(CONFIG_MEMORY_FAILURE) += opal-memory-errors.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o vas-fault.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
obj-$(CONFIG_SCOM_DEBUGFS) += opal-xscom.o
obj-$(CONFIG_PPC_SECURE_BOOT) += opal-secvar.o
diff --git a/arch/powerpc/platforms/powernv/vas-fault.c
b/arch/powerpc/platforms/powernv/vas-fault.c
new file mode 100644
index 000..b0258ed
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-fault.c
@@ -0,0 +1,73 @@
+// SPDX-License-Identifier: GPL-2.0+
+/*
+ * VAS Fault handling.
+ * Copyright 2019, IBM Corporation
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include
+#include
+
+#include "vas.h"
+
+/*
+ * The maximum FIFO size for fault window can be 8MB
+ * (VAS_RX_FIFO_SIZE_MAX). Using 4MB FIFO since each VAS
+ * instance will be having fault window.
+ * 8MB FIFO can be used if expects more faults for each VAS
+ * instance.
+ */
+#define VAS_FAULT_WIN_FIFO_SIZE(4 << 20)
+
+/*
+ * Fault window is opened per VAS instance. NX pastes fault CRB in fault
+ * FIFO upon page faults.
+ */
+int vas_setup_fault_window(struct vas_instance *vinst)
+{
+ struct vas_rx_win_attr attr;
+
+ vinst->fault_fifo_size = VAS_FAULT_WIN_FIFO_SIZE;
+ vinst->fault_fifo = kzalloc(vinst->fault_fifo_size, GFP_KERNEL);
+ if (!vinst->fault_fifo) {
+ pr_err("Unable to alloc %d bytes for fault_fifo\n",
+ vinst->fault_fifo_size);
+ return -ENOMEM;
+ }
+
+ vas_init_rx_win_attr(, VAS_COP_TYPE_FAULT);
+
+ attr.rx_fifo_size = vinst->fault_fifo_size;
+ attr.rx_fifo = vinst->fault_fifo;
+
+ /*
+* Max creds is based on number of CRBs can fit in the FIFO.
+* (fault_fifo_size/CRB_SIZE). If 8MB FIFO is used, max creds
+* will be 0x since the receive creds field is 16bits wide.
+*/
+ attr.wcreds_max = vinst->fault_fifo_size / CRB_SIZE;
+ attr.lnotify_lpid = 0;
+ attr.lnotify_pid = mfspr(SPRN_PID);
+ attr.lnotify_tid = mfspr(SPRN_PID);
+
+ vinst->fault_win = vas_rx_win_open(vinst->vas_id, VAS_COP_TYPE_FAULT,
+ );
+
+ if (IS_ERR(vinst->fault_win)) {
+ pr_err("VAS: Error %ld opening FaultWin\n",
+ PTR_ERR(vinst->fault_win));
+ kfree(vinst->fault_fifo);
+ return PTR_ERR(vinst->fault_win);
+ }
+
+ pr_devel("VAS: Created FaultWin %d, LPID/PID/TID [%d/%d/%d]\n",
+ vinst->fault_win->winid, attr.lnotify_lpid,
+ attr.lnotify_pid, attr.lnotify_tid);
+
+ return 0;
+}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 0c0d27d..f07f49a 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -827,9 +827,10 @@ void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
enum vas_cop_type cop)
rxattr->fault_win = true;
rxattr->notify_disable = true;
rxattr->rx_wcred_mode = true;
- rxattr->tx_wcred_mode = true;
rxattr->rx_win_ord_mode = true;
rxattr->tx_win_ord_mode = true;
+ rxattr->rej_no_credit = true;
+ rxattr->tc_mode = VAS_THRESH_DISABLED;
} else if (cop == VAS_COP_TYPE_FTW) {
rxattr->user_win = true;
rxattr->intr_disable = true;
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index 40d8213..850863c
[PATCH V3 03/13] powerpc/vas: Read interrupts and vas-port device tree properties
Read interrupts and vas-port device tree properties per each VAS
instance. NX generates an interrupt when it sees page fault on the
request buffer. Interrupts property is used to setup IRQ for handing
the fault and set port value for each user space send window.
Signed-off-by: Haren Myneni
---
arch/powerpc/platforms/powernv/vas.c | 40
arch/powerpc/platforms/powernv/vas.h | 2 ++
2 files changed, 34 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index ed9cc6d..40d8213 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -25,10 +25,11 @@
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, cpu, vasid;
- struct resource *res;
- struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
+ int rc, cpu, vasid, nresources = 5;
+ struct vas_instance *vinst;
+ struct resource *res;
+ uint64_t port;
rc = of_property_read_u32(dn, "ibm,vas-id", );
if (rc) {
@@ -36,7 +37,18 @@ static int init_vas_instance(struct platform_device *pdev)
return -ENODEV;
}
- if (pdev->num_resources != 4) {
+ rc = of_property_read_u64(dn, "ibm,vas-port", );
+ if (rc) {
+ pr_err("No ibm,vas-port property for %s?\n", pdev->name);
+ /* No interrupts property */
+ nresources = 4;
+ }
+
+ /*
+* interrupts property is available with 'ibm,vas-port' property.
+* 4 Resources and 1 IRQ if interrupts property is available.
+*/
+ if (pdev->num_resources != nresources) {
pr_err("Unexpected DT configuration for [%s, %d]\n",
pdev->name, vasid);
return -ENODEV;
@@ -51,6 +63,7 @@ static int init_vas_instance(struct platform_device *pdev)
mutex_init(>mutex);
vinst->vas_id = vasid;
vinst->pdev = pdev;
+ vinst->irq_port = port;
res = >resource[0];
vinst->hvwc_bar_start = res->start;
@@ -66,12 +79,23 @@ static int init_vas_instance(struct platform_device *pdev)
pr_err("Bad 'paste_win_id_shift' in DT, %llx\n", res->end);
goto free_vinst;
}
-
vinst->paste_win_id_shift = 63 - res->end;
- pr_devel("Initialized instance [%s, %d], paste_base 0x%llx, "
- "paste_win_id_shift 0x%llx\n", pdev->name, vasid,
- vinst->paste_base_addr, vinst->paste_win_id_shift);
+ /* interrupts property */
+ if (pdev->num_resources == 5) {
+ res = >resource[4];
+ vinst->virq = res->start;
+ if (vinst->virq <= 0) {
+ pr_err("IRQ resource is not available for [%s, %d]\n",
+ pdev->name, vasid);
+ vinst->virq = 0;
+ }
+ }
+
+ pr_devel("Initialized instance [%s, %d] paste_base 0x%llx
paste_win_id_shift 0x%llx IRQ %d Port 0x%llx\n",
+ pdev->name, vasid, vinst->paste_base_addr,
+ vinst->paste_win_id_shift, vinst->virq,
+ vinst->irq_port);
for_each_possible_cpu(cpu) {
if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 5574aec..598608b 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -313,6 +313,8 @@ struct vas_instance {
u64 paste_base_addr;
u64 paste_win_id_shift;
+ u64 irq_port;
+ int virq;
struct mutex mutex;
struct vas_window *rxwin[VAS_COP_TYPE_MAX];
struct vas_window *windows[VAS_WINDOWS_PER_CHIP];
--
1.8.3.1
[PATCH V3 02/13] powerpc/vas: Define nx_fault_stamp in coprocessor_request_block
Kernel sets fault address and status in CRB for NX page fault on user
space address after processing page fault. User space gets the signal
and handles the fault mentioned in CRB by bringing the page in to
memory and send NX request again.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Haren Myneni
---
arch/powerpc/include/asm/icswx.h | 18 +-
1 file changed, 17 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/icswx.h b/arch/powerpc/include/asm/icswx.h
index 9872f85..b233d1e 100644
--- a/arch/powerpc/include/asm/icswx.h
+++ b/arch/powerpc/include/asm/icswx.h
@@ -108,6 +108,17 @@ struct data_descriptor_entry {
__be64 address;
} __packed __aligned(DDE_ALIGN);
+/* 4.3.2 NX-stamped Fault CRB */
+
+#define NX_STAMP_ALIGN (0x10)
+
+struct nx_fault_stamp {
+ __be64 fault_storage_addr;
+ __be16 reserved;
+ __u8 flags;
+ __u8 fault_status;
+ __be32 pswid;
+} __packed __aligned(NX_STAMP_ALIGN);
/* Chapter 6.5.2 Coprocessor-Request Block (CRB) */
@@ -135,7 +146,12 @@ struct coprocessor_request_block {
struct coprocessor_completion_block ccb;
- u8 reserved[48];
+ union {
+ struct nx_fault_stamp nx;
+ u8 reserved[16];
+ } stamp;
+
+ u8 reserved[32];
struct coprocessor_status_block csb;
} __packed __aligned(CRB_ALIGN);
--
1.8.3.1
[PATCH V3 01/13] powerpc/vas: Describe vas-port and interrupts properties
Signed-off-by: Haren Myneni --- Documentation/devicetree/bindings/powerpc/ibm,vas.txt | 5 + 1 file changed, 5 insertions(+) diff --git a/Documentation/devicetree/bindings/powerpc/ibm,vas.txt b/Documentation/devicetree/bindings/powerpc/ibm,vas.txt index bf11d2f..12de08b 100644 --- a/Documentation/devicetree/bindings/powerpc/ibm,vas.txt +++ b/Documentation/devicetree/bindings/powerpc/ibm,vas.txt @@ -11,6 +11,8 @@ Required properties: window context start and length, OS/User window context start and length, "Paste address" start and length, "Paste window id" start bit and number of bits) +- ibm,vas-port : Port address for the interrupt. +- interrupts: IRQ value for each VAS instance and level. Example: @@ -18,5 +20,8 @@ Example: compatible = "ibm,vas", "ibm,power9-vas"; reg = <0x60191 0x200 0x60190 0x1 0x8 0x1 0x20 0x10>; name = "vas"; + interrupts = <0x1f 0>; + interrupt-parent = <>; ibm,vas-id = <0x1>; + ibm,vas-port = <0x601000100>; }; -- 1.8.3.1
[PATCH V3 00/13] powerpc/vas: Page fault handling for user space NX requests
On power9, Virtual Accelerator Switchboard (VAS) allows user space or kernel to communicate with Nest Accelerator (NX) directly using COPY/PASTE instructions. NX provides verious functionalities such as compression, encryption and etc. But only compression (842 and GZIP formats) is supported in Linux kernel on power9. 842 compression driver (drivers/crypto/nx/nx-842-powernv.c) is already included in Linux. Only GZIP support will be available from user space. Applications can issue GZIP compression / decompression requests to NX with COPY/PASTE instructions. When NX is processing these requests, can hit fault on the request buffer (not in memory). It issues an interrupt and pastes fault CRB in fault FIFO. Expects kernel to handle this fault and return credits for both send and fault windows after processing. This patch series adds IRQ and fault window setup, and NX fault handling: - Read IRQ# from "interrupts" property and configure IRQ per VAS instance. - Set port# for each window to generate an interrupt when noticed fault. - Set fault window and FIFO on which NX paste fault CRB. - Setup IRQ thread fault handler per VAS instance. - When receiving an interrupt, Read CRBs from fault FIFO and update coprocessor_status_block (CSB) in the corresponding CRB with translation failure (CSB_CC_TRANSLATION). After issuing NX requests, process polls on CSB address. When it sees translation error, can touch the request buffer to bring the page in to memory and reissue NX request. - If copy_to_user fails on user space CSB address, OS sends SEGV signal. Tested these patches with NX-GZIP support and will be posting this series soon. Patch 2: Define nx_fault_stamp on which NX writes fault status for the fault CRB Patch 3: Read interrupts and port properties per VAS instance Patch 4: Setup fault window per each VAS instance. This window is used for NX to paste fault CRB in FIFO. Patches 5 & 6: Setup threaded IRQ per VAS and register NX with fault window ID and port number for each send window so that NX paste fault CRB in this window. Patch 7: Reference to pid and mm so that pid is not used until window closed. Needed for multi thread application where child can open a window and can be used by parent later. Patches 8 and 9: Process CRBs from fault FIFO and notify tasks by updating CSB or through signals. Patches 10 and 11: Return credits for send and fault windows after handling faults. Patch 13:Fix closing send window after all credits are returned. This issue happens only for user space requests. No page faults on kernel request buffer. Changelog: V2: - Use threaded IRQ instead of own kernel thread handler - Use pswid insted of user space CSB address to find valid CRB - Removed unused macros and other changes as suggested by Christoph Hellwig V3: - Rebased to 5.5-rc2 - Use struct pid * instead of pid_t for vas_window tgid - Code cleanup as suggested by Christoph Hellwig Haren Myneni (13): powerpc/vas: Describe vas-port and interrupts properties powerpc/vas: Define nx_fault_stamp in coprocessor_request_block powerpc/vas: Read interrupts and vas-port device tree properties powerpc/vas: Setup fault window per VAS instance powerpc/vas: Setup thread IRQ handler per VAS instance powerpc/vas: Register NX with fault window ID and IRQ port value powerpc/vas: Take reference to PID and mm for user space windows powerpc/vas: Update CSB and notify process for fault CRBs powerpc/vas: Print CRB and FIFO values powerpc/vas: Do not use default credits for receive window powerpc/VAS: Return credits after handling fault powerpc/vas: Display process stuck message powerpc/vas: Free send window in VAS instance after credits returned .../devicetree/bindings/powerpc/ibm,vas.txt| 5 + arch/powerpc/include/asm/icswx.h | 18 +- arch/powerpc/platforms/powernv/Makefile| 2 +- arch/powerpc/platforms/powernv/vas-debug.c | 2 +- arch/powerpc/platforms/powernv/vas-fault.c | 328 + arch/powerpc/platforms/powernv/vas-window.c| 182 +++- arch/powerpc/platforms/powernv/vas.c | 79 - arch/powerpc/platforms/powernv/vas.h | 38 ++- scripts/pnmtologo | Bin 0 -> 71976 bytes 9 files changed, 628 insertions(+), 26 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/vas-fault.c create mode 100755 scripts/pnmtologo -- 1.8.3.1
