Re: [PATCH] ibmvnic: remove excessive irqsave
angkery [angk...@163.com] wrote:
> From: Junlin Yang
>
> ibmvnic_remove locks multiple spinlocks while disabling interrupts:
> spin_lock_irqsave(&adapter->state_lock, flags);
> spin_lock_irqsave(&adapter->rwi_lock, flags);
>
> there is no need for the second irqsave,since interrupts are disabled
> at that point, so remove the second irqsave:
> spin_lock_irqsave(&adapter->state_lock, flags);
> spin_lock(&adapter->rwi_lock);
>
> Generated by: ./scripts/coccinelle/locks/flags.cocci
> ./drivers/net/ethernet/ibm/ibmvnic.c:5413:1-18:
> ERROR: nested lock+irqsave that reuses flags from line 5404.
>
Thanks. Please add
Fixes: 4a41c421f367 ("ibmvnic: serialize access to work queue on remove")
> Signed-off-by: Junlin Yang
Reviewed-by: Sukadev Bhattiprolu
Re: [PATCH] ibmvnic: Fix possibly uninitialized old_num_tx_queues variable warning.
Michal Suchanek [msucha...@suse.de] wrote:
> GCC 7.5 reports:
> ../drivers/net/ethernet/ibm/ibmvnic.c: In function 'ibmvnic_reset_init':
> ../drivers/net/ethernet/ibm/ibmvnic.c:5373:51: warning: 'old_num_tx_queues'
> may be used uninitialized in this function [-Wmaybe-uninitialized]
> ../drivers/net/ethernet/ibm/ibmvnic.c:5373:6: warning: 'old_num_rx_queues'
> may be used uninitialized in this function [-Wmaybe-uninitialized]
>
> The variable is initialized only if(reset) and used only if(reset &&
> something) so this is a false positive. However, there is no reason to
> not initialize the variables unconditionally avoiding the warning.
Yeah, its a false positive, but initializing doesn't hurt.
>
> Fixes: 635e442f4a48 ("ibmvnic: merge ibmvnic_reset_init and ibmvnic_init")
> Signed-off-by: Michal Suchanek
Reviewed-by: Sukadev Bhattiprolu
Re: linux-next: manual merge of the net-next tree with the net tree
Stephen Rothwell [s...@canb.auug.org.au] wrote:
> Hi all,
>
> Today's linux-next merge of the net-next tree got conflicts in:
>
> drivers/net/ethernet/ibm/ibmvnic.c
> drivers/net/ethernet/ibm/ibmvnic.h
>
> between commit:
>
> 4a41c421f367 ("ibmvnic: serialize access to work queue on remove")
>
> from the net tree and commits:
>
> bab08bedcdc3 ("ibmvnic: fix block comments")
> a369d96ca554 ("ibmvnic: add comments for spinlock_t definitions")
>
> from the net-next tree.
>
> I fixed it up (see below) and can carry the fix as necessary. This
> is now fixed as far as linux-next is concerned, but any non trivial
> conflicts should be mentioned to your upstream maintainer when your tree
> is submitted for merging. You may also want to consider cooperating
> with the maintainer of the conflicting tree to minimise any particularly
> complex conflicts.
The changes look good to me. Thanks.
Sukadev
Re: [PATCH v2 3/7] powerpc: use task_pid_nr() for TID allocation
Andrew Donnellan [andrew.donnel...@au1.ibm.com] wrote: > [+ Sukadev, Christophe] > > On 18/04/18 11:08, Alastair D'Silva wrote: > > From: Alastair D'Silva > > > > The current implementation of TID allocation, using a global IDR, may > > result in an errant process starving the system of available TIDs. > > Instead, use task_pid_nr(), as mentioned by the original author. The > > scenario described which prevented it's use is not applicable, as > > set_thread_tidr can only be called after the task struct has been > > populated. > > > > Signed-off-by: Alastair D'Silva > > So it's too late in the evening for me to completely get my head around > what's going on here enough to give my Reviewed-by:, but my current thinking > is: > > - In the first version of the patch to add TIDR support > (https://patchwork.ozlabs.org/patch/799494/), it was originally proposed to > call assign_thread_id() (as it was then called) from copy_thread() > > - The comment block documents the reason why we can't use task_pid_nr() but > assumes that we're trying to assign a TIDR from within copy_thread() > > - The final patch that was accepted > (https://patchwork.ozlabs.org/patch/835552/, > ec233ede4c8654894610ea54f4dae7adc954ac62) instead sets the TIDR to 0 from > copy_thread(), so the original reasoning regarding not using task_pid_nr() > within copy_thread() is no longer applicable. > > Sukadev: does this sound right? Yes. Like with PIDR, was trying to assign TIDR initially to all threads. But since only a subset of threads need/use TIDR, we can assign the value later (when set_thread_tidr() is called). So we should be able to use task_pid_nr() then. Sukadev
[tip:perf/core] perf vendor events: Update POWER9 events
Commit-ID: 9749adc3b2a23c91b2eda8758ff0c650d731aa2f
Gitweb: https://git.kernel.org/tip/9749adc3b2a23c91b2eda8758ff0c650d731aa2f
Author: Sukadev Bhattiprolu
AuthorDate: Tue, 13 Mar 2018 12:33:29 -0500
Committer: Arnaldo Carvalho de Melo
CommitDate: Fri, 16 Mar 2018 13:57:08 -0300
perf vendor events: Update POWER9 events
Cc: linuxppc-...@lists.ozlabs.org
Signed-off-by: Sukadev Bhattiprolu
Link: https://lkml.kernel.org/r/20180313224647.ga22...@us.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo
---
.../perf/pmu-events/arch/powerpc/power9/cache.json | 25 ---
.../pmu-events/arch/powerpc/power9/frontend.json | 10 -
.../pmu-events/arch/powerpc/power9/marked.json | 5 -
.../pmu-events/arch/powerpc/power9/memory.json | 5 -
.../perf/pmu-events/arch/powerpc/power9/other.json | 241 ++---
.../pmu-events/arch/powerpc/power9/pipeline.json | 50 ++---
tools/perf/pmu-events/arch/powerpc/power9/pmc.json | 5 -
.../arch/powerpc/power9/translation.json | 10 +-
8 files changed, 178 insertions(+), 173 deletions(-)
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/cache.json
b/tools/perf/pmu-events/arch/powerpc/power9/cache.json
index 7945c5196c43..851072105054 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/cache.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/cache.json
@@ -19,11 +19,6 @@
"EventName": "PM_CMPLU_STALL_FXU",
"BriefDescription": "Finish stall due to a scalar fixed point or CR
instruction in the execution pipeline. These instructions get routed to the
ALU, ALU2, and DIV pipes"
},
- {,
-"EventCode": "0x1D15C",
-"EventName": "PM_MRK_DTLB_MISS_1G",
-"BriefDescription": "Marked Data TLB reload (after a miss) page size 2M.
Implies radix translation was used"
- },
{,
"EventCode": "0x4D12A",
"EventName": "PM_MRK_DATA_FROM_RL4_CYC",
@@ -79,21 +74,6 @@
"EventName": "PM_THRESH_EXC_4096",
"BriefDescription": "Threshold counter exceed a count of 4096"
},
- {,
-"EventCode": "0x3D156",
-"EventName": "PM_MRK_DTLB_MISS_64K",
-"BriefDescription": "Marked Data TLB Miss page size 64K"
- },
- {,
-"EventCode": "0x4C15E",
-"EventName": "PM_MRK_DTLB_MISS_16M",
-"BriefDescription": "Marked Data TLB Miss page size 16M"
- },
- {,
-"EventCode": "0x2D15E",
-"EventName": "PM_MRK_DTLB_MISS_16G",
-"BriefDescription": "Marked Data TLB Miss page size 16G"
- },
{,
"EventCode": "0x3F14A",
"EventName": "PM_MRK_DPTEG_FROM_RMEM",
@@ -123,10 +103,5 @@
"EventCode": "0x1002A",
"EventName": "PM_CMPLU_STALL_LARX",
"BriefDescription": "Finish stall because the NTF instruction was a larx
waiting to be satisfied"
- },
- {,
-"EventCode": "0x1C058",
-"EventName": "PM_DTLB_MISS_16G",
-"BriefDescription": "Data TLB Miss page size 16G"
}
]
\ No newline at end of file
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
b/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
index bd8361b5fd6a..f9fa84b16fb5 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
@@ -154,11 +154,6 @@
"EventName": "PM_MRK_DATA_FROM_RL2L3_SHR_CYC",
"BriefDescription": "Duration in cycles to reload with Shared (S) data
from another chip's L2 or L3 on the same Node or Group (Remote), as this chip
due to a marked load"
},
- {,
-"EventCode": "0x3C056",
-"EventName": "PM_DTLB_MISS_64K",
-"BriefDescription": "Data TLB Miss page size 64K"
- },
{,
"EventCode": "0x30060",
"EventName": "PM_TM_TRANS_RUN_INST",
@@ -344,11 +339,6 @@
"EventName": "PM_MRK_LARX_FIN",
"BriefDescription": "Larx finished"
},
- {,
-"EventCode": "0x4C056",
-"EventName": "PM_DTLB_MISS_16M",
-"BriefDescription": "Data TLB Miss page size 16M"
- },
{,
"EventCode": "0x1003A",
"EventName": "PM_CMPLU_STALL_LSU_FIN",
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/marked.json
b/tools/perf/pmu-events/arch/powerpc/power9/marked.json
index 22f9f32060a8..b1954c38bab1 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/marked.jso
Re: [PATCH] perf vendor events: fix processing for xfs
John Garry [john.ga...@huawei.com] wrote:
> In the recently introduced support for vendor subdirectory,
> the checking for directory entries under xfs (or any other fs
> which does not support dirent.d_type) is missing the check
> for links for current and parent directory. This can result
> in a broken pmu_events.c being generated.
I run into the problem on a 4.15-based kernel/xfs but not on a
3.10-based kernel! Have not investigated, but this fix is good.
>
> Fix this by adding the appropriate check in is_leaf_dir().
>
> Cc: Sukadev Bhattiprolu
> Signed-off-by: John Garry
Tested-by: Sukadev Bhattiprolu
Thanks John.
Sukadev
>
> diff --git a/tools/perf/pmu-events/jevents.c b/tools/perf/pmu-events/jevents.c
> index 1c01844..db3a594 100644
> --- a/tools/perf/pmu-events/jevents.c
> +++ b/tools/perf/pmu-events/jevents.c
> @@ -873,7 +873,10 @@ static int is_leaf_dir(const char *fpath)
> return 0;
>
> while ((dir = readdir(d)) != NULL) {
> - if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
> + if (!strcmp(dir->d_name, ".") || !strcmp(dir->d_name, ".."))
> + continue;
> +
> + if (dir->d_type == DT_DIR) {
> res = 0;
> break;
> } else if (dir->d_type == DT_UNKNOWN) {
> --
> 1.9.1
[GIT PULL] Please pull JSON files for POWR9 PMU events
Hi Arnaldo, Please pull an update to the JSON files for POWER9 PMU events. The following changes since commit 90d2614c4d10c2f9d0ada9a3b01e5f43ca8d1ae3: perf test: Fix exit code for record+probe_libc_inet_pton.sh (2018-03-13 15:14:43 -0300) are available in the git repository at: https://github.com/sukadev/linux/ p9-json-v5 for you to fetch changes up to 99c9dff949f2502964005f9afa8d60c89b446f2c: perf vendor events: Update POWER9 events (2018-03-13 16:48:12 -0500) Sukadev Bhattiprolu (1): perf vendor events: Update POWER9 events .../perf/pmu-events/arch/powerpc/power9/cache.json | 25 --- .../pmu-events/arch/powerpc/power9/frontend.json | 10 - .../pmu-events/arch/powerpc/power9/marked.json | 5 - .../pmu-events/arch/powerpc/power9/memory.json | 5 - .../perf/pmu-events/arch/powerpc/power9/other.json | 241 ++--- .../pmu-events/arch/powerpc/power9/pipeline.json | 50 ++--- tools/perf/pmu-events/arch/powerpc/power9/pmc.json | 5 - .../arch/powerpc/power9/translation.json | 10 +- 8 files changed, 178 insertions(+), 173 deletions(-)
Re: perf-core build fails on powerpc
John Garry [john.ga...@huawei.com] wrote: > On 13/03/2018 20:10, Sukadev Bhattiprolu wrote: > > > Hi John, > > > > I have an xfs file system which seems to have d_type == DT_UNKNOWN for all > > entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type > > may not be supported by all file systems. > > > > Not relying on ->d_type seems to fix it: > > > > Hi Sukadev, > > Thanks for debugging this. Jiri Olsa (cc'ed) warned me on this, so I did add > the check for d_type == DT_UNKNOWN. > > But, if all files have d_type == DT_UNKNOWN, you're code would from visual > observation look to be same as mine (apart from check for '.' or '..' > filename, which I would say is already covered by stat() and S_ISDIR()). Or > is d_type value just unreliable? In the current code and with DT_UNKNOWN, is_leaf_dir() returns false when it sees the "." or ".." entries right? In the new code, we skip those and return false only if we find some other directory. Thanks, Sukadev
Re: perf-core build fails on powerpc
John Garry [john.ga...@huawei.com] wrote:
> On 13/03/2018 19:17, Sukadev Bhattiprolu wrote:
> >
> >
> > Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
> > with HEAD as:
> >
> > 1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
> >
> > It maybe related to this commit:
> >
> > commit d596299
> > Author: John Garry
> > Date: Thu Mar 8 18:58:29 2018 +0800
> >
> > perf vendor events: Add support for pmu events vendor subdirectory
> >
> > Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
> > problem for me.
> >
Hi John,
I have an xfs file system which seems to have d_type == DT_UNKNOWN for all
entries in 'tools/perf/pmu-events/arch/power8'! readdir(3) says ->d_type
may not be supported by all file systems.
Not relying on ->d_type seems to fix it:
@@ -873,26 +879,26 @@ static int is_leaf_dir(const char *fpath)
return 0;
while ((dir = readdir(d)) != NULL) {
- if (dir->d_type == DT_DIR && dir->d_name[0] != '.') {
- res = 0;
- break;
- } else if (dir->d_type == DT_UNKNOWN) {
- char path[PATH_MAX];
- struct stat st;
+ char path[PATH_MAX];
+ struct stat st;
- sprintf(path, "%s/%s", fpath, dir->d_name);
- if (stat(path, &st))
- break;
+ if (strcmp(dir->d_name, ".") == 0 ||
+ strcmp(dir->d_name, "..") == 0)
+ continue;
- if (S_ISDIR(st.st_mode)) {
- res = 0;
- break;
- }
+ sprintf(path, "%s/%s", fpath, dir->d_name);
+ if (stat(path, &st))
+ break;
+
+ if (S_ISDIR(st.st_mode)) {
+ res = 0;
+ break;
perf-core build fails on powerpc
Building perf on Powerpc seems broken when using Arnaldo's perf/core branch
with HEAD as:
1b442ed ("perf test: Fix exit code for record+probe_libc_inet_pton.sh")
It maybe related to this commit:
commit d596299
Author: John Garry
Date: Thu Mar 8 18:58:29 2018 +0800
perf vendor events: Add support for pmu events vendor subdirectory
Reverting this hunk from tools/perf/pmu-events/jevents.c, seems to fix the
problem for me.
@@ -957,8 +963,12 @@ static int process_one_file(const char *fpath, const struct
/* model directory, reset topic */
+#if 0
if ((level == 1 && is_dir && is_leaf_dir(fpath)) ||
(level == 2 && is_dir)) {
+#else
+ if (level == 1 && is_dir) {
+#endif
The problem is that the tools/perf/pmu-events/pmu-events.c file generated by
the jevents program is missing the "prefix" lines like:
struct pmu_event pme_power8[] = {
and
struct pmu_event pme_power9[] = {
i.e the pmu-events.c begins like:
---
#include "../../pmu-events/pmu-events.h"
{
.name = "pm_data_from_dl2l3_mod",
.event = "event=0x4c048",
.desc = "The processor's data cache was reloaded with Modified (M) data
from another chip's L2 or L3 on a different Node or Group (Distant), as this
chip due to a demand load",
.topic = "cache",
.long_desc = "The processor's data cache was reloaded with Modified (M)
data from another chip's L2 or L3 on a different Node or Group (Distant), as
this chip due to either only demand loads or demand loads plus prefetches if
MMCR1[16] is 1",
},
---
Below is the debug and info messages at the time of failure:
---
HOSTCC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents.o
HOSTLD /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents-in.o
LINK /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/jevents
GEN /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c
d 0 50 powerpc pmu-events/arch/powerpc
d 14096 power8 pmu-events/arch/powerpc/power8
f 2 10275 cache.json pmu-events/arch/powerpc/power8/cache.json
f 2 302 floating-point.json
pmu-events/arch/powerpc/power8/floating-point.json
f 2 29283 marked.json pmu-events/arch/powerpc/power8/marked.json
f 2 12028 memory.json pmu-events/arch/powerpc/power8/memory.json
f 2 159271 other.json pmu-events/arch/powerpc/power8/other.json
f 2 10777 pipeline.jsonpmu-events/arch/powerpc/power8/pipeline.json
f 23781 pmc.json pmu-events/arch/powerpc/power8/pmc.json
f 26664 translation.json pmu-events/arch/powerpc/power8/translation.json
f 2 25104 frontend.jsonpmu-events/arch/powerpc/power8/frontend.json
d 14096 power9 pmu-events/arch/powerpc/power9
f 21096 floating-point.json
pmu-events/arch/powerpc/power9/floating-point.json
f 25503 cache.json pmu-events/arch/powerpc/power9/cache.json
f 2 14885 frontend.jsonpmu-events/arch/powerpc/power9/frontend.json
f 2 27807 marked.json pmu-events/arch/powerpc/power9/marked.json
f 24820 memory.json pmu-events/arch/powerpc/power9/memory.json
f 2 85387 other.json pmu-events/arch/powerpc/power9/other.json
f 2 23839 pipeline.jsonpmu-events/arch/powerpc/power9/pipeline.json
f 25138 pmc.json pmu-events/arch/powerpc/power9/pmc.json
f 29897 translation.json pmu-events/arch/powerpc/power9/translation.json
f 1 440 mapfile.csv pmu-events/arch/powerpc/mapfile.csv
jevents: Processing mapfile pmu-events/arch/powerpc/mapfile.csv
CC /home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.o
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:2:1:
error: expected identifier or ‘(’ before ‘{’ token
{
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:8:2:
error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:15:2:
error: expected identifier or ‘(’ before ‘,’ token
},
^
/home/suka/linux-acme.git/tools/perf/perf-obj/pmu-events/pmu-events.c:22:2:
error: expected identifier or ‘(’ before ‘,’ token
},
[PATCH] Fix cleanup when VAS is not configured
From: Sukadev Bhattiprolu
Date: Fri, 9 Feb 2018 11:49:06 -0600
Subject: [PATCH 1/1] powerpc/vas: Fix cleanup when VAS is not configured
When VAS is not configured, unregister the platform driver. Also simplify
cleanup by delaying vas debugfs init until we know VAS is configured.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v2]
- [Michael Ellerman] Move vas_init_dbgdir() into a lower level
function to keep vas_init() cleaner.
---
arch/powerpc/platforms/powernv/vas-debug.c | 11 +++
arch/powerpc/platforms/powernv/vas.c | 6 +++---
2 files changed, 14 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index b4de4c6..4f7276e 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -179,6 +179,7 @@ void vas_instance_init_dbgdir(struct vas_instance *vinst)
{
struct dentry *d;
+ vas_init_dbgdir();
if (!vas_debugfs)
return;
@@ -201,8 +202,18 @@ void vas_instance_init_dbgdir(struct vas_instance *vinst)
vinst->dbgdir = NULL;
}
+/*
+ * Set up the "root" VAS debugfs dir. Return if we already set it up
+ * (or failed to) in an earlier instance of VAS.
+ */
void vas_init_dbgdir(void)
{
+ static bool first_time = true;
+
+ if (!first_time)
+ return;
+
+ first_time = false;
vas_debugfs = debugfs_create_dir("vas", NULL);
if (IS_ERR(vas_debugfs))
vas_debugfs = NULL;
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index aebbe95..5a2b24c 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -160,8 +160,6 @@ static int __init vas_init(void)
int found = 0;
struct device_node *dn;
- vas_init_dbgdir();
-
platform_driver_register(&vas_driver);
for_each_compatible_node(dn, NULL, "ibm,vas") {
@@ -169,8 +167,10 @@ static int __init vas_init(void)
found++;
}
- if (!found)
+ if (!found) {
+ platform_driver_unregister(&vas_driver);
return -ENODEV;
+ }
pr_devel("Found %d instances\n", found);
--
2.7.4
Re: [PATCH 2/4] powerpc/vas: Fix cleanup when VAS is not configured
Michael Ellerman [m...@ellerman.id.au] wrote:
> Sukadev Bhattiprolu writes:
>
> > When VAS is not configured in the system, make sure to remove
> > the VAS debugfs directory and unregister the platform driver.
> >
> > Signed-off-by: Sukadev Bhattiprolu
> ...
> > diff --git a/arch/powerpc/platforms/powernv/vas.c
> > b/arch/powerpc/platforms/powernv/vas.c
> > index aebbe95..f83e27d8 100644
> > --- a/arch/powerpc/platforms/powernv/vas.c
> > +++ b/arch/powerpc/platforms/powernv/vas.c
> > @@ -169,8 +169,11 @@ static int __init vas_init(void)
> > found++;
> > }
> >
> > - if (!found)
> > + if (!found) {
> > + platform_driver_unregister(&vas_driver);
> > + vas_cleanup_dbgdir();
> > return -ENODEV;
> > + }
>
> The better patch would be to move the call to vas_init_dbgdir() down
> here, where we know we have successfully registered the driver.
Well, when VAS is configured, init_vas_instance() expects the top level
"vas" debugfs dir to already be setup.
We could have each init_vas_instance() assume it is the first and
unconditionally call vas_init_dbgdir(). vas_init_dbgdir() could make
sure to initialize only once.
Or, we could make a separate pass countng "ibm,vas" nodes. If there are
none, skip both steps (dbgdir and registering platform driver).
Sukadev
[PATCH 2/4] powerpc/vas: Fix cleanup when VAS is not configured
When VAS is not configured in the system, make sure to remove
the VAS debugfs directory and unregister the platform driver.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-debug.c | 5 +
arch/powerpc/platforms/powernv/vas.c | 5 -
arch/powerpc/platforms/powernv/vas.h | 1 +
3 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
index b4de4c6..e6e4067 100644
--- a/arch/powerpc/platforms/powernv/vas-debug.c
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -207,3 +207,8 @@ void vas_init_dbgdir(void)
if (IS_ERR(vas_debugfs))
vas_debugfs = NULL;
}
+
+void vas_cleanup_dbgdir(void)
+{
+ debugfs_remove_recursive(vas_debugfs);
+}
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index aebbe95..f83e27d8 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -169,8 +169,11 @@ static int __init vas_init(void)
found++;
}
- if (!found)
+ if (!found) {
+ platform_driver_unregister(&vas_driver);
+ vas_cleanup_dbgdir();
return -ENODEV;
+ }
pr_devel("Found %d instances\n", found);
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index ae0100f..2645613 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -406,6 +406,7 @@ extern struct mutex vas_mutex;
extern struct vas_instance *find_vas_instance(int vasid);
extern void vas_init_dbgdir(void);
+extern void vas_cleanup_dbgdir(void);
extern void vas_instance_init_dbgdir(struct vas_instance *vinst);
extern void vas_window_init_dbgdir(struct vas_window *win);
extern void vas_window_free_dbgdir(struct vas_window *win);
--
2.7.4
[PATCH 4/4] powerpc/vas: Add a couple of trace points
Add a couple of trace points in the VAS driver
Signed-off-by: Sukadev Bhattiprolu
---
Changelog [v2]
- Make TRACE_INCLUDE_PATH relative to
---
arch/powerpc/platforms/powernv/vas-trace.h | 112
arch/powerpc/platforms/powernv/vas-window.c | 9 +++
2 files changed, 121 insertions(+)
create mode 100644 arch/powerpc/platforms/powernv/vas-trace.h
diff --git a/arch/powerpc/platforms/powernv/vas-trace.h
b/arch/powerpc/platforms/powernv/vas-trace.h
new file mode 100644
index 000..939d85d
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-trace.h
@@ -0,0 +1,112 @@
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM vas
+
+#if !defined(_VAS_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
+
+#define _VAS_TRACE_H
+#include
+#include
+#include
+
+TRACE_EVENT( vas_rx_win_open,
+
+ TP_PROTO(struct task_struct *tsk,
+int vasid,
+int cop,
+struct vas_rx_win_attr *rxattr),
+
+ TP_ARGS(tsk, vasid, cop, rxattr),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(int, pid)
+ __field(int, cop)
+ __field(int, vasid)
+ __field(struct vas_rx_win_attr *, rxattr)
+ __field(int, lnotify_lpid)
+ __field(int, lnotify_pid)
+ __field(int, lnotify_tid)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->vasid = vasid;
+ __entry->cop = cop;
+ __entry->lnotify_lpid = rxattr->lnotify_lpid;
+ __entry->lnotify_pid = rxattr->lnotify_pid;
+ __entry->lnotify_tid = rxattr->lnotify_tid;
+ ),
+
+ TP_printk("pid=%d, vasid=%d, cop=%d, lpid=%d, pid=%d, tid=%d",
+ __entry->pid, __entry->vasid, __entry->cop,
+ __entry->lnotify_lpid, __entry->lnotify_pid,
+ __entry->lnotify_tid)
+);
+
+TRACE_EVENT( vas_tx_win_open,
+
+ TP_PROTO(struct task_struct *tsk,
+int vasid,
+int cop,
+struct vas_tx_win_attr *txattr),
+
+ TP_ARGS(tsk, vasid, cop, txattr),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(int, pid)
+ __field(int, cop)
+ __field(int, vasid)
+ __field(struct vas_tx_win_attr *, txattr)
+ __field(int, lpid)
+ __field(int, pidr)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->vasid = vasid;
+ __entry->cop = cop;
+ __entry->lpid = txattr->lpid;
+ __entry->pidr = txattr->pidr;
+ ),
+
+ TP_printk("pid=%d, vasid=%d, cop=%d, lpid=%d, pidr=%d",
+ __entry->pid, __entry->vasid, __entry->cop,
+ __entry->lpid, __entry->pidr)
+);
+
+TRACE_EVENT( vas_paste_crb,
+
+ TP_PROTO(struct task_struct *tsk,
+ struct vas_window *win),
+
+ TP_ARGS(tsk, win),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(struct vas_window *, win)
+ __field(int, pid)
+ __field(int, vasid)
+ __field(int, winid)
+ __field(unsigned long, paste_kaddr)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->vasid = win->vinst->vas_id;
+ __entry->winid = win->winid;
+ __entry->paste_kaddr = (unsigned long)win->paste_kaddr
+ ),
+
+ TP_printk("pid=%d, vasid=%d, winid=%d, paste_kaddr=0x%016lx\n",
+ __entry->pid, __entry->vasid, __entry->winid,
+ __entry->paste_kaddr)
+);
+
+#endif /* _VAS_TRACE_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH ../../arch/powerpc/platforms/powernv
+#define TRACE_INCLUDE_FILE vas-trace
+#include
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 2b3eb01..6b2de9e 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -21,6 +21,9 @@
#include "vas.h"
#include &
[PATCH RESEND 3/4] powerpc/vas: Remove a stray line in Makefile
Remove a bogus line from arch/powerpc/platforms/powernv/Makefile that
was added by commit ece4e51 ("powerpc/vas: Export HVWC to debugfs").
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/Makefile | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index 6c9d519..703a350 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -16,5 +16,4 @@ obj-$(CONFIG_OPAL_PRD)+= opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
-obj-$(CONFIG_PPC_FTW) += nx-ftw.o
obj-$(CONFIG_OCXL_BASE)+= ocxl.o
--
2.7.4
[PATCH RESEND 1/4] powerpc/vas: Fix order of cleanup in debugfs dir
Fix the order of cleanup to ensure we free the name buffer in case of an error creating 'hvwc' or 'info' files. Signed-off-by: Sukadev Bhattiprolu --- arch/powerpc/platforms/powernv/vas-debug.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-debug.c b/arch/powerpc/platforms/powernv/vas-debug.c index ca22f1e..b4de4c6 100644 --- a/arch/powerpc/platforms/powernv/vas-debug.c +++ b/arch/powerpc/platforms/powernv/vas-debug.c @@ -166,13 +166,13 @@ void vas_window_init_dbgdir(struct vas_window *window) return; -free_name: - kfree(window->dbgname); - window->dbgname = NULL; - remove_dir: debugfs_remove_recursive(window->dbgdir); window->dbgdir = NULL; + +free_name: + kfree(window->dbgname); + window->dbgname = NULL; } void vas_instance_init_dbgdir(struct vas_instance *vinst) -- 2.7.4
Re: [PATCH 5/5] powerpc/ftw: Document FTW API/usage
Randy Dunlap [rdun...@infradead.org] wrote:
> > +struct ftw_setup_attr ftwattr;
> > +
> > +fd = open("/dev/ftw", O_RDWR);
> > +
> > +memset(&rxattr, 0, sizeof(rxattr));
>
> Is that supposed to be ftwattr (2x above)?
Yes. I agree with your other comments as well and will send a new version.
Thanks for the detailed review.
Sukadev
Re: [PATCH 3/5] powerpc/ftw: Implement a simple FTW driver
Randy Dunlap [rdun...@infradead.org] wrote:
> > +
> > + default:
> > + return -EINVAL;
> > + }
> > +}
>
> Nit: some versions of gcc (or maybe clang) complain about a typed function
> not always having a return value in code like above, so it is often done as:
Ok.
>
> > +static long ftw_ioctl(struct file *fp, unsigned int cmd, unsigned long arg)
> > +{
> > + switch (cmd) {
> > +
> > + case FTW_SETUP:
> > + return ftw_ioc_ftw_setup(fp, arg);
> > +
> > + default:
> > + break;
> > + }
>
> return -EINVAL;
> > +}
>
> Do you expect to implement more ioctls? If not, just change the switch to
> an if ().
Maybe a couple more but changed it to an 'if' for now (and fixed an
error handling issue in ftw_file_init()).
Here is the updated patch.
---
>From 344ffbcc2cd1e64dd87249d508cf6000e6e41a0c Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu
Date: Fri, 4 Aug 2017 16:45:34 -0500
Subject: [PATCH 3/5] powerpc/ftw: Implement a simple FTW driver
The Fast Thread Wake-up (FTW) driver provides user space applications an
interface to the low latency Core-to-Core wakeup functionality in POWER9.
This mechanism allows a thread on one core to efficiently send a message
to a "waiting thread" on another core on the same chip, using the Virtual
Accelrator Switchboard (VAS) subsystem.
This initial FTW driver implements the ioctl and mmap operations on an
FTW device node. Using these operations, a pair of application threads
can establish a "communication channel" and use the COPY, PASTE and WAIT
instructions to wait/wake up.
PATCH 5/5 documents the API and includes an example of the usage.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v2]
- [Michael Neuling] Rename from drop "nx" from name "nx-ftw".
- [Michael Neuling] Use a single VAS_FTW_SETUP ioctl to simplify
interface.
- [Michael Ellerman] To work with paste emulation patch, mark
PTE dirty in ->mmap() to ensure there is no fault on paste
(the emulation patch must disable pagefaults when updating
thread reconfig registers).
- [Randy Dunlap] Minor cleanup in ftw_ioctl().
- Fix cleanup code in ftw_file_init()
- Check return value from set_thread_tidr().
- Move driver drivers/misc/ftw.
---
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/ftw/Kconfig | 16 +++
drivers/misc/ftw/Makefile | 4 +
drivers/misc/ftw/ftw.c| 346 ++
5 files changed, 368 insertions(+)
create mode 100644 drivers/misc/ftw/Kconfig
create mode 100644 drivers/misc/ftw/Makefile
create mode 100644 drivers/misc/ftw/ftw.c
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index f1a5c23..a9b161f 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -508,4 +508,5 @@ source "drivers/misc/mic/Kconfig"
source "drivers/misc/genwqe/Kconfig"
source "drivers/misc/echo/Kconfig"
source "drivers/misc/cxl/Kconfig"
+source "drivers/misc/ftw/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index 5ca5f64..338668c 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -52,6 +52,7 @@ obj-$(CONFIG_GENWQE) += genwqe/
obj-$(CONFIG_ECHO) += echo/
obj-$(CONFIG_VEXPRESS_SYSCFG) += vexpress-syscfg.o
obj-$(CONFIG_CXL_BASE) += cxl/
+obj-$(CONFIG_PPC_FTW) += ftw/
obj-$(CONFIG_ASPEED_LPC_CTRL) += aspeed-lpc-ctrl.o
obj-$(CONFIG_ASPEED_LPC_SNOOP) += aspeed-lpc-snoop.o
obj-$(CONFIG_PCI_ENDPOINT_TEST)+= pci_endpoint_test.o
diff --git a/drivers/misc/ftw/Kconfig b/drivers/misc/ftw/Kconfig
new file mode 100644
index 000..5454d40
--- /dev/null
+++ b/drivers/misc/ftw/Kconfig
@@ -0,0 +1,16 @@
+
+config PPC_FTW
+ tristate "IBM Fast Thread-Wakeup (FTW)"
+ depends on PPC_VAS
+ default n
+ help
+ This enables support for IBM Fast Thread-Wakeup driver.
+
+ The FTW driver allows applications to utilize a low overhead
+ core-to-core wake up mechansim in the IBM Virtual Accelerator
+ Switchboard (VAS) to improve performance.
+
+ VAS adapters are found in POWER9 based systems and are required
+ for the FTW driver to be operational.
+
+ If unsure, say N.
diff --git a/drivers/misc/ftw/Makefile b/drivers/misc/ftw/Makefile
new file mode 100644
index 000..2cfe566
--- /dev/null
+++ b/drivers/misc/ftw/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+ccflags-y := $(call cc-disable-warning,
unused-const-variable)
+ccflags-$(CONFIG_PPC_WERROR) += -Werror
+obj-$(CONFIG_PPC_FTW) += ftw.o
diff --git a/drivers/misc/ftw/ftw.c b/drivers/misc/ftw/ftw.
Re: [PATCH 2/5] powerpc/ftw: Define FTW_SETUP ioctl API
Randy Dunlap [rdun...@infradead.org] wrote:
> > +#define FTW_FLAGS_PIN_WINDOW 0x1
> > +
> > +#define FTW_SETUP _IOW('v', 1, struct ftw_setup_attr)
>
> ioctls should be documented in Documentation/ioctl/ioctl-number.txt.
> Please update that file.
Ok. Here is the updated patch.
Thanks for the review.
Sukadev
---
>From 1f347c199a0b1bbc528705c8e9ddd11c825a80fc Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu
Date: Thu, 2 Feb 2017 06:20:07 -0500
Subject: [PATCH 2/5] powerpc/ftw: Define FTW_SETUP ioctl API
Define the FTW_SETUP ioctl interface for fast thread wakeup (FTW). A
follow-on patch will implement the FTW driver and ioctl.
Thanks to input from Ben Herrenschmidt, Michael Neuling, Michael Ellerman.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v2]
- [Michael Neuling] Use a single VAS_FTW_SETUP ioctl and simplify
the interface.
- [Randy Dunlap] Reserve/document the ioctl number used.
---
Documentation/ioctl/ioctl-number.txt | 1 +
include/uapi/misc/ftw.h | 35 +++
2 files changed, 36 insertions(+)
create mode 100644 include/uapi/misc/ftw.h
diff --git a/Documentation/ioctl/ioctl-number.txt
b/Documentation/ioctl/ioctl-number.txt
index 3e3fdae..b0f323c 100644
--- a/Documentation/ioctl/ioctl-number.txt
+++ b/Documentation/ioctl/ioctl-number.txt
@@ -277,6 +277,7 @@ Code Seq#(hex) Include FileComments
'v'00-1F linux/fs.h conflict!
'v'00-0F linux/sonypi.h conflict!
'v'C0-FF linux/meye.hconflict!
+'v'20-27 include/uapi/misc/ftw.h
'w'all CERN SCI driver
'y'00-1F packet based user level communications
<mailto:zap...@interlan.net>
diff --git a/include/uapi/misc/ftw.h b/include/uapi/misc/ftw.h
new file mode 100644
index 000..99676b2
--- /dev/null
+++ b/include/uapi/misc/ftw.h
@@ -0,0 +1,35 @@
+/*
+ * Copyright 2018 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _UAPI_MISC_FTW_H
+#define _UAPI_MISC_FTW_H
+
+#include
+#include
+
+#define FTW_FLAGS_PIN_WINDOW 0x1
+
+/*
+ * Note: The range 0x20-27 for letter 'v' are reserved for FTW ioctls in
+ * Documentation/ioctl/ioctl-number.txt.
+ */
+#define FTW_SETUP _IOW('v', 0x20, struct ftw_setup_attr)
+
+struct ftw_setup_attr {
+ __s16 version;
+ __s16 vas_id; /* specific instance of vas or -1 for default */
+ __u32 reserved;
+
+ __u64 reserved1;
+
+ __u64 flags;
+ __u64 reserved2;
+};
+
+#endif /* _UAPI_MISC_FTW_H */
--
2.7.4
Re: [PATCH 1/2] powerpc: export thread-tidr interfaces
Frederic Barrat [fbar...@linux.vnet.ibm.com] wrote: > Hi, > > > > diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c > > index 2010e4c..f20c1ad 100644 > > --- a/arch/powerpc/kernel/process.c > > +++ b/arch/powerpc/kernel/process.c > > @@ -1560,6 +1560,7 @@ void clear_thread_tidr(struct task_struct *t) > > free_thread_tidr(t->thread.tidr); > > t->thread.tidr = 0; > > } > > +EXPORT_SYMBOL_GPL(clear_thread_tidr); > > Isn't it dangerous to export clear_thread_tidr()? Other modules may also > have assigned the TIDR by calling set_thread_tidr(), so clearing it could > potentially break those other modules. My understanding is that once the > TIDR is assigned, there's no safe way to reclaim it other than the thread > exiting. Or we would need some kind of reference counter. Yes the FTW driver avoids calling clear_thread_tidr() for the same reasons. I don't have a strong case for exporting clear_thread_tidr(). Here is the updated patch, exporting just the set_thread_tidr(). Thanks, Sukadev --- >From 204ee3c918f8dad46c1e40d2d3730b07c10a87a3 Mon Sep 17 00:00:00 2001 From: Sukadev Bhattiprolu Date: Mon, 15 Jan 2018 13:43:18 -0600 Subject: [PATCH 1/2] powerpc: export set_thread_tidr() Export set_thread_tidr() so it can be used by external modules. Signed-off-by: Sukadev Bhattiprolu --- Changelog [Frederic Barrat] Don't export clear_thread_tidr() --- arch/powerpc/kernel/process.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c index 2010e4c..20df2cb2 100644 --- a/arch/powerpc/kernel/process.c +++ b/arch/powerpc/kernel/process.c @@ -1592,6 +1592,7 @@ int set_thread_tidr(struct task_struct *t) return 0; } +EXPORT_SYMBOL_GPL(set_thread_tidr); #endif /* CONFIG_PPC64 */ -- 1.8.3.1
[PATCH 1/5] powerpc/vas: Remove a stray line in Makefile
Remove a bogus line from arch/powerpc/platforms/powernv/Makefile that
was added by commit ece4e51 ("powerpc/vas: Export HVWC to debugfs").
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/Makefile | 1 -
1 file changed, 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index 3732118..ca94488 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -16,4 +16,3 @@ obj-$(CONFIG_OPAL_PRD)+= opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
-obj-$(CONFIG_PPC_FTW) += nx-ftw.o
--
2.7.4
[PATCH 3/5] powerpc/ftw: Implement a simple FTW driver
The Fast Thread Wake-up (FTW) driver provides user space applications an
interface to the low latency Core-to-Core wakeup functionality in POWER9.
This mechanism allows a thread on one core to efficiently send a message
to a "waiting thread" on another core on the same chip, using the Virtual
Accelrator Switchboard (VAS) subsystem.
This initial FTW driver implements the ioctl and mmap operations on an
FTW device node. Using these operations, a pair of application threads
can establish a "communication channel" and use the COPY, PASTE and WAIT
instructions to wait/wake up.
PATCH 5/5 documents the API and includes an example of the usage.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v2]
- [Michael Neuling] Rename from drop "nx" from name "nx-ftw".
- [Michael Neuling] Use a single VAS_FTW_SETUP ioctl to simplify
interface.
- [Michael Ellerman] To work with paste emulation patch, mark
PTE dirty in ->mmap() to ensure there is no fault on paste
(the emulation patch must disable pagefaults when updating
thread reconfig registers).
- Check return value from set_thread_tidr().
- Move driver drivers/misc/ftw.
---
drivers/misc/Kconfig | 1 +
drivers/misc/Makefile | 1 +
drivers/misc/ftw/Kconfig | 16 +++
drivers/misc/ftw/Makefile | 4 +
drivers/misc/ftw/ftw.c| 346 ++
5 files changed, 368 insertions(+)
create mode 100644 drivers/misc/ftw/Kconfig
create mode 100644 drivers/misc/ftw/Makefile
create mode 100644 drivers/misc/ftw/ftw.c
diff --git a/drivers/misc/Kconfig b/drivers/misc/Kconfig
index f1a5c23..a9b161f 100644
--- a/drivers/misc/Kconfig
+++ b/drivers/misc/Kconfig
@@ -508,4 +508,5 @@ source "drivers/misc/mic/Kconfig"
source "drivers/misc/genwqe/Kconfig"
source "drivers/misc/echo/Kconfig"
source "drivers/misc/cxl/Kconfig"
+source "drivers/misc/ftw/Kconfig"
endmenu
diff --git a/drivers/misc/Makefile b/drivers/misc/Makefile
index 5ca5f64..338668c 100644
--- a/drivers/misc/Makefile
+++ b/drivers/misc/Makefile
@@ -52,6 +52,7 @@ obj-$(CONFIG_GENWQE) += genwqe/
obj-$(CONFIG_ECHO) += echo/
obj-$(CONFIG_VEXPRESS_SYSCFG) += vexpress-syscfg.o
obj-$(CONFIG_CXL_BASE) += cxl/
+obj-$(CONFIG_PPC_FTW) += ftw/
obj-$(CONFIG_ASPEED_LPC_CTRL) += aspeed-lpc-ctrl.o
obj-$(CONFIG_ASPEED_LPC_SNOOP) += aspeed-lpc-snoop.o
obj-$(CONFIG_PCI_ENDPOINT_TEST)+= pci_endpoint_test.o
diff --git a/drivers/misc/ftw/Kconfig b/drivers/misc/ftw/Kconfig
new file mode 100644
index 000..5454d40
--- /dev/null
+++ b/drivers/misc/ftw/Kconfig
@@ -0,0 +1,16 @@
+
+config PPC_FTW
+ tristate "IBM Fast Thread-Wakeup (FTW)"
+ depends on PPC_VAS
+ default n
+ help
+ This enables support for IBM Fast Thread-Wakeup driver.
+
+ The FTW driver allows applications to utilize a low overhead
+ core-to-core wake up mechansim in the IBM Virtual Accelerator
+ Switchboard (VAS) to improve performance.
+
+ VAS adapters are found in POWER9 based systems and are required
+ for the FTW driver to be operational.
+
+ If unsure, say N.
diff --git a/drivers/misc/ftw/Makefile b/drivers/misc/ftw/Makefile
new file mode 100644
index 000..2cfe566
--- /dev/null
+++ b/drivers/misc/ftw/Makefile
@@ -0,0 +1,4 @@
+# SPDX-License-Identifier: GPL-2.0
+ccflags-y := $(call cc-disable-warning,
unused-const-variable)
+ccflags-$(CONFIG_PPC_WERROR) += -Werror
+obj-$(CONFIG_PPC_FTW) += ftw.o
diff --git a/drivers/misc/ftw/ftw.c b/drivers/misc/ftw/ftw.c
new file mode 100644
index 000..6fcb4e2
--- /dev/null
+++ b/drivers/misc/ftw/ftw.c
@@ -0,0 +1,346 @@
+/*
+ * Copyright 2018 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#define pr_fmt(fmt) "ftw: " fmt
+
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+#include
+
+/*
+ * FTW is a device driver used to provide user space access to the
+ * Core-to-Core aka Fast Thread Wakeup (FTW) functionality provided by
+ * the Virtual Accelerator Subsystem (VAS) in POWER9 systems. See also
+ * arch/powerpc/platforms/powernv/vas*.
+ *
+ * The driver creates the device /dev/ftw that can be used as follows:
+ *
+ * fd = open("/dev/ftw", O_RDWR);
+ * rc = ioctl(fd, FTW_SETUP, &attr);
+ * paste_addr = mmap(NULL, PAGE_SIZE, prot, MAP_SHARED, fd, 0ULL).
+ * vas_copy(&crb, 0, 1);
+ * vas_paste(paste_addr, 0, 1);
+ *
+ * where "vas_copy" and "vas_paste" are defined in co
[PATCH 4/5] powerpc/ftw: Add a couple of trace points
Add a couple of trace points in the FTW driver
Signed-off-by: Sukadev Bhattiprolu
---
drivers/misc/ftw/ftw-trace.h | 75
drivers/misc/ftw/ftw.c | 6
2 files changed, 81 insertions(+)
create mode 100644 drivers/misc/ftw/ftw-trace.h
diff --git a/drivers/misc/ftw/ftw-trace.h b/drivers/misc/ftw/ftw-trace.h
new file mode 100644
index 000..0d96046
--- /dev/null
+++ b/drivers/misc/ftw/ftw-trace.h
@@ -0,0 +1,75 @@
+/*
+ * Copyright 2018 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM ftw
+
+#if !defined(_FTW_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
+
+#define _FTW_TRACE_H
+#include
+#include
+
+TRACE_EVENT( ftw_open_event,
+
+ TP_PROTO(struct task_struct *tsk,
+int instid),
+
+ TP_ARGS(tsk, instid),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(int, instid)
+ __field(int, pid)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->instid = instid;
+ ),
+
+ TP_printk("pid=%d, inst=%d", __entry->pid, __entry->instid)
+);
+
+TRACE_EVENT( ftw_mmap_event,
+
+ TP_PROTO(struct task_struct *tsk,
+int instid,
+unsigned long paste_addr,
+unsigned long vma_start),
+
+ TP_ARGS(tsk, instid, paste_addr, vma_start),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(int, pid)
+ __field(int, instid)
+ __field(unsigned long, paste_addr)
+ __field(unsigned long, vma_start)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->instid = instid;
+ __entry->paste_addr = paste_addr;
+ __entry->vma_start = vma_start;
+ ),
+
+ TP_printk(
+ "pid=%d, inst=%d, pasteaddr=0x%16lx, vma_start=0x%16lx",
+ __entry->pid, __entry->instid, __entry->paste_addr,
+ __entry->vma_start)
+);
+
+#endif /* _FTW_TRACE_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#define TRACE_INCLUDE_FILE ftw-trace
+#include
diff --git a/drivers/misc/ftw/ftw.c b/drivers/misc/ftw/ftw.c
index 6fcb4e2..a01c9e6 100644
--- a/drivers/misc/ftw/ftw.c
+++ b/drivers/misc/ftw/ftw.c
@@ -21,6 +21,9 @@
#include
#include
+#define CREATE_TRACE_POINTS
+#include "ftw-trace.h"
+
/*
* FTW is a device driver used to provide user space access to the
* Core-to-Core aka Fast Thread Wakeup (FTW) functionality provided by
@@ -81,6 +84,8 @@ static int ftw_open(struct inode *inode, struct file *fp)
fp->private_data = instance;
+ trace_ftw_open_event(current, instance->id);
+
return 0;
}
@@ -234,6 +239,7 @@ static int ftw_mmap(struct file *fp, struct vm_area_struct
*vma)
pr_devel("paste addr %llx at %lx, rc %d\n", paste_addr, vma->vm_start,
rc);
+ trace_ftw_mmap_event(current, instance->id, paste_addr, vma->vm_start);
set_thread_uses_vas();
--
2.7.4
[PATCH 5/5] powerpc/ftw: Document FTW API/usage
Document the usage of the VAS Fast thread-wakeup API and add an entry in MAINTAINERS file. Thanks for input/comments from Benjamin Herrenschmidt, Michael Neuling, Michael Ellerman, Robert Blackmore, Ian Munsie, Haren Myneni and Paul Mackerras. Signed-off-by: Sukadev Bhattiprolu --- Changelog[v2] - [Michael Neuling] Update API to use a single, VAS_FTW_SEUTP ioctl rather than two ioctls. - [Michael Neuling] Drop "nx" from name "nx-ftw". --- Documentation/powerpc/ftw-api.txt | 283 ++ MAINTAINERS | 8 ++ 2 files changed, 291 insertions(+) create mode 100644 Documentation/powerpc/ftw-api.txt diff --git a/Documentation/powerpc/ftw-api.txt b/Documentation/powerpc/ftw-api.txt new file mode 100644 index 000..a107628 --- /dev/null +++ b/Documentation/powerpc/ftw-api.txt @@ -0,0 +1,283 @@ +Virtual Accelerator Switchboard and Fast Thread-Wakeup API + +Power9 processor supports a hardware subystem known as the Virtual +Accelerator Switchboard (VAS) which allows two entities in the Power9 +system to efficiently exchange messages. Messages must be formatted as +Coprocessor Request Blocks (CRB) and be submitted using the COPY/PASTE +instructions (new in Power9). + +Usage of VAS depends on the entities exchanging the messages and +currently two usages have been identified. + +First usage of VAS, referred to as VAS/NX involves a software thread +submitting data compression requests to a co-processor (hardware/nest +accelerator) aka NX engine. This usage is not yet available to user +applications. + +Alternatively, VAS can be used by two software threads to efficiently +exchange messages. Initially, this mechanism is intended to wake up a +waiting thread quickly - i.e "fast thread wake-up (FTW)". This document +describes the user API for this VAS/FTW mechanism. + +Application access to the FTW mechanism is provided through the FTW +device node (/dev/ftw) implemented by the FTW device driver. + +A multi-threaded software processes that intends to use the FTW +mechanism must first setup a channel (consisting of a pair of VAS +windows) for the waiting and waking threads to communicate. The +channel is set up by opening the FTW device and issuing the FTW_SETUP +ioctl. Upon successful return from the ioctl, the waiting side of +channel is complete and a thread can issue the "Wait" instruction +to wait for an event. + +After the successful return from the FTW_SETUP ioctl, the waking +thread must use mmap() system call on the same file descriptor and +obtain a virtual address known as the "paste address". + +Once the mmap() call succeeds the setup of "waking" side of the channel +is complete. To wake up a waiting thread, the waking thread should use +the "COPY" and "PASTE" instructions to write a zero-filled CRB to the +paste-address. + +The wait and wake up operations can be repeated as long as the paste +address and the FTW file descriptor are valid (i.e until munmap() of +the paste address or a close() of the FTW fd). + +1. FTW Device Node + +There is one /dev/ftw node in the system and it provides access to the +VAS/FTW functionality. + +The only valid operations (system calls) on the FTW node are: + +- open() the device for read and write. + +- issue the FTW_SETUP ioctl to set up a channel. + +- mmap() the file descriptor + +- close the device node. + +Other file operations on the FTW node are undefined. + +Note that the COPY and PASTE operations go directly to the hardware +and do not involve system calls or go through the FTW device. + +Although a system may have several instances of the VAS in the system +(typically, one per P9 chip) there is just one FTW device node in +the system. + +When the FTW device node is opened, the kernel assigns a suitable +instance of VAS to the process. Kernel will make a best-effort attempt +to assign an optimal instance of VAS for the process - based on the CPU/ +chip that the process is running on. In the initial release, the kernel +does not support migrating the VAS instance if the process migrates from +a CPU on one chip to a CPU on another chip. + +Applications may chose a specific instance of the VAS using the 'vas_id' +field in the FTW_SETUP ioctl as detailed below. + +2. Open FTW node + +The device should be opened for read and write. No special privileges +are needed to open the device. The device may be opened multiple times. + +Each open() of the FTW device is associated with one channel of +communication. There is a system-wide limit (currently 64K windows per +chip and since some are reserved for hardware, there are about 32K +ch
[PATCH 2/5] powerpc/ftw: Define FTW_SETUP ioctl API
Define the FTW_SETUP ioctl interface for fast thread wakeup (FTW). A
follow-on patch will implement the FTW driver and ioctl.
Thanks to input from Ben Herrenschmidt, Michael Neuling, Michael Ellerman.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v2]
- [Michael Neuling] Use a single VAS_FTW_SETUP ioctl and simplify
the interface.
---
include/uapi/misc/ftw.h | 31 +++
1 file changed, 31 insertions(+)
create mode 100644 include/uapi/misc/ftw.h
diff --git a/include/uapi/misc/ftw.h b/include/uapi/misc/ftw.h
new file mode 100644
index 000..f233f51
--- /dev/null
+++ b/include/uapi/misc/ftw.h
@@ -0,0 +1,31 @@
+/*
+ * Copyright 2018 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _UAPI_MISC_FTW_H
+#define _UAPI_MISC_FTW_H
+
+#include
+#include
+
+#define FTW_FLAGS_PIN_WINDOW 0x1
+
+#define FTW_SETUP _IOW('v', 1, struct ftw_setup_attr)
+
+struct ftw_setup_attr {
+ __s16 version;
+ __s16 vas_id; /* specific instance of vas or -1 for default */
+ __u32 reserved;
+
+ __u64 reserved1;
+
+ __u64 flags;
+ __u64 reserved2;
+};
+
+#endif /* _UAPI_MISC_FTW_H */
--
2.7.4
[PATCH 0/5] Implement FTW driver
The Virtual Accelerator Switchboard (VAS) subsystem in the POWER9 processor provides a low latency Core-to-core wakeup" mechanism which allows a thread on one core the processor to efficiently send a message to a thread waiting on another core. This Fast thread-wakeup (FTW) driver provides user space applications an interface to the Core-to-core wakeup mechanism. The FTW driver uses the "external" interfaces provided by the VAS driver to interact with the VAS hardware. PATCH 5/5 documents the API. The ftw-next branch on my github has some initial test cases for the driver: https://github.com/sukadev/linux/tree/ftw-next Thanks to input from Ben Herrenschmidt, Michael Ellerman, Michael Neuling and Robert Blackmore. Sukadev Bhattiprolu (5): powerpc/vas: Remove a stray line in Makefile powerpc/ftw: Define FTW_SETUP ioctl API powerpc/ftw: Implement a simple FTW driver powerpc/ftw: Add a couple of trace points powerpc/ftw: Document FTW API/usage Documentation/powerpc/ftw-api.txt | 283 + MAINTAINERS | 8 + arch/powerpc/platforms/powernv/Makefile | 1 - drivers/misc/Kconfig| 1 + drivers/misc/Makefile | 1 + drivers/misc/ftw/Kconfig| 16 ++ drivers/misc/ftw/Makefile | 4 + drivers/misc/ftw/ftw-trace.h| 75 +++ drivers/misc/ftw/ftw.c | 352 include/uapi/misc/ftw.h | 31 +++ 10 files changed, 771 insertions(+), 1 deletion(-) create mode 100644 Documentation/powerpc/ftw-api.txt create mode 100644 drivers/misc/ftw/Kconfig create mode 100644 drivers/misc/ftw/Makefile create mode 100644 drivers/misc/ftw/ftw-trace.h create mode 100644 drivers/misc/ftw/ftw.c create mode 100644 include/uapi/misc/ftw.h -- 2.7.4
[PATCH 1/1] powerpc/vas: Add a couple of trace points
Add a couple of trace points in the VAS driver.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-trace.h | 112
arch/powerpc/platforms/powernv/vas-window.c | 9 +++
2 files changed, 121 insertions(+)
create mode 100644 arch/powerpc/platforms/powernv/vas-trace.h
diff --git a/arch/powerpc/platforms/powernv/vas-trace.h
b/arch/powerpc/platforms/powernv/vas-trace.h
new file mode 100644
index 000..c937191
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-trace.h
@@ -0,0 +1,112 @@
+
+#undef TRACE_SYSTEM
+#define TRACE_SYSTEM vas
+
+#if !defined(_VAS_TRACE_H) || defined(TRACE_HEADER_MULTI_READ)
+
+#define _VAS_TRACE_H
+#include
+#include
+#include
+
+TRACE_EVENT( vas_rx_win_open,
+
+ TP_PROTO(struct task_struct *tsk,
+int vasid,
+int cop,
+struct vas_rx_win_attr *rxattr),
+
+ TP_ARGS(tsk, vasid, cop, rxattr),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(int, pid)
+ __field(int, cop)
+ __field(int, vasid)
+ __field(struct vas_rx_win_attr *, rxattr)
+ __field(int, lnotify_lpid)
+ __field(int, lnotify_pid)
+ __field(int, lnotify_tid)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->vasid = vasid;
+ __entry->cop = cop;
+ __entry->lnotify_lpid = rxattr->lnotify_lpid;
+ __entry->lnotify_pid = rxattr->lnotify_pid;
+ __entry->lnotify_tid = rxattr->lnotify_tid;
+ ),
+
+ TP_printk("pid=%d, vasid=%d, cop=%d, lpid=%d, pid=%d, tid=%d",
+ __entry->pid, __entry->vasid, __entry->cop,
+ __entry->lnotify_lpid, __entry->lnotify_pid,
+ __entry->lnotify_tid)
+);
+
+TRACE_EVENT( vas_tx_win_open,
+
+ TP_PROTO(struct task_struct *tsk,
+int vasid,
+int cop,
+struct vas_tx_win_attr *txattr),
+
+ TP_ARGS(tsk, vasid, cop, txattr),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(int, pid)
+ __field(int, cop)
+ __field(int, vasid)
+ __field(struct vas_tx_win_attr *, txattr)
+ __field(int, lpid)
+ __field(int, pidr)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->vasid = vasid;
+ __entry->cop = cop;
+ __entry->lpid = txattr->lpid;
+ __entry->pidr = txattr->pidr;
+ ),
+
+ TP_printk("pid=%d, vasid=%d, cop=%d, lpid=%d, pidr=%d",
+ __entry->pid, __entry->vasid, __entry->cop,
+ __entry->lpid, __entry->pidr)
+);
+
+TRACE_EVENT( vas_paste_crb,
+
+ TP_PROTO(struct task_struct *tsk,
+ struct vas_window *win),
+
+ TP_ARGS(tsk, win),
+
+ TP_STRUCT__entry(
+ __field(struct task_struct *, tsk)
+ __field(struct vas_window *, win)
+ __field(int, pid)
+ __field(int, vasid)
+ __field(int, winid)
+ __field(unsigned long, paste_kaddr)
+ ),
+
+ TP_fast_assign(
+ __entry->pid = tsk->pid;
+ __entry->vasid = win->vinst->vas_id;
+ __entry->winid = win->winid;
+ __entry->paste_kaddr = (unsigned long)win->paste_kaddr
+ ),
+
+ TP_printk("pid=%d, vasid=%d, winid=%d, paste_kaddr=0x%016lx\n",
+ __entry->pid, __entry->vasid, __entry->winid,
+ __entry->paste_kaddr)
+);
+
+#endif /* _VAS_TRACE_H */
+
+#undef TRACE_INCLUDE_PATH
+#define TRACE_INCLUDE_PATH .
+#define TRACE_INCLUDE_FILE vas-trace
+#include
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 2b3eb01..6b2de9e 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -21,6 +21,9 @@
#include "vas.h"
#include "copy-paste.h"
+#define CREATE_TRACE_POINTS
+#include "vas-trace.h"
[PATCH 2/2] powerpc: export set_thread_uses_vas()
Signed-off-by: Sukadev Bhattiprolu --- arch/powerpc/kernel/process.c | 1 + 1 file changed, 1 insertion(+) diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c index f20c1ad..d22055b 100644 --- a/arch/powerpc/kernel/process.c +++ b/arch/powerpc/kernel/process.c @@ -1475,6 +1475,7 @@ int set_thread_uses_vas(void) #endif /* CONFIG_PPC_BOOK3S_64 */ return 0; } +EXPORT_SYMBOL_GPL(set_thread_uses_vas); #ifdef CONFIG_PPC64 static DEFINE_SPINLOCK(vas_thread_id_lock); -- 2.7.4
[PATCH 1/2] powerpc: export thread-tidr interfaces
Export set_thread_tidr() and clear_thread_tidr() interfaces so they
can be used by external modules.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/kernel/process.c | 2 ++
1 file changed, 2 insertions(+)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 2010e4c..f20c1ad 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1560,6 +1560,7 @@ void clear_thread_tidr(struct task_struct *t)
free_thread_tidr(t->thread.tidr);
t->thread.tidr = 0;
}
+EXPORT_SYMBOL_GPL(clear_thread_tidr);
void arch_release_task_struct(struct task_struct *t)
{
@@ -1592,6 +1593,7 @@ int set_thread_tidr(struct task_struct *t)
return 0;
}
+EXPORT_SYMBOL_GPL(set_thread_tidr);
#endif /* CONFIG_PPC64 */
--
2.7.4
[PATCH 1/1] powerpc: Emulate paste instruction
From: Michael Neuling
On POWER9 DD2.1 and below there are issues when the paste instruction
generates an error. If an error occurs when thread reconfiguration
happens (ie another thread in the core goes into/out of powersave) the
core may hang.
To avoid this a special sequence is required which stops thread
configuration so that the paste can be safely executed.
This patch assumes paste executed in userspace are trapped into the
illegal instruction exception at 0xe40.
Here we re-execute the paste instruction but with the required
sequence to ensure thread reconfiguration doesn't occur.
Signed-off-by: Michael Neuling
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v4]:
- We need to disable pagefaults after all when modifying the thread
reconfig registers. Use a mutex, rather than a spinlock around
the thread reconfig registers. Acquire the mutex first then block
interrupts so we don't sleep on the mutex with interrupts disabled.
Changlog[v3]:
- [Michael Ellerman] We don't need to disable/enable pagefaults
when emulating paste;
- [Michael Ellerman, Aneesh Kumar] Fix retval from emulate_paste()
Changelog[v2]:
[Sukadev]: Use PPC_PASTE() rather than the paste instruction since
in older versions the instruction required a third parameter.
---
arch/powerpc/include/asm/emulated_ops.h | 1 +
arch/powerpc/include/asm/ppc-opcode.h | 1 +
arch/powerpc/include/asm/reg.h | 2 +
arch/powerpc/kernel/traps.c | 73 +
4 files changed, 77 insertions(+)
diff --git a/arch/powerpc/include/asm/emulated_ops.h
b/arch/powerpc/include/asm/emulated_ops.h
index 651e135..fdc95cf 100644
--- a/arch/powerpc/include/asm/emulated_ops.h
+++ b/arch/powerpc/include/asm/emulated_ops.h
@@ -59,6 +59,7 @@ extern struct ppc_emulated {
struct ppc_emulated_entry lxvh8x;
struct ppc_emulated_entry lxvd2x;
struct ppc_emulated_entry lxvb16x;
+ struct ppc_emulated_entry paste;
#endif
} ppc_emulated;
diff --git a/arch/powerpc/include/asm/ppc-opcode.h
b/arch/powerpc/include/asm/ppc-opcode.h
index ce0930d..a55d2ef 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -229,6 +229,7 @@
#define PPC_INST_MTTMR 0x7c0003dc
#define PPC_INST_NOP 0x6000
#define PPC_INST_PASTE 0x7c20070d
+#define PPC_INST_PASTE_MASK0xfc2007ff
#define PPC_INST_POPCNTB 0x7cf4
#define PPC_INST_POPCNTB_MASK 0xfc0007fe
#define PPC_INST_POPCNTD 0x7c0003f4
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index b779f3c..3495ecf 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -469,6 +469,8 @@
#define SPRN_DBAT7U0x23E /* Data BAT 7 Upper Register */
#define SPRN_PPR 0x380 /* SMT Thread status Register */
#define SPRN_TSCR 0x399 /* Thread Switch Control Register */
+#define SPRN_TRIG1 0x371 /* WAT Trigger 1 */
+#define SPRN_TRIG2 0x372 /* WAT Trigger 2 */
#define SPRN_DEC 0x016 /* Decrement Register */
#define SPRN_DER 0x095 /* Debug Enable Register */
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index f3eb61b..e1ea3be 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -1153,6 +1153,74 @@ static inline bool tm_abort_check(struct pt_regs *regs,
int reason)
}
#endif
+static DEFINE_MUTEX(paste_emulation_mutex);
+
+static inline int paste(void *i)
+{
+ int cr;
+ long retval = 0;
+
+ /* Need per core lock to ensure trig1/2 writes don't race */
+ mutex_lock(&paste_emulation_mutex);
+
+ hard_irq_disable();
+
+ mtspr(SPRN_TRIG1, 0); /* data doesn't matter */
+ mtspr(SPRN_TRIG1, 0); /* HW says do this twice */
+ asm volatile(
+ "1: " PPC_PASTE(0, %2) "\n"
+ "2: mfcr %1\n"
+ ".section .fixup,\"ax\"\n"
+ "3: li %0,%3\n"
+ " li %2,0\n"
+ " b 2b\n"
+ ".previous\n"
+ EX_TABLE(1b, 3b)
+ : "=r" (retval), "=r" (cr)
+ : "b" (i), "i" (-EFAULT), "0" (retval));
+ mtspr(SPRN_TRIG2, 0);
+
+ local_irq_enable();
+
+ mutex_unlock(&paste_emulation_mutex);
+
+ return retval ? retval : cr;
+}
+
+static int emulate_paste(struct pt_regs *regs, u32 instword)
+{
+ const void __user *addr;
+ unsigned long ea;
+ u8 ra, rb;
+ int rc;
+
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EINVAL;
+
+ ra = (instword >> 16) & 0x1f;
+ rb = (instwo
[PATCH 1/1] vas: vas_window_init_dbgdir: fix order of cleanup.
Fix the order of cleanup to ensure we free the name buffer in case of an error creating 'hvwc' or 'info' files. Signed-off-by: Sukadev Bhattiprolu --- arch/powerpc/platforms/powernv/vas-debug.c | 8 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/arch/powerpc/platforms/powernv/vas-debug.c b/arch/powerpc/platforms/powernv/vas-debug.c index ca22f1e..b4de4c6 100644 --- a/arch/powerpc/platforms/powernv/vas-debug.c +++ b/arch/powerpc/platforms/powernv/vas-debug.c @@ -166,13 +166,13 @@ void vas_window_init_dbgdir(struct vas_window *window) return; -free_name: - kfree(window->dbgname); - window->dbgname = NULL; - remove_dir: debugfs_remove_recursive(window->dbgdir); window->dbgdir = NULL; + +free_name: + kfree(window->dbgname); + window->dbgname = NULL; } void vas_instance_init_dbgdir(struct vas_instance *vinst) -- 2.7.4
[PATCH 1/1]: powerpc: block interrupts when updating TIDR
From: Sukadev Bhattiprolu
Date: Tue, 28 Nov 2017 13:39:43 -0600
Subject: [PATCH 1/1]: powerpc: block interrupts when updating TIDR
clear_thread_tidr() is called in interrupt context as a part of delayed
put of the task structure (i.e as a part of timer interrupt). To prevent
a deadlock, block interrupts when holding vas_thread_id_lock to set/
clear TIDR for a task.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/kernel/process.c | 15 +--
1 file changed, 9 insertions(+), 6 deletions(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index bfdd783..aa8dbb9 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1509,14 +1509,15 @@ static int assign_thread_tidr(void)
{
int index;
int err;
+ unsigned long flags;
again:
if (!ida_pre_get(&vas_thread_ida, GFP_KERNEL))
return -ENOMEM;
- spin_lock(&vas_thread_id_lock);
+ spin_lock_irqsave(&vas_thread_id_lock, flags);
err = ida_get_new_above(&vas_thread_ida, 1, &index);
- spin_unlock(&vas_thread_id_lock);
+ spin_unlock_irqrestore(&vas_thread_id_lock, flags);
if (err == -EAGAIN)
goto again;
@@ -1524,9 +1525,9 @@ static int assign_thread_tidr(void)
return err;
if (index > MAX_THREAD_CONTEXT) {
- spin_lock(&vas_thread_id_lock);
+ spin_lock_irqsave(&vas_thread_id_lock, flags);
ida_remove(&vas_thread_ida, index);
- spin_unlock(&vas_thread_id_lock);
+ spin_unlock_irqrestore(&vas_thread_id_lock, flags);
return -ENOMEM;
}
@@ -1535,9 +1536,11 @@ static int assign_thread_tidr(void)
static void free_thread_tidr(int id)
{
- spin_lock(&vas_thread_id_lock);
+ unsigned long flags;
+
+ spin_lock_irqsave(&vas_thread_id_lock, flags);
ida_remove(&vas_thread_ida, id);
- spin_unlock(&vas_thread_id_lock);
+ spin_unlock_irqrestore(&vas_thread_id_lock, flags);
}
/*
--
2.7.4
[PATCH] powerpc/vas, export chip_to_vas_id()
>From 958f8db089f4b89407fc4b89bccd3eaef585aa96 Mon Sep 17 00:00:00 2001
From: Sukadev Bhattiprolu
Date: Mon, 20 Nov 2017 12:53:15 -0600
Subject: [PATCH 1/1] powerpc/vas, export chip_to_vas_id()
Export the symbol chip_to_vas_id() to fix a build failure when
CONFIG_CRYPTO_DEV_NX_COMPRESS_POWERNV=m.
Reported-by: Haren Myneni
Reported-by: Josh Boyer
Signed-off-by: Sukadev Bhattiprolu
---
This was broken by the patch https://lkml.org/lkml/2017/11/7/915.
---
arch/powerpc/platforms/powernv/vas.c | 1 +
1 file changed, 1 insertion(+)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index c488621..aebbe95 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -135,6 +135,7 @@ int chip_to_vas_id(int chipid)
}
return -1;
}
+EXPORT_SYMBOL(chip_to_vas_id);
static int vas_probe(struct platform_device *pdev)
{
--
2.7.4
[tip:perf/core] perf vendor events powerpc: Update POWER9 events
Commit-ID: e795dd42b716ff36ebaa5384fd1be8458d6c9c34
Gitweb: https://git.kernel.org/tip/e795dd42b716ff36ebaa5384fd1be8458d6c9c34
Author: Sukadev Bhattiprolu
AuthorDate: Wed, 8 Nov 2017 18:42:03 -0500
Committer: Arnaldo Carvalho de Melo
CommitDate: Thu, 16 Nov 2017 14:49:54 -0300
perf vendor events powerpc: Update POWER9 events
The POWER9 hardware has dropped support for several events, added
a few new events and changed the category for a couple of events.
Update the POWER9 events in Linux to reflect these changes.
Signed-off-by: Sukadev Bhattiprolu
Cc: Jiri Olsa
Cc: Michael Ellerman
Cc: Madhavan Srinivasan
Link: http://lkml.kernel.org/r/20171108201938.ga10...@us.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo
---
.../perf/pmu-events/arch/powerpc/power9/cache.json | 5 -
.../pmu-events/arch/powerpc/power9/frontend.json | 7 +-
.../pmu-events/arch/powerpc/power9/marked.json | 27 +-
.../perf/pmu-events/arch/powerpc/power9/other.json | 276 ++---
.../pmu-events/arch/powerpc/power9/pipeline.json | 14 +-
tools/perf/pmu-events/arch/powerpc/power9/pmc.json | 2 +-
.../arch/powerpc/power9/translation.json | 5 -
7 files changed, 88 insertions(+), 248 deletions(-)
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/cache.json
b/tools/perf/pmu-events/arch/powerpc/power9/cache.json
index 18f6645..7945c51 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/cache.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/cache.json
@@ -125,11 +125,6 @@
"BriefDescription": "Finish stall because the NTF instruction was a larx
waiting to be satisfied"
},
{,
-"EventCode": "0x3006C",
-"EventName": "PM_RUN_CYC_SMT2_MODE",
-"BriefDescription": "Cycles in which this thread's run latch is set and
the core is in SMT2 mode"
- },
- {,
"EventCode": "0x1C058",
"EventName": "PM_DTLB_MISS_16G",
"BriefDescription": "Data TLB Miss page size 16G"
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
b/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
index c63a919..bd8361b 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
@@ -1,10 +1,5 @@
[
{,
-"EventCode": "0x3E15C",
-"EventName": "PM_MRK_L2_TM_ST_ABORT_SISTER",
-"BriefDescription": "TM marked store abort for this thread"
- },
- {,
"EventCode": "0x25044",
"EventName": "PM_IPTEG_FROM_L31_MOD",
"BriefDescription": "A Page Table Entry was loaded into the TLB with
Modified (M) data from another core's L3 on the same chip due to a instruction
side request"
@@ -369,4 +364,4 @@
"EventName": "PM_IPTEG_FROM_L31_ECO_MOD",
"BriefDescription": "A Page Table Entry was loaded into the TLB with
Modified (M) data from another core's ECO L3 on the same chip due to a
instruction side request"
}
-]
+]
\ No newline at end of file
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/marked.json
b/tools/perf/pmu-events/arch/powerpc/power9/marked.json
index b9df54f..22f9f32 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/marked.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/marked.json
@@ -1,10 +1,5 @@
[
{,
-"EventCode": "0x3C052",
-"EventName": "PM_DATA_SYS_PUMP_MPRED",
-"BriefDescription": "Final Pump Scope (system) mispredicted. Either the
original scope was too small (Chip/Group) or the original scope was System and
it should have been smaller. Counts for a demand load"
- },
- {,
"EventCode": "0x3013E",
"EventName": "PM_MRK_STALL_CMPLU_CYC",
"BriefDescription": "Number of cycles the marked instruction is
experiencing a stall while it is next to complete (NTC)"
@@ -255,6 +250,11 @@
"BriefDescription": "A Page Directory Entry was reloaded to a level 1 page
walk cache from the core's L3 data cache"
},
{,
+"EventCode": "0x3C052",
+"EventName": "PM_DATA_SYS_PUMP_MPRED",
+"BriefDescription": "Final Pump Scope (system) mispredicted. Either the
original scope was too small (Chip/Group) or the original scope was System and
it should have been smaller. Counts for a demand load"
+ },
+ {,
"EventCode": "0x4D142",
"EventName": "PM_MRK_DATA_FROM_L3",
"BriefDescription": "The processor's data cache was reloaded from local
core's L3 due to a marked load"
@@ -435,2
[GIT PULL] Please pull JSON files for POWR9 PMU events
Hi Arnaldo, Please pull an update to the JSON files for POWER9 PMU events. The following changes since commit 148b43a3540bf25875bb5ab695a446950dc8d559: tools headers: Synchronize kernel ABI headers wrt SPDX tags (2017-11-07 13:41:35 -0300) are available in the git repository at: https://github.com/sukadev/linux p9-json-v4 for you to fetch changes up to 4afb062d7d306bf56dbae9b5291e3515ccfede4c: perf vendor events powerpc: Update POWER9 events (2017-11-08 18:42:03 -0500) Sukadev Bhattiprolu (1): perf vendor events powerpc: Update POWER9 events .../perf/pmu-events/arch/powerpc/power9/cache.json | 5 - .../pmu-events/arch/powerpc/power9/frontend.json | 7 +- .../pmu-events/arch/powerpc/power9/marked.json | 27 +- .../perf/pmu-events/arch/powerpc/power9/other.json | 276 ++--- .../pmu-events/arch/powerpc/power9/pipeline.json | 14 +- tools/perf/pmu-events/arch/powerpc/power9/pmc.json | 2 +- .../arch/powerpc/power9/translation.json | 5 - 7 files changed, 88 insertions(+), 248 deletions(-)
[PATCH v3 03/18] powerpc/vas: Cleanup some debug code
Clean up vas.h and the debug code around ifdef vas_debug.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v3]
- Minor tweak to a debug message
---
arch/powerpc/platforms/powernv/vas-window.c | 8 +++--
arch/powerpc/platforms/powernv/vas.h| 54 ++---
2 files changed, 17 insertions(+), 45 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a2fe120..67ffc5d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -726,7 +726,10 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
static bool rx_win_args_valid(enum vas_cop_type cop,
struct vas_rx_win_attr *attr)
{
- dump_rx_win_attr(attr);
+ pr_debug("Rxattr: fault %d, notify %d, intr %d, early %d, fifo %d\n",
+ attr->fault_win, attr->notify_disable,
+ attr->intr_disable, attr->notify_early,
+ attr->rx_fifo_size);
if (cop >= VAS_COP_TYPE_MAX)
return false;
@@ -1050,7 +1053,8 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
else
rc = -EINVAL;
- print_fifo_msg_count(txwin);
+ pr_debug("Txwin #%d: Msg count %llu\n", txwin->winid,
+ read_hvwc_reg(txwin, VREG(LRFIFO_PUSH)));
return rc;
}
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index fea0de4..63e8e03 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -259,6 +259,16 @@
#define VAS_NX_UTIL_ADDER PPC_BITMASK(32, 63)
/*
+ * VREG(x):
+ * Expand a register's short name (eg: LPID) into two parameters:
+ * - the register's short name in string form ("LPID"), and
+ * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
+ * register's offset in the window context
+ */
+#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
+#define VREG(r)VREG_SFX(r, _OFFSET)
+
+/*
* Local Notify Scope Control Register. (Receive windows only).
*/
enum vas_notify_scope {
@@ -385,43 +395,15 @@ struct vas_winctx {
extern struct vas_instance *find_vas_instance(int vasid);
-/*
- * VREG(x):
- * Expand a register's short name (eg: LPID) into two parameters:
- * - the register's short name in string form ("LPID"), and
- * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
- * register's offset in the window context
- */
-#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
-#define VREG(r)VREG_SFX(r, _OFFSET)
-
-#ifdef vas_debug
-static inline void dump_rx_win_attr(struct vas_rx_win_attr *attr)
-{
- pr_err("fault %d, notify %d, intr %d early %d\n",
- attr->fault_win, attr->notify_disable,
- attr->intr_disable, attr->notify_early);
-
- pr_err("rx_fifo_size %d, max value %d\n",
- attr->rx_fifo_size, VAS_RX_FIFO_SIZE_MAX);
-}
-
static inline void vas_log_write(struct vas_window *win, char *name,
void *regptr, u64 val)
{
if (val)
- pr_err("%swin #%d: %s reg %p, val 0x%016llx\n",
+ pr_debug("%swin #%d: %s reg %p, val 0x%016llx\n",
win->tx_win ? "Tx" : "Rx", win->winid, name,
regptr, val);
}
-#else /* vas_debug */
-
-#define vas_log_write(win, name, reg, val)
-#define dump_rx_win_attr(attr)
-
-#endif /* vas_debug */
-
static inline void write_uwc_reg(struct vas_window *win, char *name,
s32 reg, u64 val)
{
@@ -450,18 +432,4 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
-#ifdef vas_debug
-
-static void print_fifo_msg_count(struct vas_window *txwin)
-{
- uint64_t read_hvwc_reg(struct vas_window *w, char *n, uint64_t o);
- pr_devel("Winid %d, Msg count %llu\n", txwin->winid,
- (uint64_t)read_hvwc_reg(txwin, VREG(LRFIFO_PUSH)));
-}
-#else /* vas_debug */
-
-#define print_fifo_msg_count(window)
-
-#endif /* vas_debug */
-
#endif /* _VAS_H */
--
2.7.4
[PATCH v3 08/18] powerpc/vas: poll for return of window credits
Normally, the NX driver waits for the CRBs to be processed before closing
the window. But it is better to ensure that the credits are returned before
the window gets reassigned later.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 45 +
1 file changed, 45 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a59a187..23c13a7 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1063,6 +1063,49 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
EXPORT_SYMBOL_GPL(vas_paste_crb);
/*
+ * If credit checking is enabled for this window, poll for the return
+ * of window credits (i.e for NX engines to process any outstanding CRBs).
+ * Since NX-842 waits for the CRBs to be processed before closing the
+ * window, we should not have to wait for too long.
+ *
+ * TODO: We retry in 10ms intervals now. We could/should probably peek at
+ * the VAS_LRFIFO_PUSH_OFFSET register to get an estimate of pending
+ * CRBs on the FIFO and compute the delay dynamically on each retry.
+ * But that is not really needed until we support NX-GZIP access from
+ * user space. (NX-842 driver waits for CSB and Fast thread-wakeup
+ * doesn't use credit checking).
+ */
+static void poll_window_credits(struct vas_window *window)
+{
+ u64 val;
+ int creds, mode;
+
+ val = read_hvwc_reg(window, VREG(WINCTL));
+ if (window->tx_win)
+ mode = GET_FIELD(VAS_WINCTL_TX_WCRED_MODE, val);
+ else
+ mode = GET_FIELD(VAS_WINCTL_RX_WCRED_MODE, val);
+
+ if (!mode)
+ return;
+retry:
+ if (window->tx_win) {
+ val = read_hvwc_reg(window, VREG(TX_WCRED));
+ creds = GET_FIELD(VAS_TX_WCRED, val);
+ } else {
+ val = read_hvwc_reg(window, VREG(LRX_WCRED));
+ creds = GET_FIELD(VAS_LRX_WCRED, val);
+ }
+
+ if (creds < window->wcreds_max) {
+ val = 0;
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(msecs_to_jiffies(10));
+ goto retry;
+ }
+}
+
+/*
* Wait for the window to go to "not-busy" state. It should only take a
* short time to queue a CRB, so window should not be busy for too long.
* Trying 5ms intervals.
@@ -1149,6 +1192,8 @@ int vas_win_close(struct vas_window *window)
unpin_close_window(window);
+ poll_window_credits(window);
+
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
--
2.7.4
[PATCH v3 04/18] powerpc/vas: Drop poll_window_cast_out().
Polling for window cast out is listed in the spec, but turns out that
it is not strictly necessary and slows down window close. Making it a
stub for now.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 34 ++---
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 67ffc5d..8ab8a82 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1079,25 +1079,25 @@ static void poll_window_busy_state(struct vas_window
*window)
}
}
+/*
+ * Have the hardware cast a window out of cache and wait for it to
+ * be completed.
+ *
+ * NOTE: It can take a relatively long time to cast the window context
+ * out of the cache. It is not strictly necessary to cast out if:
+ *
+ * - we clear the "Pin Window" bit (so hardware is free to evict)
+ *
+ * - we re-initialize the window context when it is reassigned.
+ *
+ * We do the former in vas_win_close() and latter in vas_win_open().
+ * So, ignoring the cast-out for now. We can add it as needed. If
+ * casting out becomes necessary we should consider offloading the
+ * job to a worker thread, so the window close can proceed quickly.
+ */
static void poll_window_castout(struct vas_window *window)
{
- int cached;
- u64 val;
-
- /* Cast window context out of the cache */
-retry:
- val = read_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL));
- cached = GET_FIELD(VAS_WIN_CACHE_STATUS, val);
- if (cached) {
- val = 0ULL;
- val = SET_FIELD(VAS_CASTOUT_REQ, val, 1);
- val = SET_FIELD(VAS_PUSH_TO_MEM, val, 0);
- write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), val);
-
- set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(HZ);
- goto retry;
- }
+ /* stub for now */
}
/*
--
2.7.4
[PATCH v3 07/18] powerpc/vas: Save configured window credits
Save the configured max window credits for a window in the vas_window
structure. We will need this when polling for return of window credits.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 --
arch/powerpc/platforms/powernv/vas.h| 1 +
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1422cdd..a59a187 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -674,7 +674,7 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->rx_fifo = rxattr->rx_fifo;
winctx->rx_fifo_size = rxattr->rx_fifo_size;
- winctx->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ winctx->wcreds_max = rxwin->wcreds_max;
winctx->pin_win = rxattr->pin_win;
winctx->nx_win = rxattr->nx_win;
@@ -844,6 +844,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
+ rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
if (rxattr->user_win)
rxwin->pid = task_pid_vnr(current);
@@ -893,7 +894,7 @@ static void init_winctx_for_txwin(struct vas_window *txwin,
*/
memset(winctx, 0, sizeof(struct vas_winctx));
- winctx->wcreds_max = txattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ winctx->wcreds_max = txwin->wcreds_max;
winctx->user_win = txattr->user_win;
winctx->nx_win = txwin->rxwin->nx_win;
@@ -978,6 +979,7 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->nx_win = txwin->rxwin->nx_win;
txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
+ txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
init_winctx_for_txwin(txwin, attr, &winctx);
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 63e8e03..02d8a31 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -332,6 +332,7 @@ struct vas_window {
void *hvwc_map; /* HV window context */
void *uwc_map; /* OS/User window context */
pid_t pid; /* Linux process id of owner */
+ int wcreds_max; /* Window credits */
/* Fields applicable only to send windows */
void *paste_kaddr;
--
2.7.4
[PATCH v3 05/18] powerpc/vas: Use helper to unpin/close window
Use a helper to have the hardware unpin and mark a window closed.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 22 +++---
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8ab8a82..95622a9 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1101,6 +1101,20 @@ static void poll_window_castout(struct vas_window
*window)
}
/*
+ * Unpin and close a window so no new requests are accepted and the
+ * hardware can evict this window from cache if necessary.
+ */
+static void unpin_close_window(struct vas_window *window)
+{
+ u64 val;
+
+ val = read_hvwc_reg(window, VREG(WINCTL));
+ val = SET_FIELD(VAS_WINCTL_PIN, val, 0);
+ val = SET_FIELD(VAS_WINCTL_OPEN, val, 0);
+ write_hvwc_reg(window, VREG(WINCTL), val);
+}
+
+/*
* Close a window.
*
* See Section 1.12.1 of VAS workbook v1.05 for details on closing window:
@@ -1114,8 +1128,6 @@ static void poll_window_castout(struct vas_window *window)
*/
int vas_win_close(struct vas_window *window)
{
- u64 val;
-
if (!window)
return 0;
@@ -1131,11 +1143,7 @@ int vas_win_close(struct vas_window *window)
poll_window_busy_state(window);
- /* Unpin window from cache and close it */
- val = read_hvwc_reg(window, VREG(WINCTL));
- val = SET_FIELD(VAS_WINCTL_PIN, val, 0);
- val = SET_FIELD(VAS_WINCTL_OPEN, val, 0);
- write_hvwc_reg(window, VREG(WINCTL), val);
+ unpin_close_window(window);
poll_window_castout(window);
--
2.7.4
[PATCH v3 09/18] powerpc/vas: Create cpu to vas id mapping
Create a cpu to vasid mapping so callers can specify -1 instead of
trying to find a VAS id.
Changelog[v2]
[Michael Ellerman] Use per-cpu variables to simplify code.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas.c | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index 565a487..abb7090 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -18,15 +18,18 @@
#include
#include
#include
+#include
#include "vas.h"
static DEFINE_MUTEX(vas_mutex);
static LIST_HEAD(vas_instances);
+static DEFINE_PER_CPU(int, cpu_vas_id);
+
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, vasid;
+ int rc, cpu, vasid;
struct resource *res;
struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
@@ -74,6 +77,11 @@ static int init_vas_instance(struct platform_device *pdev)
"paste_win_id_shift 0x%llx\n", pdev->name, vasid,
vinst->paste_base_addr, vinst->paste_win_id_shift);
+ for_each_possible_cpu(cpu) {
+ if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
+ per_cpu(cpu_vas_id, cpu) = vasid;
+ }
+
mutex_lock(&vas_mutex);
list_add(&vinst->node, &vas_instances);
mutex_unlock(&vas_mutex);
@@ -98,6 +106,10 @@ struct vas_instance *find_vas_instance(int vasid)
struct vas_instance *vinst;
mutex_lock(&vas_mutex);
+
+ if (vasid == -1)
+ vasid = per_cpu(cpu_vas_id, smp_processor_id());
+
list_for_each(ent, &vas_instances) {
vinst = list_entry(ent, struct vas_instance, node);
if (vinst->vas_id == vasid) {
--
2.7.4
[PATCH v3 12/18] powerpc: have copy depend on CONFIG_BOOK3S_64
Have the COPY/PASTE instructions depend on CONFIG_BOOK3S_64 rather than
CONFIG_PPC_STD_MMU_64.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/kernel/process.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index a0c74bb..37ed60b 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1215,10 +1215,14 @@ struct task_struct *__switch_to(struct task_struct
*prev,
batch = this_cpu_ptr(&ppc64_tlb_batch);
batch->active = 1;
}
+#endif /* CONFIG_PPC_STD_MMU_64 */
if (current_thread_info()->task->thread.regs) {
+#ifdef CONFIG_PPC_STD_MMU_64
restore_math(current_thread_info()->task->thread.regs);
+#endif /* CONFIG_PPC_STD_MMU_64 */
+#ifdef CONFIG_PPC_BOOK3S_64
/*
* The copy-paste buffer can only store into foreign real
* addresses, so unprivileged processes can not see the
@@ -1237,8 +1241,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
asm volatile(PPC_COPY(%0, %1)
: : "r"(dummy_copy_buffer), "r"(0));
}
+#endif /* CONFIG_PPC_BOOK3S_64 */
}
-#endif /* CONFIG_PPC_STD_MMU_64 */
return last;
}
--
2.7.4
[PATCH v3 11/18] powerpc/vas: Export HVWC to debugfs
Export the VAS Window context information to debugfs.
We need to hold a mutex when closing the window to prevent a race
with the debugfs read(). Rather than introduce a per-instance mutex,
we use the global vas_mutex for now, since it is not heavily contended.
The window->cop field is only relevant to a receive window so we were
not setting it for a send window (which is is paired to a receive window
anyway). But to simplify reporting in debugfs, set the 'cop' field for the
send window also.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v3]:
- [Michael Ellerman] Fix couple of unininitialized variables
---
arch/powerpc/platforms/powernv/Makefile | 3 +-
arch/powerpc/platforms/powernv/vas-debug.c | 209
arch/powerpc/platforms/powernv/vas-window.c | 34 -
arch/powerpc/platforms/powernv/vas.c| 6 +-
arch/powerpc/platforms/powernv/vas.h| 14 ++
5 files changed, 257 insertions(+), 9 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-debug.c
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index 7a31c26..3732118 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -15,4 +15,5 @@ obj-$(CONFIG_TRACEPOINTS) += opal-tracepoints.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_FTW) += nx-ftw.o
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
new file mode 100644
index 000..ca22f1e
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -0,0 +1,209 @@
+/*
+ * Copyright 2016-17 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include "vas.h"
+
+static struct dentry *vas_debugfs;
+
+static char *cop_to_str(int cop)
+{
+ switch (cop) {
+ case VAS_COP_TYPE_FAULT:return "Fault";
+ case VAS_COP_TYPE_842: return "NX-842 Normal Priority";
+ case VAS_COP_TYPE_842_HIPRI:return "NX-842 High Priority";
+ case VAS_COP_TYPE_GZIP: return "NX-GZIP Normal Priority";
+ case VAS_COP_TYPE_GZIP_HIPRI: return "NX-GZIP High Priority";
+ case VAS_COP_TYPE_FTW: return "Fast Thread-wakeup";
+ default:return "Unknown";
+ }
+}
+
+static int info_dbg_show(struct seq_file *s, void *private)
+{
+ struct vas_window *window = s->private;
+
+ mutex_lock(&vas_mutex);
+
+ /* ensure window is not unmapped */
+ if (!window->hvwc_map)
+ goto unlock;
+
+ seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
+ window->tx_win ? "Send" : "Receive");
+ seq_printf(s, "Pid : %d\n", window->pid);
+
+unlock:
+ mutex_unlock(&vas_mutex);
+ return 0;
+}
+
+static int info_dbg_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, info_dbg_show, inode->i_private);
+}
+
+static const struct file_operations info_fops = {
+ .open = info_dbg_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release= single_release,
+};
+
+static inline void print_reg(struct seq_file *s, struct vas_window *win,
+ char *name, u32 reg)
+{
+ seq_printf(s, "0x%016llx %s\n", read_hvwc_reg(win, name, reg), name);
+}
+
+static int hvwc_dbg_show(struct seq_file *s, void *private)
+{
+ struct vas_window *window = s->private;
+
+ mutex_lock(&vas_mutex);
+
+ /* ensure window is not unmapped */
+ if (!window->hvwc_map)
+ goto unlock;
+
+ print_reg(s, window, VREG(LPID));
+ print_reg(s, window, VREG(PID));
+ print_reg(s, window, VREG(XLATE_MSR));
+ print_reg(s, window, VREG(XLATE_LPCR));
+ print_reg(s, window, VREG(XLATE_CTL));
+ print_reg(s, window, VREG(AMR));
+ print_reg(s, window, VREG(SEIDR));
+ print_reg(s, window, VREG(FAULT_TX_WIN));
+ print_reg(s, window, VREG(OSU_INTR_SRC_RA));
+ print_reg(s, window, VREG(HV_INTR_SRC_RA));
+ print_reg(s, window, VREG(PSWID));
+ print_reg(s, window, VREG(LFIFO_BAR));
+ print_reg(s, window, VREG(LDATA_STAMP_CTL));
+ print_reg(s, window, VREG(LDMA_CACHE_CTL));
+
[PATCH v3 06/18] powerpc/vas: Reduce polling interval for busy state
A VAS window is normally in "busy" state for only a short duration.
Reduce the time we wait for the window to go to "not-busy" state to
speed-up vas_win_close() a bit.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 95622a9..1422cdd 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1060,21 +1060,23 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
}
EXPORT_SYMBOL_GPL(vas_paste_crb);
+/*
+ * Wait for the window to go to "not-busy" state. It should only take a
+ * short time to queue a CRB, so window should not be busy for too long.
+ * Trying 5ms intervals.
+ */
static void poll_window_busy_state(struct vas_window *window)
{
int busy;
u64 val;
retry:
- /*
-* Poll Window Busy flag
-*/
val = read_hvwc_reg(window, VREG(WIN_STATUS));
busy = GET_FIELD(VAS_WIN_BUSY, val);
if (busy) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(HZ);
+ schedule_timeout(msecs_to_jiffies(5));
goto retry;
}
}
--
2.7.4
[PATCH v3 14/18] powerpc: Define set_thread_uses_vas()
A CP_ABORT instruction is required in processes that have mapped a VAS
"paste address" with the intention of using COPY/PASTE instructions.
But since CP_ABORT is expensive, we want to restrict it to only processes
that use/intend to use COPY/PASTE.
Define an interface, set_thread_uses_vas(), that VAS can use to indicate
that the current process opened a send window. During context switch,
issue CP_ABORT only for processes that have the flag set.
Thanks for input from Nick Piggin, Michael Ellerman.
Cc: Nicholas Piggin
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v3]:
- [Nick Piggin] Rename interface to set_thread_uses_vas(), tweak
comment and move the CP_ABORT from callers into the interface.
---
arch/powerpc/include/asm/processor.h | 2 ++
arch/powerpc/include/asm/switch_to.h | 2 ++
arch/powerpc/kernel/process.c| 41 +++-
3 files changed, 35 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index 58cc212..bdab3b74 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -341,7 +341,9 @@ struct thread_struct {
unsigned long sier;
unsigned long mmcr2;
unsignedmmcr0;
+
unsignedused_ebb;
+ unsigned intused_vas;
#endif
};
diff --git a/arch/powerpc/include/asm/switch_to.h
b/arch/powerpc/include/asm/switch_to.h
index ad2d762..c3ca42c 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -92,6 +92,8 @@ static inline void clear_task_ebb(struct task_struct *t)
#endif
}
+extern int set_thread_uses_vas(void);
+
extern int set_thread_tidr(struct task_struct *t);
extern void clear_thread_tidr(struct task_struct *t);
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index d861fcd..395ca80 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1234,17 +1234,17 @@ struct task_struct *__switch_to(struct task_struct
*prev,
* The copy-paste buffer can only store into foreign real
* addresses, so unprivileged processes can not see the
* data or use it in any way unless they have foreign real
-* mappings. We don't have a VAS driver that allocates those
-* yet, so no cpabort is required.
+* mappings. If the new process has the foreign real address
+* mappings, we must issue a cp_abort to clear any state and
+* prevent snooping, corruption or a covert channel.
+*
+* DD1 allows paste into normal system memory so we do an
+* unpaired copy, rather than cp_abort, to clear the buffer,
+* since cp_abort is quite expensive.
*/
- if (cpu_has_feature(CPU_FTR_POWER9_DD1)) {
- /*
-* DD1 allows paste into normal system memory, so we
-* do an unpaired copy here to clear the buffer and
-* prevent a covert channel being set up.
-*
-* cpabort is not used because it is quite expensive.
-*/
+ if (new_thread->used_vas) {
+ asm volatile(PPC_CP_ABORT);
+ } else if (cpu_has_feature(CPU_FTR_POWER9_DD1)) {
asm volatile(PPC_COPY(%0, %1)
: : "r"(dummy_copy_buffer), "r"(0));
}
@@ -1445,6 +1445,27 @@ void flush_thread(void)
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
}
+int set_thread_uses_vas(void)
+{
+#ifdef CONFIG_PPC_BOOK3S_64
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EINVAL;
+
+ current->thread.used_vas = 1;
+
+ /*
+* Even a process that has no foreign real address mapping can use
+* an unpaired COPY instruction (to no real effect). Issue CP_ABORT
+* to clear any pending COPY and prevent a covert channel.
+*
+* __switch_to() will issue CP_ABORT on future context switches.
+*/
+ asm volatile(PPC_CP_ABORT);
+
+#endif /* CONFIG_PPC_BOOK3S_64 */
+ return 0;
+}
+
#ifdef CONFIG_PPC64
static DEFINE_SPINLOCK(vas_thread_id_lock);
static DEFINE_IDA(vas_thread_ida);
--
2.7.4
[PATCH v3 17/18] powerpc/vas: Define vas_win_id()
Define an interface to return a system-wide unique id for a given VAS
window.
The vas_win_id() will be used in a follow-on patch to generate an unique
handle for a user space receive window. Applications can use this handle
to pair send and receive windows for fast thread-wakeup.
The hardware refers to this system-wide unique id as a Partition Send
Window ID which is expected to be used during fault handling. Hence the
"pswid" in the function names.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 5 +
arch/powerpc/platforms/powernv/vas-window.c | 9 +
arch/powerpc/platforms/powernv/vas.h| 28
3 files changed, 42 insertions(+)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index f98ade8..7714562 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -168,6 +168,11 @@ int vas_copy_crb(void *crb, int offset);
int vas_paste_crb(struct vas_window *win, int offset, bool re);
/*
+ * Return a system-wide unique id for the VAS window @win.
+ */
+extern u32 vas_win_id(struct vas_window *win);
+
+/*
* Return the power bus paste address associated with @win so the caller
* can map that address into their address space.
*/
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index d7d0653..8275492 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1235,3 +1235,12 @@ int vas_win_close(struct vas_window *window)
return 0;
}
EXPORT_SYMBOL_GPL(vas_win_close);
+
+/*
+ * Return a system-wide unique window id for the window @win.
+ */
+u32 vas_win_id(struct vas_window *win)
+{
+ return encode_pswid(win->vinst->vas_id, win->winid);
+}
+EXPORT_SYMBOL_GPL(vas_win_id);
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 756cbc5..ae0100f 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -447,4 +447,32 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ u32 pswid = 0;
+
+ pswid |= vasid << (31 - 7);
+ pswid |= winid;
+
+ return pswid;
+}
+
+static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
+{
+ if (vasid)
+ *vasid = pswid >> (31 - 7) & 0xFF;
+
+ if (winid)
+ *winid = pswid & 0x;
+}
#endif /* _VAS_H */
--
2.7.4
[PATCH v3 10/18] powerpc/vas, nx-842: Define and use chip_to_vas_id()
Define a helper, chip_to_vas_id() to map a given chip id to corresponding
vas id.
Normally, callers of vas_rx_win_open() and vas_tx_win_open() want the VAS
window to be on the same chip where the calling thread is executing. These
callers can pass in -1 for the VAS id.
This interface will be useful if a thread running on one chip wants to open
a window on another chip (like the NX-842 driver does during start up).
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 9 +
arch/powerpc/platforms/powernv/vas.c | 11 +++
drivers/crypto/nx/nx-842-powernv.c | 18 +++---
3 files changed, 23 insertions(+), 15 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index fd5963a..044748f 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -104,6 +104,15 @@ struct vas_tx_win_attr {
};
/*
+ * Helper to map a chip id to VAS id.
+ * For POWER9, this is a 1:1 mapping. In the future this maybe a 1:N
+ * mapping in which case, we will need to update this helper.
+ *
+ * Return the VAS id or -1 if no matching vasid is found.
+ */
+int chip_to_vas_id(int chipid);
+
+/*
* Helper to initialize receive window attributes to defaults for an
* NX window.
*/
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index abb7090..cd9a733 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -123,6 +123,17 @@ struct vas_instance *find_vas_instance(int vasid)
return NULL;
}
+int chip_to_vas_id(int chipid)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ if (cpu_to_chip_id(cpu) == chipid)
+ return per_cpu(cpu_vas_id, cpu);
+ }
+ return -1;
+}
+
static int vas_probe(struct platform_device *pdev)
{
return init_vas_instance(pdev);
diff --git a/drivers/crypto/nx/nx-842-powernv.c
b/drivers/crypto/nx/nx-842-powernv.c
index 874ddf5..eb221ed 100644
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ b/drivers/crypto/nx/nx-842-powernv.c
@@ -847,24 +847,12 @@ static int __init nx842_powernv_probe_vas(struct
device_node *pn)
return -EINVAL;
}
- for_each_compatible_node(dn, NULL, "ibm,power9-vas-x") {
- if (of_get_ibm_chip_id(dn) == chip_id)
- break;
- }
-
- if (!dn) {
- pr_err("Missing VAS device node\n");
+ vasid = chip_to_vas_id(chip_id);
+ if (vasid < 0) {
+ pr_err("Unable to map chip_id %d to vasid\n", chip_id);
return -EINVAL;
}
- if (of_property_read_u32(dn, "ibm,vas-id", &vasid)) {
- pr_err("Missing ibm,vas-id device property\n");
- of_node_put(dn);
- return -EINVAL;
- }
-
- of_node_put(dn);
-
for_each_child_of_node(pn, dn) {
if (of_device_is_compatible(dn, "ibm,p9-nx-842")) {
ret = vas_cfg_coproc_info(dn, chip_id, vasid);
--
2.7.4
[PATCH v3 15/18] powerpc: Emulate paste instruction
From: Michael Neuling
On POWER9 DD2.1 and below there are issues when the paste instruction
generates an error. If an error occurs when thread reconfiguration
happens (ie another thread in the core goes into/out of powersave) the
core may hang.
To avoid this a special sequence is required which stops thread
configuration so that the paste can be safely executed.
This patch assumes paste executed in userspace are trapped into the
illegal instruction exception at 0xe40.
Here we re-execute the paste instruction but with the required
sequence to ensure thread reconfiguration doesn't occur.
Cc: Aneesh Kumar K.V
Signed-off-by: Michael Neuling
Signed-off-by: Sukadev Bhattiprolu
---
Changlog[v3]:
- [Michael Ellerman] We don't need to disable/enable pagefaults
when emulating paste;
- [Michael Ellerman, Aneesh Kumar] Fix retval from emulate_paste()
Edit by Sukadev: Use PPC_PASTE() rather than the paste instruction since
in older versions the instruction required a third parameter.
---
arch/powerpc/include/asm/emulated_ops.h | 1 +
arch/powerpc/include/asm/ppc-opcode.h | 1 +
arch/powerpc/include/asm/reg.h | 2 +
arch/powerpc/kernel/traps.c | 67 +
4 files changed, 71 insertions(+)
diff --git a/arch/powerpc/include/asm/emulated_ops.h
b/arch/powerpc/include/asm/emulated_ops.h
index f00e10e..9247af9 100644
--- a/arch/powerpc/include/asm/emulated_ops.h
+++ b/arch/powerpc/include/asm/emulated_ops.h
@@ -55,6 +55,7 @@ extern struct ppc_emulated {
struct ppc_emulated_entry mfdscr;
struct ppc_emulated_entry mtdscr;
struct ppc_emulated_entry lq_stq;
+ struct ppc_emulated_entry paste;
#endif
} ppc_emulated;
diff --git a/arch/powerpc/include/asm/ppc-opcode.h
b/arch/powerpc/include/asm/ppc-opcode.h
index ce0930d..a55d2ef 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -229,6 +229,7 @@
#define PPC_INST_MTTMR 0x7c0003dc
#define PPC_INST_NOP 0x6000
#define PPC_INST_PASTE 0x7c20070d
+#define PPC_INST_PASTE_MASK0xfc2007ff
#define PPC_INST_POPCNTB 0x7cf4
#define PPC_INST_POPCNTB_MASK 0xfc0007fe
#define PPC_INST_POPCNTD 0x7c0003f4
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index b779f3c..3495ecf 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -469,6 +469,8 @@
#define SPRN_DBAT7U0x23E /* Data BAT 7 Upper Register */
#define SPRN_PPR 0x380 /* SMT Thread status Register */
#define SPRN_TSCR 0x399 /* Thread Switch Control Register */
+#define SPRN_TRIG1 0x371 /* WAT Trigger 1 */
+#define SPRN_TRIG2 0x372 /* WAT Trigger 2 */
#define SPRN_DEC 0x016 /* Decrement Register */
#define SPRN_DER 0x095 /* Debug Enable Register */
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index 13c9dcd..c2cce25 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -956,6 +956,68 @@ static inline bool tm_abort_check(struct pt_regs *regs,
int reason)
}
#endif
+static DEFINE_SPINLOCK(paste_emulation_lock);
+
+static inline int paste(void *i)
+{
+ int cr;
+ long retval = 0;
+
+ /* Need per core lock to ensure trig1/2 writes don't race */
+ spin_lock(&paste_emulation_lock);
+ mtspr(SPRN_TRIG1, 0); /* data doesn't matter */
+ mtspr(SPRN_TRIG1, 0); /* HW says do this twice */
+ asm volatile(
+ "1: " PPC_PASTE(0, %2) "\n"
+ "2: mfcr %1\n"
+ ".section .fixup,\"ax\"\n"
+ "3: li %0,%3\n"
+ " li %2,0\n"
+ " b 2b\n"
+ ".previous\n"
+ EX_TABLE(1b, 3b)
+ : "=r" (retval), "=r" (cr)
+ : "b" (i), "i" (-EFAULT), "0" (retval));
+ mtspr(SPRN_TRIG2, 0);
+ spin_unlock(&paste_emulation_lock);
+
+ return retval ?: cr;
+}
+
+static int emulate_paste(struct pt_regs *regs, u32 instword)
+{
+ const void __user *addr;
+ unsigned long ea;
+ u8 ra, rb;
+ int rc;
+
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EINVAL;
+
+ ra = (instword >> 16) & 0x1f;
+ rb = (instword >> 11) & 0x1f;
+
+ ea = regs->gpr[rb] + (ra ? regs->gpr[ra] : 0ul);
+ if (is_32bit_task())
+ ea &= 0xul;
+ addr = (__force const void __user *)ea;
+
+ if (!access_ok(VERIFY_WRITE, addr, 128)) // cacheline size == 128
+ return -EFAULT;
+
+ hard_irq_disable(); /* FIXME: could we jus
[PATCH v3 13/18] powerpc: Add support for setting SPRN_TIDR
We need the SPRN_TIDR to be set for use with fast thread-wakeup (core-
to-core wakeup) and also with CAPI.
Each thread in a process needs to have a unique id within the process.
But as explained below, for now, we assign globally unique thread ids
to all threads in the system.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Philippe Bergheaud
Signed-off-by: Christophe Lombard
---
Changelog[v3]
- Merge changes with and address comments to Christophe's patch.
(i.e drop CONFIG_PPC_VAS; use CONFIG_PPC64; check CPU_ARCH_300
before setting TIDR). Defer following to separate patches:
- emulation parts of Christophe's patch,
- setting TIDR for tasks other than 'current'
- setting feature bit in AT_HWCAP2
Changelog[v2]
- Michael Ellerman: Use an interface to assign TIDR so it is
assigned to only threads that need it; move assignment to
restore_sprs(). Drop lint from rebase;
---
arch/powerpc/include/asm/processor.h | 1 +
arch/powerpc/include/asm/switch_to.h | 3 +
arch/powerpc/kernel/process.c| 122 +++
3 files changed, 126 insertions(+)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index fab7ff8..58cc212 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -329,6 +329,7 @@ struct thread_struct {
*/
int dscr_inherit;
unsigned long ppr;/* used to save/restore SMT priority */
+ unsigned long tidr;
#endif
#ifdef CONFIG_PPC_BOOK3S_64
unsigned long tar;
diff --git a/arch/powerpc/include/asm/switch_to.h
b/arch/powerpc/include/asm/switch_to.h
index bf820f5..ad2d762 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -92,4 +92,7 @@ static inline void clear_task_ebb(struct task_struct *t)
#endif
}
+extern int set_thread_tidr(struct task_struct *t);
+extern void clear_thread_tidr(struct task_struct *t);
+
#endif /* _ASM_POWERPC_SWITCH_TO_H */
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 37ed60b..d861fcd 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1120,6 +1120,13 @@ static inline void restore_sprs(struct thread_struct
*old_thread,
mtspr(SPRN_TAR, new_thread->tar);
}
#endif
+#ifdef CONFIG_PPC64
+ if (old_thread->tidr != new_thread->tidr) {
+ /* TIDR should be non-zero only with ISA3.0. */
+ WARN_ON_ONCE(!cpu_has_feature(CPU_FTR_ARCH_300));
+ mtspr(SPRN_TIDR, new_thread->tidr);
+ }
+#endif
}
#ifdef CONFIG_PPC_BOOK3S_64
@@ -1438,9 +1445,117 @@ void flush_thread(void)
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
}
+#ifdef CONFIG_PPC64
+static DEFINE_SPINLOCK(vas_thread_id_lock);
+static DEFINE_IDA(vas_thread_ida);
+
+/*
+ * We need to assign a unique thread id to each thread in a process.
+ *
+ * This thread id, referred to as TIDR, and separate from the Linux's tgid,
+ * is intended to be used to direct an ASB_Notify from the hardware to the
+ * thread, when a suitable event occurs in the system.
+ *
+ * One such event is a "paste" instruction in the context of Fast Thread
+ * Wakeup (aka Core-to-core wake up in the Virtual Accelerator Switchboard
+ * (VAS) in POWER9.
+ *
+ * To get a unique TIDR per process we could simply reuse task_pid_nr() but
+ * the problem is that task_pid_nr() is not yet available copy_thread() is
+ * called. Fixing that would require changing more intrusive arch-neutral
+ * code in code path in copy_process()?.
+ *
+ * Further, to assign unique TIDRs within each process, we need an atomic
+ * field (or an IDR) in task_struct, which again intrudes into the arch-
+ * neutral code. So try to assign globally unique TIDRs for now.
+ *
+ * NOTE: TIDR 0 indicates that the thread does not need a TIDR value.
+ * For now, only threads that expect to be notified by the VAS
+ * hardware need a TIDR value and we assign values > 0 for those.
+ */
+#define MAX_THREAD_CONTEXT ((1 << 16) - 1)
+static int assign_thread_tidr(void)
+{
+ int index;
+ int err;
+
+again:
+ if (!ida_pre_get(&vas_thread_ida, GFP_KERNEL))
+ return -ENOMEM;
+
+ spin_lock(&vas_thread_id_lock);
+ err = ida_get_new_above(&vas_thread_ida, 1, &index);
+ spin_unlock(&vas_thread_id_lock);
+
+ if (err == -EAGAIN)
+ goto again;
+ else if (err)
+ return err;
+
+ if (index > MAX_THREAD_CONTEXT) {
+ spin_lock(&vas_thread_id_lock);
+ ida_remove(&vas_thread_ida, index);
+ spin_unlock(&vas_thread_id_lock);
+ return -ENOMEM;
+ }
+
+ return index;
+}
+
+static void free_thread_tid
[PATCH v3 02/18] powerpc/vas: Validate window credits
NX-842, the only user of VAS, sets the window credits to default values
but VAS should check the credits against the possible max values.
The VAS_WCREDS_MIN is not needed and can be dropped.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 ++
arch/powerpc/platforms/powernv/vas.h| 4 ++--
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index cec7ab7..a2fe120 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -738,6 +738,9 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
+ if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ return false;
+
if (attr->nx_win) {
/* cannot be fault or user window if it is nx */
if (attr->fault_win || attr->user_win)
@@ -927,6 +930,9 @@ static bool tx_win_args_valid(enum vas_cop_type cop,
if (cop > VAS_COP_TYPE_MAX)
return false;
+ if (attr->wcreds_max > VAS_TX_WCREDS_MAX)
+ return false;
+
if (attr->user_win &&
(cop != VAS_COP_TYPE_FTW || attr->rsvd_txbuf_count))
return false;
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 38dee5d..fea0de4 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -106,8 +106,8 @@
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_WCREDS_MIN 16
-#define VAS_WCREDS_MAX ((64 << 10) - 1)
+#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
+#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
/*
--
2.7.4
[PATCH v3 16/18] powerpc/vas: Define vas_win_paste_addr()
Define an interface that the NX drivers can use to find the physical
paste address of a send window. This interface is expected to be used
with the mmap() operation of the NX driver's device. i.e the user space
process can use driver's mmap() operation to map the send window's paste
address into their address space and then use copy and paste instructions
to submit the CRBs to the NX engine.
Note that kernel drivers will use vas_paste_crb() directly and don't need
this interface.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 7 +++
arch/powerpc/platforms/powernv/vas-window.c | 10 ++
2 files changed, 17 insertions(+)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index 044748f..f98ade8 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -10,6 +10,8 @@
#ifndef _ASM_POWERPC_VAS_H
#define _ASM_POWERPC_VAS_H
+struct vas_window;
+
/*
* Min and max FIFO sizes are based on Version 1.05 Section 3.1.4.25
* (Local FIFO Size Register) of the VAS workbook.
@@ -165,4 +167,9 @@ int vas_copy_crb(void *crb, int offset);
*/
int vas_paste_crb(struct vas_window *win, int offset, bool re);
+/*
+ * Return the power bus paste address associated with @win so the caller
+ * can map that address into their address space.
+ */
+extern u64 vas_win_paste_addr(struct vas_window *win);
#endif /* __ASM_POWERPC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index c030d4c..d7d0653 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -40,6 +40,16 @@ static void compute_paste_address(struct vas_window *window,
u64 *addr, int *len
pr_debug("Txwin #%d: Paste addr 0x%llx\n", winid, *addr);
}
+u64 vas_win_paste_addr(struct vas_window *win)
+{
+ u64 addr;
+
+ compute_paste_address(win, &addr, NULL);
+
+ return addr;
+}
+EXPORT_SYMBOL(vas_win_paste_addr);
+
static inline void get_hvwc_mmio_bar(struct vas_window *window,
u64 *start, int *len)
{
--
2.7.4
[PATCH v3 18/18] powerpc/vas: Add support for user receive window
Add support for user space receive window (for the Fast thread-wakeup
coprocessor type)
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v3]
- [Nick Piggin] Drop CP_ABORT since set_thread_uses_vas() does
that now (in earlier patch) and add a check for return value.
---
arch/powerpc/platforms/powernv/vas-window.c | 56 +
1 file changed, 49 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8275492..2b3eb01 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -16,7 +16,8 @@
#include
#include
#include
-
+#include
+#include
#include "vas.h"
#include "copy-paste.h"
@@ -598,6 +599,32 @@ static void put_rx_win(struct vas_window *rxwin)
}
/*
+ * Find the user space receive window given the @pswid.
+ * - We must have a valid vasid and it must belong to this instance.
+ *(so both send and receive windows are on the same VAS instance)
+ * - The window must refer to an OPEN, FTW, RECEIVE window.
+ *
+ * NOTE: We access ->windows[] table and assume that vinst->mutex is held.
+ */
+static struct vas_window *get_user_rxwin(struct vas_instance *vinst, u32 pswid)
+{
+ int vasid, winid;
+ struct vas_window *rxwin;
+
+ decode_pswid(pswid, &vasid, &winid);
+
+ if (vinst->vas_id != vasid)
+ return ERR_PTR(-EINVAL);
+
+ rxwin = vinst->windows[winid];
+
+ if (!rxwin || rxwin->tx_win || rxwin->cop != VAS_COP_TYPE_FTW)
+ return ERR_PTR(-EINVAL);
+
+ return rxwin;
+}
+
+/*
* Get the VAS receive window associated with NX engine identified
* by @cop and if applicable, @pswid.
*
@@ -610,10 +637,10 @@ static struct vas_window *get_vinst_rxwin(struct
vas_instance *vinst,
mutex_lock(&vinst->mutex);
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI)
- rxwin = vinst->rxwin[cop] ?: ERR_PTR(-EINVAL);
+ if (cop == VAS_COP_TYPE_FTW)
+ rxwin = get_user_rxwin(vinst, pswid);
else
- rxwin = ERR_PTR(-EINVAL);
+ rxwin = vinst->rxwin[cop] ?: ERR_PTR(-EINVAL);
if (!IS_ERR(rxwin))
atomic_inc(&rxwin->num_txwins);
@@ -937,10 +964,9 @@ static void init_winctx_for_txwin(struct vas_window *txwin,
winctx->tx_word_mode = txattr->tx_win_ord_mode;
winctx->rsvd_txbuf_count = txattr->rsvd_txbuf_count;
- if (winctx->nx_win) {
+ winctx->intr_disable = true;
+ if (winctx->nx_win)
winctx->data_stamp = true;
- winctx->intr_disable = true;
- }
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
@@ -985,6 +1011,14 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
if (!tx_win_args_valid(cop, attr))
return ERR_PTR(-EINVAL);
+ /*
+* If caller did not specify a vasid but specified the PSWID of a
+* receive window (applicable only to FTW windows), use the vasid
+* from that receive window.
+*/
+ if (vasid == -1 && attr->pswid)
+ decode_pswid(attr->pswid, &vasid, NULL);
+
vinst = find_vas_instance(vasid);
if (!vinst) {
pr_devel("vasid %d not found!\n", vasid);
@@ -1031,6 +1065,14 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
}
}
+ /*
+* Now that we have a send window, ensure context switch issues
+* CP_ABORT for this thread.
+*/
+ rc = -EINVAL;
+ if (set_thread_uses_vas() < 0)
+ goto free_window;
+
set_vinst_win(vinst, txwin);
return txwin;
--
2.7.4
[PATCH v3 01/18] powerpc/vas: init missing fields from [rt]xattr
Initialize a few missing window context fields from the window attributes
specified by the caller. These fields are currently set to their default
values by the caller (NX-842), but would be good to apply them anyway.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 5aae845..cec7ab7 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -679,10 +679,13 @@ static void init_winctx_for_rxwin(struct vas_window
*rxwin,
winctx->nx_win = rxattr->nx_win;
winctx->fault_win = rxattr->fault_win;
+ winctx->user_win = rxattr->user_win;
+ winctx->rej_no_credit = rxattr->rej_no_credit;
winctx->rx_word_mode = rxattr->rx_win_ord_mode;
winctx->tx_word_mode = rxattr->tx_win_ord_mode;
winctx->rx_wcred_mode = rxattr->rx_wcred_mode;
winctx->tx_wcred_mode = rxattr->tx_wcred_mode;
+ winctx->notify_early = rxattr->notify_early;
if (winctx->nx_win) {
winctx->data_stamp = true;
@@ -889,11 +892,14 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->user_win = txattr->user_win;
winctx->nx_win = txwin->rxwin->nx_win;
winctx->pin_win = txattr->pin_win;
+ winctx->rej_no_credit = txattr->rej_no_credit;
+ winctx->rsvd_txbuf_enable = txattr->rsvd_txbuf_enable;
winctx->rx_wcred_mode = txattr->rx_wcred_mode;
winctx->tx_wcred_mode = txattr->tx_wcred_mode;
winctx->rx_word_mode = txattr->rx_win_ord_mode;
winctx->tx_word_mode = txattr->tx_win_ord_mode;
+ winctx->rsvd_txbuf_count = txattr->rsvd_txbuf_count;
if (winctx->nx_win) {
winctx->data_stamp = true;
--
2.7.4
[PATCH v3 00/18] powerpc/vas: Add support for FTW
The first 10 patches in this set sanitize cpu/chip id to VAS id mapping, improve vas_win_close() performance, add a check for return of credits and cleans up some code. Patch 11 adds debugfs support for the VAS window contexts. Patches 12-18 add support for user space aka Fast thread-wakeup windows in VAS. These include a patch from Michael Neuling to support emulating the paste instruction. Changelog[v3]: - [Michael Ellerman] We don't need to disable/enable pagefaults when emulating paste; - [Michael Ellerman, Aneesh Kumar] Fix retval from emulate_paste() and paste(). - [Nick Piggin] Drop CP_ABORT since set_thread_uses_vas() does that now (in earlier patch) and add a check for return value. - [Michael Ellerman] Fix couple of unininitialized variables - Minor tweak to a debug message Michael Neuling (1): powerpc: Emulate paste instruction Sukadev Bhattiprolu (17): powerpc/vas: init missing fields from [rt]xattr powerpc/vas: Validate window credits powerpc/vas: Cleanup some debug code powerpc/vas: Drop poll_window_cast_out(). powerpc/vas: Use helper to unpin/close window powerpc/vas: Reduce polling interval for busy state powerpc/vas: Save configured window credits powerpc/vas: poll for return of window credits powerpc/vas: Create cpu to vas id mapping powerpc/vas, nx-842: Define and use chip_to_vas_id() powerpc/vas: Export HVWC to debugfs powerpc: have copy depend on CONFIG_BOOK3S_64 powerpc: Add support for setting SPRN_TIDR powerpc: Define set_thread_uses_vas() powerpc/vas: Define vas_win_paste_addr() powerpc/vas: Define vas_win_id() powerpc/vas: Add support for user receive window arch/powerpc/include/asm/emulated_ops.h | 1 + arch/powerpc/include/asm/ppc-opcode.h | 1 + arch/powerpc/include/asm/processor.h| 3 + arch/powerpc/include/asm/reg.h | 2 + arch/powerpc/include/asm/switch_to.h| 5 + arch/powerpc/include/asm/vas.h | 21 +++ arch/powerpc/kernel/process.c | 169 +-- arch/powerpc/kernel/traps.c | 67 arch/powerpc/platforms/powernv/Makefile | 3 +- arch/powerpc/platforms/powernv/vas-debug.c | 209 arch/powerpc/platforms/powernv/vas-window.c | 242 +++- arch/powerpc/platforms/powernv/vas.c| 31 +++- arch/powerpc/platforms/powernv/vas.h| 93 ++- drivers/crypto/nx/nx-842-powernv.c | 18 +-- 14 files changed, 751 insertions(+), 114 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/vas-debug.c -- 2.7.4
[PATCH v2 02/18] powerpc/vas: Validate window credits
NX-842, the only user of VAS, sets the window credits to default values
but VAS should check the credits against the possible max values.
The VAS_WCREDS_MIN is not needed and can be dropped.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 ++
arch/powerpc/platforms/powernv/vas.h| 4 ++--
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index cec7ab7..a2fe120 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -738,6 +738,9 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
+ if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ return false;
+
if (attr->nx_win) {
/* cannot be fault or user window if it is nx */
if (attr->fault_win || attr->user_win)
@@ -927,6 +930,9 @@ static bool tx_win_args_valid(enum vas_cop_type cop,
if (cop > VAS_COP_TYPE_MAX)
return false;
+ if (attr->wcreds_max > VAS_TX_WCREDS_MAX)
+ return false;
+
if (attr->user_win &&
(cop != VAS_COP_TYPE_FTW || attr->rsvd_txbuf_count))
return false;
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 38dee5d..fea0de4 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -106,8 +106,8 @@
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_WCREDS_MIN 16
-#define VAS_WCREDS_MAX ((64 << 10) - 1)
+#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
+#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
/*
--
2.7.4
[PATCH v2 01/18] powerpc/vas: init missing fields from [rt]xattr
Initialize a few missing window context fields from the window attributes
specified by the caller. These fields are currently set to their default
values by the caller (NX-842), but would be good to apply them anyway.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 5aae845..cec7ab7 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -679,10 +679,13 @@ static void init_winctx_for_rxwin(struct vas_window
*rxwin,
winctx->nx_win = rxattr->nx_win;
winctx->fault_win = rxattr->fault_win;
+ winctx->user_win = rxattr->user_win;
+ winctx->rej_no_credit = rxattr->rej_no_credit;
winctx->rx_word_mode = rxattr->rx_win_ord_mode;
winctx->tx_word_mode = rxattr->tx_win_ord_mode;
winctx->rx_wcred_mode = rxattr->rx_wcred_mode;
winctx->tx_wcred_mode = rxattr->tx_wcred_mode;
+ winctx->notify_early = rxattr->notify_early;
if (winctx->nx_win) {
winctx->data_stamp = true;
@@ -889,11 +892,14 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->user_win = txattr->user_win;
winctx->nx_win = txwin->rxwin->nx_win;
winctx->pin_win = txattr->pin_win;
+ winctx->rej_no_credit = txattr->rej_no_credit;
+ winctx->rsvd_txbuf_enable = txattr->rsvd_txbuf_enable;
winctx->rx_wcred_mode = txattr->rx_wcred_mode;
winctx->tx_wcred_mode = txattr->tx_wcred_mode;
winctx->rx_word_mode = txattr->rx_win_ord_mode;
winctx->tx_word_mode = txattr->tx_win_ord_mode;
+ winctx->rsvd_txbuf_count = txattr->rsvd_txbuf_count;
if (winctx->nx_win) {
winctx->data_stamp = true;
--
2.7.4
[PATCH v2 05/18] powerpc/vas: Use helper to unpin/close window
Use a helper to have the hardware unpin and mark a window closed.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 22 +++---
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8ab8a82..95622a9 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1101,6 +1101,20 @@ static void poll_window_castout(struct vas_window
*window)
}
/*
+ * Unpin and close a window so no new requests are accepted and the
+ * hardware can evict this window from cache if necessary.
+ */
+static void unpin_close_window(struct vas_window *window)
+{
+ u64 val;
+
+ val = read_hvwc_reg(window, VREG(WINCTL));
+ val = SET_FIELD(VAS_WINCTL_PIN, val, 0);
+ val = SET_FIELD(VAS_WINCTL_OPEN, val, 0);
+ write_hvwc_reg(window, VREG(WINCTL), val);
+}
+
+/*
* Close a window.
*
* See Section 1.12.1 of VAS workbook v1.05 for details on closing window:
@@ -1114,8 +1128,6 @@ static void poll_window_castout(struct vas_window *window)
*/
int vas_win_close(struct vas_window *window)
{
- u64 val;
-
if (!window)
return 0;
@@ -1131,11 +1143,7 @@ int vas_win_close(struct vas_window *window)
poll_window_busy_state(window);
- /* Unpin window from cache and close it */
- val = read_hvwc_reg(window, VREG(WINCTL));
- val = SET_FIELD(VAS_WINCTL_PIN, val, 0);
- val = SET_FIELD(VAS_WINCTL_OPEN, val, 0);
- write_hvwc_reg(window, VREG(WINCTL), val);
+ unpin_close_window(window);
poll_window_castout(window);
--
2.7.4
[PATCH v2 06/18] powerpc/vas: Reduce polling interval for busy state
A VAS window is normally in "busy" state for only a short duration.
Reduce the time we wait for the window to go to "not-busy" state to
speed-up vas_win_close() a bit.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 95622a9..1422cdd 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1060,21 +1060,23 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
}
EXPORT_SYMBOL_GPL(vas_paste_crb);
+/*
+ * Wait for the window to go to "not-busy" state. It should only take a
+ * short time to queue a CRB, so window should not be busy for too long.
+ * Trying 5ms intervals.
+ */
static void poll_window_busy_state(struct vas_window *window)
{
int busy;
u64 val;
retry:
- /*
-* Poll Window Busy flag
-*/
val = read_hvwc_reg(window, VREG(WIN_STATUS));
busy = GET_FIELD(VAS_WIN_BUSY, val);
if (busy) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(HZ);
+ schedule_timeout(msecs_to_jiffies(5));
goto retry;
}
}
--
2.7.4
[PATCH v2 07/18] powerpc/vas: Save configured window credits
Save the configured max window credits for a window in the vas_window
structure. We will need this when polling for return of window credits.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 --
arch/powerpc/platforms/powernv/vas.h| 1 +
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1422cdd..a59a187 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -674,7 +674,7 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->rx_fifo = rxattr->rx_fifo;
winctx->rx_fifo_size = rxattr->rx_fifo_size;
- winctx->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ winctx->wcreds_max = rxwin->wcreds_max;
winctx->pin_win = rxattr->pin_win;
winctx->nx_win = rxattr->nx_win;
@@ -844,6 +844,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
+ rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
if (rxattr->user_win)
rxwin->pid = task_pid_vnr(current);
@@ -893,7 +894,7 @@ static void init_winctx_for_txwin(struct vas_window *txwin,
*/
memset(winctx, 0, sizeof(struct vas_winctx));
- winctx->wcreds_max = txattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ winctx->wcreds_max = txwin->wcreds_max;
winctx->user_win = txattr->user_win;
winctx->nx_win = txwin->rxwin->nx_win;
@@ -978,6 +979,7 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->nx_win = txwin->rxwin->nx_win;
txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
+ txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
init_winctx_for_txwin(txwin, attr, &winctx);
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 15d2dfa..ad906f6 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -332,6 +332,7 @@ struct vas_window {
void *hvwc_map; /* HV window context */
void *uwc_map; /* OS/User window context */
pid_t pid; /* Linux process id of owner */
+ int wcreds_max; /* Window credits */
/* Fields applicable only to send windows */
void *paste_kaddr;
--
2.7.4
[PATCH v2 09/18] powerpc/vas: Create cpu to vas id mapping
Create a cpu to vasid mapping so callers can specify -1 instead of
trying to find a VAS id.
Changelog[v2]
[Michael Ellerman] Use per-cpu variables to simplify code.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas.c | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index 565a487..abb7090 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -18,15 +18,18 @@
#include
#include
#include
+#include
#include "vas.h"
static DEFINE_MUTEX(vas_mutex);
static LIST_HEAD(vas_instances);
+static DEFINE_PER_CPU(int, cpu_vas_id);
+
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, vasid;
+ int rc, cpu, vasid;
struct resource *res;
struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
@@ -74,6 +77,11 @@ static int init_vas_instance(struct platform_device *pdev)
"paste_win_id_shift 0x%llx\n", pdev->name, vasid,
vinst->paste_base_addr, vinst->paste_win_id_shift);
+ for_each_possible_cpu(cpu) {
+ if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
+ per_cpu(cpu_vas_id, cpu) = vasid;
+ }
+
mutex_lock(&vas_mutex);
list_add(&vinst->node, &vas_instances);
mutex_unlock(&vas_mutex);
@@ -98,6 +106,10 @@ struct vas_instance *find_vas_instance(int vasid)
struct vas_instance *vinst;
mutex_lock(&vas_mutex);
+
+ if (vasid == -1)
+ vasid = per_cpu(cpu_vas_id, smp_processor_id());
+
list_for_each(ent, &vas_instances) {
vinst = list_entry(ent, struct vas_instance, node);
if (vinst->vas_id == vasid) {
--
2.7.4
[PATCH v2 11/18] powerpc/vas: Export HVWC to debugfs
Export the VAS Window context information to debugfs.
We need to hold a mutex when closing the window to prevent a race
with the debugfs read(). Rather than introduce a per-instance mutex,
we use the global vas_mutex for now, since it is not heavily contended.
The window->cop field is only relevant to a receive window so we were
not setting it for a send window (which is is paired to a receive window
anyway). But to simplify reporting in debugfs, set the 'cop' field for the
send window also.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/Makefile | 3 +-
arch/powerpc/platforms/powernv/vas-debug.c | 209
arch/powerpc/platforms/powernv/vas-window.c | 34 -
arch/powerpc/platforms/powernv/vas.c| 6 +-
arch/powerpc/platforms/powernv/vas.h| 14 ++
5 files changed, 259 insertions(+), 7 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/vas-debug.c
diff --git a/arch/powerpc/platforms/powernv/Makefile
b/arch/powerpc/platforms/powernv/Makefile
index 37d60f7..17921c4 100644
--- a/arch/powerpc/platforms/powernv/Makefile
+++ b/arch/powerpc/platforms/powernv/Makefile
@@ -14,4 +14,5 @@ obj-$(CONFIG_TRACEPOINTS) += opal-tracepoints.o
obj-$(CONFIG_OPAL_PRD) += opal-prd.o
obj-$(CONFIG_PERF_EVENTS) += opal-imc.o
obj-$(CONFIG_PPC_MEMTRACE) += memtrace.o
-obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o
+obj-$(CONFIG_PPC_VAS) += vas.o vas-window.o vas-debug.o
+obj-$(CONFIG_PPC_FTW) += nx-ftw.o
diff --git a/arch/powerpc/platforms/powernv/vas-debug.c
b/arch/powerpc/platforms/powernv/vas-debug.c
new file mode 100644
index 000..ca22f1e
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas-debug.c
@@ -0,0 +1,209 @@
+/*
+ * Copyright 2016-17 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#define pr_fmt(fmt) "vas: " fmt
+
+#include
+#include
+#include
+#include
+#include "vas.h"
+
+static struct dentry *vas_debugfs;
+
+static char *cop_to_str(int cop)
+{
+ switch (cop) {
+ case VAS_COP_TYPE_FAULT:return "Fault";
+ case VAS_COP_TYPE_842: return "NX-842 Normal Priority";
+ case VAS_COP_TYPE_842_HIPRI:return "NX-842 High Priority";
+ case VAS_COP_TYPE_GZIP: return "NX-GZIP Normal Priority";
+ case VAS_COP_TYPE_GZIP_HIPRI: return "NX-GZIP High Priority";
+ case VAS_COP_TYPE_FTW: return "Fast Thread-wakeup";
+ default:return "Unknown";
+ }
+}
+
+static int info_dbg_show(struct seq_file *s, void *private)
+{
+ struct vas_window *window = s->private;
+
+ mutex_lock(&vas_mutex);
+
+ /* ensure window is not unmapped */
+ if (!window->hvwc_map)
+ goto unlock;
+
+ seq_printf(s, "Type: %s, %s\n", cop_to_str(window->cop),
+ window->tx_win ? "Send" : "Receive");
+ seq_printf(s, "Pid : %d\n", window->pid);
+
+unlock:
+ mutex_unlock(&vas_mutex);
+ return 0;
+}
+
+static int info_dbg_open(struct inode *inode, struct file *file)
+{
+ return single_open(file, info_dbg_show, inode->i_private);
+}
+
+static const struct file_operations info_fops = {
+ .open = info_dbg_open,
+ .read = seq_read,
+ .llseek = seq_lseek,
+ .release= single_release,
+};
+
+static inline void print_reg(struct seq_file *s, struct vas_window *win,
+ char *name, u32 reg)
+{
+ seq_printf(s, "0x%016llx %s\n", read_hvwc_reg(win, name, reg), name);
+}
+
+static int hvwc_dbg_show(struct seq_file *s, void *private)
+{
+ struct vas_window *window = s->private;
+
+ mutex_lock(&vas_mutex);
+
+ /* ensure window is not unmapped */
+ if (!window->hvwc_map)
+ goto unlock;
+
+ print_reg(s, window, VREG(LPID));
+ print_reg(s, window, VREG(PID));
+ print_reg(s, window, VREG(XLATE_MSR));
+ print_reg(s, window, VREG(XLATE_LPCR));
+ print_reg(s, window, VREG(XLATE_CTL));
+ print_reg(s, window, VREG(AMR));
+ print_reg(s, window, VREG(SEIDR));
+ print_reg(s, window, VREG(FAULT_TX_WIN));
+ print_reg(s, window, VREG(OSU_INTR_SRC_RA));
+ print_reg(s, window, VREG(HV_INTR_SRC_RA));
+ print_reg(s, window, VREG(PSWID));
+ print_reg(s, window, VREG(LFIFO_BAR));
+ print_reg(s, window, VREG(LDATA_STAMP_CTL));
+ print_reg(s, window, VREG(LDMA_CACHE_CTL));
+ print_reg(s, window, VREG(LRFIFO_PUSH));
+ print_reg(s, window, VREG(CURR_MSG_COUN
[PATCH v2 08/18] powerpc/vas: poll for return of window credits
Normally, the NX driver waits for the CRBs to be processed before closing
the window. But it is better to ensure that the credits are returned before
the window gets reassigned later.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 45 +
1 file changed, 45 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a59a187..23c13a7 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1063,6 +1063,49 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
EXPORT_SYMBOL_GPL(vas_paste_crb);
/*
+ * If credit checking is enabled for this window, poll for the return
+ * of window credits (i.e for NX engines to process any outstanding CRBs).
+ * Since NX-842 waits for the CRBs to be processed before closing the
+ * window, we should not have to wait for too long.
+ *
+ * TODO: We retry in 10ms intervals now. We could/should probably peek at
+ * the VAS_LRFIFO_PUSH_OFFSET register to get an estimate of pending
+ * CRBs on the FIFO and compute the delay dynamically on each retry.
+ * But that is not really needed until we support NX-GZIP access from
+ * user space. (NX-842 driver waits for CSB and Fast thread-wakeup
+ * doesn't use credit checking).
+ */
+static void poll_window_credits(struct vas_window *window)
+{
+ u64 val;
+ int creds, mode;
+
+ val = read_hvwc_reg(window, VREG(WINCTL));
+ if (window->tx_win)
+ mode = GET_FIELD(VAS_WINCTL_TX_WCRED_MODE, val);
+ else
+ mode = GET_FIELD(VAS_WINCTL_RX_WCRED_MODE, val);
+
+ if (!mode)
+ return;
+retry:
+ if (window->tx_win) {
+ val = read_hvwc_reg(window, VREG(TX_WCRED));
+ creds = GET_FIELD(VAS_TX_WCRED, val);
+ } else {
+ val = read_hvwc_reg(window, VREG(LRX_WCRED));
+ creds = GET_FIELD(VAS_LRX_WCRED, val);
+ }
+
+ if (creds < window->wcreds_max) {
+ val = 0;
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(msecs_to_jiffies(10));
+ goto retry;
+ }
+}
+
+/*
* Wait for the window to go to "not-busy" state. It should only take a
* short time to queue a CRB, so window should not be busy for too long.
* Trying 5ms intervals.
@@ -1149,6 +1192,8 @@ int vas_win_close(struct vas_window *window)
unpin_close_window(window);
+ poll_window_credits(window);
+
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
--
2.7.4
[PATCH v2 12/18] powerpc: have copy depend on CONFIG_BOOK3S_64
Have the COPY/PASTE instructions depend on CONFIG_BOOK3S_64 rather than
CONFIG_PPC_STD_MMU_64.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/kernel/process.c | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index a0c74bb..37ed60b 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1215,10 +1215,14 @@ struct task_struct *__switch_to(struct task_struct
*prev,
batch = this_cpu_ptr(&ppc64_tlb_batch);
batch->active = 1;
}
+#endif /* CONFIG_PPC_STD_MMU_64 */
if (current_thread_info()->task->thread.regs) {
+#ifdef CONFIG_PPC_STD_MMU_64
restore_math(current_thread_info()->task->thread.regs);
+#endif /* CONFIG_PPC_STD_MMU_64 */
+#ifdef CONFIG_PPC_BOOK3S_64
/*
* The copy-paste buffer can only store into foreign real
* addresses, so unprivileged processes can not see the
@@ -1237,8 +1241,8 @@ struct task_struct *__switch_to(struct task_struct *prev,
asm volatile(PPC_COPY(%0, %1)
: : "r"(dummy_copy_buffer), "r"(0));
}
+#endif /* CONFIG_PPC_BOOK3S_64 */
}
-#endif /* CONFIG_PPC_STD_MMU_64 */
return last;
}
--
2.7.4
[PATCH v2 10/18] powerpc/vas, nx-842: Define and use chip_to_vas_id()
Define a helper, chip_to_vas_id() to map a given chip id to corresponding
vas id.
Normally, callers of vas_rx_win_open() and vas_tx_win_open() want the VAS
window to be on the same chip where the calling thread is executing. These
callers can pass in -1 for the VAS id.
This interface will be useful if a thread running on one chip wants to open
a window on another chip (like the NX-842 driver does during start up).
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 9 +
arch/powerpc/platforms/powernv/vas.c | 11 +++
drivers/crypto/nx/nx-842-powernv.c | 18 +++---
3 files changed, 23 insertions(+), 15 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index fd5963a..044748f 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -104,6 +104,15 @@ struct vas_tx_win_attr {
};
/*
+ * Helper to map a chip id to VAS id.
+ * For POWER9, this is a 1:1 mapping. In the future this maybe a 1:N
+ * mapping in which case, we will need to update this helper.
+ *
+ * Return the VAS id or -1 if no matching vasid is found.
+ */
+int chip_to_vas_id(int chipid);
+
+/*
* Helper to initialize receive window attributes to defaults for an
* NX window.
*/
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index abb7090..cd9a733 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -123,6 +123,17 @@ struct vas_instance *find_vas_instance(int vasid)
return NULL;
}
+int chip_to_vas_id(int chipid)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ if (cpu_to_chip_id(cpu) == chipid)
+ return per_cpu(cpu_vas_id, cpu);
+ }
+ return -1;
+}
+
static int vas_probe(struct platform_device *pdev)
{
return init_vas_instance(pdev);
diff --git a/drivers/crypto/nx/nx-842-powernv.c
b/drivers/crypto/nx/nx-842-powernv.c
index 874ddf5..eb221ed 100644
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ b/drivers/crypto/nx/nx-842-powernv.c
@@ -847,24 +847,12 @@ static int __init nx842_powernv_probe_vas(struct
device_node *pn)
return -EINVAL;
}
- for_each_compatible_node(dn, NULL, "ibm,power9-vas-x") {
- if (of_get_ibm_chip_id(dn) == chip_id)
- break;
- }
-
- if (!dn) {
- pr_err("Missing VAS device node\n");
+ vasid = chip_to_vas_id(chip_id);
+ if (vasid < 0) {
+ pr_err("Unable to map chip_id %d to vasid\n", chip_id);
return -EINVAL;
}
- if (of_property_read_u32(dn, "ibm,vas-id", &vasid)) {
- pr_err("Missing ibm,vas-id device property\n");
- of_node_put(dn);
- return -EINVAL;
- }
-
- of_node_put(dn);
-
for_each_child_of_node(pn, dn) {
if (of_device_is_compatible(dn, "ibm,p9-nx-842")) {
ret = vas_cfg_coproc_info(dn, chip_id, vasid);
--
2.7.4
[PATCH v2 13/18] powerpc: Add support for setting SPRN_TIDR
We need the SPRN_TIDR to be set for use with fast thread-wakeup (core-
to-core wakeup) and also with CAPI.
Each thread in a process needs to have a unique id within the process.
But as explained below, for now, we assign globally unique thread ids
to all threads in the system.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Philippe Bergheaud
Signed-off-by: Christophe Lombard
---
Changelog[v3]
- Merge changes with and address comments to Christophe's patch.
(i.e drop CONFIG_PPC_VAS; use CONFIG_PPC64; check CPU_ARCH_300
before setting TIDR). Defer following to separate patches:
- emulation parts of Christophe's patch,
- setting TIDR for tasks other than 'current'
- setting feature bit in AT_HWCAP2
Changelog[v2]
- Michael Ellerman: Use an interface to assign TIDR so it is
assigned to only threads that need it; move assignment to
restore_sprs(). Drop lint from rebase;
---
arch/powerpc/include/asm/processor.h | 1 +
arch/powerpc/include/asm/switch_to.h | 3 +
arch/powerpc/kernel/process.c| 122 +++
3 files changed, 126 insertions(+)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index fab7ff8..58cc212 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -329,6 +329,7 @@ struct thread_struct {
*/
int dscr_inherit;
unsigned long ppr;/* used to save/restore SMT priority */
+ unsigned long tidr;
#endif
#ifdef CONFIG_PPC_BOOK3S_64
unsigned long tar;
diff --git a/arch/powerpc/include/asm/switch_to.h
b/arch/powerpc/include/asm/switch_to.h
index 17c8380..f5da32f 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -91,4 +91,7 @@ static inline void clear_task_ebb(struct task_struct *t)
#endif
}
+extern int set_thread_tidr(struct task_struct *t);
+extern void clear_thread_tidr(struct task_struct *t);
+
#endif /* _ASM_POWERPC_SWITCH_TO_H */
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index 37ed60b..d861fcd 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1120,6 +1120,13 @@ static inline void restore_sprs(struct thread_struct
*old_thread,
mtspr(SPRN_TAR, new_thread->tar);
}
#endif
+#ifdef CONFIG_PPC64
+ if (old_thread->tidr != new_thread->tidr) {
+ /* TIDR should be non-zero only with ISA3.0. */
+ WARN_ON_ONCE(!cpu_has_feature(CPU_FTR_ARCH_300));
+ mtspr(SPRN_TIDR, new_thread->tidr);
+ }
+#endif
}
#ifdef CONFIG_PPC_BOOK3S_64
@@ -1438,9 +1445,117 @@ void flush_thread(void)
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
}
+#ifdef CONFIG_PPC64
+static DEFINE_SPINLOCK(vas_thread_id_lock);
+static DEFINE_IDA(vas_thread_ida);
+
+/*
+ * We need to assign a unique thread id to each thread in a process.
+ *
+ * This thread id, referred to as TIDR, and separate from the Linux's tgid,
+ * is intended to be used to direct an ASB_Notify from the hardware to the
+ * thread, when a suitable event occurs in the system.
+ *
+ * One such event is a "paste" instruction in the context of Fast Thread
+ * Wakeup (aka Core-to-core wake up in the Virtual Accelerator Switchboard
+ * (VAS) in POWER9.
+ *
+ * To get a unique TIDR per process we could simply reuse task_pid_nr() but
+ * the problem is that task_pid_nr() is not yet available copy_thread() is
+ * called. Fixing that would require changing more intrusive arch-neutral
+ * code in code path in copy_process()?.
+ *
+ * Further, to assign unique TIDRs within each process, we need an atomic
+ * field (or an IDR) in task_struct, which again intrudes into the arch-
+ * neutral code. So try to assign globally unique TIDRs for now.
+ *
+ * NOTE: TIDR 0 indicates that the thread does not need a TIDR value.
+ * For now, only threads that expect to be notified by the VAS
+ * hardware need a TIDR value and we assign values > 0 for those.
+ */
+#define MAX_THREAD_CONTEXT ((1 << 16) - 1)
+static int assign_thread_tidr(void)
+{
+ int index;
+ int err;
+
+again:
+ if (!ida_pre_get(&vas_thread_ida, GFP_KERNEL))
+ return -ENOMEM;
+
+ spin_lock(&vas_thread_id_lock);
+ err = ida_get_new_above(&vas_thread_ida, 1, &index);
+ spin_unlock(&vas_thread_id_lock);
+
+ if (err == -EAGAIN)
+ goto again;
+ else if (err)
+ return err;
+
+ if (index > MAX_THREAD_CONTEXT) {
+ spin_lock(&vas_thread_id_lock);
+ ida_remove(&vas_thread_ida, index);
+ spin_unlock(&vas_thread_id_lock);
+ return -ENOMEM;
+ }
+
+ return index;
+}
+
+static void free_thread_tid
[PATCH v2 17/18] powerpc/vas: Define vas_win_id()
Define an interface to return a system-wide unique id for a given VAS
window.
The vas_win_id() will be used in a follow-on patch to generate an unique
handle for a user space receive window. Applications can use this handle
to pair send and receive windows for fast thread-wakeup.
The hardware refers to this system-wide unique id as a Partition Send
Window ID which is expected to be used during fault handling. Hence the
"pswid" in the function names.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 5 +
arch/powerpc/platforms/powernv/vas-window.c | 9 +
arch/powerpc/platforms/powernv/vas.h| 28
3 files changed, 42 insertions(+)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index f98ade8..7714562 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -168,6 +168,11 @@ int vas_copy_crb(void *crb, int offset);
int vas_paste_crb(struct vas_window *win, int offset, bool re);
/*
+ * Return a system-wide unique id for the VAS window @win.
+ */
+extern u32 vas_win_id(struct vas_window *win);
+
+/*
* Return the power bus paste address associated with @win so the caller
* can map that address into their address space.
*/
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index e4a9c7b..1d08b64 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1239,3 +1239,12 @@ int vas_win_close(struct vas_window *window)
return 0;
}
EXPORT_SYMBOL_GPL(vas_win_close);
+
+/*
+ * Return a system-wide unique window id for the window @win.
+ */
+u32 vas_win_id(struct vas_window *win)
+{
+ return encode_pswid(win->vinst->vas_id, win->winid);
+}
+EXPORT_SYMBOL_GPL(vas_win_id);
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 145749a..78a8926 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -447,4 +447,32 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
+/*
+ * Encode/decode the Partition Send Window ID (PSWID) for a window in
+ * a way that we can uniquely identify any window in the system. i.e.
+ * we should be able to locate the 'struct vas_window' given the PSWID.
+ *
+ * BitsUsage
+ * 0:7 VAS id (8 bits)
+ * 8:15Unused, 0 (3 bits)
+ * 16:31 Window id (16 bits)
+ */
+static inline u32 encode_pswid(int vasid, int winid)
+{
+ u32 pswid = 0;
+
+ pswid |= vasid << (31 - 7);
+ pswid |= winid;
+
+ return pswid;
+}
+
+static inline void decode_pswid(u32 pswid, int *vasid, int *winid)
+{
+ if (vasid)
+ *vasid = pswid >> (31 - 7) & 0xFF;
+
+ if (winid)
+ *winid = pswid & 0x;
+}
#endif /* _VAS_H */
--
2.7.4
[PATCH v2 18/18] powerpc/vas: Add support for user receive window
Add support for user space receive window (for the Fast thread-wakeup
coprocessor type)
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 59 +
1 file changed, 52 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1d08b64..99642ec 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -16,7 +16,8 @@
#include
#include
#include
-
+#include
+#include
#include "vas.h"
#include "copy-paste.h"
@@ -602,6 +603,32 @@ static void put_rx_win(struct vas_window *rxwin)
}
/*
+ * Find the user space receive window given the @pswid.
+ * - We must have a valid vasid and it must belong to this instance.
+ *(so both send and receive windows are on the same VAS instance)
+ * - The window must refer to an OPEN, FTW, RECEIVE window.
+ *
+ * NOTE: We access ->windows[] table and assume that vinst->mutex is held.
+ */
+static struct vas_window *get_user_rxwin(struct vas_instance *vinst, u32 pswid)
+{
+ int vasid, winid;
+ struct vas_window *rxwin;
+
+ decode_pswid(pswid, &vasid, &winid);
+
+ if (vinst->vas_id != vasid)
+ return ERR_PTR(-EINVAL);
+
+ rxwin = vinst->windows[winid];
+
+ if (!rxwin || rxwin->tx_win || rxwin->cop != VAS_COP_TYPE_FTW)
+ return ERR_PTR(-EINVAL);
+
+ return rxwin;
+}
+
+/*
* Get the VAS receive window associated with NX engine identified
* by @cop and if applicable, @pswid.
*
@@ -614,10 +641,10 @@ static struct vas_window *get_vinst_rxwin(struct
vas_instance *vinst,
mutex_lock(&vinst->mutex);
- if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI)
- rxwin = vinst->rxwin[cop] ?: ERR_PTR(-EINVAL);
+ if (cop == VAS_COP_TYPE_FTW)
+ rxwin = get_user_rxwin(vinst, pswid);
else
- rxwin = ERR_PTR(-EINVAL);
+ rxwin = vinst->rxwin[cop] ?: ERR_PTR(-EINVAL);
if (!IS_ERR(rxwin))
atomic_inc(&rxwin->num_txwins);
@@ -941,10 +968,9 @@ static void init_winctx_for_txwin(struct vas_window *txwin,
winctx->tx_word_mode = txattr->tx_win_ord_mode;
winctx->rsvd_txbuf_count = txattr->rsvd_txbuf_count;
- if (winctx->nx_win) {
+ winctx->intr_disable = true;
+ if (winctx->nx_win)
winctx->data_stamp = true;
- winctx->intr_disable = true;
- }
winctx->lpid = txattr->lpid;
winctx->pidr = txattr->pidr;
@@ -989,6 +1015,14 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
if (!tx_win_args_valid(cop, attr))
return ERR_PTR(-EINVAL);
+ /*
+* If caller did not specify a vasid but specified the PSWID of a
+* receive window (applicable only to FTW windows), use the vasid
+* from that receive window.
+*/
+ if (vasid == -1 && attr->pswid)
+ decode_pswid(attr->pswid, &vasid, NULL);
+
vinst = find_vas_instance(vasid);
if (!vinst) {
pr_devel("vasid %d not found!\n", vasid);
@@ -1037,6 +1071,17 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
set_vinst_win(vinst, txwin);
+ set_thread_used_vas();
+
+ /*
+* Even a process that has no foreign real address mapping can use
+* an unpaired COPY instruction (to no real effect). Issue CP_ABORT
+* to clear any pending COPY and prevent a covert channel.
+*
+* __switch_to() will issue CP_ABORT on future context switches.
+*/
+ asm volatile(PPC_CP_ABORT);
+
return txwin;
free_window:
--
2.7.4
[PATCH v2 14/18] powerpc: Define set_thread_used_vas()
A CP_ABORT instruction is required in processes that have mapped a VAS
"paste address" with the intention of using COPY/PASTE instructions.
But since CP_ABORT is expensive, we want to restrict it to only processes
that use/intend to use COPY/PASTE.
Define an interface, set_thread_used_vas(), that VAS can use to indicate
that the current process opened a send window. During context switch,
issue CP_ABORT only for processes that have the flag set.
Thanks for input from Nick Piggin, Michael Ellerman.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/processor.h | 2 ++
arch/powerpc/include/asm/switch_to.h | 2 ++
arch/powerpc/kernel/process.c| 32 ++--
3 files changed, 26 insertions(+), 10 deletions(-)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index 58cc212..bdab3b74 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -341,7 +341,9 @@ struct thread_struct {
unsigned long sier;
unsigned long mmcr2;
unsignedmmcr0;
+
unsignedused_ebb;
+ unsigned intused_vas;
#endif
};
diff --git a/arch/powerpc/include/asm/switch_to.h
b/arch/powerpc/include/asm/switch_to.h
index f5da32f..aeb305b 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -91,6 +91,8 @@ static inline void clear_task_ebb(struct task_struct *t)
#endif
}
+extern int set_thread_used_vas(void);
+
extern int set_thread_tidr(struct task_struct *t);
extern void clear_thread_tidr(struct task_struct *t);
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index d861fcd..cb5f108 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1234,17 +1234,17 @@ struct task_struct *__switch_to(struct task_struct
*prev,
* The copy-paste buffer can only store into foreign real
* addresses, so unprivileged processes can not see the
* data or use it in any way unless they have foreign real
-* mappings. We don't have a VAS driver that allocates those
-* yet, so no cpabort is required.
+* mappings. If the new process has the foreign real address
+* mappings, we must issue a cp_abort to clear any state and
+* prevent a covert channel being setup.
+*
+* DD1 allows paste into normal system memory so we do an
+* unpaired copy, rather than cp_abort, to clear the buffer,
+* since cp_abort is quite expensive.
*/
- if (cpu_has_feature(CPU_FTR_POWER9_DD1)) {
- /*
-* DD1 allows paste into normal system memory, so we
-* do an unpaired copy here to clear the buffer and
-* prevent a covert channel being set up.
-*
-* cpabort is not used because it is quite expensive.
-*/
+ if (new_thread->used_vas) {
+ asm volatile(PPC_CP_ABORT);
+ } else if (cpu_has_feature(CPU_FTR_POWER9_DD1)) {
asm volatile(PPC_COPY(%0, %1)
: : "r"(dummy_copy_buffer), "r"(0));
}
@@ -1445,6 +1445,18 @@ void flush_thread(void)
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
}
+int set_thread_used_vas(void)
+{
+#ifdef CONFIG_PPC_BOOK3S_64
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EINVAL;
+
+ current->thread.used_vas = 1;
+
+#endif /* CONFIG_PPC_BOOK3S_64 */
+ return 0;
+}
+
#ifdef CONFIG_PPC64
static DEFINE_SPINLOCK(vas_thread_id_lock);
static DEFINE_IDA(vas_thread_ida);
--
2.7.4
[PATCH v2 16/18] powerpc/vas: Define vas_win_paste_addr()
Define an interface that the NX drivers can use to find the physical
paste address of a send window. This interface is expected to be used
with the mmap() operation of the NX driver's device. i.e the user space
process can use driver's mmap() operation to map the send window's paste
address into their address space and then use copy and paste instructions
to submit the CRBs to the NX engine.
Note that kernel drivers will use vas_paste_crb() directly and don't need
this interface.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 7 +++
arch/powerpc/platforms/powernv/vas-window.c | 10 ++
2 files changed, 17 insertions(+)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index 044748f..f98ade8 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -10,6 +10,8 @@
#ifndef _ASM_POWERPC_VAS_H
#define _ASM_POWERPC_VAS_H
+struct vas_window;
+
/*
* Min and max FIFO sizes are based on Version 1.05 Section 3.1.4.25
* (Local FIFO Size Register) of the VAS workbook.
@@ -165,4 +167,9 @@ int vas_copy_crb(void *crb, int offset);
*/
int vas_paste_crb(struct vas_window *win, int offset, bool re);
+/*
+ * Return the power bus paste address associated with @win so the caller
+ * can map that address into their address space.
+ */
+extern u64 vas_win_paste_addr(struct vas_window *win);
#endif /* __ASM_POWERPC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 088ce56..e4a9c7b 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -40,6 +40,16 @@ static void compute_paste_address(struct vas_window *window,
u64 *addr, int *len
pr_debug("Txwin #%d: Paste addr 0x%llx\n", winid, *addr);
}
+u64 vas_win_paste_addr(struct vas_window *win)
+{
+ u64 addr;
+
+ compute_paste_address(win, &addr, NULL);
+
+ return addr;
+}
+EXPORT_SYMBOL(vas_win_paste_addr);
+
static inline void get_hvwc_mmio_bar(struct vas_window *window,
u64 *start, int *len)
{
--
2.7.4
[PATCH v2 00/18] powerpc/vas: Add support for FTW
The first 10 patches in this set were posted earlier[1] and don't have any significant changes since then. This set sanitizes cpu/chip id to VAS id mapping, improves vas_win_close() performance and adds a check for return of credits and cleans up some code. Patch 11 adds debugfs support for the VAS window contexts. Patches 11-18 add support for user space aka Fast thread-wakeup windows in VAS. These include a patch from Michael Neuling to support emulating the paste instruction. Michael Neuling (1): powerpc: Emulate paste instruction Sukadev Bhattiprolu (17): powerpc/vas: init missing fields from [rt]xattr powerpc/vas: Validate window credits powerpc/vas: Cleanup some debug code powerpc/vas: Drop poll_window_cast_out(). powerpc/vas: Use helper to unpin/close window powerpc/vas: Reduce polling interval for busy state powerpc/vas: Save configured window credits powerpc/vas: poll for return of window credits powerpc/vas: Create cpu to vas id mapping powerpc/vas, nx-842: Define and use chip_to_vas_id() powerpc/vas: Export HVWC to debugfs powerpc: have copy depend on CONFIG_BOOK3S_64 powerpc: Add support for setting SPRN_TIDR powerpc: Define set_thread_used_vas() powerpc/vas: Define vas_win_paste_addr() powerpc/vas: Define vas_win_id() powerpc/vas: Add support for user receive window arch/powerpc/include/asm/emulated_ops.h | 1 + arch/powerpc/include/asm/ppc-opcode.h | 1 + arch/powerpc/include/asm/processor.h| 3 + arch/powerpc/include/asm/reg.h | 2 + arch/powerpc/include/asm/switch_to.h| 5 + arch/powerpc/include/asm/vas.h | 21 +++ arch/powerpc/kernel/process.c | 160 -- arch/powerpc/kernel/traps.c | 64 arch/powerpc/platforms/powernv/Makefile | 3 +- arch/powerpc/platforms/powernv/vas-debug.c | 209 arch/powerpc/platforms/powernv/vas-window.c | 245 +++- arch/powerpc/platforms/powernv/vas.c| 31 +++- arch/powerpc/platforms/powernv/vas.h| 95 ++- drivers/crypto/nx/nx-842-powernv.c | 18 +- 14 files changed, 745 insertions(+), 113 deletions(-) create mode 100644 arch/powerpc/platforms/powernv/vas-debug.c -- 2.7.4
[PATCH v2 15/18] powerpc: Emulate paste instruction
From: Michael Neuling
On POWER9 DD2.1 and below there are issues when the paste instruction
generates an error. If an error occurs when thread reconfiguration
happens (ie another thread in the core goes into/out of powersave) the
core may hang.
To avoid this a special sequence is required which stops thread
configuration so that the paste can be safely executed.
This patch assumes paste executed in userspace are trapped into the
illegal instruction exception at 0xe40.
Here we re-execute the paste instruction but with the required
sequence to ensure thread reconfiguration doesn't occur.
Signed-off-by: Michael Neuling
Signed-off-by: Sukadev Bhattiprolu
---
Edit by Sukadev: Use PPC_PASTE() rather than the paste instruction since
in older versions the instruction required a third parameter.
---
arch/powerpc/include/asm/emulated_ops.h | 1 +
arch/powerpc/include/asm/ppc-opcode.h | 1 +
arch/powerpc/include/asm/reg.h | 2 ++
arch/powerpc/kernel/traps.c | 64 +
4 files changed, 68 insertions(+)
diff --git a/arch/powerpc/include/asm/emulated_ops.h
b/arch/powerpc/include/asm/emulated_ops.h
index f00e10e..9247af9 100644
--- a/arch/powerpc/include/asm/emulated_ops.h
+++ b/arch/powerpc/include/asm/emulated_ops.h
@@ -55,6 +55,7 @@ extern struct ppc_emulated {
struct ppc_emulated_entry mfdscr;
struct ppc_emulated_entry mtdscr;
struct ppc_emulated_entry lq_stq;
+ struct ppc_emulated_entry paste;
#endif
} ppc_emulated;
diff --git a/arch/powerpc/include/asm/ppc-opcode.h
b/arch/powerpc/include/asm/ppc-opcode.h
index ce0930d..a55d2ef 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -229,6 +229,7 @@
#define PPC_INST_MTTMR 0x7c0003dc
#define PPC_INST_NOP 0x6000
#define PPC_INST_PASTE 0x7c20070d
+#define PPC_INST_PASTE_MASK0xfc2007ff
#define PPC_INST_POPCNTB 0x7cf4
#define PPC_INST_POPCNTB_MASK 0xfc0007fe
#define PPC_INST_POPCNTD 0x7c0003f4
diff --git a/arch/powerpc/include/asm/reg.h b/arch/powerpc/include/asm/reg.h
index f92eaf7..5cde1c4 100644
--- a/arch/powerpc/include/asm/reg.h
+++ b/arch/powerpc/include/asm/reg.h
@@ -468,6 +468,8 @@
#define SPRN_DBAT7U0x23E /* Data BAT 7 Upper Register */
#define SPRN_PPR 0x380 /* SMT Thread status Register */
#define SPRN_TSCR 0x399 /* Thread Switch Control Register */
+#define SPRN_TRIG1 0x371 /* WAT Trigger 1 */
+#define SPRN_TRIG2 0x372 /* WAT Trigger 2 */
#define SPRN_DEC 0x016 /* Decrement Register */
#define SPRN_DER 0x095 /* Debug Enable Register */
diff --git a/arch/powerpc/kernel/traps.c b/arch/powerpc/kernel/traps.c
index 13c9dcd..7e6b1fe 100644
--- a/arch/powerpc/kernel/traps.c
+++ b/arch/powerpc/kernel/traps.c
@@ -956,6 +956,65 @@ static inline bool tm_abort_check(struct pt_regs *regs,
int reason)
}
#endif
+static DEFINE_SPINLOCK(paste_emulation_lock);
+
+static inline int paste(void *i)
+{
+ int cr;
+ long retval = 0;
+
+ /* Need per core lock to ensure trig1/2 writes don't race */
+ spin_lock(&paste_emulation_lock);
+ mtspr(SPRN_TRIG1, 0); /* data doesn't matter */
+ mtspr(SPRN_TRIG1, 0); /* HW says do this twice */
+ asm volatile(
+ "1: " PPC_PASTE(0, %2) "\n"
+ "2: mfcr %1\n"
+ ".section .fixup,\"ax\"\n"
+ "3: li %0,%3\n"
+ " li %2,0\n"
+ " b 2b\n"
+ ".previous\n"
+ EX_TABLE(1b, 3b)
+ : "=r" (retval), "=r" (cr)
+ : "b" (i), "i" (-EFAULT), "0" (retval));
+ mtspr(SPRN_TRIG2, 0);
+ spin_unlock(&paste_emulation_lock);
+ return cr;
+}
+
+static int emulate_paste(struct pt_regs *regs, u32 instword)
+{
+ const void __user *addr;
+ unsigned long ea;
+ u8 ra, rb;
+
+ if (!cpu_has_feature(CPU_FTR_ARCH_300))
+ return -EINVAL;
+
+ ra = (instword >> 16) & 0x1f;
+ rb = (instword >> 11) & 0x1f;
+
+ ea = regs->gpr[rb] + (ra ? regs->gpr[ra] : 0ul);
+ if (is_32bit_task())
+ ea &= 0xul;
+ addr = (__force const void __user *)ea;
+
+ if (!access_ok(VERIFY_WRITE, addr, 128)) // cacheline size == 128
+ return -EFAULT;
+
+ hard_irq_disable(); /* FIXME: could we just soft disable ?? */
+ pagefault_disable();
+
+ PPC_WARN_EMULATED(paste, regs);
+ regs->ccr = paste((void *)addr);
+
+ pagefault_enable();
+ may_hard_irq_enable();
+
+ return 0;
+}
+
static int emulate_instruction(struct pt_regs
[PATCH v2 04/18] powerpc/vas: Drop poll_window_cast_out().
Polling for window cast out is listed in the spec, but turns out that
it is not strictly necessary and slows down window close. Making it a
stub for now.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 34 ++---
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 67ffc5d..8ab8a82 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1079,25 +1079,25 @@ static void poll_window_busy_state(struct vas_window
*window)
}
}
+/*
+ * Have the hardware cast a window out of cache and wait for it to
+ * be completed.
+ *
+ * NOTE: It can take a relatively long time to cast the window context
+ * out of the cache. It is not strictly necessary to cast out if:
+ *
+ * - we clear the "Pin Window" bit (so hardware is free to evict)
+ *
+ * - we re-initialize the window context when it is reassigned.
+ *
+ * We do the former in vas_win_close() and latter in vas_win_open().
+ * So, ignoring the cast-out for now. We can add it as needed. If
+ * casting out becomes necessary we should consider offloading the
+ * job to a worker thread, so the window close can proceed quickly.
+ */
static void poll_window_castout(struct vas_window *window)
{
- int cached;
- u64 val;
-
- /* Cast window context out of the cache */
-retry:
- val = read_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL));
- cached = GET_FIELD(VAS_WIN_CACHE_STATUS, val);
- if (cached) {
- val = 0ULL;
- val = SET_FIELD(VAS_CASTOUT_REQ, val, 1);
- val = SET_FIELD(VAS_PUSH_TO_MEM, val, 0);
- write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), val);
-
- set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(HZ);
- goto retry;
- }
+ /* stub for now */
}
/*
--
2.7.4
[PATCH v2 03/18] powerpc/vas: Cleanup some debug code
Clean up vas.h and the debug code around ifdef vas_debug.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 8 +++--
arch/powerpc/platforms/powernv/vas.h| 56 +++--
2 files changed, 18 insertions(+), 46 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a2fe120..67ffc5d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -726,7 +726,10 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
static bool rx_win_args_valid(enum vas_cop_type cop,
struct vas_rx_win_attr *attr)
{
- dump_rx_win_attr(attr);
+ pr_debug("Rxattr: fault %d, notify %d, intr %d, early %d, fifo %d\n",
+ attr->fault_win, attr->notify_disable,
+ attr->intr_disable, attr->notify_early,
+ attr->rx_fifo_size);
if (cop >= VAS_COP_TYPE_MAX)
return false;
@@ -1050,7 +1053,8 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
else
rc = -EINVAL;
- print_fifo_msg_count(txwin);
+ pr_debug("Txwin #%d: Msg count %llu\n", txwin->winid,
+ read_hvwc_reg(txwin, VREG(LRFIFO_PUSH)));
return rc;
}
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index fea0de4..15d2dfa 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -259,6 +259,16 @@
#define VAS_NX_UTIL_ADDER PPC_BITMASK(32, 63)
/*
+ * VREG(x):
+ * Expand a register's short name (eg: LPID) into two parameters:
+ * - the register's short name in string form ("LPID"), and
+ * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
+ * register's offset in the window context
+ */
+#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
+#define VREG(r)VREG_SFX(r, _OFFSET)
+
+/*
* Local Notify Scope Control Register. (Receive windows only).
*/
enum vas_notify_scope {
@@ -385,43 +395,15 @@ struct vas_winctx {
extern struct vas_instance *find_vas_instance(int vasid);
-/*
- * VREG(x):
- * Expand a register's short name (eg: LPID) into two parameters:
- * - the register's short name in string form ("LPID"), and
- * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
- * register's offset in the window context
- */
-#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
-#define VREG(r)VREG_SFX(r, _OFFSET)
-
-#ifdef vas_debug
-static inline void dump_rx_win_attr(struct vas_rx_win_attr *attr)
-{
- pr_err("fault %d, notify %d, intr %d early %d\n",
- attr->fault_win, attr->notify_disable,
- attr->intr_disable, attr->notify_early);
-
- pr_err("rx_fifo_size %d, max value %d\n",
- attr->rx_fifo_size, VAS_RX_FIFO_SIZE_MAX);
-}
-
static inline void vas_log_write(struct vas_window *win, char *name,
void *regptr, u64 val)
{
- if (val)
- pr_err("%swin #%d: %s reg %p, val 0x%016llx\n",
+ if (!val)
+ pr_debug("%swin #%d: %s reg %p, val 0x%016llx\n",
win->tx_win ? "Tx" : "Rx", win->winid, name,
regptr, val);
}
-#else /* vas_debug */
-
-#define vas_log_write(win, name, reg, val)
-#define dump_rx_win_attr(attr)
-
-#endif /* vas_debug */
-
static inline void write_uwc_reg(struct vas_window *win, char *name,
s32 reg, u64 val)
{
@@ -450,18 +432,4 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
-#ifdef vas_debug
-
-static void print_fifo_msg_count(struct vas_window *txwin)
-{
- uint64_t read_hvwc_reg(struct vas_window *w, char *n, uint64_t o);
- pr_devel("Winid %d, Msg count %llu\n", txwin->winid,
- (uint64_t)read_hvwc_reg(txwin, VREG(LRFIFO_PUSH)));
-}
-#else /* vas_debug */
-
-#define print_fifo_msg_count(window)
-
-#endif /* vas_debug */
-
#endif /* _VAS_H */
--
2.7.4
[PATCH 06/10] powerpc/vas: Reduce polling interval for busy state
A VAS window is normally in "busy" state for only a short duration.
Reduce the time we wait for the window to go to "not-busy" state to
speed-up vas_win_close() a bit.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 10 ++
1 file changed, 6 insertions(+), 4 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 95622a9..1422cdd 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1060,21 +1060,23 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
}
EXPORT_SYMBOL_GPL(vas_paste_crb);
+/*
+ * Wait for the window to go to "not-busy" state. It should only take a
+ * short time to queue a CRB, so window should not be busy for too long.
+ * Trying 5ms intervals.
+ */
static void poll_window_busy_state(struct vas_window *window)
{
int busy;
u64 val;
retry:
- /*
-* Poll Window Busy flag
-*/
val = read_hvwc_reg(window, VREG(WIN_STATUS));
busy = GET_FIELD(VAS_WIN_BUSY, val);
if (busy) {
val = 0;
set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(HZ);
+ schedule_timeout(msecs_to_jiffies(5));
goto retry;
}
}
--
2.7.4
[PATCH 00/10] powerpc/vas: cleanup and optimizations
Sanitize cpu/chip id to VAS id mapping, improve vas_win_close() performance and add a check for return of credits. Also, fix up couple of initializations/error checks and cleanup some comments and debug code. Sukadev Bhattiprolu (10): powerpc/vas: init missing fields from [rt]xattr powerpc/vas: Validate window credits powerpc/vas: Cleanup some debug code powerpc/vas: Drop poll_window_cast_out(). powerpc/vas: Use helper to unpin/close window powerpc/vas: Reduce polling interval for busy state powerpc/vas: Save configured window credits powerpc/vas: poll for return of window credits powerpc/vas: Create cpu to vas id mapping powerpc/vas, nx-842: Define and use chip_to_vas_id() arch/powerpc/include/asm/vas.h | 9 ++ arch/powerpc/platforms/powernv/vas-window.c | 133 +--- arch/powerpc/platforms/powernv/vas.c| 25 +- arch/powerpc/platforms/powernv/vas.h| 61 - drivers/crypto/nx/nx-842-powernv.c | 18 +--- 5 files changed, 154 insertions(+), 92 deletions(-) -- 2.7.4
[PATCH 02/10] powerpc/vas: Validate window credits
NX-842, the only user of VAS, sets the window credits to default values
but VAS should check the credits against the possible max values.
The VAS_WCREDS_MIN is not needed and can be dropped.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 ++
arch/powerpc/platforms/powernv/vas.h| 4 ++--
2 files changed, 8 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index cec7ab7..a2fe120 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -738,6 +738,9 @@ static bool rx_win_args_valid(enum vas_cop_type cop,
if (attr->rx_fifo_size > VAS_RX_FIFO_SIZE_MAX)
return false;
+ if (attr->wcreds_max > VAS_RX_WCREDS_MAX)
+ return false;
+
if (attr->nx_win) {
/* cannot be fault or user window if it is nx */
if (attr->fault_win || attr->user_win)
@@ -927,6 +930,9 @@ static bool tx_win_args_valid(enum vas_cop_type cop,
if (cop > VAS_COP_TYPE_MAX)
return false;
+ if (attr->wcreds_max > VAS_TX_WCREDS_MAX)
+ return false;
+
if (attr->user_win &&
(cop != VAS_COP_TYPE_FTW || attr->rsvd_txbuf_count))
return false;
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 38dee5d..fea0de4 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -106,8 +106,8 @@
*
* TODO: Needs tuning for per-process credits
*/
-#define VAS_WCREDS_MIN 16
-#define VAS_WCREDS_MAX ((64 << 10) - 1)
+#define VAS_RX_WCREDS_MAX ((64 << 10) - 1)
+#define VAS_TX_WCREDS_MAX ((4 << 10) - 1)
#define VAS_WCREDS_DEFAULT (1 << 10)
/*
--
2.7.4
[PATCH 07/10] powerpc/vas: Save configured window credits
Save the configured max window credits for a window in the vas_window
structure. We will need this when polling for return of window credits.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 --
arch/powerpc/platforms/powernv/vas.h| 1 +
2 files changed, 5 insertions(+), 2 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 1422cdd..a59a187 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -674,7 +674,7 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
winctx->rx_fifo = rxattr->rx_fifo;
winctx->rx_fifo_size = rxattr->rx_fifo_size;
- winctx->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ winctx->wcreds_max = rxwin->wcreds_max;
winctx->pin_win = rxattr->pin_win;
winctx->nx_win = rxattr->nx_win;
@@ -844,6 +844,7 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
rxwin->nx_win = rxattr->nx_win;
rxwin->user_win = rxattr->user_win;
rxwin->cop = cop;
+ rxwin->wcreds_max = rxattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
if (rxattr->user_win)
rxwin->pid = task_pid_vnr(current);
@@ -893,7 +894,7 @@ static void init_winctx_for_txwin(struct vas_window *txwin,
*/
memset(winctx, 0, sizeof(struct vas_winctx));
- winctx->wcreds_max = txattr->wcreds_max ?: VAS_WCREDS_DEFAULT;
+ winctx->wcreds_max = txwin->wcreds_max;
winctx->user_win = txattr->user_win;
winctx->nx_win = txwin->rxwin->nx_win;
@@ -978,6 +979,7 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
txwin->nx_win = txwin->rxwin->nx_win;
txwin->pid = attr->pid;
txwin->user_win = attr->user_win;
+ txwin->wcreds_max = attr->wcreds_max ?: VAS_WCREDS_DEFAULT;
init_winctx_for_txwin(txwin, attr, &winctx);
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index 15d2dfa..ad906f6 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -332,6 +332,7 @@ struct vas_window {
void *hvwc_map; /* HV window context */
void *uwc_map; /* OS/User window context */
pid_t pid; /* Linux process id of owner */
+ int wcreds_max; /* Window credits */
/* Fields applicable only to send windows */
void *paste_kaddr;
--
2.7.4
[PATCH 05/10] powerpc/vas: Use helper to unpin/close window
Use a helper to have the hardware unpin and mark a window closed.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 22 +++---
1 file changed, 15 insertions(+), 7 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 8ab8a82..95622a9 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1101,6 +1101,20 @@ static void poll_window_castout(struct vas_window
*window)
}
/*
+ * Unpin and close a window so no new requests are accepted and the
+ * hardware can evict this window from cache if necessary.
+ */
+static void unpin_close_window(struct vas_window *window)
+{
+ u64 val;
+
+ val = read_hvwc_reg(window, VREG(WINCTL));
+ val = SET_FIELD(VAS_WINCTL_PIN, val, 0);
+ val = SET_FIELD(VAS_WINCTL_OPEN, val, 0);
+ write_hvwc_reg(window, VREG(WINCTL), val);
+}
+
+/*
* Close a window.
*
* See Section 1.12.1 of VAS workbook v1.05 for details on closing window:
@@ -1114,8 +1128,6 @@ static void poll_window_castout(struct vas_window *window)
*/
int vas_win_close(struct vas_window *window)
{
- u64 val;
-
if (!window)
return 0;
@@ -1131,11 +1143,7 @@ int vas_win_close(struct vas_window *window)
poll_window_busy_state(window);
- /* Unpin window from cache and close it */
- val = read_hvwc_reg(window, VREG(WINCTL));
- val = SET_FIELD(VAS_WINCTL_PIN, val, 0);
- val = SET_FIELD(VAS_WINCTL_OPEN, val, 0);
- write_hvwc_reg(window, VREG(WINCTL), val);
+ unpin_close_window(window);
poll_window_castout(window);
--
2.7.4
[PATCH 03/10] powerpc/vas: Cleanup some debug code
Cleanuup vas.h and the debug code around ifdef vas_debug.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 8 +++--
arch/powerpc/platforms/powernv/vas.h| 56 +++--
2 files changed, 18 insertions(+), 46 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a2fe120..67ffc5d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -726,7 +726,10 @@ static void init_winctx_for_rxwin(struct vas_window *rxwin,
static bool rx_win_args_valid(enum vas_cop_type cop,
struct vas_rx_win_attr *attr)
{
- dump_rx_win_attr(attr);
+ pr_debug("Rxattr: fault %d, notify %d, intr %d, early %d, fifo %d\n",
+ attr->fault_win, attr->notify_disable,
+ attr->intr_disable, attr->notify_early,
+ attr->rx_fifo_size);
if (cop >= VAS_COP_TYPE_MAX)
return false;
@@ -1050,7 +1053,8 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
else
rc = -EINVAL;
- print_fifo_msg_count(txwin);
+ pr_debug("Txwin #%d: Msg count %llu\n", txwin->winid,
+ read_hvwc_reg(txwin, VREG(LRFIFO_PUSH)));
return rc;
}
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
index fea0de4..15d2dfa 100644
--- a/arch/powerpc/platforms/powernv/vas.h
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -259,6 +259,16 @@
#define VAS_NX_UTIL_ADDER PPC_BITMASK(32, 63)
/*
+ * VREG(x):
+ * Expand a register's short name (eg: LPID) into two parameters:
+ * - the register's short name in string form ("LPID"), and
+ * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
+ * register's offset in the window context
+ */
+#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
+#define VREG(r)VREG_SFX(r, _OFFSET)
+
+/*
* Local Notify Scope Control Register. (Receive windows only).
*/
enum vas_notify_scope {
@@ -385,43 +395,15 @@ struct vas_winctx {
extern struct vas_instance *find_vas_instance(int vasid);
-/*
- * VREG(x):
- * Expand a register's short name (eg: LPID) into two parameters:
- * - the register's short name in string form ("LPID"), and
- * - the name of the macro (eg: VAS_LPID_OFFSET), defining the
- * register's offset in the window context
- */
-#define VREG_SFX(n, s) __stringify(n), VAS_##n##s
-#define VREG(r)VREG_SFX(r, _OFFSET)
-
-#ifdef vas_debug
-static inline void dump_rx_win_attr(struct vas_rx_win_attr *attr)
-{
- pr_err("fault %d, notify %d, intr %d early %d\n",
- attr->fault_win, attr->notify_disable,
- attr->intr_disable, attr->notify_early);
-
- pr_err("rx_fifo_size %d, max value %d\n",
- attr->rx_fifo_size, VAS_RX_FIFO_SIZE_MAX);
-}
-
static inline void vas_log_write(struct vas_window *win, char *name,
void *regptr, u64 val)
{
- if (val)
- pr_err("%swin #%d: %s reg %p, val 0x%016llx\n",
+ if (!val)
+ pr_debug("%swin #%d: %s reg %p, val 0x%016llx\n",
win->tx_win ? "Tx" : "Rx", win->winid, name,
regptr, val);
}
-#else /* vas_debug */
-
-#define vas_log_write(win, name, reg, val)
-#define dump_rx_win_attr(attr)
-
-#endif /* vas_debug */
-
static inline void write_uwc_reg(struct vas_window *win, char *name,
s32 reg, u64 val)
{
@@ -450,18 +432,4 @@ static inline u64 read_hvwc_reg(struct vas_window *win,
return in_be64(win->hvwc_map+reg);
}
-#ifdef vas_debug
-
-static void print_fifo_msg_count(struct vas_window *txwin)
-{
- uint64_t read_hvwc_reg(struct vas_window *w, char *n, uint64_t o);
- pr_devel("Winid %d, Msg count %llu\n", txwin->winid,
- (uint64_t)read_hvwc_reg(txwin, VREG(LRFIFO_PUSH)));
-}
-#else /* vas_debug */
-
-#define print_fifo_msg_count(window)
-
-#endif /* vas_debug */
-
#endif /* _VAS_H */
--
2.7.4
[PATCH 09/10] powerpc/vas: Create cpu to vas id mapping
Create a cpu to vasid mapping so callers can specify -1 instead of
trying to find a VAS id.
Changelog[v2]
[Michael Ellerman] Use per-cpu variables to simplify code.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas.c | 14 +-
1 file changed, 13 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index 565a487..abb7090 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -18,15 +18,18 @@
#include
#include
#include
+#include
#include "vas.h"
static DEFINE_MUTEX(vas_mutex);
static LIST_HEAD(vas_instances);
+static DEFINE_PER_CPU(int, cpu_vas_id);
+
static int init_vas_instance(struct platform_device *pdev)
{
- int rc, vasid;
+ int rc, cpu, vasid;
struct resource *res;
struct vas_instance *vinst;
struct device_node *dn = pdev->dev.of_node;
@@ -74,6 +77,11 @@ static int init_vas_instance(struct platform_device *pdev)
"paste_win_id_shift 0x%llx\n", pdev->name, vasid,
vinst->paste_base_addr, vinst->paste_win_id_shift);
+ for_each_possible_cpu(cpu) {
+ if (cpu_to_chip_id(cpu) == of_get_ibm_chip_id(dn))
+ per_cpu(cpu_vas_id, cpu) = vasid;
+ }
+
mutex_lock(&vas_mutex);
list_add(&vinst->node, &vas_instances);
mutex_unlock(&vas_mutex);
@@ -98,6 +106,10 @@ struct vas_instance *find_vas_instance(int vasid)
struct vas_instance *vinst;
mutex_lock(&vas_mutex);
+
+ if (vasid == -1)
+ vasid = per_cpu(cpu_vas_id, smp_processor_id());
+
list_for_each(ent, &vas_instances) {
vinst = list_entry(ent, struct vas_instance, node);
if (vinst->vas_id == vasid) {
--
2.7.4
[PATCH 08/10] powerpc/vas: poll for return of window credits
Normally, the NX driver waits for the CRBs to be processed before closing
the window. But it is better to ensure that the credits are returned before
the window gets reassigned later.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 45 +
1 file changed, 45 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index a59a187..8e14ce1 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1063,6 +1063,49 @@ int vas_paste_crb(struct vas_window *txwin, int offset,
bool re)
EXPORT_SYMBOL_GPL(vas_paste_crb);
/*
+ * If credit checking is enabled for this window, poll for the return
+ * of window credits (i.e for NX engines to process any outstanding CRBs).
+ * Since NX-842 waits for the CRBs to be processed before closing the
+ * window, we should not have to wait for too long.
+ *
+ * TODO: We retry in 10ms intervals now. We could/should probably peek at
+ * the VAS_LRFIFO_PUSH_OFFSET register to get an estimate of pending
+ * CRBs on the FIFO and compute the delay dynamically on each retry.
+ * But that is not really needed until we support NX-GZIP access from
+ * user space. (NX-842 driver waits for CSB and Fast thread-wakeup
+ * doesn't use credit checking).
+ */
+static void poll_window_credits(struct vas_window *window)
+{
+ u64 val;
+ int creds, mode;
+
+ val = read_hvwc_reg(window, VREG(WINCTL));
+ if (window->tx_win)
+ mode = GET_FIELD(VAS_WINCTL_TX_WCRED_MODE, val);
+ else
+ mode = GET_FIELD(VAS_WINCTL_RX_WCRED_MODE, val);
+
+ if (!mode)
+ return;
+retry:
+ if (window->tx_win) {
+ val = read_hvwc_reg(window, VREG(TX_WCRED));
+ creds = GET_FIELD(VAS_TX_WCRED, val);
+ } else {
+ val = read_hvwc_reg(window, VREG(LRX_WCRED));
+ creds = GET_FIELD(VAS_LRX_WCRED, val);
+ }
+
+ if (creds < window->wcreds_max) {
+ val = 0;
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(msecs_to_jiffies(10));
+ goto retry;
+ }
+}
+
+/*
* Wait for the window to go to "not-busy" state. It should only take a
* short time to queue a CRB, so window should not be busy for too long.
* Trying 5ms intervals.
@@ -1149,6 +1192,8 @@ int vas_win_close(struct vas_window *window)
unpin_close_window(window);
+ poll_window_credits(window);
+
poll_window_castout(window);
/* if send window, drop reference to matching receive window */
--
2.7.4
[PATCH 10/10] powerpc/vas, nx-842: Define and use chip_to_vas_id()
Define a helper, chip_to_vas_id() to map a given chip id to corresponding
vas id.
Normally, callers of vas_rx_win_open() and vas_tx_win_open() would need
the VAS window to be on the same chip where the calling thread is executing.
These callers can pass in -1 for the VAS id.
This interface will be useful if a thread running on one chip wants to open
a window on another chip (like the NX-842 driver does during start up).
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/include/asm/vas.h | 9 +
arch/powerpc/platforms/powernv/vas.c | 11 +++
drivers/crypto/nx/nx-842-powernv.c | 18 +++---
3 files changed, 23 insertions(+), 15 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index fd5963a..044748f 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -104,6 +104,15 @@ struct vas_tx_win_attr {
};
/*
+ * Helper to map a chip id to VAS id.
+ * For POWER9, this is a 1:1 mapping. In the future this maybe a 1:N
+ * mapping in which case, we will need to update this helper.
+ *
+ * Return the VAS id or -1 if no matching vasid is found.
+ */
+int chip_to_vas_id(int chipid);
+
+/*
* Helper to initialize receive window attributes to defaults for an
* NX window.
*/
diff --git a/arch/powerpc/platforms/powernv/vas.c
b/arch/powerpc/platforms/powernv/vas.c
index abb7090..cd9a733 100644
--- a/arch/powerpc/platforms/powernv/vas.c
+++ b/arch/powerpc/platforms/powernv/vas.c
@@ -123,6 +123,17 @@ struct vas_instance *find_vas_instance(int vasid)
return NULL;
}
+int chip_to_vas_id(int chipid)
+{
+ int cpu;
+
+ for_each_possible_cpu(cpu) {
+ if (cpu_to_chip_id(cpu) == chipid)
+ return per_cpu(cpu_vas_id, cpu);
+ }
+ return -1;
+}
+
static int vas_probe(struct platform_device *pdev)
{
return init_vas_instance(pdev);
diff --git a/drivers/crypto/nx/nx-842-powernv.c
b/drivers/crypto/nx/nx-842-powernv.c
index 874ddf5..eb221ed 100644
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ b/drivers/crypto/nx/nx-842-powernv.c
@@ -847,24 +847,12 @@ static int __init nx842_powernv_probe_vas(struct
device_node *pn)
return -EINVAL;
}
- for_each_compatible_node(dn, NULL, "ibm,power9-vas-x") {
- if (of_get_ibm_chip_id(dn) == chip_id)
- break;
- }
-
- if (!dn) {
- pr_err("Missing VAS device node\n");
+ vasid = chip_to_vas_id(chip_id);
+ if (vasid < 0) {
+ pr_err("Unable to map chip_id %d to vasid\n", chip_id);
return -EINVAL;
}
- if (of_property_read_u32(dn, "ibm,vas-id", &vasid)) {
- pr_err("Missing ibm,vas-id device property\n");
- of_node_put(dn);
- return -EINVAL;
- }
-
- of_node_put(dn);
-
for_each_child_of_node(pn, dn) {
if (of_device_is_compatible(dn, "ibm,p9-nx-842")) {
ret = vas_cfg_coproc_info(dn, chip_id, vasid);
--
2.7.4
[PATCH 04/10] powerpc/vas: Drop poll_window_cast_out().
Polling for window cast out is listed in the spec, but turns out that
it is not strictly necessary and slows down window close. Making it a
stub for now.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 34 ++---
1 file changed, 17 insertions(+), 17 deletions(-)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 67ffc5d..8ab8a82 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -1079,25 +1079,25 @@ static void poll_window_busy_state(struct vas_window
*window)
}
}
+/*
+ * Have the hardware cast a window out of cache and wait for it to
+ * be completed.
+ *
+ * NOTE: It can take a relatively long time to cast the window context
+ * out of the cache. It is not strictly necessary to cast out if:
+ *
+ * - we clear the "Pin Window" bit (so hardware is free to evict)
+ *
+ * - we re-initialize the window context when it is reassigned.
+ *
+ * We do the former in vas_win_close() and latter in vas_win_open().
+ * So, ignoring the cast-out for now. We can add it as needed. If
+ * casting out becomes necessary we should consider offloading the
+ * job to a worker thread, so the window close can proceed quickly.
+ */
static void poll_window_castout(struct vas_window *window)
{
- int cached;
- u64 val;
-
- /* Cast window context out of the cache */
-retry:
- val = read_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL));
- cached = GET_FIELD(VAS_WIN_CACHE_STATUS, val);
- if (cached) {
- val = 0ULL;
- val = SET_FIELD(VAS_CASTOUT_REQ, val, 1);
- val = SET_FIELD(VAS_PUSH_TO_MEM, val, 0);
- write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), val);
-
- set_current_state(TASK_UNINTERRUPTIBLE);
- schedule_timeout(HZ);
- goto retry;
- }
+ /* stub for now */
}
/*
--
2.7.4
[PATCH 01/10] powerpc/vas: init missing fields from [rt]xattr
Initialize a few missing window context fields from the window attributes
specified by the caller. These fields are currently set to their default
values by the caller (NX-842), but would be good to apply them anyway.
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 6 ++
1 file changed, 6 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 5aae845..cec7ab7 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -679,10 +679,13 @@ static void init_winctx_for_rxwin(struct vas_window
*rxwin,
winctx->nx_win = rxattr->nx_win;
winctx->fault_win = rxattr->fault_win;
+ winctx->user_win = rxattr->user_win;
+ winctx->rej_no_credit = rxattr->rej_no_credit;
winctx->rx_word_mode = rxattr->rx_win_ord_mode;
winctx->tx_word_mode = rxattr->tx_win_ord_mode;
winctx->rx_wcred_mode = rxattr->rx_wcred_mode;
winctx->tx_wcred_mode = rxattr->tx_wcred_mode;
+ winctx->notify_early = rxattr->notify_early;
if (winctx->nx_win) {
winctx->data_stamp = true;
@@ -889,11 +892,14 @@ static void init_winctx_for_txwin(struct vas_window
*txwin,
winctx->user_win = txattr->user_win;
winctx->nx_win = txwin->rxwin->nx_win;
winctx->pin_win = txattr->pin_win;
+ winctx->rej_no_credit = txattr->rej_no_credit;
+ winctx->rsvd_txbuf_enable = txattr->rsvd_txbuf_enable;
winctx->rx_wcred_mode = txattr->rx_wcred_mode;
winctx->tx_wcred_mode = txattr->tx_wcred_mode;
winctx->rx_word_mode = txattr->rx_win_ord_mode;
winctx->tx_word_mode = txattr->tx_win_ord_mode;
+ winctx->rsvd_txbuf_count = txattr->rsvd_txbuf_count;
if (winctx->nx_win) {
winctx->data_stamp = true;
--
2.7.4
[RFC PATCH v3 1/1] powerpc: Add support for setting SPRN_TIDR
We need the SPRN_TIDR to be set for use with fast thread-wakeup (core-
to-core wakeup) and also with CAPI.
Each thread in a process needs to have a unique id within the process.
But as explained below, for now, we assign globally unique thread ids
to all threads in the system.
Signed-off-by: Sukadev Bhattiprolu
Signed-off-by: Philippe Bergheaud
Signed-off-by: Christophe Lombard
---
Changelog[v3]
- Merge changes with and address comments to Christophe's patch.
(i.e drop CONFIG_PPC_VAS; use CONFIG_PPC64; check CPU_ARCH_300
before setting TIDR). Defer following to separate patches:
- emulation parts of Christophe's patch,
- setting TIDR for tasks other than 'current'
- setting feature bit in AT_HWCAP2
Changelog[v2]
- Michael Ellerman: Use an interface to assign TIDR so it is
assigned to only threads that need it; move assignment to
restore_sprs(). Drop lint from rebase;
---
arch/powerpc/include/asm/processor.h | 1 +
arch/powerpc/include/asm/switch_to.h | 3 +
arch/powerpc/kernel/process.c| 120 +++
3 files changed, 124 insertions(+)
diff --git a/arch/powerpc/include/asm/processor.h
b/arch/powerpc/include/asm/processor.h
index fab7ff8..58cc212 100644
--- a/arch/powerpc/include/asm/processor.h
+++ b/arch/powerpc/include/asm/processor.h
@@ -329,6 +329,7 @@ struct thread_struct {
*/
int dscr_inherit;
unsigned long ppr;/* used to save/restore SMT priority */
+ unsigned long tidr;
#endif
#ifdef CONFIG_PPC_BOOK3S_64
unsigned long tar;
diff --git a/arch/powerpc/include/asm/switch_to.h
b/arch/powerpc/include/asm/switch_to.h
index 17c8380..f5da32f 100644
--- a/arch/powerpc/include/asm/switch_to.h
+++ b/arch/powerpc/include/asm/switch_to.h
@@ -91,4 +91,7 @@ static inline void clear_task_ebb(struct task_struct *t)
#endif
}
+extern int set_thread_tidr(struct task_struct *t);
+extern void clear_thread_tidr(struct task_struct *t);
+
#endif /* _ASM_POWERPC_SWITCH_TO_H */
diff --git a/arch/powerpc/kernel/process.c b/arch/powerpc/kernel/process.c
index a0c74bb..8e992e9 100644
--- a/arch/powerpc/kernel/process.c
+++ b/arch/powerpc/kernel/process.c
@@ -1120,6 +1120,13 @@ static inline void restore_sprs(struct thread_struct
*old_thread,
mtspr(SPRN_TAR, new_thread->tar);
}
#endif
+#ifdef CONFIG_PPC64
+ if (old_thread->tidr != new_thread->tidr) {
+ /* TIDR should be non-zero only with ISA3.0. */
+ WARN_ON_ONCE(!cpu_has_feature(CPU_FTR_ARCH_300));
+ mtspr(SPRN_TIDR, new_thread->tidr);
+ }
+#endif
}
#ifdef CONFIG_PPC_BOOK3S_64
@@ -1434,9 +1441,117 @@ void flush_thread(void)
#endif /* CONFIG_HAVE_HW_BREAKPOINT */
}
+#ifdef CONFIG_PPC64
+static DEFINE_SPINLOCK(vas_thread_id_lock);
+static DEFINE_IDA(vas_thread_ida);
+
+/*
+ * We need to assign a unique thread id to each thread in a process.
+ *
+ * This thread id, referred to as TIDR, and separate from the Linux's tgid,
+ * is intended to be used to direct an ASB_Notify from the hardware to the
+ * thread, when a suitable event occurs in the system.
+ *
+ * One such event is a "paste" instruction in the context of Fast Thread
+ * Wakeup (aka Core-to-core wake up in the Virtual Accelerator Switchboard
+ * (VAS) in POWER9.
+ *
+ * To get a unique TIDR per process we could simply reuse task_pid_nr() but
+ * the problem is that task_pid_nr() is not yet available copy_thread() is
+ * called. Fixing that would require changing more intrusive arch-neutral
+ * code in code path in copy_process()?.
+ *
+ * Further, to assign unique TIDRs within each process, we need an atomic
+ * field (or an IDR) in task_struct, which again intrudes into the arch-
+ * neutral code. So try to assign globally unique TIDRs for now.
+ *
+ * NOTE: TIDR 0 indicates that the thread does not need a TIDR value.
+ * For now, only threads that expect to be notified by the VAS
+ * hardware need a TIDR value and we assign values > 0 for those.
+ */
+#define MAX_THREAD_CONTEXT ((1 << 16) - 1)
+static int assign_thread_tidr(void)
+{
+ int index;
+ int err;
+
+again:
+ if (!ida_pre_get(&vas_thread_ida, GFP_KERNEL))
+ return -ENOMEM;
+
+ spin_lock(&vas_thread_id_lock);
+ err = ida_get_new_above(&vas_thread_ida, 1, &index);
+ spin_unlock(&vas_thread_id_lock);
+
+ if (err == -EAGAIN)
+ goto again;
+ else if (err)
+ return err;
+
+ if (index > MAX_THREAD_CONTEXT) {
+ spin_lock(&vas_thread_id_lock);
+ ida_remove(&vas_thread_ida, index);
+ spin_unlock(&vas_thread_id_lock);
+ return -ENOMEM;
+ }
+
+ return index;
+}
+
+static void free_thread_tid
[tip:perf/urgent] perf vendor events powerpc: Remove duplicate events
Commit-ID: 2a118e1bd22cad57318520d37e3a184b8846c6a2
Gitweb: http://git.kernel.org/tip/2a118e1bd22cad57318520d37e3a184b8846c6a2
Author: Sukadev Bhattiprolu
AuthorDate: Wed, 30 Aug 2017 21:42:23 -0400
Committer: Arnaldo Carvalho de Melo
CommitDate: Fri, 1 Sep 2017 14:46:00 -0300
perf vendor events powerpc: Remove duplicate events
Some POWER PMU event names have multiple/alternate event codes. These
alternate event codes were listed in the POWER9 JSON files for
reference.
But the perf tool does not seem to handle duplicates cleanly. 'perf
list' shows such duplicate events only once, but 'perf stat' ends up
counting the first event code twice, multiplexing if necessary and we
end up with double the event counts.
Remove the duplicate event codes from the JSON files for now.
Reported-by: Michael Petlan
Signed-off-by: Sukadev Bhattiprolu
Cc: Andi Kleen
Cc: Anton Blanchard
Cc: Jiri Olsa
Cc: Michael Ellerman
Link: http://lkml.kernel.org/r/20170830231506.gb20...@us.ibm.com
Signed-off-by: Arnaldo Carvalho de Melo
---
.../pmu-events/arch/powerpc/power9/frontend.json | 7 +-
.../perf/pmu-events/arch/powerpc/power9/other.json | 120 -
.../pmu-events/arch/powerpc/power9/pipeline.json | 7 +-
tools/perf/pmu-events/arch/powerpc/power9/pmc.json | 7 +-
4 files changed, 3 insertions(+), 138 deletions(-)
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
b/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
index 7e62c46..c63a919 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/frontend.json
@@ -80,11 +80,6 @@
"BriefDescription": "Load Missed L1, counted at execution time (can be
greater than loads finished). LMQ merges are not included in this count. i.e.
if a load instruction misses on an address that is already allocated on the
LMQ, this event will not increment for that load). Note that this count is per
slice, so if a load spans multiple slices this event will increment multiple
times for a single load."
},
{,
-"EventCode": "0x400F0",
-"EventName": "PM_LD_MISS_L1",
-"BriefDescription": "Load Missed L1, counted at execution time (can be
greater than loads finished). LMQ merges are not included in this count. i.e.
if a load instruction misses on an address that is already allocated on the
LMQ, this event will not increment for that load). Note that this count is per
slice, so if a load spans multiple slices this event will increment multiple
times for a single load."
- },
- {,
"EventCode": "0x2E01A",
"EventName": "PM_CMPLU_STALL_LSU_FLUSH_NEXT",
"BriefDescription": "Completion stall of one cycle because the LSU
requested to flush the next iop in the sequence. It takes 1 cycle for the ISU
to process this request before the LSU instruction is allowed to complete"
@@ -374,4 +369,4 @@
"EventName": "PM_IPTEG_FROM_L31_ECO_MOD",
"BriefDescription": "A Page Table Entry was loaded into the TLB with
Modified (M) data from another core's ECO L3 on the same chip due to a
instruction side request"
}
-]
\ No newline at end of file
+]
diff --git a/tools/perf/pmu-events/arch/powerpc/power9/other.json
b/tools/perf/pmu-events/arch/powerpc/power9/other.json
index 00f3d2a..54cc3be 100644
--- a/tools/perf/pmu-events/arch/powerpc/power9/other.json
+++ b/tools/perf/pmu-events/arch/powerpc/power9/other.json
@@ -605,11 +605,6 @@
"BriefDescription": "RC retries on PB for any load from core (excludes
DCBFs)"
},
{,
-"EventCode": "0x3689E",
-"EventName": "PM_L2_RTY_LD",
-"BriefDescription": "RC retries on PB for any load from core (excludes
DCBFs)"
- },
- {,
"EventCode": "0xE08C",
"EventName": "PM_LSU0_ERAT_HIT",
"BriefDescription": "Primary ERAT hit. There is no secondary ERAT"
@@ -715,11 +710,6 @@
"BriefDescription": "Lifetime, sample of RD machine 0 valid"
},
{,
-"EventCode": "0x468B4",
-"EventName": "PM_L3_RD0_BUSY",
-"BriefDescription": "Lifetime, sample of RD machine 0 valid"
- },
- {,
"EventCode": "0x46080",
"EventName": "PM_L2_DISP_ALL_L2MISS",
"BriefDescription": "All successful Ld/St dispatches for this thread that
were an L2 miss (excludes i_l2mru_tch_reqs)"
@@ -850,21 +840,11 @@
"BriefDescription": "RC mach 0 Busy. Used by PMU to sample ave RC lifetime
(mach0 used as sample point)"
},
{,
-"EventCode": "0
[GIT PULL] Please pull JSON files for POWER9 PMU events
Hi Arnaldo, Please pull an update to the JSON files for POWER9 PMU events. This removes alternate event codes from the JSON files which seem to confuse perf. The following changes since commit 1b2f76d77a277bb70d38ad0991ed7f16bbc115a9: Merge tag 'perf-core-for-mingo-4.14-20170829' of git://git.kernel.org/pub/scm/linux/kernel/git/acme/linux into perf/core (2017-08-29 23:13:56 +0200) are available in the git repository at: https://github.com/sukadev/linux/ p9-json-v3 Sukadev Bhattiprolu (1): perf vendor events powerpc: remove duplicate events .../pmu-events/arch/powerpc/power9/frontend.json |7 +- .../perf/pmu-events/arch/powerpc/power9/other.json | 120 .../pmu-events/arch/powerpc/power9/pipeline.json |7 +- tools/perf/pmu-events/arch/powerpc/power9/pmc.json |7 +- 4 files changed, 3 insertions(+), 138 deletions(-)
[PATCH v8 01/10] powerpc/vas: Define macros, register fields and structures
Define macros for the VAS hardware registers and bit-fields as well
as couple of data structures needed by the VAS driver.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]
- Use u64/u32 instead of the uintXX versions.
Changelog[v7]
- Move the threshold control macros from uapi/asm/vas.h to
asm/vas.h for now. When we actually have an user space need for
them, we can move them to uapi/asm/vas.h. With this change,
uapi/asm/vas.h is empty and can be dropped from this patch.
Changelog[v6]
- Add some fields for FTW windows
Changelog[v4]
- [Michael Neuling] Move VAS code to arch/powerpc; Reorg vas.h and
vas-internal.h to kernel and uapi versions; rather than creating
separate properties for window context/address entries in device
tree, combine them into "reg" properties; drop ->hwirq and irq_port
fields from vas_window as they are only needed with user space
windows.
- Drop the error check for CONFIG_PPC_4K_PAGES. Instead in a
follow-on patch add a "depends on CONFIG_PPC_64K_PAGES".
Changelog[v3]
- Rename winctx->pid to winctx->pidr to reflect that its a value
from the PID register (SPRN_PID), not the linux process id.
- Make it easier to split header into kernel/user parts
- To keep user interface simple, use macros rather than enum for
the threshold-control modes.
- Add a pid field to struct vas_window - needed for user space
send windows.
Changelog[v2]
- Add an overview of VAS in vas-internal.h
- Get window context parameters from device tree and drop
unnecessary macros.
---
arch/powerpc/include/asm/vas.h | 45 +
arch/powerpc/platforms/powernv/vas.h | 382 +++
2 files changed, 427 insertions(+)
create mode 100644 arch/powerpc/include/asm/vas.h
create mode 100644 arch/powerpc/platforms/powernv/vas.h
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
new file mode 100644
index 000..ff87e44
--- /dev/null
+++ b/arch/powerpc/include/asm/vas.h
@@ -0,0 +1,45 @@
+/*
+ * Copyright 2016-17 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _MISC_VAS_H
+#define _MISC_VAS_H
+
+/*
+ * Min and max FIFO sizes are based on Version 1.05 Section 3.1.4.25
+ * (Local FIFO Size Register) of the VAS workbook.
+ */
+#define VAS_RX_FIFO_SIZE_MIN (1 << 10) /* 1KB */
+#define VAS_RX_FIFO_SIZE_MAX (8 << 20) /* 8MB */
+
+/*
+ * Threshold Control Mode: Have paste operation fail if the number of
+ * requests in receive FIFO exceeds a threshold.
+ *
+ * NOTE: No special error code yet if paste is rejected because of these
+ * limits. So users can't distinguish between this and other errors.
+ */
+#define VAS_THRESH_DISABLED0
+#define VAS_THRESH_FIFO_GT_HALF_FULL 1
+#define VAS_THRESH_FIFO_GT_QTR_FULL2
+#define VAS_THRESH_FIFO_GT_EIGHTH_FULL 3
+
+/*
+ * Co-processor Engine type.
+ */
+enum vas_cop_type {
+ VAS_COP_TYPE_FAULT,
+ VAS_COP_TYPE_842,
+ VAS_COP_TYPE_842_HIPRI,
+ VAS_COP_TYPE_GZIP,
+ VAS_COP_TYPE_GZIP_HIPRI,
+ VAS_COP_TYPE_FTW,
+ VAS_COP_TYPE_MAX,
+};
+
+#endif /* _MISC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas.h
b/arch/powerpc/platforms/powernv/vas.h
new file mode 100644
index 000..abb545f
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/vas.h
@@ -0,0 +1,382 @@
+/*
+ * Copyright 2016-17 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+
+#ifndef _VAS_H
+#define _VAS_H
+#include
+#include
+#include
+
+/*
+ * Overview of Virtual Accelerator Switchboard (VAS).
+ *
+ * VAS is a hardware "switchboard" that allows senders and receivers to
+ * exchange messages with _minimal_ kernel involvment. The receivers are
+ * typically NX coprocessor engines that perform compression or encryption
+ * in hardware, but receivers can also be other software threads.
+ *
+ * Senders are user/kernel threads that submit compression/encryption or
+ * other requests to the receivers. Senders must format their messages as
+ * Coprocessor Request Blocks (CRB)s and submit them using the "copy" and
+ * "paste" instructions which were introduced in Power9.
+ *
+ * A Power node can have (upto?) 8 Power chips. There is one instance of
+ * VAS in each Power9 chip. Each instance of VAS has 64K windows or ports,
+ * Senders and r
[PATCH v8 03/10] powerpc/vas: Define vas_init() and vas_exit()
Implement vas_init() and vas_exit() functions for a new VAS module.
This VAS module is essentially a library for other device drivers
and kernel users of the NX coprocessors like NX-842 and NX-GZIP.
In the future this will be extended to add support for user space
to access the NX coprocessors.
VAS is currently only supported with 64K page size.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]:
- [Michael Ellerman] VAS should be built-in and not a module;
drop the vas_exit() and free_vinst() code since its not
a module and might need some refcounting. Drop init_done,
->ready and unnecessary "len" fields in vinst; Misc cleanup.
Changelog[v5]:
- [Ben Herrenschmidt]: Create and use platform device tree nodes,
fix up the "reg" properties for the VAS DT node and use the
platform device helpers to parse the reg properties; Use linked
list of VAS instances (don't assume vasids are sequential);
Use CONFIG_PPC_VAS instead of CONFIG_VAS.
Changelog[v4]:
- [Michael Neuling] Fix some accidental deletions; fix help text
in Kconfig; change vas_initialized to a function; move from
drivers/misc to arch/powerpc/kernel
- Drop the vas_window_reset() interface. It is not needed as
window will be initialized before each use.
- Add a "depends on PPC_64K_PAGES"
Changelog[v3]:
- Zero vas_instances memory on allocation
- [Haren Myneni] Fix description in Kconfig
Changelog[v2]:
- Get HVWC, UWC and window address parameters from device tree.
---
.../devicetree/bindings/powerpc/ibm,vas.txt| 23
MAINTAINERS| 8 ++
arch/powerpc/platforms/powernv/Kconfig | 14 ++
arch/powerpc/platforms/powernv/Makefile| 1 +
arch/powerpc/platforms/powernv/vas-window.c| 19 +++
arch/powerpc/platforms/powernv/vas.c | 151 +
arch/powerpc/platforms/powernv/vas.h | 2 +
7 files changed, 218 insertions(+)
create mode 100644 Documentation/devicetree/bindings/powerpc/ibm,vas.txt
create mode 100644 arch/powerpc/platforms/powernv/vas-window.c
create mode 100644 arch/powerpc/platforms/powernv/vas.c
diff --git a/Documentation/devicetree/bindings/powerpc/ibm,vas.txt
b/Documentation/devicetree/bindings/powerpc/ibm,vas.txt
new file mode 100644
index 000..a096d73
--- /dev/null
+++ b/Documentation/devicetree/bindings/powerpc/ibm,vas.txt
@@ -0,0 +1,23 @@
+* IBM Powerpc Virtual Accelerator Switchboard (VAS)
+
+VAS is a hardware mechanism that allows kernel subsystems and user processes
+to directly submit compression and other requests to Nest accelerators (NX)
+or other coprocessors functions.
+
+Required properties:
+- compatible : should be "ibm,vas".
+- ibm,vas-id : A unique identifier for each instance of VAS in the system
+- reg : Should contain 4 pairs of 64-bit fields specifying the Hypervisor
+ window context start and length, OS/User window context start and length,
+ "Paste address" start and length, "Paste window id" start bit and number
+ of bits)
+
+Example:
+
+ vas@60191 {
+ compatible = "ibm,vas", "ibm,power9-vas";
+ reg = <0x60191 0x200 0x60190 0x1
0x8 0x1 0x20 0x10>;
+ name = "vas";
+ ibm,vas-id = <0x1>;
+ };
+
diff --git a/MAINTAINERS b/MAINTAINERS
index 1c3feff..ec68732 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6437,6 +6437,14 @@ L: net...@vger.kernel.org
S: Supported
F: drivers/net/ethernet/ibm/ibmvnic.*
+IBM Power Virtual Accelerator Switchboard
+M: Sukadev Bhattiprolu
+L: linuxppc-...@lists.ozlabs.org
+S: Supported
+F: arch/powerpc/platforms/powernv/vas*
+F: arch/powerpc/include/asm/vas.h
+F: arch/powerpc/include/uapi/asm/vas.h
+
IBM Power Virtual Ethernet Device Driver
M: Thomas Falcon
L: net...@vger.kernel.org
diff --git a/arch/powerpc/platforms/powernv/Kconfig
b/arch/powerpc/platforms/powernv/Kconfig
index 6a6f4ef..3e3bbe9 100644
--- a/arch/powerpc/platforms/powernv/Kconfig
+++ b/arch/powerpc/platforms/powernv/Kconfig
@@ -30,3 +30,17 @@ config OPAL_PRD
help
This enables the opal-prd driver, a facility to run processor
recovery diagnostics on OpenPower machines
+
+config PPC_VAS
+ bool "IBM Virtual Accelerator Switchboard (VAS)"
+ depends on PPC_POWERNV && PPC_64K_PAGES
+ default y
+ help
+ This enables support for IBM Virtual Accelerator Switchboard (VAS).
+
+ VAS allows accelerators in co-processors like NX-GZIP and NX-842
+ to be accessible to kernel subsystems and user processes.
+
+ VAS adapters are found in PO
[PATCH v8 10/10] powerpc/vas: Define copy/paste interfaces
Define interfaces (wrappers) to the 'copy' and 'paste' instructions
(which are new in PowerISA 3.0). These are intended to be used to
by NX driver(s) to submit Coprocessor Request Blocks (CRBs) to the
NX hardware engines.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]:
- [Michael Ellerman] Drop vas_initialized() check; cleanup asm code,
reuse existing macros, fix old references; add cr0 to clobbers
Changelog[v4]
- Export symbols
Changelog[v3]
- Map raw CR value from paste instruction into an error code.
Conflicts:
arch/powerpc/platforms/powernv/vas.h
---
MAINTAINERS | 1 +
arch/powerpc/include/asm/ppc-opcode.h | 2 ++
arch/powerpc/include/asm/vas.h | 12
arch/powerpc/platforms/powernv/copy-paste.h | 46
arch/powerpc/platforms/powernv/vas-window.c | 47 +
arch/powerpc/platforms/powernv/vas.h| 18 +--
6 files changed, 124 insertions(+), 2 deletions(-)
create mode 100644 arch/powerpc/platforms/powernv/copy-paste.h
diff --git a/MAINTAINERS b/MAINTAINERS
index ec68732..624c67a 100644
--- a/MAINTAINERS
+++ b/MAINTAINERS
@@ -6442,6 +6442,7 @@ M: Sukadev Bhattiprolu
L: linuxppc-...@lists.ozlabs.org
S: Supported
F: arch/powerpc/platforms/powernv/vas*
+F: arch/powerpc/platforms/powernv/copy-paste.h
F: arch/powerpc/include/asm/vas.h
F: arch/powerpc/include/uapi/asm/vas.h
diff --git a/arch/powerpc/include/asm/ppc-opcode.h
b/arch/powerpc/include/asm/ppc-opcode.h
index fa9ebae..749336d 100644
--- a/arch/powerpc/include/asm/ppc-opcode.h
+++ b/arch/powerpc/include/asm/ppc-opcode.h
@@ -414,6 +414,8 @@
___PPC_RB(b))
#define PPC_MSGCLRP(b) stringify_in_c(.long PPC_INST_MSGCLRP | \
___PPC_RB(b))
+#define PPC_PASTE(a, b)stringify_in_c(.long PPC_INST_PASTE | \
+ ___PPC_RA(a) | ___PPC_RB(b))
#define PPC_POPCNTB(a, s) stringify_in_c(.long PPC_INST_POPCNTB | \
__PPC_RA(a) | __PPC_RS(s))
#define PPC_POPCNTD(a, s) stringify_in_c(.long PPC_INST_POPCNTD | \
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index efbdde5..dfc97f5 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -145,4 +145,16 @@ struct vas_window *vas_tx_win_open(int vasid, enum
vas_cop_type cop,
*/
int vas_win_close(struct vas_window *win);
+/*
+ * Copy the co-processor request block (CRB) @crb into the local L2 cache.
+ */
+extern int vas_copy_crb(void *crb, int offset);
+
+/*
+ * Paste a previously copied CRB (see vas_copy_crb()) from the L2 cache to
+ * the hardware address associated with the window @win. @re is expected/
+ * assumed to be true for NX windows.
+ */
+extern int vas_paste_crb(struct vas_window *win, int offset, bool re);
+
#endif /* _MISC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/copy-paste.h
b/arch/powerpc/platforms/powernv/copy-paste.h
new file mode 100644
index 000..c9a5036
--- /dev/null
+++ b/arch/powerpc/platforms/powernv/copy-paste.h
@@ -0,0 +1,46 @@
+/*
+ * Copyright 2016-17 IBM Corp.
+ *
+ * This program is free software; you can redistribute it and/or
+ * modify it under the terms of the GNU General Public License
+ * as published by the Free Software Foundation; either version
+ * 2 of the License, or (at your option) any later version.
+ */
+#include
+
+#define CR0_SHIFT 28
+#define CR0_MASK 0xF
+/*
+ * Copy/paste instructions:
+ *
+ * copy RA,RB
+ * Copy contents of address (RA) + effective_address(RB)
+ * to internal copy-buffer.
+ *
+ * paste RA,RB
+ * Paste contents of internal copy-buffer to the address
+ * (RA) + effective_address(RB)
+ */
+static inline int vas_copy(void *crb, int offset)
+{
+ asm volatile(PPC_COPY(%0, %1)";"
+ :
+ : "b" (offset), "b" (crb)
+ : "memory");
+
+ return 0;
+}
+
+static inline int vas_paste(void *paste_address, int offset)
+{
+ u32 cr;
+
+ cr = 0;
+ asm volatile(PPC_PASTE(%1, %2)";"
+ "mfocrf %0, 0x80;"
+ : "=r" (cr)
+ : "b" (offset), "b" (paste_address)
+ : "memory", "cr0");
+
+ return (cr >> CR0_SHIFT) & CR0_MASK;
+}
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index cd12e44..b02f26d 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -18,6 +18,7 @@
#include
#include "vas.h"
+#include "copy-paste.h"
/*
[PATCH v8 05/10] powerpc/vas: Define helpers to init window context
Define helpers to initialize window context registers of the VAS
hardware. These will be used in follow-on patches when opening/closing
VAS windows.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]:
- Update comments (ISA references and some cleanup)
- Use 0 or 1 when setting boolean fields with SET_FIELD()
- Don't write to spare/unused registers.
- Use kernel integer types (u64/u32/s32)
Changelog[v6]
- Add support for FTW windows and drop the fault window id
code since it is not needed for FTW/kernel windows.
Changelog[v5]
- Fix: Copy the FIFO address into LFIFO_BAR register as is (don't
shift address into bits 8:53).
Changelog[v4]
- Michael Neuling] Use ilog2(), radix_enabled() helpers;
drop warning when 32-bit app uses VAS (a follow-on patch
will check and return error). Set MSR_PR state to 0 for
kernel (rather than reading from MSR).
Changelog[v3]
- Have caller, rather than init_xlate_regs() reset window regs
so we don't reset any settings caller may already have set.
- Translation mode should be 0x3 (0b11) not 0x11.
- Skip initilaizing read-only registers NX_UTIL and NX_UTIL_SE
- Skip initializing adder registers from UWC - they are already
initialized from the HVWC.
- Check winctx->user_win when setting translation registers
---
arch/powerpc/platforms/powernv/vas-window.c | 299
arch/powerpc/platforms/powernv/vas.h| 55 +
2 files changed, 354 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 642814a2..68dfe53 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -13,6 +13,7 @@
#include
#include
#include
+#include
#include "vas.h"
@@ -186,6 +187,304 @@ int map_winctx_mmio_bars(struct vas_window *window)
return 0;
}
+/*
+ * Reset all valid registers in the HV and OS/User Window Contexts for
+ * the window identified by @window.
+ *
+ * NOTE: We cannot really use a for loop to reset window context. Not all
+ * offsets in a window context are valid registers and the valid
+ * registers are not sequential. And, we can only write to offsets
+ * with valid registers.
+ */
+void reset_window_regs(struct vas_window *window)
+{
+ write_hvwc_reg(window, VREG(LPID), 0ULL);
+ write_hvwc_reg(window, VREG(PID), 0ULL);
+ write_hvwc_reg(window, VREG(XLATE_MSR), 0ULL);
+ write_hvwc_reg(window, VREG(XLATE_LPCR), 0ULL);
+ write_hvwc_reg(window, VREG(XLATE_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(AMR), 0ULL);
+ write_hvwc_reg(window, VREG(SEIDR), 0ULL);
+ write_hvwc_reg(window, VREG(FAULT_TX_WIN), 0ULL);
+ write_hvwc_reg(window, VREG(OSU_INTR_SRC_RA), 0ULL);
+ write_hvwc_reg(window, VREG(HV_INTR_SRC_RA), 0ULL);
+ write_hvwc_reg(window, VREG(PSWID), 0ULL);
+ write_hvwc_reg(window, VREG(LFIFO_BAR), 0ULL);
+ write_hvwc_reg(window, VREG(LDATA_STAMP_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(LDMA_CACHE_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(LRFIFO_PUSH), 0ULL);
+ write_hvwc_reg(window, VREG(CURR_MSG_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_AFTER_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LRX_WCRED), 0ULL);
+ write_hvwc_reg(window, VREG(LRX_WCRED_ADDER), 0ULL);
+ write_hvwc_reg(window, VREG(TX_WCRED), 0ULL);
+ write_hvwc_reg(window, VREG(TX_WCRED_ADDER), 0ULL);
+ write_hvwc_reg(window, VREG(LFIFO_SIZE), 0ULL);
+ write_hvwc_reg(window, VREG(WINCTL), 0ULL);
+ write_hvwc_reg(window, VREG(WIN_STATUS), 0ULL);
+ write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(TX_RSVD_BUF_COUNT), 0ULL);
+ write_hvwc_reg(window, VREG(LRFIFO_WIN_PTR), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_CTL), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_PID), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_LPID), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_TID), 0ULL);
+ write_hvwc_reg(window, VREG(LNOTIFY_SCOPE), 0ULL);
+ write_hvwc_reg(window, VREG(NX_UTIL_ADDER), 0ULL);
+
+ /* Skip read-only registers: NX_UTIL and NX_UTIL_SE */
+
+ /*
+* The send and receive window credit adder registers are also
+* accessible from HVWC and have been initialized above. We don't
+* need to initialize from the OS/User Window Context, so skip
+* following calls:
+*
+* write_uwc_reg(window, VREG(TX_WCRED_ADDER), 0ULL);
+* write_uwc_reg(window, VREG(LRX_WCRED_ADDER), 0ULL);
+*/
+}
+
+/*
+ * Initialize window context registers related to Address Translation.
+ * These registers are common to send/receive windows although they
+ * d
[PATCH v8 04/10] powerpc/vas: Define helpers to access MMIO regions
Define some helper functions to access the MMIO regions. We use these
in follow-on patches to read/write VAS hardware registers. They are
also used to later issue 'paste' instructions to submit requests to
the NX hardware engines.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog [v8]:
Minor cleanup of error/debug messages
Changelog [v6]:
- Minor reorg to make setup/cleanup functions more symmetric
Changelog [v5]:
- [Ben Herrenschmidt]: Need cachable mapping for paste regions
and non-cachable mapping for the MMIO regions. So, just use
ioremap() for mapping the MMIO regions; use "winctx" instead
of "wc" to avoid collision with "write combine".
Changelog [v3]:
- Minor reorg/cleanup of map/unmap functions
Changelog [v2]:
- Get HVWC, UWC and paste addresses from window->vinst (i.e DT)
rather than kernel macros.
---
arch/powerpc/platforms/powernv/vas-window.c | 174
1 file changed, 174 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index de21acb..642814a2 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -7,11 +7,185 @@
* 2 of the License, or (at your option) any later version.
*/
+#define pr_fmt(fmt) "vas: " fmt
+
#include
#include
+#include
+#include
#include "vas.h"
+/*
+ * Compute the paste address region for the window @window using the
+ * ->paste_base_addr and ->paste_win_id_shift we got from device tree.
+ */
+static void compute_paste_address(struct vas_window *window, u64 *addr, int
*len)
+{
+ int winid;
+ u64 base, shift;
+
+ base = window->vinst->paste_base_addr;
+ shift = window->vinst->paste_win_id_shift;
+ winid = window->winid;
+
+ *addr = base + (winid << shift);
+ if (len)
+ *len = PAGE_SIZE;
+
+ pr_debug("Txwin #%d: Paste addr 0x%llx\n", winid, *addr);
+}
+
+static inline void get_hvwc_mmio_bar(struct vas_window *window,
+ u64 *start, int *len)
+{
+ u64 pbaddr;
+
+ pbaddr = window->vinst->hvwc_bar_start;
+ *start = pbaddr + window->winid * VAS_HVWC_SIZE;
+ *len = VAS_HVWC_SIZE;
+}
+
+static inline void get_uwc_mmio_bar(struct vas_window *window,
+ u64 *start, int *len)
+{
+ u64 pbaddr;
+
+ pbaddr = window->vinst->uwc_bar_start;
+ *start = pbaddr + window->winid * VAS_UWC_SIZE;
+ *len = VAS_UWC_SIZE;
+}
+
+/*
+ * Map the paste bus address of the given send window into kernel address
+ * space. Unlike MMIO regions (map_mmio_region() below), paste region must
+ * be mapped cache-able and is only applicable to send windows.
+ */
+void *map_paste_region(struct vas_window *txwin)
+{
+ int len;
+ void *map;
+ char *name;
+ u64 start;
+
+ name = kasprintf(GFP_KERNEL, "window-v%d-w%d", txwin->vinst->vas_id,
+ txwin->winid);
+ if (!name)
+ goto free_name;
+
+ txwin->paste_addr_name = name;
+ compute_paste_address(txwin, &start, &len);
+
+ if (!request_mem_region(start, len, name)) {
+ pr_devel("%s(): request_mem_region(0x%llx, %d) failed\n",
+ __func__, start, len);
+ goto free_name;
+ }
+
+ map = ioremap_cache(start, len);
+ if (!map) {
+ pr_devel("%s(): ioremap_cache(0x%llx, %d) failed\n", __func__,
+ start, len);
+ goto free_name;
+ }
+
+ pr_devel("Mapped paste addr 0x%llx to kaddr 0x%p\n", start, map);
+ return map;
+
+free_name:
+ kfree(name);
+ return ERR_PTR(-ENOMEM);
+}
+
+
+static void *map_mmio_region(char *name, u64 start, int len)
+{
+ void *map;
+
+ if (!request_mem_region(start, len, name)) {
+ pr_devel("%s(): request_mem_region(0x%llx, %d) failed\n",
+ __func__, start, len);
+ return NULL;
+ }
+
+ map = ioremap(start, len);
+ if (!map) {
+ pr_devel("%s(): ioremap(0x%llx, %d) failed\n", __func__, start,
+ len);
+ return NULL;
+ }
+
+ return map;
+}
+
+static void unmap_region(void *addr, u64 start, int len)
+{
+ iounmap(addr);
+ release_mem_region((phys_addr_t)start, len);
+}
+
+/*
+ * Unmap the paste address region for a window.
+ */
+void unmap_paste_region(struct vas_window *window)
+{
+ int len;
+ u64 busaddr_start;
+
+ if (window->paste_kaddr) {
+ compute_paste_address(window, &busaddr_start, &len);
[PATCH v8 02/10] Move GET_FIELD/SET_FIELD to vas.h
Move the GET_FIELD and SET_FIELD macros to vas.h as VAS and other
users of VAS, including NX-842 can use those macros.
There is a lot of related code between the VAS/NX kernel drivers
and skiboot. For consistency, switch the order of parameters in
SET_FIELD to match the order in skiboot.
Signed-off-by: Sukadev Bhattiprolu
Reviewed-by: Dan Streetman
---
Changelog[v7]
[Michael Ellerman] Move the macros to rather than
to
Changelog[v3]
- Fix order of parameters in nx-842 driver.
---
arch/powerpc/include/asm/vas.h | 8
drivers/crypto/nx/nx-842-powernv.c | 7 ---
drivers/crypto/nx/nx-842.h | 5 -
3 files changed, 12 insertions(+), 8 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index ff87e44..33e93ca 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -30,6 +30,14 @@
#define VAS_THRESH_FIFO_GT_EIGHTH_FULL 3
/*
+ * Get/Set bit fields
+ */
+#define GET_FIELD(m, v)(((v) & (m)) >> MASK_LSH(m))
+#define MASK_LSH(m)(__builtin_ffsl(m) - 1)
+#define SET_FIELD(m, v, val) \
+ (((v) & ~(m)) | typeof(v))(val)) << MASK_LSH(m)) & (m)))
+
+/*
* Co-processor Engine type.
*/
enum vas_cop_type {
diff --git a/drivers/crypto/nx/nx-842-powernv.c
b/drivers/crypto/nx/nx-842-powernv.c
index 1710f80..3abb045 100644
--- a/drivers/crypto/nx/nx-842-powernv.c
+++ b/drivers/crypto/nx/nx-842-powernv.c
@@ -22,6 +22,7 @@
#include
#include
+#include
MODULE_LICENSE("GPL");
MODULE_AUTHOR("Dan Streetman ");
@@ -424,9 +425,9 @@ static int nx842_powernv_function(const unsigned char *in,
unsigned int inlen,
/* set up CCW */
ccw = 0;
- ccw = SET_FIELD(ccw, CCW_CT, nx842_ct);
- ccw = SET_FIELD(ccw, CCW_CI_842, 0); /* use 0 for hw auto-selection */
- ccw = SET_FIELD(ccw, CCW_FC_842, fc);
+ ccw = SET_FIELD(CCW_CT, ccw, nx842_ct);
+ ccw = SET_FIELD(CCW_CI_842, ccw, 0); /* use 0 for hw auto-selection */
+ ccw = SET_FIELD(CCW_FC_842, ccw, fc);
/* set up CRB's CSB addr */
csb_addr = nx842_get_pa(csb) & CRB_CSB_ADDRESS;
diff --git a/drivers/crypto/nx/nx-842.h b/drivers/crypto/nx/nx-842.h
index a4eee3b..30929bd 100644
--- a/drivers/crypto/nx/nx-842.h
+++ b/drivers/crypto/nx/nx-842.h
@@ -100,11 +100,6 @@ static inline unsigned long nx842_get_pa(void *addr)
return page_to_phys(vmalloc_to_page(addr)) + offset_in_page(addr);
}
-/* Get/Set bit fields */
-#define MASK_LSH(m)(__builtin_ffsl(m) - 1)
-#define GET_FIELD(v, m)(((v) & (m)) >> MASK_LSH(m))
-#define SET_FIELD(v, m, val) (((v) & ~(m)) | (((val) << MASK_LSH(m)) & (m)))
-
/**
* This provides the driver's constraints. Different nx842 implementations
* may have varying requirements. The constraints are:
--
2.7.4
[PATCH v8 08/10] powerpc/vas: Define vas_win_close() interface
Define the vas_win_close() interface which should be used to close a
send or receive windows.
While the hardware configurations required to open send and receive windows
differ, the configuration to close a window is the same for both. So we use
a single interface to close the window.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]
- [Michael Ellerman] Set task_state() and pass correct values to
schedule_timeout().
Changelog[v4]:
- Drop the poll for credits return (we can set the required credit,
but cannot really find the available credit at a point in time)
- Export the symbol
Changelog[v3]:
- Fix order of parameters in GET_FIELD().
- Update references and sequence for closing/quiescing a window.
Conflicts:
arch/powerpc/platforms/powernv/vas-window.c
---
arch/powerpc/include/asm/vas.h | 7 ++
arch/powerpc/platforms/powernv/vas-window.c | 101 ++--
2 files changed, 103 insertions(+), 5 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index 5ce800a..e124856 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -96,4 +96,11 @@ extern void vas_init_rx_win_attr(struct vas_rx_win_attr
*rxattr,
extern struct vas_window *vas_rx_win_open(int vasid, enum vas_cop_type cop,
struct vas_rx_win_attr *attr);
+/*
+ * Close the send or receive window identified by @win. For receive windows
+ * return -EAGAIN if there are active send windows attached to this receive
+ * window.
+ */
+int vas_win_close(struct vas_window *win);
+
#endif /* _MISC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 783c8c6..39aa0e4 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -130,7 +130,7 @@ static void unmap_region(void *addr, u64 start, int len)
/*
* Unmap the paste address region for a window.
*/
-void unmap_paste_region(struct vas_window *window)
+static void unmap_paste_region(struct vas_window *window)
{
int len;
u64 busaddr_start;
@@ -522,7 +522,7 @@ static int vas_assign_window_id(struct ida *ida)
return winid;
}
-void vas_window_free(struct vas_window *window)
+static void vas_window_free(struct vas_window *window)
{
int winid = window->winid;
struct vas_instance *vinst = window->vinst;
@@ -560,6 +560,14 @@ static struct vas_window *vas_window_alloc(struct
vas_instance *vinst)
return ERR_PTR(-ENOMEM);
}
+static void put_rx_win(struct vas_window *rxwin)
+{
+ /* Better not be a send window! */
+ WARN_ON_ONCE(rxwin->tx_win);
+
+ atomic_dec(&rxwin->num_txwins);
+}
+
/*
* Get the VAS receive window associated with NX engine identified
* by @cop and if applicable, @pswid.
@@ -627,7 +635,7 @@ static void set_vinst_win(struct vas_instance *vinst,
* Clear this window from the table(s) of windows for this VAS instance.
* See also function header of set_vinst_win().
*/
-void clear_vinst_win(struct vas_window *window)
+static void clear_vinst_win(struct vas_window *window)
{
int id = window->winid;
struct vas_instance *vinst = window->vinst;
@@ -839,8 +847,91 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
}
EXPORT_SYMBOL_GPL(vas_rx_win_open);
-/* stub for now */
+static void poll_window_busy_state(struct vas_window *window)
+{
+ int busy;
+ u64 val;
+
+retry:
+ /*
+* Poll Window Busy flag
+*/
+ val = read_hvwc_reg(window, VREG(WIN_STATUS));
+ busy = GET_FIELD(VAS_WIN_BUSY, val);
+ if (busy) {
+ val = 0;
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(HZ);
+ goto retry;
+ }
+}
+
+static void poll_window_castout(struct vas_window *window)
+{
+ int cached;
+ u64 val;
+
+ /* Cast window context out of the cache */
+retry:
+ val = read_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL));
+ cached = GET_FIELD(VAS_WIN_CACHE_STATUS, val);
+ if (cached) {
+ val = 0ULL;
+ val = SET_FIELD(VAS_CASTOUT_REQ, val, 1);
+ val = SET_FIELD(VAS_PUSH_TO_MEM, val, 0);
+ write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), val);
+
+ set_current_state(TASK_UNINTERRUPTIBLE);
+ schedule_timeout(HZ);
+ goto retry;
+ }
+}
+
+/*
+ * Close a window.
+ *
+ * See Section 1.12.1 of VAS workbook v1.05 for details on closing window:
+ * - Disable new paste operations (unmap paste address)
+ * - Poll for the "Window Busy" bit to be cleared
+ * - Clear the Open/Enable bit for the Window.
+ * - Poll for return of window Credits (implies FIFO empty for Rx win?)
+ * - Unpin and cast window context
[PATCH v8 07/10] powerpc/vas: Define vas_rx_win_open() interface
Define the vas_rx_win_open() interface. This interface is intended to be
used by the Nest Accelerator (NX) driver(s) to setup receive windows for
one or more NX engines (which implement compression/encryption algorithms
in the hardware).
Follow-on patches will provide an interface to close the window and to open
a send window that kernel subsystems can use to access the NX engines.
The interface to open a receive window is expected to be invoked for each
instance of VAS in the system.
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]:
- [Michael Ellerman] Drop vas_initialized() check; use pr_fmt;
user kernel integer types (u32,64 etc);
- Dropped (deferred) code that handles user rx windows.
Changelog[v7]:
- vas_rx_win_open() is simplified because API for FTW windows
are simplified. We expect the driver to open an rxwin and txwin
one after the other and we don't get back an rx_win_handle from
user space so don't have to validate the pid permissions.
Changelog[v6]:
- Add support for FTW windows
Changelog[v4]:
- Export the symbols
Changelog[v3]:
- Fault receive windows must enable interrupts and disable
notifications. NX Windows are opposite.
- Use macros rather than enum for threshold-control mode
- Ignore irq_ports for in-kernel windows. They are needed for
user space windows and will be added later
---
arch/powerpc/include/asm/vas.h | 46 +
arch/powerpc/platforms/powernv/vas-window.c | 283 +++-
arch/powerpc/platforms/powernv/vas.h| 14 ++
3 files changed, 342 insertions(+), 1 deletion(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index 33e93ca..5ce800a 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -50,4 +50,50 @@ enum vas_cop_type {
VAS_COP_TYPE_MAX,
};
+/*
+ * Receive window attributes specified by the (in-kernel) owner of window.
+ */
+struct vas_rx_win_attr {
+ void *rx_fifo;
+ int rx_fifo_size;
+ int wcreds_max;
+
+ bool pin_win;
+ bool rej_no_credit;
+ bool tx_wcred_mode;
+ bool rx_wcred_mode;
+ bool tx_win_ord_mode;
+ bool rx_win_ord_mode;
+ bool data_stamp;
+ bool nx_win;
+ bool fault_win;
+ bool user_win;
+ bool notify_disable;
+ bool intr_disable;
+ bool notify_early;
+
+ int lnotify_lpid;
+ int lnotify_pid;
+ int lnotify_tid;
+ u32 pswid;
+
+ int tc_mode;
+};
+
+/*
+ * Helper to initialize receive window attributes to defaults for an
+ * NX window.
+ */
+extern void vas_init_rx_win_attr(struct vas_rx_win_attr *rxattr,
+ enum vas_cop_type cop);
+
+/*
+ * Open a VAS receive window for the instance of VAS identified by @vasid
+ * Use @attr to initialize the attributes of the window.
+ *
+ * Return a handle to the window or ERR_PTR() on error.
+ */
+extern struct vas_window *vas_rx_win_open(int vasid, enum vas_cop_type cop,
+ struct vas_rx_win_attr *attr);
+
#endif /* _MISC_VAS_H */
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index bfc9dba..783c8c6 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -14,6 +14,8 @@
#include
#include
#include
+#include
+#include
#include "vas.h"
@@ -531,7 +533,7 @@ void vas_window_free(struct vas_window *window)
vas_release_window_id(&vinst->ida, winid);
}
-struct vas_window *vas_window_alloc(struct vas_instance *vinst)
+static struct vas_window *vas_window_alloc(struct vas_instance *vinst)
{
int winid;
struct vas_window *window;
@@ -558,6 +560,285 @@ struct vas_window *vas_window_alloc(struct vas_instance
*vinst)
return ERR_PTR(-ENOMEM);
}
+/*
+ * Get the VAS receive window associated with NX engine identified
+ * by @cop and if applicable, @pswid.
+ *
+ * See also function header of set_vinst_win().
+ */
+struct vas_window *get_vinst_rxwin(struct vas_instance *vinst,
+ enum vas_cop_type cop, u32 pswid)
+{
+ struct vas_window *rxwin;
+
+ mutex_lock(&vinst->mutex);
+
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI)
+ rxwin = vinst->rxwin[cop] ?: ERR_PTR(-EINVAL);
+ else
+ rxwin = ERR_PTR(-EINVAL);
+
+ if (!IS_ERR(rxwin))
+ atomic_inc(&rxwin->num_txwins);
+
+ mutex_unlock(&vinst->mutex);
+
+ return rxwin;
+}
+
+/*
+ * We have two tables of windows in a VAS instance. The first one,
+ * ->windows[], contains all the windows in the instance and allows
+ * looking up a window by its id. It is used to look up send windows
+ * during fault handling and receive windows when pairing user sp
[PATCH v8 06/10] powerpc/vas: Define helpers to alloc/free windows
Define helpers to allocate/free VAS window objects. These will
be used in follow-on patches when opening/closing windows.
Changelog[v8]:
- [Michael Ellerman] Make some functions static; retry if
ida_get_new() fails with EAGAIN; fix a couple of leak in ids
Signed-off-by: Sukadev Bhattiprolu
---
arch/powerpc/platforms/powernv/vas-window.c | 73 +
1 file changed, 73 insertions(+)
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 68dfe53..bfc9dba 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -485,6 +485,79 @@ int init_winctx_regs(struct vas_window *window, struct
vas_winctx *winctx)
return 0;
}
+static DEFINE_SPINLOCK(vas_ida_lock);
+
+static void vas_release_window_id(struct ida *ida, int winid)
+{
+ spin_lock(&vas_ida_lock);
+ ida_remove(ida, winid);
+ spin_unlock(&vas_ida_lock);
+}
+
+static int vas_assign_window_id(struct ida *ida)
+{
+ int rc, winid;
+
+ do {
+ rc = ida_pre_get(ida, GFP_KERNEL);
+ if (!rc)
+ return -EAGAIN;
+
+ spin_lock(&vas_ida_lock);
+ rc = ida_get_new(ida, &winid);
+ spin_unlock(&vas_ida_lock);
+ } while (rc == -EAGAIN);
+
+ if (rc)
+ return rc;
+
+ if (winid > VAS_WINDOWS_PER_CHIP) {
+ pr_err("Too many (%d) open windows\n", winid);
+ vas_release_window_id(ida, winid);
+ return -EAGAIN;
+ }
+
+ return winid;
+}
+
+void vas_window_free(struct vas_window *window)
+{
+ int winid = window->winid;
+ struct vas_instance *vinst = window->vinst;
+
+ unmap_winctx_mmio_bars(window);
+ kfree(window);
+
+ vas_release_window_id(&vinst->ida, winid);
+}
+
+struct vas_window *vas_window_alloc(struct vas_instance *vinst)
+{
+ int winid;
+ struct vas_window *window;
+
+ winid = vas_assign_window_id(&vinst->ida);
+ if (winid < 0)
+ return ERR_PTR(winid);
+
+ window = kzalloc(sizeof(*window), GFP_KERNEL);
+ if (!window)
+ goto out_free;
+
+ window->vinst = vinst;
+ window->winid = winid;
+
+ if (map_winctx_mmio_bars(window))
+ goto out_free;
+
+ return window;
+
+out_free:
+ kfree(window);
+ vas_release_window_id(&vinst->ida, winid);
+ return ERR_PTR(-ENOMEM);
+}
+
/* stub for now */
int vas_win_close(struct vas_window *window)
{
--
2.7.4
[PATCH v8 00/10] Enable VAS
- Enable send-window-credit checking - Reorg code in vas_win_close() - Minor reorgs and tweaks to register field settings to make it easier to add support for user space windows. - Skip writing to read-only registers - Start window indexing from 0 rather than 1 Changelog[v2] - Use vas-id, HVWC, UWC and paste address, entries from device tree rather than defining/computing them in kernel and reorg code. Sukadev Bhattiprolu (10): powerpc/vas: Define macros, register fields and structures Move GET_FIELD/SET_FIELD to vas.h powerpc/vas: Define vas_init() and vas_exit() powerpc/vas: Define helpers to access MMIO regions powerpc/vas: Define helpers to init window context powerpc/vas: Define helpers to alloc/free windows powerpc/vas: Define vas_rx_win_open() interface powerpc/vas: Define vas_win_close() interface powerpc/vas: Define vas_tx_win_open() powerpc/vas: Define copy/paste interfaces .../devicetree/bindings/powerpc/ibm,vas.txt| 23 + MAINTAINERS|9 + arch/powerpc/include/asm/ppc-opcode.h |2 + arch/powerpc/include/asm/vas.h | 160 +++ arch/powerpc/platforms/powernv/Kconfig | 14 + arch/powerpc/platforms/powernv/Makefile|1 + arch/powerpc/platforms/powernv/copy-paste.h| 46 + arch/powerpc/platforms/powernv/vas-window.c| 1134 arch/powerpc/platforms/powernv/vas.c | 151 +++ arch/powerpc/platforms/powernv/vas.h | 467 drivers/crypto/nx/nx-842-powernv.c |7 +- drivers/crypto/nx/nx-842.h |5 - 12 files changed, 2011 insertions(+), 8 deletions(-) create mode 100644 Documentation/devicetree/bindings/powerpc/ibm,vas.txt create mode 100644 arch/powerpc/include/asm/vas.h create mode 100644 arch/powerpc/platforms/powernv/copy-paste.h create mode 100644 arch/powerpc/platforms/powernv/vas-window.c create mode 100644 arch/powerpc/platforms/powernv/vas.c create mode 100644 arch/powerpc/platforms/powernv/vas.h -- 2.7.4
[PATCH v8 09/10] powerpc/vas: Define vas_tx_win_open()
Define an interface to open a VAS send window. This interface is
intended to be used the Nest Accelerator (NX) driver(s) to open
a send window and use it to submit compression/encryption requests
to a VAS receive window.
The receive window, identified by the [vasid, cop] parameters, must
already be open in VAS (i.e connected to an NX engine).
Signed-off-by: Sukadev Bhattiprolu
---
Changelog[v8]:
- [Michael Ellerman] Drop vas_initialized() check; defer code
that sets pswid;
Changelog[v7]:
- Initialize txwin->user_win field for FTW windows.
Changelog[v6]:
- Add support for FTW windows
Changelog[v4]:
- [Ben Herrenschmidt] MMIO regions must be mapped non-cached and
paste regions must be mapped cached. Define/use map_paste_region().
Changelog [v3]:
- Distinguish between hardware PID (SPRN_PID) and Linux pid.
- Use macros rather than enum for threshold-control mode
- Set the pid of send window from attr (needed for user space
send windows).
- Ignore irq port setting for now. They are needed for user space
windows and will be added later
---
arch/powerpc/include/asm/vas.h | 42
arch/powerpc/platforms/powernv/vas-window.c | 156 +++-
2 files changed, 195 insertions(+), 3 deletions(-)
diff --git a/arch/powerpc/include/asm/vas.h b/arch/powerpc/include/asm/vas.h
index e124856..efbdde5 100644
--- a/arch/powerpc/include/asm/vas.h
+++ b/arch/powerpc/include/asm/vas.h
@@ -81,6 +81,29 @@ struct vas_rx_win_attr {
};
/*
+ * Window attributes specified by the in-kernel owner of a send window.
+ */
+struct vas_tx_win_attr {
+ enum vas_cop_type cop;
+ int wcreds_max;
+ int lpid;
+ int pidr; /* hardware PID (from SPRN_PID) */
+ int pid;/* linux process id */
+ int pswid;
+ int rsvd_txbuf_count;
+ int tc_mode;
+
+ bool user_win;
+ bool pin_win;
+ bool rej_no_credit;
+ bool rsvd_txbuf_enable;
+ bool tx_wcred_mode;
+ bool rx_wcred_mode;
+ bool tx_win_ord_mode;
+ bool rx_win_ord_mode;
+};
+
+/*
* Helper to initialize receive window attributes to defaults for an
* NX window.
*/
@@ -97,6 +120,25 @@ extern struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
struct vas_rx_win_attr *attr);
/*
+ * Helper to initialize send window attributes to defaults for an NX window.
+ */
+extern void vas_init_tx_win_attr(struct vas_tx_win_attr *txattr,
+ enum vas_cop_type cop);
+
+/*
+ * Open a VAS send window for the instance of VAS identified by @vasid
+ * and the co-processor type @cop. Use @attr to initialize attributes
+ * of the window.
+ *
+ * Note: The instance of VAS must already have an open receive window for
+ * the coprocessor type @cop.
+ *
+ * Return a handle to the send window or ERR_PTR() on error.
+ */
+struct vas_window *vas_tx_win_open(int vasid, enum vas_cop_type cop,
+ struct vas_tx_win_attr *attr);
+
+/*
* Close the send or receive window identified by @win. For receive windows
* return -EAGAIN if there are active send windows attached to this receive
* window.
diff --git a/arch/powerpc/platforms/powernv/vas-window.c
b/arch/powerpc/platforms/powernv/vas-window.c
index 39aa0e4..cd12e44 100644
--- a/arch/powerpc/platforms/powernv/vas-window.c
+++ b/arch/powerpc/platforms/powernv/vas-window.c
@@ -64,7 +64,7 @@ static inline void get_uwc_mmio_bar(struct vas_window *window,
* space. Unlike MMIO regions (map_mmio_region() below), paste region must
* be mapped cache-able and is only applicable to send windows.
*/
-void *map_paste_region(struct vas_window *txwin)
+static void *map_paste_region(struct vas_window *txwin)
{
int len;
void *map;
@@ -100,7 +100,6 @@ void *map_paste_region(struct vas_window *txwin)
return ERR_PTR(-ENOMEM);
}
-
static void *map_mmio_region(char *name, u64 start, int len)
{
void *map;
@@ -574,7 +573,7 @@ static void put_rx_win(struct vas_window *rxwin)
*
* See also function header of set_vinst_win().
*/
-struct vas_window *get_vinst_rxwin(struct vas_instance *vinst,
+static struct vas_window *get_vinst_rxwin(struct vas_instance *vinst,
enum vas_cop_type cop, u32 pswid)
{
struct vas_window *rxwin;
@@ -847,6 +846,157 @@ struct vas_window *vas_rx_win_open(int vasid, enum
vas_cop_type cop,
}
EXPORT_SYMBOL_GPL(vas_rx_win_open);
+void vas_init_tx_win_attr(struct vas_tx_win_attr *txattr, enum vas_cop_type
cop)
+{
+ memset(txattr, 0, sizeof(*txattr));
+
+ if (cop == VAS_COP_TYPE_842 || cop == VAS_COP_TYPE_842_HIPRI) {
+ txattr->rej_no_credit = false;
+ txattr->rx_wcred_mode = true;
+ txattr->tx_wcred_mode = true;
+ txattr->rx_win_ord_mode = true;
+
Re: [PATCH v7 12/12] powerpc/vas: Define copy/paste interfaces
Michael Ellerman [m...@ellerman.id.au] wrote:
> Hi Suka,
>
> A few more things ...
>
> Sukadev Bhattiprolu writes:
>
> > diff --git a/arch/powerpc/platforms/powernv/copy-paste.h
> > b/arch/powerpc/platforms/powernv/copy-paste.h
> > new file mode 100644
> > index 000..7783bb8
> > --- /dev/null
> > +++ b/arch/powerpc/platforms/powernv/copy-paste.h
> > @@ -0,0 +1,74 @@
> > +/*
> > + * Copyright 2016 IBM Corp.
> > + *
> > + * This program is free software; you can redistribute it and/or
> > + * modify it under the terms of the GNU General Public License
> > + * as published by the Free Software Foundation; either version
> > + * 2 of the License, or (at your option) any later version.
> > + */
> > +
> > +/*
> > + * Macros taken from
> > tools/testing/selftests/powerpc/context_switch/cp_abort.c
> > + */
>
> These are both out of date, they're changed in v3.0B.
>
> > +#define PASTE(RA, RB, L, RC) \
> > + .long (0x7c00070c | (RA) << (31-15) | (RB) << (31-20) \
> > + | (L) << (31-10) | (RC) << (31-31))
>
> You should define PPC_PASTE() in ppc-opcode.h
>
> We already have PPC_INST_PASTE, so use that.
>
> L and RC are gone.
Ok. I thought they would come back later, but of course we can update
these kernel-only calls then.
>
> > +
> > +#define COPY(RA, RB, L) \
> > + .long (0x7c00060c | (RA) << (31-15) | (RB) << (31-20) \
> > + | (L) << (31-10))
>
> Use PPC_COPY().
>
Ok
> > +
> > +#define CR0_FXM"0x80"
>
> I don't think a #define for this helps readability.
>
> > +#define CR0_SHIFT 28
> > +#define CR0_MASK 0xF
>
> Not used.
Will need them now to return value in cr0?
>
> > +/*
> > + * Copy/paste instructions:
> > + *
> > + * copy RA,RB,L
> > + * Copy contents of address (RA) + effective_address(RB)
> > + * to internal copy-buffer.
> > + *
> > + * L == 1 indicates this is the first copy.
> > + *
> > + * L == 0 indicates its a continuation of a prior first copy.
> > + *
> > + * paste RA,RB,L
> > + * Paste contents of internal copy-buffer to the address
> > + * (RA) + effective_address(RB)
> > + *
> > + * L == 0 indicates its a continuation of a prior paste. i.e.
> > + * don't wait for the completion or update status.
> > + *
> > + * L == 1 indicates this is the last paste in the group (i.e.
> > + * wait for the group to complete and update status in CR0).
> > + *
> > + * For Power9, the L bit must be 'true' in both copy and paste.
> > + */
> > +
> > +static inline int vas_copy(void *crb, int offset, int first)
> > +{
> > + WARN_ON_ONCE(!first);
>
> Please change the API to not require unused parameters.
>
> Same for offset.
Ok, Haren's NX patches will need to drop those parameters as well.
>
> > +
> > + __asm__ __volatile(stringify_in_c(COPY(%0, %1, %2))";"
>
> I've never seen __volatile before.
>
> Just use: asm volatile
ok
>
>
> > + :
> > + : "b" (offset), "b" (crb), "i" (1)
> > + : "memory");
> > +
> > + return 0;
> > +}
> > +
> > +static inline int vas_paste(void *paste_address, int offset, int last)
> > +{
> > + unsigned long long cr;
>
> cr is 32-bits actually.
ok
>
> > + WARN_ON_ONCE(!last);
> > +
> > + cr = 0;
> > + __asm__ __volatile(stringify_in_c(PASTE(%1, %2, 1, 1))";"
> > + "mfocrf %0," CR0_FXM ";"
> > + : "=r" (cr)
> > + : "b" (paste_address), "b" (offset)
> > + : "memory");
>
> You need cr0 in the clobbers.
ok
>
> > +
> > + return cr;
>
> I think it would be more natural if you just returned CR0, so if you did
> shift and mask with the CR0 constants you have above.
>
ok
>
> > diff --git a/arch/powerpc/platforms/powernv/vas-window.c
> > b/arch/powerpc/platforms/powernv/vas-window.c
> > index 70762c3..73081b4 100644
> > --- a/arch/powerpc/platforms/powernv/vas-window.c
> > +++ b/arch/powerpc/platforms/powernv/vas-window.c
> > @@ -1040,6 +1041,57 @@ struct vas_window *vas_tx_win_open(int vasid, enum
> > vas_cop_type cop,
> > }
> > EXPORT_SYMBOL_GPL(vas_tx_win_open);
> >
> > +int vas_copy_crb(void *crb, int offset, bool first)
> > +{
> > + if (!vas_initialized())
> > + return -1;
> > +
> > + return vas_copy(crb, offset, first);
> > +}
> > +EXPORT_SYMBOL_GPL(vas_copy_crb);
> > +
> > +#define RMA_LSMP_REPORT_ENABLE PPC_BIT(53)
> > +int vas_paste_crb(struct vas_window *txwin, int offset, bool last, bool re)
> > +{
> > + int rc;
> > + uint64_t val;
> > + void *addr;
> > +
> > + if (!vas_initialized())
> > + return -1;
>
> This is in the fast path, or at least the runtime path. So I don't think
> these checks are wanted, how would we have got this far if vas wasn't
> initialised?
Yes, I have dropped vas_initialized() now.
>
>
>
> cheers
Re: [PATCH v7 10/12] powerpc/vas: Define vas_win_close() interface
Michael Ellerman [m...@ellerman.id.au] wrote:
> Hi Suka,
>
> More comments :)
Thanks!
>
> Sukadev Bhattiprolu writes:
>
> > diff --git a/arch/powerpc/platforms/powernv/vas-window.c
> > b/arch/powerpc/platforms/powernv/vas-window.c
> > index 2dd4b63..24288dd 100644
> > --- a/arch/powerpc/platforms/powernv/vas-window.c
> > +++ b/arch/powerpc/platforms/powernv/vas-window.c
> > @@ -879,11 +887,92 @@ struct vas_window *vas_rx_win_open(int vasid, enum
> > vas_cop_type cop,
> > }
> > EXPORT_SYMBOL_GPL(vas_rx_win_open);
> >
> > -/* stub for now */
> > +static void poll_window_busy_state(struct vas_window *window)
> > +{
> > + int busy;
> > + uint64_t val;
> > +
> > +retry:
> > + /*
> > +* Poll Window Busy flag
> > +*/
> > + val = read_hvwc_reg(window, VREG(WIN_STATUS));
> > + busy = GET_FIELD(VAS_WIN_BUSY, val);
> > + if (busy) {
> > + val = 0;
> > + schedule_timeout(2000);
>
> What's 2000?
>
> That's in jiffies, so it's not a fixed amount of time.
>
> But on a typical config that will be 20 _seconds_ ?!
Ok. Should I change to that just HZ and
>
> But you haven't set the task state, so AFAIK it will just return
> instantly.
call set_current_state(TASK_UNINTERRUPTIBLE) before the schedule_timeout()?
>
> And if there's a software/hardware bug and it never stops being busy,
> then we have a softlockup. The other option would be print a big fat
> warning and just not free the window. But maybe that doesn't work for
> other reasons.
>
> > + goto retry;
> > + }
> > +}
> > +
> > +static void poll_window_castout(struct vas_window *window)
> > +{
> > + int cached;
> > + uint64_t val;
> > +
> > + /* Cast window context out of the cache */
> > +retry:
> > + val = read_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL));
> > + cached = GET_FIELD(VAS_WIN_CACHE_STATUS, val);
> > + if (cached) {
> > + val = 0ULL;
> > + val = SET_FIELD(VAS_CASTOUT_REQ, val, 1);
> > + val = SET_FIELD(VAS_PUSH_TO_MEM, val, 0);
> > + write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), val);
>
> Sigh, I still don't like that macro :)
:-) For one thing, I have used it a lot now and secondly isn't it easier
to know that VAS_CASTOUT_REQ bit is set to 1 without worrying about its
bit position? When debugging, yes we have to ensure VAS_CASTOUT_REQ is
properly defined and we have to work out value in "val".
>
> or:
> write_hvwc_reg(window, VREG(WIN_CTX_CACHING_CTL), 1ull << 63);
>
> > +
> > + schedule_timeout(2000);
> > + goto retry;
> > + }
> > +}
> > +
> > +/*
> > + * Close a window.
> > + *
> > + * See Section 1.12.1 of VAS workbook v1.05 for details on closing window:
> > + * - Disable new paste operations (unmap paste address)
> > + * - Poll for the "Window Busy" bit to be cleared
> > + * - Clear the Open/Enable bit for the Window.
> > + * - Poll for return of window Credits (implies FIFO empty for Rx win?)
> > + * - Unpin and cast window context out of cache
> > + *
> > + * Besides the hardware, kernel has some bookkeeping of course.
> > + */
> > int vas_win_close(struct vas_window *window)
> > {
> > - return -1;
> > + uint64_t val;
> > +
> > + if (!window)
> > + return 0;
> > +
> > + if (!window->tx_win && atomic_read(&window->num_txwins) != 0) {
> > + pr_devel("VAS: Attempting to close an active Rx window!\n");
> > + WARN_ON_ONCE(1);
> > + return -EAGAIN;
>
> EAGAIN means "if you do the same thing again it might work".
>
> I don't think that's right here. The window is not in a state where it
> can be freed, the caller needs to do something to fix that.
>
> EBUSY would probably be more appropriate.
Ok. Should not happen now (or even with the fast thread-wake up code)
since only the kernel should be closing the windows - so its really a
bug. Will change to EBUSY though.
>
>
> cheers
