RE: [PATCH v2 0/2] iommu/ipmmu-vmsa: Add support for r8a779a0
Hi Joerg, Will, > From: Yoshihiro Shimoda, Sent: Tuesday, September 7, 2021 5:30 PM > > This patch series adds support for r8a779a0 (R-Car V3U). Would you review this patch series? Best regards, Yoshihiro Shimoda ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 5/8] x86/mmu: Add mm-based PASID refcounting
PASIDs are fundamentally hardware resources in a shared address space.
There is a limited number of them to use ENQCMD on shared workqueue.
They must be shared and managed. They can not, for instance, be
statically allocated to processes.
Free PASID eagerly by sending IPIs in unbind was disabled due to locking
and other issues in commit 9bfecd058339 ("x86/cpufeatures: Force disable
X86_FEATURE_ENQCMD and remove update_pasid()").
Lazy PASID free is implemented in order to re-enable the ENQCMD feature.
PASIDs are currently reference counted and are centered around device
usage. To support lazy PASID free, reference counts are tracked in the
following scenarios:
1. The PASID's reference count is initialized as 1 when the PASID is first
allocated in bind. This is already implemented.
2. A reference is taken when a device is bound to the mm and dropped
when the device is unbound from the mm. This reference tracks device
usage of the PASID. This is already implemented.
3. A reference is taken when a task's IA32_PASID MSR is initialized in
#GP fix up and dropped when the task exits. This reference tracks
the task usage of the PASID. It is implemented here.
Once a PASID is allocated to an mm in bind, it's associated to the mm until
it's freed lazily when its reference count is dropped to zero in unbind or
exit(2).
ENQCMD requires a valid IA32_PASID MSR with the PASID value and a valid
PASID table entry for the PASID. Lazy PASID free may cause the process
still has the valid PASID but the PASID table entry is removed in unbind.
In this case, workqueue submitted by ENQCMD cannot find the PASID table
entry and will generate a DMAR fault.
Here is a more detailed explanation of the life cycle of a PASID:
All processes start out without a PASID allocated (because fork(2)
clears the PASID in the child).
A PASID is allocated on the first open of an accelerator device by
a call to:
iommu_sva_bind_device()
-> intel_svm_bind()
-> intel_svm_alloc_pasid()
-> iommu_sva_alloc_pasid()
-> ioasid_alloc()
At this point mm->pasid for the process is initialized, the reference
count on that PASID is 1, but as yet no tasks within the process have
set up their MSR_IA32_PASID to be able to execute the ENQCMD instruction.
When a task in the process does execute ENQCMD there is a #GP fault.
The Linux handler notes that the process has a PASID allocated, and
attempts to fix the #GP fault by initializing MSR_IA32_PASID for this
task. It also increments the reference count for the PASID.
Additional threads in the task may also execute ENQCMD, and each
will add to the reference count of the PASID.
Tasks within the process may open additional accelerator devices.
In this case the call to iommu_sva_bind_device() merely increments
the reference count for the PASID. Since all devices use the same
PASID (all are accessing the same address space).
So the reference count on a PASID is the sum of the number of open
accelerator devices plus the number of threads that have tried to
execute ENQCMD.
The reverse happens as a process gives up resources. Each call to
iommu_sva_unbind_device() will reduce the reference count on the
PASID. Each task in the process that had set up MSR_IA32_PASID will
reduce the reference count as it exits.
When the reference count is dropped to 0 in either task exit or
unbind, the PASID will be freed.
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
arch/x86/include/asm/iommu.h | 6 +
arch/x86/include/asm/mmu_context.h | 2 ++
drivers/iommu/intel/svm.c | 39 ++
3 files changed, 47 insertions(+)
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index 9c4bf9b0702f..d00f0a3f32fb 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -28,4 +28,10 @@ arch_rmrr_sanity_check(struct acpi_dmar_reserved_memory
*rmrr)
bool __fixup_pasid_exception(void);
+#ifdef CONFIG_INTEL_IOMMU_SVM
+void pasid_put(struct task_struct *tsk, struct mm_struct *mm);
+#else
+static inline void pasid_put(struct task_struct *tsk, struct mm_struct *mm) { }
+#endif
+
#endif /* _ASM_X86_IOMMU_H */
diff --git a/arch/x86/include/asm/mmu_context.h
b/arch/x86/include/asm/mmu_context.h
index 27516046117a..3a2de87e98a9 100644
--- a/arch/x86/include/asm/mmu_context.h
+++ b/arch/x86/include/asm/mmu_context.h
@@ -12,6 +12,7 @@
#include
#include
#include
+#include
extern atomic64_t last_mm_ctx_id;
@@ -146,6 +147,7 @@ do {\
#else
#define deactivate_mm(tsk, mm) \
do { \
+ pasid_put(tsk, mm); \
load_gs_index(0); \
loadsegment(fs, 0); \
} while (0)
diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c
index ab65020019b6..8b6b8007ba2c 100644
--- a/drivers/iommu/intel/svm.c
+++ b/drivers/iommu/intel/svm.c
[PATCH 3/8] sched: Define and initialize a flag to identify valid PASID in the task
From: Peter Zijlstra
Add a new field to the task structure to track whether this task
has initialized the IA32_PASID MSR (and thus holds a reference
count on the PASID for this process).
Initialize the field to zero when creating a new task with fork/clone.
Signed-off-by: Peter Zijlstra
Co-developed-by: Fenghua Yu
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
include/linux/sched.h | 4
kernel/fork.c | 4
2 files changed, 8 insertions(+)
diff --git a/include/linux/sched.h b/include/linux/sched.h
index 39039ce8ac4c..21a8cff9155c 100644
--- a/include/linux/sched.h
+++ b/include/linux/sched.h
@@ -936,6 +936,10 @@ struct task_struct {
unsignedin_eventfd_signal:1;
#endif
+#ifdef CONFIG_IOMMU_SUPPORT
+ unsignedhas_valid_pasid:1;
+#endif
+
unsigned long atomic_flags; /* Flags requiring atomic
access. */
struct restart_blockrestart_block;
diff --git a/kernel/fork.c b/kernel/fork.c
index 38681ad44c76..e379f88260eb 100644
--- a/kernel/fork.c
+++ b/kernel/fork.c
@@ -967,6 +967,10 @@ static struct task_struct *dup_task_struct(struct
task_struct *orig, int node)
tsk->use_memdelay = 0;
#endif
+#ifdef CONFIG_IOMMU_SUPPORT
+ tsk->has_valid_pasid = 0;
+#endif
+
#ifdef CONFIG_MEMCG
tsk->active_memcg = NULL;
#endif
--
2.33.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 1/8] iommu/vt-d: Clean up unused PASID updating functions
update_pasid() and its call chain are currently unused in the tree because
Thomas disabled the ENQCMD feature. The feature will be re-enabled shortly
using a different approach and update_pasid() and its call chain will not
be used in the new approach.
Remove the useless functions.
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
arch/x86/include/asm/fpu/api.h | 2 --
drivers/iommu/intel/svm.c | 24 +---
2 files changed, 1 insertion(+), 25 deletions(-)
diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h
index 23bef08a8388..ca4d0dee1ecd 100644
--- a/arch/x86/include/asm/fpu/api.h
+++ b/arch/x86/include/asm/fpu/api.h
@@ -106,6 +106,4 @@ extern int cpu_has_xfeatures(u64 xfeatures_mask, const char
**feature_name);
*/
#define PASID_DISABLED 0
-static inline void update_pasid(void) { }
-
#endif /* _ASM_X86_FPU_API_H */
diff --git a/drivers/iommu/intel/svm.c b/drivers/iommu/intel/svm.c
index 0c228787704f..5b5d69b04fcc 100644
--- a/drivers/iommu/intel/svm.c
+++ b/drivers/iommu/intel/svm.c
@@ -505,21 +505,6 @@ int intel_svm_unbind_gpasid(struct device *dev, u32 pasid)
return ret;
}
-static void _load_pasid(void *unused)
-{
- update_pasid();
-}
-
-static void load_pasid(struct mm_struct *mm, u32 pasid)
-{
- mutex_lock(>context.lock);
-
- /* Update PASID MSR on all CPUs running the mm's tasks. */
- on_each_cpu_mask(mm_cpumask(mm), _load_pasid, NULL, true);
-
- mutex_unlock(>context.lock);
-}
-
static int intel_svm_alloc_pasid(struct device *dev, struct mm_struct *mm,
unsigned int flags)
{
@@ -614,10 +599,6 @@ static struct iommu_sva *intel_svm_bind_mm(struct
intel_iommu *iommu,
if (ret)
goto free_sdev;
- /* The newly allocated pasid is loaded to the mm. */
- if (!(flags & SVM_FLAG_SUPERVISOR_MODE) && list_empty(>devs))
- load_pasid(mm, svm->pasid);
-
list_add_rcu(>list, >devs);
success:
return >sva;
@@ -670,11 +651,8 @@ static int intel_svm_unbind_mm(struct device *dev, u32
pasid)
kfree_rcu(sdev, rcu);
if (list_empty(>devs)) {
- if (svm->notifier.ops) {
+ if (svm->notifier.ops)
mmu_notifier_unregister(>notifier,
mm);
- /* Clear mm's pasid. */
- load_pasid(mm, PASID_DISABLED);
- }
pasid_private_remove(svm->pasid);
/* We mandate that no page faults may be
outstanding
* for the PASID when intel_svm_unbind_mm() is
called.
--
2.33.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 8/8] docs: x86: Change documentation for SVA (Shared Virtual Addressing)
Since allocating, freeing and fixing up PASID are changed, the documentation is updated to reflect the changes. Originally-by: Ashok Raj Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck --- Documentation/x86/sva.rst | 81 +++ 1 file changed, 74 insertions(+), 7 deletions(-) diff --git a/Documentation/x86/sva.rst b/Documentation/x86/sva.rst index 076efd51ef1f..868ed4b25002 100644 --- a/Documentation/x86/sva.rst +++ b/Documentation/x86/sva.rst @@ -106,16 +106,83 @@ process share the same page tables, thus the same MSR value. PASID is cleared when a process is created. The PASID allocation and MSR programming may occur long after a process and its threads have been created. -One thread must call iommu_sva_bind_device() to allocate the PASID for the -process. If a thread uses ENQCMD without the MSR first being populated, a #GP -will be raised. The kernel will update the PASID MSR with the PASID for all -threads in the process. A single process PASID can be used simultaneously +One thread must call iommu_sva_bind(_device) to allocate the PASID for the process. +If a thread uses ENQCMD without the MSR first being populated, it will cause #GP. +The kernel will fix up the #GP by writing the process-wide PASID into the +thread that took the #GP. A single process PASID can be used simultaneously with multiple devices since they all share the same address space. -One thread can call iommu_sva_unbind_device() to free the allocated PASID. -The kernel will clear the PASID MSR for all threads belonging to the process. +The PASID MSR value is cleared at thread creation and is never inherited from a +parent. This ensures consistent child behavior no matter whether the thread is +created first or the PASID is allocated (and the MSR written). -New threads inherit the MSR value from the parent. + +PASID Lifecycle Management +== + +Only processes that access SVA-capable devices need to have a PASID +allocated. This allocation happens when a process opens/binds an SVA-capable +device but finds no PASID for this process. Subsequent binds of the same, or +other devices will share the same PASID. + +Although the PASID is allocated to the process by opening a device, +it is not active in any of the threads of that process. It's loaded to the +IA32_PASID MSR lazily when a thread tries to submit a work descriptor +to a device using the ENQCMD. + +That first access will trigger a #GP fault because the IA32_PASID MSR +has not been initialized with the PASID value assigned to the process +when the device was opened. The Linux #GP handler notes that a PASID has +been allocated for the process, and so initializes the IA32_PASID MSR, takes +a reference to the PASID and returns so that the ENQCMD instruction is +re-executed. + +On fork(2) or exec(2) the PASID is removed from the process as it no +longer has the same address space that it had when the device was opened. + +On clone(2) the new task shares the same address space, so will be +able to use the PASID allocated to the process. The IA32_PASID is not +preemptively initialized as the PASID value might not be allocated yet or +the kernel does not know whether this thread is going to access the device +and the cleared IA32_PASID MSR reduces context switch overhead by xstate +init optimization. Since #GP faults have to be handled on any threads that +were created before the PASID was assigned to the mm of the process, newly +created threads might as well be treated in a consistent way. + +Due to complexity of freeing the PASID and clearing all IA32_PASID MSRs in +all threads in unbind, free the PASID lazily when there is no PASID user. +Track the PASID's reference count in the following way: + +- Track device usage of the PASID: The PASID's reference count is initialized + as 1 when the PASID is allocated in the first bind. Bind takes a reference + and unbind drops the reference. +- Track task usage of the PASID: Fixing up the IA32_PASID MSR in #GP takes + reference and exit(2) drops the reference. Once the MSR is fixed up, the + PASID value stays in the MSR stays for the rest life of the task. + +The PASID is freed lazily in exit(2) or unbind when there is no reference +to the PASID. After freed, the PASID can be allocated to any process. + +ENQCMD needs at least two requirements: a valid IA32_PASID MSR with the +PASID value of the process and a valid PASID table entry for the PASID. +To execute ENQCMD, the user must ensure the device is bound to the +process so that the kernel can guarantee to meet the above two requirements. + +Lazy PASID free may cause the task still has the PASID value in IA32_PASID +while there is no PASID table entry for the PASID. The workqueue submitted +by ENQCMD in this scenario cannot find the PASID table entry and generates +a DMAR fault. Currently DMAR fault handler just prints a fault reason. +Future DMAR fault handler might notify the user the workqueue failure. +Here are
[PATCH 7/8] tools/objtool: Check for use of the ENQCMD instruction in the kernel
The ENQCMD implicitly accesses the PASID_MSR to fill in the pasid field
of the descriptor being submitted to an accelerator. But there is no
precise (and stable across kernel changes) point at which the PASID_MSR
is updated from the value for one task to the next.
Kernel code that uses accelerators must always use the ENQCMDS instruction
which does not access the PASID_MSR.
Check for use of the ENQCMD instruction in the kernel and warn on its
usage.
Checking the invalid instruction is a relatively new use of objtool and
I'm open to feedback about the approach.
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
tools/objtool/arch/x86/decode.c | 10 +-
tools/objtool/check.c| 20
tools/objtool/include/objtool/arch.h | 1 +
3 files changed, 30 insertions(+), 1 deletion(-)
diff --git a/tools/objtool/arch/x86/decode.c b/tools/objtool/arch/x86/decode.c
index bc821056aba9..e63b44ba3bd4 100644
--- a/tools/objtool/arch/x86/decode.c
+++ b/tools/objtool/arch/x86/decode.c
@@ -110,7 +110,7 @@ int arch_decode_instruction(const struct elf *elf, const
struct section *sec,
{
struct insn insn;
int x86_64, ret;
- unsigned char op1, op2,
+ unsigned char op1, op2, op3,
rex = 0, rex_b = 0, rex_r = 0, rex_w = 0, rex_x = 0,
modrm = 0, modrm_mod = 0, modrm_rm = 0, modrm_reg = 0,
sib = 0, /* sib_scale = 0, */ sib_index = 0, sib_base = 0;
@@ -137,6 +137,7 @@ int arch_decode_instruction(const struct elf *elf, const
struct section *sec,
op1 = insn.opcode.bytes[0];
op2 = insn.opcode.bytes[1];
+ op3 = insn.opcode.bytes[2];
if (insn.rex_prefix.nbytes) {
rex = insn.rex_prefix.bytes[0];
@@ -489,6 +490,13 @@ int arch_decode_instruction(const struct elf *elf, const
struct section *sec,
/* nopl/nopw */
*type = INSN_NOP;
+ } else if (op2 == 0x38 && op3 == 0xf8) {
+ if (insn.prefixes.nbytes == 1 &&
+ insn.prefixes.bytes[0] == 0xf2) {
+ /* enqcmd */
+ *type = INSN_ENQCMD;
+ }
+
} else if (op2 == 0xa0 || op2 == 0xa8) {
/* push fs/gs */
diff --git a/tools/objtool/check.c b/tools/objtool/check.c
index e5947fbb9e7a..91d13521d9d6 100644
--- a/tools/objtool/check.c
+++ b/tools/objtool/check.c
@@ -3133,6 +3133,21 @@ static int validate_reachable_instructions(struct
objtool_file *file)
return 0;
}
+static int validate_enqcmd(struct objtool_file *file)
+{
+ struct instruction *insn;
+
+ for_each_insn(file, insn) {
+ if (insn->type == INSN_ENQCMD) {
+ WARN_FUNC("enqcmd instruction", insn->sec,
+ insn->offset);
+ return 1;
+ }
+ }
+
+ return 0;
+}
+
int check(struct objtool_file *file)
{
int ret, warnings = 0;
@@ -3147,6 +3162,11 @@ int check(struct objtool_file *file)
if (list_empty(>insn_list))
goto out;
+ ret = validate_enqcmd(file);
+ if (ret < 0)
+ goto out;
+ warnings += ret;
+
if (vmlinux && !validate_dup) {
ret = validate_vmlinux_functions(file);
if (ret < 0)
diff --git a/tools/objtool/include/objtool/arch.h
b/tools/objtool/include/objtool/arch.h
index 062bb6e9b865..5147c0762b49 100644
--- a/tools/objtool/include/objtool/arch.h
+++ b/tools/objtool/include/objtool/arch.h
@@ -26,6 +26,7 @@ enum insn_type {
INSN_CLAC,
INSN_STD,
INSN_CLD,
+ INSN_ENQCMD,
INSN_OTHER,
};
--
2.33.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 2/8] x86/process: Clear PASID state for a newly forked/cloned thread
The PASID state has to be cleared on forks, since the child has a
different address space. The PASID is also cleared for thread clone. While
it would be correct to inherit the PASID in this case, it is unknown
whether the new task will use ENQCMD. Giving it the PASID "just in case"
would have the downside of increased context switch overhead to setting
the PASID MSR and losing init optimization.
Since #GP faults have to be handled on any threads that were created before
the PASID was assigned to the mm of the process, newly created threads
might as well be treated in a consistent way.
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
arch/x86/kernel/process.c | 8
1 file changed, 8 insertions(+)
diff --git a/arch/x86/kernel/process.c b/arch/x86/kernel/process.c
index 1d9463e3096b..c713986ef7d7 100644
--- a/arch/x86/kernel/process.c
+++ b/arch/x86/kernel/process.c
@@ -154,6 +154,14 @@ int copy_thread(unsigned long clone_flags, unsigned long
sp, unsigned long arg,
frame->flags = X86_EFLAGS_FIXED;
#endif
+ if (static_cpu_has(X86_FEATURE_ENQCMD)) {
+ /*
+* Clear the PASID bit in xfeatures so that the PASID MSR
+* will be initialized as init state (0).
+*/
+ p->thread.fpu.state.xsave.header.xfeatures &=
~XFEATURE_MASK_PASID;
+ }
+
/* Kernel thread ? */
if (unlikely(p->flags & PF_KTHREAD)) {
p->thread.pkru = pkru_get_init_value();
--
2.33.0
___
iommu mailing list
iommu@lists.linux-foundation.org
https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 4/8] x86/traps: Demand-populate PASID MSR via #GP
ENQCMD requires the IA32_PASID MSR has a valid PASID value which was
allocated to the process during bind. The MSR could be populated eagerly
by an IPI after the PASID is allocated in bind. But the method was
disabled in commit 9bfecd058339 ("x86/cpufeatures: Force disable
X86_FEATURE_ENQCMD and remove update_pasid()")' due to locking and other
issues.
Since the MSR was cleared in fork()/clone(), the first ENQCMD will
generate a #GP fault. The #GP fault handler will initialize the MSR
if a PASID has been allocated for this process.
The lazy enabling of the PASID MSR in the #GP handler is not an elegant
solution. But it has the least complexity that fits with h/w behavior.
Signed-off-by: Fenghua Yu
Reviewed-by: Tony Luck
---
arch/x86/include/asm/fpu/api.h | 6
arch/x86/include/asm/iommu.h | 2 ++
arch/x86/kernel/fpu/xstate.c | 59 ++
arch/x86/kernel/traps.c| 12 +++
drivers/iommu/intel/svm.c | 32 ++
5 files changed, 111 insertions(+)
diff --git a/arch/x86/include/asm/fpu/api.h b/arch/x86/include/asm/fpu/api.h
index ca4d0dee1ecd..f146849e5c8c 100644
--- a/arch/x86/include/asm/fpu/api.h
+++ b/arch/x86/include/asm/fpu/api.h
@@ -106,4 +106,10 @@ extern int cpu_has_xfeatures(u64 xfeatures_mask, const
char **feature_name);
*/
#define PASID_DISABLED 0
+#ifdef CONFIG_INTEL_IOMMU_SVM
+void fpu__pasid_write(u32 pasid);
+#else
+static inline void fpu__pasid_write(u32 pasid) { }
+#endif
+
#endif /* _ASM_X86_FPU_API_H */
diff --git a/arch/x86/include/asm/iommu.h b/arch/x86/include/asm/iommu.h
index bf1ed2ddc74b..9c4bf9b0702f 100644
--- a/arch/x86/include/asm/iommu.h
+++ b/arch/x86/include/asm/iommu.h
@@ -26,4 +26,6 @@ arch_rmrr_sanity_check(struct acpi_dmar_reserved_memory *rmrr)
return -EINVAL;
}
+bool __fixup_pasid_exception(void);
+
#endif /* _ASM_X86_IOMMU_H */
diff --git a/arch/x86/kernel/fpu/xstate.c b/arch/x86/kernel/fpu/xstate.c
index c8def1b7f8fb..8a89b2cecd77 100644
--- a/arch/x86/kernel/fpu/xstate.c
+++ b/arch/x86/kernel/fpu/xstate.c
@@ -1289,3 +1289,62 @@ int proc_pid_arch_status(struct seq_file *m, struct
pid_namespace *ns,
return 0;
}
#endif /* CONFIG_PROC_PID_ARCH_STATUS */
+
+#ifdef CONFIG_INTEL_IOMMU_SVM
+/**
+ * fpu__pasid_write - Write the current task's PASID state/MSR.
+ * @pasid: the PASID
+ *
+ * The PASID is written to the IA32_PASID MSR directly if the MSR is active.
+ * Otherwise it's written to the PASID. The IA32_PASID MSR should contain
+ * the PASID after returning to the user.
+ *
+ * This is called only when ENQCMD is enabled.
+ */
+void fpu__pasid_write(u32 pasid)
+{
+ struct xregs_state *xsave = >thread.fpu.state.xsave;
+ u64 msr_val = pasid | MSR_IA32_PASID_VALID;
+ struct fpu *fpu = >thread.fpu;
+
+ /*
+* ENQCMD always uses the compacted XSAVE format. Ensure the buffer
+* has space for the PASID.
+*/
+ BUG_ON(!(xsave->header.xcomp_bv & XFEATURE_MASK_PASID));
+
+ fpregs_lock();
+
+ /*
+* If the task's FPU doesn't need to be loaded or is valid, directly
+* write the IA32_PASID MSR. Otherwise, write the PASID state and
+* the MSR will be loaded from the PASID state before returning to
+* the user.
+*/
+ if (!test_thread_flag(TIF_NEED_FPU_LOAD) ||
+ fpregs_state_valid(fpu, smp_processor_id())) {
+ wrmsrl(MSR_IA32_PASID, msr_val);
+ } else {
+ struct ia32_pasid_state *ppasid_state;
+ /*
+* Mark XFEATURE_PASID as non-init in the XSAVE buffer.
+* This ensures that a subsequent XRSTOR will see the new
+* value instead of writing the init value to the MSR.
+*/
+ xsave->header.xfeatures |= XFEATURE_MASK_PASID;
+ ppasid_state = get_xsave_addr(xsave, XFEATURE_PASID);
+ /*
+* ppasid_state shouldn't be NULL because XFEATURE_PASID
+* was set just now.
+*
+* Please note that the following operation is a "write only"
+* operation on the PASID state and it writes the *ENTIRE*
+* state component. Please don't blindly copy this code to
+* modify other XSAVE states.
+*/
+ ppasid_state->pasid = msr_val;
+ }
+
+ fpregs_unlock();
+}
+#endif /* CONFIG_INTEL_IOMMU_SVM */
diff --git a/arch/x86/kernel/traps.c b/arch/x86/kernel/traps.c
index a58800973aed..a25d738ae839 100644
--- a/arch/x86/kernel/traps.c
+++ b/arch/x86/kernel/traps.c
@@ -61,6 +61,7 @@
#include
#include
#include
+#include
#ifdef CONFIG_X86_64
#include
@@ -526,6 +527,14 @@ static enum kernel_gp_hint get_kernel_gp_address(struct
pt_regs *regs,
return GP_CANONICAL;
}
+static bool fixup_pasid_exception(void)
+{
+ if (!cpu_feature_enabled(X86_FEATURE_ENQCMD))
+ return
[PATCH 0/8] Re-enable ENQCMD and PASID MSR
Since updating PASID (Process Address Space ID) MSR through IPI has a few issues that are beyond repair, Thomas disables ENQCMD [1]. Please check Documentation/x86/sva.rst for various concepts and terms related to PASID, ENQCMD, SVM (Shared Virtual Memory), etc. This series re-enables ENQCMD and IA32_PASID MSR by using a #GP fix up method previously published in [2]. A PASID is allocated to a mm once a SVM is bound to the mm via intel_svm_bind() API. The #GP fix up method updates the PASID MSR from the mm's PASID in #GP handler when one thread in a process executes ENQCMD for the first time and one reference is taken to the PASID. Once the MSR is uploaded, the thread keeps and can use it for the rest life time of the thread. In exit(2) or unbind, the PASID's reference is dropped and the PASID is freed if there is no reference. References: 1. ENQCMD was disabled in upstream due to serious issues: https://lore.kernel.org/linux-iommu/87mtsd6gr9@nanos.tec.linutronix.de/ 2. #GP fix up PASID MSR: https://lore.kernel.org/linux-iommu/1594684087-61184-1-git-send-email-fenghua...@intel.com/ Fenghua Yu (7): iommu/vt-d: Clean up unused PASID updating functions x86/process: Clear PASID state for a newly forked/cloned thread x86/traps: Demand-populate PASID MSR via #GP x86/mmu: Add mm-based PASID refcounting x86/cpufeatures: Re-enable ENQCMD tools/objtool: Check for use of the ENQCMD instruction in the kernel docs: x86: Change documentation for SVA (Shared Virtual Addressing) Peter Zijlstra (1): sched: Define and initialize a flag to identify valid PASID in the task Documentation/x86/sva.rst| 81 ++-- arch/x86/include/asm/disabled-features.h | 7 +- arch/x86/include/asm/fpu/api.h | 6 +- arch/x86/include/asm/iommu.h | 8 ++ arch/x86/include/asm/mmu_context.h | 2 + arch/x86/kernel/fpu/xstate.c | 59 +++ arch/x86/kernel/process.c| 8 ++ arch/x86/kernel/traps.c | 12 +++ drivers/iommu/intel/svm.c| 95 ++-- include/linux/sched.h| 4 + kernel/fork.c| 4 + tools/objtool/arch/x86/decode.c | 10 ++- tools/objtool/check.c| 20 + tools/objtool/include/objtool/arch.h | 1 + 14 files changed, 283 insertions(+), 34 deletions(-) -- 2.33.0 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
[PATCH 6/8] x86/cpufeatures: Re-enable ENQCMD
Since ENQCMD is handled by #GP fix up, it can be re-enabled. The ENQCMD feature cannot be used if CONFIG_INTEL_IOMMU_SVM is not set. Add X86_FEATURE_ENQCMD to the disabled features mask as appropriate and use cpu_feature_enabled() to check the feature. Signed-off-by: Fenghua Yu Reviewed-by: Tony Luck --- arch/x86/include/asm/disabled-features.h | 7 +-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/arch/x86/include/asm/disabled-features.h b/arch/x86/include/asm/disabled-features.h index 8f28fafa98b3..1231d63f836d 100644 --- a/arch/x86/include/asm/disabled-features.h +++ b/arch/x86/include/asm/disabled-features.h @@ -56,8 +56,11 @@ # define DISABLE_PTI (1 << (X86_FEATURE_PTI & 31)) #endif -/* Force disable because it's broken beyond repair */ -#define DISABLE_ENQCMD (1 << (X86_FEATURE_ENQCMD & 31)) +#ifdef CONFIG_INTEL_IOMMU_SVM +# define DISABLE_ENQCMD0 +#else +# define DISABLE_ENQCMD(1 << (X86_FEATURE_ENQCMD & 31)) +#endif #ifdef CONFIG_X86_SGX # define DISABLE_SGX 0 -- 2.33.0 ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v3 5/8] x86/sme: Replace occurrences of sme_active() with cc_platform_has()
On Wed, Sep 08, 2021 at 05:58:36PM -0500, Tom Lendacky wrote: > diff --git a/arch/x86/mm/mem_encrypt_identity.c > b/arch/x86/mm/mem_encrypt_identity.c > index 470b20208430..eff4d19f9cb4 100644 > --- a/arch/x86/mm/mem_encrypt_identity.c > +++ b/arch/x86/mm/mem_encrypt_identity.c > @@ -30,6 +30,7 @@ > #include > #include > #include > +#include > > #include > #include > @@ -287,7 +288,7 @@ void __init sme_encrypt_kernel(struct boot_params *bp) > unsigned long pgtable_area_len; > unsigned long decrypted_base; > > - if (!sme_active()) > + if (!cc_platform_has(CC_ATTR_HOST_MEM_ENCRYPT)) > return; > > /* This change break boot for me (in KVM on Intel host). It only reproduces with allyesconfig. More reasonable config works fine, but I didn't try to find exact cause in config. Convertion to cc_platform_has() in __startup_64() in 8/8 has the same effect. I believe it caused by sme_me_mask access from __startup_64() without fixup_pointer() magic. I think __startup_64() requires special treatement and we should avoid cc_platform_has() there (or have a special version of the helper). Note that only AMD requires these cc_platform_has() to return true. -- Kirill A. Shutemov ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
Re: [PATCH v2 0/4] swiotlb-xen: remaining fixes and adjustments
On 17.09.21 12:43, Jan Beulich wrote: The primary intention really was the last patch, there you go (on top of what is already in xen/tip.git for-linus-5.15) ... 1: swiotlb-xen: ensure to issue well-formed XENMEM_exchange requests 2: PCI: only build xen-pcifront in PV-enabled environments 3: xen/pci-swiotlb: reduce visibility of symbols 4: swiotlb-xen: this is PV-only on x86 All 4 patches pushed to xen/tip.git for-linus-5.15b Juergen OpenPGP_0xB0DE9DD628BF132F.asc Description: OpenPGP public key OpenPGP_signature Description: OpenPGP digital signature ___ iommu mailing list iommu@lists.linux-foundation.org https://lists.linuxfoundation.org/mailman/listinfo/iommu
