Re: [RFC PATCH v1 15/18] x86: Enable memory encryption on the APs

On 4/27/2016 10:58 AM, Tom Lendacky wrote: Add support to set the memory encryption enable flag on the APs during realmode initialization. When an AP is started it checks this flag, and if set, enables memory encryption on its core. Signed-off-by: Tom Lendacky ---

Re: [Qemu-devel] kvm bug in __rmap_clear_dirty during live migration

On 2/25/2017 2:44 PM, Herongguang (Stephen) wrote: On 2017/2/24 23:14, Paolo Bonzini wrote: On 24/02/2017 16:10, Chris Friesen wrote: On 02/23/2017 08:23 PM, Herongguang (Stephen) wrote: On 2017/2/22 22:43, Paolo Bonzini wrote: Hopefully Gaohuai and Rongguang can help with this too.

Re: [Qemu-devel] kvm bug in __rmap_clear_dirty during live migration

Thanks! Thanks, -Kai On 3/14/2017 5:57 AM, Paolo Bonzini wrote: On 13/03/2017 15:58, fangying wrote: Hi, Huang Kai After weeks of intensive testing, we think the problem is solved and this issue can be closed. Thanks for the update. We got to the same conclusion. Paolo

Re: [PATCH v2 2/3] nVMX: Implement emulated Page Modification Logging

On 5/6/2017 7:25 AM, Bandan Das wrote: With EPT A/D enabled, processor access to L2 guest paging structures will result in a write violation. When this happens, write the GUEST_PHYSICAL_ADDRESS to the pml buffer provided by L1 if the access is write and the dirty bit is being set. This patch

Re: [PATCH v2 1/3] kvm: x86: Add a hook for arch specific dirty logging emulation

On 5/6/2017 7:25 AM, Bandan Das wrote: When KVM updates accessed/dirty bits, this hook can be used to invoke an arch specific function that implements/emulates dirty logging such as PML. Signed-off-by: Bandan Das --- arch/x86/include/asm/kvm_host.h | 2 ++

Re: [PATCH v2 2/3] nVMX: Implement emulated Page Modification Logging

On 5/11/2017 4:00 AM, Bandan Das wrote: Paolo Bonzini writes: ... Is the purpose of returning 1 to make upper layer code to inject PML full VMEXIt to L1 in nested_ept_inject_page_fault? Yes, it triggers a fault + +gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS) &

Re: [PATCH v2 1/3] kvm: x86: Add a hook for arch specific dirty logging emulation

On 5/11/2017 3:53 AM, Bandan Das wrote: Hi Kai, "Huang, Kai" <kai.hu...@linux.intel.com> writes: On 5/6/2017 7:25 AM, Bandan Das wrote: When KVM updates accessed/dirty bits, this hook can be used to invoke an arch specific function that implements/emulates dirty log

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

[snip..] > > > > > > @@ -38,6 +39,18 @@ static LIST_HEAD(sgx_active_page_list); static > > > DEFINE_SPINLOCK(sgx_active_page_list_lock); > > > static struct task_struct *ksgxswapd_tsk; static > > > DECLARE_WAIT_QUEUE_HEAD(ksgxswapd_waitq); > > > +static struct notifier_block sgx_pm_notifier;

RE: [PATCH v13 02/13] x86/cpufeature: Add SGX and SGX_LC CPU features

> -Original Message- > From: Jarkko Sakkinen [mailto:jarkko.sakki...@linux.intel.com] > Sent: Tuesday, August 28, 2018 7:17 PM > To: Huang, Kai > Cc: x...@kernel.org; platform-driver-...@vger.kernel.org; Hansen, Dave > ; Christopherson, Sean J > ; nhor...@

Re: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

On Mon, 2018-08-27 at 21:53 +0300, Jarkko Sakkinen wrote: > From: Sean Christopherson > > Add a function to perform ENCLS(EINIT), which initializes an enclave, > which can be used by a driver for running enclaves and VMMs. > > Writing the LE hash MSRs is extraordinarily expensive, e.g. 3-4x >

RE: [PATCH v13 02/13] x86/cpufeature: Add SGX and SGX_LC CPU features

> +#define X86_FEATURE_SGX_LC (16*32+30) /* supports SGX launch > configuration */ Sorry if it was me who wrote the comment "SGX launch configuration". I think we should just use "SGX launch control". :) Thanks, -Kai > > /* AMD-defined CPU features, CPUID level 0x8007 (EBX),

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> -Original Message- > From: platform-driver-x86-ow...@vger.kernel.org [mailto:platform-driver-x86- > ow...@vger.kernel.org] On Behalf Of Sean Christopherson > Sent: Wednesday, September 5, 2018 4:36 AM > To: Jarkko Sakkinen > Cc: Huang, Kai ; platform-driver-...@vg

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> -Original Message- > From: Christopherson, Sean J > Sent: Thursday, August 30, 2018 8:34 AM > To: Huang, Kai > Cc: Jarkko Sakkinen ; platform-driver- > x...@vger.kernel.org; x...@kernel.org; nhor...@redhat.com; linux- > ker...@vger.kernel.org; t...@linutr

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> > > > > Some kind of counter is required to keep track of the power cycle. > > > > > When going to sleep the sgx_pm_cnt is increased. sgx_einit() > > > > > compares the current value of the global count to the value in > > > > > the cache entry to see whether we are in a new power cycle. > > > >

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> -Original Message- > From: linux-sgx-ow...@vger.kernel.org [mailto:linux-sgx- > ow...@vger.kernel.org] On Behalf Of Jarkko Sakkinen > Sent: Tuesday, September 4, 2018 7:19 AM > To: Christopherson, Sean J > Cc: Huang, Kai ; platform-driver-...@vger.kernel.org; >

Re: [PATCH v9 5/6] mm,thp: add read-only THP support for (non-shmem) FS

On Mon, 2019-06-24 at 17:12 -0700, Song Liu wrote: > This patch is (hopefully) the first step to enable THP for non-shmem > filesystems. > > This patch enables an application to put part of its text sections to THP > via madvise, for example: > > madvise((void *)0x60, 0x20,

Re: [PATCH, RFC 20/62] mm/page_ext: Export lookup_page_ext() symbol

On Mon, 2019-06-17 at 23:01 +1200, Kai Huang wrote: > On Mon, 2019-06-17 at 11:30 +0200, Peter Zijlstra wrote: > > On Sat, Jun 15, 2019 at 01:44:43AM +0300, Kirill A. Shutemov wrote: > > > On Fri, Jun 14, 2019 at 01:12:59PM +0200, Peter Zijlstra wrote: > > > > On Wed, May 08, 2019 at 05:43:40PM

Re: [RFC 09/16] KVM: Protected memory extension

On Mon, 2020-05-25 at 18:34 +0300, Kirill A. Shutemov wrote: > On Mon, May 25, 2020 at 05:26:37PM +0200, Vitaly Kuznetsov wrote: > > "Kirill A. Shutemov" writes: > > > > > Add infrastructure that handles protected memory extension. > > > > > > Arch-specific code has to provide hypercalls and

Re: [RFC 02/16] x86/kvm: Introduce KVM memory protection feature

On Wed, 2020-05-27 at 10:39 +0200, Vitaly Kuznetsov wrote: > Sean Christopherson writes: > > > On Mon, May 25, 2020 at 06:15:25PM +0300, Kirill A. Shutemov wrote: > > > On Mon, May 25, 2020 at 04:58:51PM +0200, Vitaly Kuznetsov wrote: > > > > > @@ -727,6 +734,15 @@ static void __init

Re: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

On Mon, 2018-08-27 at 21:53 +0300, Jarkko Sakkinen wrote: > From: Sean Christopherson > > Add a function to perform ENCLS(EINIT), which initializes an enclave, > which can be used by a driver for running enclaves and VMMs. > > Writing the LE hash MSRs is extraordinarily expensive, e.g. 3-4x >

RE: [PATCH v13 02/13] x86/cpufeature: Add SGX and SGX_LC CPU features

> +#define X86_FEATURE_SGX_LC (16*32+30) /* supports SGX launch > configuration */ Sorry if it was me who wrote the comment "SGX launch configuration". I think we should just use "SGX launch control". :) Thanks, -Kai > > /* AMD-defined CPU features, CPUID level 0x8007 (EBX),

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

[snip..] > > > > > > @@ -38,6 +39,18 @@ static LIST_HEAD(sgx_active_page_list); static > > > DEFINE_SPINLOCK(sgx_active_page_list_lock); > > > static struct task_struct *ksgxswapd_tsk; static > > > DECLARE_WAIT_QUEUE_HEAD(ksgxswapd_waitq); > > > +static struct notifier_block sgx_pm_notifier;

RE: [PATCH v13 02/13] x86/cpufeature: Add SGX and SGX_LC CPU features

> -Original Message- > From: Jarkko Sakkinen [mailto:jarkko.sakki...@linux.intel.com] > Sent: Tuesday, August 28, 2018 7:17 PM > To: Huang, Kai > Cc: x...@kernel.org; platform-driver-...@vger.kernel.org; Hansen, Dave > ; Christopherson, Sean J > ; nhor...@

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> -Original Message- > From: Christopherson, Sean J > Sent: Thursday, August 30, 2018 8:34 AM > To: Huang, Kai > Cc: Jarkko Sakkinen ; platform-driver- > x...@vger.kernel.org; x...@kernel.org; nhor...@redhat.com; linux- > ker...@vger.kernel.org; t...@linutr

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> > > > > Some kind of counter is required to keep track of the power cycle. > > > > > When going to sleep the sgx_pm_cnt is increased. sgx_einit() > > > > > compares the current value of the global count to the value in > > > > > the cache entry to see whether we are in a new power cycle. > > > >

Re: [PATCH v20 16/28] x86/sgx: Add provisioning

On Wed, 2019-04-17 at 13:39 +0300, Jarkko Sakkinen wrote: > In order to provide a mechanism for devilering provisoning rights: > > 1. Add a new device file /dev/sgx/provision that works as a token for >allowing an enclave to have the provisioning privileges. > 2. Add a new ioctl called

Re: [PATCH v19,RESEND 16/27] x86/sgx: Add the Linux SGX Enclave Driver

On Wed, 2019-03-20 at 18:21 +0200, Jarkko Sakkinen wrote: > Intel Software Guard eXtensions (SGX) is a set of CPU instructions that > can be used by applications to set aside private regions of code and > data. The code outside the enclave is disallowed to access the memory > inside the enclave by

Re: [PATCH v19,RESEND 08/27] x86/cpu/intel: Detect SGX support and update caps appropriately

On Wed, 2019-03-20 at 18:21 +0200, Jarkko Sakkinen wrote: > From: Sean Christopherson > > Similar to other large Intel features such as VMX and TXT, SGX must be > explicitly enabled in IA32_FEATURE_CONTROL MSR to be truly usable. > Clear all SGX related capabilities if SGX is not fully enabled

RE: [PATCH v19,RESEND 16/27] x86/sgx: Add the Linux SGX Enclave Driver

> On Tue, Mar 26, 2019 at 01:40:57PM +0100, Thomas Gleixner wrote: > > On Tue, 26 Mar 2019, Huang, Kai wrote: > > > On Wed, 2019-03-20 at 18:21 +0200, Jarkko Sakkinen wrote: > > > > 13 files changed, 1657 insertions(+), 2 deletions(-) create mode > > > &

RE: [PATCH v19,RESEND 08/27] x86/cpu/intel: Detect SGX support and update caps appropriately

> > On Tue, Mar 26, 2019 at 05:17:40AM -0700, Huang, Kai wrote: > > On Wed, 2019-03-20 at 18:21 +0200, Jarkko Sakkinen wrote: > > > From: Sean Christopherson > > > > > > Similar to other large Intel features such as VMX and TXT, SGX must > > > b

RE: [PATCH v19,RESEND 08/27] x86/cpu/intel: Detect SGX support and update caps appropriately

> On Tue, Mar 26, 2019 at 02:25:52PM -0700, Huang, Kai wrote: > > > > > > That being said, this in no way impacts KVM's ability to virtualize SGX, > > > e.g. > > > KVM can directly do CPUID and {RD,WR}MSR to probe the capabilities > > > of the pla

RE: x86/sgx: uapi change proposal

> > > > Can one of you explain why SGX_ENCLAVE_CREATE is better than just > > opening a new instance of /dev/sgx for each encalve? > > Directly associating /dev/sgx with an enclave means /dev/sgx can't be used > to provide ioctl()'s for other SGX-related needs, e.g. to mmap() raw EPC and > expose

RE: x86/sgx: uapi change proposal

> On Tue, Jan 08, 2019 at 11:27:11AM -0800, Huang, Kai wrote: > > > > > > > > Can one of you explain why SGX_ENCLAVE_CREATE is better than just > > > > opening a new instance of /dev/sgx for each encalve? > > > > > > Directly associating /d

RE: x86/sgx: uapi change proposal

> > > That's possible, but it has several downsides. > > > > > > - Duplicates a lot of code in KVM for managing memory regions. > > > > I don't see why there will be duplicated code. you can simply call > > __x86_set_memory_region to create private slot. It is KVM x86 > > equivalent to

Re: x86/sgx: uapi change proposal

On Thu, 2019-01-10 at 13:34 -0800, Andy Lutomirski wrote: > > > On Jan 9, 2019, at 8:31 AM, Sean Christopherson > > > wrote: > > > > > > On Tue, Jan 08, 2019 at 02:54:11PM -0800, Andy Lutomirski wrote: > > > On Tue, Jan 8, 2019 at 2:09 PM Sean Christopherson > > > wrote: > > > > > > > >

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> -Original Message- > From: linux-sgx-ow...@vger.kernel.org [mailto:linux-sgx- > ow...@vger.kernel.org] On Behalf Of Jarkko Sakkinen > Sent: Tuesday, September 4, 2018 7:19 AM > To: Christopherson, Sean J > Cc: Huang, Kai ; platform-driver-...@vger.kernel.org; >

RE: [PATCH v13 10/13] x86/sgx: Add sgx_einit() for initializing enclaves

> -Original Message- > From: platform-driver-x86-ow...@vger.kernel.org [mailto:platform-driver-x86- > ow...@vger.kernel.org] On Behalf Of Sean Christopherson > Sent: Wednesday, September 5, 2018 4:36 AM > To: Jarkko Sakkinen > Cc: Huang, Kai ; platform-driver-...@vg

Re: [Qemu-devel] kvm bug in __rmap_clear_dirty during live migration

Thanks! Thanks, -Kai On 3/14/2017 5:57 AM, Paolo Bonzini wrote: On 13/03/2017 15:58, fangying wrote: Hi, Huang Kai After weeks of intensive testing, we think the problem is solved and this issue can be closed. Thanks for the update. We got to the same conclusion. Paolo

Re: [PATCH v2 2/3] nVMX: Implement emulated Page Modification Logging

On 5/6/2017 7:25 AM, Bandan Das wrote: With EPT A/D enabled, processor access to L2 guest paging structures will result in a write violation. When this happens, write the GUEST_PHYSICAL_ADDRESS to the pml buffer provided by L1 if the access is write and the dirty bit is being set. This patch

Re: [PATCH v2 1/3] kvm: x86: Add a hook for arch specific dirty logging emulation

On 5/6/2017 7:25 AM, Bandan Das wrote: When KVM updates accessed/dirty bits, this hook can be used to invoke an arch specific function that implements/emulates dirty logging such as PML. Signed-off-by: Bandan Das --- arch/x86/include/asm/kvm_host.h | 2 ++ arch/x86/kvm/mmu.c |

Re: [PATCH v2 2/3] nVMX: Implement emulated Page Modification Logging

On 5/11/2017 4:00 AM, Bandan Das wrote: Paolo Bonzini writes: ... Is the purpose of returning 1 to make upper layer code to inject PML full VMEXIt to L1 in nested_ept_inject_page_fault? Yes, it triggers a fault + +gpa = vmcs_read64(GUEST_PHYSICAL_ADDRESS) & ~0xFFFull; + +

Re: [PATCH v2 1/3] kvm: x86: Add a hook for arch specific dirty logging emulation

On 5/11/2017 3:53 AM, Bandan Das wrote: Hi Kai, "Huang, Kai" writes: On 5/6/2017 7:25 AM, Bandan Das wrote: When KVM updates accessed/dirty bits, this hook can be used to invoke an arch specific function that implements/emulates dirty logging such as PML. Signed-off-by:

Re: [Qemu-devel] kvm bug in __rmap_clear_dirty during live migration

On 2/25/2017 2:44 PM, Herongguang (Stephen) wrote: On 2017/2/24 23:14, Paolo Bonzini wrote: On 24/02/2017 16:10, Chris Friesen wrote: On 02/23/2017 08:23 PM, Herongguang (Stephen) wrote: On 2017/2/22 22:43, Paolo Bonzini wrote: Hopefully Gaohuai and Rongguang can help with this too.

Re: [RFC PATCH v1 15/18] x86: Enable memory encryption on the APs

On 4/27/2016 10:58 AM, Tom Lendacky wrote: Add support to set the memory encryption enable flag on the APs during realmode initialization. When an AP is started it checks this flag, and if set, enables memory encryption on its core. Signed-off-by: Tom Lendacky ---

RE: [PATCH v3 00/25] KVM SGX virtualization support

> On Fri, Mar 19, 2021 at 08:29:27PM +1300, Kai Huang wrote: > > This series adds KVM SGX virtualization support. The first 14 patches > > starting with x86/sgx or x86/cpu.. are necessary changes to x86 and > > SGX core/driver to support KVM SGX virtualization, while the rest are > > patches > to

Re: [PATCH v4 03/18] x86/sgx: Add sgx_epc_lru_lists to encapsulate LRU lists

Some non-technical staff: On Tue, 2023-09-12 at 21:06 -0700, Haitao Huang wrote: > From: Kristen Carlson Accardi The patch was from Kristen, but ... > > Introduce a data structure to wrap the existing reclaimable list and its > spinlock. Each cgroup later will have one instance of this

Re: [PATCH v4 03/18] x86/sgx: Add sgx_epc_lru_lists to encapsulate LRU lists

On Thu, 2023-09-14 at 09:13 -0700, Dave Hansen wrote: > On 9/14/23 03:31, Huang, Kai wrote: > > > Signed-off-by: Haitao Huang > > > Cc: Sean Christopherson > > You don't need 'Cc:' Sean if the patch has Sean's SoB. > > It is a SoB for Sean's @intel add

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

> > > From this perspective, I think the current implementation is > "well-defined": EPC cgroup limits for VMs are only enforced at VM launch > time, not runtime. In practice, SGX VM can be launched only with fixed > EPC size and all those EPCs are fully committed to the VM once

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Mon, 2023-10-16 at 19:10 -0500, Haitao Huang wrote: > On Mon, 16 Oct 2023 16:09:52 -0500, Huang, Kai wrote: > [...] > > > still need to fix the bug mentioned above here. > > > > I really think you should just go this simple way: > > > > When you wan

Re: [PATCH v6 04/12] x86/sgx: Implement basic EPC misc cgroup functionality

On Mon, 2023-10-30 at 11:20 -0700, Haitao Huang wrote: > +static int __init sgx_epc_cgroup_init(void) > +{ > + struct misc_cg *cg; > + > + if (!boot_cpu_has(X86_FEATURE_SGX)) > + return 0; > + > + cg = misc_cg_root(); > + BUG_ON(!cg); > + > + return

Re: [PATCH v6 04/12] x86/sgx: Implement basic EPC misc cgroup functionality

> > > > +/** > > > + * sgx_epc_cgroup_try_charge() - hierarchically try to charge a single > > > EPC page > > > + * > > > + * Returns EPC cgroup or NULL on success, -errno on failure. > > > + */ > > > +struct sgx_epc_cgroup *sgx_epc_cgroup_try_charge(void) > > > +{ > > > + struct sgx_epc_cgroup

RE: [PATCH v6 04/12] x86/sgx: Implement basic EPC misc cgroup functionality

> I should have sticked to the orignial comment added in code. Actually > __sgx_alloc_epc_page() can fail if system runs out of EPC. That's the really > reason > for global reclaim. The free count enforcement is near the end of this method > after should_reclaim() check. Hi Haitao, Sorry I have

Re: [PATCH v6 04/12] x86/sgx: Implement basic EPC misc cgroup functionality

On Mon, 2023-10-30 at 11:20 -0700, Haitao Huang wrote: > From: Kristen Carlson Accardi > > Implement support for cgroup control of SGX Enclave Page Cache (EPC) > memory using the misc cgroup controller. EPC memory is independent > from normal system memory, e.g. must be reserved at boot from RAM

Re: [PATCH v5 08/18] x86/sgx: Use a list to track to-be-reclaimed pages

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > @@ -314,18 +313,22 @@ static void sgx_reclaim_pages(void) >   >   if (kref_get_unless_zero(_page->encl->refcount) != 0) { >   sgx_epc_page_set_state(epc_page, > SGX_EPC_PAGE_RECLAIM_IN_PROGRESS); > -

Re: [PATCH v5 11/18] x86/sgx: store unreclaimable pages in LRU lists

> --- a/arch/x86/kernel/cpu/sgx/encl.c > +++ b/arch/x86/kernel/cpu/sgx/encl.c > @@ -746,6 +746,7 @@ void sgx_encl_release(struct kref *ref) > xa_destroy(>page_array); > > if (!encl->secs_child_cnt && encl->secs.epc_page) { > + sgx_drop_epc_page(encl->secs.epc_page); >

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Fri, 2023-09-29 at 10:06 -0500, Haitao Huang wrote: > On Wed, 27 Sep 2023 16:21:19 -0500, Huang, Kai wrote: > > > On Wed, 2023-09-27 at 10:35 -0500, Haitao Huang wrote: > > > > > + > > > > > + /* Possible owner types */ > > > > > +

Re: [PATCH v5 01/18] cgroup/misc: Add per resource callbacks for CSS events

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Kristen Carlson Accardi > > The misc cgroup controller (subsystem) currently does not perform > resource type specific action for Cgroups Subsystem State (CSS) events: > the 'css_alloc' event when a cgroup is created and the

Re: [PATCH v5 06/18] x86/sgx: Introduce EPC page states

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > Use the lower 3 bits in the flags field of sgx_epc_page struct to > track EPC states in its life cycle and define an enum for possible > states. More state(s) will be added later. This patch does more than what the changelog claims to do.

Re: [PATCH v5 07/18] x86/sgx: Introduce RECLAIM_IN_PROGRESS state

> @@ -312,13 +312,15 @@ static void sgx_reclaim_pages(void) > list_del_init(_page->list); > encl_page = epc_page->owner; > > - if (kref_get_unless_zero(_page->encl->refcount) != 0) > + if (kref_get_unless_zero(_page->encl->refcount) != 0) { >

Re: [PATCH v5 05/18] x86/sgx: Store reclaimable EPC pages in sgx_epc_lru_lists

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > --- a/arch/x86/kernel/cpu/sgx/main.c > +++ b/arch/x86/kernel/cpu/sgx/main.c > @@ -268,7 +268,6 @@ static void sgx_reclaimer_write(struct sgx_epc_page > *epc_page, >   goto out; >   >  

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Sean Christopherson > > In a later patch, when a cgroup has exceeded the max capacity for EPC > pages, it may need to identify and OOM kill a less active enclave to > make room for other enclaves within the same group. Such a victim

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Sean Christopherson > > In a later patch, when a cgroup has exceeded the max capacity for EPC > pages, it may need to identify and OOM kill a less active enclave to > make room for other enclaves within the same group. Such a victim

Re: [PATCH v5 11/18] x86/sgx: store unreclaimable pages in LRU lists

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Sean Christopherson > > When an OOM event occurs, all pages associated with an enclave will need > to be freed, including pages that are not currently tracked by the > cgroup LRU lists. What are "cgroup LRU lists"? > > Add a new

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Wed, 2023-09-27 at 10:35 -0500, Haitao Huang wrote: > > > + > > > + /* Possible owner types */ > > > + union { > > > + struct sgx_encl_page *encl_page; > > > + struct sgx_encl *encl; > > > + }; > > > > Sadly for virtual EPC page the owner is set to the 'sgx_vepc' instance it >

Re: [PATCH v5 02/18] cgroup/misc: Add SGX EPC resource type and export APIs for SGX driver

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Kristen Carlson Accardi > > Add SGX EPC memory, MISC_CG_RES_SGX_EPC, to be a valid resource type > for the misc controller. > > Add per resource type private data so that SGX can store additional per > cgroup data in

Re: [PATCH v5 02/18] cgroup/misc: Add SGX EPC resource type and export APIs for SGX driver

On Tue, 2023-10-03 at 02:00 -0500, Haitao Huang wrote: > On Wed, 27 Sep 2023 22:59:12 -0500, Huang, Kai wrote: > > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > From: Kristen Carlson Accardi > > > > > > Add SGX EPC memory, MISC_CG_R

Re: [PATCH v5 11/18] x86/sgx: store unreclaimable pages in LRU lists

On Tue, 2023-10-03 at 00:15 -0500, Haitao Huang wrote: > On Thu, 28 Sep 2023 04:41:33 -0500, Huang, Kai wrote: > > > > > > --- a/arch/x86/kernel/cpu/sgx/encl.c > > > +++ b/arch/x86/kernel/cpu/sgx/encl.c > > > @@ -746,6 +746,7 @@ void sgx_encl_relea

Re: [PATCH v5 06/18] x86/sgx: Introduce EPC page states

On Mon, 2023-10-02 at 23:49 -0500, Haitao Huang wrote: > On Wed, 27 Sep 2023 05:28:36 -0500, Huang, Kai wrote: > > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > Use the lower 3 bits in the flags field of sgx_epc_page struct to > > > track EPC state

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Tue, 2023-10-03 at 01:45 -0500, Haitao Huang wrote: > > > > Btw, probably a dumb question: > > > > Theoretically if you only need to find a victim enclave you don't need  > > to put VA > > pages to the unreclaimable list, because those VA pages will be freed  > > anyway > > when enclave is

Re: [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller

> + > +static inline struct sgx_epc_cgroup *sgx_epc_cgroup_from_misc_cg(struct > misc_cg *cg) > +{ > + if (cg) > + return (struct sgx_epc_cgroup > *)(cg->res[MISC_CG_RES_SGX_EPC].priv); > + > + return NULL; > +} > + > Is it good idea to allow passing a NULL @cg to this

Re: [PATCH v5 15/18] x86/sgx: Prepare for multiple LRUs

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > +static inline struct sgx_epc_lru_lists *sgx_lru_lists(struct sgx_epc_page > *epc_page) > +{ > + return _global_lru; > +} > + > +static inline bool sgx_can_reclaim(void) > +{ > + return !list_empty(_global_lru.reclaimable); > +} > +

Re: [PATCH v5 13/18] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Sean Christopherson > > Adjust and expose the top-level reclaim function as > sgx_reclaim_epc_pages() for use by the upcoming EPC cgroup, which will > initiate reclaim to enforce the max limit. > > Make these adjustments to the

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Wed, 2023-10-04 at 23:22 -0500, Haitao Huang wrote: > On Wed, 04 Oct 2023 16:13:41 -0500, Huang, Kai wrote: > > > On Wed, 2023-10-04 at 10:03 -0500, Haitao Huang wrote: > > > On Tue, 03 Oct 2023 15:07:42 -0500, Huang, Kai > > > wrote: > > > &g

Re: [PATCH v5 13/18] x86/sgx: Expose sgx_reclaim_pages() for use by EPC cgroup

> > > > > > -/* > > > +/** > > > + * sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers > > > + * @nr_to_scan: Number of EPC pages to scan for reclaim > > > + * @ignore_age: Reclaim a page even if it is young > > > + * > > > * Take a fixed number of pages from

Re: [PATCH v5 15/18] x86/sgx: Prepare for multiple LRUs

On Thu, 2023-10-05 at 14:33 -0500, Haitao Huang wrote: > On Thu, 05 Oct 2023 07:30:46 -0500, Huang, Kai wrote: > > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > +static inline struct sgx_epc_lru_lists *sgx_lru_lists(struct > > > sgx_epc_page *epc

Re: [PATCH v7 00/13] selftests/sgx: Fix compilation errors

Hi Jo, Just FYI I won't review the rest patches in this series. One of the reasons is I am not that familiar with the rest. Jarkko has reviewed anyway :-). On Thu, 2023-10-05 at 17:38 +0200, Jo Van Bulck wrote: > Hi, > > This patch series ensures that all SGX selftests succeed when compiling

Re: [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller

> --- > arch/x86/Kconfig | 13 + > arch/x86/kernel/cpu/sgx/Makefile | 1 + > arch/x86/kernel/cpu/sgx/epc_cgroup.c | 415 +++ > arch/x86/kernel/cpu/sgx/epc_cgroup.h | 59 > arch/x86/kernel/cpu/sgx/main.c | 68 - >

Re: [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Kristen Carlson Accardi > > Implement support for cgroup control of SGX Enclave Page Cache (EPC) > memory using the misc cgroup controller. EPC memory is independent > from normal system memory, e.g. must be reserved at boot from RAM

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Tue, 2023-10-10 at 11:49 -0500, Haitao Huang wrote: > > > > This patch adds SGX_ENCL_NO_MEMORY.  I guess we can use it for virtual  > > EPC too? > > > > That flag is set for enclaves, do you mean we set similar flag in vepc  > struct? Yes.

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Tue, 2023-10-10 at 12:05 -0500, Haitao Huang wrote: > On Mon, 09 Oct 2023 21:12:27 -0500, Huang, Kai wrote: > > > > > > > > > > > > > > Later the hosting process could migrated/reassigned to another > > > c

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Tue, 2023-10-10 at 11:49 -0500, Haitao Huang wrote: > On Mon, 09 Oct 2023 20:34:29 -0500, Huang, Kai wrote: > > > On Tue, 2023-10-10 at 00:50 +0000, Huang, Kai wrote: > > > On Mon, 2023-10-09 at 17:23 -0700, Sean Christopherson wrote: > > > > On M

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Thu, 2023-10-12 at 08:27 -0500, Haitao Huang wrote: > On Tue, 10 Oct 2023 19:51:17 -0500, Huang, Kai wrote: > [...] > > (btw, even you track VA/SECS pages in unreclaimable list, given they > > both have > > 'enclave' as the owner, do you still

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Wed, 2023-10-11 at 01:14 +, Huang, Kai wrote: > On Tue, 2023-10-10 at 11:49 -0500, Haitao Huang wrote: > > > > > > This patch adds SGX_ENCL_NO_MEMORY.  I guess we can use it for virtual  > > > EPC too? > > > > > > > That flag is set fo

Re: [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller

> +/** > + * sgx_epc_cgroup_lru_empty() - check if a cgroup tree has no pages on its > lrus > + * @root:root of the tree to check > + * > + * Return: %true if all cgroups under the specified root have empty LRU > lists. > + * Used to avoid livelocks due to a cgroup having a non-zero charge

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Mon, 2023-10-09 at 17:23 -0700, Sean Christopherson wrote: > On Mon, Oct 09, 2023, Kai Huang wrote: > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > +/** > > > + * sgx_epc_oom() - invoke EPC out-of-memory handling on target LRU > > > + * @lru: LRU that is low > > > + * > > > + *

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Tue, 2023-10-10 at 00:50 +, Huang, Kai wrote: > On Mon, 2023-10-09 at 17:23 -0700, Sean Christopherson wrote: > > On Mon, Oct 09, 2023, Kai Huang wrote: > > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > > +/** > > > > + * sgx_epc

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

> > > > > > > Later the hosting process could migrated/reassigned to another cgroup? > > > What to do when the new cgroup is OOM? > > > > > > > You addressed in the documentation, no? > > > > +Migration > > +- > > + > > +Once an EPC page is charged to a cgroup (during allocation), it

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Mon, 2023-10-09 at 20:04 -0500, Haitao Huang wrote: > On Mon, 09 Oct 2023 18:45:06 -0500, Huang, Kai wrote: > > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > From: Sean Christopherson > > > > > > Introduce the OOM path

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > From: Sean Christopherson > > Introduce the OOM path for killing an enclave with a reclaimer that is no > longer able to reclaim enough EPC pages. Find a victim enclave, which > will be an enclave with only "unreclaimable" EPC pages left

Re: [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller

> +static inline struct sgx_epc_lru_lists *epc_cg_lru(struct sgx_epc_cgroup > *epc_cg) > +{ > + if (epc_cg) > + return _cg->lru; > + return NULL; > +} > It's legal to return NULL EPC cgroup for a given EPC page, i.e., when the enclave isn't assigned to any cgroup. But ...

Re: [PATCH v5 16/18] x86/sgx: Limit process EPC usage with misc cgroup controller

> @@ -332,6 +336,7 @@ void sgx_isolate_epc_pages(struct sgx_epc_lru_lists *lru, > size_t nr_to_scan, > * sgx_reclaim_epc_pages() - Reclaim EPC pages from the consumers > * @nr_to_scan: Number of EPC pages to scan for reclaim > * @ignore_age: Reclaim a page even

Re: [PATCH v5 12/18] x86/sgx: Add EPC OOM path to forcefully reclaim EPC

On Mon, 2023-10-09 at 20:42 -0500, Haitao Huang wrote: > Hi Sean > > On Mon, 09 Oct 2023 19:23:04 -0500, Sean Christopherson > wrote: > > > On Mon, Oct 09, 2023, Kai Huang wrote: > > > On Fri, 2023-09-22 at 20:06 -0700, Haitao Huang wrote: > > > > +/** > > > > + * sgx_epc_oom() - invoke EPC

Re: [PATCH v5 06/18] x86/sgx: Introduce EPC page states

On Wed, 2023-10-04 at 10:24 -0500, Haitao Huang wrote: > On Tue, 03 Oct 2023 15:03:48 -0500, Huang, Kai wrote: > > > On Mon, 2023-10-02 at 23:49 -0500, Haitao Huang wrote: > > > On Wed, 27 Sep 2023 05:28:36 -0500, Huang, Kai > > > wrote: > > > &g

Re: [PATCH v5 09/18] x86/sgx: Store struct sgx_encl when allocating new VA pages

On Wed, 2023-10-04 at 10:03 -0500, Haitao Huang wrote: > On Tue, 03 Oct 2023 15:07:42 -0500, Huang, Kai wrote: > > > On Tue, 2023-10-03 at 01:45 -0500, Haitao Huang wrote: > > > > > > > > Btw, probably a dumb question: > > > > > > > >

Re: [PATCH v9 07/15] x86/sgx: Expose sgx_reclaim_pages() for cgroup

On Mon, 2024-02-05 at 13:06 -0800, Haitao Huang wrote: > From: Sean Christopherson > > Each EPC cgroup will have an LRU structure to track reclaimable EPC pages. > When a cgroup usage reaches its limit, the cgroup needs to reclaim pages > from its LRU or LRUs of its descendants to make room for

Re: [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows for cgroup

> +/* > + * Get the lower bound of limits of a cgroup and its ancestors. Used in > + * sgx_epc_cgroup_reclaim_work_func() to determine if EPC usage of a cgroup > is > + * over its limit or its ancestors' hence reclamation is needed. > + */ > +static inline u64

Re: Re: [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows for cgroup

On Tue, 2024-02-20 at 14:18 +0100, Michal Koutný wrote: > On Tue, Feb 20, 2024 at 09:52:39AM +, "Huang, Kai" > wrote: > > I am not sure, but is it possible or legal for an ancestor to have less > > limit > > than children? > > Why not? > It i

Re: [PATCH v9 13/15] x86/sgx: Turn on per-cgroup EPC reclamation

On Mon, 2024-02-05 at 13:06 -0800, Haitao Huang wrote: > From: Kristen Carlson Accardi > > Previous patches have implemented all infrastructure needed for > per-cgroup EPC page tracking and reclaiming. But all reclaimable EPC > pages are still tracked in the global LRU as sgx_lru_list() returns

Re: [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows for cgroup

On Wed, 2024-02-21 at 00:44 -0600, Haitao Huang wrote: > [...] > > > > Here the @nr_to_scan is reduced by the number of pages that are > > isolated, but > > not actually reclaimed (which is reflected by @cnt). > > > > IIUC, looks you want to make this function do "each cycle" as what you > >

Re: [PATCH v9 08/15] x86/sgx: Implement EPC reclamation flows for cgroup

On Wed, 2024-02-21 at 00:23 -0600, Haitao Huang wrote: > StartHi Kai > On Tue, 20 Feb 2024 03:52:39 -0600, Huang, Kai wrote: > [...] > > > > So you introduced the work/workqueue here but there's no place which > > actually > > queues the work. IMHO you can

  1   2   >