Re: [PATCH v41 09/24] x86/sgx: Add SGX page allocator functions

[Jarkko Sakkinen]

On Sat, Nov 14, 2020 at 05:32:56PM +0800, Hillf Danton wrote:
> On Fri, 13 Nov 2020 00:01:20 +0200 Jarkko Sakkinen wrote:
> > 
> > The previous patch initialized a simple SGX page allocator.  Add functions
> > for runtime allocation and free.
> > 
> > This allocator and its algorithms are as simple as it gets.  They do a
> > linear search across all EPC sections and find the first free page.  They
> > are not NUMA aware and only hand out individual pages.  The SGX hardware
> > does not support large pages, so something more complicated like a buddy
> > allocator is unwarranted.
> > 
> > The free function (sgx_free_epc_page()) implicitly calls ENCLS[EREMOVE],
> > which returns the page to the uninitialized state.  This ensures that the
> > page is ready for use at the next allocation.
> > 
> > Acked-by: Jethro Beekman <jet...@fortanix.com> # v40
> > # Co-developed-by: Sean Christopherson <sean.j.christopher...@intel.com>
> > # Signed-off-by: Sean Christopherson <sean.j.christopher...@intel.com>
> > Signed-off-by: Jarkko Sakkinen <jar...@kernel.org>
> > ---
> >  arch/x86/kernel/cpu/sgx/main.c | 65 ++++++++++++++++++++++++++++++++++
> >  arch/x86/kernel/cpu/sgx/sgx.h  |  3 ++
> >  2 files changed, 68 insertions(+)
> > 
> > diff --git a/arch/x86/kernel/cpu/sgx/main.c b/arch/x86/kernel/cpu/sgx/main.c
> > index 187a237eec38..2e53afc288a4 100644
> > --- a/arch/x86/kernel/cpu/sgx/main.c
> > +++ b/arch/x86/kernel/cpu/sgx/main.c
> > @@ -85,6 +85,71 @@ static bool __init sgx_page_reclaimer_init(void)
> >     return true;
> >  }
> >  
> > +static struct sgx_epc_page *__sgx_alloc_epc_page_from_section(struct 
> > sgx_epc_section *section)
> > +{
> > +   struct sgx_epc_page *page;
> > +
> > +   spin_lock(&section->lock);
> > +
> > +   if (list_empty(&section->page_list)) {
> > +           spin_unlock(&section->lock);
> > +           return NULL;
> > +   }
> > +
> > +   page = list_first_entry(&section->page_list, struct sgx_epc_page, list);
> > +   list_del_init(&page->list);
> > +
> > +   spin_unlock(&section->lock);
> > +   return page;
> > +}
> > +
> > +/**
> > + * __sgx_alloc_epc_page() - Allocate an EPC page
> > + *
> > + * Iterate through EPC sections and borrow a free EPC page to the caller. 
> > When a
> > + * page is no longer needed it must be released with sgx_free_epc_page().
> > + *
> > + * Return:
> > + *   an EPC page,
> > + *   -errno on error
> > + */
> > +struct sgx_epc_page *__sgx_alloc_epc_page(void)
> > +{
> > +   struct sgx_epc_section *section;
> > +   struct sgx_epc_page *page;
> > +   int i;
> > +
> > +   for (i = 0; i < sgx_nr_epc_sections; i++) {
> > +           section = &sgx_epc_sections[i];
> > +
> > +           page = __sgx_alloc_epc_page_from_section(section);
> > +           if (page)
> > +                   return page;
> > +   }
> > +
> > +   return ERR_PTR(-ENOMEM);
> > +}
> > +
> > +/**
> > + * sgx_free_epc_page() - Free an EPC page
> > + * @page:  an EPC page
> > + *
> > + * Call EREMOVE for an EPC page and insert it back to the list of free 
> > pages.
> > + */
> > +void sgx_free_epc_page(struct sgx_epc_page *page)
> > +{
> > +   struct sgx_epc_section *section = &sgx_epc_sections[page->section];
> > +   int ret;
> > +
> > +   ret = __eremove(sgx_get_epc_virt_addr(page));
> > +   if (WARN_ONCE(ret, "EREMOVE returned %d (0x%x)", ret, ret))
> > +           return;
> > +
> > +   spin_lock(&section->lock);
> > +   list_add_tail(&page->list, &section->page_list);
> 
> Add head in favor of allocators coming before LLC becomes cold.

Nice suggestion, thank you.

/Jarkko