Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
On Fri, Jul 26, 2019 at 02:16:20PM +0300, Konstantin Khlebnikov wrote: > On 26.07.2019 3:06, Joel Fernandes wrote: > > On Thu, Jul 25, 2019 at 11:15:53AM +0300, Konstantin Khlebnikov wrote: > > [snip] > > > > > > Thanks for bringing up the swapping corner case.. Perhaps we can > > > > > > improve > > > > > > the heap profiler to detect this by looking at bits 0-4 in pagemap. > > > > > > While it > > > > > > > > > > Yeb, that could work but it could add overhead again what you want to > > > > > remove? > > > > > Even, userspace should keep metadata to identify that page was > > > > > already swapped > > > > > in last period or newly swapped in new period. > > > > > > > > Yep. > > > Between samples page could be read from swap and swapped out back > > > multiple times. > > > For tracking this swap ptes could be marked with idle bit too. > > > I believe it's not so hard to find free bit for this. > > > > > > Refault\swapout will automatically clear this bit in pte even if > > > page goes nowhere stays if swap-cache. > > > > Could you clarify more about your idea? Do you mean swapout will clear the > > new > > idle swap-pte bit if the page was accessed just before the swapout? > > > Instead, I thought of using is_swap_pte() to detect if the PTE belong to a > > page that was swapped. And if so, then assume the page was idle. Sure we > > would miss data that the page was accessed before the swap out in the > > sampling window, however if the page was swapped out, then it is likely idle > > anyway. > > > I mean page might be in swap when you mark pages idle and > then been accessed and swapped back before second pass. > > I propose marking swap pte with idle bit which will be automatically > cleared by following swapin/swapout pair: > > page alloc -> install page pte > page swapout -> install swap entry in pte > mark vm idle -> set swap-idle bit in swap pte > access/swapin -> install page pte (clear page idle if set) > page swapout -> install swap entry in pte (without swap idle bit) > scan vm idle -> see swap entry without idle bit -> page has been accessed > since marking idle > > One bit in pte is enough for tracking. This does not needs any propagation for > idle bits between page and swap, or marking pages as idle in swap cache. Ok I see the case you are referring to now. This can be a follow-up patch to address the case, because.. the limitation you mentioned is also something inherrent in the (traditional) physical page_idle tracking if that were used. The reason being, after swapping, the PTE is not mapped to any page so there is nothing to mark as idle. So if the page gets swapped out and in in the meanwhile, then you would run into the same issue. But yes, we should certainly address it in the future. I just want to keep things simple at the moment. I will make a note about your suggestion but you are welcomed to write a patch for it on top of my patch. I am about to send another revision shortly for futhre review. thanks, - Joel
Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
On Mon, Jul 22, 2019 at 03:06:39PM -0700, Andrew Morton wrote:
[snip]
> > + *end = *start + count * BITS_PER_BYTE;
> > + if (*end > max_frame)
> > + *end = max_frame;
> > + return 0;
> > +}
> > +
> >
> > ...
> >
> > +static void add_page_idle_list(struct page *page,
> > + unsigned long addr, struct mm_walk *walk)
> > +{
> > + struct page *page_get;
> > + struct page_node *pn;
> > + int bit;
> > + unsigned long frames;
> > + struct page_idle_proc_priv *priv = walk->private;
> > + u64 *chunk = (u64 *)priv->buffer;
> > +
> > + if (priv->write) {
> > + /* Find whether this page was asked to be marked */
> > + frames = (addr - priv->start_addr) >> PAGE_SHIFT;
> > + bit = frames % BITMAP_CHUNK_BITS;
> > + chunk = [frames / BITMAP_CHUNK_BITS];
> > + if (((*chunk >> bit) & 1) == 0)
> > + return;
> > + }
> > +
> > + page_get = page_idle_get_page(page);
> > + if (!page_get)
> > + return;
> > +
> > + pn = kmalloc(sizeof(*pn), GFP_ATOMIC);
>
> I'm not liking this GFP_ATOMIC. If I'm reading the code correctly,
> userspace can ask for an arbitrarily large number of GFP_ATOMIC
> allocations by doing a large read. This can potentially exhaust page
> reserves which things like networking Rx interrupts need and can make
> this whole feature less reliable.
For the revision, I will pre-allocate the page nodes in advance so it does
not need to do this. Diff on top of this patch is below. Let me know any
comments, thanks.
Btw, I also dropped the idle_page_list_lock by putting the idle_page_list
list_head on the stack instead of heap.
---8<---
From: "Joel Fernandes (Google)"
Subject: [PATCH] mm/page_idle: Avoid need for GFP_ATOMIC
GFP_ATOMIC can harm allocations does by other allocations that are in
need of reserves and the like. Pre-allocate the nodes list so that
spinlocked region can just use it.
Suggested-by: Andrew Morton
Signed-off-by: Joel Fernandes (Google)
---
mm/page_idle.c | 19 +++
1 file changed, 15 insertions(+), 4 deletions(-)
diff --git a/mm/page_idle.c b/mm/page_idle.c
index 874a60c41fef..b9c790721f16 100644
--- a/mm/page_idle.c
+++ b/mm/page_idle.c
@@ -266,6 +266,10 @@ struct page_idle_proc_priv {
unsigned long start_addr;
char *buffer;
int write;
+
+ /* Pre-allocate and provide nodes to add_page_idle_list() */
+ struct page_node *page_nodes;
+ int cur_page_node;
};
static void add_page_idle_list(struct page *page,
@@ -291,10 +295,7 @@ static void add_page_idle_list(struct page *page,
if (!page_get)
return;
- pn = kmalloc(sizeof(*pn), GFP_ATOMIC);
- if (!pn)
- return;
-
+ pn = &(priv->page_nodes[priv->cur_page_node++]);
pn->page = page_get;
pn->addr = addr;
list_add(>list, _page_list);
@@ -379,6 +380,15 @@ ssize_t page_idle_proc_generic(struct file *file, char
__user *ubuff,
priv.buffer = buffer;
priv.start_addr = start_addr;
priv.write = write;
+
+ priv.cur_page_node = 0;
+ priv.page_nodes = kzalloc(sizeof(struct page_node) * (end_frame -
start_frame),
+ GFP_KERNEL);
+ if (!priv.page_nodes) {
+ ret = -ENOMEM;
+ goto out;
+ }
+
walk.private =
walk.mm = mm;
@@ -425,6 +435,7 @@ ssize_t page_idle_proc_generic(struct file *file, char
__user *ubuff,
ret = copy_to_user(ubuff, buffer, count);
up_read(>mmap_sem);
+ kfree(priv.page_nodes);
out:
kfree(buffer);
out_mmput:
--
2.22.0.657.g960e92d24f-goog
Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
On Wed, Jul 24, 2019 at 01:28:42PM +0900, Minchan Kim wrote: > On Tue, Jul 23, 2019 at 10:20:49AM -0400, Joel Fernandes wrote: > > On Tue, Jul 23, 2019 at 03:13:58PM +0900, Minchan Kim wrote: > > > Hi Joel, > > > > > > On Mon, Jul 22, 2019 at 05:32:04PM -0400, Joel Fernandes (Google) wrote: > > > > The page_idle tracking feature currently requires looking up the pagemap > > > > for a process followed by interacting with /sys/kernel/mm/page_idle. > > > > This is quite cumbersome and can be error-prone too. If between > > > > > > cumbersome: That's the fair tradeoff between idle page tracking and > > > clear_refs because idle page tracking could check even though the page > > > is not mapped. > > > > It is fair tradeoff, but could be made simpler. The userspace code got > > reduced by a good amount as well. > > > > > error-prone: What's the error? > > > > We see in normal Android usage, that some of the times pages appear not to > > be > > idle even when they really are idle. Reproducing this is a bit unpredictable > > and happens at random occasions. With this new interface, we are seeing this > > happen much much lesser. > > I don't know how you did test. Maybe that could be contributed by > swapping out or shared pages touched by other processes or some kernel > behavior not to keep access bit of their operation. It could be something along these lines is my thinking as well. So we know its already has issues due to what you mentioned, I am not sure what else needs investigation? > Please investigate more what's the root cause. That would be important > point to justify for the patch motivation. The motivation is security. I am dropping the 'accuracy' factor I mentioned from the patch description since it created a lot of confusion. > > > > More over looking up PFN from pagemap in Android devices is not > > > > supported by unprivileged process and requires SYS_ADMIN and gives 0 for > > > > the PFN. > > > > > > > > This patch adds support to directly interact with page_idle tracking at > > > > the PID level by introducing a /proc//page_idle file. This > > > > eliminates the need for userspace to calculate the mapping of the page. > > > > It follows the exact same semantics as the global > > > > /sys/kernel/mm/page_idle, however it is easier to use for some usecases > > > > where looking up PFN is not needed and also does not require SYS_ADMIN. > > > > > > Ah, so the primary goal is to provide convinience interface and it would > > > help accurary, too. IOW, accuracy is not your main goal? > > > > There are a couple of primary goals: Security, conveience and also solving > > the accuracy/reliability problem we are seeing. Do keep in mind looking up > > PFN has security implications. The PFN field in pagemap is zeroed if the > > user > > does not have CAP_SYS_ADMIN. > > Myaybe you don't need PFN. is it? With the traditional idle tracking, PFN is needed which has the mentioned security issues. This patch solves it. And the interface is identical and familiar to the existing page_idle bitmap interface. > > > > In Android, we are using this for the heap profiler (heapprofd) which > > > > profiles and pin points code paths which allocates and leaves memory > > > > idle for long periods of time. > > > > > > So the goal is to detect idle pages with idle memory tracking? > > > > Isn't that what idle memory tracking does? > > To me, it's rather misleading. Please read motivation section in document. > The feature would be good to detect workingset pages, not idle pages > because workingset pages are never freed, swapped out and even we could > count on newly allocated pages. > > Motivation > == > > The idle page tracking feature allows to track which memory pages are being > accessed by a workload and which are idle. This information can be useful for > estimating the workload's working set size, which, in turn, can be taken into > account when configuring the workload parameters, setting memory cgroup > limits, > or deciding where to place the workload within a compute cluster. As we discussed by chat, we could collect additional metadata to check if pages were swapped or freed ever since the time we marked them as idle. However this can be incremental improvement. > > > It couldn't work well because such idle pages could finally swap out and > > > lose every flags of the page descriptor which is working mechanism of > > > idle page tracking. It should have named "workingset page tracking", > > > not "idle page tracking". > > > > The heap profiler that uses page-idle tracking is not to measure working > > set, > > but to look for pages that are idle for long periods of time. > > It's important part. Please include it in the description so that people > understands what's the usecase. As I said above, if it aims for finding > idle pages durting the period, current idle page tracking feature is not > good ironically. Ok, I will mention. > > Thanks for bringing up the swapping
Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
Hi Joel,
On Mon, Jul 22, 2019 at 05:32:04PM -0400, Joel Fernandes (Google) wrote:
> The page_idle tracking feature currently requires looking up the pagemap
> for a process followed by interacting with /sys/kernel/mm/page_idle.
> This is quite cumbersome and can be error-prone too. If between
cumbersome: That's the fair tradeoff between idle page tracking and
clear_refs because idle page tracking could check even though the page
is not mapped.
error-prone: What's the error?
> accessing the per-PID pagemap and the global page_idle bitmap, if
> something changes with the page then the information is not accurate.
What you mean with error is this timing issue?
Why do you need to be accurate? IOW, accurate is always good but what's
the scale of the accuracy?
> More over looking up PFN from pagemap in Android devices is not
> supported by unprivileged process and requires SYS_ADMIN and gives 0 for
> the PFN.
>
> This patch adds support to directly interact with page_idle tracking at
> the PID level by introducing a /proc//page_idle file. This
> eliminates the need for userspace to calculate the mapping of the page.
> It follows the exact same semantics as the global
> /sys/kernel/mm/page_idle, however it is easier to use for some usecases
> where looking up PFN is not needed and also does not require SYS_ADMIN.
Ah, so the primary goal is to provide convinience interface and it would
help accurary, too. IOW, accuracy is not your main goal?
> It ended up simplifying userspace code, solving the security issue
> mentioned and works quite well. SELinux does not need to be turned off
> since no pagemap look up is needed.
I'm not sure how it is painful to check it via pagemap for your goal
but not sure it's a good idea to create new ABI for just convinience.
I think that's library we have.
>
> In Android, we are using this for the heap profiler (heapprofd) which
> profiles and pin points code paths which allocates and leaves memory
> idle for long periods of time.
So the goal is to detect idle pages with idle memory tracking?
It couldn't work well because such idle pages could finally swap out and
lose every flags of the page descriptor which is working mechanism of
idle page tracking. It should have named "workingset page tracking",
not "idle page tracking".
>
> Documentation material:
> The idle page tracking API for virtual address indexing using virtual page
> frame numbers (VFN) is located at /proc//page_idle. It is a bitmap
> that follows the same semantics as /sys/kernel/mm/page_idle/bitmap
> except that it uses virtual instead of physical frame numbers.
>
> This idle page tracking API can be simpler to use than physical address
> indexing, since the pagemap for a process does not need to be looked up
> to mark or read a page's idle bit. It is also more accurate than
> physical address indexing since in physical address indexing, address
> space changes can occur between reading the pagemap and reading the
> bitmap. In virtual address indexing, the process's mmap_sem is held for
> the duration of the access.
>
> Cc: vdavydov@gmail.com
> Cc: Brendan Gregg
> Cc: kernel-t...@android.com
> Signed-off-by: Joel Fernandes (Google)
>
> ---
> Internal review -> v1:
> Fixes from Suren.
> Corrections to change log, docs (Florian, Sandeep)
>
> fs/proc/base.c| 3 +
> fs/proc/internal.h| 1 +
> fs/proc/task_mmu.c| 57 +++
> include/linux/page_idle.h | 4 +
> mm/page_idle.c| 305 +-
> 5 files changed, 330 insertions(+), 40 deletions(-)
>
> diff --git a/fs/proc/base.c b/fs/proc/base.c
> index 77eb628ecc7f..a58dd74606e9 100644
> --- a/fs/proc/base.c
> +++ b/fs/proc/base.c
> @@ -3021,6 +3021,9 @@ static const struct pid_entry tgid_base_stuff[] = {
> REG("smaps", S_IRUGO, proc_pid_smaps_operations),
> REG("smaps_rollup", S_IRUGO, proc_pid_smaps_rollup_operations),
> REG("pagemap",S_IRUSR, proc_pagemap_operations),
> +#ifdef CONFIG_IDLE_PAGE_TRACKING
> + REG("page_idle", S_IRUSR|S_IWUSR, proc_page_idle_operations),
> +#endif
> #endif
> #ifdef CONFIG_SECURITY
> DIR("attr", S_IRUGO|S_IXUGO, proc_attr_dir_inode_operations,
> proc_attr_dir_operations),
> diff --git a/fs/proc/internal.h b/fs/proc/internal.h
> index cd0c8d5ce9a1..bc9371880c63 100644
> --- a/fs/proc/internal.h
> +++ b/fs/proc/internal.h
> @@ -293,6 +293,7 @@ extern const struct file_operations
> proc_pid_smaps_operations;
> extern const struct file_operations proc_pid_smaps_rollup_operations;
> extern const struct file_operations proc_clear_refs_operations;
> extern const struct file_operations proc_pagemap_operations;
> +extern const struct file_operations proc_page_idle_operations;
>
> extern unsigned long task_vsize(struct mm_struct *);
> extern unsigned long task_statm(struct mm_struct *,
> diff --git a/fs/proc/task_mmu.c b/fs/proc/task_mmu.c
> index 4d2b860dbc3f..11ccc53da38e 100644
> ---
Re: [PATCH v1 1/2] mm/page_idle: Add support for per-pid page_idle using virtual indexing
On Mon, 22 Jul 2019 17:32:04 -0400 "Joel Fernandes (Google)"
wrote:
> The page_idle tracking feature currently requires looking up the pagemap
> for a process followed by interacting with /sys/kernel/mm/page_idle.
> This is quite cumbersome and can be error-prone too. If between
> accessing the per-PID pagemap and the global page_idle bitmap, if
> something changes with the page then the information is not accurate.
Well, it's never going to be "accurate" - something could change one
nanosecond after userspace has read the data...
Presumably with this approach the data will be "more" accurate. How
big a problem has this inaccuracy proven to be in real-world usage?
> More over looking up PFN from pagemap in Android devices is not
> supported by unprivileged process and requires SYS_ADMIN and gives 0 for
> the PFN.
>
> This patch adds support to directly interact with page_idle tracking at
> the PID level by introducing a /proc//page_idle file. This
> eliminates the need for userspace to calculate the mapping of the page.
> It follows the exact same semantics as the global
> /sys/kernel/mm/page_idle, however it is easier to use for some usecases
> where looking up PFN is not needed and also does not require SYS_ADMIN.
> It ended up simplifying userspace code, solving the security issue
> mentioned and works quite well. SELinux does not need to be turned off
> since no pagemap look up is needed.
>
> In Android, we are using this for the heap profiler (heapprofd) which
> profiles and pin points code paths which allocates and leaves memory
> idle for long periods of time.
>
> Documentation material:
> The idle page tracking API for virtual address indexing using virtual page
> frame numbers (VFN) is located at /proc//page_idle. It is a bitmap
> that follows the same semantics as /sys/kernel/mm/page_idle/bitmap
> except that it uses virtual instead of physical frame numbers.
>
> This idle page tracking API can be simpler to use than physical address
> indexing, since the pagemap for a process does not need to be looked up
> to mark or read a page's idle bit. It is also more accurate than
> physical address indexing since in physical address indexing, address
> space changes can occur between reading the pagemap and reading the
> bitmap. In virtual address indexing, the process's mmap_sem is held for
> the duration of the access.
>
> ...
>
> --- a/mm/page_idle.c
> +++ b/mm/page_idle.c
> @@ -11,6 +11,7 @@
> #include
> #include
> #include
> +#include
>
> #define BITMAP_CHUNK_SIZEsizeof(u64)
> #define BITMAP_CHUNK_BITS(BITMAP_CHUNK_SIZE * BITS_PER_BYTE)
> @@ -28,15 +29,12 @@
> *
> * This function tries to get a user memory page by pfn as described above.
> */
Above comment needs updating or moving?
> -static struct page *page_idle_get_page(unsigned long pfn)
> +static struct page *page_idle_get_page(struct page *page_in)
> {
> struct page *page;
> pg_data_t *pgdat;
>
> - if (!pfn_valid(pfn))
> - return NULL;
> -
> - page = pfn_to_page(pfn);
> + page = page_in;
> if (!page || !PageLRU(page) ||
> !get_page_unless_zero(page))
> return NULL;
>
> ...
>
> +static int page_idle_get_frames(loff_t pos, size_t count, struct mm_struct
> *mm,
> + unsigned long *start, unsigned long *end)
> +{
> + unsigned long max_frame;
> +
> + /* If an mm is not given, assume we want physical frames */
> + max_frame = mm ? (mm->task_size >> PAGE_SHIFT) : max_pfn;
> +
> + if (pos % BITMAP_CHUNK_SIZE || count % BITMAP_CHUNK_SIZE)
> + return -EINVAL;
> +
> + *start = pos * BITS_PER_BYTE;
> + if (*start >= max_frame)
> + return -ENXIO;
Is said to mean "The system tried to use the device represented by a
file you specified, and it couldnt find the device. This can mean that
the device file was installed incorrectly, or that the physical device
is missing or not correctly attached to the computer."
This doesn't seem appropriate in this usage and is hence possibly
misleading. Someone whose application fails with ENXIO will be
scratching their heads.
> + *end = *start + count * BITS_PER_BYTE;
> + if (*end > max_frame)
> + *end = max_frame;
> + return 0;
> +}
> +
>
> ...
>
> +static void add_page_idle_list(struct page *page,
> +unsigned long addr, struct mm_walk *walk)
> +{
> + struct page *page_get;
> + struct page_node *pn;
> + int bit;
> + unsigned long frames;
> + struct page_idle_proc_priv *priv = walk->private;
> + u64 *chunk = (u64 *)priv->buffer;
> +
> + if (priv->write) {
> + /* Find whether this page was asked to be marked */
> + frames = (addr - priv->start_addr) >> PAGE_SHIFT;
> + bit = frames % BITMAP_CHUNK_BITS;
> + chunk = [frames / BITMAP_CHUNK_BITS];
> + if (((*chunk >> bit) & 1) == 0)
> +
