Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-28 Thread Anshuman Khandual
On 05/21/2019 04:04 PM, Michal Hocko wrote: > On Tue 21-05-19 08:25:55, Anshuman Khandual wrote: >> On 05/20/2019 10:29 PM, Tim Murray wrote: > [...] >>> not seem to introduce a noticeable hot start penalty, not does it >>> cause an increase in performance problems later in the app's >>> lifecyc

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-27 Thread Minchan Kim
On Thu, May 23, 2019 at 10:07:17PM +0900, Minchan Kim wrote: > On Wed, May 22, 2019 at 09:01:33AM -0700, Daniel Colascione wrote: > > On Wed, May 22, 2019 at 9:01 AM Christian Brauner > > wrote: > > > > > > On Wed, May 22, 2019 at 08:57:47AM -0700, Daniel Colascione wrote: > > > > On Wed, May 22,

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-23 Thread Minchan Kim
On Wed, May 22, 2019 at 09:01:33AM -0700, Daniel Colascione wrote: > On Wed, May 22, 2019 at 9:01 AM Christian Brauner > wrote: > > > > On Wed, May 22, 2019 at 08:57:47AM -0700, Daniel Colascione wrote: > > > On Wed, May 22, 2019 at 8:48 AM Christian Brauner > > > wrote: > > > > > > > > On Wed,

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Daniel Colascione
On Wed, May 22, 2019 at 9:01 AM Christian Brauner wrote: > > On Wed, May 22, 2019 at 08:57:47AM -0700, Daniel Colascione wrote: > > On Wed, May 22, 2019 at 8:48 AM Christian Brauner > > wrote: > > > > > > On Wed, May 22, 2019 at 08:17:23AM -0700, Daniel Colascione wrote: > > > > On Wed, May 22,

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Christian Brauner
On Wed, May 22, 2019 at 08:57:47AM -0700, Daniel Colascione wrote: > On Wed, May 22, 2019 at 8:48 AM Christian Brauner > wrote: > > > > On Wed, May 22, 2019 at 08:17:23AM -0700, Daniel Colascione wrote: > > > On Wed, May 22, 2019 at 7:52 AM Christian Brauner > > > wrote: > > > > I'm not going t

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Daniel Colascione
On Wed, May 22, 2019 at 8:48 AM Christian Brauner wrote: > > On Wed, May 22, 2019 at 08:17:23AM -0700, Daniel Colascione wrote: > > On Wed, May 22, 2019 at 7:52 AM Christian Brauner > > wrote: > > > I'm not going to go into yet another long argument. I prefer pidfd_*. > > > > Ok. We're each allo

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Christian Brauner
On Wed, May 22, 2019 at 08:17:23AM -0700, Daniel Colascione wrote: > On Wed, May 22, 2019 at 7:52 AM Christian Brauner > wrote: > > I'm not going to go into yet another long argument. I prefer pidfd_*. > > Ok. We're each allowed our opinion. > > > It's tied to the api, transparent for userspace

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Daniel Colascione
On Wed, May 22, 2019 at 7:52 AM Christian Brauner wrote: > I'm not going to go into yet another long argument. I prefer pidfd_*. Ok. We're each allowed our opinion. > It's tied to the api, transparent for userspace, and disambiguates it > from process_vm_{read,write}v that both take a pid_t. Sp

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Christian Brauner
On Wed, May 22, 2019 at 06:16:35AM -0700, Daniel Colascione wrote: > On Wed, May 22, 2019 at 1:22 AM Christian Brauner > wrote: > > > > On Wed, May 22, 2019 at 7:12 AM Daniel Colascione wrote: > > > > > > On Tue, May 21, 2019 at 4:39 AM Christian Brauner > > > wrote: > > > > > > > > On Tue, Ma

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Daniel Colascione
On Wed, May 22, 2019 at 1:22 AM Christian Brauner wrote: > > On Wed, May 22, 2019 at 7:12 AM Daniel Colascione wrote: > > > > On Tue, May 21, 2019 at 4:39 AM Christian Brauner > > wrote: > > > > > > On Tue, May 21, 2019 at 01:30:29PM +0200, Christian Brauner wrote: > > > > On Tue, May 21, 2019

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-22 Thread Christian Brauner
On Wed, May 22, 2019 at 7:12 AM Daniel Colascione wrote: > > On Tue, May 21, 2019 at 4:39 AM Christian Brauner > wrote: > > > > On Tue, May 21, 2019 at 01:30:29PM +0200, Christian Brauner wrote: > > > On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: > > > > On Tue, May 21, 2019 at 10

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Daniel Colascione
On Tue, May 21, 2019 at 4:39 AM Christian Brauner wrote: > > On Tue, May 21, 2019 at 01:30:29PM +0200, Christian Brauner wrote: > > On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: > > > On Tue, May 21, 2019 at 10:42:00AM +0200, Christian Brauner wrote: > > > > On Mon, May 20, 2019 at

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Brian Geffon
To expand on the ChromeOS use case we're in a very similar situation to Android. For example, the Chrome browser uses a separate process for each individual tab (with some exceptions) and over time many tabs remain open in a back-grounded or idle state. Given that we have a lot of information about

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Shakeel Butt
On Mon, May 20, 2019 at 7:55 PM Anshuman Khandual wrote: > > > > On 05/20/2019 10:29 PM, Tim Murray wrote: > > On Sun, May 19, 2019 at 11:37 PM Anshuman Khandual > > wrote: > >> > >> Or Is the objective here is reduce the number of processes which get > >> killed by > >> lmkd by triggering swapp

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Shakeel Butt
On Sun, May 19, 2019 at 8:53 PM Minchan Kim wrote: > > - Background > > The Android terminology used for forking a new process and starting an app > from scratch is a cold start, while resuming an existing app is a hot start. > While we continually try to improve the performance of cold starts, ho

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Oleksandr Natalenko
On Tue, May 21, 2019 at 02:04:00PM +0200, Christian Brauner wrote: > On May 21, 2019 1:41:20 PM GMT+02:00, Minchan Kim wrote: > >On Tue, May 21, 2019 at 01:30:32PM +0200, Christian Brauner wrote: > >> On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: > >> > On Tue, May 21, 2019 at 10:42

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Christian Brauner
On May 21, 2019 1:41:20 PM GMT+02:00, Minchan Kim wrote: >On Tue, May 21, 2019 at 01:30:32PM +0200, Christian Brauner wrote: >> On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: >> > On Tue, May 21, 2019 at 10:42:00AM +0200, Christian Brauner wrote: >> > > On Mon, May 20, 2019 at 12:52:

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Minchan Kim
On Tue, May 21, 2019 at 01:30:32PM +0200, Christian Brauner wrote: > On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: > > On Tue, May 21, 2019 at 10:42:00AM +0200, Christian Brauner wrote: > > > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > > > - Background > > > > >

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Christian Brauner
On Tue, May 21, 2019 at 01:30:29PM +0200, Christian Brauner wrote: > On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: > > On Tue, May 21, 2019 at 10:42:00AM +0200, Christian Brauner wrote: > > > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > > > - Background > > > > >

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Christian Brauner
On Tue, May 21, 2019 at 08:05:52PM +0900, Minchan Kim wrote: > On Tue, May 21, 2019 at 10:42:00AM +0200, Christian Brauner wrote: > > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > > - Background > > > > > > The Android terminology used for forking a new process and starting an a

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Minchan Kim
On Tue, May 21, 2019 at 10:42:00AM +0200, Christian Brauner wrote: > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > - Background > > > > The Android terminology used for forking a new process and starting an app > > from scratch is a cold start, while resuming an existing app is

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Michal Hocko
On Tue 21-05-19 08:25:55, Anshuman Khandual wrote: > On 05/20/2019 10:29 PM, Tim Murray wrote: [...] > > not seem to introduce a noticeable hot start penalty, not does it > > cause an increase in performance problems later in the app's > > lifecycle. I've measured with and without process_madvise,

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-21 Thread Christian Brauner
On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > - Background > > The Android terminology used for forking a new process and starting an app > from scratch is a cold start, while resuming an existing app is a hot start. > While we continually try to improve the performance of cold st

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Michal Hocko
[linux-api] On Mon 20-05-19 18:44:52, Matthew Wilcox wrote: > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > IMHO we should spell it out that this patchset complements MADV_WONTNEED > > and MADV_FREE by adding non-destructive ways to gain some free memory > > space. MADV_COLD is

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Michal Hocko
[Cc linux-api] On Tue 21-05-19 13:39:50, Minchan Kim wrote: > On Mon, May 20, 2019 at 12:46:05PM -0400, Johannes Weiner wrote: > > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > > - Approach > > > > > > The approach we chose was to use a new interface to allow userspace to > > >

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Minchan Kim
On Tue, May 21, 2019 at 08:25:55AM +0530, Anshuman Khandual wrote: > > > On 05/20/2019 10:29 PM, Tim Murray wrote: > > On Sun, May 19, 2019 at 11:37 PM Anshuman Khandual > > wrote: > >> > >> Or Is the objective here is reduce the number of processes which get > >> killed by > >> lmkd by trigger

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Minchan Kim
On Mon, May 20, 2019 at 06:44:52PM -0700, Matthew Wilcox wrote: > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > IMHO we should spell it out that this patchset complements MADV_WONTNEED > > and MADV_FREE by adding non-destructive ways to gain some free memory > > space. MADV_COLD

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Minchan Kim
On Mon, May 20, 2019 at 12:46:05PM -0400, Johannes Weiner wrote: > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > - Approach > > > > The approach we chose was to use a new interface to allow userspace to > > proactively reclaim entire processes by leveraging platform information.

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Minchan Kim
On Mon, May 20, 2019 at 04:42:00PM +0200, Oleksandr Natalenko wrote: > Hi. > > On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > > - Background > > > > The Android terminology used for forking a new process and starting an app > > from scratch is a cold start, while resuming an exist

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Anshuman Khandual
On 05/20/2019 10:29 PM, Tim Murray wrote: > On Sun, May 19, 2019 at 11:37 PM Anshuman Khandual > wrote: >> >> Or Is the objective here is reduce the number of processes which get killed >> by >> lmkd by triggering swapping for the unused memory (user hinted) sooner so >> that >> they dont get

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Matthew Wilcox
On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > IMHO we should spell it out that this patchset complements MADV_WONTNEED > and MADV_FREE by adding non-destructive ways to gain some free memory > space. MADV_COLD is similar to MADV_WONTNEED in a way that it hints the > kernel that mem

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Tim Murray
On Sun, May 19, 2019 at 11:37 PM Anshuman Khandual wrote: > > Or Is the objective here is reduce the number of processes which get killed by > lmkd by triggering swapping for the unused memory (user hinted) sooner so that > they dont get picked by lmkd. Under utilization for zram hardware is a con

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Johannes Weiner
On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > - Approach > > The approach we chose was to use a new interface to allow userspace to > proactively reclaim entire processes by leveraging platform information. > This allowed us to bypass the inaccuracy of the kernel’s LRUs for pages

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Oleksandr Natalenko
Hi. On Mon, May 20, 2019 at 12:52:47PM +0900, Minchan Kim wrote: > - Background > > The Android terminology used for forking a new process and starting an app > from scratch is a cold start, while resuming an existing app is a hot start. > While we continually try to improve the performance of co

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-20 Thread Michal Hocko
[Cc linux-api] On Mon 20-05-19 12:52:47, Minchan Kim wrote: > - Background > > The Android terminology used for forking a new process and starting an app > from scratch is a cold start, while resuming an existing app is a hot start. > While we continually try to improve the performance of cold st

Re: [RFC 0/7] introduce memory hinting API for external process

2019-05-19 Thread Anshuman Khandual
On 05/20/2019 09:22 AM, Minchan Kim wrote: > - Problem > > Naturally, cached apps were dominant consumers of memory on the system. > However, they were not significant consumers of swap even though they are > good candidate for swap. Under investigation, swapping out only begins > once the low

[RFC 0/7] introduce memory hinting API for external process

2019-05-19 Thread Minchan Kim
- Background The Android terminology used for forking a new process and starting an app from scratch is a cold start, while resuming an existing app is a hot start. While we continually try to improve the performance of cold starts, hot starts will always be significantly less power hungry as well