Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-09-03 Thread Khalid Aziz
On 9/2/19 2:02 AM, Michal Hocko wrote: > On Fri 30-08-19 15:35:06, Khalid Aziz wrote: > [...] >> - Kernel is not self-tuning and is dependent upon a userspace tool to >> perform well in a fundamental area of memory management. > > You keep bringing this up without an actual analysis of a wider ran

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-09-02 Thread Michal Hocko
On Fri 30-08-19 15:35:06, Khalid Aziz wrote: [...] > - Kernel is not self-tuning and is dependent upon a userspace tool to > perform well in a fundamental area of memory management. You keep bringing this up without an actual analysis of a wider range of workloads that would prove that the default

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-30 Thread Khalid Aziz
On 8/27/19 12:16 AM, Michal Hocko wrote: > On Tue 27-08-19 02:14:20, Bharath Vedartham wrote: >> Hi Michal, >> >> Here are some of my thoughts, >> On Wed, Aug 21, 2019 at 04:06:32PM +0200, Michal Hocko wrote: >>> On Thu 15-08-19 14:51:04, Khalid Aziz wrote: Hi Michal, The smarts for

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-28 Thread Michal Hocko
On Wed 28-08-19 18:39:22, Bharath Vedartham wrote: [...] > > Therefore I would like to shift the discussion towards existing APIs and > > whether they are suitable for such an advance auto-tuning. I haven't > > heard any arguments about missing pieces. > I understand your concern here. Just confirm

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-28 Thread Bharath Vedartham
Hi Michal, Thank you for spending your time on this. On Tue, Aug 27, 2019 at 08:16:06AM +0200, Michal Hocko wrote: > On Tue 27-08-19 02:14:20, Bharath Vedartham wrote: > > Hi Michal, > > > > Here are some of my thoughts, > > On Wed, Aug 21, 2019 at 04:06:32PM +0200, Michal Hocko wrote: > > > On Th

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-26 Thread Michal Hocko
On Tue 27-08-19 02:14:20, Bharath Vedartham wrote: > Hi Michal, > > Here are some of my thoughts, > On Wed, Aug 21, 2019 at 04:06:32PM +0200, Michal Hocko wrote: > > On Thu 15-08-19 14:51:04, Khalid Aziz wrote: > > > Hi Michal, > > > > > > The smarts for tuning these knobs can be implemented in u

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-26 Thread Bharath Vedartham
Hi Michal, Here are some of my thoughts, On Wed, Aug 21, 2019 at 04:06:32PM +0200, Michal Hocko wrote: > On Thu 15-08-19 14:51:04, Khalid Aziz wrote: > > Hi Michal, > > > > The smarts for tuning these knobs can be implemented in userspace and > > more knobs added to allow for what is missing toda

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-21 Thread Michal Hocko
On Thu 15-08-19 14:51:04, Khalid Aziz wrote: > Hi Michal, > > The smarts for tuning these knobs can be implemented in userspace and > more knobs added to allow for what is missing today, but we get back to > the same issue as before. That does nothing to make kernel self-tuning > and adds possibly

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-15 Thread Khalid Aziz
On 8/15/19 11:02 AM, Michal Hocko wrote: > On Thu 15-08-19 10:27:26, Khalid Aziz wrote: >> On 8/14/19 2:58 AM, Michal Hocko wrote: >>> On Tue 13-08-19 09:20:51, Khalid Aziz wrote: On 8/13/19 8:05 AM, Michal Hocko wrote: > On Mon 12-08-19 19:40:10, Khalid Aziz wrote: > [...] >> Patc

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-15 Thread Michal Hocko
On Thu 15-08-19 10:27:26, Khalid Aziz wrote: > On 8/14/19 2:58 AM, Michal Hocko wrote: > > On Tue 13-08-19 09:20:51, Khalid Aziz wrote: > >> On 8/13/19 8:05 AM, Michal Hocko wrote: > >>> On Mon 12-08-19 19:40:10, Khalid Aziz wrote: > >>> [...] > Patch 1 adds code to maintain a sliding lookback

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-15 Thread Khalid Aziz
On 8/14/19 2:58 AM, Michal Hocko wrote: > On Tue 13-08-19 09:20:51, Khalid Aziz wrote: >> On 8/13/19 8:05 AM, Michal Hocko wrote: >>> On Mon 12-08-19 19:40:10, Khalid Aziz wrote: >>> [...] Patch 1 adds code to maintain a sliding lookback window of (time, number of free pages) points which

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-14 Thread Michal Hocko
On Tue 13-08-19 09:20:51, Khalid Aziz wrote: > On 8/13/19 8:05 AM, Michal Hocko wrote: > > On Mon 12-08-19 19:40:10, Khalid Aziz wrote: > > [...] > >> Patch 1 adds code to maintain a sliding lookback window of (time, number > >> of free pages) points which can be updated continuously and adds code

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-13 Thread Khalid Aziz
On 8/13/19 8:05 AM, Michal Hocko wrote: > On Mon 12-08-19 19:40:10, Khalid Aziz wrote: > [...] >> Patch 1 adds code to maintain a sliding lookback window of (time, number >> of free pages) points which can be updated continuously and adds code to >> compute best fit line across these points. It als

Re: [RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-13 Thread Michal Hocko
On Mon 12-08-19 19:40:10, Khalid Aziz wrote: [...] > Patch 1 adds code to maintain a sliding lookback window of (time, number > of free pages) points which can be updated continuously and adds code to > compute best fit line across these points. It also adds code to use the > best fit lines to dete

[RFC PATCH 0/2] Add predictive memory reclamation and compaction

2019-08-12 Thread Khalid Aziz
Page reclamation and compaction is triggered in response to reaching low watermark. This makes reclamation/compaction reactive based upon a snapshot of the system at a point in time. When that point is reached, system is already suffering from free memory shortage and must now try to recover. Rec