Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-25 Thread Huang, Ying
Shaohua Li writes: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: >> Rik van Riel writes: >> >> > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> >. >> >> > - It will help the memory fragmentation, esp

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-25 Thread Minchan Kim
On Sun, Sep 25, 2016 at 12:18:49PM -0700, Shaohua Li wrote: > On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > > Rik van Riel writes: > > > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > >> >  > > >> >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-25 Thread Shaohua Li
On Fri, Sep 23, 2016 at 10:32:39AM +0800, Huang, Ying wrote: > Rik van Riel writes: > > > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >> >  > >> > - It will help the memory fragmentation, especially when the THP is > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-22 Thread Huang, Ying
Rik van Riel writes: > On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: >> On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >  >> > - It will help the memory fragmentation, especially when the THP is >> >   heavily used by the applications.  The 2M continuous pages will >> > be

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-22 Thread Huang, Ying
Hi, Shaohua, Thanks for comments! Shaohua Li writes: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> >> The advantages of the THP swap support include: Sorry for confusing. This is the advantages of the final goal, that is, avoid splitting/collapsing the THP during swap out

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-22 Thread Rik van Riel
On Thu, 2016-09-22 at 15:56 -0700, Shaohua Li wrote: > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > >  > > - It will help the memory fragmentation, especially when the THP is > >   heavily used by the applications.  The 2M continuous pages will > > be > >   free up after THP swapp

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-22 Thread Andi Kleen
"Chen, Tim C" writes: >> >>So this is impossible without THP swapin. While 2M swapout makes a lot of >>sense, I doubt 2M swapin is really useful. What kind of application is >>'optimized' >>to do sequential memory access? Anything that touches regions larger than 4K and we want to do the kernel

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-22 Thread Chen, Tim C
> >So this is impossible without THP swapin. While 2M swapout makes a lot of >sense, I doubt 2M swapin is really useful. What kind of application is >'optimized' >to do sequential memory access? We waste a lot of cpu cycles to re-compact 4K pages back to a large page under THP. Swapping it back

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-22 Thread Shaohua Li
On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > The advantages of the THP swap support include: > > - Batch the swap operations for the THP to reduce lock > acquiring/releasing, including allocating/freeing the swap space, > adding/deleting to/from the swap cache, and writing

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Minchan Kim
Hi Huang, On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: > Hi, Minchan, > > Minchan Kim writes: > > Hi Huang, > > > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Huang, Ying
Minchan Kim writes: > Hi Huang, > > On Tue, Sep 20, 2016 at 10:54:35AM +0800, Huang, Ying wrote: >> Hi, Minchan, >> >> Minchan Kim writes: >> > Hi Huang, >> > >> > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > On Tue, Sep 13, 2016 at 04:5

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Huang, Ying
Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrot

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Hugh Dickins
On Wed, 7 Sep 2016, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Minchan and Rik, could you help me to r

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Tim Chen
On Mon, 2016-09-19 at 16:11 +0900, Minchan Kim wrote: > Hi Tim, > > On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > > > > > > > > > > > > > > > > > - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > Yes, if you want, please give us how bad it is. > > > > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Minchan Kim
Hi Tim, On Tue, Sep 13, 2016 at 11:52:27PM +, Chen, Tim C wrote: > >> > >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > > > >Yes, if you want, please give us how bad it is. > > > > It could be pretty bad. In an experiment with THP turned on and we > enter swap, 50% of

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-19 Thread Minchan Kim
Hi Huang, On Sun, Sep 18, 2016 at 09:53:39AM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> >> Minchan Kim writes: > >> >> > >>

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-17 Thread Huang, Ying
Minchan Kim writes: > On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> >> Minchan Kim writes: >> >> >> >> > Hi Huang, >> >> > >> >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wr

RE: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-13 Thread Chen, Tim C
>> >> - Avoid CPU time for splitting, collapsing THP across swap out/in. > >Yes, if you want, please give us how bad it is. > It could be pretty bad. In an experiment with THP turned on and we enter swap, 50% of the cpu are spent in the page compaction path. So if we could deal with units of la

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-13 Thread Andrea Arcangeli
Hello, On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > I am glad to discuss my final goal, that is, swapping out/in the full > THP without splitting. Why I want to do that is copied as below, I think that is a fine objective. It wasn't implemented initially just to keep things sim

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-13 Thread Minchan Kim
On Tue, Sep 13, 2016 at 04:53:49PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > >> Minchan Kim writes: > >> > >> > Hi Huang, > >> > > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > >> > > >> > < snip > > >

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-13 Thread Huang, Ying
Minchan Kim writes: > On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: >> Minchan Kim writes: >> >> > Hi Huang, >> > >> > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: >> > >> > < snip > >> > >> >> >> Recently, the performance of the storage devices improved so fast th

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-13 Thread Minchan Kim
On Tue, Sep 13, 2016 at 02:40:00PM +0800, Huang, Ying wrote: > Minchan Kim writes: > > > Hi Huang, > > > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > > > < snip > > > > >> >> Recently, the performance of the storage devices improved so fast that > >> >> we cannot saturate th

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-12 Thread Huang, Ying
Minchan Kim writes: > Hi Huang, > > On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: > > < snip > > >> >> Recently, the performance of the storage devices improved so fast that >> >> we cannot saturate the disk bandwidth when do page swap out even on a >> >> high-end server machine.

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-12 Thread Minchan Kim
Hi Huang, On Fri, Sep 09, 2016 at 01:35:12PM -0700, Huang, Ying wrote: < snip > > >> Recently, the performance of the storage devices improved so fast that > >> we cannot saturate the disk bandwidth when do page swap out even on a > >> high-end server machine. Because the performance of the sto

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-09 Thread Huang, Ying
Hi, Minchan, Minchan Kim writes: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: >> From: Huang Ying >> >> This patchset is to optimize the performance of Transparent Huge Page >> (THP) swap. >> >> Hi, Andrew, could you help me to check whether the overall design i

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-09 Thread Tim Chen
On Fri, 2016-09-09 at 14:43 +0900, Minchan Kim wrote: > Hi Huang, > > On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > > > > From: Huang Ying > > > > This patchset is to optimize the performance of Transparent Huge Page > > (THP) swap. > > > > Hi, Andrew, could you help me to che

Re: [PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-08 Thread Minchan Kim
Hi Huang, On Wed, Sep 07, 2016 at 09:45:59AM -0700, Huang, Ying wrote: > From: Huang Ying > > This patchset is to optimize the performance of Transparent Huge Page > (THP) swap. > > Hi, Andrew, could you help me to check whether the overall design is > reasonable? > > Hi, Hugh, Shaohua, Mincha

[PATCH -v3 00/10] THP swap: Delay splitting THP during swapping out

2016-09-07 Thread Huang, Ying
From: Huang Ying This patchset is to optimize the performance of Transparent Huge Page (THP) swap. Hi, Andrew, could you help me to check whether the overall design is reasonable? Hi, Hugh, Shaohua, Minchan and Rik, could you help me to review the swap part of the patchset? Especially [01/10],