Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Christoph Hellwig writes: > Hi Ying, > >> Any update to this regression? > > Not really. We've optimized everything we could in XFS without > dropping the architecture that we really want to move to. Now we're > waiting for some MM behavior to be fixed that this unconvered. But >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Christoph Hellwig writes: > Hi Ying, > >> Any update to this regression? > > Not really. We've optimized everything we could in XFS without > dropping the architecture that we really want to move to. Now we're > waiting for some MM behavior to be fixed that this unconvered. But > in the end

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi Ying, > Any update to this regression? Not really. We've optimized everything we could in XFS without dropping the architecture that we really want to move to. Now we're waiting for some MM behavior to be fixed that this unconvered. But in the end will probabkly stuck with a slight

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi Ying, > Any update to this regression? Not really. We've optimized everything we could in XFS without dropping the architecture that we really want to move to. Now we're waiting for some MM behavior to be fixed that this unconvered. But in the end will probabkly stuck with a slight

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Christoph, "Huang, Ying" writes: > Hi, Christoph, > > "Huang, Ying" writes: > >> Christoph Hellwig writes: >> >>> Snipping the long contest: >>> >>> I think there are three observations here: >>> >>> (1) removing the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Christoph, "Huang, Ying" writes: > Hi, Christoph, > > "Huang, Ying" writes: > >> Christoph Hellwig writes: >> >>> Snipping the long contest: >>> >>> I think there are three observations here: >>> >>> (1) removing the mark_page_accessed (which is the only significant >>> change in

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Mel Gorman writes: > On Fri, Sep 02, 2016 at 09:32:58AM +1000, Dave Chinner wrote: >> On Fri, Aug 19, 2016 at 04:08:34PM +0100, Mel Gorman wrote: >> > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: >> > > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Mel Gorman writes: > On Fri, Sep 02, 2016 at 09:32:58AM +1000, Dave Chinner wrote: >> On Fri, Aug 19, 2016 at 04:08:34PM +0100, Mel Gorman wrote: >> > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: >> > > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: >> > > > On Wed,

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Sep 02, 2016 at 09:32:58AM +1000, Dave Chinner wrote: > On Fri, Aug 19, 2016 at 04:08:34PM +0100, Mel Gorman wrote: > > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > > > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > > > On Wed, Aug 17, 2016 at 04:49:07PM

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Sep 02, 2016 at 09:32:58AM +1000, Dave Chinner wrote: > On Fri, Aug 19, 2016 at 04:08:34PM +0100, Mel Gorman wrote: > > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > > > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > > > On Wed, Aug 17, 2016 at 04:49:07PM

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Aug 19, 2016 at 04:08:34PM +0100, Mel Gorman wrote: > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > > > Yes, we could try to batch the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Aug 19, 2016 at 04:08:34PM +0100, Mel Gorman wrote: > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > > > Yes, we could try to batch the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed, Aug 24, 2016 at 08:40:37AM -0700, Huang, Ying wrote: > Mel Gorman writes: > > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > >> > Yes, we could try to batch the locking like DaveC already suggested > >> > (ie we could move the locking to the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed, Aug 24, 2016 at 08:40:37AM -0700, Huang, Ying wrote: > Mel Gorman writes: > > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > >> > Yes, we could try to batch the locking like DaveC already suggested > >> > (ie we could move the locking to the caller, and then make > >> >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Mel, Mel Gorman writes: > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: >> > Yes, we could try to batch the locking like DaveC already suggested >> > (ie we could move the locking to the caller, and then make >> > shrink_page_list() just try to

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Mel, Mel Gorman writes: > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: >> > Yes, we could try to batch the locking like DaveC already suggested >> > (ie we could move the locking to the caller, and then make >> > shrink_page_list() just try to keep the lock held for a few

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Christoph, "Huang, Ying" writes: > Christoph Hellwig writes: > >> Snipping the long contest: >> >> I think there are three observations here: >> >> (1) removing the mark_page_accessed (which is the only significant >> change in the parent commit)

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Christoph, "Huang, Ying" writes: > Christoph Hellwig writes: > >> Snipping the long contest: >> >> I think there are three observations here: >> >> (1) removing the mark_page_accessed (which is the only significant >> change in the parent commit) hurts the >>

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Sat, Aug 20, 2016 at 09:48:39AM +1000, Dave Chinner wrote: > On Fri, Aug 19, 2016 at 11:49:46AM +0100, Mel Gorman wrote: > > On Thu, Aug 18, 2016 at 03:25:40PM -0700, Linus Torvalds wrote: > > > It *could* be as simple/stupid as just saying "let's allocate the page > > > cache for new pages

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Sat, Aug 20, 2016 at 09:48:39AM +1000, Dave Chinner wrote: > On Fri, Aug 19, 2016 at 11:49:46AM +0100, Mel Gorman wrote: > > On Thu, Aug 18, 2016 at 03:25:40PM -0700, Linus Torvalds wrote: > > > It *could* be as simple/stupid as just saying "let's allocate the page > > > cache for new pages

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Aug 19, 2016 at 4:48 PM, Dave Chinner wrote: > > Well, it depends on the speed of the storage. The higher the speed > of the storage, the less we care about stalling on dirty pages > during reclaim Actually, that's largely true independently of the speed of the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Aug 19, 2016 at 4:48 PM, Dave Chinner wrote: > > Well, it depends on the speed of the storage. The higher the speed > of the storage, the less we care about stalling on dirty pages > during reclaim Actually, that's largely true independently of the speed of the storage, I feel. On

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Aug 19, 2016 at 11:49:46AM +0100, Mel Gorman wrote: > On Thu, Aug 18, 2016 at 03:25:40PM -0700, Linus Torvalds wrote: > > It *could* be as simple/stupid as just saying "let's allocate the page > > cache for new pages from the current node" - and if the process that > > dirties pages just

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Fri, Aug 19, 2016 at 11:49:46AM +0100, Mel Gorman wrote: > On Thu, Aug 18, 2016 at 03:25:40PM -0700, Linus Torvalds wrote: > > It *could* be as simple/stupid as just saying "let's allocate the page > > cache for new pages from the current node" - and if the process that > > dirties pages just

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > > Yes, we could try to batch the locking like DaveC already suggested > > > > (ie we could move the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > > Yes, we could try to batch the locking like DaveC already suggested > > > > (ie we could move the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 03:25:40PM -0700, Linus Torvalds wrote: > >> In fact, looking at the __page_cache_alloc(), we already have that > >> "spread pages out" logic. I'm assuming Dave doesn't actually have that > >> bit set (I don't think it's the default), but I'm also envisioning > >> that

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 03:25:40PM -0700, Linus Torvalds wrote: > >> In fact, looking at the __page_cache_alloc(), we already have that > >> "spread pages out" logic. I'm assuming Dave doesn't actually have that > >> bit set (I don't think it's the default), but I'm also envisioning > >> that

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu 18-08-16 15:25:40, Linus Torvalds wrote: [...] > So just for testing purposes, you could try changing that > > return alloc_pages(gfp, 0); > > in __page_cache_alloc() into something like > > return alloc_pages_node(cpu_to_node(raw_smp_processor_id())), gfp, 0); That

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu 18-08-16 15:25:40, Linus Torvalds wrote: [...] > So just for testing purposes, you could try changing that > > return alloc_pages(gfp, 0); > > in __page_cache_alloc() into something like > > return alloc_pages_node(cpu_to_node(raw_smp_processor_id())), gfp, 0); That

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 10:55:01AM -0700, Linus Torvalds wrote: > On Thu, Aug 18, 2016 at 6:24 AM, Mel Gorman > wrote: > > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > >> FWIW, I just remembered about /proc/sys/vm/zone_reclaim_mode. > >> > > > >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 10:55:01AM -0700, Linus Torvalds wrote: > On Thu, Aug 18, 2016 at 6:24 AM, Mel Gorman > wrote: > > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > >> FWIW, I just remembered about /proc/sys/vm/zone_reclaim_mode. > >> > > > > That is a terrifying "fix" for

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 2:19 PM, Dave Chinner wrote: > > For streaming or use-once IO it makes a lot of sense to restrict the > locality of the page cache. The faster the IO device, the less dirty > page buffering we need to maintain full device bandwidth. And the > larger

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 2:19 PM, Dave Chinner wrote: > > For streaming or use-once IO it makes a lot of sense to restrict the > locality of the page cache. The faster the IO device, the less dirty > page buffering we need to maintain full device bandwidth. And the > larger the machine the greater

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 6:24 AM, Mel Gorman wrote: > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: >> FWIW, I just remembered about /proc/sys/vm/zone_reclaim_mode. >> > > That is a terrifying "fix" for this problem. It just happens to work > because

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 6:24 AM, Mel Gorman wrote: > On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: >> FWIW, I just remembered about /proc/sys/vm/zone_reclaim_mode. >> > > That is a terrifying "fix" for this problem. It just happens to work > because there is no spillover to other

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > > Yes, we could try to batch the locking like DaveC already suggested > > > > (ie we could move the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 05:11:11PM +1000, Dave Chinner wrote: > On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > > Yes, we could try to batch the locking like DaveC already suggested > > > > (ie we could move the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > Yes, we could try to batch the locking like DaveC already suggested > > > (ie we could move the locking to the caller, and then make > > > shrink_page_list() just try to

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Thu, Aug 18, 2016 at 01:45:17AM +0100, Mel Gorman wrote: > On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > > Yes, we could try to batch the locking like DaveC already suggested > > > (ie we could move the locking to the caller, and then make > > > shrink_page_list() just try to

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > On Tue, Aug 16, 2016 at 10:47:36AM -0700, Linus Torvalds wrote: > > I've always preferred to see direct reclaim as the primary model for > > reclaim, partly in order to throttle the actual "bad" process, but > > also because "kswapd

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > On Tue, Aug 16, 2016 at 10:47:36AM -0700, Linus Torvalds wrote: > > I've always preferred to see direct reclaim as the primary model for > > reclaim, partly in order to throttle the actual "bad" process, but > > also because "kswapd

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > Yes, we could try to batch the locking like DaveC already suggested > > (ie we could move the locking to the caller, and then make > > shrink_page_list() just try to keep the lock held for a few pages if > > the mapping doesn't

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed, Aug 17, 2016 at 04:49:07PM +0100, Mel Gorman wrote: > > Yes, we could try to batch the locking like DaveC already suggested > > (ie we could move the locking to the caller, and then make > > shrink_page_list() just try to keep the lock held for a few pages if > > the mapping doesn't

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed 17-08-16 17:48:25, Michal Hocko wrote: [...] > I will try to catch up with the rest of the email thread but from a > quick glance it just feels like we are doing more more work under the > lock. Hmm, so it doesn't seem to be more work in __remove_mapping as pointed out in

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Wed 17-08-16 17:48:25, Michal Hocko wrote: [...] > I will try to catch up with the rest of the email thread but from a > quick glance it just feels like we are doing more more work under the > lock. Hmm, so it doesn't seem to be more work in __remove_mapping as pointed out in

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 07:03:00AM +0200, Ingo Molnar wrote: > > * Linus Torvalds wrote: > > > Make sure you actually use "perf record -e cycles:pp" or something > > that uses PEBS to get real profiles using CPU performance counters. > > Btw., 'perf record -e

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 07:03:00AM +0200, Ingo Molnar wrote: > > * Linus Torvalds wrote: > > > Make sure you actually use "perf record -e cycles:pp" or something > > that uses PEBS to get real profiles using CPU performance counters. > > Btw., 'perf record -e cycles:pp' is the default now for

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 10:47:36AM -0700, Linus Torvalds wrote: > I've always preferred to see direct reclaim as the primary model for > reclaim, partly in order to throttle the actual "bad" process, but > also because "kswapd uses lots of CPU time" is such a nasty thing to > even begin guessing

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 10:47:36AM -0700, Linus Torvalds wrote: > I've always preferred to see direct reclaim as the primary model for > reclaim, partly in order to throttle the actual "bad" process, but > also because "kswapd uses lots of CPU time" is such a nasty thing to > even begin guessing

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue 16-08-16 10:47:36, Linus Torvalds wrote: > Mel, > thanks for taking a look. Your theory sounds more complete than mine, > and since Dave is able to see the problem with 4.7, it would be nice > to hear about the 4.6 behavior and commit ede37713737 in particular. > > That one seems more

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue 16-08-16 10:47:36, Linus Torvalds wrote: > Mel, > thanks for taking a look. Your theory sounds more complete than mine, > and since Dave is able to see the problem with 4.7, it would be nice > to hear about the 4.6 behavior and commit ede37713737 in particular. > > That one seems more

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 3:02 PM, Dave Chinner wrote: >> >> What does your profile show for when you actually dig into >> __remove_mapping() itself?, Looking at your flat profile, I'm assuming >> you get > > - 22.26% 0.93% [kernel] [k] __remove_mapping >-

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 3:02 PM, Dave Chinner wrote: >> >> What does your profile show for when you actually dig into >> __remove_mapping() itself?, Looking at your flat profile, I'm assuming >> you get > > - 22.26% 0.93% [kernel] [k] __remove_mapping >- 3.86% __remove_mapping

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 06:51:42PM -0700, Linus Torvalds wrote: > Anyway, including the direct reclaim call paths gets > __remove_mapping() a bit higher, and _raw_spin_lock_irqsave climbs to > 0.26%. But perhaps more importlantly, looking at what __remove_mapping > actually *does* (apart from the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 06:51:42PM -0700, Linus Torvalds wrote: > Anyway, including the direct reclaim call paths gets > __remove_mapping() a bit higher, and _raw_spin_lock_irqsave climbs to > 0.26%. But perhaps more importlantly, looking at what __remove_mapping > actually *does* (apart from the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Mel, thanks for taking a look. Your theory sounds more complete than mine, and since Dave is able to see the problem with 4.7, it would be nice to hear about the 4.6 behavior and commit ede37713737 in particular. That one seems more likely to affect contention than the zone/node one I found

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Mel, thanks for taking a look. Your theory sounds more complete than mine, and since Dave is able to see the problem with 4.7, it would be nice to hear about the 4.6 behavior and commit ede37713737 in particular. That one seems more likely to affect contention than the zone/node one I found

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:48:36PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds > wrote: > > > > None of this code is all that new, which is annoying. This must have > > gone on forever, > > ... ooh. > > Wait, I take that back. >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:48:36PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds > wrote: > > > > None of this code is all that new, which is annoying. This must have > > gone on forever, > > ... ooh. > > Wait, I take that back. > > We actually have some very

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: Snipping the long contest: I think there are three observations here: (1) removing the mark_page_accessed (which is the only significant change in the parent commit) hurts the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: Snipping the long contest: I think there are three observations here: (1) removing the mark_page_accessed (which is the only significant change in the parent commit) hurts the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 07:22:40AM +1000, Dave Chinner wrote: On Mon, Aug 15, 2016 at 10:14:55PM +0800, Fengguang Wu wrote: Hi Christoph, On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: >Snipping the long contest: > >I think there are three observations here: > >(1) removing

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 07:22:40AM +1000, Dave Chinner wrote: On Mon, Aug 15, 2016 at 10:14:55PM +0800, Fengguang Wu wrote: Hi Christoph, On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: >Snipping the long contest: > >I think there are three observations here: > >(1) removing

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 5:19 PM, Dave Chinner wrote: > >> None of this code is all that new, which is annoying. This must have >> gone on forever, > > Yes, it has been. Just worse than I've notice before, probably > because of all the stuff put under the tree lock in the past

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 5:19 PM, Dave Chinner wrote: > >> None of this code is all that new, which is annoying. This must have >> gone on forever, > > Yes, it has been. Just worse than I've notice before, probably > because of all the stuff put under the tree lock in the past couple > of years.

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 5:38 PM, Dave Chinner wrote: > > Same in 4.7 (flat profile numbers climbed higher after this > snapshot was taken, as can be seen by the callgraph numbers): Ok, so it's not the zone-vs-node thing. It's just that nobody has looked at that load in

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 5:38 PM, Dave Chinner wrote: > > Same in 4.7 (flat profile numbers climbed higher after this > snapshot was taken, as can be seen by the callgraph numbers): Ok, so it's not the zone-vs-node thing. It's just that nobody has looked at that load in recent times. Where

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 5:17 PM, Dave Chinner wrote: > > Read the code, Linus? I am. It's how I came up with my current pet theory. But I don't actually have enough sane numbers to make it much more than a cute pet theory. It *might* explain why you see tons of kswap time

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 5:17 PM, Dave Chinner wrote: > > Read the code, Linus? I am. It's how I came up with my current pet theory. But I don't actually have enough sane numbers to make it much more than a cute pet theory. It *might* explain why you see tons of kswap time and bad lock

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:48:36PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds > wrote: > > > > None of this code is all that new, which is annoying. This must have > > gone on forever, > > ... ooh. > > Wait, I take that back. >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:48:36PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds > wrote: > > > > None of this code is all that new, which is annoying. This must have > > gone on forever, > > ... ooh. > > Wait, I take that back. > > We actually have some very

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:20:55PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 3:42 PM, Dave Chinner wrote: > > > > 31.18% [kernel] [k] __pv_queued_spin_lock_slowpath > >9.90% [kernel] [k] copy_user_generic_string > >3.65% [kernel] [k]

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:20:55PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 3:42 PM, Dave Chinner wrote: > > > > 31.18% [kernel] [k] __pv_queued_spin_lock_slowpath > >9.90% [kernel] [k] copy_user_generic_string > >3.65% [kernel] [k]

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 05:15:47PM -0700, Linus Torvalds wrote: > DaveC - does the spinlock contention go away if you just go back to > 4.7? If so, I think it's the new zone thing. But it would be good to > verify - maybe it's something entirely different and it goes back much > further. Same in

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 05:15:47PM -0700, Linus Torvalds wrote: > DaveC - does the spinlock contention go away if you just go back to > 4.7? If so, I think it's the new zone thing. But it would be good to > verify - maybe it's something entirely different and it goes back much > further. Same in

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:01:00PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 3:22 PM, Dave Chinner wrote: > > > > Right, but that does not make the profile data useless, > > Yes it does. Because it basically hides everything that happens inside > the lock, which

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 04:01:00PM -0700, Linus Torvalds wrote: > On Mon, Aug 15, 2016 at 3:22 PM, Dave Chinner wrote: > > > > Right, but that does not make the profile data useless, > > Yes it does. Because it basically hides everything that happens inside > the lock, which is what causes the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 10:22:43AM -0700, Huang, Ying wrote: > Hi, Chinner, > > Dave Chinner writes: > > > On Wed, Aug 10, 2016 at 06:00:24PM -0700, Linus Torvalds wrote: > >> On Wed, Aug 10, 2016 at 5:33 PM, Huang, Ying wrote: > >> > > >> > Here it

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 10:22:43AM -0700, Huang, Ying wrote: > Hi, Chinner, > > Dave Chinner writes: > > > On Wed, Aug 10, 2016 at 06:00:24PM -0700, Linus Torvalds wrote: > >> On Wed, Aug 10, 2016 at 5:33 PM, Huang, Ying wrote: > >> > > >> > Here it is, > >> > >> Thanks. > >> > >> Appended

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds wrote: > > But I'll try to see what happens > on my profile, even if I can't recreate the contention itself, just > trying to see what happens inside of that region. Yeah, since I run my machines on encrypted disks,

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds wrote: > > But I'll try to see what happens > on my profile, even if I can't recreate the contention itself, just > trying to see what happens inside of that region. Yeah, since I run my machines on encrypted disks, my profile shows 60% kthread,

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds wrote: > > None of this code is all that new, which is annoying. This must have > gone on forever, ... ooh. Wait, I take that back. We actually have some very recent changes that I didn't even think about that went

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 4:20 PM, Linus Torvalds wrote: > > None of this code is all that new, which is annoying. This must have > gone on forever, ... ooh. Wait, I take that back. We actually have some very recent changes that I didn't even think about that went into this very merge window.

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 3:42 PM, Dave Chinner wrote: > > 31.18% [kernel] [k] __pv_queued_spin_lock_slowpath >9.90% [kernel] [k] copy_user_generic_string >3.65% [kernel] [k] __raw_callee_save___pv_queued_spin_unlock >2.62% [kernel] [k]

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 3:42 PM, Dave Chinner wrote: > > 31.18% [kernel] [k] __pv_queued_spin_lock_slowpath >9.90% [kernel] [k] copy_user_generic_string >3.65% [kernel] [k] __raw_callee_save___pv_queued_spin_unlock >2.62% [kernel] [k] __block_commit_write.isra.29 >2.26%

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 3:22 PM, Dave Chinner wrote: > > Right, but that does not make the profile data useless, Yes it does. Because it basically hides everything that happens inside the lock, which is what causes the contention in the first place. So stop making inane and

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 3:22 PM, Dave Chinner wrote: > > Right, but that does not make the profile data useless, Yes it does. Because it basically hides everything that happens inside the lock, which is what causes the contention in the first place. So stop making inane and stupid arguments,

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 08:22:11AM +1000, Dave Chinner wrote: > On Sun, Aug 14, 2016 at 10:12:20PM -0700, Linus Torvalds wrote: > > On Aug 14, 2016 10:00 PM, "Dave Chinner" wrote: > > > > > > > What does it say if you annotate that _raw_spin_unlock_irqrestore() > > function?

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Tue, Aug 16, 2016 at 08:22:11AM +1000, Dave Chinner wrote: > On Sun, Aug 14, 2016 at 10:12:20PM -0700, Linus Torvalds wrote: > > On Aug 14, 2016 10:00 PM, "Dave Chinner" wrote: > > > > > > > What does it say if you annotate that _raw_spin_unlock_irqrestore() > > function? > > > > > >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Sun, Aug 14, 2016 at 10:12:20PM -0700, Linus Torvalds wrote: > On Aug 14, 2016 10:00 PM, "Dave Chinner" wrote: > > > > > What does it say if you annotate that _raw_spin_unlock_irqrestore() > function? > > > >¿ > >¿Disassembly of section load0: > >

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Sun, Aug 14, 2016 at 10:12:20PM -0700, Linus Torvalds wrote: > On Aug 14, 2016 10:00 PM, "Dave Chinner" wrote: > > > > > What does it say if you annotate that _raw_spin_unlock_irqrestore() > function? > > > >¿ > >¿Disassembly of section load0: > >¿ > >¿

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 10:14:55PM +0800, Fengguang Wu wrote: > Hi Christoph, > > On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: > >Snipping the long contest: > > > >I think there are three observations here: > > > >(1) removing the mark_page_accessed (which is the only

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

On Mon, Aug 15, 2016 at 10:14:55PM +0800, Fengguang Wu wrote: > Hi Christoph, > > On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: > >Snipping the long contest: > > > >I think there are three observations here: > > > >(1) removing the mark_page_accessed (which is the only

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Christoph Hellwig writes: > Snipping the long contest: > > I think there are three observations here: > > (1) removing the mark_page_accessed (which is the only significant > change in the parent commit) hurts the > aim7/1BRD_48G-xfs-disk_rr-3000-performance/ivb44 test.

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Christoph Hellwig writes: > Snipping the long contest: > > I think there are three observations here: > > (1) removing the mark_page_accessed (which is the only significant > change in the parent commit) hurts the > aim7/1BRD_48G-xfs-disk_rr-3000-performance/ivb44 test. > I'd

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Chinner, Dave Chinner writes: > On Wed, Aug 10, 2016 at 06:00:24PM -0700, Linus Torvalds wrote: >> On Wed, Aug 10, 2016 at 5:33 PM, Huang, Ying wrote: >> > >> > Here it is, >> >> Thanks. >> >> Appended is a munged "after" list, with the "before"

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi, Chinner, Dave Chinner writes: > On Wed, Aug 10, 2016 at 06:00:24PM -0700, Linus Torvalds wrote: >> On Wed, Aug 10, 2016 at 5:33 PM, Huang, Ying wrote: >> > >> > Here it is, >> >> Thanks. >> >> Appended is a munged "after" list, with the "before" values in >> parenthesis. It actually

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi Christoph, On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: Snipping the long contest: I think there are three observations here: (1) removing the mark_page_accessed (which is the only significant change in the parent commit) hurts the

Re: [LKP] [lkp] [xfs] 68a9f5e700: aim7.jobs-per-min -13.6% regression

Hi Christoph, On Sun, Aug 14, 2016 at 06:17:24PM +0200, Christoph Hellwig wrote: Snipping the long contest: I think there are three observations here: (1) removing the mark_page_accessed (which is the only significant change in the parent commit) hurts the

  1   2   3   >