Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-06-26 Thread Lutz Vieweg
Testing the patch took much longer than I anticipated due to pre-4.1-kernels being "too risky" for use on our servers, but now it's done and I can say: This patch, as integrated in linux-4.1, has successfully removed the lags. Thanks! Regards, Lutz Vieweg On 04/22/2015 06:09 PM, Lutz Vieweg

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-25 Thread Filipe David Manana
On Fri, Apr 24, 2015 at 4:05 PM, Filipe David Manana wrote: > On Fri, Apr 24, 2015 at 2:55 PM, Chris Mason wrote: >> On 04/24/2015 09:43 AM, Filipe David Manana wrote: >>> On Fri, Apr 24, 2015 at 2:00 PM, Chris Mason wrote: >> Can you please bang on this and get a more reliable reproduction

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-24 Thread Filipe David Manana
On Fri, Apr 24, 2015 at 2:55 PM, Chris Mason wrote: > On 04/24/2015 09:43 AM, Filipe David Manana wrote: >> On Fri, Apr 24, 2015 at 2:00 PM, Chris Mason wrote: > >>> Can you please bang on this and get a more reliable reproduction? I'll >>> take a look. >> >> Not really that easy to get a more re

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-24 Thread Chris Mason
On 04/24/2015 09:43 AM, Filipe David Manana wrote: > On Fri, Apr 24, 2015 at 2:00 PM, Chris Mason wrote: >> Can you please bang on this and get a more reliable reproduction? I'll >> take a look. > > Not really that easy to get a more reliable reproducer - just run > fsstress with multiple proces

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-24 Thread Filipe David Manana
On Fri, Apr 24, 2015 at 2:00 PM, Chris Mason wrote: > On 04/24/2015 02:34 AM, Filipe David Manana wrote: >> On Thu, Apr 23, 2015 at 8:50 PM, Chris Mason wrote: >>> On 04/23/2015 03:43 PM, Filipe David Manana wrote: On Thu, Apr 23, 2015 at 4:48 PM, Filipe David Manana wrote: > On T

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-24 Thread Chris Mason
On Fri, Apr 24, 2015 at 07:34:43AM +0100, Filipe David Manana wrote: > There's also one list corruption I didn't get before and happened > while running fsstress (btrfs/078), apparently due to some race: > > [25590.799058] [ cut here ] > [25590.800204] WARNING: CPU: 3 PID:

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-24 Thread Chris Mason
On 04/24/2015 02:34 AM, Filipe David Manana wrote: > On Thu, Apr 23, 2015 at 8:50 PM, Chris Mason wrote: >> On 04/23/2015 03:43 PM, Filipe David Manana wrote: >>> On Thu, Apr 23, 2015 at 4:48 PM, Filipe David Manana >>> wrote: On Thu, Apr 23, 2015 at 4:17 PM, Chris Mason wrote: > On Th

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Filipe David Manana
On Thu, Apr 23, 2015 at 8:50 PM, Chris Mason wrote: > On 04/23/2015 03:43 PM, Filipe David Manana wrote: >> On Thu, Apr 23, 2015 at 4:48 PM, Filipe David Manana >> wrote: >>> On Thu, Apr 23, 2015 at 4:17 PM, Chris Mason wrote: On Thu, Apr 23, 2015 at 02:05:48PM +0100, Filipe David Manana w

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Chris Mason
On 04/23/2015 03:43 PM, Filipe David Manana wrote: > On Thu, Apr 23, 2015 at 4:48 PM, Filipe David Manana > wrote: >> On Thu, Apr 23, 2015 at 4:17 PM, Chris Mason wrote: >>> On Thu, Apr 23, 2015 at 02:05:48PM +0100, Filipe David Manana wrote: >> Trying the current integration-4.1 branch, I r

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Filipe David Manana
On Thu, Apr 23, 2015 at 4:48 PM, Filipe David Manana wrote: > On Thu, Apr 23, 2015 at 4:17 PM, Chris Mason wrote: >> On Thu, Apr 23, 2015 at 02:05:48PM +0100, Filipe David Manana wrote: >>> >> Trying the current integration-4.1 branch, I ran into the following >>> >> during xfstests/btrfs/049: >>

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Chris Mason
On 04/23/2015 12:34 PM, Holger Hoffstätte wrote: > kmem_cache_destroy btrfs_transaction: Slab cache still has objects Josef has a fix for this, but the fix was causing other problems. It's not a new bug, but we'll get it fixed up. -chris -- To unsubscribe from this list: send the line "unsubscr

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Holger Hoffstätte
On Wed, 22 Apr 2015 12:55:26 -0400, Chris Mason wrote: > Great to hear. I recommend just using my for-linus-4.1 branch, since it > has all the good things in one place. Even with the latest patch for xfstests/049 I reproducibly get: [ 405.052123] =

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Filipe David Manana
On Thu, Apr 23, 2015 at 4:17 PM, Chris Mason wrote: > On Thu, Apr 23, 2015 at 02:05:48PM +0100, Filipe David Manana wrote: >> >> Trying the current integration-4.1 branch, I ran into the following >> >> during xfstests/btrfs/049: >> >> >> > >> > Ugh, I must not be waiting correctly in one of the i

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Chris Mason
On Thu, Apr 23, 2015 at 02:05:48PM +0100, Filipe David Manana wrote: > >> Trying the current integration-4.1 branch, I ran into the following > >> during xfstests/btrfs/049: > >> > > > > Ugh, I must not be waiting correctly in one of the inode cache writeout > > sections. But I've run 049 a whole

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Filipe David Manana
On Thu, Apr 23, 2015 at 1:52 PM, Chris Mason wrote: > On 04/23/2015 08:45 AM, Filipe David Manana wrote: >> On Wed, Apr 22, 2015 at 5:55 PM, Chris Mason wrote: >>> On 04/22/2015 12:37 PM, Holger Hoffstätte wrote: On Wed, 22 Apr 2015 18:09:18 +0200, Lutz Vieweg wrote: > On 04/13/2015

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Chris Mason
On 04/23/2015 08:52 AM, Chris Mason wrote: > On 04/23/2015 08:45 AM, Filipe David Manana wrote: >> On Wed, Apr 22, 2015 at 5:55 PM, Chris Mason wrote: >>> On 04/22/2015 12:37 PM, Holger Hoffstätte wrote: On Wed, 22 Apr 2015 18:09:18 +0200, Lutz Vieweg wrote: > On 04/13/2015 09:52 PM,

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Chris Mason
On 04/23/2015 08:45 AM, Filipe David Manana wrote: > On Wed, Apr 22, 2015 at 5:55 PM, Chris Mason wrote: >> On 04/22/2015 12:37 PM, Holger Hoffstätte wrote: >>> On Wed, 22 Apr 2015 18:09:18 +0200, Lutz Vieweg wrote: >>> On 04/13/2015 09:52 PM, Chris Mason wrote: > Large filesystems with l

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-23 Thread Filipe David Manana
On Wed, Apr 22, 2015 at 5:55 PM, Chris Mason wrote: > On 04/22/2015 12:37 PM, Holger Hoffstätte wrote: >> On Wed, 22 Apr 2015 18:09:18 +0200, Lutz Vieweg wrote: >> >>> On 04/13/2015 09:52 PM, Chris Mason wrote: Large filesystems with lots of block groups can suffer long stalls during com

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-22 Thread Chris Mason
On 04/22/2015 12:37 PM, Holger Hoffstätte wrote: > On Wed, 22 Apr 2015 18:09:18 +0200, Lutz Vieweg wrote: > >> On 04/13/2015 09:52 PM, Chris Mason wrote: >>> Large filesystems with lots of block groups can suffer long stalls during >>> commit while we create and send down all of the block group ca

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-22 Thread Holger Hoffstätte
On Wed, 22 Apr 2015 18:09:18 +0200, Lutz Vieweg wrote: > On 04/13/2015 09:52 PM, Chris Mason wrote: >> Large filesystems with lots of block groups can suffer long stalls during >> commit while we create and send down all of the block group caches. The >> more blocks groups dirtied in a transactio

Re: [PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-22 Thread Lutz Vieweg
On 04/13/2015 09:52 PM, Chris Mason wrote: Large filesystems with lots of block groups can suffer long stalls during commit while we create and send down all of the block group caches. The more blocks groups dirtied in a transaction, the longer these stalls can be. Some workloads average 10 seco

[PATCH 0/4] btrfs: reduce block group cache writeout times during commit

2015-04-13 Thread Chris Mason
Large filesystems with lots of block groups can suffer long stalls during commit while we create and send down all of the block group caches. The more blocks groups dirtied in a transaction, the longer these stalls can be. Some workloads average 10 seconds per commit, but see peak times much highe