Re: btrfs-transacti hammering the system

2017-12-04 Thread Austin S. Hemmelgarn
On 2017-12-04 09:10, Duncan wrote: Austin S. Hemmelgarn posted on Mon, 04 Dec 2017 07:18:11 -0500 as excerpted: On 2017-12-01 16:50, Matt McKinnon wrote: Well, it's at zero now... # btrfs fi df /export/ Data, single: total=30.45TiB, used=30.25TiB System, DUP: total=32.00MiB, used=3.62MiB Meta

Re: btrfs-transacti hammering the system

2017-12-04 Thread Duncan
Austin S. Hemmelgarn posted on Mon, 04 Dec 2017 07:18:11 -0500 as excerpted: > On 2017-12-01 16:50, Matt McKinnon wrote: >> Well, it's at zero now... >> >> # btrfs fi df /export/ >> Data, single: total=30.45TiB, used=30.25TiB >> System, DUP: total=32.00MiB, used=3.62MiB >> Metadata, DUP: total=66

Re: btrfs-transacti hammering the system

2017-12-04 Thread Austin S. Hemmelgarn
On 2017-12-01 16:50, Matt McKinnon wrote: Well, it's at zero now... # btrfs fi df /export/ Data, single: total=30.45TiB, used=30.25TiB System, DUP: total=32.00MiB, used=3.62MiB Metadata, DUP: total=66.50GiB, used=65.16GiB GlobalReserve, single: total=512.00MiB, used=0.00B GlobalReserve seems to

Re: btrfs-transacti hammering the system

2017-12-02 Thread Andrei Borzenkov
01.12.2017 21:04, Austin S. Hemmelgarn пишет: > On 2017-12-01 12:13, Andrei Borzenkov wrote: >> 01.12.2017 20:06, Hans van Kranenburg пишет: >>> >>> Additional tips (forgot to ask for your /proc/mounts before): >>> * Use the noatime mount option, so that only accessing files does not >>> lead to ch

Re: btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
Well, it's at zero now... # btrfs fi df /export/ Data, single: total=30.45TiB, used=30.25TiB System, DUP: total=32.00MiB, used=3.62MiB Metadata, DUP: total=66.50GiB, used=65.16GiB GlobalReserve, single: total=512.00MiB, used=0.00B On 01/12/17 16:47, Duncan wrote: Hans van Kranenburg posted on

Re: btrfs-transacti hammering the system

2017-12-01 Thread Duncan
Hans van Kranenburg posted on Fri, 01 Dec 2017 18:06:23 +0100 as excerpted: > On 12/01/2017 05:31 PM, Matt McKinnon wrote: >> Sorry, I missed your in-line reply: >> >> >>> 2) How big is this filesystem? What does your `btrfs fi df >>> /mountpoint` say? >>> >> >> # btrfs fi df /export/ >> Data,

Re: btrfs-transacti hammering the system

2017-12-01 Thread Chris Murphy
On Fri, Dec 1, 2017 at 12:07 PM, Matt McKinnon wrote: > Right. The file system is 48T, with 17T available, so we're not quite > pushing it yet. > > So far so good on the space_cache=v2 mount. I'm surprised this isn't on the > gotcha page in the wiki; it may end up making a world of difference to

Re: btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
Right. The file system is 48T, with 17T available, so we're not quite pushing it yet. So far so good on the space_cache=v2 mount. I'm surprised this isn't on the gotcha page in the wiki; it may end up making a world of difference to the users here Thanks again, Matt On 01/12/17 13:24, Han

Re: btrfs-transacti hammering the system

2017-12-01 Thread Hans van Kranenburg
On 12/01/2017 06:57 PM, Holger Hoffstätte wrote: > On 12/01/17 18:34, Matt McKinnon wrote: >> Thanks, I'll give space_cache=v2 a shot. > > Yes, very much recommended. > >> My mount options are: rw,relatime,space_cache,autodefrag,subvolid=5,subvol=/ > > Turn autodefrag off and use noatime instead

Re: btrfs-transacti hammering the system

2017-12-01 Thread Austin S. Hemmelgarn
On 2017-12-01 12:13, Andrei Borzenkov wrote: 01.12.2017 20:06, Hans van Kranenburg пишет: Additional tips (forgot to ask for your /proc/mounts before): * Use the noatime mount option, so that only accessing files does not lead to changes in metadata, Is not 'lazytime" default today? It gives

Re: btrfs-transacti hammering the system

2017-12-01 Thread Holger Hoffstätte
On 12/01/17 18:34, Matt McKinnon wrote: > Thanks, I'll give space_cache=v2 a shot. Yes, very much recommended. > My mount options are: rw,relatime,space_cache,autodefrag,subvolid=5,subvol=/ Turn autodefrag off and use noatime instead of relatime. Your filesystem also seems very full, that's bad

Re: btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
Thanks, I'll give space_cache=v2 a shot. My mount options are: rw,relatime,space_cache,autodefrag,subvolid=5,subvol=/ -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majo

Re: btrfs-transacti hammering the system

2017-12-01 Thread Andrei Borzenkov
01.12.2017 20:06, Hans van Kranenburg пишет: > > Additional tips (forgot to ask for your /proc/mounts before): > * Use the noatime mount option, so that only accessing files does not > lead to changes in metadata, Is not 'lazytime" default today? It gives you correct atime + no extra metadata upd

Re: btrfs-transacti hammering the system

2017-12-01 Thread Hans van Kranenburg
On 12/01/2017 05:31 PM, Matt McKinnon wrote: > Sorry, I missed your in-line reply: > > >> 1) The one right above, btrfs_write_out_cache, is the write-out of the >> free space cache v1. Do you see this for multiple seconds going on, and >> does it match the time when it's writing X MB/s to disk? >

Re: btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
Sorry, I missed your in-line reply: 1) The one right above, btrfs_write_out_cache, is the write-out of the free space cache v1. Do you see this for multiple seconds going on, and does it match the time when it's writing X MB/s to disk? It seems to only last until the next watch update. [] i

Re: btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
These seem to come up most often: [] transaction_kthread+0x133/0x1c0 [btrfs] [] kthread+0x109/0x140 [] ret_from_fork+0x25/0x30 -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kern

Re: btrfs-transacti hammering the system

2017-12-01 Thread Hans van Kranenburg
On 12/01/2017 04:24 PM, Matt McKinnon wrote: > Thanks for this.  Here's what I get: Ok, and which one is displaying most of the time? > [...] > > [] io_schedule+0x16/0x40 > [] get_request+0x23e/0x720 > [] blk_queue_bio+0xc1/0x3a0 > [] generic_make_request+0xf8/0x2a0 > [] submit_bio+0x75/0x150 >

Re: btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
Thanks for this. Here's what I get: [] transaction_kthread+0x133/0x1c0 [btrfs] [] kthread+0x109/0x140 [] ret_from_fork+0x25/0x30 ... [] io_schedule+0x16/0x40 [] get_request+0x23e/0x720 [] blk_queue_bio+0xc1/0x3a0 [] generic_make_request+0xf8/0x2a0 [] submit_bio+0x75/0x150 [] btrfs_map_bio+0xe

Re: btrfs-transacti hammering the system

2017-12-01 Thread Hans van Kranenburg
On 12/01/2017 03:25 PM, Matt McKinnon wrote: > > Is there any way to figure out what exactly btrfs-transacti is chugging > on?  I have a few file systems that seem to get wedged for days on end > with this process pegged around 100%.  I've stopped all snapshots, made > sure no quotas were enabled,

btrfs-transacti hammering the system

2017-12-01 Thread Matt McKinnon
Hi All, Is there any way to figure out what exactly btrfs-transacti is chugging on? I have a few file systems that seem to get wedged for days on end with this process pegged around 100%. I've stopped all snapshots, made sure no quotas were enabled, turned on autodefrag in the mount options,