kernel BUG at fs/btrfs/ctree.h:2417

2014-06-15 Thread Tobias Hunger
I keep running into this bug on a 2.14.3 kernel from arch linux. It seems to be triggered by docker removing a machine. Docker is using the btrfs backend here, so doing so involves some btrfs subvolume magic (or so I guess:-). Jun 14 22:18:05 ron kernel: [ cut here ] Jun

Re: kernel BUG at fs/btrfs/ctree.h:2417

2014-06-15 Thread Tobias Hunger
Some more information: [root@ron ~]# uname -a Linux ron 3.14.3-2-ARCH #1 SMP PREEMPT Mon May 12 20:52:20 CEST 2014 x86_64 GNU/Linux [root@ron ~]# btrfsck /dev/mapper/raid0d0 Checking filesystem on /dev/mapper/raid0d0 UUID: 18d8b702-5519-40dc-ab14-c425 checking extents read block failed

Re: Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Kai Krakow
Duncan 1i5t5.dun...@cox.net schrieb: Back to the extents counts: What I did next was implementing a defrag job that regularly defrags the journal (actually, the complete log directory as other log files suffer the same problem): $ cat /usr/local/sbin/defrag-logs.rb #!/bin/sh exec btrfs

Re: Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Martin Steigerwald
Am Samstag, 14. Juni 2014, 02:53:20 schrieb Duncan: I am reaching the conclusion that fallocate is not the problem. The fallocate increase the filesize of about 8MB, which is enough for some logging. So it is not called very often. But... If a file isn't (properly[1]) set NOCOW (and

Re: Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Hugo Mills
On Sun, Jun 15, 2014 at 11:31:07PM +0200, Martin Steigerwald wrote: Am Samstag, 14. Juni 2014, 02:53:20 schrieb Duncan: I am reaching the conclusion that fallocate is not the problem. The fallocate increase the filesize of about 8MB, which is enough for some logging. So it is not called

Re: Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Martin Steigerwald
Am Samstag, 14. Juni 2014, 12:59:31 schrieb Kai Krakow: Well, how did I accomblish that? Setting no cow and defragmenting regularily? Quite a complex setup for a casual Linux user. Any solution should be automatic. I´d suggest by a combination of sane application behaviour and measures within

Re: Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Hugo Mills
On Sun, Jun 15, 2014 at 11:45:37PM +0200, Martin Steigerwald wrote: Am Samstag, 14. Juni 2014, 12:59:31 schrieb Kai Krakow: Well, how did I accomblish that? Setting no cow and defragmenting regularily? Quite a complex setup for a casual Linux user. Any solution should be automatic. I´d

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Lennart Poettering
On Sat, 14.06.14 09:52, Goffredo Baroncelli (kreij...@libero.it) wrote: Which effectively means that by the time the 8 MiB is filled, each 4 KiB block has been rewritten to a new location and is now an extent unto itself. So now that 8 MiB is composed of 2048 new extents, each one a

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Lennart Poettering
On Wed, 11.06.14 20:32, Chris Murphy (li...@colorremedies.com) wrote: systemd has a very stupid journal write pattern. It checks if there is space in the file for the write, and if not it fallocates the small amount of space it needs (it does *4 byte* fallocate calls!) Not really the case.

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Lennart Poettering
On Sun, 15.06.14 05:43, Duncan (1i5t5.dun...@cox.net) wrote: The base problem isn't fallocate per se, rather, tho it's the trigger in this case. The base problem is that for COW-based filesystems, *ANY* rewriting of existing file content results in fragmentation. It just so happens that

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Lennart Poettering
On Sat, 14.06.14 12:59, Kai Krakow (hurikha...@gmail.com) wrote: Duncan 1i5t5.dun...@cox.net schrieb: As they say, Whoosh! At least here, I interpreted that remark as primarily sarcastic commentary on the systemd devs' apparent attitude, which can be (controversially) summarized

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Russell Coker
On Mon, 16 Jun 2014 00:13:07 Lennart Poettering wrote: On Sat, 14.06.14 09:52, Goffredo Baroncelli (kreij...@libero.it) wrote: Which effectively means that by the time the 8 MiB is filled, each 4 KiB block has been rewritten to a new location and is now an extent unto itself. So now

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread John Williams
On Sun, Jun 15, 2014 at 5:17 PM, Russell Coker russ...@coker.com.au wrote: I just did some tests using fallocate(1). I did the tests both with and without the -n option which appeared to make no difference. I started by allocating a 24G file on a 106G filesystem that had 30G free according

Re: Transaction commit: none (default)

2014-06-15 Thread Wang Shilong
On 06/15/2014 12:05 AM, Marc MERLIN wrote: Taking snapshots is now output this in addition of the snapshot operation. Transaction commit: none (default) 1) Is that expected/normal? It looks kind of spamming/useless to me? 2) If it's useful, what's the use I'm not getting? Addressed a issue

Re: Transaction commit: none (default)

2014-06-15 Thread Marc MERLIN
On Mon, Jun 16, 2014 at 09:35:23AM +0800, Wang Shilong wrote: On 06/15/2014 12:05 AM, Marc MERLIN wrote: Taking snapshots is now output this in addition of the snapshot operation. Transaction commit: none (default) 1) Is that expected/normal? It looks kind of spamming/useless to me? 2)

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Russell Coker
On Sun, 15 Jun 2014 18:06:34 John Williams wrote: Why does it take 2 minutes? On XFS or ext4, fallocate is almost instantaneous, even for multi-Terabyte allocations. According the fallocate man page, preallocation should be quick and require no IO: fallocate is used to manipulate

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread Chris Murphy
On Jun 15, 2014, at 4:34 PM, Lennart Poettering lenn...@poettering.net wrote: On Wed, 11.06.14 20:32, Chris Murphy (li...@colorremedies.com) wrote: systemd has a very stupid journal write pattern. It checks if there is space in the file for the write, and if not it fallocates the small

Re: [systemd-devel] Slow startup of systemd-journal on BTRFS

2014-06-15 Thread cwillu
Fallocate is a red herring except insofar as it's a hint that btrfs isn't making much use of: you see the same behaviour with small writes to an mmap'ed file that's msync'ed after each write, and likewise with plain old appending small writes with an fsync after each write, with or without