Re: btrfs-transaction blocked for more than 120 seconds

2014-01-08 Thread Duncan
Marc MERLIN posted on Tue, 07 Jan 2014 19:22:58 -0800 as excerpted: On Fri, Jan 03, 2014 at 09:34:10PM +, Duncan wrote: IIRC someone also mentioned problems with autodefrag and an about 3/4 gig systemd journal. My gut feeling (IOW, *NOT* benchmarked!) is that double-digit MiB files

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-07 Thread Brendan Hide
On 2014/01/06 12:57 AM, Roman Mamedov wrote: Did you align your partitions to accommodate for the 4K sector of the EARS? I had, yes. I had to do a lot of research to get the array working optimally. I didn't need to repartition the spare so this carried over to its being used as an OS disk.

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-07 Thread Marc MERLIN
On Fri, Jan 03, 2014 at 09:34:10PM +, Duncan wrote: IIRC someone also mentioned problems with autodefrag and an about 3/4 gig systemd journal. My gut feeling (IOW, *NOT* benchmarked!) is that double- digit MiB files should /normally/ be fine, but somewhere in the lower triple digits,

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Marc MERLIN
On Fri, Jan 03, 2014 at 09:34:10PM +, Duncan wrote: Thank you for that tip, I had been unaware of it 'till now. This will make my virtualbox image directory much happier :) I think I said it, but it bears repeating. Once you set that attribute on the dir, you may want to move the

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 4, 2014, at 11:39 PM, Marc MERLIN m...@merlins.org wrote: Nope, I never posted anything until now. Hopefully you agree that it's not ok for btrfs/kernel to just kill my system for over 2H until I power it off before of defragging one file. I did hit a severe performance but if it's

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Jim Salter
On 01/05/2014 12:09 PM, Chris Murphy wrote: I haven't read anything so far indicating defrag applies to the VM container use case, rather nodatacow via xattr +C is the way to go. At least for now. Can you elaborate on the rationale behind database or VM binaries being set nodatacow? I

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Duncan
Jim Salter posted on Sun, 05 Jan 2014 12:54:44 -0500 as excerpted: On 01/05/2014 12:09 PM, Chris Murphy wrote: I haven't read anything so far indicating defrag applies to the VM container use case, rather nodatacow via xattr +C is the way to go. At least for now. Well, NOCOW from the get-go

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Dec 31, 2013, at 4:46 AM, Sulla su...@gmx.at wrote: Dear all! On my Ubuntu Server 13.10 I use a RAID5 blockdevice consisting of 3 WD20EARS Sulla is this md raid5? If so can you report the result from mdadm -D mddevice, I'm curious what the chunk size is. Thanks. Chris Murphy-- To

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 12:57 PM, Duncan 1i5t5.dun...@cox.net wrote: But I do very little snapshotting here, and as a result hadn't considered the knockon effect of 100K-plus extents in perhaps 1000 snapshots. I wonder if this is an issue with snapshot aware defrag? Some problems were fixed

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Sulla
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear Chris! Certainly: I have 3 HDDs, all of which WD20EARS. Originally I wanted to let btrfs handle all 3 devices directly without making partitions, but this was impossible, as at least /boot needed to be ext4, at least back then when I set up the

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Duncan
On Sun, 05 Jan 2014 08:42:46 -0500 Jim Salter j...@jrs-s.net wrote: On Jan 5, 2014 1:39 AM, Marc MERLIN m...@merlins.org wrote: On Fri, Jan 03, 2014 at 09:34:10PM +, Duncan wrote: Yes, I got that. That why I ran btrfs defrag on the files after that Why are you trying to defrag an

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Brendan Hide
On 2014/01/05 11:17 PM, Sulla wrote: Certainly: I have 3 HDDs, all of which WD20EARS. Maybe/maybe-not off-topic: Poor hardware performance, though not necessarily the root cause, can be a major factor with these errors. WD Greens (Reds too, for that matter) have poor non-sequential

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Roman Mamedov
On Mon, 06 Jan 2014 00:36:22 +0200 Brendan Hide bren...@swiftspirit.co.za wrote: I had 8x 1.5TB WD1500EARS drives in an mdRAID5 array. With it I had a single 250GB IDE disk for the OS. When the very old IDE disk inevitably died, I decided to use a spare 1.5TB drive for the OS. Performance

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 2:17 PM, Sulla su...@gmx.at wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear Chris! Certainly: I have 3 HDDs, all of which WD20EARS. These drives don't have a configurable SCT ERC, so you need to modify the SCSI block layer timeout: echo 120

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 4:48 PM, Chris Murphy li...@colorremedies.com wrote: On Jan 5, 2014, at 2:17 PM, Sulla su...@gmx.at wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Dear Chris! Certainly: I have 3 HDDs, all of which WD20EARS. These drives don't have a configurable SCT ERC,

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 3:36 PM, Brendan Hide bren...@swiftspirit.co.za wrote: WD Greens (Reds too, for that matter) have poor non-sequential performance. An educated guess I'd say there's a 15% chance this is a major factor to the problem and, perhaps, a 60% chance it is merely a small

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 5:15 PM, Chris Murphy li...@colorremedies.com wrote: On Jan 5, 2014, at 3:36 PM, Brendan Hide bren...@swiftspirit.co.za wrote: WD Greens (Reds too, for that matter) have poor non-sequential performance. An educated guess I'd say there's a 15% chance this is a major

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Sulla
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thanks Chris! Thanks for your support. echo 120 /sys/block/sdX/device/timeout timeout is 30 for my HDDs. I'm well aware that the WD green HDDs are not the perfect ones for servers, but they were cheaper - and quieter - than the black ones for

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 5:25 PM, Sulla su...@gmx.at wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Thanks Chris! Thanks for your support. echo 120 /sys/block/sdX/device/timeout timeout is 30 for my HDDs. I don't think those drives support a configurable time out; the Green hasn't

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-05 Thread Chris Murphy
On Jan 5, 2014, at 6:29 PM, Sulla su...@gmx.at wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Hi Chris! # sudo smartctl -l scterc /dev/sda tells me SCT Error Recovery Control command not supported you're right. the /sys/block/sdX/device/timeout file probably is useless then.

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-04 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/01/14 09:25, Marc MERLIN wrote: Is there even a reason for this not to become a default mount option in newer kernels? autodefrag can go insane because it is unbounded. For example I have a 4GB RAM system (3.12, no gui) that kept hanging. I

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-04 Thread Sulla
Oh gosh, I don't know what went wrong with my btrfs root filesystem, and I probably will never know, too: The sudo balance start / was running fine for about 4 or 5 hours, running at a system load of ~3 when balance status / told me the balancing was on its way and had completed 19 out of 23

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-03 Thread Duncan
Kai Krakow posted on Fri, 03 Jan 2014 02:24:01 +0100 as excerpted: Duncan 1i5t5.dun...@cox.net schrieb: But because a full balance rewrites everything anyway, it'll effectively defrag too. Is that really true? I thought it just rewrites each distinct extent and shuffels chunks around...

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-03 Thread Marc MERLIN
First, a big thank you for taking the time to post this very informative message. On Wed, Jan 01, 2014 at 12:37:42PM +, Duncan wrote: Apparently the way some distribution installation scripts work results in even a brand new installation being highly fragmented. =:^( If in addition they

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-03 Thread Duncan
Marc MERLIN posted on Fri, 03 Jan 2014 09:25:06 -0800 as excerpted: First, a big thank you for taking the time to post this very informative message. On Wed, Jan 01, 2014 at 12:37:42PM +, Duncan wrote: Apparently the way some distribution installation scripts work results in even a

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-02 Thread Duncan
Sulla posted on Wed, 01 Jan 2014 20:08:21 + as excerpted: Dear Duncan! Thanks very much for your exhaustive answer. Hm, I also thought of fragmentation. Alhtough I don't think this is really very likely, as my server doesn't serve things that likely cause fragmentation. It is a

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-02 Thread Kai Krakow
Duncan 1i5t5.dun...@cox.net schrieb: But because a full balance rewrites everything anyway, it'll effectively defrag too. Is that really true? I thought it just rewrites each distinct extent and shuffels chunks around... This would mean it does not merge extents together. Regards, Kai --

Re: btrfs-transaction blocked for more than 120 seconds

2014-01-01 Thread Sulla
Dear Duncan! Thanks very much for your exhaustive answer. Hm, I also thought of fragmentation. Alhtough I don't think this is really very likely, as my server doesn't serve things that likely cause fragmentation. It is a mailserver (but only maildir-format), fileserver for windows clients (huge

btrfs-transaction blocked for more than 120 seconds

2013-12-31 Thread Sulla
Dear all! On my Ubuntu Server 13.10 I use a RAID5 blockdevice consisting of 3 WD20EARS drives. On this I built a LVM and in this LVM I use quite normal partitions /, /home, SWAP (/boot resides on a RAID1.) and also a custom /data partition. Everything (except boot and swap) is on btrfs.