Re: Triple parity and beyond

2013-11-22 Thread Russell Coker
On Thu, 21 Nov 2013 20:54:54 Adam Goryachev wrote: On a pure storage server, the CPU would normally have nothing to do, except a little interrupt handling, it is just shuffling bytes around. Of course, if you need RAID7.5 then you probably have a dedicated storage server, so I don't see the

Re: Triple parity and beyond

2013-11-22 Thread Stan Hoeppner
Hi David, On 11/21/2013 3:07 AM, David Brown wrote: On 21/11/13 02:28, Stan Hoeppner wrote: ... WRT rebuild times, once drives hit 20TB we're looking at 18 hours just to mirror a drive at full streaming bandwidth, assuming 300MB/s average--and that is probably being kind to the drive makers.

Re: Triple parity and beyond

2013-11-22 Thread Stan Hoeppner
On 11/21/2013 3:07 AM, David Brown wrote: For example, with 20 disks at 1 TB each, you can have: All correct, and these are maximum redundancies. Maximum: raid5 = 19TB, 1 disk redundancy raid6 = 18TB, 2 disk redundancy raid6.3 = 17TB, 3 disk redundancy raid6.4 = 16TB, 4 disk redundancy

Re: [PATCH 1/5] Btrfs: wake up the tasks that wait for the io earlier

2013-11-22 Thread Liu Bo
On Fri, Nov 22, 2013 at 03:28:32PM +0800, Miao Xie wrote: On Fri, 22 Nov 2013 12:30:40 +0800, Liu Bo wrote: On Thu, Nov 21, 2013 at 09:43:14PM +0800, Miao Xie wrote: The tasks that wait for the IO_DONE flag just care about the io of the dirty pages, so it is better to wake up them

Re: progs integration branch moved to master (new default leafsize)

2013-11-22 Thread Martin
On 21/11/13 23:37, Chris Mason wrote: Quoting Martin (2013-11-08 18:53:06) On 08/11/13 22:01, Chris Mason wrote: Hi everyone, This patch is now the tip of the master branch for btrfs-progs, which has been updated to include most of the backlogged progs patches. Please take a look and give

Re: Triple parity and beyond

2013-11-22 Thread Stan Hoeppner
On 11/21/2013 5:38 PM, John Williams wrote: On Thu, Nov 21, 2013 at 2:57 PM, Stan Hoeppner s...@hardwarefreak.com wrote: He wrote that article in late 2009. It seems pretty clear he wasn't looking 10 years forward to 20TB drives, where the minimum mirror rebuild time will be ~18 hours, and

Re: Receive fails (Could not find parent subvolume)

2013-11-22 Thread Joshua Varghese
Hi, I'm trying to implement a mechanism for incremental backup using btrfs. I have two hardisks and each has a btrfs partition on it. I am able to take a snapshot of the source partition and send it to the backup disk using the send-recieve mechanism. When I try to send an incremental backup of a

Re: Receive fails (Could not find parent subvolume)

2013-11-22 Thread Stefan Behrens
On Fri, 22 Nov 2013 01:47:43 -0800, Joshua Varghese wrote: Hi, I'm trying to implement a mechanism for incremental backup using btrfs. I have two hardisks and each has a btrfs partition on it. I am able to take a snapshot of the source partition and send it to the backup disk using the

Re: Receive fails (Could not find parent subvolume)

2013-11-22 Thread Joshua Varghese
Hi Stefan, Thank you for replying. I am running on a 3.10 kernel, will upgrade it and check. Regards On Fri 22 Nov 2013 10:34:08 GMT, Stefan Behrens wrote: Re: Receive fails (Could not find parent subvolume) On Fri, 22 Nov 2013 01:47:43 -0800, Joshua Varghese wrote: Hi, I'm trying to

[PATCH] Btrfs: don't clear the default compression type

2013-11-22 Thread Miao Xie
We met a oops caused by the wrong compression type: [ 556.512356] BUG: unable to handle kernel NULL pointer dereference at (null) [ 556.512370] IP: [811dbaa0] __list_del_entry+0x1/0x98 [SNIP] [ 556.512490] [811dbb44] ? list_del+0xd/0x2b [ 556.512539]

Re: Receive fails (Could not find parent subvolume)

2013-11-22 Thread Joshua Varghese
Thank you. it worked with 3.12.0 :) On Fri 22 Nov 2013 10:44:41 GMT, Joshua wrote: Hi Stefan, Thank you for replying. I am running on a 3.10 kernel, will upgrade it and check. Regards On Fri 22 Nov 2013 10:34:08 GMT, Stefan Behrens wrote: Re: Receive fails (Could not find parent

Re: progs integration branch moved to master (new default leafsize)

2013-11-22 Thread Chris Mason
Quoting Martin (2013-11-22 04:03:41) On 21/11/13 23:37, Chris Mason wrote: Quoting Martin (2013-11-08 18:53:06) On 08/11/13 22:01, Chris Mason wrote: Hi everyone, This patch is now the tip of the master branch for btrfs-progs, which has been updated to include most of the backlogged

Nagios probe for btrfs RAID status?

2013-11-22 Thread Daniel Pocock
I just did a search and couldn't find any probe for btrfs RAID status The check_raid plugin seems to recognise mdadm and various other types of RAID but not btrfs Has anybody seen a plugin for Nagios or could anybody comment on how it should work if somebody wants to make one? For example,

RE: Why Does Your Website : mail-archive.com Needs an SEO Expert?

2013-11-22 Thread outpourings29223
Hello Website Owner We agree that you must be receiving many emails from companies regarding SEO services. Our questions is have you ever given a serious thought to it for even a moment that your website, especially in the current scenario, needs SEO? Keeping in mind what google actually

Re: Triple parity and beyond

2013-11-22 Thread John Williams
On Fri, Nov 22, 2013 at 1:35 AM, Stan Hoeppner s...@hardwarefreak.com wrote: Only one graph goes to 2019, the rest are 2010 or less. That being the case, his 2019 graph deals with projected reliability of single, double, and triple parity. The whole article goes to 2019 (or longer). He shows

Re: Triple parity and beyond

2013-11-22 Thread Stan Hoeppner
On 11/22/2013 2:13 AM, Stan Hoeppner wrote: Hi David, On 11/21/2013 3:07 AM, David Brown wrote: ... I don't see that there needs to be any changes to the existing md code to make raid15 work - it is merely a raid 5 made from a set of raid1 pairs. The sole purpose of the parity layer of

Fwd: [virt-devel] btrfs NOCOW for VM disk images

2013-11-22 Thread John Dulaney
- Forwarded Message - From: Stefan Hajnoczi stefa...@redhat.com To: Eric Sandeen sand...@redhat.com Cc: virt-de...@redhat.com, Kevin Wolf kw...@redhat.com Sent: Friday, November 22, 2013 9:20:51 AM Subject: [virt-devel] btrfs NOCOW for VM disk images Hi, In upstream QEMU we're

Re: Triple parity and beyond

2013-11-22 Thread Mark Knecht
On Fri, Nov 22, 2013 at 12:13 AM, Stan Hoeppner s...@hardwarefreak.com wrote: Hi David, On 11/21/2013 3:07 AM, David Brown wrote: SNIP Shouldn't we be talking about RAID 15 here, rather than RAID 51 ? I interpret RAID 15 to be like RAID 10 - a raid5 set of raid1 mirrors, while RAID 51 would

[PATCH] Btrfs: avoid unnecessary ordered extent cache resets

2013-11-22 Thread Filipe David Borba Manana
After an ordered extent completes, don't blindly reset the inode's ordered tree last accessed ordered extent pointer. While running the xfstests I noticed that about 29% of the time the ordered extent to which tree-last pointed was not the same as our just completed ordered extent. After that I

Re: [PATCH] Btrfs: avoid unnecessary ordered extent cache resets

2013-11-22 Thread Josef Bacik
On Fri, Nov 22, 2013 at 04:52:43PM +, Filipe David Borba Manana wrote: After an ordered extent completes, don't blindly reset the inode's ordered tree last accessed ordered extent pointer. While running the xfstests I noticed that about 29% of the time the ordered extent to which

Re: [PATCH] Btrfs: don't clear the default compression type

2013-11-22 Thread Josef Bacik
On Fri, Nov 22, 2013 at 06:47:59PM +0800, Miao Xie wrote: We met a oops caused by the wrong compression type: [ 556.512356] BUG: unable to handle kernel NULL pointer dereference at (null) [ 556.512370] IP: [811dbaa0] __list_del_entry+0x1/0x98 [SNIP] [ 556.512490]

Re: [PATCH 5/5] Btrfs: reclaim the reserved metadata space at background

2013-11-22 Thread Josef Bacik
On Thu, Nov 21, 2013 at 09:43:18PM +0800, Miao Xie wrote: Before applying this patch, the task had to reclaim the metadata space by itself if the metadata space was not enough. And When the task started the space reclamation, all the other tasks which wanted to reserve the metadata space were

[PATCH v2] Btrfs: avoid unnecessary ordered extent cache resets

2013-11-22 Thread Filipe David Borba Manana
After an ordered extent completes, don't blindly reset the inode's ordered tree last accessed ordered extent pointer. While running the xfstests I noticed that about 29% of the time the ordered extent to which tree-last pointed was not the same as our just completed ordered extent. After that I

Re: progs integration branch moved to master (new default leafsize)

2013-11-22 Thread Martin
On 22/11/13 13:40, Chris Mason wrote: Quoting Martin (2013-11-22 04:03:41) * QA Notice: Package triggers severe warnings which indicate that it *may exhibit random runtime failures. * disk-io.c:91:5: warning: dereferencing type-punned pointer will break strict-aliasing rules

Re: Triple parity and beyond

2013-11-22 Thread Duncan
Mark Knecht posted on Fri, 22 Nov 2013 08:50:32 -0800 as excerpted: On Fri, Nov 22, 2013 at 12:13 AM, Stan Hoeppner s...@hardwarefreak.com wrote: Now that you mention it, yes, RAID 15 would fit much better with convention. Not sure why I thought 51. So it's RAID 15 from here. SNIP For

Re: progs integration branch moved to master (new default leafsize)

2013-11-22 Thread Chris Mason
Quoting Martin (2013-11-22 14:50:17) On 22/11/13 13:40, Chris Mason wrote: Quoting Martin (2013-11-22 04:03:41) * QA Notice: Package triggers severe warnings which indicate that it *may exhibit random runtime failures. * disk-io.c:91:5: warning: dereferencing type-punned

Re: Triple parity and beyond

2013-11-22 Thread Piergiorgio Sartor
Hi David, On Fri, Nov 22, 2013 at 01:32:09AM +0100, David Brown wrote: One typical case is when many errors are found, belonging to the same disk. This case clearly shows the disk is to be replaced or the interface checked... But, again, the user is the master, not the machine... :-)

Re: Fwd: [virt-devel] btrfs NOCOW for VM disk images

2013-11-22 Thread Duncan
John Dulaney posted on Fri, 22 Nov 2013 11:17:34 -0500 as excerpted: In upstream QEMU we're discussing patches that set the NOCOW flag on disk image files. We're told that this increases btrfs performance greatly since the file system will modify data in-place like ext4/xfs. Indeed. For VM

Re: [virt-devel] btrfs NOCOW for VM disk images

2013-11-22 Thread Roman Mamedov
On Fri, 22 Nov 2013 21:26:16 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: During testing I found that the NOCOW flag prevents file cloning from working. cp --reflink fails with EINVAL when the source file has the NOCOW flag set. That would be expected, since disabling COW means the

Re: progs integration branch moved to master (new default leafsize)

2013-11-22 Thread Duncan
Chris Mason posted on Fri, 22 Nov 2013 08:40:38 -0500 as excerpted: Does gentoo modify the optimizations from the Makefile? We actually have many strict-aliasing warnings, but I didn't think they came up until -O2. At any rate, I'm adding -fno-strict-aliasing just to be sure. FWIW, Gentoo

Re: [virt-devel] btrfs NOCOW for VM disk images

2013-11-22 Thread Chris Murphy
On Nov 22, 2013, at 9:17 AM, John Dulaney jdula...@redhat.com wrote: In upstream QEMU we're discussing patches that set the NOCOW flag on disk image files. We're told that this increases btrfs performance greatly since the file system will modify data in-place like ext4/xfs. The best

Re: Triple parity and beyond

2013-11-22 Thread Stan Hoeppner
On 11/22/2013 9:01 AM, John Williams wrote: snip I see no advantage of RAID 15, and several disadvantages. Of course not, just as I sated previously. On 11/22/2013 2:13 AM, Stan Hoeppner wrote: Parity users who currently shun RAID 10 for this reason will also shun this RAID 15. With that

Re: Triple parity and beyond

2013-11-22 Thread NeilBrown
On Fri, 22 Nov 2013 10:07:09 -0600 Stan Hoeppner s...@hardwarefreak.com wrote: On 11/22/2013 2:13 AM, Stan Hoeppner wrote: Hi David, On 11/21/2013 3:07 AM, David Brown wrote: ... I don't see that there needs to be any changes to the existing md code to make raid15 work - it is merely

Re: Triple parity and beyond

2013-11-22 Thread NeilBrown
On Thu, 21 Nov 2013 16:57:48 -0600 Stan Hoeppner s...@hardwarefreak.com wrote: On 11/21/2013 1:05 AM, John Williams wrote: On Wed, Nov 20, 2013 at 10:52 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 11/20/2013 8:46 PM, John Williams wrote: For myself or any machines I managed for

Re: progs integration branch moved to master (new default leafsize)

2013-11-22 Thread Martin
On 22/11/13 19:57, Chris Mason wrote: Quoting Martin (2013-11-22 14:50:17) On 22/11/13 13:40, Chris Mason wrote: Quoting Martin (2013-11-22 04:03:41) * QA Notice: Package triggers severe warnings which indicate that it *may exhibit random runtime failures. * disk-io.c:91:5:

Re: [PATCH 0/5] Add support for object properties

2013-11-22 Thread David Sterba
On Tue, Nov 12, 2013 at 01:41:41PM +, Filipe David Borba Manana wrote: This is a revised version of the original proposal/work from Alexander Block to introduce a generic framework to set properties on btrfs filesystem objects (inodes, subvolumes, filesystems, devices). Currently

Re: [virt-devel] btrfs NOCOW for VM disk images

2013-11-22 Thread David Sterba
On Sat, Nov 23, 2013 at 04:00:28AM +0600, Roman Mamedov wrote: On Fri, 22 Nov 2013 21:26:16 + (UTC) Duncan 1i5t5.dun...@cox.net wrote: During testing I found that the NOCOW flag prevents file cloning from working. cp --reflink fails with EINVAL when the source file has the NOCOW

Re: Triple parity and beyond

2013-11-22 Thread Stan Hoeppner
On 11/22/2013 5:07 PM, NeilBrown wrote: On Thu, 21 Nov 2013 16:57:48 -0600 Stan Hoeppner s...@hardwarefreak.com wrote: On 11/21/2013 1:05 AM, John Williams wrote: On Wed, Nov 20, 2013 at 10:52 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 11/20/2013 8:46 PM, John Williams wrote: For

Re: Nagios probe for btrfs RAID status?

2013-11-22 Thread Anand Jain
For example, would the command btrfs filesystem show --all-devices give a non-zero error status or some other clue if any of the devices are at risk? No there isn't any good way as of now. that's something to fix. Thanks, Anand -- To unsubscribe from this list: send the line

Re: Triple parity and beyond

2013-11-22 Thread NeilBrown
On Fri, 22 Nov 2013 21:46:50 -0600 Stan Hoeppner s...@hardwarefreak.com wrote: On 11/22/2013 5:07 PM, NeilBrown wrote: On Thu, 21 Nov 2013 16:57:48 -0600 Stan Hoeppner s...@hardwarefreak.com wrote: On 11/21/2013 1:05 AM, John Williams wrote: On Wed, Nov 20, 2013 at 10:52 PM, Stan

Re: Triple parity and beyond

2013-11-22 Thread John Williams
On Fri, Nov 22, 2013 at 9:04 PM, NeilBrown ne...@suse.de wrote: I guess with that many drives you could hit PCI bus throughput limits. A 16-lane PCIe 4.0 could just about give 100MB/s to each of 16 devices. So you would really need top-end hardware to keep all of 16 drives busy in a

Re: Triple parity and beyond

2013-11-22 Thread NeilBrown
On Fri, 22 Nov 2013 21:34:41 -0800 John Williams jwilliams4...@gmail.com wrote: On Fri, Nov 22, 2013 at 9:04 PM, NeilBrown ne...@suse.de wrote: I guess with that many drives you could hit PCI bus throughput limits. A 16-lane PCIe 4.0 could just about give 100MB/s to each of 16 devices.

Re: Triple parity and beyond

2013-11-22 Thread Andrea Mazzoleni
Hi Piergiorgio, How about par2? How does this work? I checked the matrix they use, and sometimes it contains some singular square submatrix. It seems that in GF(2^16) these cases are just less common. Maybe they were just unnoticed. Anyway, this seems to be an already known problem for PAR2,