Re: Actual effect of mkfs.btrfs -m raid10 /dev/sdX ... -d raid10 /dev/sdX ...

2013-11-20 Thread Hugo Mills
On Tue, Nov 19, 2013 at 11:16:58PM +, Duncan wrote: Hugo Mills posted on Tue, 19 Nov 2013 09:06:02 + as excerpted: This will happen with RAID-10. The allocator will write stripes as wide as it can: in this case, the first stripes will run across all 8 devices, until the SSDs are

btrfs still core dumps

2013-11-20 Thread Marc MERLIN
Here is another way in which btrfs-tools crashes if it sees things it doesn't like. Is anyone interested in those, or not really? gandalfthegreat:/tmp# btrfs-find-root /dev/mapper/ssdcrypt Check tree block failed, want=20971520, have=10422900740180685247 Check tree block failed, want=20971520,

Re: Actual effect of mkfs.btrfs -m raid10 /dev/sdX ... -d raid10 /dev/sdX ...

2013-11-20 Thread Chris Murphy
On Nov 19, 2013, at 11:35 PM, Martin m_bt...@ml1.co.uk wrote: On 19/11/13 23:16, Duncan wrote: So we have: 1) raid1 is exactly two copies of data, paired devices. 2) raid0 is a stripe exactly two devices wide (reinforced by to read a stripe takes only two devices), so again paired

Re: Actual effect of mkfs.btrfs -m raid10 /dev/sdX ... -d raid10 /dev/sdX ...

2013-11-20 Thread Russell Coker
On Wed, 20 Nov 2013, Chris Murphy li...@colorremedies.com wrote: Hot spares are worse than useless. Especially for raid10. The drive takes up space doing nothing but suck power, rather than adding space or performance. Somehow this idea comes from cheap companies who seem to think their data

Btrfs assertion failed: !memcmp_extent_buffer() and bug at ctree.h:3832

2013-11-20 Thread Szőts Ákos
Dear list members, After a quick directory defragging, I tried to start the mysql daemon. Then suddenly to following appeared in the kernel log and the whole system hung up. This log is also from an instant copy over SSH. Now I'm planning to run a btrfsck on the file system, also. [

Re: Triple parity and beyond

2013-11-20 Thread David Brown
On 20/11/13 02:23, John Williams wrote: On Tue, Nov 19, 2013 at 4:54 PM, Chris Murphy li...@colorremedies.com wrote: If anything, I'd like to see two implementations of RAID 6 dual parity. The existing implementation in the md driver and btrfs could remain the default, but users could opt

Re: Btrfs assertion failed: !memcmp_extent_buffer() and bug at ctree.h:3832

2013-11-20 Thread Filipe David Manana
On Wed, Nov 20, 2013 at 10:28 AM, Szőts Ákos szots...@gmail.com wrote: Dear list members, After a quick directory defragging, I tried to start the mysql daemon. Then suddenly to following appeared in the kernel log and the whole system hung up. This log is also from an instant copy over SSH.

[PATCH] btrfs: update kconfig help text

2013-11-20 Thread David Sterba
Reflect the current status. Portions of the text taken from the wiki pages. Signed-off-by: David Sterba dste...@suse.cz --- Please enqueue for 3.13 fs/btrfs/Kconfig | 15 ++- 1 file changed, 10 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/Kconfig b/fs/btrfs/Kconfig index

[PATCH 1/2] Documentation: filesystems: add new btrfs mount options

2013-11-20 Thread David Sterba
Two new options were added in 3.12: commit and rescan_uuid_tree CC: linux-...@vger.kernel.org Signed-off-by: David Sterba dste...@suse.cz --- Documentation/filesystems/btrfs.txt | 12 +++- 1 file changed, 11 insertions(+), 1 deletion(-) diff --git a/Documentation/filesystems/btrfs.txt

[PATCH 2/2] Documentation: filesystems: update btrfs tools section

2013-11-20 Thread David Sterba
The tools mentioned have been obsoleted long ago, replace with the current ones. CC: linux-...@vger.kernel.org Signed-off-by: David Sterba dste...@suse.cz --- Documentation/filesystems/btrfs.txt | 22 ++ 1 file changed, 6 insertions(+), 16 deletions(-) diff --git

Re: [PATCH v4 2/3] btrfs-progs: fs show should handle if subvol(s) mounted

2013-11-20 Thread David Sterba
On Fri, Nov 15, 2013 at 07:25:34PM +0800, Anand Jain wrote: static int btrfs_scan_kernel(void *search) { - int ret = 0, fd; - FILE *f; - struct mntent *mnt; - struct btrfs_ioctl_fs_info_args fs_info_arg; - struct btrfs_ioctl_dev_info_args *dev_info_arg = NULL; -

Btrfsck complains about fs tree 264 refs 1 not found

2013-11-20 Thread Pedro Fonseca
Hi, I've been getting the error message fs tree 264 refs 1 not found when running btrfsck (v0.19) after a test case. The test case creates and then deletes a subvolume while concurrently creating a snapshot of the parent directory. This situation occurred with kernel version 3.11.1. Here's

Re: fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format arguments

2013-11-20 Thread Kees Cook
On Tue, Nov 19, 2013 at 8:05 PM, Fengguang Wu fengguang...@intel.com wrote: On Tue, Nov 19, 2013 at 07:56:35PM -0800, Kees Cook wrote: Hi! Which tree is 'devel-snb'? I don't see that on the kernel.org trees. It's my local merge branch, based on the latest upstream release. Hm, which

Re: Triple parity and beyond

2013-11-20 Thread James Plank
Hi all -- no real comments, except as I mentioned to Ric, my tutorial in FAST last February presents Reed-Solomon coding with Cauchy matrices, and then makes special note of the common pitfall of assuming that you can append a Vandermonde matrix to an identity matrix. Please see

Re: Actual effect of mkfs.btrfs -m raid10 /dev/sdX ... -d raid10 /dev/sdX ...

2013-11-20 Thread Hugo Mills
On Wed, Nov 20, 2013 at 04:43:57PM +, Duncan wrote: Hugo Mills posted on Wed, 20 Nov 2013 08:09:58 + as excerpted: RAID-0: min 2 devices RAID-10: min 4 devices RAID-5: min 2 devices (I think) RAID-6: min 3 devices (I think) RAID-5 should be 3-device minimum (each stripe

Re: btrfsck --repair /dev/sdc (Was: [PATCH] Btrfs-progs: allow --init-extent-tree to work when extent tree is borked)

2013-11-20 Thread Duncan
Martin posted on Wed, 20 Nov 2013 06:51:20 + as excerpted: It's now gone back to a pattern from a full week ago: (gdb) bt #0 0x0042d576 in read_extent_buffer () #1 0x0041ee79 in btrfs_check_node () #2 0x00420211 in check_block () #3 0x00420813 in

Re: Btrfsck complains about fs tree 264 refs 1 not found

2013-11-20 Thread Duncan
Pedro Fonseca posted on Wed, 20 Nov 2013 16:22:18 +0100 as excerpted: I've been getting the error message fs tree 264 refs 1 not found when running btrfsck (v0.19) after a test case. The test case creates and then deletes a subvolume while concurrently creating a snapshot of the parent

Re: fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format arguments

2013-11-20 Thread Chris Mason
Quoting Fengguang Wu (2013-11-19 23:05:51) On Tue, Nov 19, 2013 at 07:56:35PM -0800, Kees Cook wrote: Hi! Which tree is 'devel-snb'? I don't see that on the kernel.org trees. It's my local merge branch, based on the latest upstream release. Let's CC the btrfs developers for this

Re: [PATCH] Btrfs: add missing sysfs group remove / fix leak

2013-11-20 Thread Jeff Mahoney
On 11/19/13, 11:07 AM, Filipe David Borba Manana wrote: From kmemleak: hex dump (first 32 bytes): 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 backtrace: [8175f5f6]

Re: fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format arguments

2013-11-20 Thread Jeff Mahoney
On 11/20/13, 12:30 PM, Chris Mason wrote: Quoting Fengguang Wu (2013-11-19 23:05:51) On Tue, Nov 19, 2013 at 07:56:35PM -0800, Kees Cook wrote: Hi! Which tree is 'devel-snb'? I don't see that on the kernel.org trees. It's my local merge branch, based on the latest upstream release. Let's

Re: Triple parity and beyond

2013-11-20 Thread John Williams
On Wed, Nov 20, 2013 at 2:31 AM, David Brown david.br...@hesbynett.no wrote: That's certainly a reasonable way to look at it. We should not limit the possibilities for high-end systems because of the limitations of low-end systems that are unlikely to use 3+ parity anyway. I've also looked

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi David, The choice of ZFS to use powers of 4 was likely not optimal, because to multiply by 4, it has to do two multiplications by 2. I can agree with that. I didn't copy ZFS's choice here David, it was not my intention to suggest that you copied from ZFS. Sorry to have expressed myself

Re: fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format arguments

2013-11-20 Thread Kees Cook
On Wed, Nov 20, 2013 at 10:05 AM, Jeff Mahoney je...@suse.com wrote: On 11/20/13, 12:30 PM, Chris Mason wrote: Quoting Fengguang Wu (2013-11-19 23:05:51) On Tue, Nov 19, 2013 at 07:56:35PM -0800, Kees Cook wrote: Hi! Which tree is 'devel-snb'? I don't see that on the kernel.org trees. It's

Re: Triple parity and beyond

2013-11-20 Thread H. Peter Anvin
It is also possible to quickly multiply by 2^-1 which makes for an interesting R parity. Andrea Mazzoleni amadva...@gmail.com wrote: Hi David, The choice of ZFS to use powers of 4 was likely not optimal, because to multiply by 4, it has to do two multiplications by 2. I can agree with that.

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi John, Yes. There are still AMD CPUs sold without SSSE3. Most notably Athlon. Instead, Intel is providing SSSE3 from the Core 2 Duo. A detailed list is available at: http://en.wikipedia.org/wiki/SSSE3 Ciao, Andrea On Wed, Nov 20, 2013 at 7:09 PM, John Williams jwilliams4...@gmail.com wrote:

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi, Yep. At present to multiply for 2^-1 I'm using in C: static inline uint64_t d2_64(uint64_t v) { uint64_t mask = v 0x0101010101010101U; mask = (mask 8) - mask; v = (v 1) 0x7f7f7f7f7f7f7f7fU; v ^= mask 0x8e8e8e8e8e8e8e8eU; return v; } and for SSE2:

Re: Triple parity and beyond

2013-11-20 Thread H. Peter Anvin
On 11/20/2013 10:56 AM, Andrea Mazzoleni wrote: Hi, Yep. At present to multiply for 2^-1 I'm using in C: static inline uint64_t d2_64(uint64_t v) { uint64_t mask = v 0x0101010101010101U; mask = (mask 8) - mask; v = (v 1) 0x7f7f7f7f7f7f7f7fU; v ^=

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi Jim, I build the matrix in a way that results in coefficients matching Linux RAID for the first two rows, and at the same time gives the guarantee that all the square submatrices are not singular, resulting in a MDS code. I start forming a Cauchy matrix setting each element to 1/(xi+yj) where

Re: Triple parity and beyond

2013-11-20 Thread H. Peter Anvin
On 11/20/2013 11:05 AM, Andrea Mazzoleni wrote: For the first row with j=0, I use xi = 2^-i and y0 = 0, that results in: How can xi = 2^-i if x is supposed to be constant? That doesn't mean that your approach isn't valid, of course, but it might not be a Cauchy matrix and thus needs

Re: btrfsck --repair /dev/sdc (Was: [PATCH] Btrfs-progs: allow --init-extent-tree to work when extent tree is borked)

2013-11-20 Thread Martin
On 20/11/13 17:08, Duncan wrote: Martin posted on Wed, 20 Nov 2013 06:51:20 + as excerpted: It's now gone back to a pattern from a full week ago: (gdb) bt #0 0x0042d576 in read_extent_buffer () #1 0x0041ee79 in btrfs_check_node () #2 0x00420211 in check_block

Re: btrfsck --repair /dev/sdc (Was: [PATCH] Btrfs-progs: allow --init-extent-tree to work when extent tree is borked)

2013-11-20 Thread Martin
On 20/11/13 17:08, Duncan wrote: Which leads to the question of what to do next. Obviously, there have been a number of update patches since then, some of which might address your problem. You could update your kernel and userspace and try again... /if/ you have the patience... This is

Kernel panic at (among others): free-space-cache.c:1179

2013-11-20 Thread Szőts Ákos
Dear list members, When the kernel had paniced because of the key_search_validate() defrag bug, on the next restart it started to recreate its space- and inode caches. Both were finished in 5 minutes but when I tried to enter back to init 3, the kernel paniced and hung up again. Versions: -

Re: Triple parity and beyond

2013-11-20 Thread James Plank
Peter, I think I understand it differently. Concrete example in GF(256) for k=6, m=4: First, create a 3 by 6 cauchy matrix, using x_i = 2^-i, and y_i = 0 for i=0, and y_i = 2^i for other i. In this case: x = { 1, 142, 71, 173, 216, 108 } y = { 0, 2, 4). The cauchy matrix is: 1 2 4

Re: [PATCH] Btrfs: add missing sysfs group remove / fix leak

2013-11-20 Thread Filipe David Manana
On Wed, Nov 20, 2013 at 6:02 PM, Jeff Mahoney je...@suse.com wrote: On 11/19/13, 11:07 AM, Filipe David Borba Manana wrote: From kmemleak: hex dump (first 32 bytes): 03 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi Peter, static inline uint64_t d2_64(uint64_t v) { uint64_t mask = v 0x0101010101010101U; mask = (mask 8) - mask; (mask 7) I assume... No. It's (mask 8) - mask. We want to expand the bit at position 0 (in each byte) to the full byte, resulting in 0xFF if the bit is at

Re: Triple parity and beyond

2013-11-20 Thread H. Peter Anvin
On 11/20/2013 01:04 PM, Andrea Mazzoleni wrote: Hi Peter, static inline uint64_t d2_64(uint64_t v) { uint64_t mask = v 0x0101010101010101U; mask = (mask 8) - mask; (mask 7) I assume... No. It's (mask 8) - mask. We want to expand the bit at position 0 (in each byte)

Re: Actual effect of mkfs.btrfs -m raid10 /dev/sdX ... -d raid10 /dev/sdX ...

2013-11-20 Thread Duncan
Hugo Mills posted on Wed, 20 Nov 2013 16:52:47 + as excerpted: Perhaps it's time I get that wiki account and edit some of this stuff myself... Do check the assumptions first. :) Of course. =:^) -- Duncan - List replies preferred. No HTML msgs. Every nonfree program has a lord, a

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi Peter, Now, that doesn't sound like something that can get neatly meshed into the Cauchy matrix scheme, I assume. You are correct. Multiplication by 2^-1 cannot be used for the Cauchy method. I used it to implement an alternate triple parity not requiring PSHUFB that I used as reference for

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi, First, create a 3 by 6 cauchy matrix, using x_i = 2^-i, and y_i = 0 for i=0, and y_i = 2^i for other i. In this case: x = { 1, 142, 71, 173, 216, 108 } y = { 0, 2, 4). The cauchy matrix is: 1 2 4 8 16 32 244 83 78 183 118 47 167 39 213 59 153 82 Divide row 2 by

Re: Triple parity and beyond

2013-11-20 Thread H. Peter Anvin
On 11/20/2013 12:30 PM, James Plank wrote: Peter, I think I understand it differently. Concrete example in GF(256) for k=6, m=4: First, create a 3 by 6 cauchy matrix, using x_i = 2^-i, and y_i = 0 for i=0, and y_i = 2^i for other i. In this case: x = { 1, 142, 71, 173, 216, 108 } y

Re: Triple parity and beyond

2013-11-20 Thread Andrea Mazzoleni
Hi Piergiorgio, In RAID-6 (as per raid6check) there is an easy way to verify where an HDD has incorrect data. I suspect, for each 2 parity block it should be possible to find 1 error (and if this is true, then quad parity is more attractive than triple one). Yes. The theory say that with

[PATCH] btrfs: fix static checker warnings

2013-11-20 Thread Jeff Mahoney
This patch fixes the following warnings: fs/btrfs/extent-tree.c:6201:12: sparse: symbol 'get_raid_name' was not declared. Should it be static? fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format arguments [-Werror=format-security] get_raid_name(index));

Re: Triple parity and beyond

2013-11-20 Thread Piergiorgio Sartor
On Wed, Nov 20, 2013 at 11:44:39AM +0100, David Brown wrote: [...] In RAID-6 (as per raid6check) there is an easy way to verify where an HDD has incorrect data. I think the way to do that is just to generate the parity blocks from the data blocks, and compare them to the existing parity

[PATCH] btrfs: fix leaks during sysfs teardown

2013-11-20 Thread Jeff Mahoney
Filipe noticed that we were leaking the features attribute group after umount. His fix of just calling sysfs_remove_group() wasn't enough since that removes just the supported features and not the unknown features (but a regular test wouldn't show that). This patch changes the unknown feature

Re: [PATCH] btrfs: fix static checker warnings

2013-11-20 Thread Kees Cook
On Wed, Nov 20, 2013 at 1:50 PM, Jeff Mahoney je...@suse.com wrote: This patch fixes the following warnings: fs/btrfs/extent-tree.c:6201:12: sparse: symbol 'get_raid_name' was not declared. Should it be static? fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format

Re: Triple parity and beyond

2013-11-20 Thread Piergiorgio Sartor
On Mon, Nov 18, 2013 at 11:08:59PM +0100, Andrea Mazzoleni wrote: [...] I've a side question, a bit OT, but maybe you could help with the answer. How about par2? How does this work? They claim Vendermonde matrix and they seem to be quite flexible in amount of parities. The could be in GF(2^16),

Re: fs/btrfs/extent-tree.c:8430:9: error: format not a string literal and no format arguments

2013-11-20 Thread Fengguang Wu
On Wed, Nov 20, 2013 at 08:04:58AM -0800, Kees Cook wrote: On Tue, Nov 19, 2013 at 8:05 PM, Fengguang Wu fengguang...@intel.com wrote: On Tue, Nov 19, 2013 at 07:56:35PM -0800, Kees Cook wrote: Hi! Which tree is 'devel-snb'? I don't see that on the kernel.org trees. It's my local

Re: Triple parity and beyond

2013-11-20 Thread John Williams
For myself or any machines I managed for work that do not need high IOPS, I would definitely choose triple- or quad-parity over RAID 51 or similar schemes with arrays of 16 - 32 drives. No need to go into detail here on a subject Adam Leventhal has already covered in detail in an article

[PATCH v5 2/3] btrfs-progs: fs show should handle if subvol(s) mounted

2013-11-20 Thread Anand Jain
as of now with out this patch user would see fsinfo per btrfs mount path but which mean multiple entry if more than one subvol is mounted of the same fsid. so this patch will handle that nicely. Signed-off-by: Anand Jain anand.j...@oracle.com --- v5: fixup missed mem free, thanks David v4:

Re: [PATCH v4 2/3] btrfs-progs: fs show should handle if subvol(s) mounted

2013-11-20 Thread Anand Jain
On 11/20/2013 10:18 PM, David Sterba wrote: On Fri, Nov 15, 2013 at 07:25:34PM +0800, Anand Jain wrote: static int btrfs_scan_kernel(void *search) { - int ret = 0, fd; - FILE *f; - struct mntent *mnt; - struct btrfs_ioctl_fs_info_args fs_info_arg; - struct

Re: Triple parity and beyond

2013-11-20 Thread Stan Hoeppner
On 11/20/2013 12:44 PM, Andrea Mazzoleni wrote: Yes. There are still AMD CPUs sold without SSSE3. Most notably Athlon. Instead, Intel is providing SSSE3 from the Core 2 Duo. I hate branding discontinuity, due to the resulting confusion... Athlon, Athlon64, Athlon64 X2, Athlon X2 (K10), Athlon

Re: Triple parity and beyond

2013-11-20 Thread John Williams
On Wed, Nov 20, 2013 at 10:52 PM, Stan Hoeppner s...@hardwarefreak.com wrote: On 11/20/2013 8:46 PM, John Williams wrote: For myself or any machines I managed for work that do not need high IOPS, I would definitely choose triple- or quad-parity over RAID 51 or similar schemes with arrays of 16