Re: [patch 01/99] btrfs: Add btrfs_panic()

2011-11-23 Thread David Brown
On Wed, Nov 23, 2011 at 07:35:34PM -0500, Jeff Mahoney wrote: As part of the effort to eliminate BUG_ON as an error handling technique, we need to determine which errors are actual logic errors, which are on-disk corruption, and which are normal runtime errors e.g. -ENOMEM. Annotating these erro

Re: [patch 01/99] btrfs: Add btrfs_panic()

2011-11-23 Thread David Brown
On Wed, Nov 23, 2011 at 09:22:06PM -0500, Jeff Mahoney wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/23/2011 09:05 PM, David Brown wrote: On Wed, Nov 23, 2011 at 07:35:34PM -0500, Jeff Mahoney wrote: As part of the effort to eliminate BUG_ON as an error handling technique, we

Re: [patch 01/99] btrfs: Add btrfs_panic()

2011-11-24 Thread David Brown
On Thu, Nov 24, 2011 at 09:36:55PM -0500, Jeff Mahoney wrote: Probably best not to, it makes them inconsistent with the rest of the kernel's history when imported into git. The body becomes the commit text directly. I'll change them to do this since you're obviously correct. You're the first

Re: btrfsprogs source code

2012-01-04 Thread David Brown
On Tue, Jan 03, 2012 at 01:05:07PM -0500, Calvin Walton wrote: The best way to get the btrfs-progs source is probably via git; Chris Mason's repository for it can be found at http://git.kernel.org/?p=linux/kernel/git/mason/btrfs-progs.git Chris, The wiki at https://btrfs.wiki.kernel.org/artic

Odd behavior of subvolume find-new

2012-01-09 Thread David Brown
I've been creating some time-based snapshots, e.g. # btrfs subvolume snapshot @root 2012-01-09-@root After some changes, I wanted to see what had changed, so I tried: # btrfs subvolume find-new @root 2012-01-09-@root transid marker was 37 which doesn't print anything out. Curio

Re: [patch] Btrfs: silence a compiler warning

2012-02-22 Thread David Brown
On Wed, Feb 22, 2012 at 10:30:55AM +0300, Dan Carpenter wrote: Gcc warns that "ret" can be used uninitialized. It can't actually be used uninitialized because btrfs_num_copies() always returns 1 or more. Signed-off-by: Dan Carpenter diff --git a/fs/btrfs/check-integrity.c b/fs/btrfs/check-int

Re: converting one-disk btrfs into RAID-1?

2010-10-12 Thread David Brown
On 11/10/2010 19:06, Chris Ball wrote: Hi, > Is it possible to turn a 1-disk (partition) btrfs filesystem into > RAID-1? Not yet, but I'm pretty sure it's on the roadmap. - Chris. Is it possible to view the raid levels of data and meta data for an existing btrfs filesystem? It's

Re: converting one-disk btrfs into RAID-1?

2010-10-12 Thread David Brown
On 12/10/2010 11:34, Tomasz Torcz wrote: On Tue, Oct 12, 2010 at 11:32:07AM +0200, David Brown wrote: On 11/10/2010 19:06, Chris Ball wrote: Hi, > Is it possible to turn a 1-disk (partition) btrfs filesystem into > RAID-1? Not yet, but I'm pretty sure it'

Re: Replacing the top-level root

2010-10-26 Thread David Brown
On Mon, Oct 25, 2010 at 03:20:58PM -0500, C Anthony Risinger wrote: For example, right now extlinux support booting btrfs, but _only_ from the top-level root. if i just had a way to "swap" the top-level root with a different subvol, i could overcome several problems i have with users all at onc

Re: Triple parity and beyond

2013-11-19 Thread David Brown
On 19/11/13 00:25, H. Peter Anvin wrote: > On 11/18/2013 02:35 PM, Andrea Mazzoleni wrote: >> Hi Peter, >> >> The Cauchy matrix has the mathematical property to always have itself >> and all submatrices not singular. So, we are sure that we can always >> solve the equations to recover the data disk

Re: Triple parity and beyond

2013-11-20 Thread David Brown
Oracle/Sun, Dell/Compellent ZFS: 3 parity drives > NEC HydraStor: 3 parity drives > EMC/Isilon: 4 parity drives > Amplidata: 4 parity drives > CleverSafe: 6 parity drives > StreamScale/BigParity: 7 parity drives > > And Btrfs with six parities would be surely cool :) > >

Re: Triple parity and beyond

2013-11-20 Thread David Brown
On 20/11/13 02:23, John Williams wrote: > On Tue, Nov 19, 2013 at 4:54 PM, Chris Murphy > wrote: >> If anything, I'd like to see two implementations of RAID 6 dual >> parity. The existing implementation in the md driver and btrfs could >> remain the default, but users could opt into Cauchy matrix

Re: Triple parity and beyond

2013-11-20 Thread David Brown
On 19/11/13 19:12, Piergiorgio Sartor wrote: > On Mon, Nov 18, 2013 at 11:08:59PM +0100, Andrea Mazzoleni wrote: > > Hi Andrea, > > great job, this was exactly what I was looking for. > > Do you know if there is a "fast" way not to correct > errors, but to find them? > > In RAID-6 (as per ra

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 20/11/13 19:09, John Williams wrote: > On Wed, Nov 20, 2013 at 2:31 AM, David Brown wrote: >> That's certainly a reasonable way to look at it. We should not limit >> the possibilities for high-end systems because of the limitations of >> low-end systems that are

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 20/11/13 19:34, Andrea Mazzoleni wrote: > Hi David, > >>> The choice of ZFS to use powers of 4 was likely not optimal, >>> because to multiply by 4, it has to do two multiplications by 2. >> I can agree with that. I didn't copy ZFS's choice here > David, it was not my intention to suggest that

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 21/11/13 02:28, Stan Hoeppner wrote: > On 11/20/2013 10:16 AM, James Plank wrote: >> Hi all -- no real comments, except as I mentioned to Ric, my tutorial >> in FAST last February presents Reed-Solomon coding with Cauchy >> matrices, and then makes special note of the common pitfall of >> assumi

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 20/11/13 22:59, Piergiorgio Sartor wrote: > On Wed, Nov 20, 2013 at 11:44:39AM +0100, David Brown wrote: > [...] >>> In RAID-6 (as per raid6check) there is an easy way >>> to verify where an HDD has incorrect data. >>> >> >> I think the way to do th

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 21/11/13 10:54, Adam Goryachev wrote: > On 21/11/13 20:07, David Brown wrote: >> I can see plenty of reasons why raid15 might be a good idea, and even >> raid16 for 5 disk redundancy, compared to multi-parity sets. However, >> it costs a lot in disk space. For example,

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 21/11/13 21:05, Piergiorgio Sartor wrote: > On Thu, Nov 21, 2013 at 11:13:29AM +0100, David Brown wrote: > [...] >> Ah, you are trying to find which disk has incorrect data so that you can >> change just that one disk? There are dangers with that... > > Hi David, >

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 21/11/13 21:52, Piergiorgio Sartor wrote: > Hi David, > > On Thu, Nov 21, 2013 at 09:31:46PM +0100, David Brown wrote: > [...] >> If this can all be done to give the user an informed choice, then it >> sounds good. > > that would be my target. > To _offer_ mo

Re: Triple parity and beyond

2013-11-21 Thread David Brown
On 22/11/13 01:30, Stan Hoeppner wrote: > I don't like it either. It's a compromise. But as RAID1/10 will soon > be unusable due to URE probability during rebuild, I think it's a > relatively good compromise for some users, some workloads. An alternative is to move to 3-way raid1 mirrors rather

Re: Triple parity and beyond

2013-11-22 Thread David Brown
On 22/11/13 09:13, Stan Hoeppner wrote: > Hi David, > > On 11/21/2013 3:07 AM, David Brown wrote: >> On 21/11/13 02:28, Stan Hoeppner wrote: > ... >>> WRT rebuild times, once drives hit 20TB we're looking at 18 hours just >>> to mirror a drive at f

Re: Triple parity and beyond

2013-11-22 Thread David Brown
On 22/11/13 09:38, Stan Hoeppner wrote: > On 11/21/2013 3:07 AM, David Brown wrote: > >> For example, with 20 disks at 1 TB each, you can have: > > All correct, and these are maximum redundancies. > > Maximum: > >> raid5 = 19TB, 1 disk redundancy >> rai

Re: Triple parity and beyond

2013-11-23 Thread David Brown
On 22/11/13 23:59, NeilBrown wrote: On Fri, 22 Nov 2013 10:07:09 -0600 Stan Hoeppner wrote: In the event of a double drive failure in one mirror, the RAID 1 code will need to be modified in such a way as to allow the RAID 5 code to rebuild the first replacement disk, because the RAID 1 devic

Re: Triple parity and beyond

2013-11-25 Thread David Brown
On 24/11/13 22:13, Stan Hoeppner wrote: > On 11/23/2013 11:14 PM, John Williams wrote: >> On Sat, Nov 23, 2013 at 8:03 PM, Stan Hoeppner >> wrote: >> >>> Parity array rebuilds are read-modify-write operations. The main >>> difference from normal operation RMWs is that the write is always to the

Re: Triple parity and beyond

2013-11-25 Thread David Brown
On 25/11/13 03:14, Russell Coker wrote: > On Mon, 25 Nov 2013, Stan Hoeppner wrote: >>> If that is the problem then the solution would be to just enable >>> read-ahead. Don't we already have that in both the OS and the disk >>> hardware? The hard- drive read-ahead buffer should at least cover the

Re: Triple parity and beyond

2013-11-28 Thread David Brown
On 28/11/13 08:16, Stan Hoeppner wrote: > Late reply. This one got lost in the flurry of activity... > > On 11/22/2013 7:24 AM, David Brown wrote: >> On 22/11/13 09:38, Stan Hoeppner wrote: >>> On 11/21/2013 3:07 AM, David Brown wrote: >>> >>>> For

Re: Can moving data to a subvolume not take as long as a fully copy?

2013-01-14 Thread David Brown
Marc MERLIN writes: > I made a mistake and copied data in the root of a new btrfs filesystem. > I created a subvolume, and used mv to put everything in there. > Something like: > cd /mnt > btrfs subvolume create dir > mv * dir > > Except it's been running for over a day now (ok, it's 5TB of data)

Re: 3.14.0rc3: did not find backref in send_root

2014-05-05 Thread David Brown
On Mon, Feb 24, 2014 at 10:36:52PM -0800, Marc MERLIN wrote: I got this during a btrfs send: BTRFS error (device dm-2): did not find backref in send_root. inode=22672, offset=524288, disk_byte=1490517954560 found extent=1490517954560 I'll try a scrub when I've finished my backup, but is there a

Re: 3.14.0rc3: did not find backref in send_root

2014-05-10 Thread David Brown
On Mon, May 05, 2014 at 11:10:54PM -0700, David Brown wrote: On Mon, Feb 24, 2014 at 10:36:52PM -0800, Marc MERLIN wrote: I got this during a btrfs send: BTRFS error (device dm-2): did not find backref in send_root. inode=22672, offset=524288, disk_byte=1490517954560 found extent=1490517954560

Re: 3.15-rc5 btrfs send/receive corruption errors? Does scrub

2014-05-10 Thread David Brown
On Sat, May 10, 2014 at 04:57:18PM -0700, Marc MERLIN wrote: On Fri, May 09, 2014 at 11:39:13AM -0700, Anacron wrote: /etc/cron.daily/btrfs-scrub: scrub device /dev/mapper/cryptroot (id 1) done scrub started at Fri May 9 06:09:14 2014 and finished after 19153 seconds total byt

Re: lsetxattr error when doing send/receive

2014-05-13 Thread David Brown
On Tue, May 13, 2014 at 08:44:44PM -0300, Bernardo Donadio wrote: Hi! I'm trying to do a send/receive of a snapshot between two disks on Fedora 20 with Linux 3.15-rc5 (and also tried with 3.14 and 3.11) and SELinux disabled, and then I'm receiving the following error: [root@darwin /]# btrfs

Re: lsetxattr error when doing send/receive

2014-05-14 Thread David Brown
On Wed, May 14, 2014 at 12:52:50AM -0600, Chris Murphy wrote: On May 13, 2014, at 7:57 PM, David Brown wrote: On Tue, May 13, 2014 at 08:44:44PM -0300, Bernardo Donadio wrote: Hi! I'm trying to do a send/receive of a snapshot between two disks on Fedora 20 with Linux 3.15-rc5 (and

Re: btrfs send/receive still gets out of sync in 3.14.0

2014-04-22 Thread David Brown
On Sat, Mar 22, 2014 at 02:04:56PM -0700, Marc MERLIN wrote: After deleting a huge directory tree in my /home subvolume, syncing snapshots now fails with: ERROR: rmdir o1952777-157-0 failed. No such file or directory Error line 156 with status 1 DIE: Code dump: 153 if [[ -n "$init" ]]; then

Content based storage

2010-03-16 Thread David Brown
Hi, I was wondering if there has been any thought or progress in content-based storage for btrfs beyond the suggestion in the "Project ideas" wiki page? The basic idea, as I understand it, is that a longer data extent checksum is used (long enough to make collisions unrealistic), and merge

Re: Content based storage

2010-03-17 Thread David Brown
On 16/03/2010 23:45, Fabio wrote: Some years ago I was searching for that kind of functionality and found an experimental ext3 patch to allow the so-called COW-links: http://lwn.net/Articles/76616/ I'd read about the COW patches for ext3 before. While there is certainly some similarity here,

Re: Content based storage

2010-03-17 Thread David Brown
On 17/03/2010 01:45, Hubert Kario wrote: On Tuesday 16 March 2010 10:21:43 David Brown wrote: Hi, I was wondering if there has been any thought or progress in content-based storage for btrfs beyond the suggestion in the "Project ideas" wiki page? The basic idea, as I understand it,

UUID of subvolumes

2010-04-08 Thread David Brown
he device number can change depending on what else might be used. - Any other ideas on a unique key I could use for a given subvolume to identify the files on that volume, even if it moves around? Thanks, David Brown -- To unsubscribe from this list: send the line "unsubscribe linux-bt

Re: [PATCH] fs/btrfs: Return EPERM for rmdir on subvolumes and snapshots

2010-04-08 Thread David Brown
On Thu, Apr 08, 2010 at 01:35:31PM -0700, Harshavardhana wrote: if (inode->i_size > BTRFS_EMPTY_DIR_SIZE || inode->i_ino == BTRFS_FIRST_FREE_OBJECTID) - return -ENOTEMPTY; + return -EPERM; Don't you want to still return ENOTEMPTY for the size ch

Re: Confused about resizing

2010-05-26 Thread David Brown
On 26/05/2010 10:41, David Pottage wrote: On Wed, May 26, 2010 3:46 am, Charlie Brune wrote: I think I'm not understanding something fundamental about btrfs: what am I able to resize? Resizing would be nice, given that it's so hard to do with ext3 (or even LVM). I created a btrfs filesystem o

Re: default subvolume abilities/restrictions

2010-06-12 Thread David Brown
On Sat, Jun 12, 2010 at 06:06:23PM -0500, C Anthony Risinger wrote: # btrfs subvolume create new_root # mv . new_root/old_root can i at least get confirmation that the above is possible? I've had no problem with # btrfs subvolume snapshot . new_root # mkdir old_root # mv * old_root

Re: Confused by performance

2010-06-16 Thread David Brown
On 16/06/2010 21:35, Freddie Cash wrote: That's all well and good, but you missed the part where he said ext2 on a 5-way LVM stripeset is many times faster than btrfs on a 5-way btrfs stripeset. IOW, same 5-way stripeset, different filesystems and volume managers, and very different performanc

Re: Hardlinks-per-directory limit?

2010-08-01 Thread David Brown
On Wednesday 28 July 2010, Ken D'Ambrosio said: > Hello, all. I'm thinking of rolling out a BackupPC server, and -- > based on the strength of the recent Phoronix benchmarks > (http://benchmarkreviews.com/index.php?option=com_content&task=view&id=11156&Itemid=23) > -- had been strongly considerin

Re: BTRFS && SSD

2010-09-30 Thread David Brown
On 29/09/2010 23:31, Yuehai Xu wrote: On Wed, Sep 29, 2010 at 3:59 PM, Sean Bartell wrote: On Wed, Sep 29, 2010 at 02:45:29PM -0400, Yuehai Xu wrote: On Wed, Sep 29, 2010 at 1:08 PM, Sean Bartell wrote: On Wed, Sep 29, 2010 at 11:30:14AM -0400, Yuehai Xu wrote: I know BTRFS is a kind of Log