[PATCH 0/3] btrfs: fix RCU string sparse noise

2014-11-30 Thread Omar Sandoval
Hi everyone, These patches clean up the big stack of sparse RCU errors I introduced into the integration tree as reported by the kbuild test robot: On Thu, Nov 27, 2014 at 06:45:20AM +0800, kbuild test robot wrote: tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git

[PATCH 3/3] btrfs: refactor btrfs_device-name updates

2014-11-30 Thread Omar Sandoval
The rcu_string API introduced some new sparse errors but also revealed existing ones. First of all, the name in struct btrfs_device should be annotated as __rcu to prevent unsafe reads. Additionally, updates should go through rcu_dereference_protected to make it clear what's going on. This

[PATCH 2/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-11-30 Thread Omar Sandoval
A naked read of the value of an RCU pointer isn't safe. Put the whole access in an RCU critical section, not just the pointer dereference. Signed-off-by: Omar Sandoval osan...@osandov.com --- fs/btrfs/ioctl.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git

[PATCH 1/3] rcustring: clean up botched __rcu annotations

2014-11-30 Thread Omar Sandoval
The rcu_string returned by rcu_string_strdup isn't technically under RCU yet, and it makes more sense not to treat it as such. Additionally, an rcu_string passed to rcu_string_free should already be rcu_dereferenced and therefore not in the __rcu address space. Signed-off-by: Omar Sandoval

Re: Moving an entire subvol?

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 9:51 AM, Marc MERLIN m...@merlins.org wrote: So the Ubuntu Wiki BtrFS entry advises against using subvol set-default because it boots its kernel using root=subvol=@ and home as subvol=@home, and these two subvols are only present under the subvol with ID 5. But isn't

Re: Skript for backup btrfs on external HD

2014-11-30 Thread Jakob Schürz
subdirectories from the 4th coloumn (leave this coloumn blank, no auto-transfer to the external HD!!!) in this subdirectories you get subvolumes like @.20141130-115001.hourly_0 @home.20141130-115001.hourly_0 @var.20141130-115001.hourly_0 AND they are rotated automatically. And my script changes

pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Gour
Hello, recently I was migrating from Debian to openSUSE and wanted to make it smooth by dismantling my 2x1TB raid-1 array, install SUSE on the 1st disk, cp /home data, check everything is OK and then add 2nd disk containing Debian into raid-1 array. However, in orde to accomplish it I learnt

Re: root subvol id is 0 or 5?

2014-11-30 Thread Hugo Mills
On Sun, Nov 30, 2014 at 09:01:37AM +0530, Shriramana Sharma wrote: I am confused with this: should I call it the root subvol or top-level subvol or default subvol or doesn't it matter? Are all subvols equal, or some are more equal than others [hark to Orwell's Animal Farm ;-)]? I try to

Re: Skript for backup btrfs on external HD

2014-11-30 Thread Jakob Schürz
Am 2014-11-29 um 22:11 schrieb Marc MERLIN: On Sat, Nov 29, 2014 at 09:34:01PM +0100, Jakob Schürz wrote: Hi there! I made a script to do backup with btrfs on a external HD. You can see the function, how it works, and how it's to be used on my site

Re: Moving contents from one subvol to another

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 9:23 AM, Shriramana Sharma samj...@gmail.com wrote: Why should noCoW affect cp --reflink anyhow? I just created a 500 MiB file from /dev/urandom under a chattr +C-ed dir, and copied to another subvol using cp --reflink, and fi df still shows 500 MiB, not 1 GiB. Looks

Re: Change total in btrfs filesystem df output to alloc

2014-11-30 Thread Shriramana Sharma
Attached patch. On Sun, Nov 30, 2014 at 9:30 AM, Shriramana Sharma samj...@gmail.com wrote: On Sun, Aug 31, 2014 at 7:25 AM, Shriramana Sharma samj...@gmail.com wrote: Hello. There seem to be lots of questions in various forums re the output of btrfs fi df -- especially w.r.t. the usage of the

Re: root subvol id is 0 or 5?

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 5:29 PM, Hugo Mills h...@carfax.org.uk wrote: In the data structures on disk, it's 5. The kernel aliases 0 to mean subvolid 5. So why 5 and not just 0 which seems a logical choice? On top of this, one needs to alias 0 to 5! -- Shriramana Sharma ஶ்ரீரமணஶர்மா

Re: root subvol id is 0 or 5?

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 7:08 PM, Shriramana Sharma samj...@gmail.com wrote: So why 5 and not just 0 which seems a logical choice? On top of this, one needs to alias 0 to 5! Attached patch clarifying this in the documentation. (Should have done this with the previous mail. Sorry for multiple

Considerations in snapshotting and send/receive of nocow files?

2014-11-30 Thread Shriramana Sharma
Given that snapshotting effectively reduces the usefulness of nocow, I suppose the preferable model to snapshotting and send/receiving such files would be different than other files. Should nocow files (for me only VBox images) preferably be: 1) under a separate subvolume? 2) said subvol

Re: root subvol id is 0 or 5?

2014-11-30 Thread Hugo Mills
On Sun, Nov 30, 2014 at 07:08:51PM +0530, Shriramana Sharma wrote: On Sun, Nov 30, 2014 at 5:29 PM, Hugo Mills h...@carfax.org.uk wrote: In the data structures on disk, it's 5. The kernel aliases 0 to mean subvolid 5. So why 5 and not just 0 which seems a logical choice? On top of

Re: Running out of disk space during BTRFS_IOC_CLONE - rebalance doesn't help

2014-11-30 Thread Liu Bo
On Sun, Nov 30, 2014 at 08:29:42AM +0100, Guenther Starnberger wrote: I'm having an issue with a filesystem where I'm regularly running out of disk space during deduplication with bedup. Rebalancing does not help and the same issue occurs even after a full rebalance. Main use-case for this

Re: [PATCH 2/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-11-30 Thread Pranith Kumar
On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval osan...@osandov.com wrote: A naked read of the value of an RCU pointer isn't safe. Put the whole access in an RCU critical section, not just the pointer dereference. Signed-off-by: Omar Sandoval osan...@osandov.com You can use

Re: [PATCH 3/3] btrfs: refactor btrfs_device-name updates

2014-11-30 Thread Pranith Kumar
On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval osan...@osandov.com wrote: The rcu_string API introduced some new sparse errors but also revealed existing ones. First of all, the name in struct btrfs_device should be annotated as __rcu to prevent unsafe reads. Additionally, updates should go

[RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread Goffredo Baroncelli
Hi all, this patch provides a mount.btrfs helper for the mount command. A btrfs filesystem could span several disks. This helper scans all the partitions to discover all the disks required to mount a filesystem. So it would not necessary any-more to scan the partitions to mount a filesystem.

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Russell Coker
When the 2 disks have different data mdadm has no way of knowing which one is correct and has a 50% chance of overwriting good data. But BTRFS does checksums on all reads and solves the problem of corrupt data - as long as you don't have 2 corrupt sectors in matching blocks. -- Sent from my

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread Dimitri John Ledkov
Hello, On 30 November 2014 at 17:43, Goffredo Baroncelli kreij...@libero.it wrote: Hi all, this patch provides a mount.btrfs helper for the mount command. A btrfs filesystem could span several disks. This helper scans all the partitions to discover all the disks required to mount a

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread cwillu
In ubuntu, the initfs runs a btrfs dev scan, which should catch anything that would be missed there. On Sun, Nov 30, 2014 at 4:11 PM, Dimitri John Ledkov x...@debian.org wrote: Hello, On 30 November 2014 at 17:43, Goffredo Baroncelli kreij...@libero.it wrote: Hi all, this patch provides a

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Christoph Anton Mitterer
Agree with others about -C 256...-C sha256 is only three letters more ;) Ideally, sha2-256 would be used, since there will be (are) other versions of sha which have 256 bits size. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread Dimitri John Ledkov
On 30 November 2014 at 22:31, cwillu cwi...@cwillu.com wrote: In ubuntu, the initfs runs a btrfs dev scan, which should catch anything that would be missed there. I'm sorry, udev rule(s) is not sufficient in the initramfs-less case, as outlined. In case of booting with initramfs, indeed,

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Christoph Anton Mitterer
Agree with others about -C 256...-C sha256 is only three letters more ;) Ideally, sha2-256 would be used, since there will be (are) other versions of sha which have 256 bits size. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Dimitri John Ledkov
On 30 November 2014 at 22:59, Christoph Anton Mitterer cales...@scientia.net wrote: Agree with others about -C 256...-C sha256 is only three letters more ;) Ideally, sha2-256 would be used, since there will be (are) other versions of sha which have 256 bits size. Nope, we should use standard

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread cwillu
Sorry, misread initrdless as initramfs. In #btrfs, I usually say something like do you gain enough by not using an initfs for this to be worth the hassle?, but of course, that's not an argument against making mount smarter. On Sun, Nov 30, 2014 at 4:57 PM, Dimitri John Ledkov x...@debian.org

Re: Moving an entire subvol?

2014-11-30 Thread Marc MERLIN
On Sun, Nov 30, 2014 at 03:57:06PM +0530, Shriramana Sharma wrote: On Sun, Nov 30, 2014 at 9:51 AM, Marc MERLIN m...@merlins.org wrote: So the Ubuntu Wiki BtrFS entry advises against using subvol set-default because it boots its kernel using root=subvol=@ and home as subvol=@home, and

Re: Moving an entire subvol?

2014-11-30 Thread Chris Murphy
On Sat, Nov 29, 2014 at 8:31 PM, Shriramana Sharma samj...@gmail.com wrote: So the Ubuntu Wiki BtrFS entry advises against using subvol set-default because it boots its kernel using root=subvol=@ and home as subvol=@home, and these two subvols are only present under the subvol with ID 5. The

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Chris Murphy
On Sun, Nov 30, 2014 at 3:06 PM, Russell Coker russ...@coker.com.au wrote: When the 2 disks have different data mdadm has no way of knowing which one is correct and has a 50% chance of overwriting good data. But BTRFS does checksums on all reads and solves the problem of corrupt data - as

Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
[BACKGROUND] I'm trying to implement the function to repair missing inode item. Under that case, inode type must be salvaged(although it can be fallback to FILE). One case should be, if there is any dir_item/index or inode_ref refers the inode as parent, the type of that inode must be DIR.

Re: [PATCH 3/3] btrfs: refactor btrfs_device-name updates

2014-11-30 Thread Omar Sandoval
On Sun, Nov 30, 2014 at 10:26:43AM -0500, Pranith Kumar wrote: On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval osan...@osandov.com wrote: The rcu_string API introduced some new sparse errors but also revealed existing ones. First of all, the name in struct btrfs_device should be annotated

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Christoph Anton Mitterer
On Sun, 2014-11-30 at 23:05 +, Dimitri John Ledkov wrote: Nope, we should use standard names. Well I wouldn't know that there is really a standardised name in the sense that it tells it's mandatory. People use SHA2-xxx, SHA-xxx, SHAxxx and probably even more combinations. And just because

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Duncan
Qu Wenruo posted on Mon, 01 Dec 2014 09:58:27 +0800 as excerpted: [CRAZY IDEA] So why not using SQL to implement the btrfsck inode-record things? 2. Heavy dependency If use it, btrfs-progs will include RDBMS as the make and runtime dependency. Such low level progs depend on high

Re: [PATCH 2/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-11-30 Thread Omar Sandoval
On Sun, Nov 30, 2014 at 10:11:41AM -0500, Pranith Kumar wrote: On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval osan...@osandov.com wrote: A naked read of the value of an RCU pointer isn't safe. Put the whole access in an RCU critical section, not just the pointer dereference.

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Duncan 1i5t5.dun...@cox.net To: linux-btrfs@vger.kernel.org Date: 2014年12月01日 11:08 Qu Wenruo posted on Mon, 01 Dec 2014 09:58:27 +0800 as excerpted: [CRAZY IDEA] So why not

[PATCH v2] btrfs: remove empty fs_devices to prevent memory runout

2014-11-30 Thread Gui Hecheng
There is a global list @fs_uuids to keep @fs_devices object for each created btrfs. But when a btrfs becomes empty (all devices belong to it are gone), its @fs_devices remains in @fs_uuids list until module exit. If we keeps mkfs.btrfs on the same device again and again, all empty @fs_devices

Re: root subvol id is 0 or 5?

2014-11-30 Thread Duncan
Hugo Mills posted on Sun, 30 Nov 2014 13:53:28 + as excerpted: On Sun, Nov 30, 2014 at 07:08:51PM +0530, Shriramana Sharma wrote: On Sun, Nov 30, 2014 at 5:29 PM, Hugo Mills h...@carfax.org.uk wrote: In the data structures on disk, it's 5. The kernel aliases 0 to mean subvolid 5.

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Robert White
On 11/30/2014 05:58 PM, Qu Wenruo wrote: (why not use SQL to... suggestion) SQL, as in Structured Query Language, is _terrible_ for recursion. It expresses all of its elements in terms of set theory and really can only implement union and intersection of flat sets. Several companies offer

Re: [PATCH] fstests: add btrfs test to stress chunk allocation/removal and fstrim

2014-11-30 Thread Dave Chinner
On Wed, Nov 26, 2014 at 03:30:39PM +, Filipe Manana wrote: Stress btrfs' block group allocation and deallocation while running fstrim in parallel. Part of the goal is also to get data block groups deallocated so that new metadata block groups, using the same physical device space ranges,

Re: Considerations in snapshotting and send/receive of nocow files?

2014-11-30 Thread Duncan
Shriramana Sharma posted on Sun, 30 Nov 2014 19:17:42 +0530 as excerpted: Given that snapshotting effectively reduces the usefulness of nocow, I suppose the preferable model to snapshotting and send/receiving such files would be different than other files. Should nocow files (for me only

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Roman Mamedov
On Sun, 30 Nov 2014 12:11:47 +0100 Gour g...@atmarama.net wrote: However, I wonder if there are some 'cons' in having raid-1 partition under mdadm and not using native mirroring capabilities of btrfs fs? Pros: * mdadm RAID has much better read balancing; Btrfs reads are satisfied from

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Duncan
Qu Wenruo posted on Mon, 01 Dec 2014 11:24:50 +0800 as excerpted: The db file is mostly used in memory, only when the metadata is really really big, maybe when the fs tree's level is 7 or 8 we may need to use db file. So fscking the database in ordered to fsck the database isn't an issue.

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Robert White rwh...@pobox.com To: Qu Wenruo quwen...@cn.fujitsu.com, linux-btrfs linux-btrfs@vger.kernel.org Date: 2014年12月01日 12:03 On 11/30/2014 05:58 PM, Qu Wenruo wrote:

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Duncan 1i5t5.dun...@cox.net To: linux-btrfs@vger.kernel.org Date: 2014年12月01日 13:47 Qu Wenruo posted on Mon, 01 Dec 2014 11:24:50 +0800 as excerpted: The db file is mostly