[PATCH 0/3] btrfs: fix RCU string sparse noise

2014-11-30 Thread Omar Sandoval
Hi everyone, These patches clean up the big stack of sparse RCU errors I introduced into the integration tree as reported by the kbuild test robot: On Thu, Nov 27, 2014 at 06:45:20AM +0800, kbuild test robot wrote: > tree: git://git.kernel.org/pub/scm/linux/kernel/git/mason/linux-btrfs.git > i

[PATCH 3/3] btrfs: refactor btrfs_device->name updates

2014-11-30 Thread Omar Sandoval
The rcu_string API introduced some new sparse errors but also revealed existing ones. First of all, the name in struct btrfs_device should be annotated as __rcu to prevent unsafe reads. Additionally, updates should go through rcu_dereference_protected to make it clear what's going on. This introduc

[PATCH 2/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-11-30 Thread Omar Sandoval
A naked read of the value of an RCU pointer isn't safe. Put the whole access in an RCU critical section, not just the pointer dereference. Signed-off-by: Omar Sandoval --- fs/btrfs/ioctl.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/btrfs/

[PATCH 1/3] rcustring: clean up botched __rcu annotations

2014-11-30 Thread Omar Sandoval
The rcu_string returned by rcu_string_strdup isn't technically under RCU yet, and it makes more sense not to treat it as such. Additionally, an rcu_string passed to rcu_string_free should already be rcu_dereferenced and therefore not in the __rcu address space. Signed-off-by: Omar Sandoval --- i

Re: Moving an entire subvol?

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 9:51 AM, Marc MERLIN wrote: > >> So the Ubuntu Wiki BtrFS entry advises against using subvol >> set-default because it boots its kernel using root=subvol=@ and home >> as subvol=@home, and these two subvols are only present under the >> subvol with ID 5. But isn't it just p

Re: Skript for backup btrfs on external HD

2014-11-30 Thread Jakob Schürz
p", including also the three given subdirectories from the 4th coloumn (leave this coloumn blank, no auto-transfer to the external HD!!!) in this subdirectories you get subvolumes like @.20141130-115001.hourly_0 @home.20141130-115001.hourly_0 @var.20141130-115001.hourly_0 AND they are rotated

pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Gour
Hello, recently I was migrating from Debian to openSUSE and wanted to make it smooth by dismantling my 2x1TB raid-1 array, install SUSE on the 1st disk, cp /home data, check everything is OK and then add 2nd disk containing Debian into raid-1 array. However, in orde to accomplish it I learnt that

Re: root subvol id is 0 or 5?

2014-11-30 Thread Hugo Mills
On Sun, Nov 30, 2014 at 09:01:37AM +0530, Shriramana Sharma wrote: > I am confused with this: should I call it the "root subvol" or > "top-level subvol" or "default subvol" or doesn't it matter? Are all > subvols equal, or some are more equal than others [hark to Orwell's > Animal Farm ;-)]? I

Re: Skript for backup btrfs on external HD

2014-11-30 Thread Jakob Schürz
Am 2014-11-29 um 22:11 schrieb Marc MERLIN: On Sat, Nov 29, 2014 at 09:34:01PM +0100, Jakob Schürz wrote: Hi there! I made a script to do backup with btrfs on a external HD. You can see the function, how it works, and how it's to be used on my site http://linux.xundeenergie.at/doku.php?id=mkbtr

Re: Moving contents from one subvol to another

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 9:23 AM, Shriramana Sharma wrote: > > > Why should noCoW affect cp --reflink anyhow? I just created a 500 MiB > file from /dev/urandom under a chattr +C-ed dir, and copied to another > subvol using cp --reflink, and fi df still shows 500 MiB, not 1 GiB. Looks like I might

Re: Change "total" in btrfs filesystem df output to "alloc"

2014-11-30 Thread Shriramana Sharma
Attached patch. On Sun, Nov 30, 2014 at 9:30 AM, Shriramana Sharma wrote: > On Sun, Aug 31, 2014 at 7:25 AM, Shriramana Sharma wrote: >> Hello. There seem to be lots of questions in various forums re the >> output of btrfs fi df -- especially w.r.t. the usage of the word >> "total". For example

Re: root subvol id is 0 or 5?

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 5:29 PM, Hugo Mills wrote: > >In the data structures on disk, it's 5. The kernel aliases 0 to > mean subvolid 5. So why 5 and not just 0 which seems a logical choice? On top of this, one needs to alias 0 to 5! -- Shriramana Sharma ஶ்ரீரமணஶர்மா श्रीरमणशर्मा

Re: root subvol id is 0 or 5?

2014-11-30 Thread Shriramana Sharma
On Sun, Nov 30, 2014 at 7:08 PM, Shriramana Sharma wrote: > > So why 5 and not just 0 which seems a logical choice? On top of this, > one needs to alias 0 to 5! Attached patch clarifying this in the documentation. (Should have done this with the previous mail. Sorry for multiple mails.) -- Shri

Considerations in snapshotting and send/receive of nocow files?

2014-11-30 Thread Shriramana Sharma
Given that snapshotting effectively reduces the usefulness of nocow, I suppose the preferable model to snapshotting and send/receiving such files would be different than other files. Should nocow files (for me only VBox images) preferably be: 1) under a separate subvolume? 2) said subvol snapsho

Re: root subvol id is 0 or 5?

2014-11-30 Thread Hugo Mills
On Sun, Nov 30, 2014 at 07:08:51PM +0530, Shriramana Sharma wrote: > On Sun, Nov 30, 2014 at 5:29 PM, Hugo Mills wrote: > > > >In the data structures on disk, it's 5. The kernel aliases 0 to > > mean subvolid 5. > > So why 5 and not just 0 which seems a logical choice? On top of this, > one n

Re: Running out of disk space during BTRFS_IOC_CLONE - rebalance doesn't help

2014-11-30 Thread Liu Bo
On Sun, Nov 30, 2014 at 08:29:42AM +0100, Guenther Starnberger wrote: > I'm having an issue with a filesystem where I'm regularly running out of disk > space during deduplication with bedup. Rebalancing does not help and the same > issue occurs even after a full rebalance. > > Main use-case for th

Re: [PATCH 2/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-11-30 Thread Pranith Kumar
On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval wrote: > A naked read of the value of an RCU pointer isn't safe. Put the whole access > in > an RCU critical section, not just the pointer dereference. > > Signed-off-by: Omar Sandoval You can use rcu_access_pointer() in the if() condition check ra

Re: [PATCH 3/3] btrfs: refactor btrfs_device->name updates

2014-11-30 Thread Pranith Kumar
On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval wrote: > The rcu_string API introduced some new sparse errors but also revealed > existing > ones. First of all, the name in struct btrfs_device should be annotated as > __rcu to prevent unsafe reads. Additionally, updates should go through > rcu_der

[RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread Goffredo Baroncelli
Hi all, this patch provides a "mount.btrfs" helper for the mount command. A btrfs filesystem could span several disks. This helper scans all the partitions to discover all the disks required to mount a filesystem. So it would not necessary any-more to "scan" the partitions to mount a filesystem.

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Russell Coker
When the 2 disks have different data mdadm has no way of knowing which one is correct and has a 50% chance of overwriting good data. But BTRFS does checksums on all reads and solves the problem of corrupt data - as long as you don't have 2 corrupt sectors in matching blocks. -- Sent from my Sam

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread Dimitri John Ledkov
Hello, On 30 November 2014 at 17:43, Goffredo Baroncelli wrote: > Hi all, > > this patch provides a "mount.btrfs" helper for the mount command. > A btrfs filesystem could span several disks. This helper scans all the > partitions to discover all the disks required to mount a filesystem. > So it w

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread cwillu
In ubuntu, the initfs runs a btrfs dev scan, which should catch anything that would be missed there. On Sun, Nov 30, 2014 at 4:11 PM, Dimitri John Ledkov wrote: > Hello, > > On 30 November 2014 at 17:43, Goffredo Baroncelli wrote: >> Hi all, >> >> this patch provides a "mount.btrfs" helper for t

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Christoph Anton Mitterer
>Agree with others about -C 256...-C sha256 is only three >letters more ;) Ideally, sha2-256 would be used, since there will be (are) other versions of sha which have 256 bits size. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread Dimitri John Ledkov
On 30 November 2014 at 22:31, cwillu wrote: > > In ubuntu, the initfs runs a btrfs dev scan, which should catch > anything that would be missed there. > I'm sorry, udev rule(s) is not sufficient in the initramfs-less case, as outlined. In case of booting with initramfs, indeed, both Debian & Ubu

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Christoph Anton Mitterer
>Agree with others about -C 256...-C sha256 is only three >letters more ;) Ideally, sha2-256 would be used, since there will be (are) other versions of sha which have 256 bits size. Cheers, Chris. smime.p7s Description: S/MIME cryptographic signature

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Dimitri John Ledkov
On 30 November 2014 at 22:59, Christoph Anton Mitterer wrote: >>Agree with others about -C 256...-C sha256 is only three >>letters more ;) > > Ideally, sha2-256 would be used, since there will be (are) other > versions of sha which have 256 bits size. > Nope, we should use standard names. SHA-2 2

Re: [RFC][PATCH v2] mount.btrfs helper

2014-11-30 Thread cwillu
Sorry, misread "initrdless" as "initramfs". In #btrfs, I usually say something like "do you gain enough by not using an initfs for this to be worth the hassle?", but of course, that's not an argument against making mount smarter. On Sun, Nov 30, 2014 at 4:57 PM, Dimitri John Ledkov wrote: > On 3

Re: Moving an entire subvol?

2014-11-30 Thread Marc MERLIN
On Sun, Nov 30, 2014 at 03:57:06PM +0530, Shriramana Sharma wrote: > On Sun, Nov 30, 2014 at 9:51 AM, Marc MERLIN wrote: > > > >> So the Ubuntu Wiki BtrFS entry advises against using subvol > >> set-default because it boots its kernel using root=subvol=@ and home > >> as subvol=@home, and these tw

Re: Moving an entire subvol?

2014-11-30 Thread Chris Murphy
On Sat, Nov 29, 2014 at 8:31 PM, Shriramana Sharma wrote: > So the Ubuntu Wiki BtrFS entry advises against using subvol > set-default because it boots its kernel using root=subvol=@ and home > as subvol=@home, and these two subvols are only present under the > subvol with ID 5. The advice may hav

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Chris Murphy
On Sun, Nov 30, 2014 at 3:06 PM, Russell Coker wrote: > When the 2 disks have different data mdadm has no way of knowing which one is > correct and has a 50% chance of overwriting good data. But BTRFS does > checksums on all reads and solves the problem of corrupt data - as long as > you don't

Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
[BACKGROUND] I'm trying to implement the function to repair missing inode item. Under that case, inode type must be salvaged(although it can be fallback to FILE). One case should be, if there is any dir_item/index or inode_ref refers the inode as parent, the type of that inode must be DIR. Howev

Re: [PATCH 3/3] btrfs: refactor btrfs_device->name updates

2014-11-30 Thread Omar Sandoval
On Sun, Nov 30, 2014 at 10:26:43AM -0500, Pranith Kumar wrote: > On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval wrote: > > The rcu_string API introduced some new sparse errors but also revealed > > existing > > ones. First of all, the name in struct btrfs_device should be annotated as > > __rcu t

Re: [RFC PATCH] Btrfs: add sha256 checksum option

2014-11-30 Thread Christoph Anton Mitterer
On Sun, 2014-11-30 at 23:05 +, Dimitri John Ledkov wrote: > Nope, we should use standard names. Well I wouldn't know that there is really a standardised name in the sense that it tells it's mandatory. People use SHA2-xxx, SHA-xxx, SHAxxx and probably even more combinations. And just because s

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Duncan
Qu Wenruo posted on Mon, 01 Dec 2014 09:58:27 +0800 as excerpted: > [CRAZY IDEA] > So why not using SQL to implement the btrfsck inode-record things? > 2. Heavy dependency > If use it, btrfs-progs will include RDBMS as the make and runtime > dependency. Such low level progs depend on hig

Re: [PATCH 2/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-11-30 Thread Omar Sandoval
On Sun, Nov 30, 2014 at 10:11:41AM -0500, Pranith Kumar wrote: > On Sun, Nov 30, 2014 at 3:26 AM, Omar Sandoval wrote: > > A naked read of the value of an RCU pointer isn't safe. Put the whole > > access in > > an RCU critical section, not just the pointer dereference. > > > > Signed-off-by: Omar

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Duncan <1i5t5.dun...@cox.net> To: Date: 2014年12月01日 11:08 Qu Wenruo posted on Mon, 01 Dec 2014 09:58:27 +0800 as excerpted: [CRAZY IDEA] So why not using SQL to implement the

[PATCH v2] btrfs: remove empty fs_devices to prevent memory runout

2014-11-30 Thread Gui Hecheng
There is a global list @fs_uuids to keep @fs_devices object for each created btrfs. But when a btrfs becomes "empty" (all devices belong to it are gone), its @fs_devices remains in @fs_uuids list until module exit. If we keeps mkfs.btrfs on the same device again and again, all empty @fs_devices pro

Re: root subvol id is 0 or 5?

2014-11-30 Thread Duncan
Hugo Mills posted on Sun, 30 Nov 2014 13:53:28 + as excerpted: > On Sun, Nov 30, 2014 at 07:08:51PM +0530, Shriramana Sharma wrote: >> On Sun, Nov 30, 2014 at 5:29 PM, Hugo Mills wrote: >> > >> >In the data structures on disk, it's 5. The kernel aliases 0 to >> > mean subvolid 5. >> >> S

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Robert White
On 11/30/2014 05:58 PM, Qu Wenruo wrote: ("why not use SQL to..." suggestion) SQL, as in Structured Query Language, is _terrible_ for recursion. It expresses all of its elements in terms of set theory and really can only implement union and intersection of flat sets. Several companies offer

Re: [PATCH] fstests: add btrfs test to stress chunk allocation/removal and fstrim

2014-11-30 Thread Dave Chinner
On Wed, Nov 26, 2014 at 03:30:39PM +, Filipe Manana wrote: > Stress btrfs' block group allocation and deallocation while running > fstrim in parallel. Part of the goal is also to get data block groups > deallocated so that new metadata block groups, using the same physical > device space ranges

Re: Considerations in snapshotting and send/receive of nocow files?

2014-11-30 Thread Duncan
Shriramana Sharma posted on Sun, 30 Nov 2014 19:17:42 +0530 as excerpted: > Given that snapshotting effectively reduces the usefulness of nocow, I > suppose the preferable model to snapshotting and send/receiving such > files would be different than other files. > > Should nocow files (for me onl

Re: pro/cons of raid1 with mdadm/lvm2

2014-11-30 Thread Roman Mamedov
On Sun, 30 Nov 2014 12:11:47 +0100 Gour wrote: > However, I wonder if there are some 'cons' in having raid-1 partition > under mdadm and not using native mirroring capabilities of btrfs fs? Pros: * mdadm RAID has much better read balancing; Btrfs reads are satisfied from what's in effect

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Duncan
Qu Wenruo posted on Mon, 01 Dec 2014 11:24:50 +0800 as excerpted: > The db file is mostly used in memory, only when the metadata is really > really big, maybe when the fs tree's level is 7 or 8 we may need to use > db file. So fscking the database in ordered to fsck the database isn't an issue.

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Robert White To: Qu Wenruo , linux-btrfs Date: 2014年12月01日 12:03 On 11/30/2014 05:58 PM, Qu Wenruo wrote: ("why not use SQL to..." suggestion) SQL, as in Structured Query

Re: Crazy idea of cleanup the inode_record btrfsck things with SQL?

2014-11-30 Thread Qu Wenruo
Original Message Subject: Re: Crazy idea of cleanup the inode_record btrfsck things with SQL? From: Duncan <1i5t5.dun...@cox.net> To: Date: 2014年12月01日 13:47 Qu Wenruo posted on Mon, 01 Dec 2014 11:24:50 +0800 as excerpted: The db file is mostly used in memory, only when the