Re: btrfs device delete missing - Input/output error

2014-12-07 Thread Tomasz Chmielewski
On 2014-12-07 06:26, Chris Murphy wrote: On Sat, Dec 6, 2014 at 2:17 AM, Tomasz Chmielewski t...@virtall.com wrote: After we run again btrfs device delete missing /home, the newly created directory eventually (/home/backup/ma-int/weekly.tmp) is being detected as csum failed ino If

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Hi Shriramana! Am Sonntag, 7. Dezember 2014, 20:45:59 schrieb Shriramana Sharma: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many extents of data and metadata there are, and how many

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Shriramana Sharma
On Sun, Dec 7, 2014 at 9:03 PM, Martin Steigerwald mar...@lichtvoll.de wrote: I never read that the actual disk usage is unknown. But I read that the actual what is free is unknown. And there are several reasons for that: That is totally understood. But I guess when your alloc space is nearing

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Am Sonntag, 7. Dezember 2014, 16:33:37 schrieb Martin Steigerwald: What might be possible but still has the limitation of the fourth point above, would be a query: How much free space do you have *right* know, on this directory path, if I write with standard settings. But the only guarantee

Re: If btrfs-find-root doesn't find tree roots is the filesystem lost?

2014-12-07 Thread Sandy McArthur Jr
On Sun, Dec 7, 2014 at 12:50 AM, Sandy McArthur Jr sandy...@gmail.com wrote: Would the `btrfs check` output below indicate only small error and that a --repair is likely to succeed? It did. Thanks Qu and Duncan. dmesg suggests I have some scrubbing to do: [220283.369150] BTRFS: bdev /dev/sdh1

Re: abort device removal?

2014-12-07 Thread Marc MERLIN
On Fri, Dec 05, 2014 at 12:28:57PM -0500, moparisthebest wrote: Hello all, I had a 6-device array I added a 4tb device to last night and ran the command to remove a previous 4tb device that still worked fine overnight. Unfortunately, one of the OTHER devices completely failed while this

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Martin, I read that the actual what is free is unknown. And there are several reasons for that: 1) On a compressed filesystem you cannot know, but only estimate the compression ratio for future data. It is NOT the job of BTRFS, or ANY file-system, to try to prodict the future. The future

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Hugo Mills
On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashf...@whisperpc.com wrote: [snip] 3) From what I gathered it is planned to allow different raid / redundancy levels for different subvolumes. BTRFS can´t know beforehand where applications request to save future data, i.e. in which subvolume.

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Am Sonntag, 7. Dezember 2014, 10:20:27 schrieb ashf...@whisperpc.com: Martin, I read that the actual what is free is unknown. And there are several reasons for that: 1) On a compressed filesystem you cannot know, but only estimate the compression ratio for future data. It is NOT

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Am Sonntag, 7. Dezember 2014, 18:34:44 schrieb Hugo Mills: On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashf...@whisperpc.com wrote: [snip] 3) From what I gathered it is planned to allow different raid / redundancy levels for different subvolumes. BTRFS can´t know beforehand where

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Goffredo Baroncelli
On 12/07/2014 04:33 PM, Martin Steigerwald wrote: Hi Shriramana! Am Sonntag, 7. Dezember 2014, 20:45:59 schrieb Shriramana Sharma: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many

Re: Kernel lockup: fs/btrfs/delayed-inoce.c:1410 btrfs_assert_delayed_root_empty

2014-12-07 Thread Chris Murphy
On Sat, Dec 6, 2014 at 7:45 PM, Andrew Wade andrew.j.w...@gmail.com wrote: I've had repeated lockups as well; I can trigger them by attempting to copy files off of a damaged DVD. I saw the btrfs_assert_delayed_root_empty warning as well, but not during the lockup; instead I saw it on a boot

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashf...@whisperpc.com wrote: [snip] 3) From what I gathered it is planned to allow different raid / redundancy levels for different subvolumes. BTRFS can´t know beforehand where applications request to save future data, i.e. in which subvolume.

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Am Sonntag, 7. Dezember 2014, 10:20:27 schrieb ashf...@whisperpc.com: Martin, I read that the actual what is free is unknown. And there are several reasons for that: 1) On a compressed filesystem you cannot know, but only estimate the compression ratio for future data. It is NOT the

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
3.1) even in the case of a single disk filesystem, data and metadata have different profiles: the data chunk doesn't have any redundancy, so 64kb of data consume 64kb of disk space. The metadata chunks usually are stored as DUP, so 64kb of metadata consume 128kb on disk. Moreover you have to

[PATCH v2] Btrfs: fix fs corruption on transaction abort if device supports discard

2014-12-07 Thread Filipe Manana
When we abort a transaction we iterate over all the ranges marked as dirty in fs_info-freed_extents[0] and fs_info-freed_extents[1], clear them from those trees, add them back (unpin) to the free space caches and, if the fs was mounted with -o discard, perform a discard on those regions. Also,

Re: Fixing Btrfs Filesystem Full Problems typo?

2014-12-07 Thread Marc MERLIN
On Sun, Nov 23, 2014 at 07:52:29AM +, Duncan wrote: Right. So, why would you rebalance empty chunks or near empty chunks? Don't you want to rebalance almost full chunks first, and work you way to less and less full as needed? No, the closer to empty a chunk is, the more effect you can

Re: PROBLEM: #89121 BTRFS mixes up mounted devices with their snapshots

2014-12-07 Thread Konstantin
Anand Jain wrote on 02.12.2014 at 12:54: On 02/12/2014 19:14, Goffredo Baroncelli wrote: I further investigate this issue. MegaBrutal, reported the following issue: doing a lvm snapshot of the device of a mounted btrfs fs, the new snapshot device name replaces the name of the original

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Goffredo, So in case you have a raid1 filesystem on two disks; each disk has 300GB free; which is the free space that you expected: 300GB or 600GB and why ? You should see 300GB free. That's what you'll see with RAID-1 with a hardware RAID controller, and with MD RAID. Why would you expect

Re: PROBLEM: #89121 BTRFS mixes up mounted devices with their snapshots

2014-12-07 Thread Konstantin
Phillip Susi wrote on 02.12.2014 at 20:19: On 12/1/2014 4:45 PM, Konstantin wrote: The bug appears also when using mdadm RAID1 - when one of the drives is detached from the array then the OS discovers it and after a while (not directly, it takes several minutes) it appears under

Re: System/single + Metadata/single as leftover cruft of mkfs?

2014-12-07 Thread Qu Wenruo
Original Message Subject: Re: System/single + Metadata/single as leftover cruft of mkfs? From: Goffredo Baroncelli kreij...@inwind.it To: Shriramana Sharma samj...@gmail.com, linux-btrfs linux-btrfs@vger.kernel.org Date: 2014年12月05日 02:32 On 12/04/2014 02:53 PM, Shriramana

Re: [PATCH 1/5] Avoid to consider lvm snapshots when scanning devices.

2014-12-07 Thread Qu Wenruo
Original Message Subject: [PATCH 1/5] Avoid to consider lvm snapshots when scanning devices. From: Goffredo Baroncelli kreij...@gmail.com To: linux-btrfs@vger.kernel.org Date: 2014年12月05日 02:39 LVM snapshots create a problem to the btrfs devices management. BTRFS assumes that

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Qu Wenruo
Original Message Subject: Re: Why is the actual disk usage of btrfs considered unknowable? From: ashf...@whisperpc.com To: kreij...@inwind.it Date: 2014年12月08日 08:12 Goffredo, So in case you have a raid1 filesystem on two disks; each disk has 300GB free; which is the free

[PATCH v8 3/3] btrfs: fix suspicious RCU in BTRFS_IOC_DEV_INFO

2014-12-07 Thread Omar Sandoval
A naked read of the value of an RCU pointer isn't safe. Put the whole access in an RCU critical section, not just the pointer dereference. Signed-off-by: Omar Sandoval osan...@osandov.com --- fs/btrfs/ioctl.c | 10 +- 1 file changed, 5 insertions(+), 5 deletions(-) diff --git

[PATCH v8 0/3] Introduce RCU string API

2014-12-07 Thread Omar Sandoval
This patch series on top of v3.18 introduces the RCU string API and cleans up the wreckage of sparse warnings which follow from it (shown here from when the patch was briefly in the btrfs integration tree): On Thu, Nov 27, 2014 at 06:45:20AM +0800, kbuild test robot wrote: tree:

[PATCH v8 1/3] Move BTRFS RCU string to common library

2014-12-07 Thread Omar Sandoval
The RCU-friendly string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality but instead just moves the code and documents the existing API. Reviewed-by: Josh Triplett j...@joshtriplett.org Acked-by: Paul E. McKenney paul...@linux.vnet.ibm.com

[PATCH v8 2/3] btrfs: refactor btrfs_device-name updates

2014-12-07 Thread Omar Sandoval
The rcu_string API introduced some new sparse errors but also revealed existing ones. First of all, the name in struct btrfs_device should be annotated as __rcu to prevent unsafe reads. Additionally, updates should go through rcu_dereference_protected to make it clear what's going on. This

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Robert White
On 12/07/2014 07:15 AM, Shriramana Sharma wrote: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many extents of data and metadata there are, and how many bytes on disk these occcupy, no matter

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Chris Murphy
On Sun, Dec 7, 2014 at 11:34 AM, Hugo Mills h...@carfax.org.uk wrote: *Any* value shown here is going to be inaccurate, and whatever way round we show it, someone will complain. Yeah I'd suggest that for regular df command, when multiple device volumes exist, they're shown with ?? for Avail

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Robert White
On 12/07/2014 07:40 AM, Martin Steigerwald wrote: Well what would be possible I bet would be a kind of system call like this: I need to write 5 GB of data in 100 of files to /opt/mynewshinysoftware, can I do it *and* give me a guarentee I can. So like a more flexible fallocate approach as

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Martin, Excellent analysis. On 12/07/2014 07:40 AM, Martin Steigerwald wrote: So while the core problem isn't insoluble, in real life it is _not_ _worth_ _solving_. I agree. There is inadequate return on the investment. In addition, the number of corner cases increases dramatically,

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Zygo Blaxell
On Sun, Dec 07, 2014 at 08:45:59PM +0530, Shriramana Sharma wrote: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many extents of data and metadata there are, and how many bytes on disk

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Robert White
On 12/07/2014 10:20 PM, ashf...@whisperpc.com wrote: Martin, Excellent analysis. On 12/07/2014 07:40 AM, Martin Steigerwald wrote: So while the core problem isn't insoluble, in real life it is _not_ _worth_ _solving_. Your email quoting things is messed up... I wrote that analysis... 8-)