Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread ashford
Original Message Subject: Re: Why is the actual disk usage of btrfs considered unknowable? From: ashf...@whisperpc.com To: kreij...@inwind.it Date: 2014年12月08日 08:12 Goffredo, So in case you have a raid1 filesystem on two disks; each disk has 300GB free; which

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread Chris Murphy
On Sun, Dec 7, 2014 at 1:32 PM, ashf...@whisperpc.com wrote: I disagree. My experiences with other file-systems, including ZFS, show that the most common solution is to just deliver to the user the actual amount of unused disk space. Anything else changes this known value into a guess or

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread Goffredo Baroncelli
On 12/08/2014 01:12 AM, ashf...@whisperpc.com wrote: Goffredo, So in case you have a raid1 filesystem on two disks; each disk has 300GB free; which is the free space that you expected: 300GB or 600GB and why ? You should see 300GB free. That's what you'll see with RAID-1 with a hardware

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread Martin Steigerwald
Hi, Am Sonntag, 7. Dezember 2014, 21:32:01 schrieb Robert White: On 12/07/2014 07:40 AM, Martin Steigerwald wrote: Well what would be possible I bet would be a kind of system call like this: I need to write 5 GB of data in 100 of files to /opt/mynewshinysoftware, can I do it *and*

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread Austin S Hemmelgarn
On 2014-12-08 09:47, Martin Steigerwald wrote: Hi, Am Sonntag, 7. Dezember 2014, 21:32:01 schrieb Robert White: On 12/07/2014 07:40 AM, Martin Steigerwald wrote: Well what would be possible I bet would be a kind of system call like this: I need to write 5 GB of data in 100 of files to

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread Martin Steigerwald
Am Montag, 8. Dezember 2014, 09:57:50 schrieb Austin S Hemmelgarn: On 2014-12-08 09:47, Martin Steigerwald wrote: Hi, Am Sonntag, 7. Dezember 2014, 21:32:01 schrieb Robert White: On 12/07/2014 07:40 AM, Martin Steigerwald wrote: Well what would be possible I bet would be a kind of

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-08 Thread Zygo Blaxell
On Mon, Dec 08, 2014 at 03:47:23PM +0100, Martin Steigerwald wrote: Am Sonntag, 7. Dezember 2014, 21:32:01 schrieb Robert White: On 12/07/2014 07:40 AM, Martin Steigerwald wrote: Almost full filesystems are their own reward. So you basically say that BTRFS with compression does not meet

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Hi Shriramana! Am Sonntag, 7. Dezember 2014, 20:45:59 schrieb Shriramana Sharma: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many extents of data and metadata there are, and how many

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Shriramana Sharma
On Sun, Dec 7, 2014 at 9:03 PM, Martin Steigerwald mar...@lichtvoll.de wrote: I never read that the actual disk usage is unknown. But I read that the actual what is free is unknown. And there are several reasons for that: That is totally understood. But I guess when your alloc space is nearing

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Am Sonntag, 7. Dezember 2014, 16:33:37 schrieb Martin Steigerwald: What might be possible but still has the limitation of the fourth point above, would be a query: How much free space do you have *right* know, on this directory path, if I write with standard settings. But the only guarantee

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Martin, I read that the actual what is free is unknown. And there are several reasons for that: 1) On a compressed filesystem you cannot know, but only estimate the compression ratio for future data. It is NOT the job of BTRFS, or ANY file-system, to try to prodict the future. The future

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Hugo Mills
On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashf...@whisperpc.com wrote: [snip] 3) From what I gathered it is planned to allow different raid / redundancy levels for different subvolumes. BTRFS can´t know beforehand where applications request to save future data, i.e. in which subvolume.

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Am Sonntag, 7. Dezember 2014, 10:20:27 schrieb ashf...@whisperpc.com: Martin, I read that the actual what is free is unknown. And there are several reasons for that: 1) On a compressed filesystem you cannot know, but only estimate the compression ratio for future data. It is NOT

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Martin Steigerwald
Am Sonntag, 7. Dezember 2014, 18:34:44 schrieb Hugo Mills: On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashf...@whisperpc.com wrote: [snip] 3) From what I gathered it is planned to allow different raid / redundancy levels for different subvolumes. BTRFS can´t know beforehand where

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Goffredo Baroncelli
On 12/07/2014 04:33 PM, Martin Steigerwald wrote: Hi Shriramana! Am Sonntag, 7. Dezember 2014, 20:45:59 schrieb Shriramana Sharma: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
On Sun, Dec 07, 2014 at 10:20:27AM -0800, ashf...@whisperpc.com wrote: [snip] 3) From what I gathered it is planned to allow different raid / redundancy levels for different subvolumes. BTRFS can´t know beforehand where applications request to save future data, i.e. in which subvolume.

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Am Sonntag, 7. Dezember 2014, 10:20:27 schrieb ashf...@whisperpc.com: Martin, I read that the actual what is free is unknown. And there are several reasons for that: 1) On a compressed filesystem you cannot know, but only estimate the compression ratio for future data. It is NOT the

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
3.1) even in the case of a single disk filesystem, data and metadata have different profiles: the data chunk doesn't have any redundancy, so 64kb of data consume 64kb of disk space. The metadata chunks usually are stored as DUP, so 64kb of metadata consume 128kb on disk. Moreover you have to

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Goffredo, So in case you have a raid1 filesystem on two disks; each disk has 300GB free; which is the free space that you expected: 300GB or 600GB and why ? You should see 300GB free. That's what you'll see with RAID-1 with a hardware RAID controller, and with MD RAID. Why would you expect

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Qu Wenruo
Original Message Subject: Re: Why is the actual disk usage of btrfs considered unknowable? From: ashf...@whisperpc.com To: kreij...@inwind.it Date: 2014年12月08日 08:12 Goffredo, So in case you have a raid1 filesystem on two disks; each disk has 300GB free; which is the free

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Robert White
On 12/07/2014 07:15 AM, Shriramana Sharma wrote: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many extents of data and metadata there are, and how many bytes on disk these occcupy, no matter

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Chris Murphy
On Sun, Dec 7, 2014 at 11:34 AM, Hugo Mills h...@carfax.org.uk wrote: *Any* value shown here is going to be inaccurate, and whatever way round we show it, someone will complain. Yeah I'd suggest that for regular df command, when multiple device volumes exist, they're shown with ?? for Avail

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Robert White
On 12/07/2014 07:40 AM, Martin Steigerwald wrote: Well what would be possible I bet would be a kind of system call like this: I need to write 5 GB of data in 100 of files to /opt/mynewshinysoftware, can I do it *and* give me a guarentee I can. So like a more flexible fallocate approach as

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread ashford
Martin, Excellent analysis. On 12/07/2014 07:40 AM, Martin Steigerwald wrote: So while the core problem isn't insoluble, in real life it is _not_ _worth_ _solving_. I agree. There is inadequate return on the investment. In addition, the number of corner cases increases dramatically,

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Zygo Blaxell
On Sun, Dec 07, 2014 at 08:45:59PM +0530, Shriramana Sharma wrote: IIUC: 1) btrfs fi df already shows the alloc-ed space and the space used out of that. 2) Despite snapshots, CoW and compression, the tree knows how many extents of data and metadata there are, and how many bytes on disk

Re: Why is the actual disk usage of btrfs considered unknowable?

2014-12-07 Thread Robert White
On 12/07/2014 10:20 PM, ashf...@whisperpc.com wrote: Martin, Excellent analysis. On 12/07/2014 07:40 AM, Martin Steigerwald wrote: So while the core problem isn't insoluble, in real life it is _not_ _worth_ _solving_. Your email quoting things is messed up... I wrote that analysis... 8-)