Re: Weird space issues (with more than plenty left; over 100GiB free on 465.63GiB RAID partition)

2015-11-23 Thread Duncan
Steven Hoff posted on Mon, 23 Nov 2015 19:31:43 -0700 as excerpted:

> BTRFS Community,
> 
> I seem to be having a bit of an issue. A comment on
> https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_get_.22
No_space_left_on_device.22_errors.2C_but_df_says_I.27ve_got_lots_of_space
> suggested reporting this to the mailing list.
> 
> Issue: BTRFS claims to be running out of space
> 
> Setup:
> /home: 2 500GB hard drives in MDADM RAID1 with BTRFS being the primary
> filesystem
> 
> Output of various commands:
> 
> $ sudo btrfs fi df
> /home Data, single: total=334.00GiB, used=333.84GiB
> System, DUP: total=32.00MiB, used=80.00KiB
> Metadata, DUP: total=1.50GiB, used=987.17MiB
> unknown, single: total=336.00MiB, used=0.00
> 
> $ sudo btrfs fi show /home
> Label: none  uuid: (private information removed)
>  Total devices 1 FS bytes used 334.80GiB
>  devid1 size 465.63GiB used 337.06GiB path /dev/md0p1
> Btrfs v3.12
> 
> $ df /home
> Filesystem 1K-blocks  Used Available Use% Mounted on
> /dev/md0p1 488252416 352418008 134988424  73% /home
> 
> 
> I would like to note that "total" in the following line:
> Data, single: total=334.00GiB, used=333.84GiB gets smaller whenever data
> is deleted or moved from /home. I'm not sure if that is by design, or if
> it is a bug. Either way, every once in a while applications will start
> closing unexpectedly complaining that there is no space left.


Critical information not provided: kernel version.  However, if it's 
anything as old as the 3.12 btrfs-progs userspace that's being reported, 
you're running seriously ancient versions, and an upgrade to something 
semi-recent is *strongly* recommended.

Normally, the kernel version is most critical, and running either a 
current release series (4.3), or an LTS (long term support) version no 
further back than the penultimate (next to latest, 3.18 as 4.1 is the 
latest LTS, but the currently in development 4.4 has already been 
announced as another LTS so people on 3.18 should be strongly considering 
an upgrade to 4.1 by now).

As for btrfs userspace, while it's generally not so critical, 3.12 is 
getting a bit ridiculously old now, and a good rule of thumb is to run at 
least the version matching that of your kernel series, since a userspace 
release of a particular version comes shortly after the kernel release, 
and is being coded at the same time, so they're designed with each other 
in mind.  Tho running a newer userspace shouldn't be a problem, either.  
Following that rule of thumb, if you're following the kernel rule as 
well, you'll be running at least a 3.18 series kernel, so should be 
running at least a 3.18 series userspace.  That will ensure at least that 
it doesn't get a ridiculously old as your current 3.12 userspace.

But you do seem to be running at least a newer kernel than your 
userspace, as otherwise you'd not be getting that unknown line in btrfs 
fi df.  FWIW, newer userspace correctly reports that as global reserve.

And while it doesn't yet report it as such, the global reserve (again, 
reported as unknown on your ancient userspace) actually comes from the 
metadata number, so can be added to it.  There's actually some very 
recent on-list discussion on this topic, suggesting either making the 
global reserve a sub-entry under metadata, or eliminating it entirely, 
simply adding the global reserve size to metadata's used figure.

If you do that, the metadata line becomes 1.5 gig size, 1.3 gig used.  As 
metadata chunks are nominally a quarter gig but your metadata is dup 
mode, so writing will be to two of them at once, you're into your last 
already allocated set of metadata chunks, and similarly, data usage is 
into its last data chunk (nominally 1 GiB in size).

So that means before you do much of anything, either data or metadata 
will run out on its existing allocation, and new chunks should be 
allocated.

Now according to btrfs fi show, you have plenty of unallocated space 
(device line, 337 gig allocated, of 465 gig size, so over a hundred gig 
of unallocated free space), so allocating fresh chunks shouldn't be a 
problem at all.

But evidently in your case, at least some of the time, those allocations 
are failing.  Certainly, in the past, that was known to happen with some 
versions.

But that would be kernel code, and you didn't post what kernel version 
you're running.

Assuming based on your positively ancient btrfs userspace and the above 
hints that your kernel version, while newer, isn't exactly current... the 
first thing to do would be to get a current 4.3 series kernel, at least 
for testing, and see if the problem goes away.  If so and you you want to 
return to an older version and you weren't already on one of them, try 
the latest release of either the 3.18 or 4.1 kernel series, and see if 
the bug is still gone or is there again.

Based on the results of the above, we'll know whether it's a current 
kernel bug, or corrected in current but still in the 

Weird space issues (with more than plenty left; over 100GiB free on 465.63GiB RAID partition)

2015-11-23 Thread Steven Hoff

BTRFS Community,

I seem to be having a bit of an issue. A comment on
https://btrfs.wiki.kernel.org/index.php/Problem_FAQ#I_get_.22No_space_left_on_device.22_errors.2C_but_df_says_I.27ve_got_lots_of_space
suggested reporting this to the mailing list.

Issue: BTRFS claims to be running out of space

Setup:
/home: 2 500GB hard drives in MDADM RAID1 with BTRFS being the primary 
filesystem


Output of various commands:

$ sudo btrfs fi df /home
Data, single: total=334.00GiB, used=333.84GiB
System, DUP: total=32.00MiB, used=80.00KiB
Metadata, DUP: total=1.50GiB, used=987.17MiB
unknown, single: total=336.00MiB, used=0.00

$ sudo btrfs fi show /home
Label: none  uuid: (private information removed)
Total devices 1 FS bytes used 334.80GiB
devid1 size 465.63GiB used 337.06GiB path /dev/md0p1
Btrfs v3.12

$ df /home
Filesystem 1K-blocks  Used Available Use% Mounted on
/dev/md0p1 488252416 352418008 134988424  73% /home


I would like to note that "total" in the following line:
Data, single: total=334.00GiB, used=333.84GiB
gets smaller whenever data is deleted or moved from /home. I'm not sure if
that is by design, or if it is a bug. Either way, every once in a while 
applications will

start closing unexpectedly complaining that there is no space left. I've run
sudo btrfs fi balance /home
with no discernible result (other than relocating ~300 blocks). I have a 
system
monitoring tool that updates usage stats on my hard drives once every 
couple
seconds. It shows no abnormal disk usage during the times when 
applications start

complaining about no room left.

Any thoughts/ideas/help would be appreciated.

Regards,

Lectrode

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html