Re: [3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free
Woops, just noticed I copied and pasted a typo there. Sorry for the trouble. It should be: Tested-by: Johannes Henninger <johannes+bt...@henninger.io> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup double free
Tested-by: Johannes Henninger <johannes+bt...@henniger.io> -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exclusive quota of snapshot exceeded despite no space used
On 27.10.2015 02:06, Qu Wenruo wrote:
>
>
> Johannes Henninger wrote on 2015/10/27 01:15 +0100:
>> On 26.10.2015 08:12, Qu Wenruo wrote:
>>>
>>>>>> Thanks a lot for your reply!
>>>>>>
>>>>>> While remounting the filesystem fixes the issue temporary, it
>>>>>> doesn't
>>>>>> take very long for the bug to happen again so it's not really a
>>>>>> workaround I can work with.
>>>>>>
>>>>>> I did recompile the kernel using your patches, but unfortunately the
>>>>>> problem still appears.
>>>>>>
>>>>>> Thanks,
>>>>>> Johannes
>>>>>>
>>>>> Interesting, just touching file will cause EQUOTA is quite a big
>>>>> problem.
>>>>>
>>>>> I'll try to reproduce it with my patchset and see what really caused
>>>>> the problem.
>>>>> The problem seems to do with snapshot qgroup hacking.
>>>>> But I'm not completely sure yet.
>>>>>
>>>>> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K?
>>>>> 16K is the correct number with only 6 empty files, just in case.
>>>>>
>>>>> Thanks,
>>>>> Qu
>>>>
>>>> I ran my example from the first mail again and managed to write 7
>>>> files
>>>> this time, "qgroup show" still shows 16kB after sync:
>>>>
>>>> root@t420:/media/extern/snap# btrfs qg limit -e 50M .
>>>> root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
>>>> sleep 5m; done
>>>> touch: cannot touch ‘8’: Disk quota exceeded
>>>> ^C
>>>> root@t420:/media/extern/snap# sync
>>>> root@t420:/media/extern/snap# btrfs qgroup show -pcre .
>>>> qgroupid rfer excl max_rfer max_excl parent
>>>> child
>>>> --
>>>> -
>>>> 0/5 16.00KiB 16.00KiB none none
>>>> --- ---
>>>> 0/25716.00KiB 16.00KiB none none
>>>> --- ---
>>>> 0/25816.00KiB 16.00KiB none 50.00MiB
>>>> --- ---
>>>> root@t420:/media/extern/snap# btrfs fi sync .
>>>> FSSync '.'
>>>> root@t420:/media/extern/snap# btrfs qgroup show -pcre .
>>>> qgroupid rfer excl max_rfer max_excl parent
>>>> child
>>>> --
>>>> -
>>>> 0/5 16.00KiB 16.00KiB none none
>>>> --- ---
>>>> 0/25716.00KiB 16.00KiB none none
>>>> --- ---
>>>> 0/25816.00KiB 16.00KiB none 50.00MiB
>>>> --- ---
>>>>
>>>> By the way, I don't if its relevant but the problem is not limited to
>>>> exclusive quotas, but also happens when setting a "referenced" limit
>>>> (qgroup limit without "-e").
>>>>
>>>> Thanks,
>>>> Johannes
>>>>
>>>
>>> The bug is located, and turns out to be quite a stupid problem caused
>>> by myself.
>>>
>>> I just forgot to include a cleanup patch during rebase AGAIN!!!
>>>
>>> You can apply the following patch to resolve it:
>>> [PATCH 3/3] btrfs: qgroup: Fix a rebase bug which will cause qgroup
>>> double free
>>>
>>> Or just apply the whole patchset:
>>> [4.4][PATCH 0/3] btrfs: Qgroup hotfix
>>>
>>> At least, with the patchset based on Chris' integration-4.4 branch, it
>>> succeeded in touching all the 100 files in my test box.
>>>
>>> Thanks,
>>> Qu
>>>
>>
>> It's working! Thank you so much for fixing this bug, you don't even know
>> how much this has helped me!
>>
>> Thanks!
>> Johannes
>> --
>> To unsubscribe from this list: send the line "unsubscribe
>> linux-btrfs" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at http://vger.kernel.org/majordomo-info.html
>>
> Glad to hear that.
>
> If it's working for you, it would be better to add a 'Tested-by' tag
> for the 3rd patch.
>
> Thanks,
> Qu
Sure! Is there anything I have to do? I'm a kernel and mailing list noob :)
Thanks,
Johannes
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: Exclusive quota of snapshot exceeded despite no space used
On 25.10.2015 02:44, Qu Wenruo wrote:
>
>
> 在 2015年10月23日 23:05, Johannes Henninger 写道:
>> On 23.10.2015 01:47, Qu Wenruo wrote:
>>> 在 2015年10月23日 04:38, Johannes Henninger 写道:
>>>> I'm having a weird problem with snapshots and exclusive quotas. After
>>>> creating a snapshot of a subvolume and setting an exclusive quota of
>>>> 50MB for the snapshot, everything seems to work fine. I can write
>>>> approximately 50MB before the quota kicks in.
>>>>
>>>> However, if I create a snapshot, set an exclusive quota and just wait
>>>> for some time, I suddenly cannot even create an empty file because I'm
>>>> getting a "quota exceeded" error. The time until the bug appears seems
>>>> to vary. During the waiting time, I'm changing neither the snapshot
>>>> nor
>>>> the original subvolume. "qgroup show -e" reports an exclusive use of
>>>> only a few kilobytes for the snapshot, which is nowhere near the
>>>> limit.
>>>>
>>>> Steps to reproduce (/media/extern is a fresh and empty btrfs
>>>> partition):
>>>>
>>>> Enable quota and create an empty subvolume:
>>>> root@t420:/media/extern# btrfs quota enable .
>>>> root@t420:/media/extern# btrfs subvolume create sub
>>>> Create subvolume './sub'
>>>>
>>>> Snapshot the subvolume and set a limit:
>>>> root@t420:/media/extern# btrfs subvolume snapshot sub snap
>>>> Create a snapshot of 'sub' in './snap'
>>>> root@t420:/media/extern# cd snap/
>>>> root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .
>>>>
>>>> Sometimes it takes "longer" for the quota to kick in, so I'm
>>>> touching a
>>>> file every 5 minutes here:
>>>>
>>>> root@t420:/media/extern/snap# for file in {1..100}; do touch
>>>> $file;
>>>> sleep 5m; done
>>>> touch: cannot touch ‘7’: Disk quota exceeded
>>>> ^C
>>>> root@t420:/media/extern/snap# btrfs qgroup show -e .
>>>> qgroupid rfer excl max_excl
>>>>
>>>> 0/5 16.00KiB 16.00KiB none
>>>> 0/25716.00KiB 16.00KiB none
>>>> 0/25816.00KiB 16.00KiB 50.00MiB
>>>>
>>>> Any idea why this happens?
>>> BTW, to make btrfs qgroup show work, it's better to call sync before
>>> qgroup show.
>>>
>>> It's a known bug that even after qgroup accounting rework, qgroup
>>> reserve still has bug and can cause reserved space to underflow,
>>> making such problem happen.
>>>
>>> For such case, btrfs qgroup show won't help as reserved space is not
>>> shown in the output.
>>>
>>> One workaround would be, umount the filesystem and mount again.
>>> Which will reset the underflow reserved space and work for sometime.
>>>
>>> If it's OK for you to recompile the kernel, you can try the following
>>> patchset:
>>> [PATCH v3 00/21] Rework btrfs qgroup reserved space framework
>>>
>>> Which should solve the problem.
>>>
>>> Thanks,
>>> Qu
>>>
>>
>> Thanks a lot for your reply!
>>
>> While remounting the filesystem fixes the issue temporary, it doesn't
>> take very long for the bug to happen again so it's not really a
>> workaround I can work with.
>>
>> I did recompile the kernel using your patches, but unfortunately the
>> problem still appears.
>>
>> Thanks,
>> Johannes
>>
> Interesting, just touching file will cause EQUOTA is quite a big problem.
>
> I'll try to reproduce it with my patchset and see what really caused
> the problem.
> The problem seems to do with snapshot qgroup hacking.
> But I'm not completely sure yet.
>
> BTW, does "sync; btrfs qgroup show -prce" still show excl as 16K?
> 16K is the correct number with only 6 empty files, just in case.
>
> Thanks,
> Qu
I ran my example from the first mail again and managed to write 7 files
this time, "qgroup show" still shows 16kB after sync:
root@t420:/media/extern/snap# btrfs qg limit -e 50M .
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘8’: Disk quota exceeded
^C
root@t420:/media/extern/snap# sync
root@t420:/media/extern/snap# btrfs qgr
Re: Exclusive quota of snapshot exceeded despite no space used
On 23.10.2015 01:47, Qu Wenruo wrote:
> 在 2015年10月23日 04:38, Johannes Henninger 写道:
>> I'm having a weird problem with snapshots and exclusive quotas. After
>> creating a snapshot of a subvolume and setting an exclusive quota of
>> 50MB for the snapshot, everything seems to work fine. I can write
>> approximately 50MB before the quota kicks in.
>>
>> However, if I create a snapshot, set an exclusive quota and just wait
>> for some time, I suddenly cannot even create an empty file because I'm
>> getting a "quota exceeded" error. The time until the bug appears seems
>> to vary. During the waiting time, I'm changing neither the snapshot nor
>> the original subvolume. "qgroup show -e" reports an exclusive use of
>> only a few kilobytes for the snapshot, which is nowhere near the limit.
>>
>> Steps to reproduce (/media/extern is a fresh and empty btrfs partition):
>>
>> Enable quota and create an empty subvolume:
>> root@t420:/media/extern# btrfs quota enable .
>> root@t420:/media/extern# btrfs subvolume create sub
>> Create subvolume './sub'
>>
>> Snapshot the subvolume and set a limit:
>> root@t420:/media/extern# btrfs subvolume snapshot sub snap
>> Create a snapshot of 'sub' in './snap'
>> root@t420:/media/extern# cd snap/
>> root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .
>>
>> Sometimes it takes "longer" for the quota to kick in, so I'm touching a
>> file every 5 minutes here:
>>
>> root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
>> sleep 5m; done
>> touch: cannot touch ‘7’: Disk quota exceeded
>> ^C
>> root@t420:/media/extern/snap# btrfs qgroup show -e .
>> qgroupid rfer excl max_excl
>>
>> 0/5 16.00KiB 16.00KiB none
>> 0/25716.00KiB 16.00KiB none
>> 0/25816.00KiB 16.00KiB 50.00MiB
>>
>> Any idea why this happens?
> BTW, to make btrfs qgroup show work, it's better to call sync before
> qgroup show.
>
> It's a known bug that even after qgroup accounting rework, qgroup
> reserve still has bug and can cause reserved space to underflow,
> making such problem happen.
>
> For such case, btrfs qgroup show won't help as reserved space is not
> shown in the output.
>
> One workaround would be, umount the filesystem and mount again.
> Which will reset the underflow reserved space and work for sometime.
>
> If it's OK for you to recompile the kernel, you can try the following
> patchset:
> [PATCH v3 00/21] Rework btrfs qgroup reserved space framework
>
> Which should solve the problem.
>
> Thanks,
> Qu
>
Thanks a lot for your reply!
While remounting the filesystem fixes the issue temporary, it doesn't
take very long for the bug to happen again so it's not really a
workaround I can work with.
I did recompile the kernel using your patches, but unfortunately the
problem still appears.
Thanks,
Johannes
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
Exclusive quota of snapshot exceeded despite no space used
I'm having a weird problem with snapshots and exclusive quotas. After
creating a snapshot of a subvolume and setting an exclusive quota of
50MB for the snapshot, everything seems to work fine. I can write
approximately 50MB before the quota kicks in.
However, if I create a snapshot, set an exclusive quota and just wait
for some time, I suddenly cannot even create an empty file because I'm
getting a "quota exceeded" error. The time until the bug appears seems
to vary. During the waiting time, I'm changing neither the snapshot nor
the original subvolume. "qgroup show -e" reports an exclusive use of
only a few kilobytes for the snapshot, which is nowhere near the limit.
Steps to reproduce (/media/extern is a fresh and empty btrfs partition):
Enable quota and create an empty subvolume:
root@t420:/media/extern# btrfs quota enable .
root@t420:/media/extern# btrfs subvolume create sub
Create subvolume './sub'
Snapshot the subvolume and set a limit:
root@t420:/media/extern# btrfs subvolume snapshot sub snap
Create a snapshot of 'sub' in './snap'
root@t420:/media/extern# cd snap/
root@t420:/media/extern/snap# btrfs qgroup limit -e 50M .
Sometimes it takes "longer" for the quota to kick in, so I'm touching a
file every 5 minutes here:
root@t420:/media/extern/snap# for file in {1..100}; do touch $file;
sleep 5m; done
touch: cannot touch ‘7’: Disk quota exceeded
^C
root@t420:/media/extern/snap# btrfs qgroup show -e .
qgroupid rfer excl max_excl
0/5 16.00KiB 16.00KiB none
0/25716.00KiB 16.00KiB none
0/25816.00KiB 16.00KiB 50.00MiB
Any idea why this happens?
Thanks,
Johannes
System info:
Linux t420 4.3.0-rc5 #1 SMP Tue Oct 13 13:21:02 CEST 2015 x86_64
GNU/Linux
Label: none uuid: 9551e3ca-1608-469c-9d8c-77b99ce0e8ec
Total devices 1 FS bytes used 816.00KiB
devid1 size 931.51GiB used 2.04GiB path /dev/sdb1
btrfs-progs v4.1.2
Data, single: total=8.00MiB, used=256.00KiB
System, DUP: total=8.00MiB, used=16.00KiB
System, single: total=4.00MiB, used=0.00B
Metadata, DUP: total=1.00GiB, used=544.00KiB
Metadata, single: total=8.00MiB, used=0.00B
GlobalReserve, single: total=16.00MiB, used=0.00B
[249174.151820] sdb: sdb1
[249184.387377] sdb: sdb1
[249184.573096] sdb: sdb1
[249184.656274] BTRFS: device fsid
9551e3ca-1608-469c-9d8c-77b99ce0e8ec devid 1 transid 3 /dev/sdb1
[249186.323915] sdb: sdb1
[249186.534505] sdb: sdb1
[249186.538420] sdb: sdb1
[249196.781978] BTRFS info (device sdb1): disk space caching is enabled
[249196.781986] BTRFS: has skinny extents
[249196.781990] BTRFS: flagging fs with big metadata feature
[249196.818164] BTRFS: creating UUID tree
[249202.311983] BTRFS info (device sdb1): qgroup scan completed
(inconsistency flag cleared)
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at http://vger.kernel.org/majordomo-info.html
