The -c option of btrfs qgroup limit

2013-03-26 Thread Wang Shilong
Hello Koen, Although we offer '-c' option for btrfs qgroup limit, it has been not implemented yet. So until now, btrfs quota just limits the real space that writes to the space. Thanks, Wang -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of

Re: The -c option of btrfs qgroup limit

2013-03-26 Thread Jan Schmidt
Hi Koen, On 27.03.2013 00:36, Koen De Wit wrote: > The "btrfs qgroup limit" command has an option -c which means "limit amount > of data after compression". Whether this option is specified or not, I seem > to be able to write more well-compressible data than the specified limit. I > was expecting

Re: btrfs "stuck" on

2013-03-26 Thread Mike Dilger
I have a remarkably similar problem as Ask Bjørn Hansen. I have a 2TB btrfs filesystem on a SATA drive, on top of an msdos partition table, partition 1, without any form of RAID. I have been using it to make backups using rsync like this: 1) snapshot the previous backup subvolume 2) rsync into

Re: [PATCH 0/5] [RFC] RAID-level terminology change

2013-03-26 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 26/03/13 21:27, Brendan Hide wrote: > On 11/03/13 02:21, Roger Binns wrote: >> Why does all data have to be rewritten? Why does every piece of data >> have to have exactly the same storage parameters in terms of >> non-redundancy/performance/stri

Re: [PATCH 0/5] [RFC] RAID-level terminology change

2013-03-26 Thread Brendan Hide
Bit late - but that could be explored in future. The main downside I see with "automatic" redundancy/optimisation is the complexity it introduces. Likely this would be best served with user-space tools. On 11/03/13 02:21, Roger Binns wrote: On 10/03/13 15:04, Hugo Mills wrote: Given that this

[RFC] How to enable btrfs reserve block space from specific device?

2013-03-26 Thread Zhi Yong Wu
HI, When i work on btrfs hot relocation feature, i hit one question about block reservation. btrfs hot relocation need to reserve block space from specific devices such as SSD, but current btrfs reserving code doesn't differentiate if block space is reserved from SSD or HDD. In order to make b

The -c option of btrfs qgroup limit

2013-03-26 Thread Koen De Wit
All, The "btrfs qgroup limit" command has an option -c which means "limit amount of data after compression". Whether this option is specified or not, I seem to be able to write more well-compressible data than the specified limit. I was expecting that omitting the -c option would never allow me t

"No space left on device" although df reports only 55% in use

2013-03-26 Thread Clemens Eisserer
Hi, I am using a btrfs loopback mounted file with lzo-compression on Linux-3.7.9, and I ran into "No space left on device" messages, although df reports only 55% of space is used: # touch testfile touch: cannot touch `testfile': No space left on device # df Filesystem 1K-blocks Used Ava

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe
HI, Am 26.03.2013 20:38, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 01:22:20PM -0600, Stefan Priebe wrote: Hi, but when i transfer big files i see now this one: [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. [20368.821978] "echo 0 > /proc/sys/kernel/hung_task_timeout

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 01:22:20PM -0600, Stefan Priebe wrote: > Hi, > > but when i transfer big files i see now this one: > [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. > [20368.821978] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" > disables this message. > [203

[PATCH V2] btrfs: document mount options in Documentation/fs/btrfs.txt

2013-03-26 Thread Eric Sandeen
Document all current btrfs mount options. Signed-off-by: Eric Sandeen --- V2: * reflect that btrfs is no longer "new" ;) * make it clear that alloc_start is for each device * highlight potential perf impacts of -o discard * reword skip_balance docs to refer to "resume" diff --git a/Documentati

[PATCH] Btrfs: limit the global reserve to 512mb

2013-03-26 Thread Josef Bacik
A user reported a problem where he was getting early ENOSPC with hundreds of gigs of free data space and 6 gigs of free metadata space. This is because the global block reserve was taking up the entire free metadata space. This is ridiculous, we have infrastructure in place to throttle if we star

[PATCH] Btrfs: hold the ordered operations mutex when waiting on ordered extents

2013-03-26 Thread Josef Bacik
We need to hold the ordered_operations mutex while waiting on ordered extents since we splice and run the ordered extents list. We need to make sure anybody else who wants to wait on ordered extents does actually wait for them to be completed. This will keep us from bailing out of flushing in cas

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe
Hi, but when i transfer big files i see now this one: [20368.784736] INFO: task rsync:14911 blocked for more than 120 seconds. [20368.821978] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message. [20368.895140] rsync D 8160f580 0 14911 1 0x

[PATCH] Btrfs: fix space accounting for unlink and rename

2013-03-26 Thread Josef Bacik
We are way over-reserving for unlink and rename. Rename is just some random huge number and unlink accounts for tree log operations that don't actually happen during unlink, not to mention the tree log doesn't take from the trans block rsv anyway so it's completely useless. Thanks, Signed-off-by

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 01:05:36PM -0600, Stefan Priebe wrote: > Hi Josef, > > Am 26.03.2013 18:45, schrieb Josef Bacik: > >> Am 26.03.2013 16:25, schrieb Josef Bacik: > >>> On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG > >>> wrote: > Hi, > Am 26.03.2013 15:44,

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe
Hi Josef, Am 26.03.2013 18:45, schrieb Josef Bacik: Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount

Re: Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 12:25:44PM -0600, Szőts Ákos wrote: > If my memory servers me well, it was taken in unmounted state. Do you > want me to take an other one and compare the SHA1 values? Nope I found the bug, just a problem with the restore. Thanks, Josef -- To unsubscribe from this list: s

Re: Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Szőts Ákos
If my memory servers me well, it was taken in unmounted state. Do you want me to take an other one and compare the SHA1 values? -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.

Re:Re: Re: compression btrfs

2013-03-26 Thread yiletian
I think the biggest problem is how we can reclaim the space when the extent is a compressed one. In this case, we may need to read and decompress data in the extent, and then compress the valid range to generate a new extent. Is this process a performance killer? At 2013-03-27 02:03:57,"Josef Bac

Re: Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 09:59:57AM -0600, Szőts Ákos wrote: > The error was just a silly "permission denied" one. > > I uploaded the created image to www.morrohun.hu/temp/btrfs/btrfs.img > SHA1: 90ee61e0724700495d26678d58dc229e04a04cc4 > Please, write me when you downloaded and I'll delete it. >

Re: Re: compression btrfs

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 10:27:34AM -0600, yiletian wrote: > Yes, I use compress-force=zlib for my partition. > > Consider this scenario. > > We first write a file with size of 256KB. Assume all data is compressed to > 128KB size, > btrfs create a extent item in extent-tree to record the 128KB di

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 10:19:19AM -0600, Stefan Priebe wrote: > HI, > > > Am 26.03.2013 16:25, schrieb Josef Bacik: > > On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG > > wrote: > >> Hi, > >> Am 26.03.2013 15:44, schrieb Josef Bacik: > >> Am 26.03.2013 13:53, schrieb

Re: Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 09:59:57AM -0600, Szőts Ákos wrote: > The error was just a silly "permission denied" one. > > I uploaded the created image to www.morrohun.hu/temp/btrfs/btrfs.img > SHA1: 90ee61e0724700495d26678d58dc229e04a04cc4 > Please, write me when you downloaded and I'll delete it. >

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe
HI, Am 26.03.2013 16:25, schrieb Josef Bacik: On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /

Re: Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Szőts Ákos
The error was just a silly "permission denied" one. I uploaded the created image to www.morrohun.hu/temp/btrfs/btrfs.img SHA1: 90ee61e0724700495d26678d58dc229e04a04cc4 Please, write me when you downloaded and I'll delete it. I have the idea to run btrfsck from your tree on the file system again,

Re: [bug] mount and /proc/mounts disagrees

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 09:24:02AM -0600, anand jain wrote: > > 3.8.0+ #3 > > This happened after 'umount /btrfs' was interrupted by ctl-C > > # mount | egrep btrfs > /dev/mapper/mpathe on /btrfs type btrfs (rw,degraded) > > # cat /etc/mtab | egrep btrfs > /dev/mapper/mpathe /btrfs btrfs rw,deg

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 09:03:11AM -0600, Stefan Priebe - Profihost AG wrote: > Hi, > Am 26.03.2013 15:44, schrieb Josef Bacik: > Am 26.03.2013 13:53, schrieb Josef Bacik: > no - it's just mounted with mount -o noatime > > :~# cat /proc/mounts | grep btrfs > /dev/mapper/rai

[bug] mount and /proc/mounts disagrees

2013-03-26 Thread anand jain
3.8.0+ #3 This happened after 'umount /btrfs' was interrupted by ctl-C # mount | egrep btrfs /dev/mapper/mpathe on /btrfs type btrfs (rw,degraded) # cat /etc/mtab | egrep btrfs /dev/mapper/mpathe /btrfs btrfs rw,degraded 0 0 # cat /proc/mounts | egrep btrfs # umount /btrfs umount: /btrfs: not

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe - Profihost AG
Hi, Am 26.03.2013 15:44, schrieb Josef Bacik: Am 26.03.2013 13:53, schrieb Josef Bacik: no - it's just mounted with mount -o noatime :~# cat /proc/mounts | grep btrfs /dev/mapper/raid54tb1 /mnt btrfs rw,noatime,space_cache 0 0 >>> >>> Ok I think I see what's going on.

Re: Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 08:33:27AM -0600, Szőts Ákos wrote: > Dear list members, > > In my previous thread at > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg2.html > there was a space_cache kernel bug/panic on kernel 3.8. I could > successfully "fix" that with rebuilding the cach

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 07:49:16AM -0600, Stefan Priebe - Profihost AG wrote: > Hi Josef, > > > Am 26.03.2013 14:30, schrieb Josef Bacik: > > On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG > > wrote: > >> Hi Josef, > >> > >> Am 26.03.2013 13:53, schrieb Josef Bacik: > >>>

Kernel bug on mismatching generation_v2 in inode.c:835

2013-03-26 Thread Szőts Ákos
Dear list members, In my previous thread at http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg2.html there was a space_cache kernel bug/panic on kernel 3.8. I could successfully "fix" that with rebuilding the cache. But some files were missing/corrupted. So I booted a rescue CD with k

Re: Kernel BUG on corrupted space cache under 3.7 and 3.8

2013-03-26 Thread Szőts Ákos
I fixed my file system here, so I can't provide more details now. Anyway, I cloned your repository and successfully compiled the tree (except for the mkfs program). But when I try to issue the command "./btrfs-image -c9 /dev/sda7 /home/linux/image.img" I get the following: Could not open /dev/sda

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe - Profihost AG
Hi Josef, Am 26.03.2013 14:30, schrieb Josef Bacik: > On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG wrote: >> Hi Josef, >> >> Am 26.03.2013 13:53, schrieb Josef Bacik: >>> On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: Hi, output here:

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 06:55:23AM -0600, Stefan Priebe - Profihost AG wrote: > Hi Josef, > > Am 26.03.2013 13:53, schrieb Josef Bacik: > > On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: > >> Hi, > >> > >> output here: > >> [ 590.546162] returning enospc, space_info 3, size 0 rese

Re: [PATCH] btrfs: document mount options in Documentation/fs/btrfs.txt

2013-03-26 Thread Karel Zak
On Sat, Mar 23, 2013 at 01:45:36PM -0500, Eric Sandeen wrote: > The mount manpage is maintained in the util-linux pkg, > but it's not kernel-version specific, and the util-linux > maintainer does not have specific knowledge of all filesytem > options. Absolutely true. The util-linux maintainer is

Re: compression btrfs

2013-03-26 Thread Josef Bacik
On Mon, Mar 25, 2013 at 10:03:20PM -0600, lonat_fr...@163.com wrote: > Hi everyone, > >   I have used btrfs as a work partition with compression=zlib. The > compression ratio is not satisfied to me.  > So you probably want compress-force=zlib. With just compress we will bail out of the compres

Re: Kernel BUG on corrupted space cache under 3.7 and 3.8

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 05:37:35AM -0600, Szőts Ákos wrote: > Dear list members, > > Yesterday I was about to restart my computer and that's why I closed every > opened applications: three Eclipse instances, two other Java applications, 2 > web browsers etc. It involved a lot of IO operations an

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe - Profihost AG
Hi Josef, Am 26.03.2013 13:53, schrieb Josef Bacik: > On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: >> Hi, >> >> output here: >> [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush >> 2, flush_state 7 dumping space info >> [ 590.548280] space_info 4 has 6439

Re: No space left on device (28)

2013-03-26 Thread Josef Bacik
On Tue, Mar 26, 2013 at 01:45:42AM -0600, Stefan Priebe wrote: > Hi, > > output here: > [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush > 2, flush_state 7 dumping space info > [ 590.548280] space_info 4 has 6439292928 free, is full > [ 590.548283] space_info total=25748

Re: [PATCH 4/7] Btrfs: add a necessary check before assigning/removing qgroup relations

2013-03-26 Thread Wang Shilong
Hello Arne, I have sent a patch-set about btrfs quota, would you please review them for me. Any comment is welcome. Thanks, Wang > From: Wang Shilong > > This is one of most important reason why we introduce a mutex here, > the previous code dosen't check 'src' and 'dst' before assigning/remov

[PATCH 7/7] Btrfs: remove reduplicate check about quota_root in qgroup.c

2013-03-26 Thread Wang Shilong
From: Wang Shilong The check has been done just before starting/joining a transaction, so we don't need to check it again. Signed-off-by: Wang Shilong Reviewed-by: Miao Xie --- fs/btrfs/qgroup.c | 20 1 files changed, 0 insertions(+), 20 deletions(-) diff --git a/fs/bt

[PATCH 6/7] Btrfs: remove unnecessary spin_lock holding for find_qgroup_rb()

2013-03-26 Thread Wang Shilong
From: Wang Shilong Since mutex has been used for quota operations, we don't have to hold spin_lock when calling find_qgroup_rb. Signed-off-by: Wang Shilong Reviewed-by: Miao Xie --- fs/btrfs/qgroup.c | 15 ++- 1 files changed, 6 insertions(+), 9 deletions(-) diff --git a/fs/btr

[PATCH 4/7] Btrfs: add a necessary check before assigning/removing qgroup relations

2013-03-26 Thread Wang Shilong
From: Wang Shilong This is one of most important reason why we introduce a mutex here, the previous code dosen't check 'src' and 'dst' before assigning/removing a qgroup relations. Without the mutex lock, and checks may leads a race conditon. Signed-off-by: Wang Shilong Reviewed-by: Miao Xie -

[PATCH 5/7] Btrfs: fix missing check before btrfs_qgroup_inherit()

2013-03-26 Thread Wang Shilong
From: Wang Shilong The original code dosen't have necessary checks before doing qgroup_inherit. Fix it. Signed-off-by: Wang Shilong Reviewed-by: Miao Xie --- fs/btrfs/ioctl.c | 50 ++ 1 files changed, 50 insertions(+), 0 deletions(-) diff --g

[PATCH 3/7] Btrfs: add a necessary check before creating/destroying a qgroup

2013-03-26 Thread Wang Shilong
From: Wang Shilong This patch try to add a check before creating/destroying a qgroup. Signed-off-by: Wang Shilong Reviewed-by: Miao Xie --- fs/btrfs/ioctl.c | 41 - 1 files changed, 40 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/ioctl.c b/fs/

[PATCH 2/7] Btrfs: add a necessary check before updating qgroup limit

2013-03-26 Thread Wang Shilong
From: Wang Shilong Add a necessary check before updating qgroup limit, if the relative qgroup dosen't exsit, do not join transaction. Signed-off-by: Wang Shilong Reviewed-by: Miao Xie --- fs/btrfs/ioctl.c | 43 --- 1 files changed, 36 insertions(+), 7

[PATCH 1/7] Btrfs: introduce a mutex lock for btrfs quota operations

2013-03-26 Thread Wang Shilong
From: Wang Shilong The original code only has one spin_lock 'qgroup_lock' to protect quota configurations on memory, if we want to add a BTRFS_QGROUP_INFO_KEY, it will be added to Btree firstly and then update quota configuration on memory,this case works ok just because of Btrfs internal btree l

Kernel BUG on corrupted space cache under 3.7 and 3.8

2013-03-26 Thread Szőts Ákos
Dear list members, Yesterday I was about to restart my computer and that's why I closed every opened applications: three Eclipse instances, two other Java applications, 2 web browsers etc. It involved a lot of IO operations and before everything could settle down my system became unresponsive.

Re: No space left on device (28)

2013-03-26 Thread Stefan Priebe
Hi, output here: [ 590.546162] returning enospc, space_info 3, size 0 reserved 0, flush 2, flush_state 7 dumping space info [ 590.548280] space_info 4 has 6439292928 free, is full [ 590.548283] space_info total=25748307968, used=19308916736, pinned=0, reserved=32768, may_use=6438354944, re