Re: [PATCH 2/2][v2] blk-plug: don't flush nested plug lists

2015-04-08 Thread Dave Chinner
[ Sending again with a trimmed CC list to just the lists. Jeff - cc lists that large get blocked by mailing lists... ] On Tue, Apr 07, 2015 at 02:55:13PM -0400, Jeff Moyer wrote: The way the on-stack plugging currently works, each nesting level flushes its own list of I/Os. This can be less

snapshot space use

2015-04-08 Thread Russell Coker
# zfs list -t snapshot NAMEUSED AVAIL REFER MOUNTPOINT hetz0/be0-mail@2015-03-10 2.88G - 387G - hetz0/be0-mail@2015-03-11 1.12G - 388G - hetz0/be0-mail@2015-03-12 1.11G - 388G - hetz0/be0-mail@2015-03-13 1.19G - 388G -

Re: [PATCH] btrfs-progs: use local btrfs-image in leaf corruption test

2015-04-08 Thread Qu Wenruo
Currently this test uses the system btrfs-image. If there isn't a btrfs-image on $PATH, the test fails. The test should be using the locally compiled btrfs-image, not the system one. Good catch. Reviewed-by: Qu Wenruo quwen...@cn.fujitsu.com Thanks, Qu --- diff --git

Re: snapshot space use

2015-04-08 Thread Qu Wenruo
Hi, You can use btrfs quota feature to do it. Like this: # btrfs quota enable MNT_POINT # btrfs quota rescan -w MNT_POINT # btrfs qgroup show -prce MNT_POINT qgroupid rferexclmax_rfer max_excl parent child -- - 0/5 2248704 12288

[PATCH 8/9] btrfs: wait for delayed iputs on no space

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com btrfs will report no_space when we run following write and delete file loop: # FILE_SIZE_M=[ 75% of fs space ] # DEV=[ some dev ] # MNT=[ some dir ] # # mkfs.btrfs -f $DEV # mount -o nodatacow $DEV $MNT # for ((i = 0; i 100; i++)); do dd if=/dev/zero

[PATCH 3/9] btrfs: Adjust commit-transaction condition to avoid NO_SPACE more

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com If we have any chance to make a successful write, we should not give up. This patch adjust commit-transaction condition from: pinned = wanted to left + pinned = wanted Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 3 ++- 1

[PATCH 7/9] btrfs: Support busy loop of write and delete

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com Reproduce: while true; do dd if=/dev/zero of=/mnt/btrfs/file count=[75% fs_size] rm /mnt/btrfs/file done Then we can see above loop failed on NO_SPACE. It it long-term problem since very beginning, because delayed-iput after rm are not run. We

[PATCH 4/9] btrfs: Set relative data on clear btrfs_block_group_cache-pinned

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com Bug1: space_info-bytes_readonly was set to very large(negative) value in btrfs_remove_block_group(). Reason: Current code set block_group_cache-pinned = 0 in btrfs_delete_unused_bgs(), but above space was not counted to space_info-bytes_readonly.

[PATCH v2 0/9] btrfs: Fix no_space on dd and rm loop

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com This is v2 of resend-fix-no-space. Most of them are send in single patch, I resend them in patchset to make it easy to access. Notice that Btrfs: fix find_free_dev_extent() malfunction in case device tree has hole from Forrest Liu in:

Re: Are attachments allowed?

2015-04-08 Thread Jim Mills
I have read through them, and even checked my attachment (multipart MIME), but with only 2 parts, text, and attachment. It doesn't seem to go through. I checked the size, and it is smaller than 100 kB. I did the same email to the test account, autoans...@vger.kernel.org, and I got an answer.

[PATCH 9/9] btrfs: cleanup unused alloc_chunk varible

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com Remove int alloc_chunk in btrfs_check_data_free_space() for not necessary. Signed-off-by: Zhao Lei zhao...@cn.fujitsu.com --- fs/btrfs/extent-tree.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/extent-tree.c

[PATCH 5/9] btrfs: add WARN_ON() to check is space_info op current

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com space_info's value calculation is some complex and easy to cause bug, add WARN_ON() to help debug. Changelog v1-v2: Put WARN_ON()s under the ENOSPC_DEBUG mount option. Suggested by: David Sterba dste...@suse.cz Signed-off-by: Zhao Lei

[PATCH 6/9] btrfs: Fix NO_SPACE bug caused by delayed-iput

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com Steps to reproduce: while true; do dd if=/dev/zero of=/btrfs_dir/file count=[fs_size * 75%] rm /btrfs_dir/file sync done And we'll see dd failed because btrfs return NO_SPACE. Reason: Normally, btrfs_commit_transaction() call

[PATCH 2/9] btrfs: Fix tail space processing in find_free_dev_extent()

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com It is another reason for NO_SPACE case. When we found enough free space in loop and saved them to max_hole_start/size before, and tail space contains pending extent, origional innocent max_hole_start/size are reset in retry. As a result,

Re: Re: Re: Porting BTRFS to user space

2015-04-08 Thread 인정식
My goal is to make a native DFS, with native as contrast to overlay. It's about performance and features. 1. With overlay DFS, overlying system and underlying system do pretty much overlaping tasks: maintain/lookup meta-data, keep data/meta-data consistency, and etc. They are both

[PATCH] Btrfs: fill -last_trans for delayed inode in btrfs_fill_inode.

2015-04-08 Thread Dongsheng Yang
We need to fill inode when we found a node for it in delayed_nodes_tree. But we did not fill the -last_trans currently, it will cause the test of xfstest/generic/311 fail. Scenario of the 311 is shown as below: Problem: (1). test_fd = open(fname, O_RDWR|O_DIRECT) (2).

[PATCH 1/9] btrfs: fix condition of commit transaction

2015-04-08 Thread Zhaolei
From: Zhao Lei zhao...@cn.fujitsu.com Old code bypass commit transaction when we don't have enough pinned space, but another case is there exist freed bgs in current transction, it have possibility to make alloc_chunk success. This patch modify the condition to: if (have_free_bg ||

Re: [PATCH] btrfs: Add show_path function for btrfs_super_ops.

2015-04-08 Thread Qu Wenruo
Hi, all. Patch from long long ago. Anyone to review/comment/merge it? Thanks Qu Original Message Subject: [PATCH] btrfs: Add show_path function for btrfs_super_ops. From: Qu Wenruo quwen...@cn.fujitsu.com To: linux-btrfs@vger.kernel.org Date: 2014年07月21日 17:02 show_path()

Re: [PATCH 3/3] Btrfs: show subvol= and subvolid= in /proc/mounts

2015-04-08 Thread Qu Wenruo
Original Message Subject: [PATCH 3/3] Btrfs: show subvol= and subvolid= in /proc/mounts From: Omar Sandoval osan...@osandov.com To: Chris Mason c...@fb.com, Josef Bacik jba...@fb.com, David Sterba dste...@suse.cz, linux-btrfs@vger.kernel.org Date: 2015年04月08日 13:34

Re: [PATCH 2/3] Btrfs: unify subvol= and subvolid= mounting

2015-04-08 Thread Qu Wenruo
Original Message Subject: [PATCH 2/3] Btrfs: unify subvol= and subvolid= mounting From: Omar Sandoval osan...@osandov.com To: Chris Mason c...@fb.com, Josef Bacik jba...@fb.com, David Sterba dste...@suse.cz, linux-btrfs@vger.kernel.org Date: 2015年04月08日 13:34 Currently,

Re: Re: Porting BTRFS to user space

2015-04-08 Thread 인정식
Thank you for the advise. I am still wonder why there are same-name files in btrfs(kernel source) and btrfs-progs. They are quite many as follows. backref.{c, h} ctree.{c, h} dir-item.c disk-io.{c, h} extent_io.{c, h} extent-tree.c file.c file-item.c

Re: Porting BTRFS to user space

2015-04-08 Thread Austin S Hemmelgarn
On 2015-04-07 19:57, 인정식 wrote: Thank you for the information. I just found that btrfs-progs includes several files that seem modified from btrfs kernel source. I am not sure exactly what they are. Web pages say libbtrfs is to provide interface for apps that use btrfs. Why should there be

Re: [PATCH 2/3] Btrfs: unify subvol= and subvolid= mounting

2015-04-08 Thread Omar Sandoval
On Wed, Apr 08, 2015 at 02:06:14PM +0800, Qu Wenruo wrote: Original Message Subject: [PATCH 2/3] Btrfs: unify subvol= and subvolid= mounting From: Omar Sandoval osan...@osandov.com To: Chris Mason c...@fb.com, Josef Bacik jba...@fb.com, David Sterba dste...@suse.cz,

Re: [PATCH 2/3] Btrfs: unify subvol= and subvolid= mounting

2015-04-08 Thread Qu Wenruo
Original Message Subject: Re: [PATCH 2/3] Btrfs: unify subvol= and subvolid= mounting From: Omar Sandoval osan...@osandov.com To: Qu Wenruo quwen...@cn.fujitsu.com Date: 2015年04月08日 15:17 On Wed, Apr 08, 2015 at 02:06:14PM +0800, Qu Wenruo wrote: Original

Re: Porting BTRFS to user space

2015-04-08 Thread Chris Samuel
On Wed, 8 Apr 2015 12:27:57 PM Hugo Mills wrote: (NFS, Ceph, Gluster certainly work this way. I would be surprised if any of the others out there at the moment didn't work like that). GPFS has its own underlying filesystem format, you can upgrade the format on the fly with mmchfs once all your

Re: Porting BTRFS to user space

2015-04-08 Thread Stef Bon
Hi all, maybe interesting to know, I'm working on a backup service, which uses btrfs for making snapshots. The user can assign a folder for backup very easy, and create rules per backup of what to backup. Right now it works with assigning mimetypes to a backup: the files ot these mimetypes will

Re: Re: Porting BTRFS to user space

2015-04-08 Thread Hugo Mills
On Wed, Apr 08, 2015 at 12:03:29PM +, 인정식 wrote: Thank you for the advise. I am still wonder why there are same-name files in btrfs(kernel source) and btrfs-progs. They are quite many as follows. backref.{c, h} ctree.{c, h} dir-item.c disk-io.{c, h} extent_io.{c,

[PATCH] btrfs-progs: use local btrfs-image in leaf corruption test

2015-04-08 Thread WorMzy Tykashi
Currently this test uses the system btrfs-image. If there isn't a btrfs-image on $PATH, the test fails. The test should be using the locally compiled btrfs-image, not the system one. --- diff --git a/tests/fsck-tests/012-leaf-corruption/test.sh b/tests/fsck-tests/012-leaf-corruption/test.sh index

Re: Recovering BTRFS from bcache failure.

2015-04-08 Thread Dan Merillat
It's a known bug with bcache and enabling discard, it was discarding sections containing data it wanted. After a reboot bcache refused to accept the cache data, and of course it was dirty because I'm frankly too stupid to breathe sometimes. So yes, it's a bcache issue, but that's unresolvable.

Re: Recovering BTRFS from bcache failure.

2015-04-08 Thread Dan Merillat
Sorry I pressed send before I finished my thoughts. btrfs restore gets nowhere with any options. btrfs-recover says the superblocks are fine, and chunk recover does nothing after a few hours of reading. Everything else bails out with the errors I listed above. On Wed, Apr 8, 2015 at 2:36 PM,

Re: Recovering BTRFS from bcache failure.

2015-04-08 Thread Kai Krakow
Dan Merillat dan.meril...@gmail.com schrieb: Bcache failures are nasty, because they leave a mix of old and new data on the disk. In this case, there was very little dirty data, but of course the tree roots were dirty and out-of-sync. fileserver:/usr/src/btrfs-progs# ./btrfs --version

Re: Recovering BTRFS from bcache failure.

2015-04-08 Thread Dan Merillat
Any ideas on where to start with this? I did flush the cache out to disk before I made changes to the bcache configuration, so there shouldn't be anything completely missing, just some bits of stale metadata. If I can get the tools to take the closest match and run with it it would probably