BTRFS critical (device dm-0): invalid dir item name len: 45389

2014-09-03 Thread john terragon
Hi. When I traverse one of my btrfs, for example with a simple "find /", I get the following in kmsg BTRFS critical (device dm-0): invalid dir item name len: 45389 The message appears just one time (so I guess it involves just one file/dir). dm-0 is the first dmcrypt device of a pair on which I

[PATCH] btrfs-progs: remove wrong set_argv0 for restore

2014-09-03 Thread Gui Hecheng
Before this patch, you could see the following after exec restore # :too few arguments The tool name "btrfs restore" is missing. The @set_argv0() function is introduced by: commit a184abc70f7b1468e6036ab576f1587ee0574668 btrfs-progs: move the check_argc_* functions into ut

[PATCH] btrfs-progs: remove btrfs_release_path before btrfs_free_path

2014-09-03 Thread Gui Hecheng
The btrfs_free_path calls btrfs_release_path internally. Signed-off-by: Gui Hecheng --- disk-io.c | 1 - file-item.c | 1 - inode-map.c | 2 -- 3 files changed, 4 deletions(-) diff --git a/disk-io.c b/disk-io.c index 9e44f10..0f9f374 100644 --- a/disk-io.c +++ b/disk-io.c @@ -628,7 +628,6 @@

Re: fs corruption report

2014-09-03 Thread Gui Hecheng
On Mon, 2014-09-01 at 15:25 +, Zooko Wilcox-OHearn wrote: > I'm more than happy to try out patches and even focus my own brain on > diagnosing it, if I can. I'm hoping to regain access to some of my > files on my btrfs partition, and also I would enjoy helping get this > improved. :-) > > So i

[PATCH] btrfs-progs: fix find_mount_root() to handle duplicated mount point correctly

2014-09-03 Thread Qu Wenruo
Original find_mount_root() will use the first mount point match and return it. It was OK until the following commit, which will also check the fstype: de22c28ef31d9721606ba059 btrfs-progs: Check fstype in find_mount_root() With fstype check, we should check the last match, not only the first one.

Re: Btrfs stable updates for v3.16

2014-09-03 Thread Chris Mason
On 09/03/2014 07:36 PM, Holger Hoffstätte wrote: > On Wed, 03 Sep 2014 16:50:47 -0400, Chris Mason wrote: > >> Hi everyone, >> >> For 3.16, please pull these into stable, I've cherry picked and tested >> them here. For 3.15 and earlier there are a few conflicts, so I'll make >> a git tree with

Re: Btrfs stable updates for v3.16

2014-09-03 Thread Holger Hoffstätte
On Wed, 03 Sep 2014 16:50:47 -0400, Chris Mason wrote: > Hi everyone, > > For 3.16, please pull these into stable, I've cherry picked and tested > them here. For 3.15 and earlier there are a few conflicts, so I'll make > a git tree with things to pull. > > 8d875f95da43c6a8f18f77869f2ef26e9594fe

Re: INFO: task btrfs-transacti:2408 blocked for more than 120 seconds.

2014-09-03 Thread Duncan
Martin Steigerwald posted on Thu, 04 Sep 2014 00:02:03 +0200 as excerpted: > Am Mittwoch, 3. September 2014, 19:17:17 schrieben Sie: >> At a 32 bit stable Gentoo Linux I do have 2 BTRFS file systems : >> >> $ mount | grep btrfs /var/lib/portage.fs on /usr/portage type btrfs >> (rw,noatime,compres

Re: INFO: task btrfs-transacti:2408 blocked for more than 120 seconds.

2014-09-03 Thread Martin Steigerwald
Am Mittwoch, 3. September 2014, 19:17:17 schrieben Sie: > At a 32 bit stable Gentoo Linux I do have 2 BTRFS file systems : > > $ mount | grep btrfs > /var/lib/portage.fs on /usr/portage type btrfs (rw,noatime,compress=lzo) > /var/lib/pkg.fs on /var/db/pkg type btrfs (rw,noatime,compress=lzo) > >

Re: Btrfs stable updates for v3.16

2014-09-03 Thread Greg KH
On Wed, Sep 03, 2014 at 04:50:47PM -0400, Chris Mason wrote: > Hi everyone, > > For 3.16, please pull these into stable, I've cherry picked and tested > them here. For 3.15 and earlier there are a few conflicts, so I'll make > a git tree with things to pull. > > 8d875f95da43c6a8f18f77869f2ef26e9

Re: Btrfs stable updates for 3.16.x (and others)

2014-09-03 Thread Greg KH
On Tue, Aug 19, 2014 at 01:10:45PM +0200, David Sterba wrote: > Hi stable team, > > please add the following patches to stable trees. > > Patch #3 applies to all currently live stables, a 7 years old bug. I've > briefly reviewed all 3 patches against 3.10/12/14/16 (ie. 3.4 skips #1 > and #2). >

Btrfs stable updates for v3.16

2014-09-03 Thread Chris Mason
Hi everyone, For 3.16, please pull these into stable, I've cherry picked and tested them here. For 3.15 and earlier there are a few conflicts, so I'll make a git tree with things to pull. 8d875f95da43c6a8f18f77869f2ef26e9594fecc v3.15+ 38c1c2e44bacb37efd68b90b3f70386a8ee370ee v3.11+ f6dc45c7a93a

Re: Large files, nodatacow and fragmentation

2014-09-03 Thread Clemens Eisserer
Hi Richard, > It is interesting that for me the number of extents before and after > bcache are essentially the same. > > The lesson here for me there is that the fragmentation of a btrfs > nodatacow file is not mitigated by bcache. There seems to be nothing I > can do to prevent that fragmentatio

kernel BUG at fs/btrfs/extent-tree.c:7727! with 3.17-rc3

2014-09-03 Thread Tomasz Chmielewski
Got the following with 3.17-rc3 and running balance (had to power cycle after that): [ 1329.952600] [ cut here ] [ 1329.952671] WARNING: CPU: 7 PID: 3106 at fs/btrfs/extent-tree.c:876 btrfs_lookup_extent_info+0x377/0x3eb [btrfs]() [ 1329.952726] Modules linked in: ipt_MA

INFO: task btrfs-transacti:2408 blocked for more than 120 seconds.

2014-09-03 Thread Toralf Förster
At a 32 bit stable Gentoo Linux I do have 2 BTRFS file systems : $ mount | grep btrfs /var/lib/portage.fs on /usr/portage type btrfs (rw,noatime,compress=lzo) /var/lib/pkg.fs on /var/db/pkg type btrfs (rw,noatime,compress=lzo) holding a lot of small Gentoo-package-Manager-related files. The first

Re: Large files, nodatacow and fragmentation

2014-09-03 Thread G. Richard Bellamy
It is interesting that for me the number of extents before and after bcache are essentially the same. The lesson here for me there is that the fragmentation of a btrfs nodatacow file is not mitigated by bcache. There seems to be nothing I can do to prevent that fragmentation, and may in fact be ex

Re: kernel 3.17-rc3: task rsync:2524 blocked for more than 120 seconds

2014-09-03 Thread Chris Murphy
On Sep 3, 2014, at 8:11 AM, john terragon wrote: > It's a usb2 device but doesn't it seem kind of slow? Not atypical, I have one that's the same, and another that's ~21MB/s, both are USB 2. [Certain older Apple Mac firmware boot faster with the slow stick than the fast one, and it turns out

Re: kernel 3.17-rc3: task rsync:2524 blocked for more than 120 seconds

2014-09-03 Thread john terragon
I wasn't sure what you meant with so I dd'd all the three possible cases: 1) here's the dmcrypt device on which I mkfs.btrfs 2097152000 bytes (2.1 GB) copied, 487.265 s, 4.3 MB/s 2) here's the partition of the usb stick (which has another partition containing /boot) on top of which the dmcry

[PATCH 18/18] Btrfs: modify rw_devices counter under chunk_mutex context

2014-09-03 Thread Miao Xie
rw_devices counter is often used to tune the profile when doing chunk allocation, so we should modify it under the chunk_mutex context to avoid getting wrong chunk profile. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/

[PATCH 13/18] Btrfs: fix unprotected device list access when cloning fs devices

2014-09-03 Thread Miao Xie
We can build a new filesystem based a seed filesystem, and we need clone the fs devices when we open the new filesystem. But someone might clear the seed flag of the seed filesystem, then mount that filesystem and remove some device. If we mount the new filesystem, we might access a device list whi

[PATCH 1/5] block: export disk_class and disk_type for btrfs

2014-09-03 Thread Miao Xie
Btrfs can make filesystem cross several disks/partitions, in order to load all the disks/partitions which belong to the same filesystem, we need scan the system and find all the devices, and then register them into the kernel. Currently, we do it by user tool. But if we forget to do it, we can not

[PATCH 15/18] Btrfs: make the logic of source device removing more clear

2014-09-03 Thread Miao Xie
Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 3 +-- fs/btrfs/volumes.c | 19 +++ 2 files changed, 8 insertions(+), 14 deletions(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index e9cbbdb..6f662b3 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/d

[PATCH RFC 0/5] Scan all devices to build fs device list

2014-09-03 Thread Miao Xie
This patchset implements device list automatic building function. As we know, currently we need scan the devices to build device list by a user tool before mounting the filesystem, especially mount the filesystem after we re-install btrfs module. It is not convenient. This patchset can improve that

[PATCH 11/18] Btrfs: fix unprotected device list access when getting the fs information

2014-09-03 Thread Miao Xie
When we get the fs information, we forgot to acquire the mutex of device list, it might cause the problem we might access a device that was removed. Fix it by acquiring the device list mutex. Signed-off-by: Miao Xie --- fs/btrfs/super.c | 8 +++- 1 file changed, 7 insertions(+), 1 deletion(-

[PATCH 09/18] Btrfs: fix unprotected device's variants on 32bits machine

2014-09-03 Thread Miao Xie
->total_bytes,->disk_total_bytes,->bytes_used is protected by chunk lock when we change them, but sometimes we read them without any lock, and we might get unexpected value. We fix this problem like inode's i_size. Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 15 + fs/btrfs/ioctl

[PATCH 07/18] Btrfs: fix unprotected device->bytes_used update

2014-09-03 Thread Miao Xie
We should update device->bytes_used in the lock context of chunk_mutex, or we would get wrong data. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index 1524b3f..45e0b5d 100644 --- a/fs

[PATCH 05/18] Btrfs: fix wrong device bytes_used in the super block

2014-09-03 Thread Miao Xie
device->bytes_used will be changed when allocating a new chunk, and disk_total_size will be changed if resizing is successful. Meanwhile, the on-disk super blocks of the previous transaction might not be updated. Considering the consistency of the metadata in the previous transaction, We should use

[PATCH 5/5] Btrfs: scan all the devices and build the fs device list by btrfs's self

2014-09-03 Thread Miao Xie
The original code need scan the devices and build the fs device list by the user tool by udev or users' selves. It is flexible. But if someone re-install the filesystem module, and forget to scan the devices by himself, or we plug some devices with btrfs, but udev thread is blocked and doesn't regi

[PATCH 16/18] Btrfs: stop mounting the fs if the non-ENOENT errors happen when opening seed fs

2014-09-03 Thread Miao Xie
When we open a seed filesystem, if the degraded mount option is set, we continue to mount the fs if we don't find some devices in the seed filesystem. But we should stop mounting if other errors happen. Fix it Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 2 +- 1 file changed, 1 insertion(+

[PATCH 14/18] Btrfs: fix use-after-free problem of the device during device replace

2014-09-03 Thread Miao Xie
The problem is: Task0(device scan task) Task1(device replace task) scan_one_device() mutex_lock(&uuid_mutex) device = find_device() mutex_lock(&device_list_mutex) lock_chunk()

[PATCH 08/18] Btrfs: update free_chunk_space during allocting a new chunk

2014-09-03 Thread Miao Xie
We should update free_chunk_space in time when we allocate a new chunk, not when we deal with the pending device update and block group insertion, because we need the real free_chunk_space data to calculate the reserved space, if we don't update it in time, we would consider the disk space which ha

[PATCH 2/5] Btrfs: don't return btrfs_fs_devices if the caller doesn't want it

2014-09-03 Thread Miao Xie
We will implement the function that the filesystem scan all the devices in the system and build the device set for btrfs. In this case, we needn't get btrfs_fs_devices when adding a device into list. This patch changes device_add_list and implement this feature. Signed-off-by: Miao Xie --- fs/bt

[PATCH 10/18] Btrfs: fix unprotected system chunk array insertion

2014-09-03 Thread Miao Xie
We didn't protect the system chunk array when we added a new system chunk into it, it would cause the array be corrupted if someone remove/add some system chunk into array at the same time. Fix it by chunk lock. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 7 ++- 1 file changed, 6 insert

[PATCH 12/18] Btrfs: Fix misuse of chunk mutex

2014-09-03 Thread Miao Xie
There were several problems about chunk mutex usage: - Lock chunk mutex when updating metadata. It would cause the nested deadlock because updating metadata might need allocate new chunks that need acquire chunk mutex. We remove chunk mutex at this case, because b-tree lock and other lock mec

[PATCH 04/18] Btrfs: fix wrong disk size when writing super blocks

2014-09-03 Thread Miao Xie
total_size will be changed when resizing a device, and disk_total_size will be changed if resizing is successful. Meanwhile, the on-disk super blocks of the previous transaction might not be updated. Considering the consistency of the metadata in the previous transaction, We should use the size in

[PATCH 17/18] Btrfs: move the missing device to its own fs device list

2014-09-03 Thread Miao Xie
For a missing device, we don't know it belong to which fs before we read its fsid from the chunk tree. So we add them into the current fs device list at first. When we get its fsid, we should move them to their own fs device list. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 78

[PATCH 4/5] Btrfs: restructure btrfs_get_bdev_and_sb and pick up some code used later

2014-09-03 Thread Miao Xie
Some code in btrfs_get_bdev_and_sb will be re-used by the other function later, so restructure btrfs_get_bdev_and_sb and pick up those code to make a new function. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 66 +- 1 file changed, 36 inser

[PATCH 03/18] Btrfs: fix unprotected assignment of the target device

2014-09-03 Thread Miao Xie
We didn't protect the assignment of the target device, it might cause the problem that the super block update was skipped because we might find wrong size of the target device during the assignment. Fix it by moving the assignment sentences into the initialization function of the target device. And

[PATCH 3/5] Btrfs: restructure btrfs_scan_one_device

2014-09-03 Thread Miao Xie
Some code in btrfs_scan_one_device will be re-used by the other function later, so restructure btrfs_scan_one_device and pick up those code to make a new function. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 57 +++--- 1 file changed, 33 inser

[PATCH 01/18] Btrfs: cleanup unused num_can_discard in fs_devices

2014-09-03 Thread Miao Xie
The member variants - num_can_discard - of fs_devices structure are set, but no one use them to do anything. so remove them. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 16 ++-- fs/btrfs/volumes.h | 1 - 2 files changed, 2 insertions(+), 15 deletions(-) diff --git a/fs/btrfs/v

[PATCH 02/18] Btrfs: cleanup double assignment of device->bytes_used when device replace finishes

2014-09-03 Thread Miao Xie
Signed-off-by: Miao Xie --- fs/btrfs/dev-replace.c | 1 - 1 file changed, 1 deletion(-) diff --git a/fs/btrfs/dev-replace.c b/fs/btrfs/dev-replace.c index a85b5f5..10dfb41 100644 --- a/fs/btrfs/dev-replace.c +++ b/fs/btrfs/dev-replace.c @@ -550,7 +550,6 @@ static int btrfs_dev_replace_finishing(

[PATCH 06/18] Btrfs: Fix wrong free_chunk_space assignment during removing a device

2014-09-03 Thread Miao Xie
During removing a device, we have modified free_chunk_space when we shrink the device, so we needn't assign a new value to it after the device shrink. Fix it. Signed-off-by: Miao Xie --- fs/btrfs/volumes.c | 5 - 1 file changed, 5 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volu

Re: kernel 3.17-rc3: task rsync:2524 blocked for more than 120 seconds

2014-09-03 Thread Chris Mason
On 09/02/2014 09:31 PM, john terragon wrote: > Rsync finished. FWIW in the end it reported an average speed of about > 900K/sec. Without autodefrag there have been no messages about hung > kworkers even though rsync seemingly keeps getting hung for several > minutes throughout the whole execution.

Problem with applying incremental btrfs-send

2014-09-03 Thread Harald Hoyer
Hi, maybe someone can enlighten me. I am doing btrfs send & receive with full snapshots and incremental updates. It basically looks like this: vol-0 and vol-1 are full subvolume image sends. inc-1 and inc-2 are incremental images with: # btrfs send -f inc-1 -p vol vol' # btrfs send -f inc-2 -p v

kernel BUG at fs/btrfs/relocation.c:1065 in 3.14.16 to 3.17-rc3

2014-09-03 Thread Olivier Bonvalet
Hi, I have a btrfs partition which throw kernel BUG, even with linux 3.17-rc3 (I tried 3.14.16, 3.16.1 and 3.17-rc3 kernels) : [ 45.058466] [ cut here ] [ 45.058539] kernel BUG at fs/btrfs/relocation.c:1065! [ 45.058578] invalid opcode: [#1] SMP [ 45.058655]

Re: kernel 3.17-rc3: task rsync:2524 blocked for more than 120 seconds

2014-09-03 Thread Liu Bo
On Wed, Sep 03, 2014 at 08:03:51AM +0200, john terragon wrote: > I tried the same routine on 32GB usb sticks. Same exact problems. 32GB > seems a bit much for a --mixed btrfs. > I haven't tried ssd_spread, maybe it's beneficial. However, as I wrote > above, disabling autodefrag gets rid completely

Re: [PATCH v2 11/12] Btrfs: implement repair function when direct read fails

2014-09-03 Thread Miao Xie
On Tue, 2 Sep 2014 09:05:15 -0400, Chris Mason wrote: > diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c > index 08e65e9..56b1546 100644 > --- a/fs/btrfs/disk-io.c > +++ b/fs/btrfs/disk-io.c > @@ -698,7 +719,12 @@ static void end_workqueue_bio(struct bio *bio, int > err

Re: Btrfs-progs-3.16: fs metadata is both single and dup?

2014-09-03 Thread Duncan
Hugo Mills posted on Wed, 03 Sep 2014 08:33:05 +0100 as excerpted: > On Wed, Sep 03, 2014 at 04:53:39AM +, Duncan wrote: >> Hugo Mills posted on Tue, 02 Sep 2014 13:13:49 +0100 as excerpted: >> >> >> [A] btrfs fi df on a new filesystem always seems to have those extra >> unused single profil

Re: Btrfs-progs-3.16: fs metadata is both single and dup?

2014-09-03 Thread Hugo Mills
On Wed, Sep 03, 2014 at 04:53:39AM +, Duncan wrote: > Hugo Mills posted on Tue, 02 Sep 2014 13:13:49 +0100 as excerpted: > > > On Tue, Sep 02, 2014 at 12:05:33PM +, Holger Hoffstätte wrote: > >> So where does the confusing initial display come from? [I] don't > >> remember ever seeing this

[PATCH v2 06/10] Btrfs: Fix the problem that the dirty flag of dev stats is cleared

2014-09-03 Thread Miao Xie
The io error might happen during writing out the device stats, and the device stats information and dirty flag would be update at that time, but the current code didn't consider this case, just clear the dirty flag, it would cause that we forgot to write out the new device stats information. Fix it