Re: [PATCH RFC 0/5] Scan all devices to build fs device list

2014-09-08 Thread David Sterba
On Sat, Sep 06, 2014 at 04:05:20PM -0400, Chris Mason wrote: On 09/03/2014 09:36 AM, Miao Xie wrote: This patchset implements device list automatic building function. As we know, currently we need scan the devices to build device list by a user tool before mounting the filesystem,

Re: Linux 3.14.18 : kernel BUG at fs/btrfs/file.c:890!

2014-09-08 Thread Olivier Bonvalet
Le dimanche 07 septembre 2014 à 17:48 +, Holger Hoffstätte a écrit : On Sun, 07 Sep 2014 18:39:40 +0200, Olivier Bonvalet wrote: I cannot really offer much assistance, but that particular area of code looks suspiciously like something fixed later and *not* yet backported to 3.14.8:

Re: [PATCH 00/15 v2] xfstests: new btrfs stress test cases

2014-09-08 Thread Dave Chinner
On Thu, Aug 28, 2014 at 09:47:41PM +0800, Eryu Guan wrote: This patchset add new stress test cases for btrfs by running two different btrfs operations simultaneously under fsstress to ensure btrfs doesn't hang or oops in such situations. btrfs scrub and btrfs check will be run after each test.

Re: Linux 3.14.18 : kernel BUG at fs/btrfs/file.c:890!

2014-09-08 Thread Holger Hoffstätte
On Mon, 08 Sep 2014 11:39:57 +0200, Olivier Bonvalet wrote: Ok, so the cause is probably solved, this is a very good news, thanks. I don't *know* that this is your specific problem, it just looks like it might be judging by the description in the patch. And yes, I can easily use 3.17-rc3

Re: [PATCH 01/15 v2] btrfs: new test to run btrfs balance and subvolume test simultaneously

2014-09-08 Thread Dave Chinner
On Thu, Aug 28, 2014 at 09:47:42PM +0800, Eryu Guan wrote: Run btrfs balance and subvolume create/mount/umount/delete simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 24 ++ tests/btrfs/059 | 129

Re: [PATCH 02/15 v2] btrfs: new test to run btrfs balance and scrub simultaneously

2014-09-08 Thread Dave Chinner
On Thu, Aug 28, 2014 at 09:47:43PM +0800, Eryu Guan wrote: Run btrfs balance and scrub operations simultaneously with fsstress running in background. +run_test() +{ + local mkfs_opts=$1 + + echo Test $mkfs_opts $seqres.full + + # dup only works on single device + if [[

Re: [PATCH 04/15 v2] btrfs: new case to run btrfs balance and remount with different compress algorithms

2014-09-08 Thread Dave Chinner
On Thu, Aug 28, 2014 at 09:47:45PM +0800, Eryu Guan wrote: Run btrfs balance and remount with different compress algorithms simultaneously, with fsstress running in background. Signed-off-by: Eryu Guan eg...@redhat.com --- common/rc | 10 tests/btrfs/062 | 128

Re: Linux 3.14.18 : kernel BUG at fs/btrfs/file.c:890!

2014-09-08 Thread Olivier Bonvalet
Le lundi 08 septembre 2014 à 09:59 +, Holger Hoffstätte a écrit : By default btrfsck doesn't do anything. What happens when you run it without --repair, just to see what it finds? In fact I haven't enough memory to run it (8GB of RAM only, with a 2*3TB FS in mirroring, btrfs mirroring).

Re: No space on empty, degraded raid10

2014-09-08 Thread Austin S Hemmelgarn
On 2014-09-07 16:38, Or Tal wrote: Hi, I've created a new raid10 array from 4, 4TB drives in order to migrate old data to it. As I didn't have enough sata ports, I: - disconnected one of the raid10 disks to free a sata port, - connected an old disk I wanted to migrate, - mounted the array

Re: [PATCH RFC 0/5] Scan all devices to build fs device list

2014-09-08 Thread Anand Jain
On 08/09/2014 17:09, David Sterba wrote: On Sat, Sep 06, 2014 at 04:05:20PM -0400, Chris Mason wrote: On 09/03/2014 09:36 AM, Miao Xie wrote: This patchset implements device list automatic building function. As we know, currently we need scan the devices to build device list by a user tool

Re: Linux 3.14.18 : kernel BUG at fs/btrfs/file.c:890!

2014-09-08 Thread Holger Hoffstätte
On Mon, 08 Sep 2014 12:48:20 +0200, Olivier Bonvalet wrote: Le lundi 08 septembre 2014 à 09:59 +, Holger Hoffstätte a écrit : By default btrfsck doesn't do anything. What happens when you run it without --repair, just to see what it finds? In fact I haven't enough memory to run it (8GB

[PATCH] btrfs: remove the wrong comments

2014-09-08 Thread roy . qing . li
From: Li RongQing roy.qing...@gmail.com This comments became wrong after c3c532[bdi: add helper function for doing init and register of a bdi for a file system], so remove them. Signed-off-by: Li RongQing roy.qing...@gmail.com --- fs/btrfs/disk-io.c |4 1 file changed, 4 deletions(-)

[RFC PATCH V6 15/15] Btrfs: subpagesize-blocksize: Revert commit fc4adbff823f76577ece26dcb88bf6f8392dbd43.

2014-09-08 Thread Chandan Rajendra
In subpagesize-blocksize, we have multiple blocks in a page. Checking for existence of a page in the page cache isn't a sufficient check, since we could be truncating a subset of the blocks mapped by the page. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/btrfs_inode.h

[RFC PATCH V6 14/15] Btrfs: subpagesize-blocksize: Explicitly Track I/O status of blocks of an ordered extent.

2014-09-08 Thread Chandan Rajendra
In subpagesize-blocksize scenario a page can have more than one block. So in addition to PagePrivate2 flag, we would have to track the I/O status of each block of a page to reliably mark the ordered extent as complete. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com ---

[RFC PATCH V6 04/15] Btrfs: subpagesize-blocksize: Define extent_buffer_head.

2014-09-08 Thread Chandan Rajendra
From: Chandra Seetharaman sekha...@us.ibm.com In order to handle multiple extent buffers per page, first we need to create a way to handle all the extent buffers that are attached to a page. This patch creates a new data structure 'struct extent_buffer_head', and moves fields that are common to

[RFC PATCH V6 13/15] Btrfs: subpagesize-blocksize: Deal with partial ordered extent allocations.

2014-09-08 Thread Chandan Rajendra
In subpagesize-blocksize scenario, extent allocations for only some of the dirty blocks of a page can succeed, while allocation for rest of the blocks can fail. This patch allows I/O against such partially allocated ordered extents to be submitted. Signed-off-by: Chandan Rajendra

[RFC PATCH V6 11/15] Btrfs: subpagesize-blocksize: btrfs_page_mkwrite: Reserve space in sectorsized units.

2014-09-08 Thread Chandan Rajendra
In subpagesize-blocksize scenario, if i_size occurs in a block which is not the last block in the page, then the space to be reserved should be calculated appropriately. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/inode.c | 33 ++--- 1 file

[RFC PATCH V6 06/15] Btrfs: subpagesize-blocksize: Write only dirty extent buffers belonging to a page

2014-09-08 Thread Chandan Rajendra
For the subpagesize-blocksize scenario, This patch adds the ability to write a single extent buffer to the disk. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/disk-io.c | 20 ++-- fs/btrfs/extent_io.c | 300 --- 2 files

[RFC PATCH V6 12/15] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page.

2014-09-08 Thread Chandan Rajendra
In subpagesize-blocksize scenario it is not sufficient to search using the first byte of the page to make sure that there are no ordered extents present across the page. Fix this. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/extent_io.c | 3 ++- fs/btrfs/inode.c |

[RFC PATCH V6 10/15] Btrfs: subpagesize-blocksize: fallocate: Work with sectorsized units.

2014-09-08 Thread Chandan Rajendra
While at it, this commit changes btrfs_truncate_page() to truncate sectorsized blocks instead of pages. Hence the function has been renamed to btrfs_truncate_block(). Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/ctree.h | 2 +- fs/btrfs/file.c | 41

[RFC PATCH V6 03/15] Btrfs: subpagesize-blocksize: __btrfs_buffered_write: Reserve/release extents aligned to block size.

2014-09-08 Thread Chandan Rajendra
Currently, the code reserves/releases extents in multiples of PAGE_CACHE_SIZE units. Fix this. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/file.c | 32 1 file changed, 20 insertions(+), 12 deletions(-) diff --git a/fs/btrfs/file.c

[RFC PATCH V6 07/15] Btrfs: subpagesize-blocksize: Allow mounting filesystems where sectorsize != PAGE_SIZE

2014-09-08 Thread Chandan Rajendra
From: Chandra Seetharaman sekha...@us.ibm.com This patch allows mounting filesystems with blocksize smaller than the PAGE_SIZE. Signed-off-by: Chandra Seetharaman sekha...@us.ibm.com Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/disk-io.c | 6 -- 1 file changed, 6

[RFC PATCH V6 08/15] Btrfs: subpagesize-blocksize: Compute and look up csums based on sectorsized blocks.

2014-09-08 Thread Chandan Rajendra
Checksums are applicable to sectorsize units. The current code uses bio-bv_len units to compute and look up checksums. This works on machines where sectorsize == PAGE_CACHE_SIZE. This patch makes the checksum computation and look up code to work with sectorsize units. Signed-off-by: Chandan

[RFC PATCH V6 00/15] ] Btrfs: Subpagesize-blocksize: Get rid of whole page I/O.

2014-09-08 Thread Chandan Rajendra
This patchset continues with the work posted earlier at https://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg35926.html. Changes from V1: 1. Remove usage of bio_vec-bv_{len,offset} in end_bio_extent_readpage() and end_bio_extent_writepage(). Changes from V2: 1. Get __extent_writepage()

[RFC PATCH V6 02/15] Btrfs: subpagesize-blocksize: Get rid of whole page writes.

2014-09-08 Thread Chandan Rajendra
This commit brings back functions that set/clear EXTENT_WRITEBACK bits. These are required to reliably clear PG_writeback page flag. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/extent_io.c | 47 +++ fs/btrfs/inode.c | 40

[RFC PATCH V6 09/15] Btrfs: subpagesize-blocksize: __extent_writepage: Write only dirty blocks of a page.

2014-09-08 Thread Chandan Rajendra
The code now loops across 'ordered extents' instead of 'extent maps' to figure out the dirty blocks of the page to be submitted for a write operation. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/extent_io.c | 74 1

[RFC PATCH V6 05/15] Btrfs: subpagesize-blocksize: Read tree blocks whose size is PAGE_CACHE_SIZE.

2014-09-08 Thread Chandan Rajendra
In the case of subpagesize-blocksize, this patch makes it possible to read only a single metadata block from the disk instead of all the metadata blocks that map into a page. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com --- fs/btrfs/disk-io.c | 45 -

[RFC PATCH V6 01/15] Btrfs: subpagesize-blocksize: Get rid of whole page reads.

2014-09-08 Thread Chandan Rajendra
Based on original patch from Aneesh Kumar K.V aneesh.ku...@linux.vnet.ibm.com For the subpagesize-blocksize scenario, a page can contain multiple blocks. This patch handles this case. This patch also brings back check_page_locked() to reliably unlock pages in readpage's end bio function.

Re: [PATCH RFC 0/5] Scan all devices to build fs device list

2014-09-08 Thread Goffredo Baroncelli
Hi David, On 09/08/2014 11:09 AM, David Sterba wrote: Automatic scanning in usperspace can be done via the mount helper and this is IMO the way to go. There's a patch for that from Goffredo, not merged yet, the patch backlog is still too long. There is something that I can do to help you

Re: [PATCH RFC 0/5] Scan all devices to build fs device list

2014-09-08 Thread Goffredo Baroncelli
On 09/08/2014 01:04 PM, Anand Jain wrote: On 08/09/2014 17:09, David Sterba wrote: On Sat, Sep 06, 2014 at 04:05:20PM -0400, Chris Mason wrote: On 09/03/2014 09:36 AM, Miao Xie wrote: This patchset implements device list automatic building function. As we know, currently we need scan the

Re: kernel BUG at fs/btrfs/extent-tree.c:7727! with 3.17-rc3

2014-09-08 Thread Tomasz Chmielewski
On 2014-09-03 19:42 (Wed), Tomasz Chmielewski wrote: Got the following with 3.17-rc3 and running balance (had to power cycle after that): I'm seeing similar BUG with 3.17-rc4: [ 1049.755843] BTRFS info (device sdb5): found 35715 extents [ 1050.257075] BTRFS info (device sdb5): relocating

Re: [PATCH] Btrfs: fix data corruption after fast fsync and writeback error

2014-09-08 Thread Chris Mason
On 09/05/2014 10:14 AM, Filipe Manana wrote: When we do a fast fsync, we start all ordered operations and then while they're running in parallel we visit the list of modified extent maps and construct their matching file extent items and write them to the log btree. After that, in

[PATCH] Btrfs: fix directory recovery from fsync log

2014-09-08 Thread Filipe Manana
When replaying a directory from the fsync log, if a directory entry exists both in the fs/subvol tree and in the log, the directory's inode got its i_size updated incorrectly, accounting for the dentry's name twice. Reproducer, from a test for xfstests: _scratch_mkfs $seqres.full 21

[PATCH] xfstests: generic: add dir fsync test, motivated by a btrfs bug

2014-09-08 Thread Filipe Manana
This test is motivated by a bug found in btrfs when fsync'ing a directory. The issue was that if a directory entry is both found in the persisted metadata and in the fsync log, at log replay time the directory got set ith a wrong i_size. This was fixed in btrfs with the following linux kernel

[PATCH v5] Btrfs: fix fsync data loss after a ranged fsync

2014-09-08 Thread Filipe Manana
While we're doing a full fsync (when the inode has the flag BTRFS_INODE_NEEDS_FULL_SYNC set) that is ranged too (covers only a portion of the file), we might have ordered operations that are started before or while we're logging the inode and that fall outside the fsync range. Therefore when a

[PATCH v2] xfstests: generic: add dir fsync test, motivated by a btrfs bug

2014-09-08 Thread Filipe Manana
This test is motivated by a bug found in btrfs when fsync'ing a directory. The issue was that if a directory entry is both found in the persisted metadata and in the fsync log, at log replay time the directory got set ith a wrong i_size. This was fixed in btrfs with the following linux kernel

Re: btrfs differential receive has become excrutiatingly slow on one machine

2014-09-08 Thread Filipe David Manana
On Mon, Sep 8, 2014 at 2:51 AM, Marc MERLIN m...@merlins.org wrote: Hi Filipe and others, TL;DR: - btrfs send of a 1.5GB filesystem takes 4mn to replicate that filesystem on the target machine. - btrfs send -p between 2 snapshots happens in 1 second - copying that diff to the other

Re: btrfs 3.16.2 system crash

2014-09-08 Thread Chris Mason
On 09/08/2014 06:51 PM, Morten Stevens wrote: Hi, After some testing with the latest stable 3.16.2 kernel on my Fedora 20 test-vm and a small filesystem test with btrfs causes the system to crash. Kernel: 3.16.2 Mount options: rw,seclabel,relatime,compress=lzo,space_cache The test

[PATCH] btrfs-progs: doc: fix invalid reference to setattr(8)

2014-09-08 Thread Satoru Takeuchi
From: Naohiro Aota na...@elisp.net man 8 btrfs-property refers to `setattr(8)` which does not actually exist. It should refer to `chattr (1)` instead. Signed-off-by: Naohiro Aota na...@elisp.net --- Documentation/btrfs-property.txt | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff

Re: [PATCH RFC 0/5] Scan all devices to build fs device list

2014-09-08 Thread Anand Jain
Since we are on the topic of scanning. A point on improving the btrfs-progs scan method which I am working on... - Scan of all system devices is an expensive task. But btrfs-progs do it very liberally. Due to this there are some serious problem like - btrfs fi show is too

Re: [PATCH] btrfs-progs: doc: fix invalid reference to setattr(8)

2014-09-08 Thread Qu Wenruo
Original Message Subject: [PATCH] btrfs-progs: doc: fix invalid reference to setattr(8) From: Satoru Takeuchi takeuchi_sat...@jp.fujitsu.com To: linux-btrfs@vger.kernel.org linux-btrfs@vger.kernel.org Date: 2014年09月09日 10:09 From: Naohiro Aota na...@elisp.net man 8