[PATCH] Btrfs: calc file extent num_bytes correctly in file clone

2011-08-23 Thread Li Zefan
# touch /mnt/dst # clone_range -s 8192 -l 4096 /mnt/src /mnt/dst # ls -l /mnt/dst -rw-r--r-- 1 root root 4096 dst # umount /mnt # btrfs-debug-tree /dev/sda7 ... item 11 key (258 EXTENT_DATA 0) itemoff 3230 itemsize 53 extent data disk byte 12582912 nr 49152

[PATCH] Btrfs: add dummy extent if dst offset excceeds file end in file clone

2011-08-23 Thread Li Zefan
# touch /mnt/dst # clone_range -d 4096 /mnt/src /mnt/dst # umount /mnt # btrfs-debug-tree /dev/sda7 ... item 10 key (258 INODE_REF 256) itemoff 3283 itemsize 13 inode ref index 3 namelen 3 name: tmp item 11 key (258 EXTENT_DATA 4096) itemoff 3230 itemsize 53

Re: [PATCH 2/3] Btrfs: fix the file extent gap when doing direct IO

2011-08-23 Thread Li Zefan
> diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c > index 010aec8..a9c4636 100644 > --- a/fs/btrfs/file.c > +++ b/fs/btrfs/file.c > @@ -1073,12 +1073,6 @@ static noinline int prepare_pages(struct btrfs_root > *root, struct file *file, > start_pos = pos & ~((u64)root->sectorsize - 1); >

Re: Suddenly, a dead filesystem

2011-08-23 Thread Avi Kivity
On Tue, Aug 23, 2011 at 5:27 PM, Hugo Mills wrote: > >   Could you give a bit more information about what happened in the > crash immediately prior to the FS being unmountable, I was talking on the phone (HTC Desire running Android 2.2). > and how your > storage is configured? /dev/sda2 -> phys

Re: Suddenly, a dead filesystem

2011-08-23 Thread Avi Kivity
> This is fixed upstream, I've sent the patch to -stable so hopefully it > will show up in fedora soon, but in the meantime can you try Linus's > tree and verify that it fixes it?  Thanks, Thanks, it appears to be fixed (at least in a VM). Will try native soon. -- To unsubscribe from this list: s

[PATCH] Btrfs-progs: add restriper commands

2011-08-23 Thread Ilya Dryomov
Signed-off-by: Ilya Dryomov --- btrfs.c | 25 +++- btrfs_cmds.c | 508 +- btrfs_cmds.h |5 + ctree.h |9 + ioctl.h | 44 + print-tree.c |3 + volumes.h| 42 + 7 files changed, 632 insertions(+),

[PATCH] [RFC] Btrfs-progs: restriper interface

2011-08-23 Thread Ilya Dryomov
Hello, This is a RFC, man pages are absent and just one big commit. Here are some specs for the new interface: ./btrfs fi restripe start [-d[filters]] [-m[filters]] [-s[filters]] [-vf] where 'filters' is comma-separated list of filters (comma == AND): o profiles={profiles mask} - profiles

[PATCH 21/21] Btrfs: add restripe progress reporting

2011-08-23 Thread Ilya Dryomov
Signed-off-by: Ilya Dryomov --- fs/btrfs/ioctl.c | 45 + fs/btrfs/ioctl.h |2 ++ fs/btrfs/volumes.c | 40 ++-- fs/btrfs/volumes.h |3 +++ 4 files changed, 84 insertions(+), 6 deletions(-) diff --git a

[PATCH 20/21] Btrfs: get rid of btrfs_balance() function

2011-08-23 Thread Ilya Dryomov
Remove btrfs_balance(). The old balancing ioctl now uses restriper infrastructure, just w/o using any filters. Signed-off-by: Ilya Dryomov --- fs/btrfs/ioctl.c | 38 +- fs/btrfs/volumes.c | 115 --- fs/btrfs/volumes.h |1 -

[PATCH 19/21] Btrfs: add skip_restripe mount option

2011-08-23 Thread Ilya Dryomov
Since restriper kthread starts involuntarily on mount and can suck cpu and memory bandwidth add a mount option to forcefully skip it. The restriper in that case hangs around in paused state and can be resumed from userspace when it's convenient. Signed-off-by: Ilya Dryomov --- fs/btrfs/ctree.h

[PATCH 18/21] Btrfs: allow for resuming restriper after it was paused

2011-08-23 Thread Ilya Dryomov
Implement an ioctl for resuming restriper. We use the same heuristics used when recovering restripe after a crash to try to start where we left off last time. If needed those parameters can be made configurable through the userspace "resume" command in future. Signed-off-by: Ilya Dryomov --- f

[PATCH 17/21] Btrfs: allow for pausing restriper

2011-08-23 Thread Ilya Dryomov
Implement an ioctl for pausing restriper. This pauses the relocation, but restripe is still considered to be "in progress": restriper item is not deleted, other volume operations cannot be started, etc. If paused in the middle of profile changing operation we will continue making allocations with

[PATCH 16/21] Btrfs: allow for cancelling restriper

2011-08-23 Thread Ilya Dryomov
Implement an ioctl for cancelling restriper. Currently we wait until relocation of the current block group is finished, in future this can be done by triggering a commit. Restripe item is deleted and no memory about the interrupted restripe is kept. Signed-off-by: Ilya Dryomov --- fs/btrfs/ctr

[PATCH 15/21] Btrfs: recover restripe on mount

2011-08-23 Thread Ilya Dryomov
On mount, if restripe item is found, resume restripe in a separate kernel thread. Try to be smart to continue roughly where previous balance (or convert) was interrupted. For chunk types that were being converted to some profile we turn on soft convert, in case of a simple balance we turn on usag

[PATCH 14/21] Btrfs: save restripe parameters to disk

2011-08-23 Thread Ilya Dryomov
Introduce a new btree objectid for storing restripe item. The reason is to be able to resume restriper after a crash with the same parameters. Restripe item has a very high objectid and goes into tree of tree roots. The key for the new item is as follows: [ BTRFS_RESTRIPE_OBJECTID ; 0 ; 0 ]

[PATCH 13/21] Btrfs: virtual address space subset filter

2011-08-23 Thread Ilya Dryomov
Select chunks which have at least one byte located inside a given [vstart, vend) virtual address space range. Signed-off-by: Ilya Dryomov --- fs/btrfs/volumes.c | 20 fs/btrfs/volumes.h |3 ++- 2 files changed, 22 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/vo

[PATCH 12/21] Btrfs: devid subset filter

2011-08-23 Thread Ilya Dryomov
Select chunks which have at least one byte of at least one stripe located on a device with devid X in a given [pstart,pend) physical address range. This filter only works when devid filter is turned on. Signed-off-by: Ilya Dryomov --- fs/btrfs/volumes.c | 45 ++

[PATCH 11/21] Btrfs: devid filter

2011-08-23 Thread Ilya Dryomov
Relocate chunks which have at least one stripe located on a device with devid X. Signed-off-by: Ilya Dryomov --- fs/btrfs/volumes.c | 23 +++ fs/btrfs/volumes.h |1 + 2 files changed, 24 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c

[PATCH 10/21] Btrfs: usage filter

2011-08-23 Thread Ilya Dryomov
Select chunks that are less than X percent full. Signed-off-by: Ilya Dryomov --- fs/btrfs/volumes.c | 33 + fs/btrfs/volumes.h |1 + 2 files changed, 34 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index f045615..b49ecf

[PATCH 09/21] Btrfs: profiles filter

2011-08-23 Thread Ilya Dryomov
Select chunks based on a given profile mask. Signed-off-by: Ilya Dryomov --- fs/btrfs/volumes.c | 20 fs/btrfs/volumes.h |5 + 2 files changed, 25 insertions(+), 0 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c index ff252ef..f045615 100644 ---

[PATCH 08/21] Btrfs: soft profile changing mode (aka soft convert)

2011-08-23 Thread Ilya Dryomov
When doing convert from one profile to another if soft mode is on restriper won't touch chunks that already have the profile we are converting to. This is useful if e.g. half of the fs was converted earlier. The soft mode switch is per-type (like everything else). This means that we can convert

[PATCH 07/21] Btrfs: add basic infrastructure for selective balancing

2011-08-23 Thread Ilya Dryomov
This allows to have a separate set of filters for each chunk type (data,meta,sys). The code however is generic and switch on chunk type is only done once. This commit also adds a type filter: it allows to balance for example meta and system chunks w/o touching data ones. Signed-off-by: Ilya Dryo

[PATCH 05/21] Btrfs: add basic restriper infrastructure

2011-08-23 Thread Ilya Dryomov
Add basic restriper infrastructure: ioctl to start restripe, all restripe ioctl data structures, add data structure for tracking restriper's state to fs_info. Duplicate balancing code for restriper, btrfs_balance() will be removed when restriper is implemented. Explicitly disallow any volume oper

[PATCH 06/21] Btrfs: implement online profile changing

2011-08-23 Thread Ilya Dryomov
Profile changing is done by initializing target field in respective btrfs_restripe_args structs and launching a balance. Reducing code in this mode will pick restriper's target profile if it's available instead of doing a blind reduce. If target profile is not yet available go back to plain reduc

[PATCH 04/21] Btrfs: make avail_*_alloc_bits fields dynamic

2011-08-23 Thread Ilya Dryomov
Currently when new chunks are created respective avail_alloc_bits field is updated to reflect profiles of all chunks present in the system. However when chunks are removed, corresponding profile bits are never cleared. This patch clears corresponding bit of avail_alloc_bits field when the last chu

[PATCH 03/21] Btrfs: add BTRFS_AVAIL_ALLOC_BIT_SINGLE bit

2011-08-23 Thread Ilya Dryomov
Right now on-disk BTRFS_BLOCK_GROUP_* profile bits are used for avail_{data,metadata,system}_alloc_bits fields, which are there to tell us about available allocation profiles in the fs. When chunk is created, it's profile is OR'ed with respective avail_alloc_bits field. Since SINGLE is denoted by

[PATCH 02/21] Btrfs: introduce masks for chunk type and profile

2011-08-23 Thread Ilya Dryomov
Chunk's type and profile are encoded in u64 flags field. Introduce masks to easily access them. Signed-off-by: Ilya Dryomov --- fs/btrfs/ctree.h |8 fs/btrfs/extent-tree.c | 12 +++- fs/btrfs/volumes.c | 11 ++- 3 files changed, 13 insertions(+), 18 de

[PATCH 01/21] Btrfs: get rid of *_alloc_profile fields

2011-08-23 Thread Ilya Dryomov
{data,metadata,system}_alloc_profile fields have been unused for a long time now. Get rid of them. Signed-off-by: Ilya Dryomov --- fs/btrfs/ctree.h |3 --- fs/btrfs/disk-io.c |3 --- fs/btrfs/extent-tree.c | 10 -- fs/btrfs/volumes.c |6 ++ 4 files change

[PATCH 00/21] [RFC] Btrfs: restriper

2011-08-23 Thread Ilya Dryomov
Hello, This patch series adds an initial implementation of restriper (it's a clever name for relocation framework that allows to do selective profile changing and selective balancing with some goodies like pausing/resuming and reporting progress to the user. Profile changing is global (per-FS) s

Re: [PATCH 1/4] fs: add SEEK_HOLE and SEEK_DATA flags

2011-08-23 Thread Marco Stornelli
Il 22/08/2011 23:22, Sunil Mushran ha scritto: On 08/22/2011 10:56 AM, Marco Stornelli wrote: Il 22/08/2011 17:57, Sunil Mushran ha scritto: The following test was used to test the early implementations. http://oss.oracle.com/~smushran/seek_data/ Thank you very much!! I found another point.

Re: Swap file on btrfs fails ("swapfile has holes")

2011-08-23 Thread David Nicol
it seems to me that the swap file interface doesn't make sense; to "eliminate the middleman" and provide a direct swap tree type would be the way to go, so a btrfs can hold swapped pages. Doing this would AIUI require: 1) direct interface to btrfs trees, not via file-system interface 2) modifying s

Re: Swap file on btrfs fails ("swapfile has holes")

2011-08-23 Thread Gour-Gadadhara Dasa
On Sun, 9 Jan 2011 19:19:44 +0100 Diego Calleja wrote: Hello, > AFAIK swapfiles are not supported, at least for now. At the moment I use FreeBSD with zfs (two disks in mirror) and plan to return back to (Arch)linux. Considering that with zfs I've everything in one (z)pool - root, swap - I'd lik

Re: BTRFS should increase the hard-link in the same directory limit

2011-08-23 Thread David Nicol
so there are these hundreds of message files, and when one is read, a new link to The Markfile appears with a similar name as the read file? Is that right? if the point is to save inodes by making a directory entry that's a hardlink to something already existing, why not link to the message file?

Re: Suddenly, a dead filesystem

2011-08-23 Thread Hugo Mills
On Tue, Aug 23, 2011 at 03:27:27PM +0100, Hugo Mills wrote: > On Tue, Aug 23, 2011 at 05:15:46PM +0300, Avi Kivity wrote: > > I've been using btrfs for a while as my /home (converted from ext4; > > encrypted lvm) when it died on me.  Mounting it crashes immediately, > > here's a log: > [...] > > [

Re: Suddenly, a dead filesystem

2011-08-23 Thread Hugo Mills
On Tue, Aug 23, 2011 at 05:15:46PM +0300, Avi Kivity wrote: > I've been using btrfs for a while as my /home (converted from ext4; > encrypted lvm) when it died on me.  Mounting it crashes immediately, > here's a log: [...] > [    6.456117] Call Trace: > [    6.456117]  [] add_inode_ref+0x2e6/0x37c

Re: Suddenly, a dead filesystem

2011-08-23 Thread Josef Bacik
On 08/23/2011 10:15 AM, Avi Kivity wrote: > I've been using btrfs for a while as my /home (converted from ext4; > encrypted lvm) when it died on me. Mounting it crashes immediately, > here's a log: > > [6.455721] [ cut here ] > [6.456117] kernel BUG at fs/btrfs/ino

Suddenly, a dead filesystem

2011-08-23 Thread Avi Kivity
I've been using btrfs for a while as my /home (converted from ext4; encrypted lvm) when it died on me.  Mounting it crashes immediately, here's a log: [    6.455721] [ cut here ] [    6.456117] kernel BUG at fs/btrfs/inode.c:4586! [    6.456117] invalid opcode: [#1] SM

Re: btrfs over nfs

2011-08-23 Thread Jim Maloney
Thanks Hugo, I totally missed the fsid's. That works perfectly. Jim On Mon, Aug 22, 2011 at 5:28 PM, Hugo Mills wrote: > On Mon, Aug 22, 2011 at 05:07:45PM -0400, Jim Maloney wrote: >> I have been experimenting exporting btrfs subvolumes over nfs.  Main >> subvolume is filesys1 mounted at /files

[PATCH] Btrfs: fix an oops when deleting snapshots

2011-08-23 Thread Liu Bo
We can reproduce this oops via the following steps: $ mkfs.btrfs /dev/sdb7 $ mount /dev/sdb7 /mnt/btrfs $ for ((i=0; i<3; i++)); do btrfs sub snap /mnt/btrfs /mnt/btrfs/s_$i; done $ rm -fr /mnt/btrfs/* $ rm -fr /mnt/btrfs/* then we'll get [ cut here ] kernel BUG at fs/btrf