[RFC][PATCH 0/2] btrfs-progs: introduce kernelsrc directory to update it

2013-02-12 Thread Filipe Brandenburger
Hello, This is a proposal on a way to keep btrfs-progs in sync with kernel code. https://btrfs.wiki.kernel.org/index.php/Cleanup_ideas#Use_the_kernel_code_in_user_mode For now, the patch introduces a method to import and update the kernel source files into the btrfs-progs tree. Further work will

[PATCH 1/2] btrfs-progs: anchor gitignore patterns on the root of the tree

2013-02-12 Thread Filipe Brandenburger
This is needed in order to allow checking out fs/btrfs/, otherwise the ignore rule for the btrfs binary would conflict with it. Signed-off-by: Filipe Brandenburger --- .gitignore | 32 +++- 1 file changed, 19 insertions(+), 13 deletions(-) diff --git a/.gitignore b/.

[PATCH 2/2] btrfs-progs: introduce kernelsrc directory and script to update it

2013-02-12 Thread Filipe Brandenburger
The kernelsrc directory will contain a copy of the Btrfs-related files from the Linux kernel source tree (mainly the files in fs/btrfs/ directory.) This commit introduces an initial filelist (generated by comparing the list of files in both directories) and a script to fetch the files from a local

Re: [PATCH] Btrfs: fix the deadlock between the transaction attach and commit

2013-02-12 Thread David Sterba
On Mon, Feb 11, 2013 at 03:35:37PM -0500, Josef Bacik wrote: > or something like that. Me and kdave reproduced by running 274 in a loop, it > happpened pretty quick. I'd fix it myself but I have to leave my house for > people to come look at it. If you haven't fixed this by tomorrow I'll fix it

Re: [PATCH] Btrfs-progs: add static compile target

2013-02-12 Thread David Sterba
On Sat, Feb 09, 2013 at 12:19:29AM +0100, Ian Kumlien wrote: > +# Define static compilation flags > +STATIC_CFLAGS = $(CFLAGS) -static -ffunction-sections -fdata-sections > +STATIC_LDFLAGS = -Wl,--gc-sections > +STATIC_LIBS = $(LIBS) -lpthread > +btrfs.static: $(static_objects) btrfs.static.o help

Re: [PATCH] Btrfs-progs: -U_FORTIFY_SOURCE before -D

2013-02-12 Thread David Sterba
On Sat, Feb 09, 2013 at 09:02:06PM +0300, Sergei Trofimovich wrote: > On Sat, 9 Feb 2013 00:30:21 +0100 > Ian Kumlien wrote: > > > My builds are cluttered with: > > :0:0: warning: "_FORTIFY_SOURCE" redefined [enabled by > > default] > > > > Which makes it hard to tell if something breaks or not

Re: [PATCH] Btrfs: fix the deadlock between the transaction attach and commit

2013-02-12 Thread Josef Bacik
On Thu, Feb 07, 2013 at 11:55:51PM -0700, Miao Xie wrote: > Here is the whole story: > Trans_Attach_Task Trans_Commit_Task > btrfs_commit_transaction() >|->wait writers to be 1 > btrfs_attach_tra

Re: [PATCH] Btrfs-progs: check out if the swap device

2013-02-12 Thread David Sterba
On Tue, Feb 12, 2013 at 10:25:23AM +0900, Tsutomu Itoh wrote: > Currently, the following commands succeed. > > # cat /proc/swaps > FilenameTypeSizeUsed > Priority > /dev/sda3 partition 8388604 0 -1 >

Re: experimental raid5/6 code in git

2013-02-12 Thread Kaspar Schleiser
Hey Chris, On 02/02/2013 05:02 PM, Chris Mason wrote: > Btrfs -- 604MB/s > MD-- 162MB/s > > > MD -- 800MB/s very little system time > Btrfs -- 3.8GB/s one CPU mostly pegged > Btrfs -- 380MB/s seen by fio > MD-- 174MB/s seen by fio > Creating 12 million files on Btrfs raid5 took 226 sec

Re: [PATCH 6/6] Btrfs-progs: add the rescue section to btrfs

2013-02-12 Thread David Sterba
On Fri, Feb 08, 2013 at 01:37:02AM +0100, Ian Kumlien wrote: > the btrfs command now lists: > btrfs rescue select-super -s > Select a superblock > btrfs rescue dump-super > Dump a superblock to disk > btrfs rescue debug-tree [options] > Debug the filesystem

Re: experimental raid5/6 code in git

2013-02-12 Thread Chris Mason
On Tue, Feb 12, 2013 at 08:16:49AM -0700, Kaspar Schleiser wrote: > Hey Chris, > > On 02/02/2013 05:02 PM, Chris Mason wrote: > > Btrfs -- 604MB/s > > MD-- 162MB/s > > > > > > MD -- 800MB/s very little system time > > Btrfs -- 3.8GB/s one CPU mostly pegged > > > Btrfs -- 380MB/s seen by fio

[PATCH] Btrfs: eliminate a use-after-free in btrfs_balance()

2013-02-12 Thread Ilya Dryomov
Commit 5af3e8cc introduced a use-after-free at volumes.c:3139: bctl is freed above in __cancel_balance() in all cases except for balance pause. Fix this by moving the offending check a couple statements above, the meaning of the check is preserved. Reported-by: Chris Mason Signed-off-by: Ilya Dr

[PATCH] Btrfs: allow for selecting only completely empty chunks

2013-02-12 Thread Ilya Dryomov
Enhance balance usage filter by making it possible to balance out only completely empty chunks. Today, usage filter properly acts on values from 1 to 99 inclusive, usage=100 selects all chunks, and usage=0 selects no chunks. This commit changes the usage=0 case: the new meaning is to restripe onl

Re: [PATCH] btrfs: accept zero for balance usage filter

2013-02-12 Thread Ilya Dryomov
On Mon, Feb 11, 2013 at 01:33:52PM +0100, David Sterba wrote: > The condition can be relaxed to accept also 0 which will delete > unoccupied chunks and does not need space for the actual data > relocation. Until there is an automatic empty chunk reclaim, we > can use this as a last resort option un

Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs

2013-02-12 Thread David Sterba
On Fri, Feb 08, 2013 at 01:36:58AM +0100, Ian Kumlien wrote: > -btrfsck: $(objects) btrfsck.o > - @echo "[LD] $@" > - $(Q)$(CC) $(CFLAGS) -o btrfsck btrfsck.o $(objects) $(LDFLAGS) $(LIBS) > - Do we want to get rid of the btrfsck binary completely? Using the term 'btrsfck' is commo

[PATCH] Btrfs-progs: make 0 a valid usage filter argument

2013-02-12 Thread Ilya Dryomov
This is a progs counterpart to a "Btrfs: allow for selecting only completely empty chunks". usage=0 now means "select only only completely empty chunks and nothing else". Signed-off-by: Ilya Dryomov --- cmds-balance.c |2 +- 1 files changed, 1 insertions(+), 1 deletions(-) diff --git a/cmd

Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs

2013-02-12 Thread Filipe Brandenburger
Hi, On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: > +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's > named 'btrfsck' > +btrfsck: btrfs > + @echo "[CP] $@" > + $(Q)cp btrfs btrfsck > + I think the idea was that btrfsck becomes a link (either s

Re: [PATCH] Btrfs-progs: check out if the swap device

2013-02-12 Thread Eric Sandeen
On 2/11/13 11:50 PM, Tsutomu Itoh wrote: > Hi, Eric, > > Thanks for your comment. > > On 2013/02/12 13:22, Eric Sandeen wrote: >> On 2/11/13 7:25 PM, Tsutomu Itoh wrote: >>> Currently, the following commands succeed. >>> >>> # cat /proc/swaps >>> FilenameType

Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs

2013-02-12 Thread Goffredo Baroncelli
On 02/12/2013 06:37 PM, Filipe Brandenburger wrote: > Hi, > > On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: >> +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's >> named 'btrfsck' >> +btrfsck: btrfs >> + @echo "[CP] $@" >> + $(Q)cp btrfs btrfsck

Re: [PATCH] Btrfs-progs: check out if the swap device

2013-02-12 Thread Goffredo Baroncelli
On 02/12/2013 02:25 AM, Tsutomu Itoh wrote: > Currently, the following commands succeed. > > # cat /proc/swaps > FilenameTypeSizeUsed > Priority > /dev/sda3 partition 8388604 0 -1 > /dev/sdc8

What can I do to make btrfs work?

2013-02-12 Thread Richard W.M. Jones
Btrfs has been broken for me for ages. I first reported it on this list 5 months ago[1]. Below is a very simple reproducer that anyone can run. *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an unused block device! -

Re: What can I do to make btrfs work?

2013-02-12 Thread Chris Mason
On Tue, Feb 12, 2013 at 11:54:49AM -0700, Richard W.M. Jones wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unu

[PATCH] Btrfs: relax the block group size limit for bitmaps

2013-02-12 Thread Josef Bacik
Dave pointed out that xfstests 273 will tell you that it failed to load the space cache for a block group when it remounts. This is because we run out of space writing out the block group cache. This is ok and is working as it should, but let's try to be a bit nicer. This happens because the blo

Re: What can I do to make btrfs work?

2013-02-12 Thread Josef Bacik
On Tue, Feb 12, 2013 at 11:54:49AM -0700, Richard W.M. Jones wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unu

[PATCH] Btrfs-progs: move crc32c optimization init

2013-02-12 Thread Ilya Dryomov
Don't call crc32c_optimization_init() until we know that a command is actually going to be invoked. Signed-off-by: Ilya Dryomov --- btrfs.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/btrfs.c b/btrfs.c index 687acec..7752bd6 100644 --- a/btrfs.c +++ b/btrfs.c @@ -

Re: What can I do to make btrfs work?

2013-02-12 Thread Zach Brown
On Tue, Feb 12, 2013 at 06:54:49PM +, Richard W.M. Jones wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. The very simple reproducer doesn't fail over here on bare hardware for me. #

Re: What can I do to make btrfs work?

2013-02-12 Thread Roman Mamedov
On Tue, 12 Feb 2013 18:54:49 + "Richard W.M. Jones" wrote: > Btrfs has been broken for me for ages. I first reported it on this > list 5 months ago[1]. Below is a very simple reproducer that anyone > can run. > > *NB* before you run this, adjust /dev/sda & /dev/sda1 to point to an > unused

Re: [PATCH] Btrfs-progs: check out if the swap device

2013-02-12 Thread Zach Brown
> > > > So, I chose this one. (read /proc/swaps) > > Sure, I think your change is good. I just think perhaps mkfs should also try > to open O_EXCL after all those other tests, as a last safety check. I think mkfs should first try an O_EXCL open. If that works it doesn't need to do any of this

Re: What can I do to make btrfs work?

2013-02-12 Thread Richard W.M. Jones
On Tue, Feb 12, 2013 at 02:16:37PM -0500, Josef Bacik wrote: > On Tue, Feb 12, 2013 at 11:54:49AM -0700, Richard W.M. Jones wrote: > > Btrfs has been broken for me for ages. I first reported it on this > > list 5 months ago[1]. Below is a very simple reproducer that anyone > > can run. > > > > *

Re: What can I do to make btrfs work?

2013-02-12 Thread Richard W.M. Jones
On Tue, Feb 12, 2013 at 11:44:56AM -0800, Zach Brown wrote: > On Tue, Feb 12, 2013 at 06:54:49PM +, Richard W.M. Jones wrote: > > Btrfs has been broken for me for ages. I first reported it on this > > list 5 months ago[1]. Below is a very simple reproducer that anyone > > can run. > > The ve

Re: btrfs and LVM snapshots (Re: kernel BUG at fs/btrfs/extent-tree.c:1772)

2013-02-12 Thread Piotr Pawłow
I can confirm that, even with a single-device btrfs filesystem. However I am curious why you want to use the lvm snapshot capability instead of the btrfs one. You can't use btrfs snapshots on a broken FS. LVM snapshots would be useful to save the original state, before any potentially destructi

Re: What can I do to make btrfs work?

2013-02-12 Thread Chris Mason
On Tue, Feb 12, 2013 at 02:05:35PM -0700, Richard W.M. Jones wrote: > > Yes, this is inside a very recent KVM (qemu 1.3.0), using virtio-scsi > as the backing disk. Ok, can you please run this on your virtio device file? It will overwrite the first 256K, so don't do this on a file you care about

[RFC] fix async ordered operations flush deadlock

2013-02-12 Thread Josef Bacik
Hello, So btrfs_commit_transaction does this ret = btrfs_run_ordered_operations(root, 0); which async flushes all inodes on the ordered operations list. The problem with this is that we wait for this flushing to finish, so we end up with this Task 1 Task 2

Re: [PATCH] btrfs: accept zero for balance usage filter

2013-02-12 Thread David Sterba
On Tue, Feb 12, 2013 at 06:36:17PM +0200, Ilya Dryomov wrote: > Josef, please don't pull this one into btrfs-next, it's been deprecated > by "Btrfs: allow for selecting only completely empty chunks" from me. > David and I discussed this on IRC, he is OK with it. Ack. david -- To unsubscribe from

Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs

2013-02-12 Thread David Sterba
On Tue, Feb 12, 2013 at 07:01:33PM +0100, Goffredo Baroncelli wrote: > On 02/12/2013 06:37 PM, Filipe Brandenburger wrote: > > Hi, > > > > On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: > >> +# For backward compatibility, 'btrfs' changes behaviour to fsck if it's > >> named 'btrfsck' > >>

Re: [PATCH 2/6] Btrfs-progs: add btrfsck functionality to btrfs

2013-02-12 Thread Goffredo Baroncelli
On 02/12/2013 11:52 PM, David Sterba wrote: > On Tue, Feb 12, 2013 at 07:01:33PM +0100, Goffredo Baroncelli wrote: >> On 02/12/2013 06:37 PM, Filipe Brandenburger wrote: >>> Hi, >>> >>> On Tue, Feb 12, 2013 at 8:39 AM, David Sterba wrote: +# For backward compatibility, 'btrfs' changes behavio

Integration branch of btrfs-progs 2013-02

2013-02-12 Thread David Sterba
Hi, a few more fixes and enhancements: check and restore subcommands, support for send mode NO_FILE_DATA and partial "libifying". The libify patchset porting was not straightforward (based on old sources) and I fixed it here up to my satisfaction. The changes were mostly small but there was quit

[PATCH 20/21] btrfs-progs: don't link binaries to a dynamic library

2013-02-12 Thread David Sterba
Linking 'btrfs' and other binaries against the dynamic library makes it tedious to use directly from the git repo. This is useful for testing various fixes, but now it'd need to also set LD_LIBRARY_PATH or install the library to a known path. Add a target for static library and use it for linking,

[PATCH 17/21] btrfs-progs: libify some parts of btrfs-progs

2013-02-12 Thread Mark Fasheh
External software wanting to use the functionality provided by the btrfs send ioctl has a hard time doing so without replicating tons of work. Of particular interest are functions like btrfs_read_and_process_send_stream() and subvol_uuid_search(). As that functionality requires a bit more than just

[PATCH 13/21] btrfs-progs: build btrsfck to keep compatibility

2013-02-12 Thread David Sterba
The command 'btrfsck' is commonly used and we should build it by default. Signed-off-by: David Sterba --- Makefile | 10 -- 1 file changed, 8 insertions(+), 2 deletions(-) diff --git a/Makefile b/Makefile index 039f0dc..7a679d0 100644 --- a/Makefile +++ b/Makefile @@ -1,4 +1,5 @@ CC =

Rebalancing RAID1

2013-02-12 Thread Fredrik Tolf
Dear list, I'm sorry if this is a dumb n3wb question, but I couldn't find anything about it, so please bear with me. I just decided to try BtrFS for the first time, to replace an old ReiserFS data partition currently on a mdadm mirror. To do so, I'm using two 3 TB disks that were initially d

Re: Rebalancing RAID1

2013-02-12 Thread Chris Murphy
On Feb 12, 2013, at 4:01 PM, Fredrik Tolf wrote: > > mkfs.btrfs -d raid -m raid1 /dev/sd{d,e}1 Is that a typo? -d raid isn't valid. What do you get for: btrfs fi df /mnt Please report the result for each drive: smartctl -a /dev/sdX smartctl -l scterc /dev/sdX > > Also, why does it say that

Re: Integration branch of btrfs-progs 2013-02

2013-02-12 Thread Filipe Brandenburger
Hi David, I really have concerns about the libification, in particular this commit: 6fc8b21 btrfs-progs: libify some parts of btrfs-progs The relevant snippets that concern me below: +libbtrfs_headers = send-stream.h send-utils.h send.h rbtree.h btrfs-list.h \ + crc32c.h list.h ker

Re: [PATCH] Btrfs-progs: check out if the swap device

2013-02-12 Thread Tsutomu Itoh
Hi, All, Thanks for advice. On 2013/02/13 5:57, Zach Brown wrote: So, I chose this one. (read /proc/swaps) Sure, I think your change is good. I just think perhaps mkfs should also try to open O_EXCL after all those other tests, as a last safety check. I think mkfs should first try an O_EX

Re: Rebalancing RAID1

2013-02-12 Thread Fredrik Tolf
On Tue, 12 Feb 2013, Chris Murphy wrote: On Feb 12, 2013, at 4:01 PM, Fredrik Tolf wrote: mkfs.btrfs -d raid -m raid1 /dev/sd{d,e}1 Is that a typo? -d raid isn't valid. Ah yes, sorry. That was a typo. What do you get for: btrfs fi df /mnt $ sudo ./btrfs fi df /mnt Data, RAID1: total=2