Re: [RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-01-15 Thread Liu Bo
On Wed, Jan 15, 2014 at 08:00:58PM +0800, Miao Xie wrote: > When we mounted the filesystem after the crash, we got the following > message: > BTRFS error (device xxx): block group 4315938816 has wrong amount of free > space > BTRFS error (device xxx): failed to load free space cache for block

Re: BTRFS critical: unable to find logical 0 len 4096, No mapping for 0-4096

2014-01-15 Thread Wang Shilong
Hello Tomasz, This problem should be fixed by Filipe, and pused into josef's btrfs-next:-) : patch url is: https://git.kernel.org/cgit/linux/kernel/git/josef/btrfs-next.git/commit/?id=5d157162be99f950a0b64b526c04dff03f8c4eeb Thanks, Wang On 01/16/2014 01:27 PM, Tomasz Chmielewski wrote: Yet

BTRFS critical: unable to find logical 0 len 4096, No mapping for 0-4096

2014-01-15 Thread Tomasz Chmielewski
Yet another btrfs filesystem getting bad: [43013.108612] [ cut here ] [43013.108675] WARNING: CPU: 2 PID: 30756 at fs/btrfs/ctree.c:1321 btrfs_search_old_slot+0x322/0x7ea [btrfs]() [43013.108766] Modules linked in: ipt_MASQUERADE iptable_nat nf_conntrack_ipv4 nf_defrag_ip

[PATCH] btrfs-progs: add help switches where needed

2014-01-15 Thread Mitchel Humpherys
Some of the btrfs programs don't support -h or --help. It's not really a problem since everyone seems to fall back to printing usage text when an option is not recognized, so passing -h or --help usually gets you the usage text. However, it also usually gets you a "invalid option -- 'h'". Fix this

[PATCH] add help switches

2014-01-15 Thread Mitchel Humpherys
Some -h and --help love. Mitchel Humpherys (1): btrfs-progs: add help switches where needed btrfs-calc-size.c | 3 ++- btrfs-convert.c | 3 ++- btrfs-corrupt-block.c | 3 ++- btrfs-debug-tree.c| 3 ++- btrfs-find-root.c | 3 ++- btrfs-image.c | 3 ++- btrfs-map-logica

[PATCH 1/2 v3] xfstests: make btrfs/001.out in line with subvol delete output

2014-01-15 Thread Anand Jain
From: Anand Jain btrfs/001 is failing as the below btrfs-progs patch changed the output during subvol delete. Patch : btrfs-progs: add options to set commit mode after subvol delete adding it to the filter Signed-off-by: Anand Jain --- v3: takes in David suggested v2: accepts Wang suggested

Re: kernel BUG at fs/btrfs/relocation.c:4026!

2014-01-15 Thread Wang Shilong
Hello Tomasz, Looking at lines 4026, it is a BUG_ON() triggered because we fail to commit transaction,from dmesg, it is because of ENOSPC issue. However, we should not BUG_ON here, it has been deleted from flighting patches and haven't been merge into upstream yet.^_^ Thanks, Wang On 01/16/2014

Re: [PATCH v2 2/4] Btrfs: fix protection between send and root deletion

2014-01-15 Thread Miao Xie
On wed, 15 Jan 2014 17:40:38 +0100, David Sterba wrote: > On Tue, Jan 14, 2014 at 10:22:56AM +0800, Miao Xie wrote: >> On mon, 13 Jan 2014 19:27:45 +0100, David Sterba wrote: >> The root what is about to be sent out may be deleted after we get it, if >> there is no in-memory i-node in it, it will b

Re: 2 year old raid1 issue chunk-recovery help

2014-01-15 Thread Miao Xie
On wed, 15 Jan 2014 11:40:09 -0800, Vladi Gergov wrote: > Hi, in 2010 i had an issue with my raid1 when one drive failed and i > added another drive to the array and tried to rebuild. Here is what bug > I hit according to Chris Mason > http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg0686

[PATCH] btrfs: add simple debugfs interface

2014-01-15 Thread David Sterba
Help during debugging to export various interesting infromation and tunables without the need of extra mount options or ioctls. Usage: * declare your variable in sysfs.h, and include where you need it * define the variable in sysfs.c and make it visible via debugfs_create_TYPE Depends on CONFIG

kernel BUG at fs/btrfs/relocation.c:4026!

2014-01-15 Thread Tomasz Chmielewski
I was doing balance and hit these: [34363.498746] [ cut here ] [34363.498825] WARNING: CPU: 3 PID: 2237 at fs/btrfs/super.c:254 __btrfs_abort_transaction+0x4d/0xff [btrfs]() [34363.498937] btrfs: Transaction aborted (error -28) [34363.498938] Modules linked in: xt_tcpudp i

Re: why am I getting "No space left on device" here?

2014-01-15 Thread Duncan
Martin Steigerwald posted on Wed, 15 Jan 2014 20:40:29 +0100 as excerpted: > Am Mittwoch, 15. Januar 2014, 19:05:41 schrieb Duncan: >> But just as my already allocated mixed-mode chunks were just about full >> and I needed another one allocated to complete the job, so your data >> chunks are full

Re: why am I getting "No space left on device" here?

2014-01-15 Thread Tomasz Chmielewski
> > # dd if=/dev/urandom of=bigfile > > dd: writing to `bigfile': No space left on device > > 186+0 records in > > 185+0 records out > > 94720 bytes (95 kB) copied, 0.0144045 s, 6.6 MB/s > > > > > > I don't understand why - can anyone explain? > > What kernel version? Can you: > > umount > dmes

Re: [PATCH] btrfs: call permission checks earlier in ioctls and return EPERM

2014-01-15 Thread Gerhard Heift
2014/1/15 David Sterba : > The owner and capability checks in IOC_SUBVOL_SETFLAGS and > SET_RECEIVED_SUBVOL should be called before any other checks are done. > > Also unify the error code to EPERM. > > Signed-off-by: David Sterba > --- > fs/btrfs/ioctl.c | 22 +- > 1 file cha

2 year old raid1 issue chunk-recovery help

2014-01-15 Thread Vladi Gergov
Hi, in 2010 i had an issue with my raid1 when one drive failed and i added another drive to the array and tried to rebuild. Here is what bug I hit according to Chris Mason http://www.mail-archive.com/linux-btrfs@vger.kernel.org/msg06868.html I have since updated to lastes btrfs-tools 3.12 + kernel

Re: why am I getting "No space left on device" here?

2014-01-15 Thread Martin Steigerwald
Am Mittwoch, 15. Januar 2014, 19:05:41 schrieb Duncan: > But just as my already allocated mixed-mode chunks were just about full > and I needed another one allocated to complete the job, so your data > chunks are full or very close, according to btrfs fi df, and you need a > new one allocated (a

Re: why am I getting "No space left on device" here?

2014-01-15 Thread Chris Murphy
On Jan 15, 2014, at 3:55 AM, Tomasz Chmielewski wrote: > I'm no longer able to write to this btrfs filesystem: > > # df -h /home > Filesystem Size Used Avail Use% Mounted on > /dev/sdb4 5.2T 3.6T 1.6T 71% /home > > > # btrfs fi show /home > Label: crawler-btrfs uuid: 60f1759c-

Re: BTRFS_SEARCH_ARGS_BUFSIZE too small

2014-01-15 Thread Gerhard Heift
I have already patches for such an ioctl. Which form do you prefer? struct btrfs_ioctl_search_args_inline { struct btrfs_ioctl_search_key key; __u64 buf_len char buf[0]; }; or struct btrfs_ioctl_search_args_extern { struct btrfs_ioctl_search_key key; __u64

Re: why am I getting "No space left on device" here?

2014-01-15 Thread Duncan
Tomasz Chmielewski posted on Wed, 15 Jan 2014 11:55:43 +0100 as excerpted: > I'm no longer able to write to this btrfs filesystem: > > # df -h /home > Filesystem Size Used Avail Use% Mounted on > /dev/sdb4 5.2T 3.6T 1.6T 71% /home FWIW, standard df doesn't really know how to work

[PATCH] Btrfs: make fsync latency less sucky

2014-01-15 Thread Josef Bacik
Looking into some performance related issues with large amounts of metadata revealed that we can have some pretty huge swings in fsync() performance. If we have a lot of delayed refs backed up (as you will tend to do with lots of metadata) fsync() will wander off and try to run some of those delay

Re: BTRFS_SEARCH_ARGS_BUFSIZE too small

2014-01-15 Thread David Sterba
On Fri, Jan 10, 2014 at 12:44:17AM +0100, Gerhard Heift wrote: > I'm playing around with the BTRFS_IOC_SEARCH_TREE to extract the csums > of the physical blocks. During the tests some item_header had len = 0, > which indicates the buffer was to small to hold the item. I added a > printk into the ke

Btrfs-progs Relicensing effort has begun

2014-01-15 Thread Andy Grover
Hi all, We want to relicense btrfs-progs, using the LGPLv2.1 instead of GPLv2. We can then build btrfs-progs code as a shared library, libbtrfs, and this library can be used by other code -- GPLv3+, BSD/Apache, and non-free, without causing the GPLv2 to apply to the other code. This will all

[PATCH] btrfs: call permission checks earlier in ioctls and return EPERM

2014-01-15 Thread David Sterba
The owner and capability checks in IOC_SUBVOL_SETFLAGS and SET_RECEIVED_SUBVOL should be called before any other checks are done. Also unify the error code to EPERM. Signed-off-by: David Sterba --- fs/btrfs/ioctl.c | 22 +- 1 file changed, 9 insertions(+), 13 deletions(-) d

Re: Hitting error after failed balance

2014-01-15 Thread Duncan
Mitchel Humpherys posted on Tue, 14 Jan 2014 15:21:19 -0800 as excerpted: > On Tue, Jan 14 2014 at 11:47:18 AM, Mitchel Humpherys > wrote: >> >> Btrfs v3.12 >> >> $ uname -a >> Linux mitchelh-linux 3.12.6-1-ARCH #1 SMP PREEMPT Fri >> Dec 20 19:39:00 CET 2013 x86_64 GNU/Linux

[PATCH] btrfs: restrict snapshotting to own subvolumes

2014-01-15 Thread David Sterba
Currently, any user can snapshot any subvolume if the path is accessible and thus indirectly create and keep files he does not own under his direcotries. This is not possible with traditional directories. In security context, a user can snapshot root filesystem and pin any potentially buggy binari

Re: btrfsck failes

2014-01-15 Thread Chris Murphy
On Jan 15, 2014, at 9:15 AM, Mitch Harder wrote: > On Mon, Jan 13, 2014 at 6:37 PM, Chris Murphy wrote: >> >> On Jan 13, 2014, at 3:58 PM, Holger Brandsmeier >> wrote: >>> >>> Currently btrfsck failes to repair my partition, I get the output: >>> >>> [root@ho-think bholger]# btrfsck --repa

Re: btrfsck failes

2014-01-15 Thread Chris Murphy
On Jan 15, 2014, at 8:15 AM, Marc MERLIN wrote: > On Wed, Jan 15, 2014 at 08:53:55AM +0100, Holger Brandsmeier wrote: >> # btrfs-zero-log /dev/sda5 >> parent transid verify failed on 602529792 wanted 23460 found 23463 >> parent transid verify failed on 602529792 wanted 23460 found 23463 >> paren

Re: [PATCH v2 2/4] Btrfs: fix protection between send and root deletion

2014-01-15 Thread David Sterba
On Tue, Jan 14, 2014 at 10:22:56AM +0800, Miao Xie wrote: > On mon, 13 Jan 2014 19:27:45 +0100, David Sterba wrote: > The root what is about to be sent out may be deleted after we get it, if > there is no in-memory i-node in it, it will be inserted into the dead_roots > list > immediately, then th

Ext4 (and xfs and btrfs?) meetings around LSF

2014-01-15 Thread Ric Wheeler
On 01/13/2014 12:45 AM, Jan Kara wrote: Hello, On Sat 11-01-14 13:05:20, Ted Tso wrote: On Fri, Jan 10, 2014 at 12:58:34PM +0100, Jan Kara wrote: do you plan to organize an ext4 meeting around LSF? And do you plan it during Collab summit or on Sunday before LSF? Thanks for answer in adva

Re: btrfsck failes

2014-01-15 Thread Mitch Harder
On Mon, Jan 13, 2014 at 6:37 PM, Chris Murphy wrote: > > On Jan 13, 2014, at 3:58 PM, Holger Brandsmeier wrote: >> >> Currently btrfsck failes to repair my partition, I get the output: >> >> [root@ho-think bholger]# btrfsck --repair /dev/sda5 > > This is almost the last resort and you probably sh

Re: [PATCH 1/5] Btrfs: change the members' order of btrfs_space_info structure to reduce the cache miss

2014-01-15 Thread David Sterba
On Wed, Jan 15, 2014 at 08:00:54PM +0800, Miao Xie wrote: > It is better that the position of the lock is close to the data which is > protected by it, because they may be in the same cache line, we will load > less cache lines when we access them. So we rearrange the members' position > of btrfs_s

Re: btrfsck failes

2014-01-15 Thread Marc MERLIN
On Mon, Jan 13, 2014 at 05:37:31PM -0700, Chris Murphy wrote: > > On Jan 13, 2014, at 3:58 PM, Holger Brandsmeier wrote: > > > > Currently btrfsck failes to repair my partition, I get the output: > > > > [root@ho-think bholger]# btrfsck --repair /dev/sda5 > > This is almost the last resort and

Re: Fwd: btrfsck failes

2014-01-15 Thread Marc MERLIN
On Wed, Jan 15, 2014 at 08:53:55AM +0100, Holger Brandsmeier wrote: > # btrfs-zero-log /dev/sda5 > parent transid verify failed on 602529792 wanted 23460 found 23463 > parent transid verify failed on 602529792 wanted 23460 found 23463 > parent transid verify failed on 602529792 wanted 23460 found 2

Re: [PATCH 1/2 v2] xfstests: make btrfs/001.out in line with subvol delete output

2014-01-15 Thread David Sterba
On Tue, Jan 14, 2014 at 11:23:48AM +0800, Anand Jain wrote: > --- a/common/filter > +++ b/common/filter > @@ -313,5 +313,9 @@ _filter_ro_mount() { > -e "s/mount: cannot mount block device/mount: cannot mount/g" > } > > +_filter_transcation_commit_default() { > + sed -e "/Transactio

Re: [PATCH] Btrfs: don't miss roots in deletion during walking backrefs

2014-01-15 Thread Wang Shilong
Please ignore this patch, there is still a race condition between root deletion and walking backrefs, i will send them together later! Thanks, Wang On 01/14/2014 08:36 PM, Wang Shilong wrote: btrfs_read_fs_root_no_name() will check its root refs, and will return us ENOENT if this root in deletio

[PATCH 4/5] Btrfs: fix wrong block group in trace during the free space allocation

2014-01-15 Thread Miao Xie
We allocate the free space from the former block group, not the current one, so should use the former one to output the trace information. Signed-off-by: Miao Xie --- fs/btrfs/extent-tree.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/ex

[RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-01-15 Thread Miao Xie
When we mounted the filesystem after the crash, we got the following message: BTRFS error (device xxx): block group 4315938816 has wrong amount of free space BTRFS error (device xxx): failed to load free space cache for block group 4315938816 It is because we didn't update the metadata of th

[PATCH 3/5] Btrfs: cleanup the code of used_block_group in find_free_extent()

2014-01-15 Thread Miao Xie
used_block_group is just used for the space cluster which doesn't belong to the current block group, the other place needn't use it. Or the logic of code seems unclear. Signed-off-by: Miao Xie --- fs/btrfs/extent-tree.c | 33 + 1 file changed, 13 insertions(+), 20

[PATCH 2/5] Btrfs: cleanup the redundant code for the block group allocation and init

2014-01-15 Thread Miao Xie
Signed-off-by: Miao Xie --- fs/btrfs/extent-tree.c | 94 +++--- 1 file changed, 44 insertions(+), 50 deletions(-) diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 45d98d0..69a79ed 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/ext

[PATCH 1/5] Btrfs: change the members' order of btrfs_space_info structure to reduce the cache miss

2014-01-15 Thread Miao Xie
It is better that the position of the lock is close to the data which is protected by it, because they may be in the same cache line, we will load less cache lines when we access them. So we rearrange the members' position of btrfs_space_info structure to make the lock be closer to the its data. S

why am I getting "No space left on device" here?

2014-01-15 Thread Tomasz Chmielewski
I'm no longer able to write to this btrfs filesystem: # df -h /home Filesystem Size Used Avail Use% Mounted on /dev/sdb4 5.2T 3.6T 1.6T 71% /home # btrfs fi show /home Label: crawler-btrfs uuid: 60f1759c-45f6-4484-9f60-66a4e9bbf2b6 Total devices 2 FS bytes used 1.80TiB

[RFC PATCH] btrfs: fix null pointer deference at btrfs_sysfs_add_one+0x105

2014-01-15 Thread Anand Jain
bdev is null when disk has disappeared and mounted with the degrade option stack trace - btrfs_sysfs_add_one+0x105/0x1c0 [btrfs] open_ctree+0x15f3/0x1fe0 [btrfs] btrfs_mount+0x5db/0x790 [btrfs] ? alloc_pages_current+0xa4/0x160 mount_fs+0x34/0x1b0 vfs_kern_mount+0x62/0xf0 do_mount+0x22e/0xa