Re: Receive on same subvolume

2014-02-03 Thread Felix Blanke
Hi Matthew, I'm not sure what you are trying to achive. Couldn't you simply do another snapshot of the subvolume? I don't understand why you want to use send/receive on the same subvolume to be honest. Regards, Felix On Wed, Jan 29, 2014 at 10:26 PM, Matthew Lai m...@matthewlai.ca wrote:

Re: [PATCH] Btrfs: disable snapshot aware defrag for now

2014-02-03 Thread David Sterba
On Wed, Jan 29, 2014 at 04:05:30PM -0500, Josef Bacik wrote: It's just broken and it's taking a lot of effort to fix it, so for now just disable it so people can defrag in peace. Thanks, Cc: sta...@vger.kernel.org Signed-off-by: Josef Bacik jba...@fb.com --- fs/btrfs/inode.c | 2 +- 1

[PATCH] Btrfs: add regression test for running snapshot and send concurrently

2014-02-03 Thread Wang Shilong
From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Signed-off-by: Wang Shilong wangsl.f...@cn.fujitsu.com --- tests/btrfs/034 | 75 +

btrfs raid5 unmountable

2014-02-03 Thread Tetja Rediske
Hi, since Freenode is doomed today, i ask the direct way. Following Filesystem: Label: 'data' uuid: 3a6fd6d7-5943-4cad-b56f-2e6dcabff453 Total devices 6 FS bytes used 7.02TiB devid1 size 1.82TiB used 1.82TiB path /dev/sda3 devid2 size 2.73TiB used 2.48TiB path

[PATCH RESEND] xfstests: add test for btrfs incremental send data corruption

2014-02-03 Thread Filipe David Borba Manana
Btrfs incremental send had an issue where it would detect a non-existent file hole and then overwrite the file section that hole covers with zeroes, overriding file data that it shouldn't. The respective btrfs kernel patch that fixed this issue is titled: Btrfs: fix send file hole detection

[PATCH] btrfs: send: replace check with an assert in gen_unique_name

2014-02-03 Thread David Sterba
The buffer passed to snprintf can hold the fully expanded format string, 64 = 3x largest ULL + 3x char + trailing null. I don't think that removing the check entirely is a good idea, hence the ASSERT. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/send.c |6 +- 1 files

Re: [PATCH] Btrfs: disable snapshot aware defrag for now

2014-02-03 Thread Josef Bacik
On 02/03/2014 09:48 AM, David Sterba wrote: On Wed, Jan 29, 2014 at 04:05:30PM -0500, Josef Bacik wrote: It's just broken and it's taking a lot of effort to fix it, so for now just disable it so people can defrag in peace. Thanks, Cc: sta...@vger.kernel.org Signed-off-by: Josef Bacik

Re: Receive on same subvolume

2014-02-03 Thread Chris Murphy
On Jan 29, 2014, at 2:26 PM, Matthew Lai m...@matthewlai.ca wrote: Hello, Is this supposed to work? (/data is the root volume, /data/a is a subvolume) btrfs subvolume snapshot /data/a /data/b # make some changes in b btrfs send -p /data/a /data/b delta btrfs receive /data/a delta

Re: [GIT PULL] Btrfs

2014-02-03 Thread David Sterba
On Thu, Jan 30, 2014 at 04:52:54PM -0500, Chris Mason wrote: Chris Mason (3) commits (+64/-32): Btrfs: setup inode location during btrfs_init_inode_locked (+9/-9) Btrfs: don't use ram_bytes for uncompressed inline items (+52/-22) The patches are CC: stable, but haven't gone through the

[PATCH v3] btrfs: add simple debugfs interface

2014-02-03 Thread David Sterba
Help during debugging to export various interesting information and tunables without the need of extra mount options or ioctls. Usage: * declare your variable in sysfs.h, and include where you need it * define the variable in sysfs.c and make it visible via debugfs_create_TYPE Depends on

Re: [GIT PULL] Btrfs

2014-02-03 Thread Chris Mason
On Mon 03 Feb 2014 12:54:05 PM EST, David Sterba wrote: On Thu, Jan 30, 2014 at 04:52:54PM -0500, Chris Mason wrote: Chris Mason (3) commits (+64/-32): Btrfs: setup inode location during btrfs_init_inode_locked (+9/-9) Btrfs: don't use ram_bytes for uncompressed inline items (+52/-22)

[PATCH 5/6] btrfs: send: remove BUG from process_all_refs

2014-02-03 Thread David Sterba
There are only 2 static callers, the BUG would normally be never reached, but let's be nice. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/send.c |5 - 1 files changed, 4 insertions(+), 1 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index

[PATCH 6/6] btrfs: send: remove BUG_ON from name_cache_delete

2014-02-03 Thread David Sterba
If cleaning the name cache fails, we could try to proceed at the cost of some memory leak. This is not expected to happen often. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/send.c | 11 +-- 1 files changed, 9 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/send.c

[PATCH 0/6] Btrfs send updates - reduce memory consumption

2014-02-03 Thread David Sterba
[Sorry if you see this twice, first attempt hasn't appeared in the list yet] This reduces size of the path buffer in common case. Has been tested by xfstests, but at the moment v3.13 with or without this patch blows, so I'm sending it anyway. Based on current btrfs-next/master. David Sterba

[PATCH 1/6] btrfs: send: remove prepared member from fs_path

2014-02-03 Thread David Sterba
The member is used only to return value back from fs_path_prepare_for_add, we can do it locally and save 8 bytes for the inline_buf path. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/send.c | 26 +- 1 files changed, 13 insertions(+), 13 deletions(-) diff

[PATCH 3/6] btrfs: send: squeeze bitfilelds in fs_path

2014-02-03 Thread David Sterba
We know that buf_len is at most PATH_MAX, 4k, and can merge it with the reversed member. This saves 3 bytes in favor of inline_buf. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/send.c |4 ++-- 1 files changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/send.c

[PATCH 4/6] btrfs: send: lower memory requirements in common case

2014-02-03 Thread David Sterba
The fs_path structure uses an inline buffer and falls back to a chain of allocations, but vmalloc is not necessary because PATH_MAX fits into PAGE_SIZE. The size of fs_path has been reduced to 256 bytes from PAGE_SIZE, usually 4k. Experimental measurements show that most paths on a single

[PATCH 2/6] btrfs: send: remove virtual_mem member from fs_path

2014-02-03 Thread David Sterba
We don't need to keep track of that, it's available via is_vmalloc_addr. Signed-off-by: David Sterba dste...@suse.cz --- fs/btrfs/send.c |8 ++-- 1 files changed, 2 insertions(+), 6 deletions(-) diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c index 524086a882f9..ea427624e842 100644 ---

Re: Receive on same subvolume

2014-02-03 Thread Matthew Lai
Thanks. I should clarify what I'm trying to do. I'm trying to use btrfs send for backup, without having another btrfs volume. So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the whole file, and doesn't have space for one). At the same time I'm

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-03 Thread Johannes Hirte
On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik jba...@fb.com wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because we have like 30 gigs of metadata, so our

Re: Receive on same subvolume

2014-02-03 Thread Chris Murphy
On Feb 3, 2014, at 11:19 AM, Matthew Lai m...@matthewlai.ca wrote: Thanks. I should clarify what I'm trying to do. I'm trying to use btrfs send for backup, without having another btrfs volume. So the initial backup is a complete send, piped to Amazon Glacier (so my machine never has the

Re: lost with degraded RAID1

2014-02-03 Thread Johan Kröckel
2014-01-30 Chris Murphy li...@colorremedies.com: On Jan 30, 2014, at 10:58 AM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Jan 30, 2014 at 10:33:21AM -0700, Chris Murphy wrote: You're doing an online conversion of a degraded raid1 volume into single? Does anyone know if this is expected or

Re: lost with degraded RAID1

2014-02-03 Thread Chris Murphy
On Feb 3, 2014, at 1:55 PM, Johan Kröckel johan.kroec...@gmail.com wrote: 2014-01-30 Chris Murphy li...@colorremedies.com: On Jan 30, 2014, at 10:58 AM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Jan 30, 2014 at 10:33:21AM -0700, Chris Murphy wrote: You're doing an online conversion of

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-03 Thread Josef Bacik
On 02/03/2014 01:28 PM, Johannes Hirte wrote: On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik jba...@fb.com wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because our transaction commit was taking like 3 minutes to complete. This is because

hitting BUG_ON on troublesome FS

2014-02-03 Thread Remco Hosman - Yerf-it.com
FIrst, a bit of history of the filesystem: used to be 6 disks, now 5. partially raid1 / raid10. been migrating back and forth a few times. As some point, a balance would not complete and would end with 164 ENOSPC’ses, while there was plenty of unallocated space on each disk. i scanned for

Re: lost with degraded RAID1

2014-02-03 Thread Johan Kröckel
State is: I wont use this filesystem again. I have a backup. So I am interested to give the necessary information for debuging it and afterwards format it and create a new one. I already did fscks and btrfschk --repair and pushed the output to txt-files but they are more than 4 mb in size. So I

Re: [PATCH] Btrfs: convert to add transaction protection for btrfs send

2014-02-03 Thread Josef Bacik
On 01/31/2014 11:37 AM, Wang Shilong wrote: Hello Josef, 在 2014-1-31,上午12:23,Josef Bacik jba...@fb.com 写道: On 01/30/2014 11:20 AM, Wang Shilong wrote: Hello Josef, On 01/30/2014 04:42 AM, Wang Shilong wrote: Hi Josef, On 01/29/2014 10:32 AM, Wang Shilong wrote: From: Wang Shilong

Re: Receive on same subvolume

2014-02-03 Thread Matthew Lai
On 03/02/2014 11:26 AM, Chris Murphy wrote: On Feb 3, 2014, at 11:19 AM, Matthew Lai m...@matthewlai.ca wrote: Thanks. I should clarify what I'm trying to do. I'm trying to use btrfs send for backup, without having another btrfs volume. So the initial backup is a complete send, piped to

Re: [PATCH] Btrfs: throttle delayed refs better

2014-02-03 Thread Johannes Hirte
On Mon, 3 Feb 2014 16:08:08 -0500 Josef Bacik jba...@fb.com wrote: On 02/03/2014 01:28 PM, Johannes Hirte wrote: On Thu, 23 Jan 2014 13:07:52 -0500 Josef Bacik jba...@fb.com wrote: On one of our gluster clusters we noticed some pretty big lag spikes. This turned out to be because

Re: [PATCH] Btrfs: add regression test for running snapshot and send concurrently

2014-02-03 Thread Dave Chinner
On Mon, Feb 03, 2014 at 11:22:36PM +0800, Wang Shilong wrote: From: Wang Shilong wangsl.f...@cn.fujitsu.com Btrfs would fail to send if snapshot run concurrently, this test is to make sure we have fixed the bug. Couple of comments below. +_scratch_mkfs /dev/null 21 +_scratch_mount + +

Re: Receive on same subvolume

2014-02-03 Thread Chris Murphy
On Feb 3, 2014, at 3:53 PM, Matthew Lai m...@matthewlai.ca wrote: On 03/02/2014 11:26 AM, Chris Murphy wrote: On Feb 3, 2014, at 11:19 AM, Matthew Lai m...@matthewlai.ca wrote: Thanks. I should clarify what I'm trying to do. I'm trying to use btrfs send for backup, without having another

Re: Receive on same subvolume

2014-02-03 Thread Matthew Lai
On 03/02/2014 4:34 PM, Chris Murphy wrote: On Feb 3, 2014, at 3:53 PM, Matthew Lai m...@matthewlai.ca wrote: On 03/02/2014 11:26 AM, Chris Murphy wrote: On Feb 3, 2014, at 11:19 AM, Matthew Lai m...@matthewlai.ca wrote: Thanks. I should clarify what I'm trying to do. I'm trying to use

Re: [PATCH] Btrfs: disable snapshot aware defrag for now

2014-02-03 Thread Roger Binns
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 03/02/14 09:27, Josef Bacik wrote: It is so totally broken that I don't want it being turned on by anybody who can't edit this and change it themselves. The symptoms I saw are huge amounts of kernel memory consumption, possibly till exhaustion

Re: btrfs raid5 unmountable

2014-02-03 Thread Duncan
Tetja Rediske posted on Mon, 03 Feb 2014 17:12:24 +0100 as excerpted: [...] What happened before: One disk was faulty, I added a new one and removed the old one, followed by a balance. So far so good. Some days after this I accidently removed a SATA Power Connector from another