Re: Any hope of pool recovery?

2015-07-03 Thread Martin Steigerwald
On Friday 03 July 2015 09:31:03 Duncan wrote: Donald Pearson posted on Thu, 02 Jul 2015 13:19:41 -0500 as excerpted: btrfs restore complains that every device is missing except the one that you specify on executing the command. Multiple devices as a parameter isn't an option. Specifcy

Re: Any hope of pool recovery?

2015-07-03 Thread Donald Pearson
Thanks for the inputs guys. Yes I did learn to perform a device scan --all-devices. It seems that the chunk tree is vital to a lot of functionality and the recovery tools are no exception. I suspect that I ran in to the raid56 caveat btrfs does not deal well with a drive that is present but not

Re: [PATCH 0/2] btrfsck repair for I_ERR_FILE_WRONG_NBYTES and test case

2015-07-03 Thread David Sterba
On Thu, Jul 02, 2015 at 02:52:31PM +0800, Qu Wenruo wrote: Add repair function for I_ERR_FILE_WRONG_NBYTES and a test case for it. Rebased to devel branch. The second patch contains binary data, so created a pull request for it. https://github.com/kdave/btrfs-progs/pull/7 Qu Wenruo (2):

Re: btrfs full, but not full, can't rebalance

2015-07-03 Thread Rich Rauenzahn
I'm still seeing periodic issues here. The filesystem will go full occasionally even though there appears to be plenty of space. Running another rebalance seems to fix it ... but I don't see why the system thinks it needs to be rebalanced. # while ! btrfs balance start /; do btrfs fi show /;

Re: [PATCH] Btrfs: fix memory leak in the extent_same ioctl

2015-07-03 Thread Mark Fasheh
That looks great, thanks for fixing this Filipe. On Fri, Jul 03, 2015 at 08:48:22AM +0100, fdman...@kernel.org wrote: From: Filipe Manana fdman...@suse.com We were allocating memory with memdup_user() but we were never releasing that memory. This affected pretty much every call to the ioctl,

Re: btrfs full, but not full, can't rebalance

2015-07-03 Thread Chris Murphy
What kernel version are you using? I suggest reading the whole thread but in particular this note about CentOS 7 and Btrfs. http://www.spinics.net/lists/linux-btrfs/msg40850.html So kernel 3.16 isn't necessarily too old, but I know there's been a lot of on-going balance related work that's not

Re: [PATCH] Integer underflow in ctree.c

2015-07-03 Thread David Sterba
On Fri, Jun 19, 2015 at 11:31:16AM -0500, Sandino Araico Sánchez wrote: :btrfs check crashed while trying to fix my corrupted filesystem. btrfs check --repair /dev/sdd3 enabling repair mode Checking filesystem on /dev/sdd3 UUID: 58222ebc-79ca-4dc4-891f-129aae342313 checking extents bad

Re: Any hope of pool recovery?

2015-07-03 Thread Chris Murphy
On Fri, Jul 3, 2015 at 9:05 AM, Donald Pearson donaldwhpear...@gmail.com wrote: I did some more digging and found that I had a lot of errors basically every drive. Ick. Sucks for you but then makes this less of a Btrfs problem because it can really only do so much if more than the number of

Re: 4.1-rc6 - kernel crash after doing chattr +C

2015-07-03 Thread Filipe David Manana
On Sat, Jun 6, 2015 at 7:07 AM, Tomasz Chmielewski t...@virtall.com wrote: 4.1-rc6, busy filesystem. I was running mongo import which made quite a lot of IO. During the import, I did chattr +C /var/lib/mongodb - shortly after I saw this in dmesg and server died: [57860.149839] BUG: unable

[PATCH] Btrfs: fix list transaction-pending_ordered corruption

2015-07-03 Thread fdmanana
From: Filipe Manana fdman...@suse.com When we call btrfs_commit_transaction(), we splice the list ordered of our transaction handle into the transaction's pending_ordered list, but we don't reinitialize the ordered list of our transaction handle, this means it still points to the same elements it

Re: btrfs full, but not full, can't rebalance

2015-07-03 Thread Donald Pearson
what does the fi df , or btrfs fi usage show now On Fri, Jul 3, 2015 at 1:03 AM, Rich Rauenzahn rraue...@gmail.com wrote: Yes -- I just figured that out as well! Now why did it suddenly fill up? (I still get the failure rebalancing ...) # btrfs fi show / Label: 'centos7' uuid:

[PATCH] Btrfs: fix memory leak in the extent_same ioctl

2015-07-03 Thread fdmanana
From: Filipe Manana fdman...@suse.com We were allocating memory with memdup_user() but we were never releasing that memory. This affected pretty much every call to the ioctl, whether it deduplicated extents or not. This issue was reported on IRC by Julian Taylor and on the mailing list by Marcel

Re: linux 4.1 - memory leak (possibly dedup related)

2015-07-03 Thread Filipe David Manana
On Fri, Jul 3, 2015 at 7:58 AM, Marcel Ritter ritter.mar...@gmail.com wrote: Hi, I've been running some btrfs tests (mainly duperemove related) with linux kernel 4.1 for the last few days. Now I noticed by accident (dying processes), that all my memory (128 GB!) is gone. Gone meaning,

Re: btrfs full, but not full, can't rebalance

2015-07-03 Thread Rich Rauenzahn
Donald and I went offline so I will summarize where I am now: (1) Since I'm RAID1, I had to add two loopbacks. That makes sense! (2) The two 5GB loopbacks filled up almost instantly and did me no good. (3) I created two more 100 GB loopbacks (hint: use /usr/bin/truncate not /usr/bin/dd to

Re: btrfs full, but not full, can't rebalance

2015-07-03 Thread Rich Rauenzahn
Yes -- I just figured that out as well! Now why did it suddenly fill up? (I still get the failure rebalancing ...) # btrfs fi show / Label: 'centos7' uuid: 35f0ce3f-0902-47a3-8ad8-86179d1f3e3a Total devices 4 FS bytes used 17.12GiB devid1 size 111.11GiB used 111.05GiB path

Re: btrfs full, but not full, can't rebalance

2015-07-03 Thread Donald Pearson
Because this is raid1 I believe you need another for that to work. On Fri, Jul 3, 2015 at 12:57 AM, Rich Rauenzahn rraue...@gmail.com wrote: Yes, I tried that -- and adding the loopback device. # btrfs device add /dev/loop1 / Performing full device TRIM (5.00GiB) ... # btrfs fi show /

linux 4.1 - memory leak (possibly dedup related)

2015-07-03 Thread Marcel Ritter
Hi, I've been running some btrfs tests (mainly duperemove related) with linux kernel 4.1 for the last few days. Now I noticed by accident (dying processes), that all my memory (128 GB!) is gone. Gone meaning, there's no user space process allocating this memory. Digging deeper I found the

Re: Any hope of pool recovery?

2015-07-03 Thread Duncan
Donald Pearson posted on Thu, 02 Jul 2015 13:19:41 -0500 as excerpted: btrfs restore complains that every device is missing except the one that you specify on executing the command. Multiple devices as a parameter isn't an option. Specifcy /dev/disk/by-uuid/uuid claims that all devices are

Re: [RFC PATCH V11 05/21] Btrfs: subpagesize-blocksize: Read tree blocks whose size is PAGE_SIZE.

2015-07-03 Thread Chandan Rajendra
On Wednesday 01 Jul 2015 22:40:08 Liu Bo wrote: On Mon, Jun 01, 2015 at 08:52:40PM +0530, Chandan Rajendra wrote: In the case of subpagesize-blocksize, this patch makes it possible to read only a single metadata block from the disk instead of all the metadata blocks that map into a page.

Re: [RFC PATCH V11 12/21] Btrfs: subpagesize-blocksize: Search for all ordered extents that could span across a page.

2015-07-03 Thread Chandan Rajendra
On Wednesday 01 Jul 2015 22:47:10 Liu Bo wrote: On Mon, Jun 01, 2015 at 08:52:47PM +0530, Chandan Rajendra wrote: In subpagesize-blocksize scenario it is not sufficient to search using the first byte of the page to make sure that there are no ordered extents present across the page. Fix

Re: [RFC PATCH V11 09/21] Btrfs: subpagesize-blocksize: Direct I/O read: Work on sectorsized blocks.

2015-07-03 Thread Chandan Rajendra
On Wednesday 01 Jul 2015 22:45:00 Liu Bo wrote: On Mon, Jun 01, 2015 at 08:52:44PM +0530, Chandan Rajendra wrote: The direct I/O read's endio and corresponding repair functions work on page sized blocks. Fix this. Signed-off-by: Chandan Rajendra chan...@linux.vnet.ibm.com ---

[PATCH v2] Btrfs: fix memory leak in the extent_same ioctl

2015-07-03 Thread fdmanana
From: Filipe Manana fdman...@suse.com We were allocating memory with memdup_user() but we were never releasing that memory. This affected pretty much every call to the ioctl, whether it deduplicated extents or not. This issue was reported on IRC by Julian Taylor and on the mailing list by Marcel