Re: "btrfs rescue super-recover" memory corruption

2014-09-18 Thread Wang Shilong
Hi Eric, > This: > > # truncate --size=8g > # dd if=/dev/zero of=file conv=notrunc bs=4 seek=16384 count=1 > # valgrind ./btrfs rescue super-recover file -v > > yields: > > ==4604== Memcheck, a memory error detector > ==4604== Copyright (C) 2002-2012, and GNU GPL'd, by Julian Seward et al. > =

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Liu Bo
On Thu, Sep 18, 2014 at 01:36:47PM +0800, Qu Wenruo wrote: > > Original Message > Subject: Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() > to insert best fitted extent map > From: Liu Bo > To: Qu Wenruo > Date: 2014年09月18日 12:21 > >On Wed, Sep 17, 2014 at 11:53:35AM

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map From: Liu Bo To: Qu Wenruo Date: 2014年09月18日 15:33 On Thu, Sep 18, 2014 at 01:36:47PM +0800, Qu Wenruo wrote: Original Message Subject: Re

Re: fs corruption report

2014-09-18 Thread Marc Dietrich
Am Donnerstag, 18. September 2014, 11:39:21 schrieb Gui Hecheng: > On Mon, 2014-09-01 at 15:25 +, Zooko Wilcox-OHearn wrote: > > I'm more than happy to try out patches and even focus my own brain on > > diagnosing it, if I can. I'm hoping to regain access to some of my > > files on my btrfs par

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Liu Bo
On Thu, Sep 18, 2014 at 03:58:26PM +0800, Qu Wenruo wrote: > [...] > > > >original: (start >= existing->start && start < extent_map_end(existing)) > >this patch: (start < extent_map_end(existing) && start + len > > >existing->start) > > > >(start + len > existing->start) doesn't equal to start

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map From: Liu Bo To: Qu Wenruo Date: 2014年09月18日 16:20 On Thu, Sep 18, 2014 at 03:58:26PM +0800, Qu Wenruo wrote: [...] original: (start >= existing->start &&

Re: [PATCH] btrfs-progs: fix page align issue for lzo compress in restore

2014-09-18 Thread Marc Dietrich
Hello Gui, Am Donnerstag, 18. September 2014, 11:34:43 schrieb Gui Hecheng: > When runing restore under lzo compression, "bad compress length" > problems are encountered. > It is because there is a page align problem with the @decompress_lzo, > as follows: > |--| ||-| |--

[PATCH 1/5] btrfs: nodatasum drop compress

2014-09-18 Thread Satoru Takeuchi
From: Naohiro Aota We can enable both compress option and nodatasum option at the same time with the following command. === $ sudo mount -o remount,compress,nodatasum / $ mount | grep btrfs /dev/vda2 on / type btrfs (rw,relatime,seclabel,nodatasum,compress=zlib,space_cache) === This behavior c

[PATCH 2/5] btrfs: correct a message on setting nodatacow

2014-09-18 Thread Satoru Takeuchi
From: Naohiro Aota If we set nodatacow mount option after compress-force option, we don't get compression disabling message. === $ sudo mount -o remount,compress-force,nodatacow /; dmesg|tail -n 3 [ 3845.719047] BTRFS info (device vda2): force zlib compression [ 3845.719052] BTRFS info (device v

[PATCH 3/5] btrfs: notice nodatasum is also enabled with nodatacow

2014-09-18 Thread Satoru Takeuchi
From: Naohiro Aota Documentation/filesystems/btrfs.txt states the following: === ... Nodatacow implies nodatasum, and disables all compression. ... === However the current message only mentions nodatacow. It's better to also mention nodatasum. Signed-off-by: Naohiro Aota Signed-off-by: Sato

[PATCH 4/5] btrfs: suppress a verbose message about enabling space cache on remounting

2014-09-18 Thread Satoru Takeuchi
From: Naohiro Aota Remount operation always tells you it enabled space cache. I consider it's a bit verbose and is better not to show this message on remounting. Signed-off-by: Naohiro Aota Signed-off-by: Satoru Takeuchi --- fs/btrfs/super.c | 5 ++--- 1 file changed, 2 insertions(+), 3 delet

[PATCH 5/5] btrfs: rework compression related options processing

2014-09-18 Thread Satoru Takeuchi
From: Naohiro Aota There are mutual exclusive mount options corresponding to compression options. If we set these options at once, we get improper messages and states, and a verbose message. * Improper states - We can't disable "compress forcing" with compress={zlib,lzo} mount option.

[PATCH v2] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Qu Wenruo
The following commit enhanced the merge_extent_mapping() to reduce fragment in extent map tree, but it can't handle case which existing lies before map_start: 51f39 btrfs: Use right extent length when inserting overlap extent map. [BUG] When existing extent map's start is before map_start, the em-

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Liu Bo
On Thu, Sep 18, 2014 at 04:24:48PM +0800, Qu Wenruo wrote: > > Original Message > Subject: Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() > to insert best fitted extent map > From: Liu Bo > To: Qu Wenruo > Date: 2014年09月18日 16:20 > >On Thu, Sep 18, 2014 at 03:58:26PM

[PATCH] Btrfs-progs: super-recover: fix double free fs_devices memory

2014-09-18 Thread Wang Shilong
super-recover collects btrfs devices infomation using existed functions scan_one_devices(). Problem is fs_devices is freed twice in close_ctree() and free_recover_superblock() for super correction path. Fix this problem by checking whether fs_devices memory have been freed before we free it. Cc:

Re: [PATCH] btrfs-progs: fix page align issue for lzo compress in restore

2014-09-18 Thread Gui Hecheng
On Thu, 2014-09-18 at 10:25 +0200, Marc Dietrich wrote: > Hello Gui, > > Am Donnerstag, 18. September 2014, 11:34:43 schrieb Gui Hecheng: > > When runing restore under lzo compression, "bad compress length" > > problems are encountered. > > It is because there is a page align problem with the @dec

Re: [PATCH] Btrfs-progs: super-recover: fix double free fs_devices memory

2014-09-18 Thread Gui Hecheng
On Thu, 2014-09-18 at 05:01 -0400, Wang Shilong wrote: > super-recover collects btrfs devices infomation using existed > functions scan_one_devices(). > > Problem is fs_devices is freed twice in close_ctree() and > free_recover_superblock() for super correction path. > > Fix this problem by check

Re: [PATCH] btrfs-progs: fix page align issue for lzo compress inrestore

2014-09-18 Thread Marc Dietrich
Am Donnerstag, 18. September 2014, 17:10:54 schrieb Gui Hecheng: > On Thu, 2014-09-18 at 10:25 +0200, Marc Dietrich wrote: > > Hello Gui, > > > > Am Donnerstag, 18. September 2014, 11:34:43 schrieb Gui Hecheng: > > > When runing restore under lzo compression, "bad compress length" > > > problems a

[PATCH] Move BTRFS RCU string to common library

2014-09-18 Thread Omar Sandoval
The RCU-friendy string API used internally by BTRFS is generic enough for common use. This doesn't add any new functionality, but instead just moves the code and documents the existing API. Signed-off-by: Omar Sandoval --- This patch applies to 3.17-rc5. fs/btrfs/check-integrity.c | 6 ++-- fs

Re: [PATCH] btrfs-progs: fix page align issue for lzo compress inrestore

2014-09-18 Thread Gui Hecheng
On Thu, 2014-09-18 at 11:25 +0200, Marc Dietrich wrote: > Am Donnerstag, 18. September 2014, 17:10:54 schrieb Gui Hecheng: > > On Thu, 2014-09-18 at 10:25 +0200, Marc Dietrich wrote: > > > Hello Gui, > > > > > > Am Donnerstag, 18. September 2014, 11:34:43 schrieb Gui Hecheng: > > > > When runing r

Re: fs corruption report

2014-09-18 Thread Zooko Wilcox-OHearn
Thank you! I will try to restore using this patch. What branch of what btrfs tools git repo should I apply the patch to? Regards, Zooko -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://v

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Filipe David Manana
On Wed, Sep 17, 2014 at 4:53 AM, Qu Wenruo wrote: > The following commit enhanced the merge_extent_mapping() to reduce > fragment in extent map tree, but it can't handle case which existing > lies before map_start: > 51f39 btrfs: Use right extent length when inserting overlap extent map. > > [BUG]

Re: Problem with a filesystem

2014-09-18 Thread Tovo Rabemanantsoa
On 09/17/2014 08:49 PM, Chris Murphy wrote: > > This isn't what I expect as a candidate for btrfs-zero-log, I don't know what > the last message means. You could run a > btrfs check > and then ask about both the failure to read block groups -5 and also the > btrfs check (without --repair) resu

corruption with btrfs on nocow loop file

2014-09-18 Thread Jan Killius
Hello, I get corruption with btrfs in a loop device with linux-3.16.3 (https://git.kernel.org/cgit/linux/kernel/git/mason/linux-btrfs.git/ for-linus applied). I never had a problem with this setup until 3.16. This happened just after unpacking a few thousand small files and reading them back with

Re: [PATCH] Move BTRFS RCU string to common library

2014-09-18 Thread Mathieu Desnoyers
- Original Message - > From: "Omar Sandoval" > To: "Chris Mason" , "Josef Bacik" , > linux-btrfs@vger.kernel.org, "Paul E. McKenney" > , "Josh Triplett" , > "Steven Rostedt" , > "Mathieu Desnoyers" , "Lai Jiangshan" > , > linux-ker...@vger.kernel.org > Sent: Thursday, September 18, 2014

kernel integration branch updated

2014-09-18 Thread Chris Mason
Hi everyone, I've added a few more patches to the kernel integration branch, and rebased onto rc5. This should be my last rebase before sending into linux-next, please take a look. It's still missing three patches from Josef, which we're updating. I can put more patches on top, but I'd prefer n

[PATCH] Revert "Btrfs: device_list_add() should not update list when

2014-09-18 Thread Chris Mason
Johannes and Sam, could you please confirm this patch fixes your mount regression for now? Anand, please make sure I kept the generation check properly. This reverts commit b96de000bc8bc9688b3a2abea4332bd57648a49f. This commit is triggering failures to mount by subvolume id in some configuratio

[PATCH] Btrfs: remove empty block groups automatically V3

2014-09-18 Thread Josef Bacik
One problem that has plagued us is that a user will use up all of his space with data, remove a bunch of that data, and then try to create a bunch of small files and run out of space. This happens because all the chunks were allocated for data since the metadata requirements were so low. But now

[PATCH] Btrfs: don't do async reclaim during log replay V2

2014-09-18 Thread Josef Bacik
Trying to reproduce a log enospc bug I hit a panic in the async reclaim code during log replay. This is because we use fs_info->fs_root as our root for shrinking and such. Technically we can use whatever root we want, but let's just not allow async reclaim while we're doing log replay. Thanks,

btrfs receive: could not find parent subvolume

2014-09-18 Thread Marc MERLIN
While debugging a btrfs send/receive slow problem, I now getting this: legolas:/mnt/btrfs_pool1# btrfs send -p tmp_ggm_daily_ro.20140917_06:29:58 tmp_ggm_daily_ro.20140918_02:48:24 | ssh gargamel btrfs receive -v /mnt/btrfs_pool2/backup/debian64/legolas At subvol tmp_ggm_daily_ro.20140918_02:48:2

[PATCH] Btrfs: try not to ENOSPC on log replay V2

2014-09-18 Thread Josef Bacik
When doing log replay we may have to update inodes, which traditionally goes through our delayed inode stuff. This will try to move space over from the trans handle, but we don't reserve space in our trans handle on replay since we don't know how much we will need, so instead we try to flush. But

Re: btrfs receive: could not find parent subvolume

2014-09-18 Thread Hugo Mills
On Thu, Sep 18, 2014 at 08:27:18AM -0700, Marc MERLIN wrote: > While debugging a btrfs send/receive slow problem, I now getting this: > legolas:/mnt/btrfs_pool1# btrfs send -p tmp_ggm_daily_ro.20140917_06:29:58 > tmp_ggm_daily_ro.20140918_02:48:24 | ssh gargamel btrfs receive -v > /mnt/btrfs_pool

Re: btrfs receive: could not find parent subvolume

2014-09-18 Thread Marc MERLIN
On Thu, Sep 18, 2014 at 04:39:34PM +0100, Hugo Mills wrote: > > Now it seems that the UUID is different on all my snapshots created by > > btrfs send, so maybe it doesn't match UUID? > > > > Given that, what is btrfs receive using to get a match? > >There's a "recieved UUID" field on each sub

Re: Problem with unmountable filesystem.

2014-09-18 Thread Austin S Hemmelgarn
On 09/17/2014 02:57 PM, Chris Murphy wrote: > > On Sep 17, 2014, at 5:23 AM, Austin S Hemmelgarn wrote: >> >> Thanks for all the help. > > Well, it's not much help. It seems possible to "corrupt" a primary superblock > that points to a corrupt tree root, and use btrfs rescure super-recover to

Re: Problem with unmountable filesystem.

2014-09-18 Thread Austin S Hemmelgarn
On 09/17/2014 04:22 PM, Duncan wrote: > Austin S Hemmelgarn posted on Wed, 17 Sep 2014 07:23:46 -0400 as > excerpted: > >> I've also discovered, when trying to use btrfs restore to copy out the >> data to a different system, that 3.14.1 restore apparently chokes on >> filesystem that have lzo comp

XFS Tests for Btrfs

2014-09-18 Thread nick
Hey Fellow Btrfs Developers, I am wondering how to run the xfs tests for btrfs as I tried to do it based on a link online written I believe a few years ago. If someone can help me get set up for testing the btrfs code using xfs tests that would be great. In addition afterwards I already build ke

Re: Problem with unmountable filesystem.

2014-09-18 Thread Chris Murphy
On Sep 18, 2014, at 11:12 AM, Austin S Hemmelgarn wrote: > On 09/17/2014 02:57 PM, Chris Murphy wrote: >> >> On Sep 17, 2014, at 5:23 AM, Austin S Hemmelgarn >> wrote: >>> >>> Thanks for all the help. >> >> Well, it's not much help. It seems possible to "corrupt" a primary >> superblock th

Re: Problem with unmountable filesystem.

2014-09-18 Thread Duncan
Austin S Hemmelgarn posted on Thu, 18 Sep 2014 13:12:03 -0400 as excerpted: > Secondarily, this almost makes me want to set the ssd option on all > BTRFS filesystems, just to get the rotating superblock updates, because > if it weren't for that behavior, I probably wouldn't have been able to > rec

Re: Btrfs-progs releases

2014-09-18 Thread WorMzy Tykashi
On 26 August 2014 15:22, Chris Mason wrote: > Hi everyone, > > I've just tagged btrfs-progs v3.16. Like the last few releases, this > one was prepared by Dave Sterba, who has been wrangling btrfs-progs > patches for some time now. > > Dave has asked to officially take over the btrfs-progs mainten

Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map

2014-09-18 Thread Qu Wenruo
Original Message Subject: Re: [PATCH] btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map From: Filipe David Manana To: Qu Wenruo Date: 2014年09月18日 21:16 On Wed, Sep 17, 2014 at 4:53 AM, Qu Wenruo wrote: The following commit enhanced the merge_e

Re: XFS Tests for Btrfs

2014-09-18 Thread Qu Wenruo
Hi, Xfstests uses several environment variants to setup the test environment. You can set it manually using export, or write them into local.conf in xfstests directory. These environment variants are mandatory: TEST_DEV: device for normal tests, like all generic tests TEST_DIR: where TEST_DEV

Re: XFS Tests for Btrfs

2014-09-18 Thread nick
On 14-09-18 09:08 PM, Qu Wenruo wrote: > Hi, > > Xfstests uses several environment variants to setup the test environment. > You can set it manually using export, or write them into local.conf in > xfstests directory. > > These environment variants are mandatory: > TEST_DEV: device for normal

Re: fs corruption report

2014-09-18 Thread Gui Hecheng
On Thu, 2014-09-18 at 12:47 +, Zooko Wilcox-OHearn wrote: > Thank you! I will try to restore using this patch. > > What branch of what btrfs tools git repo should I apply the patch to? > > Regards, > > Zooko At least I think the following repo/v3.17.x branch has all restore-related patches

Re: XFS Tests for Btrfs

2014-09-18 Thread Qu Wenruo
Original Message Subject: Re: XFS Tests for Btrfs From: nick To: Qu Wenruo , Date: 2014年09月19日 09:17 On 14-09-18 09:08 PM, Qu Wenruo wrote: Hi, Xfstests uses several environment variants to setup the test environment. You can set it manually using export, or write them int

Re: XFS Tests for Btrfs

2014-09-18 Thread nick
On 14-09-18 09:40 PM, Qu Wenruo wrote: > > Original Message > Subject: Re: XFS Tests for Btrfs > From: nick > To: Qu Wenruo , > Date: 2014年09月19日 09:17 >> >> On 14-09-18 09:08 PM, Qu Wenruo wrote: >>> Hi, >>> >>> Xfstests uses several environment variants to setup the test en

Re: kernel integration branch updated

2014-09-18 Thread Qu Wenruo
Hi Chris, I'm sorry that the commit 'btrfs: Fix and enhance merge_extent_mapping() to insert best fitted extent map' has a V2 patch, so the one in tree is not up-to-data. Although the v2 change is quite small and it's relevantly dependent, so it should not be a pain change. Thanks, Qu ---

Re: XFS Tests for Btrfs

2014-09-18 Thread Qu Wenruo
Original Message Subject: Re: XFS Tests for Btrfs From: nick To: Qu Wenruo , Date: 2014年09月19日 09:41 On 14-09-18 09:40 PM, Qu Wenruo wrote: Original Message Subject: Re: XFS Tests for Btrfs From: nick To: Qu Wenruo , Date: 2014年09月19日 09:17 On 14-09-18

[PATCH] btrfs-progs: fix many typos in documents

2014-09-18 Thread Satoru Takeuchi
From: Naohiro Aota There are many trivial typos in Documentation/*.txt. All of these use "exist status" to mean "exit status" by mistake. I guess someone first made this mistake and it has spread by copy-and-paste :-D Signed-off-by: Naohiro Aota Signed-off-by: Satoru Takeuchi --- Documentatio

Re: XFS Tests for Btrfs

2014-09-18 Thread nick
On 14-09-18 09:46 PM, Qu Wenruo wrote: > > Original Message > Subject: Re: XFS Tests for Btrfs > From: nick > To: Qu Wenruo , > Date: 2014年09月19日 09:41 >> >> On 14-09-18 09:40 PM, Qu Wenruo wrote: >>> Original Message >>> Subject: Re: XFS Tests for Btrfs >>>

Re: [PATCH 2/5] btrfs: correct a message on setting nodatacow

2014-09-18 Thread Qu Wenruo
Original Message Subject: [PATCH 2/5] btrfs: correct a message on setting nodatacow From: Satoru Takeuchi To: linux-btrfs@vger.kernel.org Date: 2014年09月18日 16:28 From: Naohiro Aota If we set nodatacow mount option after compress-force option, we don't get compression disabl

Re: kernel integration branch updated

2014-09-18 Thread Miao Xie
Chris On Fri, 19 Sep 2014 09:45:17 +0800, Qu Wenruo wrote: > Hi Chris, > > > I'm sorry that the commit 'btrfs: Fix and enhance merge_extent_mapping() to > insert best fitted extent map' > has a V2 patch, so the one in tree is not up-to-data. > > Although the v2 change is quite small and it's r

Re: [PATCH] xfstests: remove check_scratch_fs in btrfs/012

2014-09-18 Thread Eryu Guan
On Wed, Sep 03, 2014 at 11:25:59AM +0800, Liu Bo wrote: > From: Liu Bo > > btrfs/012 is a case to verify btrfs-convert feature, it converts an ext4 to > btrfs firstly and do something, then rolls back to ext4. > > So at last we have a ext4 on the scratch device, but setting _require_scratch > wi

Re: [PATCH 2/5] btrfs: correct a message on setting nodatacow

2014-09-18 Thread Satoru Takeuchi
Hi Qu, Thank you for your comment. (2014/09/19 11:03), Qu Wenruo wrote: Original Message Subject: [PATCH 2/5] btrfs: correct a message on setting nodatacow From: Satoru Takeuchi To: linux-btrfs@vger.kernel.org Date: 2014年09月18日 16:28 From: Naohiro Aota If we set nodataco