Re: WARNING at fs/btrfs/inode.c:9261 btrfs_destroy_inode()

On Tue, May 10, 2016 at 06:19:20PM -0500, Eric Biggers wrote: > Hello, > > The following warning has been triggering for me since about v4.6-rc3: > > WARN_ON(BTRFS_I(inode)->csum_bytes); > > On one machine the warning has occurred 657 times since v4.6-rc5. On another > it > has occurred

Re: [PATCH] fstests: generic: Test SHARED flag about fiemap ioctl before and after sync

On 05/13/2016 01:17 PM, Darrick J. Wong wrote: On Fri, May 13, 2016 at 09:52:52AM +0800, Qu Wenruo wrote: The test case will check SHARED flag returned by fiemap ioctl on reflinked files before and after sync. Normally SHARED flag won't change just due to a normal sync operation. But btrfs

[PATCH 1/7] Btrfs: replace BUG() with WARN_ONCE in raid56

This BUG() has been triggered by a fuzz testing image, but in fact btrfs can handle this gracefully by returning -EIO. Thus, use WARN_ONCE for warning purpose and don't leave a possible kernel panic. Signed-off-by: Liu Bo --- fs/btrfs/raid56.c | 2 +- 1 file changed, 1

[PATCH 3/7] Btrfs: check if extent buffer is aligned to sectorsize

Thanks to fuzz testing, we can pass an invalid bytenr to extent buffer via alloc_extent_buffer(). An unaligned eb can have more pages than it should have, which ends up extent buffer's leak or some corrupted content in extent buffer. This adds a warning to let us quickly know what was happening.

[PATCH 2/7] Btrfs: replace BUG_ON with WARN_ONCE in cow_file_range

This BUG_ON is more like a warning since an invalid btrfs_super_total_bytes() doesn't affect other stuff. Thus, use WARN_ONCE for warning purpose and don't leave a possible kernel panic here. Signed-off-by: Liu Bo --- fs/btrfs/inode.c | 7 +-- 1 file changed, 5

[PATCH 6/7] Btrfs: fix eb memory leak due to readpage failure

eb->io_pages is set in read_extent_buffer_pages(). In case of readpage failure, for pages that have been added to bio, it calls bio_endio and later readpage_io_failed_hook() does the work. When this eb's page (couldn't be the 1st page) fails to add itself to bio due to failure in merge_bio(), it

[PATCH 7/7] Btrfs: fix memory leak due to invalid btree height

Thanks to fuzz testing, we can have invalid btree root node height. Btrfs limits btree height to 7 and if the given height is 9, then btrfs will have problems in both releasing root node's lock and freeing the node. Signed-off-by: Liu Bo --- fs/btrfs/ctree.c | 7 +++ 1

[PATCH 4/7] Btrfs: free sys_array eb as soon as possible

While reading sys_chunk_array in superblock, btrfs creates a temporary extent buffer. Since we don't use it after finishing reading sys_chunk_array, we don't need to keep it in memory. Signed-off-by: Liu Bo --- fs/btrfs/volumes.c | 4 ++-- 1 file changed, 2

[PATCH 5/7] Btrfs: replace BUG_ON with WARN in merge_bio

We have two BUG_ON in merge_bio, but since it can gracefully return errors to callers, use WARN_ONCE to give error information and don't leave a possible panic. Signed-off-by: Liu Bo --- fs/btrfs/extent_io.c | 1 - fs/btrfs/inode.c | 6 -- 2 files changed, 4

Re: [PATCH 2/2] Btrfs: add valid checks for chunk loading

On Wed, May 04, 2016 at 03:56:26PM +0200, David Sterba wrote: > A few minor comments below > > On Mon, May 02, 2016 at 11:15:51AM -0700, Liu Bo wrote: > > --- a/fs/btrfs/volumes.c > > +++ b/fs/btrfs/volumes.c > > @@ -6206,27 +6206,23 @@ struct btrfs_device *btrfs_alloc_device(struct > >

Re: [PATCH 1/2] Btrfs: add more valid checks for superblock

On 05/14/2016 02:14 AM, Liu Bo wrote: On Mon, May 09, 2016 at 09:31:37AM +0800, Qu Wenruo wrote: David Sterba wrote on 2016/05/06 16:35 +0200: On Thu, May 05, 2016 at 09:08:54AM +0800, Qu Wenruo wrote: An early check can compare against some reasonable value, but the total_bytes value

Re: btrfs ate my data in just two days, after a fresh install. ram and disk are ok. it still mounts, but I cannot repair

On Fri, May 13, 2016 at 6:10 AM, Niccolò Belli wrote: > On venerdì 13 maggio 2016 13:35:01 CEST, Austin S. Hemmelgarn wrote: >> >> The fact that you're getting an OOPS involving core kernel threads >> (kswapd) is a pretty good indication that either there's a bug

Re: fsck: to repair or not to repair

On Fri, May 13, 2016 at 9:28 AM, Nikolaus Rath wrote: > On May 13 2016, Duncan <1i5t5.dun...@cox.net> wrote: >> Because btrfs can be multi-device, it needs some way to track which >> devices belong to each filesystem, and it uses filesystem UUID for this >> purpose. >> >> If

Re: [PATCH 1/2] Btrfs: add more valid checks for superblock

On Mon, May 09, 2016 at 09:31:37AM +0800, Qu Wenruo wrote: > > > David Sterba wrote on 2016/05/06 16:35 +0200: > > On Thu, May 05, 2016 at 09:08:54AM +0800, Qu Wenruo wrote: > > > > > An early check can compare against some reasonable value, but the > > > > > total_bytes value must be equal to

Re: BTRFS Data at Rest File Corruption

On Thu, May 12, 2016 at 10:49 PM, Richard A. Lochner wrote: > My apologies, they were from different boots. After the dd, I get > these: > > [109479.550836] BTRFS warning (device sdb1): csum failed ino 1437377 > off 75754369024 csum 1689728329 expected csum 2165338402 >

Re: BTRFS Data at Rest File Corruption

On 2016-05-13 12:28, Goffredo Baroncelli wrote: On 2016-05-11 21:26, Austin S. Hemmelgarn wrote: (although it can't tell the difference between a corrupted checksum and a corrupted block of data). I don't think so. The data checksums are stored in metadata blocks, and as metadata block,

Re: About in-band dedupe for v4.7

On Fri, May 13, 2016 at 08:14:10AM -0400, Austin S. Hemmelgarn wrote: > On 2016-05-12 16:54, Mark Fasheh wrote: > >Now ask yourself the question - would you accept a write cache which is > >expensive to fill and would only have a hit rate of less than 5%? > In-band deduplication is a feature that

[GIT PULL] Btrfs fixes for 4.7

From: Filipe Manana Hi Chris, Please consider the following changes for the merge window for 4.7. There's an implementation for the rename exchange and rename whiteout operations, from Dan Fuhry, which was originally in David's integration branches and linux-next but then we

Re: BTRFS Data at Rest File Corruption

On 2016-05-11 21:26, Austin S. Hemmelgarn wrote: > (although it can't tell the difference between a corrupted checksum and a > corrupted block of data). I don't think so. The data checksums are stored in metadata blocks, and as metadata block, these have their checksums. So btrfs know if the

Re: fsck: to repair or not to repair

On May 13 2016, Duncan <1i5t5.dun...@cox.net> wrote: > Because btrfs can be multi-device, it needs some way to track which > devices belong to each filesystem, and it uses filesystem UUID for this > purpose. > > If you clone a filesystem (for instance using dd or lvm snapshotting, > doesn't

Re: About in-band dedupe for v4.7

On 05/13/2016 08:14 PM, Austin S. Hemmelgarn wrote: On 2016-05-12 16:54, Mark Fasheh wrote: On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: Taking your history with qgroups out of this btw, my opinion does not

Re: About in-band dedupe for v4.7

On 2016-05-12 16:54, Mark Fasheh wrote: On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: On Tue, May 10, 2016 at 07:52:11PM -0700, Mark Fasheh wrote: Taking your history with qgroups out of this btw, my opinion does not change. With respect to in-memory only dedupe, it is my

Re: btrfs ate my data in just two days, after a fresh install. ram and disk are ok. it still mounts, but I cannot repair

On venerdì 13 maggio 2016 13:35:01 CEST, Austin S. Hemmelgarn wrote: The fact that you're getting an OOPS involving core kernel threads (kswapd) is a pretty good indication that either there's a bug elsewhere in the kernel, or that something is wrong with your hardware. it's really difficult

[PATCH 1/3] [RFC] fstests: btrfs: add functions to set and reset required number of SCRATCH_DEV_POOL

This patch provides functions _scratch_dev_pool_get() _scratch_dev_pool_put() Which will help to set/reset SCRATCH_DEV_POOL with the required number of devices. SCRATCH_DEV_POOL_SAVED will hold all the devices. Usage: _scratch_dev_pool_get() :: do stuff _scratch_dev_pool_put()

[PATCH 3/3] fstests: btrfs: 027 make use of new device get and put helper functions

Below patches added helper function to get the requested number of devices for scratch and spare device fstest: btrfs: add functions to get and put a device for replace target fstests: btrfs: add functions to set and reset required number of SCRATCH_DEV_POOL This patch makes use of them.

[PATCH 2/3] [RFC] fstests: btrfs: add functions to get and put a device for replace target

For the replace tests we need a device as a spare device, here functions _spare_dev_get() and _spare_dev_put() will get it from the SCRATCH_DEV_POOL_SAVED, which is set when _scratch_dev_pool_get() is called, and is based on how many has already been assigned to SCRATCH_DEV_POOL. usage:

Re: btrfs ate my data in just two days, after a fresh install. ram and disk are ok. it still mounts, but I cannot repair

On 2016-05-13 07:07, Niccolò Belli wrote: On giovedì 12 maggio 2016 17:43:38 CEST, Austin S. Hemmelgarn wrote: That's probably a good indication of the CPU and the MB being OK, but not necessarily the RAM. There's two other possible options for testing the RAM that haven't been mentioned yet

Re: btrfs ate my data in just two days, after a fresh install. ram and disk are ok. it still mounts, but I cannot repair

On giovedì 12 maggio 2016 17:43:38 CEST, Austin S. Hemmelgarn wrote: That's probably a good indication of the CPU and the MB being OK, but not necessarily the RAM. There's two other possible options for testing the RAM that haven't been mentioned yet though (which I hadn't thought of myself

RE: [PATCH] btrfs: fix memory leak during RAID 5/6 device replacement

Hi, Scott Talbert * From: David Sterba [mailto:dste...@suse.cz] > Sent: Tuesday, May 10, 2016 1:32 AM > To: Scott Talbert > Cc: linux-btrfs@vger.kernel.org; Zhao Lei > Subject: Re: [PATCH] btrfs: fix memory leak during RAID 5/6 device replacement

Re: [PATCH] btrfs-progs: added quiet-option for scripts

Hi, On Thu, May 12, 2016 at 09:49:25PM +0200, bt...@oss.m-berberich.de wrote: > From: M G Berberich > > -q,--quiet to prevent status-messages on stderr > --verbose as alternative for -v > moved 'Mode NO_FILE_DATA enabled' message to stderr > changed default for

Re: [PATCH 0/1] btrfs-progs: Typo review of strings and comments

On Thu, May 12, 2016 at 04:56:46PM -0400, Nicholas D Steeves wrote: > Thank you for reviewing this patch. Sorry for missing the context of > the strtoull comment; I should have been able to infer that and am > embarrassed that I failed to. Also, embarrassed because I think I've > used it in some

Re: Input/output error on newly created file

Am Friday 13 May 2016 schrieb Duncan <1i5t5.dun...@cox.net> >Szalma László posted on Thu, 12 May 2016 20:28:24 +0200 as excerpted: >> The files that rarely become unreadable (I/O error but no error in dmesg >> or anywhere) are mysql MyIsam database files, and they are always small. >> Like

Re: Undelete deleted subvolume?

Andrei Borzenkov posted on Thu, 12 May 2016 17:19:11 +0300 as excerpted: > I accidentally deleted wrong snapshot using SUSE snapper. Is it possible > to undelete subvolume? I know that it is possible to extract files from > old tree (although SLES12 does not seem to offer btrfs-find-root), but >

Re: Input/output error on newly created file

Szalma László posted on Thu, 12 May 2016 20:28:24 +0200 as excerpted: > The files that rarely become unreadable (I/O error but no error in dmesg > or anywhere) are mysql MyIsam database files, and they are always small. > Like 16kbyte for example, or smaller. Sometimes dropping the fs cache >

Re: About in-band dedupe for v4.7

Mark Fasheh posted on Thu, 12 May 2016 13:54:26 -0700 as excerpted: > For example, my 'large' duperemove test involves about 750 gigabytes of > general purpose data - quite literally /home off my workstation. > > After the run I'm usually seeing between 65-75 gigabytes saved for a > total of

Re: fsck: to repair or not to repair

Henk Slager posted on Thu, 12 May 2016 19:02:56 +0200 as excerpted: > Maybe you first want to test it on an overlay > of the device or copy the whole fs with dd. WARNING! Because btrfs can be multi-device, it needs some way to track which devices belong to each filesystem, and it uses

Re: About in-band dedupe for v4.7

On Fri, May 13, 2016 at 11:44:40AM +0800, Qu Wenruo wrote: > Although maybe out of your expectation, inband de-dedupe did exposed some > existing bugs we didn't ever found before. > And they are all reproducible without inband dedupe. > > Some examples: > [...] > 3) Slow backref walk. >

Re: Amount of scrubbed data goes from 15.90GiB to 26.66GiB after defragment -r -v -clzo on a fs always mounted with compress=lzo

Niccolò Belli posted on Thu, 12 May 2016 15:56:20 +0200 as excerpted: > Thanks for the detailed explanation, hopefully in the future someone > will be able to make defrag snapshot/reflink aware in a scalable manner. It's still planned, AFAIK, but one of the scaling issues in particular, quotas,

Re: About in-band dedupe for v4.7

On Wed, May 11, 2016 at 07:36:59PM +0200, David Sterba wrote: > I like the in-memory dedup backend. It's lightweight, only a heuristic, > does not need any IO or persistent storage. OTOH I consider it a subpart > of the in-band deduplication that does all the persistency etc. So I > treat the