[PATCH 1/6] btrfs-progs: check: enable repair in lowmem mode

2017-08-22 Thread Lu Fengqi
From: Su Yue Turn on the option --repair with --mode==lowmem in btrfsck. Signed-off-by: Su Yue --- cmds-check.c | 8 +++- 1 file changed, 3 insertions(+), 5 deletions(-) diff --git a/cmds-check.c b/cmds-check.c index c5faa2b..829f7c5

[PATCH 4/6] btrfs-progs: check: Introduce repair_chunk_item()

2017-08-22 Thread Lu Fengqi
From: Su Yue Because this patchset concentrates on repair of extent tree, repair_chunk_item() now only inserts missed chunk group item into extent tree. There are some things left TODO, for example dev_item. Signed-off-by: Su Yue ---

[PATCH 3/6] btrfs-progs: check: delete wrong items in lowmem repair

2017-08-22 Thread Lu Fengqi
From: Su Yue Introduce delete_extent_tree_item() and repair_extent_item() to do delete only. while checking a extent tree, just delete wrong item. For extent item, free wrong backref. Otherwise, do delete. So the rest items in extent tree should be correct.

[PATCH 5/6] [btrfs-progs: check: Introduce repair_tree_block_ref()

2017-08-22 Thread Lu Fengqi
From: Su Yue The only thing repair_tree_block_ref() does is that adding backref of the tree_block. Just like what origin repair do: It first searches the correspond extent item then 1. If the extent item exists but backref is missing, add one backref to the extent.

[PATCH 0/6] btrfs-progs: check: extent tree lowmem repair

2017-08-22 Thread Lu Fengqi
From: Su Yue This is part 2 of lowmem repair patchsets: 1. Change the way of traversal under lowmem check to use walk_up_tree_v2() and walk_down_tree_v2() and it scans all trees now. 2. Repair cases: block group missing, tree block backref missing, extent item

[PATCH 6/6] btrfs-progs: check: Introduce repair_extent_data_item()

2017-08-22 Thread Lu Fengqi
From: Su Yue The only thing repair_extent_data_item() does is that adding backref of the tree_block. Just like what origin repair do: It first searches the correspond extent item then 1. If the extent item exists but backref is missing, add one backref to the extent.

[PATCH 2/6] btrfs-progs: check: change traversal way of lowmem mode

2017-08-22 Thread Lu Fengqi
From: Su Yue This patch is a preparation for extent-tree repair in lowmem mode. In the lowmem mode, checking tree blocks of various tree is in recursive way. But if during repair, add or delete of item(s) may modify upper nodes which will cause the repair to be

RE: [PATCH] btrfs-progs: mkfs: Replace number with enum

2017-08-22 Thread Gu, Jinxiang
> -Original Message- > From: David Sterba [mailto:dste...@suse.cz] > Sent: Tuesday, August 22, 2017 10:04 PM > To: Gu, Jinxiang/顾 金香 ; linux-btrfs@vger.kernel.org > Subject: Re: [PATCH] btrfs-progs: mkfs: Replace number with enum > > On Mon, Aug 21, 2017 at

[PATCH v5 1/6] Btrfs: heuristic make use compression workspaces

2017-08-22 Thread Timofey Titovets
Move heuristic to external file Implement compression workspaces support for heuristic resources Signed-off-by: Timofey Titovets --- fs/btrfs/Makefile | 2 +- fs/btrfs/compression.c | 18 + fs/btrfs/compression.h | 7 - fs/btrfs/heuristic.c | 70

[PATCH v5 2/6] Btrfs: heuristic workspace add bucket and sample items

2017-08-22 Thread Timofey Titovets
Heuristic workspace: - Add bucket for storing byte type counters - Add sample array for storing partial copy of input data range - Add counter for store current sample size to workspace Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 30

[PATCH v5 3/6] Btrfs: implement heuristic sampling logic

2017-08-22 Thread Timofey Titovets
Copy sample data from input data range to sample buffer then calculate byte type count for that sample into bucket. Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 31 +-- 1 file changed, 29 insertions(+), 2 deletions(-) diff --git

[PATCH v5 4/6] Btrfs: heuristic add detection of zeroed sample

2017-08-22 Thread Timofey Titovets
Use memcmp for check sample data to zeroes. Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 18 ++ 1 file changed, 18 insertions(+) diff --git a/fs/btrfs/heuristic.c b/fs/btrfs/heuristic.c index 5336638a3b7c..4557ea1db373 100644 ---

[PATCH v5 5/6] Btrfs: heuristic add byte set calculation

2017-08-22 Thread Timofey Titovets
Calculate byte set size for data sample: Calculate how many unique bytes has been in sample By count all bytes in bucket with count > 0 If byte set low (~25%), data are easily compressible Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 26

[PATCH v5 6/6] Btrfs: heuristic add byte core set calculation

2017-08-22 Thread Timofey Titovets
Calculate byte core set for data sample: Sort bucket's numbers in decreasing order Count how many numbers use 90% of sample If core set are low (<=25%), data are easily compressible If core set high (>=80%), data are not compressible Signed-off-by: Timofey Titovets ---

[PATCH v5 0/6] Btrfs: populate heuristic with code

2017-08-22 Thread Timofey Titovets
Based on kdave for-next Patches short: 1. Move heuristic to use compression workspaces Bit tricky, but works. 2. Add heuristic counters and buffer to workspaces 3. Implement simple input data sampling It's get 16 byte samples with 256 bytes shifts over input data. Collect info about

Re: Btrfs Raid5 issue.

2017-08-22 Thread Qu Wenruo
On 2017年08月23日 00:37, Robert LeBlanc wrote: Thanks for the explanations. Chris, I don't think 'degraded' did anything to help the mounting, I just passed it in to see if it would help (I'm not sure if btrfs is "smart" enough to ignore a drive if it would increase the chance of mounting the

user snapshots

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (19:36), Peter Grandi wrote: > For somewhat good reasons subvolumes including snapshots cannot be > deleted by users though unless mount option 'user_subvol_rm_allowed' is > used. Also in https://btrfs.wiki.kernel.org/index.php/Mount_options "user_subvol_rm_allowed (...) Use

Re: netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (19:36), Peter Grandi wrote: > Indeed and there is a fair description of some options for > subvolume nesting policies here which may be interesting to the > original poster: > > https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout > > It is unsurprising to me that

[josef-btrfs:slab-priority 4/6] fs//ntfs/attrib.c:2549:35: error: implicit declaration of function 'inode_to_bdi'

2017-08-22 Thread kbuild test robot
tree: https://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git slab-priority head: a1be3b41415243d20c90e9e92e82808fe1ff91a0 commit: fe049b0156a10dd0bb3fbf3d4dad3ca943874f10 [4/6] remove mapping from balance_dirty_pages*() config: i386-randconfig-a1-201734 (attached as .config)

[PATCH][v2] btrfs: change how we decide to commit transactions during flushing

2017-08-22 Thread josef
From: Josef Bacik Nikolay reported that generic/273 was failing currently with ENOSPC. Turns out this is because we get to the point where the outstanding reservations are greater than the pinned space on the fs. This is a mistake, previously we used the current reservation

Re: netapp-alike snapshots?

2017-08-22 Thread Peter Grandi
[ ... ] It is beneficial to not have snapshots in-place. With a local directory of snapshots, [ ... ] Indeed and there is a fair description of some options for subvolume nesting policies here which may be interesting to the original poster:

Re: [PATCH 0/5 v2] btrfs-progs: convert fixes + reiserfs support

2017-08-22 Thread David Sterba
On Thu, Jul 27, 2017 at 11:47:18AM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > Changes since v1: > - reiserfs conversion: > - use bool instead of int > - catch 'impossible' condition of multiple discontiguous tails > - properly handle hole followed by tail > -

Re: netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (22:36), Roman Mamedov wrote: > > My users want the snapshots locally in a .snapshot subdirectory. > > Because Netapp do it this way - for at least 20 years and we have a > > multi-PB Netapp storage environment. > > Just a side note, you do know that only subvolumes can be

Re: netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (19:19), A L wrote: > Perhaps using a bind mount? It would look and work the same as a ordinary fs. > Just need to make sure du uses one filesystem. > > From: Ulli Horlacher -- Sent: 2017-08-22 > - 18:57 > > > On Tue 2017-08-22

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Peter Grandi
[ ... ] >> There is no fixed relationship between the root directory >> inode of a subvolume and the root directory inode of any >> other subvolume or the main volume. > Actually, there is, because it's inherently rooted in the > hierarchy of the volume itself. That root inode for the >

Re: netapp-alike snapshots?

2017-08-22 Thread Roman Mamedov
On Tue, 22 Aug 2017 18:57:25 +0200 Ulli Horlacher wrote: > On Tue 2017-08-22 (21:45), Roman Mamedov wrote: > > > It is beneficial to not have snapshots in-place. With a local directory of > > snapshots, issuing things like "find", "grep -r" or even "du" will take

Re: netapp-alike snapshots?

2017-08-22 Thread A L
Perhaps using a bind mount? It would look and work the same as a ordinary fs. Just need to make sure du uses one filesystem. From: Ulli Horlacher -- Sent: 2017-08-22 - 18:57 > On Tue 2017-08-22 (21:45), Roman Mamedov wrote: > >> It is beneficial to

Re: [PATCH 3/3] btrfs-progs: convert: add support for converting reiserfs

2017-08-22 Thread David Sterba
On Tue, Jul 25, 2017 at 04:54:43PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > This patch adds support to convert reiserfs file systems in-place to btrfs. > > It will convert extended attribute files to btrfs extended attributes, > translate ACLs, coalesce tails that

Re: netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (21:45), Roman Mamedov wrote: > It is beneficial to not have snapshots in-place. With a local directory of > snapshots, issuing things like "find", "grep -r" or even "du" will take an > inordinate amount of time and will produce a result you do not expect. Netapp snapshots are

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Roman Mamedov
On Tue, 22 Aug 2017 17:45:37 +0200 Ulli Horlacher wrote: > In perl I have now: > > $root = $volume; > while (`btrfs subvolume show "$root" 2>/dev/null` !~ /toplevel subvolume/) { > $root = dirname($root); > last if $root eq '/'; > } > > If you are okay with

Re: netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (18:08), Peter Becker wrote: > This is possible. Use the -b or -B option. > > -b basedir places the snapshot in basedir with a directory structure > that mimics the mountpoint > -B basedir places the snapshots in basedir with NO additional > subdirectory structure > >

Re: netapp-alike snapshots?

2017-08-22 Thread Roman Mamedov
On Tue, 22 Aug 2017 16:24:51 +0200 Ulli Horlacher wrote: > On Tue 2017-08-22 (15:44), Peter Becker wrote: > > Is use: https://github.com/jf647/btrfs-snap > > > > 2017-08-22 15:22 GMT+02:00 Ulli Horlacher : > > > With Netapp/waffle

Re: Btrfs Raid5 issue.

2017-08-22 Thread Robert LeBlanc
Thanks for the explanations. Chris, I don't think 'degraded' did anything to help the mounting, I just passed it in to see if it would help (I'm not sure if btrfs is "smart" enough to ignore a drive if it would increase the chance of mounting the volume even if it is degraded, but one could hope).

Re: netapp-alike snapshots?

2017-08-22 Thread Peter Becker
This is possible. Use the -b or -B option. -b basedir places the snapshot in basedir with a directory structure that mimics the mountpoint -B basedir places the snapshots in basedir with NO additional subdirectory structure 2017-08-22 16:24 GMT+02:00 Ulli Horlacher

Re: [PATCH 3/7] btrfs-progs: extent-cache: actually cache extent buffers

2017-08-22 Thread David Sterba
On Tue, Jul 25, 2017 at 04:51:34PM -0400, je...@suse.com wrote: > From: Jeff Mahoney > > We have the infrastructure to cache extent buffers but we don't actually > do the caching. As soon as the last reference is dropped, the buffer > is dropped. This patch keeps the extent

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (11:03), Austin S. Hemmelgarn wrote: > Or alternatively, repeatedly call `btrfs filesystem show` on the path, > removing one component from the end each time until you get a zero > return code. The path you called it on that got a zero return code is > where the mount is

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Austin S. Hemmelgarn
On 2017-08-22 10:43, Peter Grandi wrote: How do I find the root filesystem of a subvolume? Example: root@fex:~# df -T Filesystem Type 1K-blocks Used Available Use% Mounted on - -1073740800 104244552 967773976 10% /local/.backup/home [ ... ] I know, the root

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Peter Grandi
> How do I find the root filesystem of a subvolume? > Example: > root@fex:~# df -T > Filesystem Type 1K-blocks Used Available Use% Mounted on > - -1073740800 104244552 967773976 10% /local/.backup/home [ ... ] > I know, the root filesystem is /local, That question is

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Austin S. Hemmelgarn
On 2017-08-22 10:23, Hugo Mills wrote: On Tue, Aug 22, 2017 at 10:12:25AM -0400, Austin S. Hemmelgarn wrote: On 2017-08-22 09:53, Ulli Horlacher wrote: On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote: root@fex:~# df -T /local/.backup/home Filesystem Type 1K-blocks Used

Re: [PATCH 2/2] btrfs-progs: Use named constants for common sizes

2017-08-22 Thread David Sterba
On Thu, Jul 27, 2017 at 11:17:00AM +0300, Nikolay Borisov wrote: > There multiple places where we use well-known sizes - 1,8,16,32 megabytes. We > also have them defined as constants in the sizes.h header. So let's use them. > No functional changes. Both applied, thanks. -- To unsubscribe from

Re: [PATCH 2/2] btrfs-progs: Use named constants for common sizes

2017-08-22 Thread David Sterba
On Thu, Jul 27, 2017 at 09:02:12PM +, Duncan wrote: > Nikolay Borisov posted on Thu, 27 Jul 2017 11:17:00 +0300 as excerpted: > > > diff --git a/convert/main.c b/convert/main.c > > index c56382e915fd..49ab829b5641 100644 > > --- a/convert/main.c > > +++ b/convert/main.c > > > @@ -1586,7

Re: netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (15:44), Peter Becker wrote: > Is use: https://github.com/jf647/btrfs-snap > > 2017-08-22 15:22 GMT+02:00 Ulli Horlacher : > > With Netapp/waffle you have automatic hourly/daily/weekly snapshots. > > You can find these snapshots in every local

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Hugo Mills
On Tue, Aug 22, 2017 at 10:12:25AM -0400, Austin S. Hemmelgarn wrote: > On 2017-08-22 09:53, Ulli Horlacher wrote: > >On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote: > > > >>>root@fex:~# df -T /local/.backup/home > >>>Filesystem Type 1K-blocks Used Available Use% Mounted on > >>>-

Re: [PATCH] Btrfs-progs: Check root before printing item

2017-08-22 Thread David Sterba
On Mon, Aug 21, 2017 at 03:57:13PM +0800, zhangyu-f...@cn.fujitsu.com wrote: > From: Zhang Yu > > [TEST/fuzz] case: 004-simple-dump-tree > > Since the wrong key(DATA_RELOC_TREE CHUNK_ITEM 0) in root tree, > error calling print_chunk(), resulting in num_stripes == 0.

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Austin S. Hemmelgarn
On 2017-08-22 09:53, Ulli Horlacher wrote: On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote: root@fex:~# df -T /local/.backup/home Filesystem Type 1K-blocks Used Available Use% Mounted on - -1073740800 104252160 967766336 10% /local/.backup/home Hmm, now I'm

Re: [PATCH] btrfs-progs: Make in-place exit to a common exit block

2017-08-22 Thread David Sterba
On Tue, Aug 22, 2017 at 01:35:06PM +0800, Gu Jinxiang wrote: > As comment pointed out by David, make in-place exit > to a common exit block of mkfs. > > v1: > Add some close(fd) when error occures in mkfs. > And add close(fd) when end use it. > > Signed-off-by: Gu Jinxiang

Re: [PATCH] btrfs-progs: mkfs: Replace number with enum

2017-08-22 Thread David Sterba
On Mon, Aug 21, 2017 at 07:39:49PM +0200, David Sterba wrote: > > +/* roots: root tree, extent tree, chunk tree, dev tree, fs tree, csum tree > > */ > > +enum btrfs_mkfs_block { > > + SUPER_BLOCK = 0, > > + ROOT_TREE, > > + EXTENT_TREE, > > + CHUNK_TREE, > > + DEV_TREE, > > + FS_TREE,

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote: > > root@fex:~# df -T /local/.backup/home > > Filesystem Type 1K-blocks Used Available Use% Mounted on > > - -1073740800 104252160 967766336 10% /local/.backup/home > > Hmm, now I'm really confused, I just checked

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Marat Khalili
Hmm, now I'm really confused, I just checked on the Ubuntu 17.04 and 16.04.3 VM's I have (I only run current and the most recent LTS version), and neither of them behave like this. Was also shocked, but: $ lsb_release -a No LSB modules are available. Distributor ID:Ubuntu Description:

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Marat Khalili
I have no subvol=/ option at all: Probably depends on kernel, but I presume missing subvol means the same as subvol=/ . I am only interested in mounted volumes. If your initial path (/local/.backup/home) is a subvolume but it's not itself present in /proc/mounts then it's probably mounted as

Re: netapp-alike snapshots?

2017-08-22 Thread Peter Becker
Is use: https://github.com/jf647/btrfs-snap 2017-08-22 15:22 GMT+02:00 Ulli Horlacher : > With Netapp/waffle you have automatic hourly/daily/weekly snapshots. > You can find these snapshots in every local directory (readonly). > Example: > > framstag@fex:/sw/share:

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Austin S. Hemmelgarn
On 2017-08-22 09:30, Ulli Horlacher wrote: On Tue 2017-08-22 (09:27), Austin S. Hemmelgarn wrote: root@fex:~# df -T Filesystem Type 1K-blocks Used Available Use% Mounted on - -1073740800 104244552 967773976 10% /local/.backup/home I've never seen the "- -"

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (09:27), Austin S. Hemmelgarn wrote: > >>> root@fex:~# df -T > >>> Filesystem Type 1K-blocks Used Available Use% Mounted on > >>> - -1073740800 104244552 967773976 10% > >>> /local/.backup/home > >> > >> I've never seen the "- -" output from df

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Austin S. Hemmelgarn
On 2017-08-22 08:50, Ulli Horlacher wrote: On Tue 2017-08-22 (12:40), Hugo Mills wrote: On Tue, Aug 22, 2017 at 02:23:50PM +0200, Ulli Horlacher wrote: How do I find the root filesystem of a subvolume? Example: root@fex:~# df -T Filesystem Type 1K-blocks Used Available Use% Mounted

netapp-alike snapshots?

2017-08-22 Thread Ulli Horlacher
With Netapp/waffle you have automatic hourly/daily/weekly snapshots. You can find these snapshots in every local directory (readonly). Example: framstag@fex:/sw/share: ll .snapshot/ drwxr-xr-x framstag root - 2017-08-14 10:21:47 .snapshot/daily.2017-08-15_0010 drwxr-xr-x framstag root -

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (15:58), Marat Khalili wrote: > On 22/08/17 15:50, Ulli Horlacher wrote: > > > It seems, I have to scan the subvolume path upwards until I found a real > > mount point, > > I think searching /proc/mounts for the same device and subvol=/ in > options is most straightforward. I

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Marat Khalili
On 22/08/17 15:50, Ulli Horlacher wrote: It seems, I have to scan the subvolume path upwards until I found a real mount point, I think searching /proc/mounts for the same device and subvol=/ in options is most straightforward. But what makes you think it's mounted at all? -- With Best

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Ulli Horlacher
On Tue 2017-08-22 (12:40), Hugo Mills wrote: > On Tue, Aug 22, 2017 at 02:23:50PM +0200, Ulli Horlacher wrote: > > > How do I find the root filesystem of a subvolume? > > Example: > > > > root@fex:~# df -T > > Filesystem Type 1K-blocks Used Available Use% Mounted on > > -

Re: finding root filesystem of a subvolume?

2017-08-22 Thread Hugo Mills
On Tue, Aug 22, 2017 at 02:23:50PM +0200, Ulli Horlacher wrote: > How do I find the root filesystem of a subvolume? > Example: > > root@fex:~# df -T > Filesystem Type 1K-blocks Used Available Use% Mounted on > - -1073740800 104244552 967773976 10% /local/.backup/home

finding root filesystem of a subvolume?

2017-08-22 Thread Ulli Horlacher
How do I find the root filesystem of a subvolume? Example: root@fex:~# df -T Filesystem Type 1K-blocks Used Available Use% Mounted on - -1073740800 104244552 967773976 10% /local/.backup/home root@fex:~# btrfs subvolume show /local/.backup/home /local/.backup/home

Re: [PATCH 3/3] btrfs: Add sanity check for EXTENT_DATA when reading out leaf

2017-08-22 Thread Nikolay Borisov
On 22.08.2017 14:23, Qu Wenruo wrote: > > > On 2017年08月22日 19:00, Nikolay Borisov wrote: >> >> >> On 22.08.2017 13:57, Nikolay Borisov wrote: >>> >>> >>> On 22.08.2017 10:37, Qu Wenruo wrote: Add extra checker for item with EXTENT_DATA type. This checks the following thing: 1)

Re: [PATCH 3/3] btrfs: Add sanity check for EXTENT_DATA when reading out leaf

2017-08-22 Thread Qu Wenruo
On 2017年08月22日 19:00, Nikolay Borisov wrote: On 22.08.2017 13:57, Nikolay Borisov wrote: On 22.08.2017 10:37, Qu Wenruo wrote: Add extra checker for item with EXTENT_DATA type. This checks the following thing: 1) Item size Plain text inline file extent size must match item size.

Re: [PATCH 3/3] btrfs: Add sanity check for EXTENT_DATA when reading out leaf

2017-08-22 Thread Nikolay Borisov
On 22.08.2017 13:57, Nikolay Borisov wrote: > > > On 22.08.2017 10:37, Qu Wenruo wrote: >> Add extra checker for item with EXTENT_DATA type. >> This checks the following thing: >> 1) Item size >>Plain text inline file extent size must match item size. >>(compressed inline file extent

Re: [PATCH 0/3] Introduce comprehensive sanity check framework and

2017-08-22 Thread Nikolay Borisov
On 22.08.2017 10:37, Qu Wenruo wrote: > The patchset introduce a new framework to do more comprehensive (if not > the most) sanity check when reading out a leaf. > > The new sanity checker will include: > > 1) Key order >Existing code > > 2) Item boundary >Existing code with enhanced

Re: [PATCH 3/3] btrfs: Add sanity check for EXTENT_DATA when reading out leaf

2017-08-22 Thread Nikolay Borisov
On 22.08.2017 10:37, Qu Wenruo wrote: > Add extra checker for item with EXTENT_DATA type. > This checks the following thing: > 1) Item size >Plain text inline file extent size must match item size. >(compressed inline file extent has no info about its on-disk size) >

Re: degraded BTRFS RAID 1 not mountable: open_ctree failed, unable to find block group for 0

2017-08-22 Thread Dmitrii Tcvetkov
On Tue, 22 Aug 2017 11:31:23 +0200 g6094...@freenet.de wrote: > So 1st should be investigating why did the disk not get removed > correctly? Btrfs dev del should remove the device corretly, right? Is > there a bug? It should and probably did. To check that we need to see output of btrfs

RE: [PATCH 00/15] Btrfs-progs offline scrub

2017-08-22 Thread Gu, Jinxiang
Ping -Original Message- From: linux-btrfs-ow...@vger.kernel.org [mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Gu Jinxiang Sent: Tuesday, July 18, 2017 2:34 PM To: linux-btrfs@vger.kernel.org Cc: quwenruo.bt...@gmx.com Subject: [PATCH 00/15] Btrfs-progs offline scrub For any

Re: degraded BTRFS RAID 1 not mountable: open_ctree failed, unable to find block group for 0

2017-08-22 Thread g6094199
He guys, picking up this old topic cause i'm running into a similar problem. Running a Ubuntu 16.04 (HWE K4.8) server with 2 nvme SSD as Raid1 as /. Since one nvme died i had to replace it, where the trouble began. I replaced the nvme, bootet degraded, added the new disk to the raid (btrfs dev

[PATCH 1/3] btrfs: Refactor check_leaf function for later expansion.

2017-08-22 Thread Qu Wenruo
Current check_leaf() function does a good job checking key orders and item offset/size. However it only checks from slot 0 to the last but one slot, this is good but makes later expansion hard. So this refactoring iterates from slot 0 to the last slot. For key comparison, it uses a key with all

[PATCH 3/3] btrfs: Add sanity check for EXTENT_DATA when reading out leaf

2017-08-22 Thread Qu Wenruo
Add extra checker for item with EXTENT_DATA type. This checks the following thing: 1) Item size Plain text inline file extent size must match item size. (compressed inline file extent has no info about its on-disk size) Regular/preallocated file extent size must be a fixed value. 2)

[PATCH 0/3] Introduce comprehensive sanity check framework and

2017-08-22 Thread Qu Wenruo
The patchset introduce a new framework to do more comprehensive (if not the most) sanity check when reading out a leaf. The new sanity checker will include: 1) Key order Existing code 2) Item boundary Existing code with enhanced checker to ensure item pointer doesn't overlap with item

[PATCH 2/3] btrfs: Check if item pointer overlap with item itself

2017-08-22 Thread Qu Wenruo
Function check_leaf() checks if any item pointer points outside of the leaf, but it doesn't check if the pointer overlap with the item itself. Normally only the last item may be the victim, but add such check is never a bad idea anyway. Signed-off-by: Qu Wenruo ---

Re: Btrfs Raid5 issue.

2017-08-22 Thread Qu Wenruo
On 2017年08月22日 13:19, Robert LeBlanc wrote: Chris and Qu thanks for your help. I was able to restore the data off the volume. I only could not read one file that I tried to rsync (a MySQl bin log), but it wasn't critical as I had an off-site snapshot from that morning and ownclould could