From: Su Yue
Turn on the option --repair with --mode==lowmem in btrfsck.
Signed-off-by: Su Yue
---
cmds-check.c | 8 +++-
1 file changed, 3 insertions(+), 5 deletions(-)
diff --git a/cmds-check.c b/cmds-check.c
index c5faa2b..829f7c5
From: Su Yue
Because this patchset concentrates on repair of extent tree,
repair_chunk_item() now only inserts missed chunk group item into
extent tree.
There are some things left TODO, for example dev_item.
Signed-off-by: Su Yue
---
From: Su Yue
Introduce delete_extent_tree_item() and repair_extent_item() to do
delete only.
while checking a extent tree, just delete wrong item.
For extent item, free wrong backref. Otherwise, do delete.
So the rest items in extent tree should be correct.
From: Su Yue
The only thing repair_tree_block_ref() does is that adding backref of the
tree_block. Just like what origin repair do:
It first searches the correspond extent item then
1. If the extent item exists but backref is missing, add one backref to the
extent.
From: Su Yue
This is part 2 of lowmem repair patchsets:
1. Change the way of traversal under lowmem check to use walk_up_tree_v2() and
walk_down_tree_v2() and it scans all trees now.
2. Repair cases: block group missing, tree block backref missing,
extent item
From: Su Yue
The only thing repair_extent_data_item() does is that adding backref of the
tree_block. Just like what origin repair do:
It first searches the correspond extent item then
1. If the extent item exists but backref is missing, add one backref to the
extent.
From: Su Yue
This patch is a preparation for extent-tree repair in lowmem mode.
In the lowmem mode, checking tree blocks of various tree is in
recursive way.
But if during repair, add or delete of item(s) may modify upper nodes
which will cause the repair to be
> -Original Message-
> From: David Sterba [mailto:dste...@suse.cz]
> Sent: Tuesday, August 22, 2017 10:04 PM
> To: Gu, Jinxiang/顾 金香 ; linux-btrfs@vger.kernel.org
> Subject: Re: [PATCH] btrfs-progs: mkfs: Replace number with enum
>
> On Mon, Aug 21, 2017 at
Move heuristic to external file
Implement compression workspaces support for
heuristic resources
Signed-off-by: Timofey Titovets
---
fs/btrfs/Makefile | 2 +-
fs/btrfs/compression.c | 18 +
fs/btrfs/compression.h | 7 -
fs/btrfs/heuristic.c | 70
Heuristic workspace:
- Add bucket for storing byte type counters
- Add sample array for storing partial copy of
input data range
- Add counter for store current sample size to workspace
Signed-off-by: Timofey Titovets
---
fs/btrfs/heuristic.c | 30
Copy sample data from input data range to sample buffer
then calculate byte type count for that sample into bucket.
Signed-off-by: Timofey Titovets
---
fs/btrfs/heuristic.c | 31 +--
1 file changed, 29 insertions(+), 2 deletions(-)
diff --git
Use memcmp for check sample data to zeroes.
Signed-off-by: Timofey Titovets
---
fs/btrfs/heuristic.c | 18 ++
1 file changed, 18 insertions(+)
diff --git a/fs/btrfs/heuristic.c b/fs/btrfs/heuristic.c
index 5336638a3b7c..4557ea1db373 100644
---
Calculate byte set size for data sample:
Calculate how many unique bytes has been in sample
By count all bytes in bucket with count > 0
If byte set low (~25%), data are easily compressible
Signed-off-by: Timofey Titovets
---
fs/btrfs/heuristic.c | 26
Calculate byte core set for data sample:
Sort bucket's numbers in decreasing order
Count how many numbers use 90% of sample
If core set are low (<=25%), data are easily compressible
If core set high (>=80%), data are not compressible
Signed-off-by: Timofey Titovets
---
Based on kdave for-next
Patches short:
1. Move heuristic to use compression workspaces
Bit tricky, but works.
2. Add heuristic counters and buffer to workspaces
3. Implement simple input data sampling
It's get 16 byte samples with 256 bytes shifts
over input data. Collect info about
On 2017年08月23日 00:37, Robert LeBlanc wrote:
Thanks for the explanations. Chris, I don't think 'degraded' did
anything to help the mounting, I just passed it in to see if it would
help (I'm not sure if btrfs is "smart" enough to ignore a drive if it
would increase the chance of mounting the
On Tue 2017-08-22 (19:36), Peter Grandi wrote:
> For somewhat good reasons subvolumes including snapshots cannot be
> deleted by users though unless mount option 'user_subvol_rm_allowed' is
> used.
Also in https://btrfs.wiki.kernel.org/index.php/Mount_options
"user_subvol_rm_allowed (...) Use
On Tue 2017-08-22 (19:36), Peter Grandi wrote:
> Indeed and there is a fair description of some options for
> subvolume nesting policies here which may be interesting to the
> original poster:
>
> https://btrfs.wiki.kernel.org/index.php/SysadminGuide#Layout
>
> It is unsurprising to me that
tree: https://git.kernel.org/pub/scm/linux/kernel/git/josef/btrfs-next.git
slab-priority
head: a1be3b41415243d20c90e9e92e82808fe1ff91a0
commit: fe049b0156a10dd0bb3fbf3d4dad3ca943874f10 [4/6] remove mapping from
balance_dirty_pages*()
config: i386-randconfig-a1-201734 (attached as .config)
From: Josef Bacik
Nikolay reported that generic/273 was failing currently with ENOSPC.
Turns out this is because we get to the point where the outstanding
reservations are greater than the pinned space on the fs. This is a
mistake, previously we used the current reservation
[ ... ]
It is beneficial to not have snapshots in-place. With a local
directory of snapshots, [ ... ]
Indeed and there is a fair description of some options for
subvolume nesting policies here which may be interesting to the
original poster:
On Thu, Jul 27, 2017 at 11:47:18AM -0400, je...@suse.com wrote:
> From: Jeff Mahoney
>
> Changes since v1:
> - reiserfs conversion:
> - use bool instead of int
> - catch 'impossible' condition of multiple discontiguous tails
> - properly handle hole followed by tail
> -
On Tue 2017-08-22 (22:36), Roman Mamedov wrote:
> > My users want the snapshots locally in a .snapshot subdirectory.
> > Because Netapp do it this way - for at least 20 years and we have a
> > multi-PB Netapp storage environment.
>
> Just a side note, you do know that only subvolumes can be
On Tue 2017-08-22 (19:19), A L wrote:
> Perhaps using a bind mount? It would look and work the same as a ordinary fs.
> Just need to make sure du uses one filesystem.
>
> From: Ulli Horlacher -- Sent: 2017-08-22
> - 18:57
>
> > On Tue 2017-08-22
[ ... ]
>> There is no fixed relationship between the root directory
>> inode of a subvolume and the root directory inode of any
>> other subvolume or the main volume.
> Actually, there is, because it's inherently rooted in the
> hierarchy of the volume itself. That root inode for the
>
On Tue, 22 Aug 2017 18:57:25 +0200
Ulli Horlacher wrote:
> On Tue 2017-08-22 (21:45), Roman Mamedov wrote:
>
> > It is beneficial to not have snapshots in-place. With a local directory of
> > snapshots, issuing things like "find", "grep -r" or even "du" will take
Perhaps using a bind mount? It would look and work the same as a ordinary fs.
Just need to make sure du uses one filesystem.
From: Ulli Horlacher -- Sent: 2017-08-22 -
18:57
> On Tue 2017-08-22 (21:45), Roman Mamedov wrote:
>
>> It is beneficial to
On Tue, Jul 25, 2017 at 04:54:43PM -0400, je...@suse.com wrote:
> From: Jeff Mahoney
>
> This patch adds support to convert reiserfs file systems in-place to btrfs.
>
> It will convert extended attribute files to btrfs extended attributes,
> translate ACLs, coalesce tails that
On Tue 2017-08-22 (21:45), Roman Mamedov wrote:
> It is beneficial to not have snapshots in-place. With a local directory of
> snapshots, issuing things like "find", "grep -r" or even "du" will take an
> inordinate amount of time and will produce a result you do not expect.
Netapp snapshots are
On Tue, 22 Aug 2017 17:45:37 +0200
Ulli Horlacher wrote:
> In perl I have now:
>
> $root = $volume;
> while (`btrfs subvolume show "$root" 2>/dev/null` !~ /toplevel subvolume/) {
> $root = dirname($root);
> last if $root eq '/';
> }
>
>
If you are okay with
On Tue 2017-08-22 (18:08), Peter Becker wrote:
> This is possible. Use the -b or -B option.
>
> -b basedir places the snapshot in basedir with a directory structure
> that mimics the mountpoint
> -B basedir places the snapshots in basedir with NO additional
> subdirectory structure
>
>
On Tue, 22 Aug 2017 16:24:51 +0200
Ulli Horlacher wrote:
> On Tue 2017-08-22 (15:44), Peter Becker wrote:
> > Is use: https://github.com/jf647/btrfs-snap
> >
> > 2017-08-22 15:22 GMT+02:00 Ulli Horlacher :
> > > With Netapp/waffle
Thanks for the explanations. Chris, I don't think 'degraded' did
anything to help the mounting, I just passed it in to see if it would
help (I'm not sure if btrfs is "smart" enough to ignore a drive if it
would increase the chance of mounting the volume even if it is
degraded, but one could hope).
This is possible. Use the -b or -B option.
-b basedir places the snapshot in basedir with a directory structure
that mimics the mountpoint
-B basedir places the snapshots in basedir with NO additional
subdirectory structure
2017-08-22 16:24 GMT+02:00 Ulli Horlacher
On Tue, Jul 25, 2017 at 04:51:34PM -0400, je...@suse.com wrote:
> From: Jeff Mahoney
>
> We have the infrastructure to cache extent buffers but we don't actually
> do the caching. As soon as the last reference is dropped, the buffer
> is dropped. This patch keeps the extent
On Tue 2017-08-22 (11:03), Austin S. Hemmelgarn wrote:
> Or alternatively, repeatedly call `btrfs filesystem show` on the path,
> removing one component from the end each time until you get a zero
> return code. The path you called it on that got a zero return code is
> where the mount is
On 2017-08-22 10:43, Peter Grandi wrote:
How do I find the root filesystem of a subvolume?
Example:
root@fex:~# df -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
- -1073740800 104244552 967773976 10% /local/.backup/home
[ ... ]
I know, the root
> How do I find the root filesystem of a subvolume?
> Example:
> root@fex:~# df -T
> Filesystem Type 1K-blocks Used Available Use% Mounted on
> - -1073740800 104244552 967773976 10% /local/.backup/home
[ ... ]
> I know, the root filesystem is /local,
That question is
On 2017-08-22 10:23, Hugo Mills wrote:
On Tue, Aug 22, 2017 at 10:12:25AM -0400, Austin S. Hemmelgarn wrote:
On 2017-08-22 09:53, Ulli Horlacher wrote:
On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote:
root@fex:~# df -T /local/.backup/home
Filesystem Type 1K-blocks Used
On Thu, Jul 27, 2017 at 11:17:00AM +0300, Nikolay Borisov wrote:
> There multiple places where we use well-known sizes - 1,8,16,32 megabytes. We
> also have them defined as constants in the sizes.h header. So let's use them.
> No functional changes.
Both applied, thanks.
--
To unsubscribe from
On Thu, Jul 27, 2017 at 09:02:12PM +, Duncan wrote:
> Nikolay Borisov posted on Thu, 27 Jul 2017 11:17:00 +0300 as excerpted:
>
> > diff --git a/convert/main.c b/convert/main.c
> > index c56382e915fd..49ab829b5641 100644
> > --- a/convert/main.c
> > +++ b/convert/main.c
>
> > @@ -1586,7
On Tue 2017-08-22 (15:44), Peter Becker wrote:
> Is use: https://github.com/jf647/btrfs-snap
>
> 2017-08-22 15:22 GMT+02:00 Ulli Horlacher :
> > With Netapp/waffle you have automatic hourly/daily/weekly snapshots.
> > You can find these snapshots in every local
On Tue, Aug 22, 2017 at 10:12:25AM -0400, Austin S. Hemmelgarn wrote:
> On 2017-08-22 09:53, Ulli Horlacher wrote:
> >On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote:
> >
> >>>root@fex:~# df -T /local/.backup/home
> >>>Filesystem Type 1K-blocks Used Available Use% Mounted on
> >>>-
On Mon, Aug 21, 2017 at 03:57:13PM +0800, zhangyu-f...@cn.fujitsu.com wrote:
> From: Zhang Yu
>
> [TEST/fuzz] case: 004-simple-dump-tree
>
> Since the wrong key(DATA_RELOC_TREE CHUNK_ITEM 0) in root tree,
> error calling print_chunk(), resulting in num_stripes == 0.
On 2017-08-22 09:53, Ulli Horlacher wrote:
On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote:
root@fex:~# df -T /local/.backup/home
Filesystem Type 1K-blocks Used Available Use% Mounted on
- -1073740800 104252160 967766336 10% /local/.backup/home
Hmm, now I'm
On Tue, Aug 22, 2017 at 01:35:06PM +0800, Gu Jinxiang wrote:
> As comment pointed out by David, make in-place exit
> to a common exit block of mkfs.
>
> v1:
> Add some close(fd) when error occures in mkfs.
> And add close(fd) when end use it.
>
> Signed-off-by: Gu Jinxiang
On Mon, Aug 21, 2017 at 07:39:49PM +0200, David Sterba wrote:
> > +/* roots: root tree, extent tree, chunk tree, dev tree, fs tree, csum tree
> > */
> > +enum btrfs_mkfs_block {
> > + SUPER_BLOCK = 0,
> > + ROOT_TREE,
> > + EXTENT_TREE,
> > + CHUNK_TREE,
> > + DEV_TREE,
> > + FS_TREE,
On Tue 2017-08-22 (09:37), Austin S. Hemmelgarn wrote:
> > root@fex:~# df -T /local/.backup/home
> > Filesystem Type 1K-blocks Used Available Use% Mounted on
> > - -1073740800 104252160 967766336 10% /local/.backup/home
>
> Hmm, now I'm really confused, I just checked
Hmm, now I'm really confused, I just checked on the Ubuntu 17.04 and
16.04.3 VM's I have (I only run current and the most recent LTS
version), and neither of them behave like this.
Was also shocked, but:
$ lsb_release -a
No LSB modules are available.
Distributor ID:Ubuntu
Description:
I have no subvol=/ option at all:
Probably depends on kernel, but I presume missing subvol means the same
as subvol=/ .
I am only interested in mounted volumes.
If your initial path (/local/.backup/home) is a subvolume but it's not
itself present in /proc/mounts then it's probably mounted as
Is use: https://github.com/jf647/btrfs-snap
2017-08-22 15:22 GMT+02:00 Ulli Horlacher :
> With Netapp/waffle you have automatic hourly/daily/weekly snapshots.
> You can find these snapshots in every local directory (readonly).
> Example:
>
> framstag@fex:/sw/share:
On 2017-08-22 09:30, Ulli Horlacher wrote:
On Tue 2017-08-22 (09:27), Austin S. Hemmelgarn wrote:
root@fex:~# df -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
- -1073740800 104244552 967773976 10% /local/.backup/home
I've never seen the "- -"
On Tue 2017-08-22 (09:27), Austin S. Hemmelgarn wrote:
> >>> root@fex:~# df -T
> >>> Filesystem Type 1K-blocks Used Available Use% Mounted on
> >>> - -1073740800 104244552 967773976 10%
> >>> /local/.backup/home
> >>
> >> I've never seen the "- -" output from df
On 2017-08-22 08:50, Ulli Horlacher wrote:
On Tue 2017-08-22 (12:40), Hugo Mills wrote:
On Tue, Aug 22, 2017 at 02:23:50PM +0200, Ulli Horlacher wrote:
How do I find the root filesystem of a subvolume?
Example:
root@fex:~# df -T
Filesystem Type 1K-blocks Used Available Use% Mounted
With Netapp/waffle you have automatic hourly/daily/weekly snapshots.
You can find these snapshots in every local directory (readonly).
Example:
framstag@fex:/sw/share: ll .snapshot/
drwxr-xr-x framstag root - 2017-08-14 10:21:47 .snapshot/daily.2017-08-15_0010
drwxr-xr-x framstag root -
On Tue 2017-08-22 (15:58), Marat Khalili wrote:
> On 22/08/17 15:50, Ulli Horlacher wrote:
>
> > It seems, I have to scan the subvolume path upwards until I found a real
> > mount point,
>
> I think searching /proc/mounts for the same device and subvol=/ in
> options is most straightforward.
I
On 22/08/17 15:50, Ulli Horlacher wrote:
It seems, I have to scan the subvolume path upwards until I found a real
mount point,
I think searching /proc/mounts for the same device and subvol=/ in
options is most straightforward. But what makes you think it's mounted
at all?
--
With Best
On Tue 2017-08-22 (12:40), Hugo Mills wrote:
> On Tue, Aug 22, 2017 at 02:23:50PM +0200, Ulli Horlacher wrote:
>
> > How do I find the root filesystem of a subvolume?
> > Example:
> >
> > root@fex:~# df -T
> > Filesystem Type 1K-blocks Used Available Use% Mounted on
> > -
On Tue, Aug 22, 2017 at 02:23:50PM +0200, Ulli Horlacher wrote:
> How do I find the root filesystem of a subvolume?
> Example:
>
> root@fex:~# df -T
> Filesystem Type 1K-blocks Used Available Use% Mounted on
> - -1073740800 104244552 967773976 10% /local/.backup/home
How do I find the root filesystem of a subvolume?
Example:
root@fex:~# df -T
Filesystem Type 1K-blocks Used Available Use% Mounted on
- -1073740800 104244552 967773976 10% /local/.backup/home
root@fex:~# btrfs subvolume show /local/.backup/home
/local/.backup/home
On 22.08.2017 14:23, Qu Wenruo wrote:
>
>
> On 2017年08月22日 19:00, Nikolay Borisov wrote:
>>
>>
>> On 22.08.2017 13:57, Nikolay Borisov wrote:
>>>
>>>
>>> On 22.08.2017 10:37, Qu Wenruo wrote:
Add extra checker for item with EXTENT_DATA type.
This checks the following thing:
1)
On 2017年08月22日 19:00, Nikolay Borisov wrote:
On 22.08.2017 13:57, Nikolay Borisov wrote:
On 22.08.2017 10:37, Qu Wenruo wrote:
Add extra checker for item with EXTENT_DATA type.
This checks the following thing:
1) Item size
Plain text inline file extent size must match item size.
On 22.08.2017 13:57, Nikolay Borisov wrote:
>
>
> On 22.08.2017 10:37, Qu Wenruo wrote:
>> Add extra checker for item with EXTENT_DATA type.
>> This checks the following thing:
>> 1) Item size
>>Plain text inline file extent size must match item size.
>>(compressed inline file extent
On 22.08.2017 10:37, Qu Wenruo wrote:
> The patchset introduce a new framework to do more comprehensive (if not
> the most) sanity check when reading out a leaf.
>
> The new sanity checker will include:
>
> 1) Key order
>Existing code
>
> 2) Item boundary
>Existing code with enhanced
On 22.08.2017 10:37, Qu Wenruo wrote:
> Add extra checker for item with EXTENT_DATA type.
> This checks the following thing:
> 1) Item size
>Plain text inline file extent size must match item size.
>(compressed inline file extent has no info about its on-disk size)
>
On Tue, 22 Aug 2017 11:31:23 +0200
g6094...@freenet.de wrote:
> So 1st should be investigating why did the disk not get removed
> correctly? Btrfs dev del should remove the device corretly, right? Is
> there a bug?
It should and probably did. To check that we need to see output of
btrfs
Ping
-Original Message-
From: linux-btrfs-ow...@vger.kernel.org
[mailto:linux-btrfs-ow...@vger.kernel.org] On Behalf Of Gu Jinxiang
Sent: Tuesday, July 18, 2017 2:34 PM
To: linux-btrfs@vger.kernel.org
Cc: quwenruo.bt...@gmx.com
Subject: [PATCH 00/15] Btrfs-progs offline scrub
For any
He guys,
picking up this old topic cause i'm running into a similar problem.
Running a Ubuntu 16.04 (HWE K4.8) server with 2 nvme SSD as Raid1 as /.
Since one nvme died i had to replace it, where the trouble began. I
replaced the nvme, bootet degraded, added the new disk to the raid
(btrfs dev
Current check_leaf() function does a good job checking key orders and
item offset/size.
However it only checks from slot 0 to the last but one slot, this is
good but makes later expansion hard.
So this refactoring iterates from slot 0 to the last slot.
For key comparison, it uses a key with all
Add extra checker for item with EXTENT_DATA type.
This checks the following thing:
1) Item size
Plain text inline file extent size must match item size.
(compressed inline file extent has no info about its on-disk size)
Regular/preallocated file extent size must be a fixed value.
2)
The patchset introduce a new framework to do more comprehensive (if not
the most) sanity check when reading out a leaf.
The new sanity checker will include:
1) Key order
Existing code
2) Item boundary
Existing code with enhanced checker to ensure item pointer doesn't
overlap with item
Function check_leaf() checks if any item pointer points outside of the
leaf, but it doesn't check if the pointer overlap with the item itself.
Normally only the last item may be the victim, but add such check is
never a bad idea anyway.
Signed-off-by: Qu Wenruo
---
On 2017年08月22日 13:19, Robert LeBlanc wrote:
Chris and Qu thanks for your help. I was able to restore the data off
the volume. I only could not read one file that I tried to rsync (a
MySQl bin log), but it wasn't critical as I had an off-site snapshot
from that morning and ownclould could
73 matches
Mail list logo