Re: [PATCH] btrfs: resume qgroup rescan on rw remount

2017-08-25 Thread Aleksa Sarai
On 07/12/2017 03:03 AM, David Sterba wrote: On Mon, Jul 10, 2017 at 04:56:36PM +0300, Nikolay Borisov wrote: On 10.07.2017 16:12, Nikolay Borisov wrote: On 4.07.2017 14:49, Aleksa Sarai wrote: Several distributions mount the "proper root" as ro during initrd and then remount it as rw before p

Re: status of inline deduplication in btrfs

2017-08-25 Thread Duncan
shally verma posted on Fri, 25 Aug 2017 23:01:10 +0530 as excerpted: > On Thu, Aug 24, 2017 at 6:39 AM, Tsutomu Itoh > wrote: >> On 2017/08/23 23:52, shally verma wrote: >>> HI >>> >>> Through btrfs wiki, I got to know about inline patch and this git >>> location https://github.com/adam900710/lin

[PATCH 1/1] btrfs-progs: mkfs: add subvolume support to mkfs

2017-08-25 Thread Yingyi Luo
From: yingyil Add -S/--subvol [NAME] option to configure. It enables users to create a subvolume under the toplevel volume and populate the created subvolume with files from the rootdir specified by -r/--rootdir option. Two functions link_subvol() and create_subvol() are moved from convert/main.

[PATCH 0/1] btrfs-progs: mkfs: add subvolume support to mkfs

2017-08-25 Thread Yingyi Luo
From: yingyil Add -S/--subvol [NAME] option to configure. It enables users to create a subvolume under the toplevel volume and populate the created subvolume with files from the rootdir specified by -r/--rootdir option. Two functions link_subvol() and create_subvol() are moved from convert/main.

Re: number of subvolumes

2017-08-25 Thread Austin S. Hemmelgarn
On 2017-08-25 08:55, Ferry Toth wrote: Op Fri, 25 Aug 2017 07:45:44 -0400, schreef Austin S. Hemmelgarn: On 2017-08-24 17:56, Ferry Toth wrote: Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: We find that typically apt is very slow on a machine with 50 or so snapshots and raid10.

Re: [PATCH] btrfs-progs: Enable ThreadSanitizer, using D=tsan.

2017-08-25 Thread David Sterba
On Wed, Jul 12, 2017 at 01:04:49PM -0700, Adam Buchbinder wrote: > Tested with clang-3.9. > > Signed-off-by: Adam Buchbinder > --- > Makefile | 6 ++ > 1 file changed, 6 insertions(+) > > diff --git a/Makefile b/Makefile > index 81598df..8948301 100644 > --- a/Makefile > +++ b/Makefile > @@

Re: status of inline deduplication in btrfs

2017-08-25 Thread shally verma
On Thu, Aug 24, 2017 at 6:39 AM, Tsutomu Itoh wrote: > On 2017/08/23 23:52, shally verma wrote: >> HI >> >> Through btrfs wiki, I got to know about inline patch and this git >> location https://github.com/adam900710/linux but I am not sure what's >> progress and status on this. Could any one pleas

Re: [PATCH][v2] btrfs-progs: print the csum length in debug-tree

2017-08-25 Thread Nikolay Borisov
On 25.08.2017 18:11, jo...@toxicpanda.com wrote: > From: Josef Bacik > > While looking at a log of a corrupted fs I needed to verify we were > missing csums for a given range. Make this easier by printing out the > range of bytes a csum item covers. > > Signed-off-by: Josef Bacik Reviewed-b

[PATCH][v2] btrfs-progs: print the csum length in debug-tree

2017-08-25 Thread josef
From: Josef Bacik While looking at a log of a corrupted fs I needed to verify we were missing csums for a given range. Make this easier by printing out the range of bytes a csum item covers. Signed-off-by: Josef Bacik --- v1->v2: - print the byte range instead of just the length print-tree.c

Re: [PATCH] btrfs-progs: print the csum length in debug-tree

2017-08-25 Thread Josef Bacik
On Fri, Aug 25, 2017 at 04:48:43PM +0300, Nikolay Borisov wrote: > > > On 25.08.2017 16:13, jo...@toxicpanda.com wrote: > > From: Josef Bacik > > > > While looking at a log of a corrupted fs I needed to verify we were > > missing csums for a given range. Make this easier by printing out how >

Re: [PATCH] btrfs-progs: fix cross-compile build

2017-08-25 Thread Goffredo Baroncelli
On 08/24/2017 07:01 PM, David Sterba wrote: > On Wed, Aug 16, 2017 at 08:17:00AM +0800, Qu Wenruo wrote: >> >> >> On 2017年08月16日 02:11, Eric Sandeen wrote: >>> The mktables binary needs to be build with the host >>> compiler at built time, not the target compiler, because >>> it runs at build time

Re: [PATCH 00/14 RFC] Btrfs: Add journal for raid5/6 writes

2017-08-25 Thread Goffredo Baroncelli
On 08/23/2017 05:28 PM, Chris Murphy wrote: > - dynamically sized stripes, so that writes can always be full stripe > writes, no overwrites, and atomic Think about also that a block could be deallocated (i.e. canceling a file of 4kb). This leads to have some holes that you cannot fill without a RM

Re: [PATCH] btrfs-progs: print the csum length in debug-tree

2017-08-25 Thread Nikolay Borisov
On 25.08.2017 16:13, jo...@toxicpanda.com wrote: > From: Josef Bacik > > While looking at a log of a corrupted fs I needed to verify we were > missing csums for a given range. Make this easier by printing out how > many bytes a csum extent item represents when using btrfs_debug_tree. > > Sign

[PATCH] btrfs-progs: print the csum length in debug-tree

2017-08-25 Thread josef
From: Josef Bacik While looking at a log of a corrupted fs I needed to verify we were missing csums for a given range. Make this easier by printing out how many bytes a csum extent item represents when using btrfs_debug_tree. Signed-off-by: Josef Bacik --- print-tree.c | 10 -- 1 file

Re: number of subvolumes

2017-08-25 Thread Ferry Toth
Op Fri, 25 Aug 2017 07:45:44 -0400, schreef Austin S. Hemmelgarn: > On 2017-08-24 17:56, Ferry Toth wrote: >> Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: >> We find that typically apt is very slow on a machine with 50 or so snapshots and raid10. Slow as in probably 10x sl

Re: number of subvolumes

2017-08-25 Thread Austin S. Hemmelgarn
On 2017-08-24 17:56, Ferry Toth wrote: Op Thu, 24 Aug 2017 22:40:54 +0300, schreef Marat Khalili: We find that typically apt is very slow on a machine with 50 or so snapshots and raid10. Slow as in probably 10x slower as doing the same update on a machine with 'single' and no snapshots. Other

Re: user snapshots

2017-08-25 Thread Austin S. Hemmelgarn
On 2017-08-23 17:13, Ulli Horlacher wrote: On Wed 2017-08-23 (12:42), Peter Grandi wrote: So, still: What is the problem with user_subvol_rm_allowed? As usual, it is complicated: mostly that while subvol creation is very cheap, subvol deletion can be very expensive. But then so can be creating

[PATCH v7 4/6] Btrfs: heuristic add detection of repeated data patterns

2017-08-25 Thread Timofey Titovets
Walk over data sample and use memcmp to detect repeated data (like zeroed) Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 16 1 file changed, 16 insertions(+) diff --git a/fs/btrfs/heuristic.c b/fs/btrfs/heuristic.c index 5192e51ab81e..f1fa6e4f1c11 100644 --- a/fs/b

[PATCH v7 3/6] Btrfs: implement heuristic sampling logic

2017-08-25 Thread Timofey Titovets
Copy sample data from input data range to sample buffer then calculate byte type count for that sample into bucket. Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 38 +- 1 file changed, 37 insertions(+), 1 deletion(-) diff --git a/fs/btrfs/heurist

[PATCH v7 0/6] Btrfs: populate heuristic with code

2017-08-25 Thread Timofey Titovets
Based on kdave for-next Patches short: 1. Move heuristic to use compression workspaces Bit tricky, but works. 2. Add heuristic counters and buffer to workspaces 3. Implement simple input data sampling It's get 16 byte samples with 256 bytes shifts over input data. Collect info about how

[PATCH v7 1/6] Btrfs: heuristic make use compression workspaces

2017-08-25 Thread Timofey Titovets
Move heuristic to external file Implement compression workspaces support for heuristic resources Signed-off-by: Timofey Titovets --- fs/btrfs/Makefile | 2 +- fs/btrfs/compression.c | 18 + fs/btrfs/compression.h | 7 - fs/btrfs/heuristic.c | 73 +

[PATCH v7 5/6] Btrfs: heuristic add byte set calculation

2017-08-25 Thread Timofey Titovets
Calculate byte set size for data sample: Calculate how many unique bytes has been in sample By count all bytes in bucket with count > 0 If byte set low (~25%), data are easily compressible Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 26 ++ 1 file changed, 2

[PATCH v7 2/6] Btrfs: heuristic workspace add bucket and sample items

2017-08-25 Thread Timofey Titovets
Heuristic workspace: - Add bucket for storing byte type counters - Add sample array for storing partial copy of input data range - Add counter for store current sample size to workspace Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 23 +++ 1 file changed, 2

[PATCH v7 6/6] Btrfs: heuristic add byte core set calculation

2017-08-25 Thread Timofey Titovets
Calculate byte core set for data sample: Sort bucket's numbers in decreasing order Count how many numbers use 90% of sample If core set are low (<=25%), data are easily compressible If core set high (>=80%), data are not compressible Signed-off-by: Timofey Titovets --- fs/btrfs/heuristic.c | 51

[PATCH v2] Btrfs: handle unaligned tail of data ranges more efficient

2017-08-25 Thread Timofey Titovets
At now while switch page bits in data ranges we always hande +1 page, for cover case where end of data range is not page aligned Let's handle that case more obvious and efficient Check end aligment directly and touch +1 page only then needed Signed-off-by: Timofey Titovets --- fs/btrfs/extent_i

Re: [PATCH][v2] btrfs: change how we decide to commit transactions during flushing

2017-08-25 Thread Nikolay Borisov
On 24.08.2017 17:43, Nikolay Borisov wrote: > > > On 22.08.2017 23:00, jo...@toxicpanda.com wrote: >> From: Josef Bacik >> >> Nikolay reported that generic/273 was failing currently with ENOSPC. >> Turns out this is because we get to the point where the outstanding >> reservations are greater

Re: [PATCH] Btrfs: do not reset bio->bi_ops while writing bio

2017-08-25 Thread Christoph Hellwig
On Thu, Aug 24, 2017 at 11:04:40AM -0700, Liu Bo wrote: > On Thu, Aug 24, 2017 at 07:10:41AM -0700, Christoph Hellwig wrote: > > On Wed, Aug 23, 2017 at 12:15:09PM -0600, Liu Bo wrote: > > > flush_epd_write_bio() sets bio->bi_ops by itself to honor REQ_SYNC, > > > but it's not needed at all since b

[PATCH] btrfs-progs: Make the process of create root tree to a function

2017-08-25 Thread Gu Jinxiang
make_btrfs is too long to understand, make creatation of root tree in a function. Signed-off-by: Gu Jinxiang --- mkfs/common.c | 136 +++--- 1 file changed, 64 insertions(+), 72 deletions(-) diff --git a/mkfs/common.c b/mkfs/common.c index 7d4