[PATCH v3] btrfs-progs: send-test: add checking of clone-src option

Sending stream size of clone-src(-c) option is checked. Fixed by "btrfs-progs: send: fix handling of -c option". Signed-off-by: Tsutomu Itoh --- V2: old sending stream image is used V3: image file has been compressed by gzip ---

[PATCH v2] fstests: btrfs: Check false ENOSPC bug caused by incorrect metadata reserve

Due to the fact that btrfs uses different max extent size for compressed and non-compressed write, it calculates metadata reservation incorrectly. This can leads to false ENOSPC alert for compressed write. This test case will check it by using small fs and large nodesize, and do parallel

Re: btrfs support for filesystems >8TB on 32bit architectures

On Wed, Nov 09, 2016 at 09:50:08AM +0800, Qu Wenruo wrote: > Yeah, quite possible! > > The truth is, current btrfs check only checks: > 1) Metadata >while --check-data-csum option will check data, but still >follow the restriction 3). > 2) Crossing reference of metadata (contents of

Re: [PATCH v2] btrfs-progs: send-test: add checking of clone-src option

On 2016/11/09 10:39, Qu Wenruo wrote: > > > At 11/09/2016 09:25 AM, Tsutomu Itoh wrote: >> Sending stream size of clone-src(-c) option is checked. >> Fixed by "btrfs-progs: send: fix handling of -c option". >> >> Signed-off-by: Tsutomu Itoh >> --- >> V2: old sending

Re: btrfs support for filesystems >8TB on 32bit architectures

At 11/08/2016 11:24 PM, Marc MERLIN wrote: On Tue, Nov 08, 2016 at 09:17:43AM +0800, Qu Wenruo wrote: At 11/08/2016 09:06 AM, Marc MERLIN wrote: On Tue, Nov 08, 2016 at 08:43:34AM +0800, Qu Wenruo wrote: That's strange, balance is done completely in kernel space. Unless we're calling

Re: [PATCH v2] btrfs-progs: send-test: add checking of clone-src option

At 11/09/2016 09:25 AM, Tsutomu Itoh wrote: Sending stream size of clone-src(-c) option is checked. Fixed by "btrfs-progs: send: fix handling of -c option". Signed-off-by: Tsutomu Itoh --- V2: old sending stream image is used ---

[PATCH v2] btrfs-progs: send-test: add checking of clone-src option

Sending stream size of clone-src(-c) option is checked. Fixed by "btrfs-progs: send: fix handling of -c option". Signed-off-by: Tsutomu Itoh --- V2: old sending stream image is used --- .../016-send-clone-src/send-stream-v4.8.2.img | Bin 0 -> 6299778 bytes

Re: [PATCH] fstests: btrfs: Check false ENOSPC bug caused by incorrect metadata reserve

At 11/08/2016 06:58 PM, Eryu Guan wrote: On Tue, Nov 08, 2016 at 05:15:15PM +0800, Qu Wenruo wrote: Due to the fact that btrfs uses different max extent size for compressed and non-compressed write, it calculates metadata reservation incorrectly. This can leads to false ENOSPC alert for

Re: [PATCH] btrfs-progs: send-test: add checking of clone-src option

On 2016/11/08 21:47, David Sterba wrote: > On Tue, Nov 08, 2016 at 08:53:04AM +0900, Tsutomu Itoh wrote: >> On 2016/11/08 0:16, David Sterba wrote: >>> On Fri, Nov 04, 2016 at 05:35:18PM +0900, Tsutomu Itoh wrote: +before_size=`ls -l "$here"/send.stream.before | awk '{print $5}'`

Re: Could receive allow updating an existing subvolume?

On Tue, Nov 8, 2016, at 03:15 PM, Ian Kelling wrote: > On Tue, Nov 8, 2016, at 03:00 PM, Hugo Mills wrote: > > > >If the sender sends an incremental stream, that assumes an *exact* > > subvol state on the receiving side. If the subvol on the receiving > > side is modified, then the receive

Re: Could receive allow updating an existing subvolume?

On Tue, Nov 8, 2016, at 03:00 PM, Hugo Mills wrote: > On Tue, Nov 08, 2016 at 02:48:56PM -0800, Ian Kelling wrote: > > It seems to be an artificially imposed limitation which hurts which > > hurts its usefulness. Let me know if this makes sense. If so, perhaps it > > can be implemented eventually.

Re: Could receive allow updating an existing subvolume?

On Tue, Nov 08, 2016 at 02:48:56PM -0800, Ian Kelling wrote: > It seems to be an artificially imposed limitation which hurts which > hurts its usefulness. Let me know if this makes sense. If so, perhaps it > can be implemented eventually. It seems a bit obvious but I couldn't > find any existing

Could receive allow updating an existing subvolume?

It seems to be an artificially imposed limitation which hurts which hurts its usefulness. Let me know if this makes sense. If so, perhaps it can be implemented eventually. It seems a bit obvious but I couldn't find any existing discussion of it. Say you have this situation: a/1, a/2 (parent is

Re: Announcing btrfs-dedupe

On Sun, 6 Nov 2016 14:30:52 +0100, James Pharaoh wrote : > Hi all, > > I'm pleased to announce my btrfs deduplication utility, written in > Rust. This operates on whole files, is fast, and I believe > complements the existing utilities (duperemove, bedup), which

Re: Announcing btrfs-dedupe

On Tue, Nov 08, 2016 at 10:59:56AM -0800, Mark Fasheh wrote: > On Mon, Nov 7, 2016 at 6:17 PM, Darrick J. Wong > wrote: > > On Mon, Nov 07, 2016 at 09:54:09PM +0100, Adam Borowski wrote: > >> Mark has already included XFS in documentation of duperemove, all that > >>

Re: Announcing btrfs-dedupe

On Mon, Nov 7, 2016 at 6:17 PM, Darrick J. Wong wrote: > On Mon, Nov 07, 2016 at 09:54:09PM +0100, Adam Borowski wrote: >> Mark has already included XFS in documentation of duperemove, all that looks >> amiss is btrfs-extent-same having an obsolete name. But then, I

Re: Announcing btrfs-dedupe

On Mon, Nov 7, 2016 at 6:40 PM, Christoph Anton Mitterer wrote: > On Mon, 2016-11-07 at 15:02 +0100, David Sterba wrote: >> I think adding a whole-file dedup mode to duperemove would be better >> (from user's POV) than writing a whole new tool > > What would IMO be really

Re: Announcing btrfs-dedupe

On martedì 8 novembre 2016 17:58:52 CET, James Pharaoh wrote: Yes, everything you have described here is something I intend to create, and might as well include in the tool itself. I'll add it to the roadmap ;-) Sounds good, but I have yet another feature request which is even more

Re: Announcing btrfs-dedupe

On 2016-11-08 11:57, Darrick J. Wong wrote: On Tue, Nov 08, 2016 at 08:26:02AM -0500, Austin S. Hemmelgarn wrote: On 2016-11-07 21:40, Christoph Anton Mitterer wrote: On Mon, 2016-11-07 at 15:02 +0100, David Sterba wrote: I think adding a whole-file dedup mode to duperemove would be better

Re: Announcing btrfs-dedupe

Yes, everything you have described here is something I intend to create, and might as well include in the tool itself. I'll add it to the roadmap ;-) James On 08/11/16 17:57, Niccolò Belli wrote: On martedì 8 novembre 2016 12:38:48 CET, James Pharaoh wrote: You can't deduplicate a read-only

Re: Announcing btrfs-dedupe

On Tue, Nov 08, 2016 at 08:26:02AM -0500, Austin S. Hemmelgarn wrote: > On 2016-11-07 21:40, Christoph Anton Mitterer wrote: > >On Mon, 2016-11-07 at 15:02 +0100, David Sterba wrote: > >>I think adding a whole-file dedup mode to duperemove would be better > >>(from user's POV) than writing a whole

Re: Announcing btrfs-dedupe

On martedì 8 novembre 2016 12:38:48 CET, James Pharaoh wrote: You can't deduplicate a read-only snapshot, but you can create read-write snapshots from them, deduplicate those, and then recreate the read-only ones. This is what I've done. Since snapper creates hundreds of snapshots, isn't this

Re: btrfs support for filesystems >8TB on 32bit architectures

On Tue, Nov 08, 2016 at 09:17:43AM +0800, Qu Wenruo wrote: > > > At 11/08/2016 09:06 AM, Marc MERLIN wrote: > >On Tue, Nov 08, 2016 at 08:43:34AM +0800, Qu Wenruo wrote: > >>That's strange, balance is done completely in kernel space. > >> > >>Unless we're calling vfs_* function we won't go

Re: btrfs btree_ctree_super fault

On 11/08/2016 09:59 AM, Dave Jones wrote: On Sun, Nov 06, 2016 at 11:55:39AM -0500, Dave Jones wrote: > > > On Mon, Oct 31, 2016 at 01:44:55PM -0600, Chris Mason wrote: > > On Mon, Oct 31, 2016 at 12:35:16PM -0700, Linus Torvalds wrote: > > >On Mon, Oct 31, 2016 at 11:55 AM, Dave Jones

Re: btrfs btree_ctree_super fault

On Sun, Nov 06, 2016 at 11:55:39AM -0500, Dave Jones wrote: > > > On Mon, Oct 31, 2016 at 01:44:55PM -0600, Chris Mason wrote: > > On Mon, Oct 31, 2016 at 12:35:16PM -0700, Linus Torvalds wrote: > > >On Mon, Oct 31, 2016 at 11:55 AM, Dave Jones > wrote: > >

Re: [PATCH v2] btrfs: make block group flags in balance printks human-readable

On 11/07/16 22:40, Adam Borowski wrote: > They're not even documented anywhere, letting users with no recourse but > to RTFS. It's no big burden to output the bitfield as words. > > Also, display unknown flags as hex. > > Signed-off-by: Adam Borowski [..] > > /* > + *

Re: Announcing btrfs-dedupe

On 2016-11-07 21:40, Christoph Anton Mitterer wrote: On Mon, 2016-11-07 at 15:02 +0100, David Sterba wrote: I think adding a whole-file dedup mode to duperemove would be better (from user's POV) than writing a whole new tool What would IMO be really good from a user's POV was, if one of the

Re: [PATCH v2 0/5] Make btrfs-progs really compatible with any kernel version

Hi David, This patch isn't integrated, any idea why ? Just a note, if it matters, this set has already been integrated into Oracle Linux. Thanks, Anand On 11/23/15 20:56, Anand Jain wrote: Btrfs-progs is a tool for the btrfs kernel and we hope latest btrfs-progs be compatible w any set of

Re: [PATCH] btrfs-progs: send-test: add checking of clone-src option

On Tue, Nov 08, 2016 at 08:53:04AM +0900, Tsutomu Itoh wrote: > On 2016/11/08 0:16, David Sterba wrote: > > On Fri, Nov 04, 2016 at 05:35:18PM +0900, Tsutomu Itoh wrote: > >> +before_size=`ls -l "$here"/send.stream.before | awk '{print $5}'` > >> +after_size=`ls -l "$here"/send.stream.after | awk

Re: [PATCH] btrfs: Introduce device pool sysfs attributes

David, Any comments on this patch ? It would need rebase though, I shall do it based on your inputs. sysfs was most challenging to get correct. (There are defunct procfs and ioctl based patches as well which does the same thing). Thanks, Anand On 10/10/15 01:46, Anand Jain wrote: This

Re: [PATCH] btrfs: Introduce device pool sysfs attributes

David, Any comments on this patch ? It would need rebase though, I shall do it based on your inputs. sysfs was most challenging to get correct. (There are defunct procfs and ioctl based patches as well which does the same thing). Thanks, Anand On 10/10/15 01:46, Anand Jain wrote: This

Re: [PATCH 02/13] btrfs: Do per-chunk check for mount time check

Hi David, This and its related patches 1/13..5/13 provides a good interim workaround to the regression caused by the patch commit 95669976bd7d30ae265db938ecb46a6b7f8cb893 Author: Miao Xie Date: Thu Jul 24 11:37:14 2014 +0800 Btrfs: don't consider the missing

Re: [PATCH 12/13] btrfs: check device for critical errors and mark failed

This patch is independent of the hot-space as such. 11/13 introduced new device state. This patch 12/13 brings the device to those new device states up on errors. Would like to know your opinion on this as well. Thanks, Anand On 05/10/16 22:09, Anand Jain wrote: From: Anand Jain

Re: [PATCH 11/13] btrfs: introduce device dynamic state transition to offline or failed

Hi David, This patch isn't integrated, so when there is flank/failing device, btrfs would never stop sending new read/write to the device. Would want to know your opinion if that the right/final behavior ? Thanks, Anand On 05/10/16 22:09, Anand Jain wrote: From: Anand Jain

Re: Announcing btrfs-dedupe

On 08/11/16 12:06, Niccolò Belli wrote: Nice, you should probably update the btrfs wiki as well, because there is no mention of btrfs-dedupe. I am planning to, I had to apply for an account, which has now been approved. First question, why this name? Don't you plan to support xfs as well?

Re: [PATCH v3 2/3] btrfs: Add trace point for qgroup reserved space

On Tue, 8 Nov 2016 10:20:43 +0800 Qu Wenruo wrote: > Introduce the following trace points: > qgroup_update_reserve > qgroup_meta_reserve > > These trace points are handy to trace qgroup reserve space related > problems. > > Signed-off-by: Qu Wenruo

Re: Announcing btrfs-dedupe

Nice, you should probably update the btrfs wiki as well, because there is no mention of btrfs-dedupe. First question, why this name? Don't you plan to support xfs as well? Second question, I'm trying deduplication tools for the very first time and I still have to figure out how to handle

Re: [PATCH] fstests: btrfs: Check false ENOSPC bug caused by incorrect metadata reserve

On Tue, Nov 08, 2016 at 05:15:15PM +0800, Qu Wenruo wrote: > Due to the fact that btrfs uses different max extent size for > compressed and non-compressed write, it calculates metadata reservation > incorrectly. > > This can leads to false ENOSPC alert for compressed write. > > This test case

[PATCH] btrfs: limit the number of asynchronous delalloc pages to reasonable value

The current limit of number of asynchronous delalloc pages is (10 * SZ_1M). For 4K page, the total ram bytes would be 40G, very big value, I think in most cases, this limit will not work, here I set limit of the number of asynchronous delalloc pages to SZ_1M(4GB ram bytes). Signed-off-by: Wang

[PATCH] fstests: btrfs: Check false ENOSPC bug caused by incorrect metadata reserve

Due to the fact that btrfs uses different max extent size for compressed and non-compressed write, it calculates metadata reservation incorrectly. This can leads to false ENOSPC alert for compressed write. This test case will check it by using small fs and large nodesize, and do parallel