[PATCH] btrfs-progs: qgroup: Fix a bug that fails to skip rescan running case

[Qu Wenruo]

Commit 6bdf962fe35a8648d(btrfs-progs: Read qgroup status for qgroup
verify) will read qgroup status, and then use it to skip qgroup
reporting.

However since the rescan_running/inconsistent flags are only 1 bit long,
while qgroup flags & BTRFS_QGROUP_FLAGS returns value longer than 1bit,
it doesn't work.

Fix it by doing double negation on (flags & BTRFS_QGROUP_FLAGS) to
convert it to either 1 or 0.

Signed-off-by: Qu Wenruo 
---
 qgroup-verify.c | 9 +++--
 1 file changed, 7 insertions(+), 2 deletions(-)

diff --git a/qgroup-verify.c b/qgroup-verify.c
index 1a0d38c..86dcd6d 100644
--- a/qgroup-verify.c
+++ b/qgroup-verify.c
@@ -711,8 +711,13 @@ static void read_qgroup_status(struct btrfs_path *path,
status_item = btrfs_item_ptr(path->nodes[0], path->slots[0],
 struct btrfs_qgroup_status_item);
flags = btrfs_qgroup_status_flags(path->nodes[0], status_item);
-   counts->qgroup_inconsist = flags & 
BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT;
-   counts->rescan_running = flags & BTRFS_QGROUP_STATUS_FLAG_RESCAN;
+   /*
+* Since qgroup_inconsist/rescan_running is just one bit,
+* assign value directly won't work.
+*/
+   counts->qgroup_inconsist = !!(flags &
+   BTRFS_QGROUP_STATUS_FLAG_INCONSISTENT);
+   counts->rescan_running = !!(flags & BTRFS_QGROUP_STATUS_FLAG_RESCAN);
 }
 
 static int load_quota_info(struct btrfs_fs_info *info)
-- 
2.8.3



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Process is blocked for more than 120 seconds

[Dmitry Katsubo]

On 2015-11-11 12:38, Dmitry Katsubo wrote:
> On 2015-11-09 14:25, Austin S Hemmelgarn wrote:
>> On 2015-11-07 07:22, Dmitry Katsubo wrote:
>>> Hi everyone,
>>>
>>> I have noticed the following in the log. The system continues to run,
>>> but I am not sure for how long it will be stable. Should I start
>>> worrying? Thanks in advance for the opinion.
>>>
>> This just means that a process was stuck in the D state (uninterruptible
>> I/O sleep) for more than 120 seconds.  Depending on a number of factors,
>> this happening could mean:
>> 1. Absolutely nothing (if you have low-powered or older hardware, for
>> example, I get these regularly on a first generation Raspberry Pi if I
>> don't increase the timeout significantly)
>> 2. The program is doing a very large chunk of I/O (usually with the
>> O_DIRECT flag, although this probably isn't the case here)
>> 3. There's a bug in the blocked program (this is rarely the case when
>> this type of thing happens)
>> 4. There's a bug in the kernel (which is why this dumps a stack trace)
>> 5. The filesystem itself is messed up somehow, and the kernel isn't
>> handling it properly (technically a bug, but a more specific case of it).
>> 6. You're hardware is misbehaving, failing, or experienced a transient
>> error.
>>
>> Assuming you can rule out possibilities 1 and 6, I think that 4 is the
>> most likely cause, as all of the listed programs (I'm assuming that
>> 'master' is from postfix) are relatively well audited, and all of them
>> hit this at the same time.
>>
>> For what it's worth, if you want you can do:
>> echo 0 > /proc/sys/kernel/hung_task_timeout_secs
>> like the message says to stop these from appearing in the future, or use
>> some arbitrary number to change the timeout before these messages appear
>> (I usually use at least 150 on production systems, and more often 300,
>> although on something like a Raspberry Pi I often use timeouts as high
>> as 1800 seconds).
> 
> Thanks for comments, Austin.
> 
> The system is "normal" PC, running Intel Core 2 Duo Mobile @1.66GHz.
> "master" is indeed a postfix process.
> 
> I haven't seen anything like that when I was on 3.16 kernel, but after I
> have upgraded to 4.2.3, I caught that message. I/O and CPU load are
> usually low, but it could be (6) from your list, as the system is
> generally very old (5+ years).
> 
> As the problem appeared only once for passed 15 days, I think it is just
> a transient error. Thanks for clarifying the possible reasons.

The problem (rarely) re-occurs. It does not happen on XFS filesystem (root)
but only on btrfs. I will increase timeout up to 300 and see what happens.

=== cut dmesg ===
INFO: task fail2ban-server:1747 blocked for more than 120 seconds.
  Tainted: GW   4.4.0-1-rt-686-pae #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
fail2ban-server D 001f 0  1747  1 0x
 f1ca1bc0 00200086 f2d24190 001f  f79ca4c0 f3d21bc0 
 c1168726 f1ca1bc0 e9152000  e9151d8c c156075f c0d25a90 f1ca1bc0
 e9151db4 c1561ed4   f1ca1bc0 0002 eab98940 c0d25a90
Call Trace:
 [] ? __filemap_fdatawrite_range+0xb6/0xf0
 [] ? schedule+0x3f/0xd0
 [] ? __rt_mutex_slowlock+0x74/0x140
 [] ? rt_mutex_slowlock+0xf3/0x250
 [] ? btrfs_write_marked_extents+0xae/0x190 [btrfs]
 [] ? rt_mutex_lock+0x45/0x50
 [] ? btrfs_sync_log+0x1d5/0x9a0 [btrfs]
 [] ? pin_current_cpu+0x71/0x1a0
 [] ? preempt_count_add+0x8a/0xb0
 [] ? unpin_current_cpu+0x13/0x70
 [] ? btrfs_sync_file+0x3ce/0x410 [btrfs]
 [] ? start_ordered_ops+0x40/0x40 [btrfs]
 [] ? vfs_fsync_range+0x47/0xb0
 [] ? do_fsync+0x3c/0x60
 [] ? SyS_fdatasync+0x15/0x20
 [] ? do_fast_syscall_32+0x8d/0x150
 [] ? sysenter_past_esp+0x3d/0x61
 [] ? pci_mmcfg_check_reserved+0x90/0xb0
INFO: task cleanup:2093 blocked for more than 120 seconds.
  Tainted: GW   4.4.0-1-rt-686-pae #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this message.
cleanup D d0f5de0c 0  2093  28135 0x
 eab99bc0 00200086 c1836760 d0f5de0c 0002 f79ca4c0 f3d21bc0 
 0001 eab99bc0 d0f5e000 f325d2c4 d0f5ddfc c156075f f325d000 0088
 d0f5de30 f8c72f36 0310 f325d290  eab99bc0 c10afd70 f325d2dc
Call Trace:
 [] ? schedule+0x3f/0xd0
 [] ? wait_log_commit+0xc6/0xf0 [btrfs]
 [] ? wake_atomic_t_function+0x70/0x70
 [] ? btrfs_sync_log+0x36a/0x9a0 [btrfs]
 [] ? pin_current_cpu+0x71/0x1a0
 [] ? preempt_count_add+0x8a/0xb0
 [] ? unpin_current_cpu+0x13/0x70
 [] ? btrfs_log_dentry_safe+0x64/0x70 [btrfs]
 [] ? btrfs_sync_file+0x3ce/0x410 [btrfs]
 [] ? do_sys_truncate+0xb0/0xb0
 [] ? start_ordered_ops+0x40/0x40 [btrfs]
 [] ? vfs_fsync_range+0x47/0xb0
 [] ? do_fsync+0x3c/0x60
 [] ? SyS_fsync+0x12/0x20
 [] ? do_fast_syscall_32+0x8d/0x150
 [] ? sysenter_past_esp+0x3d/0x61
INFO: task lmtpd:2101 blocked for more than 120 seconds.
  Tainted: GW   4.4.0-1-rt-686-pae #1
"echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 

Re: [PATCH] Btrfs: track transid for delayed ref flushing

[Liu Bo]

On Wed, Apr 27, 2016 at 09:59:38AM -0400, Chris Mason wrote:
> On Mon, Apr 11, 2016 at 05:37:40PM -0400, Josef Bacik wrote:
> > Using the offwakecputime bpf script I noticed most of our time was spent 
> > waiting
> > on the delayed ref throttling.  This is what is supposed to happen, but
> > sometimes the transaction can commit and then we're waiting for throttling 
> > that
> > doesn't matter anymore.  So change this stuff to be a little smarter by 
> > tracking
> > the transid we were in when we initiated the throttling.  If the 
> > transaction we
> > get is different then we can just bail out.  This resulted in a 50% speedup 
> > in
> > my fs_mark test, and reduced the amount of time spent throttling by 60 
> > seconds
> > over the entire run (which is about 30 minutes).  Thanks,
> > 
> > Signed-off-by: Josef Bacik 
> > ---
> >  fs/btrfs/ctree.h   |  2 +-
> >  fs/btrfs/extent-tree.c | 15 ---
> >  fs/btrfs/inode.c   |  1 +
> >  fs/btrfs/transaction.c |  3 ++-
> >  4 files changed, 16 insertions(+), 5 deletions(-)
> > 
> > diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> > index 55a24c5..4222936 100644
> > --- a/fs/btrfs/ctree.h
> > +++ b/fs/btrfs/ctree.h
> > @@ -3505,7 +3505,7 @@ void btrfs_put_block_group(struct 
> > btrfs_block_group_cache *cache);
> >  int btrfs_run_delayed_refs(struct btrfs_trans_handle *trans,
> >struct btrfs_root *root, unsigned long count);
> >  int btrfs_async_run_delayed_refs(struct btrfs_root *root,
> > -unsigned long count, int wait);
> > +unsigned long count, u64 transid, int wait);
> >  int btrfs_lookup_data_extent(struct btrfs_root *root, u64 start, u64 len);
> >  int btrfs_lookup_extent_info(struct btrfs_trans_handle *trans,
> >  struct btrfs_root *root, u64 bytenr,
> > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
> > index 4b5a517..f23f426 100644
> > --- a/fs/btrfs/extent-tree.c
> > +++ b/fs/btrfs/extent-tree.c
> > @@ -2839,6 +2839,7 @@ int btrfs_should_throttle_delayed_refs(struct 
> > btrfs_trans_handle *trans,
> >  
> >  struct async_delayed_refs {
> > struct btrfs_root *root;
> > +   u64 transid;
> > int count;
> > int error;
> > int sync;
> > @@ -2854,9 +2855,16 @@ static void delayed_ref_async_start(struct 
> > btrfs_work *work)
> >  
> > async = container_of(work, struct async_delayed_refs, work);
> >  
> > -   trans = btrfs_join_transaction(async->root);
> > +   trans = btrfs_attach_transaction(async->root);
> > if (IS_ERR(trans)) {
> > -   async->error = PTR_ERR(trans);
> > +   if (PTR_ERR(trans) != -ENOENT)
> > +   async->error = PTR_ERR(trans);
> > +   goto done;
> > +   }
> 
> This ends up deadlocking because btrfs_attach_transaction waits in ways
> that join does not.  The differences between these two are really
> subtle, and we manage to make this mistake every year or so.
> 
> Subject: [PATCH] btrfs: fix deadlock in delayed_ref_async_start
> 
> "Btrfs: track transid for delayed ref flushing" was deadlocking on
> btrfs_attach_transaction because its not safe to call from the async
> delayed ref start code.  This commit brings back btrfs_join_transaction
> instead and checks for a blocked commit.

Are we going to take this patch?

Thanks,

-liubo

> 
> Signed-off-by: Josef Bacik 
> Signed-off-by: Chris Mason 
> ---
>  fs/btrfs/extent-tree.c | 20 +++-
>  1 file changed, 11 insertions(+), 9 deletions(-)
> 
> diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c
> index 6ce5b6c..44da4ac 100644
> --- a/fs/btrfs/extent-tree.c
> +++ b/fs/btrfs/extent-tree.c
> @@ -2845,16 +2845,13 @@ static void delayed_ref_async_start(struct btrfs_work 
> *work)
>  
>   async = container_of(work, struct async_delayed_refs, work);
>  
> - trans = btrfs_attach_transaction(async->root);
> - if (IS_ERR(trans)) {
> - if (PTR_ERR(trans) != -ENOENT)
> - async->error = PTR_ERR(trans);
> + /* if the commit is already started, we don't need to wait here */
> + if (btrfs_transaction_blocked(async->root->fs_info))
>   goto done;
> - }
>  
> - /* Don't bother flushing if we got into a different transaction */
> - if (trans->transid != async->transid) {
> - btrfs_end_transaction(trans, async->root);
> + trans = btrfs_join_transaction(async->root);
> + if (IS_ERR(trans)) {
> + async->error = PTR_ERR(trans);
>   goto done;
>   }
>  
> @@ -2863,10 +2860,15 @@ static void delayed_ref_async_start(struct btrfs_work 
> *work)
>* wait on delayed refs
>*/
>   trans->sync = true;
> +
> + /* Don't bother flushing if we got into a different transaction */
> + if (trans->transid > async->transid)
> + goto end;
> +
>   ret = btrfs_run_delayed_refs(trans, async->root, 

Re: Replacing drives with larger ones in a 4 drive raid1

[boli]

>> So I was back to a 4-drive raid1, with 3x 6 TB drives and 1x 8 TB drive
>> (though that 8 TB drive had very little data on it). Then I tried to
>> "remove" (without "-r" this time) the 6 TB drive with the least amount
>> of data on it (one had 4.0 TiB, where the other two had 5.45 TiB each).
>> This failed after a few minutes because of "no space left on device".
>> 
>> […]
>> 
>> For now I avoided that by removing one of the other two (rather full) 6
>> TB drives at random, and this has been going on for the last 20 hours or
>> so. Thanks to running it in a screen I can check the progress this time
>> around, and it's doing its thing at ~41 MiB/s, or ~7 hours per TiB, on
>> average.
> 
> The ENOSPC errors are likely due to the fact that the raid1 allocator 
> needs _two_ devices with free space.  If your 6T devices get too full, 
> even if the 8T device is nearly empty, you'll run into ENOSPC, because 
> you have just one device with unallocated space and the raid1 allocator 
> needs two.

I see, now this makes total sense. Two of the 6 TB drives were almost 
completely full, at 5.45 used of 5.46 TiB capacity. Note to self: Maybe I 
should start using the --si option to make such a condition more obvious when 
mentally comparing to the advertised capacity of 6 TB (when comparing to to 
5.46 TiB would have been correct)

"remove"-ing one of these almost-full-drives did finish successfully, and a 
"replace" of the 3rd 6 TB drive onto a second 8 TB drive is currently in 
progress (at high speed).

> btrfs device usage should help diagnose this condition, with btrfs 
> filesystem show also showing the individual device space allocation but 
> not as much other information as usage will.

I had mostly been using btrfs filesystem usage, thanks for the reminder about 
device usage, which is easier to read in this case.

> If you run into this, you may just have to do the hardware yank and 
> replace-missing thing again, yanking a 6T and replacing with an 8T.  
> Don't forget the resize.  That should leave you with two devices with 
> free space and thus hopefully allow normal raid1 reallocation with a 
> device remove again.

Good to know. For now this doesn't seem necessary, even the other drive that 
was almost completely full before looks much better now at 4.8/5.46 TiB or 
5.27/6.0 TB in --si :), as some data was moved to the first 8 TB drive during 
the last "remove".

So far everything is looking good, thanks very much for the help everyone.

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: let super_stripesize match with sectorsize

[Liu Bo]

On Wed, Jun 15, 2016 at 03:50:17PM +0530, Chandan Rajendra wrote:
> On Wednesday, June 15, 2016 09:12:28 AM Chandan Rajendra wrote:
> > Hello Liu Bo,
> > 
> > We have to fix the following check in check_super() as well,
> > 
> >if (btrfs_super_stripesize(sb) != 4096) {
> > error("invalid stripesize %u", btrfs_super_stripesize(sb));
> > goto error_out;
> > }
> > 
> > i.e. btrfs_super_stripesize(sb) must be equal to
> > btrfs_super_sectorsize(sb).
> > 
> > However in btrfs-progs (mkfs.c to be precise) since we had stripesize
> > hardcoded to 4096, setting stripesize to the value of sectorsize in
> > mkfs.c will cause the following to occur when mkfs.btrfs is invoked for
> > devices with existing Btrfs filesystem instances,
> > 
> > NOTE: Assume we have changed the stripesize validation in btrfs-progs'
> > check_super() to,
> > 
> > if (btrfs_super_stripesize(sb) != btrfs_super_sectorsize(sb)) {
> > error("invalid stripesize %u", btrfs_super_stripesize(sb));
> > goto error_out;
> > }
> > 
> > 
> > main()
> >  for each device file passed as an argument,
> >test_dev_for_mkfs()
> >  check_mounted
> >check_mounted_where
> >  btrfs_scan_one_device
> >btrfs_read_dev_super
> >  check_super() call will fail for existing filesystems which
> > have stripesize set to 4k. All existing filesystem instances will fall into
> > this category.
> > 
> > This error value is pushed up the call stack and this causes the device to
> > not get added to the fs_devices_mnt list in check_mounted_where(). Hence we
> > would fail to correctly check the mount status of the multi-device btrfs
> > filesystems.
> > 
> 
> 
> We can end up in the following scenario,
> - /dev/loop0, /dev/loop1 and /dev/loop2 are mounted as a single
>   filesystem. The filesystem was created by an older version of mkfs.btrfs
>   which set stripesize to 4k. 
> - losetup -a
>/dev/loop0: [0030]:19477 (/root/disk-imgs/file-0.img)  
>/dev/loop1: [0030]:16577 (/root/disk-imgs/file-1.img)  
>/dev/loop2: [64770]:3423229 (/root/disk-imgs/file-2.img)  
> - /etc/mtab lists only /dev/loop0
> - "losetup /dev/loop4 /root/disk-imgs/file-1.img"
>The new mkfs.btrfs invoked as 'mkfs.btrfs -f /dev/loop4' succeeds even
>though /dev/loop1 has already been mounted and has
>/root/disk-imgs/file-1.img as its backing file.
> 
> So IMHO the only solution is to have the stripesize check in check_super() to
> allow both '4k' and 'sectorsize' as valid values i.e.
> 
> if ((btrfs_super_stripesize(sb) != 4096)
>   && (btrfs_super_stripesize(sb) != btrfs_super_sectorsize(sb))) {
> error("invalid stripesize %u", btrfs_super_stripesize(sb));
> goto error_out;
> }

That's a good one.

But if we go back to the original point, in the kernel side,
1. in open_ctree(), root->stripesize = btrfs_super_stripesize();

2. in find_free_extent(),
...
search_start = ALIGN(offset, root->stripesize);
...
3. in btrfs_alloc_tree_block(),
...
ret = btrfs_reserve_extent(..., , ...);
...
buf = btrfs_init_new_buffer(trans, root, ins.objectid, level);

4. in btrfs_init_new_buffer(),
...
buf = btrfs_find_create_tree_block(root, bytenr);
...

Because 'num_bytes' we pass to find_free_extent() is aligned to
sectorsize, the free space we can find is aligned to sectorsize,
which means 'offset' in '1. find_free_extent()' is aligned to sectorsize.

If our stripesize is larger than sectorsize, say 4 * sectorsize,
for data allocation it's fine while for metadata block allocations it's
not.  It is possible that when we allocate a new metadata block, we can
end up with an existing eb returned by '4. in btrfs_init_new_buffer()'.

PS: There is something wrong around '2. in find_free_extent()',
we only do alignment on offset, but for num_bytes, we don't do that,
so we may end up with a overlap with other data extents or metadata
blocks.

So I think we can just replace this ALING with a warning since the offset
returned by searching free space tree is aligned to 
block_group->full_stripe_len,
which is either sectorsize or BTRFS_STRIPE_LEN * nr_stripes (for
raid56), then we can just drop the check for stripesize everywhere.

Thanks,

-liubo
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH 2/2] btrfs-progs: btrfsck: verify qgroups above level 0

[Mark Fasheh]

At the moment we only check subvolume quota groups (level 0). With this
patch we can check groups above 0, thus verifying the entire qgroup
hierarchy on a file system.  The accounting portion of this patch is modeled
after the kernel - we are essentially reimplementing the 'quota rescan' case
here. Most other sections of this code went unchanged, in particular the
root counting works independently of the accounting.

Signed-off-by: Mark Fasheh 
---
 qgroup-verify.c | 305 ++--
 1 file changed, 251 insertions(+), 54 deletions(-)

diff --git a/qgroup-verify.c b/qgroup-verify.c
index 7b78504..4d2ea66 100644
--- a/qgroup-verify.c
+++ b/qgroup-verify.c
@@ -35,7 +35,8 @@
 /*#define QGROUP_VERIFY_DEBUG*/
 static unsigned long tot_extents_scanned = 0;
 
-static void add_bytes(u64 root_objectid, u64 num_bytes, int exclusive);
+struct qgroup_count;
+static struct qgroup_count *find_count(u64 qgroupid);
 
 struct qgroup_info {
u64 referenced;
@@ -54,6 +55,14 @@ struct qgroup_count {
struct qgroup_info info;
 
struct rb_node rb_node;
+
+   struct list_head groups;  /* parents when we are a child group */
+   struct list_head members; /* children when we are a parent
+group (not currently used but
+maintained to mirror kernel
+handling of qgroups) */
+
+   u64 cur_refcnt;
 };
 
 static struct counts_tree {
@@ -66,6 +75,39 @@ static struct counts_tree {
 static struct rb_root by_bytenr = RB_ROOT;
 
 /*
+ * glue structure to represent the relations between qgroups. Mirrored
+ * from kernel.
+ */
+struct btrfs_qgroup_list {
+   struct list_head next_group;
+   struct list_head next_member;
+   struct qgroup_count *group; /* Parent group */
+   struct qgroup_count *member;
+};
+
+/* allow us to reset ref counts during accounting without zeroing each group */
+static u64 qgroup_seq = 1ULL;
+
+static inline void update_cur_refcnt(struct qgroup_count *c)
+{
+   if (c->cur_refcnt < qgroup_seq)
+   c->cur_refcnt = qgroup_seq;
+   c->cur_refcnt += 1;
+}
+
+static inline u64 group_get_cur_refcnt(struct qgroup_count *c)
+{
+   if (c->cur_refcnt < qgroup_seq)
+   return 0;
+   return c->cur_refcnt - qgroup_seq;
+}
+
+static void inc_qgroup_seq(int root_count)
+{
+   qgroup_seq += root_count + 1;
+}
+
+/*
  * List of interior tree blocks. We walk this list after loading the
  * extent tree to resolve implied refs. For each interior node we'll
  * place a shared ref in the ref tree against each child object. This
@@ -296,9 +338,10 @@ static void find_parent_roots(struct ulist *roots, u64 
parent)
}
 
do {
-   if (ref->root)
-   ulist_add(roots, ref->root, 0, 0);
-   else
+   if (ref->root) {
+   if (is_fstree(ref->root))
+   ulist_add(roots, ref->root, 0, 0);
+   } else
find_parent_roots(roots, ref->parent);
 
node = rb_next(node);
@@ -307,6 +350,116 @@ static void find_parent_roots(struct ulist *roots, u64 
parent)
} while (node && ref->bytenr == parent);
 }
 
+static int account_one_extent(struct ulist *roots, u64 bytenr, u64 num_bytes)
+{
+   int ret;
+   u64 id, nr_roots, nr_refs;
+   struct qgroup_count *count;
+   struct ulist *counts = ulist_alloc(0);
+   struct ulist *tmp = ulist_alloc(0);
+   struct ulist_iterator uiter;
+   struct ulist_iterator tmp_uiter;
+   struct ulist_node *unode;
+   struct ulist_node *tmp_unode;
+   struct btrfs_qgroup_list *glist;
+
+   if (!counts || !tmp) {
+   ulist_free(counts);
+   ulist_free(tmp);
+   return ENOMEM;
+   }
+
+   ULIST_ITER_INIT();
+   while ((unode = ulist_next(roots, ))) {
+   BUG_ON(unode->val == 0ULL);
+
+   /*
+* For each root, find their corresponding tracking group and
+* add it to our qgroups list.
+*/
+   count = find_count(unode->val);
+   if (!count)
+   continue;
+
+   BUG_ON(!is_fstree(unode->val));
+   ret = ulist_add(counts, count->qgroupid, ptr_to_u64(count), 0);
+   if (ret < 0)
+   goto out;
+
+   /*
+* Now we look for parents (and parents of
+* those...). Use a tmp ulist here to avoid re-walking
+* (and re-incrementing) our already added items on every
+* loop iteration.
+*/
+   ulist_reinit(tmp);
+   ret = ulist_add(tmp, count->qgroupid, ptr_to_u64(count), 0);
+   if (ret < 0)
+   goto out;
+
+   

[PATCH 0/2] btrfs-progs: qgroup verification update

[Mark Fasheh]

Hi David,

The following two patches update the qgroup verification code in btrfsck to
understand entire qgroup hierarchies.  The first patch is a simple bugfix
for some leaked objects and can be taken separately if you like.  The 2nd
patch implements the actual verification update.

If you prefer to pull via git there is a branch at:

https://github.com/markfasheh/btrfs-progs-patches qgroup-hierarchy

Review and comments are welcome.

Thanks,
--Mark
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: add missing check for writeback errors on fsync

[Liu Bo]

On Tue, Jun 14, 2016 at 08:50:22PM +0100, fdman...@kernel.org wrote:
> From: Filipe Manana 
> 
> When we start an fsync we start ordered extents for all delalloc ranges.
> However before attempting to log the inode, we only wait for those ordered
> extents if we are not doing a full sync (bit BTRFS_INODE_NEEDS_FULL_SYNC
> is set in the inode's flags). This means that if an ordered extent
> completes with an IO error before we check if we can skip logging the
> inode, we will not catch and report the IO error to user space. This is
> because on an IO error, when the ordered extent completes we do not
> update the inode, so if the inode was not previously updated by the
> current transaction we end up not logging it through calls to fsync and
> therefore not check its mapping flags for the presence of IO errors.
> 
> Fix this by checking for errors in the flags of the inode's mapping when
> we notice we can skip logging the inode.
> 
> This caused sporadic failures in the test generic/331 (which explicitly
> tests for IO errors during an fsync call).

Good catch.

Reviewed-by: Liu Bo 

Thanks,

-liubo

> 
> Signed-off-by: Filipe Manana 
> ---
>  fs/btrfs/file.c | 8 
>  1 file changed, 8 insertions(+)
> 
> diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
> index 159a934..6c6863a 100644
> --- a/fs/btrfs/file.c
> +++ b/fs/btrfs/file.c
> @@ -2039,6 +2039,14 @@ int btrfs_sync_file(struct file *file, loff_t start, 
> loff_t end, int datasync)
>*/
>   clear_bit(BTRFS_INODE_NEEDS_FULL_SYNC,
> _I(inode)->runtime_flags);
> + /*
> +  * An ordered extent might have started before and completed
> +  * already with io errors, in which case the inode was not
> +  * updated and we end up here. So check the inode's mapping
> +  * flags for any errors that might have happened while doing
> +  * writeback of file data.
> +  */
> + ret = btrfs_inode_check_errors(inode);
>   inode_unlock(inode);
>   goto out;
>   }
> -- 
> 2.7.0.rc3
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BUG: unable to mount btrfs on ppc64 starting from v4.7-rc3 kernel

[David Sterba]

On Tue, Jun 14, 2016 at 11:24:04AM -0700, Liu Bo wrote:
> > [ 1910.048650] BTRFS: device fsid 06813ff6-d585-4c54-b4df-b7d6920d27ba 
> > devid 1 transid 3 /dev/vda3
> > [ 1913.152085] BTRFS error (device vda3): invalid stripesize 4096
> > [ 1913.154349] BTRFS error (device vda3): superblock contains fatal errors
> > [ 1913.200300] BTRFS: open_ctree failed
> 
> Ah, that's right, we need to update btrfs-progs to set super_stripesize to
> sectorsize.
> 
> In mkfs.c we have,
> 
> {
>   u32 sectorsize = 4096;
>   u32 stripesize = 4096;
>   ...
>   sectorsize = max(sectorsize, (u32)sysconf(_SC_PAGESIZE));
>   ...
>   mkfs_cfg.sectorsize = sectorsize;
>   mkfs_cfg.stripesize = stripesize;
> 
>   ret = make_btrfs(fd, _cfg, NULL);
>   ...
> }

Yeah we need to fix progs and allow the 4k value in kernel, Chandan will
send a patch.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Balance fails with unallocated diskspace

[Duncan]

Nisse Karlsson posted on Wed, 15 Jun 2016 11:58:36 +0200 as excerpted:

> Since I have about 1.5TB unallocated space it's my understanding that
> this shouldn't be a problem, or am I completely wrong?

Quick partial answer...

It's not just unallocated space in total that matters, but also the 
number of devices with unallocated space.  To write raid10 chunks 
requires four devices.  To write raid1 chunks requires two.

However, your usage output does indicate several hundred gigs of 
unallocated space on multiple devices, so convert to raid1 shouldn't be 
ENOSPCing...  That it is would indicate a bug.  There have been various 
bugs of this sort, but I'm just a list regular, not a dev, and I'm not 
sure of the current status, whether any such bugs are currently known or 
not.

You don't mention kernel version in your post and I don't have time to 
look up the bug ATM, but you might try the latest 4.1 and 4.4 LTS kernels 
as well as the latest 4.6 and 4.7-rc kernels to see if it makes a 
difference.  If it's a problem with 4.6 and 4.7-rc then it becomes 
interesting, and if it works with 4.4 or 4.1, even more so.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Balance fails with unallocated diskspace

[Nisse Karlsson]

I've now tried mounting with nospace_cache as suggested in another thread,
no difference.

I also tried remounting with enospc_debug, but that didn't give me more info
than before
BTRFS info (device sdb): 523 enospc errors during balance

//Nisse


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cannot balance FS (No space left on device)

[E V]

In my experience phantom ENOSPC messages are frequently due to the
free space cache being corrupt. Mounting with nospace_cache or
space_cache=v2 may help.

On Wed, Jun 15, 2016 at 6:59 AM, ojab //  wrote:
> On Fri, Jun 10, 2016 at 2:58 PM, ojab //  wrote:
>> [Please CC me since I'm not subscribed to the list]
>
> So I'm still playing w/ btrfs and again I have 'No space left on
> device' during balance:
>>$ sudo /usr/bin/btrfs balance start --full-balance /mnt/xxx/
>>ERROR: error during balancing '/mnt/xxx/': No space left on device
>>There may be more info in syslog - try dmesg | tail
>>$ sudo dmesg -T  | grep BTRFS | tail
>>[Wed Jun 15 10:28:53 2016] BTRFS info (device sdc1): relocating block group 
>>13043037372416 flags 9
>>[Wed Jun 15 10:28:53 2016] BTRFS info (device sdc1): relocating block group 
>>13041963630592 flags 20
>>[Wed Jun 15 10:29:54 2016] BTRFS info (device sdc1): found 25155 extents
>>[Wed Jun 15 10:29:54 2016] BTRFS info (device sdc1): relocating block group 
>>13040889888768 flags 20
>>[Wed Jun 15 10:30:50 2016] BTRFS info (device sdc1): found 63700 extents
>>[Wed Jun 15 10:30:50 2016] BTRFS info (device sdc1): relocating block group 
>>13040856334336 flags 18
>>[Wed Jun 15 10:30:51 2016] BTRFS info (device sdc1): found 9 extents
>>[Wed Jun 15 10:30:52 2016] BTRFS info (device sdc1): relocating block group 
>>13039782592512 flags 20
>>[Wed Jun 15 10:32:08 2016] BTRFS info (device sdc1): found 61931 extents
>>[Wed Jun 15 10:32:08 2016] BTRFS info (device sdc1): 896 enospc errors during 
>>balance
>>$ sudo /usr/bin/btrfs balance start -dusage=75 /mnt/xxx/
>>Done, had to relocate 1 out of 901 chunks
>>$ sudo /usr/bin/btrfs balance start -dusage=76 /mnt/xxx/
>>ERROR: error during balancing '/mnt/xxx/': No space left on device
>>There may be more info in syslog - try dmesg | tail
>>$ sudo /usr/bin/btrfs fi usage /mnt/xxx/
>>Overall:
>>Device size:   1.98TiB
>>Device allocated:  1.85TiB
>>Device unallocated:135.06GiB
>>Device missing:0.00B
>>Used:  1.85TiB
>>Free (estimated):  135.68GiB  (min: 68.15GiB)
>>Data ratio:1.00
>>Metadata ratio:2.00
>>Global reserve:512.00MiB  (used: 0.00B)
>>
>>Data,RAID0: Size:1.84TiB, Used:1.84TiB
>>   /dev/sdb1   895.27GiB
>>   /dev/sdc1   895.27GiB
>>   /dev/sdd1   37.27GiB
>>   /dev/sdd2   37.27GiB
>>   /dev/sde1   11.27GiB
>>   /dev/sde2   11.27GiB
>>
>>Metadata,RAID1: Size:4.00GiB, Used:2.21GiB
>>   /dev/sdb1   2.00GiB
>>   /dev/sdc1   2.00GiB
>>   /dev/sde1   2.00GiB
>>   /dev/sde2   2.00GiB
>>
>>System,RAID1: Size:32.00MiB, Used:160.00KiB
>>   /dev/sde132.00MiB
>>   /dev/sde232.00MiB
>>
>>Unallocated:
>>   /dev/sdb1  34.25GiB
>>   /dev/sdc1  34.25GiB
>>   /dev/sdd1  1.11MiB
>>   /dev/sdd2  1.05MiB
>>   /dev/sde1  33.28GiB
>>   /dev/sde2  33.28GiB
>>$ sudo /usr/bin/btrfs fi show /mnt/xxx/
>>Label: none  uuid: 8a65465d-1a8c-4f80-abc6-c818c38567c3
>>   Total devices 6 FS bytes used 1.84TiB
>>   devid1 size 931.51GiB used 897.27GiB path /dev/sdc1
>>   devid2 size 931.51GiB used 897.27GiB path /dev/sdb1
>>   devid3 size 37.27GiB used 37.27GiB path /dev/sdd1
>>   devid4 size 37.27GiB used 37.27GiB path /dev/sdd2
>>   devid5 size 46.58GiB used 13.30GiB path /dev/sde1
>>   devid6 size 46.58GiB used 13.30GiB path /dev/sde2
>
> show_usage.py output can be found here:
> https://gist.github.com/ojab/a24ce373ce5bede001140c572879fce8
>
> Balance always fails with '896 enospc errors during balance' message
> in dmesg. I don't quite understand the logic: there is a plenty of
> space on four devices, why `btrfs` apparently trying to use sdd[0-1]
> drives, is it a bug or intended behaviour?
> What is the proper way of fixing such an issue in general, adding more
> devices and rebalancing? How can I determine how many devices should
> be added and their capacity?
>
> I'm still on 4.6.2 vanilla kernel and using btrfs-progs-4.6.
>
> //wbr ojab
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Cannot balance FS (No space left on device)

[ojab //]

On Fri, Jun 10, 2016 at 2:58 PM, ojab //  wrote:
> [Please CC me since I'm not subscribed to the list]

So I'm still playing w/ btrfs and again I have 'No space left on
device' during balance:
>$ sudo /usr/bin/btrfs balance start --full-balance /mnt/xxx/
>ERROR: error during balancing '/mnt/xxx/': No space left on device
>There may be more info in syslog - try dmesg | tail
>$ sudo dmesg -T  | grep BTRFS | tail
>[Wed Jun 15 10:28:53 2016] BTRFS info (device sdc1): relocating block group 
>13043037372416 flags 9
>[Wed Jun 15 10:28:53 2016] BTRFS info (device sdc1): relocating block group 
>13041963630592 flags 20
>[Wed Jun 15 10:29:54 2016] BTRFS info (device sdc1): found 25155 extents
>[Wed Jun 15 10:29:54 2016] BTRFS info (device sdc1): relocating block group 
>13040889888768 flags 20
>[Wed Jun 15 10:30:50 2016] BTRFS info (device sdc1): found 63700 extents
>[Wed Jun 15 10:30:50 2016] BTRFS info (device sdc1): relocating block group 
>13040856334336 flags 18
>[Wed Jun 15 10:30:51 2016] BTRFS info (device sdc1): found 9 extents
>[Wed Jun 15 10:30:52 2016] BTRFS info (device sdc1): relocating block group 
>13039782592512 flags 20
>[Wed Jun 15 10:32:08 2016] BTRFS info (device sdc1): found 61931 extents
>[Wed Jun 15 10:32:08 2016] BTRFS info (device sdc1): 896 enospc errors during 
>balance
>$ sudo /usr/bin/btrfs balance start -dusage=75 /mnt/xxx/
>Done, had to relocate 1 out of 901 chunks
>$ sudo /usr/bin/btrfs balance start -dusage=76 /mnt/xxx/
>ERROR: error during balancing '/mnt/xxx/': No space left on device
>There may be more info in syslog - try dmesg | tail
>$ sudo /usr/bin/btrfs fi usage /mnt/xxx/
>Overall:
>Device size:   1.98TiB
>Device allocated:  1.85TiB
>Device unallocated:135.06GiB
>Device missing:0.00B
>Used:  1.85TiB
>Free (estimated):  135.68GiB  (min: 68.15GiB)
>Data ratio:1.00
>Metadata ratio:2.00
>Global reserve:512.00MiB  (used: 0.00B)
>
>Data,RAID0: Size:1.84TiB, Used:1.84TiB
>   /dev/sdb1   895.27GiB
>   /dev/sdc1   895.27GiB
>   /dev/sdd1   37.27GiB
>   /dev/sdd2   37.27GiB
>   /dev/sde1   11.27GiB
>   /dev/sde2   11.27GiB
>
>Metadata,RAID1: Size:4.00GiB, Used:2.21GiB
>   /dev/sdb1   2.00GiB
>   /dev/sdc1   2.00GiB
>   /dev/sde1   2.00GiB
>   /dev/sde2   2.00GiB
>
>System,RAID1: Size:32.00MiB, Used:160.00KiB
>   /dev/sde132.00MiB
>   /dev/sde232.00MiB
>
>Unallocated:
>   /dev/sdb1  34.25GiB
>   /dev/sdc1  34.25GiB
>   /dev/sdd1  1.11MiB
>   /dev/sdd2  1.05MiB
>   /dev/sde1  33.28GiB
>   /dev/sde2  33.28GiB
>$ sudo /usr/bin/btrfs fi show /mnt/xxx/
>Label: none  uuid: 8a65465d-1a8c-4f80-abc6-c818c38567c3
>   Total devices 6 FS bytes used 1.84TiB
>   devid1 size 931.51GiB used 897.27GiB path /dev/sdc1
>   devid2 size 931.51GiB used 897.27GiB path /dev/sdb1
>   devid3 size 37.27GiB used 37.27GiB path /dev/sdd1
>   devid4 size 37.27GiB used 37.27GiB path /dev/sdd2
>   devid5 size 46.58GiB used 13.30GiB path /dev/sde1
>   devid6 size 46.58GiB used 13.30GiB path /dev/sde2

show_usage.py output can be found here:
https://gist.github.com/ojab/a24ce373ce5bede001140c572879fce8

Balance always fails with '896 enospc errors during balance' message
in dmesg. I don't quite understand the logic: there is a plenty of
space on four devices, why `btrfs` apparently trying to use sdd[0-1]
drives, is it a bug or intended behaviour?
What is the proper way of fixing such an issue in general, adding more
devices and rebalancing? How can I determine how many devices should
be added and their capacity?

I'm still on 4.6.2 vanilla kernel and using btrfs-progs-4.6.

//wbr ojab
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] Btrfs: add missing check for writeback errors on fsync

[fdmanana]

From: Filipe Manana 

When we start an fsync we start ordered extents for all delalloc ranges.
However before attempting to log the inode, we only wait for those ordered
extents if we are not doing a full sync (bit BTRFS_INODE_NEEDS_FULL_SYNC
is set in the inode's flags). This means that if an ordered extent
completes with an IO error before we check if we can skip logging the
inode, we will not catch and report the IO error to user space. This is
because on an IO error, when the ordered extent completes we do not
update the inode, so if the inode was not previously updated by the
current transaction we end up not logging it through calls to fsync and
therefore not check its mapping flags for the presence of IO errors.

Fix this by checking for errors in the flags of the inode's mapping when
we notice we can skip logging the inode.

This caused sporadic failures in the test generic/331 (which explicitly
tests for IO errors during an fsync call).

Signed-off-by: Filipe Manana 
---
 fs/btrfs/file.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/fs/btrfs/file.c b/fs/btrfs/file.c
index 159a934..6c6863a 100644
--- a/fs/btrfs/file.c
+++ b/fs/btrfs/file.c
@@ -2039,6 +2039,14 @@ int btrfs_sync_file(struct file *file, loff_t start, 
loff_t end, int datasync)
 */
clear_bit(BTRFS_INODE_NEEDS_FULL_SYNC,
  _I(inode)->runtime_flags);
+   /*
+* An ordered extent might have started before and completed
+* already with io errors, in which case the inode was not
+* updated and we end up here. So check the inode's mapping
+* flags for any errors that might have happened while doing
+* writeback of file data.
+*/
+   ret = btrfs_inode_check_errors(inode);
inode_unlock(inode);
goto out;
}
-- 
2.7.0.rc3

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] Btrfs: let super_stripesize match with sectorsize

[Chandan Rajendra]

On Wednesday, June 15, 2016 09:12:28 AM Chandan Rajendra wrote:
> Hello Liu Bo,
> 
> We have to fix the following check in check_super() as well,
> 
>if (btrfs_super_stripesize(sb) != 4096) {
> error("invalid stripesize %u", btrfs_super_stripesize(sb));
> goto error_out;
> }
> 
> i.e. btrfs_super_stripesize(sb) must be equal to
> btrfs_super_sectorsize(sb).
> 
> However in btrfs-progs (mkfs.c to be precise) since we had stripesize
> hardcoded to 4096, setting stripesize to the value of sectorsize in
> mkfs.c will cause the following to occur when mkfs.btrfs is invoked for
> devices with existing Btrfs filesystem instances,
> 
> NOTE: Assume we have changed the stripesize validation in btrfs-progs'
> check_super() to,
> 
> if (btrfs_super_stripesize(sb) != btrfs_super_sectorsize(sb)) {
> error("invalid stripesize %u", btrfs_super_stripesize(sb));
> goto error_out;
> }
> 
> 
> main()
>  for each device file passed as an argument,
>test_dev_for_mkfs()
>  check_mounted
>check_mounted_where
>  btrfs_scan_one_device
>btrfs_read_dev_super
>  check_super() call will fail for existing filesystems which
> have stripesize set to 4k. All existing filesystem instances will fall into
> this category.
> 
> This error value is pushed up the call stack and this causes the device to
> not get added to the fs_devices_mnt list in check_mounted_where(). Hence we
> would fail to correctly check the mount status of the multi-device btrfs
> filesystems.
> 


We can end up in the following scenario,
- /dev/loop0, /dev/loop1 and /dev/loop2 are mounted as a single
  filesystem. The filesystem was created by an older version of mkfs.btrfs
  which set stripesize to 4k. 
- losetup -a
   /dev/loop0: [0030]:19477 (/root/disk-imgs/file-0.img)  
   /dev/loop1: [0030]:16577 (/root/disk-imgs/file-1.img)  
   /dev/loop2: [64770]:3423229 (/root/disk-imgs/file-2.img)  
- /etc/mtab lists only /dev/loop0
- "losetup /dev/loop4 /root/disk-imgs/file-1.img"
   The new mkfs.btrfs invoked as 'mkfs.btrfs -f /dev/loop4' succeeds even
   though /dev/loop1 has already been mounted and has
   /root/disk-imgs/file-1.img as its backing file.

So IMHO the only solution is to have the stripesize check in check_super() to
allow both '4k' and 'sectorsize' as valid values i.e.

if ((btrfs_super_stripesize(sb) != 4096)
&& (btrfs_super_stripesize(sb) != btrfs_super_sectorsize(sb))) {
error("invalid stripesize %u", btrfs_super_stripesize(sb));
goto error_out;
}
-- 
chandan

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs: avoid blocking open_ctree from cleaner_kthread

[David Sterba]

On Sun, Jun 12, 2016 at 11:39:58PM -0400, Zygo Blaxell wrote:
> This fixes a problem introduced in commit 
> 2f3165ecf103599f82bf0ea254039db335fb5005
> "btrfs: don't force mounts to wait for cleaner_kthread to delete one or more 
> subvolumes".
> 
> open_ctree eventually calls btrfs_replay_log which in turn calls
> btrfs_commit_super which tries to lock the cleaner_mutex, causing a
> recursive mutex deadlock during mount.
> 
> Instead of playing whack-a-mole trying to keep up with all the
> functions that may want to lock cleaner_mutex, put all the cleaner_mutex
> lockers back where they were, and attack the problem more directly:
> keep cleaner_kthread asleep until the filesystem is mounted.

This approach looks good to me.

> When filesystems are mounted read-only and later remounted read-write,
> open_ctree did not set fs_info->open and neither does anything else.
> Set this flag in btrfs_remount so that neither btrfs_delete_unused_bgs
> nor cleaner_kthread get confused by the common case of "/" filesystem
> read-only mount followed by read-write remount.
> 
> Signed-off-by: Zygo Blaxell 

Reviewed-by: David Sterba 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 6/6] fstests: btrfs: test RAID5 device reappear and balance

[Anand Jain]

On 06/12/2016 01:08 PM, Eryu Guan wrote:

On Tue, May 17, 2016 at 10:32:10PM +0800, Anand Jain wrote:

The test does the following:
Initialize a RAID5 with some data

Re-mount RAID5 degraded with _dev3_ missing and write data.
Save md5sum checkpoint1

Re-mount healthy RAID5

Let balance fix degraded blocks.
Save md5sum checkpoint2

Re-mount RAID1 degraded now with _dev1_ missing.
Save md5sum checkpoint3

Verify if all three md5sum matches

Signed-off-by: Anand Jain 


All comments for 5/6 apply here for 6/6 :)


yep. Taken care in v2. Thanks for all the review comments.

Anand



Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 6/6] fstests: btrfs: test RAID5 device reappear and balance

[Anand Jain]

From: Anand Jain 

The test does the following:
Initialize a RAID5 with some data

Re-mount RAID5 degraded with _dev3_ missing and write data.
Save md5sum checkpoint1

Re-mount healthy RAID5

Let balance fix degraded blocks.
Save md5sum checkpoint2

Re-mount RAID1 degraded now with _dev1_ missing.
Save md5sum checkpoint3

Verify if all three md5sum matches

Signed-off-by: Anand Jain 
---
v2:
  add tmp= and its rm
  add comments to why _reload_btrfs_ko is used
  add missing put and test_mount at notrun exit
  use echo instead of _fail when checkpoints are checked
  .out updated to remove Silence..
 tests/btrfs/124 | 183 
 tests/btrfs/124.out |   7 ++
 tests/btrfs/group   |   1 +
 3 files changed, 191 insertions(+)
 create mode 100755 tests/btrfs/124
 create mode 100644 tests/btrfs/124.out

diff --git a/tests/btrfs/124 b/tests/btrfs/124
new file mode 100755
index ..8ef7c62c4380
--- /dev/null
+++ b/tests/btrfs/124
@@ -0,0 +1,183 @@
+#! /bin/bash
+# FS QA Test 124
+#
+# This test verify if the reconstructed data on the RAID5 is good.
+# Steps:
+# Initialize RAID5 with some data
+#
+# Re-mount RAID5 degraded with dev3 missing and write data
+# Save md5sum checkpoint1
+#
+# Re-mount healthy RAID5
+#
+# Let balance fix the RAID5.
+# Save md5sum checkpoint2
+#
+# Re-mount RAID5 degraded with dev1 as missing.
+# Save md5sum checkpoint3
+#
+# Verify if all three checkpoints match
+#
+#-
+# Copyright (c) 2016 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /
+   rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch_nocheck
+_require_scratch_dev_pool 3
+
+# we need btrfs to unload, need test dir unmounted
+_test_unmount
+_require_btrfs_loadable
+
+_scratch_dev_pool_get 3
+
+dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+dev3=`echo $SCRATCH_DEV_POOL | awk '{print $3}'`
+
+echo dev1=$dev1 >> $seqres.full
+echo dev2=$dev2 >> $seqres.full
+echo dev3=$dev3 >> $seqres.full
+
+# Balance won't be successful if filled too much
+dev1_sz=`blockdev --getsize64 $dev1`
+dev2_sz=`blockdev --getsize64 $dev2`
+dev3_sz=`blockdev --getsize64 $dev3`
+
+# get min of both
+max_fs_sz=`echo -e "$dev1_sz\n$dev2_sz\n$dev3_sz" | sort | head -1`
+if [ $max_fs_sz -gt 100 ]; then
+   bs="1M"
+   count=$(( max_fs_sz/100 ))
+   count=$(( count/100 ))
+else
+   _scratch_dev_pool_put
+   _test_mount
+   _notrun "Smallest dev size $max_fs_sz, Need at least 2M"
+fi
+
+#-init---
+echo >> $seqres.full
+echo "max_fs_sz=$max_fs_sz count=$count" >> $seqres.full
+echo "-Initialize -" >> $seqres.full
+_scratch_pool_mkfs "-mraid5 -draid5" >> $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+dd if=/dev/zero of="$SCRATCH_MNT"/tf1 bs=$bs count=1 \
+   >>$seqres.full 2>&1
+_run_btrfs_util_prog filesystem show
+_run_btrfs_util_prog filesystem df $SCRATCH_MNT
+count=$(( count-- ))
+
+
+#-degraded-init---
+echo >> $seqres.full
+echo "-Write degraded mount fill upto $max_fs_sz bytes-" >> 
$seqres.full
+echo
+echo "Write data with degraded mount"
+
+echo "unmount" >> $seqres.full
+_scratch_unmount
+echo "clean btrfs ko" >> $seqres.full
+# un-scan the btrfs devices
+_reload_btrfs_ko
+_mount -o degraded,device=$dev2 $dev1 $SCRATCH_MNT >>$seqres.full 2>&1
+dd if=/dev/zero of="$SCRATCH_MNT"/tf2 bs=$bs count=$count \
+   >>$seqres.full 2>&1
+_run_btrfs_util_prog filesystem show
+_run_btrfs_util_prog filesystem df $SCRATCH_MNT
+checkpoint1=`md5sum $SCRATCH_MNT/tf2`
+echo $checkpoint1 >> $seqres.full 2>&1
+
+#-balance---
+echo >> 

[PATCH v2 5/6] fstests: btrfs: test RAID1 device reappear and balanced

[Anand Jain]

From: Anand Jain 

The test does the following:
  Initialize a RAID1 with some data

  Re-mount RAID1 degraded with _dev1_ and write up to
  half of the FS capacity
  Save md5sum checkpoint1

  Re-mount healthy RAID1

  Let balance re-silver.
  Save md5sum checkpoint2

  Re-mount RAID1 degraded with _dev2_
  Save md5sum checkpoint3

  Verify if all three md5sum match

Signed-off-by: Anand Jain 
---
v2:
  add tmp= and its rm
  add comments to why _reload_btrfs_ko is used
  add missing put and test_mount at notrun exit
  use echo instead of _fail when checkpoints are checked
  .out updated to remove Silence..

 tests/btrfs/123 | 169 
 tests/btrfs/123.out |   7 +++
 tests/btrfs/group   |   1 +
 3 files changed, 177 insertions(+)
 create mode 100755 tests/btrfs/123
 create mode 100644 tests/btrfs/123.out

diff --git a/tests/btrfs/123 b/tests/btrfs/123
new file mode 100755
index ..33decfd1c434
--- /dev/null
+++ b/tests/btrfs/123
@@ -0,0 +1,169 @@
+#! /bin/bash
+# FS QA Test 123
+#
+# This test verify the RAID1 reconstruction on the reappeared
+# device. By using the following steps:
+# Initialize a RAID1 with some data
+#
+# Re-mount RAID1 degraded with dev2 missing and write up to
+# half of the FS capacity.
+# Save md5sum checkpoint1
+#
+# Re-mount healthy RAID1
+#
+# Let balance re-silver.
+# Save md5sum checkpoint2
+#
+# Re-mount RAID1 degraded with dev1 missing
+# Save md5sum checkpoint3
+#
+# Verify if all three checkpoints match
+#
+#-
+# Copyright (c) 2016 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /
+   rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch_nocheck
+_require_scratch_dev_pool 2
+
+# the mounted test dir prevent btrfs unload, we need to unmount
+_test_unmount
+_require_btrfs_loadable
+
+_scratch_dev_pool_get 2
+
+dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+
+dev1_sz=`blockdev --getsize64 $dev1`
+dev2_sz=`blockdev --getsize64 $dev2`
+# get min of both
+max_fs_sz=`echo -e "$dev1_sz\n$dev2_sz" | sort | head -1`
+max_fs_sz=$(( max_fs_sz/2 ))
+if [ $max_fs_sz -gt 100 ]; then
+   bs="1M"
+   count=$(( max_fs_sz/100 ))
+else
+   max_fs_sz=$(( max_fs_sz*2 ))
+   _scratch_dev_pool_put
+   _test_mount
+   _notrun "Smallest dev size $max_fs_sz, Need at least 2M"
+fi
+
+echo >> $seqres.full
+echo "max_fs_sz=$max_fs_sz count=$count" >> $seqres.full
+echo "-Initialize -" >> $seqres.full
+_scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+_run_btrfs_util_prog filesystem show
+dd if=/dev/zero of="$SCRATCH_MNT"/tf1 bs=$bs count=1 \
+   >>$seqres.full 2>&1
+count=$(( count-- ))
+echo "unmount" >> $seqres.full
+echo "clean btrfs ko" >> $seqres.full
+_scratch_unmount
+
+# un-scan the btrfs devices
+_reload_btrfs_ko
+
+
+echo >> $seqres.full
+echo "-Write degraded mount fill upto $max_fs_sz bytes-" >> 
$seqres.full
+echo
+echo "Write data with degraded mount"
+# Since we didn't run dev scan, btrfs kernel does not know
+# about the dev2
+# don't use _scratch_mount as we want to control
+# the device used for mounting.
+
+_mount -o degraded $dev1 $SCRATCH_MNT >>$seqres.full 2>&1
+_run_btrfs_util_prog filesystem show
+dd if=/dev/zero of="$SCRATCH_MNT"/tf2 bs=$bs count=$count \
+   >>$seqres.full 2>&1
+checkpoint1=`md5sum $SCRATCH_MNT/tf2`
+echo $checkpoint1 >> $seqres.full 2>&1
+_scratch_unmount
+echo "unmount" >> $seqres.full
+
+echo >> $seqres.full
+echo "-Mount normal-" >> $seqres.full
+echo
+echo "Mount normal after balance"
+_run_btrfs_util_prog device scan
+_scratch_mount >> $seqres.full 2>&1

[PATCH v2 4/6] fstests: btrfs: add helper function to check if btrfs is module

[Anand Jain]

From: Anand Jain 

We need btrfs to be a module so that it can unloaded and reloaded,
so that we can clean up the btrfs internal in memory device list.

This patch adds _require_btrfs_unloadable() and _reload_btrfs_ko()
to help with the same.

Signed-off-by: Anand Jain 
---
v2: rename _require_btrfs_unloadable to _require_btrfs_loadable

 common/rc | 12 
 1 file changed, 12 insertions(+)

diff --git a/common/rc b/common/rc
index ffdbfcdae68d..89281171365f 100644
--- a/common/rc
+++ b/common/rc
@@ -1441,6 +1441,18 @@ _supported_os()
 _notrun "not suitable for this OS: $HOSTOS"
 }
 
+_require_btrfs_loadable()
+{
+   modprobe -r btrfs || _notrun "btrfs unloadable"
+   modprobe btrfs || _notrun "Can't load btrfs"
+}
+
+_reload_btrfs_ko()
+{
+   modprobe -r btrfs || _fail "btrfs unload failed"
+   modprobe btrfs || _fail "btrfs load failed"
+}
+
 # this test needs a scratch partition - check we're ok & unmount it
 # No post-test check of the device is required. e.g. the test intentionally
 # finishes the test with the filesystem in a corrupt state
-- 
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 2/6] fstests: btrfs: add functions to get and put a device for replace target

[Anand Jain]

From: Anand Jain 

For the replace tests we need a device as a spare device,
here functions _spare_dev_get() and _spare_dev_put()
will get it from the SCRATCH_DEV_POOL_SAVED, which is set
when _scratch_dev_pool_get() is called, and is based on how
many has already been assigned to SCRATCH_DEV_POOL.

 usage:
   _scratch_dev_pool_get 3
   _spare_dev_get

  SPARE_DEV will have a device set which can be
  used as the replace target device.

   _spare_dev_put
   _scratch_dev_pool_put

_spare_dev_get() will pick the next device after SCRATCH_DEV_POOL
devices, from the SCRATCH_DEV_POOL_SAVED, and assigns it to
SPARE_DEV. _spare_dev_put() will set to SPARE_DEV to null.

Signed-off-by: Anand Jain 
---
 v2: Added comments

 common/rc | 59 +++
 1 file changed, 59 insertions(+)

diff --git a/common/rc b/common/rc
index 31c46ba1226e..ffdbfcdae68d 100644
--- a/common/rc
+++ b/common/rc
@@ -802,6 +802,65 @@ _scratch_mkfs()
 esac
 }
 
+# Helper function to get a spare or replace-target device from
+# configured SCRATCH_DEV_POLL, must call _scratch_dev_pool_get()
+# before _spare_dev_get(). Replace-target-device/Spare-device will
+# be assigned to SPARE_DEV.
+# As of now only one replace-target-device/spare-device can be
+# assigned.
+#
+# Usage:
+#  _scratch_dev_pool_get() 
+# _spare_dev_get()
+# :: do stuff
+# _spare_dev_put()
+#  _scratch_dev_pool_put()
+#
+_spare_dev_get()
+{
+   typeset -p SCRATCH_DEV_POOL_SAVED >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: unset val, must call _scratch_dev_pool_get before 
_spare_dev_get"
+   fi
+
+   if [ -z "$SCRATCH_DEV_POOL_SAVED" ]; then
+   _fail "Bug: str empty, must call _scratch_dev_pool_get before 
_spare_dev_get"
+   fi
+
+   # Check if the spare is already assigned
+   typeset -p SPARE_DEV >/dev/null 2>&1
+   if [ $? == 0 ]; then
+   if [ ! -z "$SPARE_DEV" ]; then
+   _fail "Bug: SPARE_DEV = $SPARE_DEV already assigned"
+   fi
+   fi
+
+   local ndevs=`echo $SCRATCH_DEV_POOL| wc -w`
+   local config_ndevs=`echo $SCRATCH_DEV_POOL_SAVED| wc -w`
+
+   if [ $ndevs -eq $config_ndevs ]; then
+   _notrun "All devs used no spare"
+   fi
+   # Get a dev that is not used
+   local devs[]="( $SCRATCH_DEV_POOL_SAVED )"
+   SPARE_DEV=${devs[@]:$ndevs:1}
+   export SPARE_DEV
+}
+
+_spare_dev_put()
+{
+   typeset -p SPARE_DEV >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: unset val, must call _spare_dev_get before its put"
+   fi
+
+   if [ -z "$SPARE_DEV" ]; then
+   _fail "Bug: str empty, must call _spare_dev_get before its put"
+   fi
+
+   export SPARE_DEV=""
+}
+
 #
 # Generally test cases will have..
 #   _require_scratch_dev_pool X
-- 
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH v2 1/6] fstests: btrfs: add functions to set and reset required number of SCRATCH_DEV_POOL

[Anand Jain]

From: Anand Jain 

This patch provides functions
 _scratch_dev_pool_get()
 _scratch_dev_pool_put()

Which will help to set/reset SCRATCH_DEV_POOL with the required
number of devices. SCRATCH_DEV_POOL_SAVED will hold all the devices.

Usage:
  _scratch_dev_pool_get() 
  :: do stuff

  _scratch_dev_pool_put()

Signed-off-by: Anand Jain 
---
 v2: Error message and comment section updates.

 common/rc | 55 +++
 1 file changed, 55 insertions(+)

diff --git a/common/rc b/common/rc
index 51092a0644f0..31c46ba1226e 100644
--- a/common/rc
+++ b/common/rc
@@ -802,6 +802,61 @@ _scratch_mkfs()
 esac
 }
 
+#
+# Generally test cases will have..
+#   _require_scratch_dev_pool X
+# to make sure it has the enough scratch devices including
+# replace-target and spare device. Now arg1 here is the
+# required number of scratch devices by a-test-case excluding
+# the replace-target and spare device. So this function will
+# set SCRATCH_DEV_POOL to the specified number of devices.
+#
+# Usage:
+#  _scratch_dev_pool_get() 
+# :: do stuff
+#
+#  _scratch_dev_pool_put()
+#
+_scratch_dev_pool_get()
+{
+   if [ $# != 1 ]; then
+   _fail "Usage: _scratch_dev_pool_get ndevs"
+   fi
+
+   local test_ndevs=$1
+   local config_ndevs=`echo $SCRATCH_DEV_POOL| wc -w`
+   local devs[]="( $SCRATCH_DEV_POOL )"
+
+   typeset -p config_ndevs >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: cant find SCRATCH_DEV_POOL ndevs"
+   fi
+
+   if [ $config_ndevs -lt $test_ndevs ]; then
+   _notrun "Need at least test requested number of ndevs 
$test_ndevs"
+   fi
+
+   SCRATCH_DEV_POOL_SAVED=${SCRATCH_DEV_POOL}
+   export SCRATCH_DEV_POOL_SAVED
+   SCRATCH_DEV_POOL=${devs[@]:0:$test_ndevs}
+   export SCRATCH_DEV_POOL
+}
+
+_scratch_dev_pool_put()
+{
+   typeset -p SCRATCH_DEV_POOL_SAVED >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: unset val, must call _scratch_dev_pool_get before 
_scratch_dev_pool_put"
+   fi
+
+   if [ -z "$SCRATCH_DEV_POOL_SAVED" ]; then
+   _fail "Bug: str empty, must call _scratch_dev_pool_get before 
_scratch_dev_pool_put"
+   fi
+
+   export SCRATCH_DEV_POOL=$SCRATCH_DEV_POOL_SAVED
+   export SCRATCH_DEV_POOL_SAVED=""
+}
+
 _scratch_pool_mkfs()
 {
 case $FSTYP in
-- 
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 5/6] fstests: btrfs: test RAID1 device reappear and balanced

[Anand Jain]

On 06/12/2016 01:06 PM, Eryu Guan wrote:

On Tue, May 17, 2016 at 10:32:09PM +0800, Anand Jain wrote:

The test does the following:
  Initialize a RAID1 with some data

  Re-mount RAID1 degraded with _dev1_ and write up to
  half of the FS capacity
  Save md5sum checkpoint1

  Re-mount healthy RAID1

  Let balance re-silver.
  Save md5sum checkpoint2

  Re-mount RAID1 degraded with _dev2_
  Save md5sum checkpoint3

  Verify if all three md5sum match

Signed-off-by: Anand Jain 
---
 tests/btrfs/121 | 163 
 tests/btrfs/121.out |   8 +++
 tests/btrfs/group   |   1 +
 3 files changed, 172 insertions(+)
 create mode 100755 tests/btrfs/121
 create mode 100644 tests/btrfs/121.out

diff --git a/tests/btrfs/121 b/tests/btrfs/121
new file mode 100755
index ..81b7deb047af
--- /dev/null
+++ b/tests/btrfs/121
@@ -0,0 +1,163 @@
+#! /bin/bash
+# FS QA Test 121
+#
+# This test verify the RAID1 reconstruction on the reappeared
+# device. By using the following steps:
+# Initialize a RAID1 with some data
+#
+# Re-mount RAID1 degraded with dev2 missing and write up to
+# half of the FS capacity.
+# Save md5sum checkpoint1
+#
+# Re-mount healthy RAID1
+#
+# Let balance re-silver.
+# Save md5sum checkpoint2
+#
+# Re-mount RAID1 degraded with dev1 missing
+# Save md5sum checkpoint3
+#
+# Verify if all three checkpoints match
+#
+#-
+# Copyright (c) 2016 Oracle.  All Rights Reserved.
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#-
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+here=`pwd`
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /


The "rm -f $tmp.*" is still needed even $tmp.* is not used in the test
itself, it's still possible to be used by some helper functions.


 I didn't set the tmp= either, so is it mandatory to set tmp=?
 Anyway have done it in v2.


+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# remove previous $seqres.full before test
+rm -f $seqres.full
+
+# real QA test starts here
+
+_supported_fs btrfs
+_supported_os Linux
+_require_scratch_nocheck
+_require_scratch_dev_pool 2
+
+# the mounted test dir prevent btrfs unload, we need to unmount
+_test_unmount
+_require_btrfs_unloadable
+
+_scratch_dev_pool_get 2
+
+dev1=`echo $SCRATCH_DEV_POOL | awk '{print $1}'`
+dev2=`echo $SCRATCH_DEV_POOL | awk '{print $2}'`
+
+dev1_sz=`blockdev --getsize64 $dev1`
+dev2_sz=`blockdev --getsize64 $dev2`
+# get min of both
+max_fs_sz=`echo -e "$dev1_sz\n$dev2_sz" | sort | head -1`
+max_fs_sz=$(( max_fs_sz/2 ))
+if [ $max_fs_sz -gt 100 ]; then
+   bs="1M"
+   count=$(( max_fs_sz/100 ))
+else
+   max_fs_sz=$(( max_fs_sz*2 ))
+   _notrun "Smallest dev size $max_fs_sz, Need at least 2M"
+fi
+
+echo >> $seqres.full
+echo "max_fs_sz=$max_fs_sz count=$count" >> $seqres.full
+echo "-Initialize -" >> $seqres.full
+_scratch_pool_mkfs "-mraid1 -draid1" >> $seqres.full 2>&1
+_scratch_mount >> $seqres.full 2>&1
+_run_btrfs_util_prog filesystem show
+dd if=/dev/zero of="$SCRATCH_MNT"/tf1 bs=$bs count=1 \
+   >>$seqres.full 2>&1
+count=$(( count-- ))
+echo "unmount" >> $seqres.full
+echo "clean btrfs ko" >> $seqres.full
+_scratch_unmount
+_reload_btrfs_ko


A comment to explain the reason to reload btrfs.ko would be good.


yes. added.


+
+
+echo >> $seqres.full
+echo "-Write degraded mount fill upto $max_fs_sz bytes-" >> 
$seqres.full
+echo
+echo "Write data with degraded mount"
+# Since we didn't run dev scan, btrfs kernel does not know
+# about the dev2
+# don't use _scratch_mount as we want to control
+# the device used for mounting.
+
+_mount -o degraded $dev1 $SCRATCH_MNT >>$seqres.full 2>&1
+_run_btrfs_util_prog filesystem show
+dd if=/dev/zero of="$SCRATCH_MNT"/tf2 bs=$bs count=$count \
+   >>$seqres.full 2>&1
+checkpoint1=`md5sum $SCRATCH_MNT/tf2`
+echo $checkpoint1 >> $seqres.full 2>&1
+_scratch_unmount
+echo "unmount" >> $seqres.full
+
+echo >> $seqres.full
+echo "-Mount normal-" >> $seqres.full
+echo
+echo "Mount normal after balance"

Re: [PATCH 2/6] fstests: btrfs: add functions to get and put a device for replace target

[Anand Jain]

On 06/12/2016 12:42 PM, Eryu Guan wrote:

On Tue, May 17, 2016 at 10:32:06PM +0800, Anand Jain wrote:

For the replace tests we need a device as a spare device,
here functions _spare_dev_get() and _spare_dev_put()
will get it from the SCRATCH_DEV_POOL_SAVED, which is set
when _scratch_dev_pool_get() is called, and is based on how
many has already been assigned to SCRATCH_DEV_POOL.

 usage:
   _scratch_dev_pool_get 3
   _spare_dev_get

  SPARE_DEV will have a device set which can be
  used as the replace target device.

   _spare_dev_put
   _scratch_dev_pool_put

_spare_dev_get() will pick the next device after SCRATCH_DEV_POOL
devices, from the SCRATCH_DEV_POOL_SAVED, and assigns it to
SPARE_DEV. _spare_dev_put() will set to SPARE_DEV to null.

Signed-off-by: Anand Jain 
---
 common/rc | 45 +
 1 file changed, 45 insertions(+)

diff --git a/common/rc b/common/rc
index 33632fd8e4a3..f2b39ddbee0c 100644
--- a/common/rc
+++ b/common/rc
@@ -786,6 +786,51 @@ _scratch_mkfs()
 esac
 }

+_spare_dev_get()


I'd like to see some comments about these helpers in the code. Otherwise
look good to me.


 Added in v2. Thanks Anand



Thanks,
Eryu


+{
+   typeset -p SCRATCH_DEV_POOL_SAVED >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: unset val, must call _scratch_dev_pool_get before 
_spare_dev_get"
+   fi
+
+   if [ -z "$SCRATCH_DEV_POOL_SAVED" ]; then
+   _fail "Bug: str empty, must call _scratch_dev_pool_get before 
_spare_dev_get"
+   fi
+
+   # Check if the spare is already assigned
+   typeset -p SPARE_DEV >/dev/null 2>&1
+   if [ $? == 0 ]; then
+   if [ ! -z "$SPARE_DEV" ]; then
+   _fail "Bug: SPARE_DEV = $SPARE_DEV already assigned"
+   fi
+   fi
+
+   local ndevs=`echo $SCRATCH_DEV_POOL| wc -w`
+   local config_ndevs=`echo $SCRATCH_DEV_POOL_SAVED| wc -w`
+
+   if [ $ndevs -eq $config_ndevs ]; then
+   _notrun "All devs used no spare"
+   fi
+   # Get a dev that is not used
+   local devs[]="( $SCRATCH_DEV_POOL_SAVED )"
+   SPARE_DEV=${devs[@]:$ndevs:1}
+   export SPARE_DEV
+}
+
+_spare_dev_put()
+{
+   typeset -p SPARE_DEV >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: unset val, must call _spare_dev_get before its put"
+   fi
+
+   if [ -z "$SPARE_DEV" ]; then
+   _fail "Bug: str empty, must call _spare_dev_get before its put"
+   fi
+
+   export SPARE_DEV=""
+}
+
 #
 # $1 Number of the scratch devs required
 #
--
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 1/6] fstests: btrfs: add functions to set and reset required number of SCRATCH_DEV_POOL

[Anand Jain]

Thanks for reviewing.

On 06/12/2016 12:40 PM, Eryu Guan wrote:

On Tue, May 17, 2016 at 10:32:05PM +0800, Anand Jain wrote:

This patch provides functions
 _scratch_dev_pool_get()
 _scratch_dev_pool_put()

Which will help to set/reset SCRATCH_DEV_POOL with the required
number of devices. SCRATCH_DEV_POOL_SAVED will hold all the devices.

Usage:
  _scratch_dev_pool_get() 
  :: do stuff

  _scratch_dev_pool_put()

Signed-off-by: Anand Jain 


I'm not sure about the usefulness and the implementation of these
helpers (include the _spare_dev_get|_put helpers), but they look good to
me. It'd better to have btrfs developers review them as well.


---
 common/rc | 47 +++
 1 file changed, 47 insertions(+)

diff --git a/common/rc b/common/rc
index 91e8f1c8e693..33632fd8e4a3 100644
--- a/common/rc
+++ b/common/rc
@@ -786,6 +786,53 @@ _scratch_mkfs()
 esac
 }

+#
+# $1 Number of the scratch devs required
+#


Some comments about its usage/purpose would be good, otherwise one has
to search for the commit log and look into it to understand its purpose


added in v2.


+_scratch_dev_pool_get()
+{
+   if [ $# != 1 ]; then
+   _fail "Usage: _scratch_dev_pool_get ndevs"
+   fi
+
+   local test_ndevs=$1
+   local config_ndevs=`echo $SCRATCH_DEV_POOL| wc -w`
+   local devs[]="( $SCRATCH_DEV_POOL )"
+
+   typeset -p config_ndevs >/dev/null 2>&1
+   if [ $? != 0 ]; then
+   _fail "Bug: unset val, must call _scratch_dev_pool_get before 
_scratch_dev_pool_put"


This error message seems confusing, it's _scratch_dev_pool_get being
called here, not _put.


ah, copy paste typo. fixed it in v2.


+   fi
+
+   # _require_scratch_dev_pool $test_ndevs
+   # must have already checked the min required devices
+   # but just in case, trap here for any potential bugs
+   # perpetuating any further
+   if [ $config_ndevs -lt $test_ndevs ]; then
+   _notrun "Need at least test requested number of ndevs 
$test_ndevs"


This message is not clear to me either.


 ok, updated.

Thanks, Anand



Thanks,
Eryu


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH 4/6] fstests: btrfs: add helper function to check if btrfs is module

[Anand Jain]

On 06/12/2016 12:53 PM, Eryu Guan wrote:

On Tue, May 17, 2016 at 10:32:08PM +0800, Anand Jain wrote:

We need btrfs to be a module so that it can unloaded and reloaded,
so that we can clean up the btrfs internal in memory device list.


It looks like a bug to me if btrfs needs to reload module to clean up
the device list.

 Definitely not a bug, but a pending enhancement.


If a user builds btrfs in kernel then he cannot replace
btrfs device properly?

 No. That's not true.


Or it's not totally clear to me why a test needs
to reload btrfs module.


 Sure.
 Test cases such 5/6 and 6/6 needs devices to be un-scanned.
 But as said, as of now we don't have such a feature and
 in a way _reload_btrfs_ko() will help to achieve the device
 un-scan.

 Further we have seen situations where previous test cases
 affects the next test case results. These are bugs,
 but are unintentional at that test case point of viwe, and
 is disturbingly deviating, so these helpers will help to
 start with a clean btrfs memory, so that test results are
 consistent to what is being tested.



This patch adds _require_btrfs_unloadable() and _reload_btrfs_ko()
to help with the same.

Signed-off-by: Anand Jain 
---
 common/rc | 12 
 1 file changed, 12 insertions(+)

diff --git a/common/rc b/common/rc
index f2b39ddbee0c..a9391b4cbf57 100644
--- a/common/rc
+++ b/common/rc
@@ -1403,6 +1403,18 @@ _supported_os()
 _notrun "not suitable for this OS: $HOSTOS"
 }

+_require_btrfs_unloadable()


The name seems misleading, the name indicates that it requires btrfs to
be unloadable, but in the code it _notrun if btrfs is unloadable.


 ok. Will rename it.

Thanks, Anand



Thanks,
Eryu


+{
+   modprobe -r btrfs || _notrun "btrfs unloadable"
+   modprobe btrfs || _notrun "Can't load btrfs"
+}
+
+_reload_btrfs_ko()
+{
+   modprobe -r btrfs || _fail "btrfs unload failed"
+   modprobe btrfs || _fail "btrfs load failed"
+}
+
 # this test needs a scratch partition - check we're ok & unmount it
 # No post-test check of the device is required. e.g. the test intentionally
 # finishes the test with the filesystem in a corrupt state
--
2.7.0

--
To unsubscribe from this list: send the line "unsubscribe fstests" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] btrfs-progs: doc: correct the destination of btrfs-receive

[David Sterba]

On Wed, Jun 15, 2016 at 10:55:45AM +0900, Satoru Takeuchi wrote:
> >> ie. all the context lines start with two spaces instead of one. I'll
> >> apply this patch manually but please have a look.
> > 
> >Looking at this, I suspect it's a consequence of sending it as
> > "Content-Type: format=flowed; delsp=yes". I'm not sure which of those
> > two options is the culprit. When I look at the message in my client
> > (mutt), it looks absolutely fine. When I pipe it to hexdump, the
> > double-spacing is apparent.
> 
> You're right. These are added to charset="iso-2022-jp" plain text
> mail since thunderbird 45.
> 
> I disabled the setting that appends the above mentioned options.
> So, probably the following sample patch doesn't have strange spaces.

Yes, looks good now, thanks.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: BTRFS, remarkable problem: filesystem turns to read-only caused by firefox download

[Duncan]

Paul Verreth posted on Wed, 15 Jun 2016 08:29:49 +0200 as excerpted:

> When I download a video using  Firefox DownloadHelper addon, the
> filesystem suddenly turns read only. Not a coincedence, I tried it
> several times, and it happened every time again
> 
> Info:
> Linux wolfgang 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 UTC
> 2016 x86_64 x86_64 x86_64 GNU/Linux

Well first of all, that 4.2 kernel version isn't really supported any 
more, except possibly by your distro.  It's not a mainstream LTS kernel, 
with 4.1 and 4.4 being the LTS kernels on either side, and as a short-
term support kernel, its mainstream support lasted thru 4.3...

Of course distros can choose to support whatever kernel they like, but 
then it's them doing the patch backporting and we don't track what 
they've backported and what they haven't.  So if you want to stick with a 
distro-supported kernel, the best option is to get your support from them 
as they know what they've backported and are thus in the best position to 
support it.

As far as this list, 4.6 is the current stable kernel, with 4.7 in 
development, so 4.5 is the earliest non-LTS kernel series that's really 
supported.  Or go LTS series and choose, as mentioned, 4.1 or 4.4.  3.18 
was the LTS previous to that, but it's getting a bit long in tooth by now.

> btrfs --version btrfs-progs v4.0

That too is a bit dated.  While userspace version isn't as critical for 
runtime operations as it's mostly calling on the kernel to do the real 
work, once something goes wrong and you're trying to repair it, userspace 
code becomes vitally important.  With btrfs still stabilizing, not fully 
stable and mature, and still under heavy development, and with userspace 
versions synced to kernelspace releases and about five kernel series 
releases per year, given that 4.6 is current, 4.0 is over a year outdated 
now.  And a lot of bugs have been fixed in that year-plus...

> extract from dmesg:

> [171145.415466] BTRFS error (device sda5): unable to find ref byte
> nr 75093794816 parent 0 root 257  owner 0 offset 0
> [171145.415467] [ cut here ]
> [171145.415473] WARNING: CPU: 3 PID: 15124 at
> /build/linux-HVWSXI/linux-4.2.0/fs/btrfs/extent-tree.c:6264
> __btrfs_free_extent.isra.69+0x92f/0xd70 [btrfs]()
> [171145.415474] BTRFS: Transaction aborted (error -2)
> [171145.415492] CPU: 3 PID: 15124 Comm: kworker/u16:0 Tainted: G
>  W   4.2.0-35-generic #40-Ubuntu
> [171145.415493] Hardware name: ASUS All Series/Z87-PLUS, BIOS 1707
> 12/13/2013
> [171145.415500] Workqueue: btrfs-extent-refs
> btrfs_extent_refs_helper [btrfs]

[...]

> [171145.415568] ---[ end trace 42e5b5054b17a8a2 ]---
> [171145.415570] BTRFS: error (device sda5) in __btrfs_free_extent:6264:
> errno=-2 No such entry
> [171145.415571] BTRFS info (device sda5): forced
> readonly [171145.415572] BTRFS: error (device sda5) in
> btrfs_run_delayed_refs:2788: errno=-2 No such entry


I'm not a dev, just a btrfs user and list regular, so the stack dump 
doesn't mean a whole lot to me.  What I can say, however, is that yes, 
this is btrfs involved...

And it's pretty standard for btrfs to force itself read-only when it sees 
an unexpected error that could otherwise lead to further damage, with the 
force to read-only thus protecting the filesystem from that further 
damage.

> I was able to mount RW again using -o recovery.
> 
> Based on these messages, I thought it would be usefull to do btrfs
> balance but it gave a segmentation fault after some minutes:
> 
> [246678.922508] BTRFS: error (device sdb5) in __btrfs_free_extent:6549:
> errno=-2 No such entry
> [246678.922509] BTRFS info (device sdb5): forced readonly
> [246678.922510] BTRFS: error (device sdb5) in
> btrfs_run_delayed_refs:2927: errno=-2 No such entry
> [246678.922520] BTRFS error (device sdb5): Error removing orphan entry,
> stopping orphan cleanup
> [246678.922521] BTRFS error (device sdb5): could not do orphan
> cleanup -22
> [246678.937230] BTRFS error (device sdb5): cleaner transaction attach
> returned -30

FWIW, orphans are files that were deleted when the file was in-use, 
typically *.so libraries that were replaced on package update, but where 
some executable that was running at the time was still using them, so 
they couldn't be fully deleted as there was still and open reference to 
them.

Normally, orphans will be deleted on umount or for the root filesystem, 
on (normal) remount-read-only, after whatever executables that were 
holding them open have terminated or been killed.  However, that doesn't 
have a chance to happen when the filesystem is forced read-only due to 
error, as above, so then they have to be cleaned up when the filesystem 
is remounted writable once again.

Btrfs does this normally, so this would have been unrelated to the 
balance.

> Balance:
> 
> btrfs balance start -v -dconvert=raid1 -mconvert=raid1 /mnt
> Dumping filters: flags 0x7, state 0x0, force is off
> DATA (flags 0x100): converting, 

Re: BTRFS, remarkable problem: filesystem turns to read-only caused by firefox download

[Paul Verreth]

Hello Fajar.

Thanks.

But since I can't boot anymore from the filesystem I tried to repair
the situation using a Xenial Live USB-stick with kernel
4.4.0-22-generic.

Without any success. The balance seems active, but there is no
noticable progress in DMESG, kern.log or using balance status.

Kind regards.

2016-06-15 8:42 GMT+02:00 Fajar A. Nugraha :
> On Wed, Jun 15, 2016 at 1:29 PM, Paul Verreth  wrote:
>> Dear all.
>>
>> When I download a video using  Firefox DownloadHelper addon, the
>> filesystem suddenly turns read only. Not a coincedence, I tried it
>> several times, and it happened every time again
>>
>> Info:
>> Linux wolfgang 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 UTC
>> 2016 x86_64 x86_64 x86_64 GNU/Linux
>
>> Segmentation fault
>>
>> Jun  5 15:03:15 ubuntu kernel: [ 2062.544303] BTRFS info (device
>> sdb5): relocating block group 383447465984 flags 17
>
>
>> What can I do to repair this problem?
>
> The usual starting advice would be "try with latest kernel and see if
> you can still reproduce the problem". Is it ubuntu wily? It'd go end
> of in July anyway, so you might want to upgrade to xenial (or at
> least, just the kernel, for the purpose of troubleshooting your
> problem).
>
> Or even try http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/
> (should be usable, but might report some errors/warning due to missing
> ubuntu patches)
>
> --
> Fajar



-- 

Paul Verreth

Het verschil tussen theorie en praktijk
is klein in theorie,
maar in praktijk daarentegen ...
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: BTRFS, remarkable problem: filesystem turns to read-only caused by firefox download

[Fajar A. Nugraha]

On Wed, Jun 15, 2016 at 1:29 PM, Paul Verreth  wrote:
> Dear all.
>
> When I download a video using  Firefox DownloadHelper addon, the
> filesystem suddenly turns read only. Not a coincedence, I tried it
> several times, and it happened every time again
>
> Info:
> Linux wolfgang 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 UTC
> 2016 x86_64 x86_64 x86_64 GNU/Linux

> Segmentation fault
>
> Jun  5 15:03:15 ubuntu kernel: [ 2062.544303] BTRFS info (device
> sdb5): relocating block group 383447465984 flags 17


> What can I do to repair this problem?

The usual starting advice would be "try with latest kernel and see if
you can still reproduce the problem". Is it ubuntu wily? It'd go end
of in July anyway, so you might want to upgrade to xenial (or at
least, just the kernel, for the purpose of troubleshooting your
problem).

Or even try http://kernel.ubuntu.com/~kernel-ppa/mainline/daily/current/
(should be usable, but might report some errors/warning due to missing
ubuntu patches)

-- 
Fajar
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Fwd: BTRFS, remarkable problem: filesystem turns to read-only caused by firefox download

[Paul Verreth]

Dear all.

When I download a video using  Firefox DownloadHelper addon, the
filesystem suddenly turns read only. Not a coincedence, I tried it
several times, and it happened every time again

Info:
Linux wolfgang 4.2.0-35-generic #40-Ubuntu SMP Tue Mar 15 22:15:45 UTC
2016 x86_64 x86_64 x86_64 GNU/Linux

btrfs --version
btrfs-progs v4.0

firefox --version
Mozilla Firefox 46.0

package: btrfs-tools
State: installed
Automatically installed: no
Version: 4.0-2
Priority: optional
Section: admin
Maintainer: Ubuntu Developers 
Architecture: amd64
Uncompressed Size: 3.518 k
Depends: e2fslibs (>= 1.42), libblkid1 (>= 2.17.2), libc6 (>= 2.8), libcomerr2
 (>= 1.01), liblzo2-2, libuuid1 (>= 2.16), zlib1g (>= 1:1.2.0)
Conflicts: btrfs-tools
Description: Checksumming Copy on Write Filesystem utilities

Homepage: http://btrfs.wiki.kernel.org/


extract from dmesg:

[171145.415378] tree block key (18446744073709551611 48 3255000) level 0
[171145.415379] shared block backref parent 203105845248
[171145.415379] item 4 key (75093737472 168 4096) itemoff 3740 itemsize 51
[171145.415380] extent refs 1 gen 1551545 flags 258
[171145.415381] tree block key (3547221 12 3547219) level 0
[171145.415381] shared block backref parent 75092348928
[171145.415382] item 5 key (75093741568 168 4096) itemoff 3689 itemsize 51
[171145.415382] extent refs 1 gen 1452265 flags 258
[171145.415383] tree block key (3280755 12 3280753) level 0
[171145.415383] shared block backref parent 203105845248
[171145.415384] item 6 key (75093745664 168 4096) itemoff 3638 itemsize 51
[171145.415385] extent refs 1 gen 1452265 flags 258
[171145.415385] tree block key (18446744073709551611 48 3255000) level 0
[171145.415386] shared block backref parent 203105845248
[171145.415386] item 7 key (75093749760 168 4096) itemoff 3587 itemsize 51
[171145.415387] extent refs 1 gen 1514341 flags 258
[171145.415387] tree block key (3473870 1 0) level 0
[171145.415388] shared block backref parent 75091329024
[171145.415388] item 8 key (75093753856 168 4096) itemoff 3536 itemsize 51
[171145.415389] extent refs 1 gen 1597177 flags 258
[171145.415390] tree block key (2921841 108 6848512) level 0
[171145.415390] shared block backref parent 75091030016
[171145.415391] item 9 key (75093757952 168 4096) itemoff 3485 itemsize 51
[171145.415391] extent refs 1 gen 1452265 flags 258
[171145.415392] tree block key (18446744073709551611 48 3254998) level 0
[171145.415392] shared block backref parent 203105845248
[171145.415393] item 10 key (75093766144 168 4096) itemoff 3434 itemsize 51
[171145.415394] extent refs 1 gen 1452265 flags 258
[171145.415394] tree block key (3280757 96 12) level 0
[171145.415395] shared block backref parent 203105845248
[171145.415395] item 11 key (75093770240 168 4096) itemoff 3383 itemsize 51
[171145.415396] extent refs 1 gen 1452265 flags 258
[171145.415396] tree block key (18446744073709551611 48 3254998) level 0
[171145.415397] shared block backref parent 203105845248
[171145.415398] item 12 key (75093774336 168 4096) itemoff 3332 itemsize 51
[171145.415398] extent refs 1 gen 1452265 flags 258
[171145.415399] tree block key (3280738 84 4205285998) level 0
[171145.415399] shared block backref parent 203105845248
[171145.415400] item 13 key (75093778432 168 4096) itemoff 3281 itemsize 51
[171145.415400] extent refs 1 gen 1452265 flags 258
[171145.415401] tree block key (18446744073709551611 48 3254998) level 0
[171145.415401] shared block backref parent 203105845248
[171145.415402] item 14 key (75093782528 168 4096) itemoff 3230 itemsize 51
[171145.415403] extent refs 1 gen 1551545 flags 258
[171145.415403] tree block key (3547236 84 3743801254) level 0
[171145.415404] shared block backref parent 75092348928
[171145.415404] item 15 key (75093790720 168 4096) itemoff 3179 itemsize 51
[171145.415405] extent refs 1 gen 305740 flags 258
[171145.415406] tree block key (831798 96 265) level 0
[171145.415406] shared block backref parent 25729994752
[171145.415407] item 16 key (75093794816 168 4096) itemoff 3128 itemsize 51
[171145.415407] extent refs 1 gen 1525268 flags 2
[171145.415408] tree block key (11528453 1 0) level 0
[171145.415408] tree block backref root 281474976710913
[171145.415409] item 17 key (75093798912 168 4096) itemoff 3077 itemsize 51
[171145.415410] extent refs 1 gen 1452265 flags 258
[171145.415410] tree block key (18446744073709551611 48 3254998) level 0
[171145.415411] shared block backref parent 203105845248
[171145.415411] item 18 key (75093803008 168 4096) itemoff 3026 itemsize 51
[171145.415412] extent refs 1 gen 1452265 flags 258
[171145.415413] tree block key (18446744073709551611 48 3255000) level 0
[171145.415413] shared block backref parent 203105845248
[171145.415414] item 19 key (75093807104 168 4096) itemoff 2975 itemsize 51
[171145.415414] extent refs 1 gen 1141992 flags 2
[171145.415415] tree block key (25600704512 168 4096) level 0
[171145.415415] tree block backref root 2