date:20180114

Re: [PATCH] fstests: common/rc: fix device still mounted error with SCRATCH_DEV_POOL

2018-01-14 Thread Eryu Guan

On Fri, Jan 12, 2018 at 06:04:59PM -0700, Liu Bo wrote:
> One of btrfs tests, btrfs/011, uses SCRATCH_DEV_POOL and puts a 
> non-SCRATCH_DEV
> device as the first one when doing mkfs, and this makes
> _require_scratch{_nocheck} fail to umount $SCRATCH_MNT since it checks mount
> point with SCRATCH_DEV only, and for sure it finds nothing to umount and the
> following tests complain about 'device still mounted' alike errors.
> 
> Introduce a helper to address this special case where both btrfs and scratch
> dev pool are in use.
> 
> Signed-off-by: Liu Bo 

Hmm, I didn't see this problem, I ran btrfs/011 then another tests that
uses $SCRATCH_DEV, and the second test ran fine too. Can you please
provide more details?

Anyway, I think we should fix btrfs/011 to either not use $SCRATCH_DEV
in replace operations (AFAIK, other btrfs replace tests do this) or
umount all devices before exit. And I noticed btrfs/011 does umount
$SCRATCH_MNT at the end of workout(), so usually all should be fine
(perhaps it would leave a device mounted if interrupted in the middle of
test run, because _cleanup() doesn't do umount).

Thanks,
Eryu
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs: Add chunk allocation ENOSPC debug message for enospc_debug mount option

2018-01-14 Thread Qu Wenruo

Enospc_debug makes extent allocator to print more debug messages,
however for chunk allocation, there is no debug message for enospc_debug
at all.

This patch will add message for the following parts of chunk allocator:

1) No rw device at all
   Quite rare, but at least output one message for this case.

2) No enough space for some device
   This debug message is quite handy for unbalanced disks with stripe
   based profiles (RAID0/10/5/6).

3) Not enough free devices
   This debug message should tell us if current chunk allocator is
   working correctly on minimal device requirement.

Although under most case, we will hit other ENOSPC before we even hit a
chunk allocator ENOSPC, but such debug info won't help.

Signed-off-by: Qu Wenruo 
---
 fs/btrfs/volumes.c | 19 +--
 1 file changed, 17 insertions(+), 2 deletions(-)

diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index a25684287501..664d8a1b90b3 100644
--- a/fs/btrfs/volumes.c
+++ b/fs/btrfs/volumes.c
@@ -4622,8 +4622,11 @@ static int __btrfs_alloc_chunk(struct btrfs_trans_handle 
*trans,
 
BUG_ON(!alloc_profile_is_valid(type, 0));
 
-   if (list_empty(&fs_devices->alloc_list))
+   if (list_empty(&fs_devices->alloc_list)) {
+   if (btrfs_test_opt(info, ENOSPC_DEBUG))
+   btrfs_warn(info, "%s: No writable device", __func__);
return -ENOSPC;
+   }
 
index = __get_raid_index(type);
 
@@ -4705,8 +4708,14 @@ static int __btrfs_alloc_chunk(struct btrfs_trans_handle 
*trans,
if (ret == 0)
max_avail = max_stripe_size * dev_stripes;
 
-   if (max_avail < BTRFS_STRIPE_LEN * dev_stripes)
+   if (max_avail < BTRFS_STRIPE_LEN * dev_stripes) {
+   if (btrfs_test_opt(info, ENOSPC_DEBUG))
+   btrfs_debug(info,
+   "%s: devid %llu has no free space, have=%llu want=%u",
+   __func__, device->devid, max_avail,
+   BTRFS_STRIPE_LEN * dev_stripes);
continue;
+   }
 
if (ndevs == fs_devices->rw_devices) {
WARN(1, "%s: found more than %llu devices\n",
@@ -4731,6 +4740,12 @@ static int __btrfs_alloc_chunk(struct btrfs_trans_handle 
*trans,
 
if (ndevs < devs_increment * sub_stripes || ndevs < devs_min) {
ret = -ENOSPC;
+   if (btrfs_test_opt(info, ENOSPC_DEBUG)) {
+   btrfs_debug(info,
+   "%s: not enough devices with free space: have=%d minimal=%d 
increment=%d",
+   __func__, ndevs, devs_min,
+   devs_increment * sub_stripes);
+   }
goto error;
}
 
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

[PATCH] btrfs-progs: fsck: Continue check even if we find space cache problem

2018-01-14 Thread Qu Wenruo

Space cache itself is not a fatal part of btrfs (especially for v1 space
cache), and if we find something wrong it can be caused by other more
serious problem.

So continue to check more important trees even we found something wrong
in free space cache.

Signed-off-by: Qu Wenruo 
---
 cmds-check.c | 5 -
 1 file changed, 4 insertions(+), 1 deletion(-)

diff --git a/cmds-check.c b/cmds-check.c
index 7fc30da83ea1..6083ac0dd032 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -15000,7 +15000,10 @@ int cmd_check(int argc, char **argv)
error("errors found in free space tree");
else
error("errors found in free space cache");
-   goto out;
+   /*
+* Space cache problem is not fatal, and normally indicates more
+* serious problem, so continue check to expose the cause.
+*/
}
 
/*
-- 
2.15.1

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-14 Thread Qu Wenruo



On 2018年01月15日 09:59, Lu Fengqi wrote:
> On Mon, Jan 15, 2018 at 09:26:27AM +0800, Qu Wenruo wrote:
>>
>>
>> On 2018年01月15日 00:38, Sebastian Andrzej Siewior wrote:
>>> On 2018-01-14 08:36:41 [+0800], Qu Wenruo wrote:
 Still needs more. (and maybe even more depending on the output)

 The original mode doesn't report error clear enough, so it would help if
 --mode=lowmem can be used.

 # btrfs check --mode=lowmem /dev/sdb4
>>>
>>> ~# btrfs check --mode=lowmem /dev/sdb4
>>> Checking filesystem on /dev/sdb4
>>> UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
>>> checking extents
>>> checking free space cache
>>> Wanted bytes 4096, found 8192 for off 159178047488
>>> Wanted bytes 1073725440, found 8192 for off 159178047488
>>> cache appears valid but isn't 159178031104
>>> ERROR: errors found in free space cache
>>
>> A new bug is exposed for lowmem mode.
>> It doesn't check fs tree if space cache has any problem.
>>
> 
> Just a reminder, the problem is not only for lowmem mode.

Right, I'll fix it soon.

And BTW what makes the the output different from the original one?

Sebastaian, did you do extra write or other operation to the fs after
previous btrfs check?

Thanks,
Qu

> 
> ret = check_space_cache(root);
> err |= !!ret;
> if (ret) {
> if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE))
> error("errors found in free space tree");
> else
> error("errors found in free space cache");
> goto out;
> ^^^
> }
> 
> /*
>  * We used to have to have these hole extents in between our real
>  * extents so if we don't have this flag set we need to make sure 
> there
>  * are no gaps in the file extents for inodes, otherwise we can just
>  * ignore it when this happens.
>  */
> no_holes = btrfs_fs_incompat(root->fs_info, NO_HOLES);
> ret = do_check_fs_roots(info, &root_cache);
>   ^^^
> err |= !!ret;
> if (ret) {
> error("errors found in fs roots");
> goto out;
> }
> 



signature.asc
Description: OpenPGP digital signature

Re: invalid files names, btrfs check can't repair it

2018-01-14 Thread Lu Fengqi

On Mon, Jan 15, 2018 at 09:26:27AM +0800, Qu Wenruo wrote:
>
>
>On 2018年01月15日 00:38, Sebastian Andrzej Siewior wrote:
>> On 2018-01-14 08:36:41 [+0800], Qu Wenruo wrote:
>>> Still needs more. (and maybe even more depending on the output)
>>>
>>> The original mode doesn't report error clear enough, so it would help if
>>> --mode=lowmem can be used.
>>>
>>> # btrfs check --mode=lowmem /dev/sdb4
>> 
>> ~# btrfs check --mode=lowmem /dev/sdb4
>> Checking filesystem on /dev/sdb4
>> UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
>> checking extents
>> checking free space cache
>> Wanted bytes 4096, found 8192 for off 159178047488
>> Wanted bytes 1073725440, found 8192 for off 159178047488
>> cache appears valid but isn't 159178031104
>> ERROR: errors found in free space cache
>
>A new bug is exposed for lowmem mode.
>It doesn't check fs tree if space cache has any problem.
>

Just a reminder, the problem is not only for lowmem mode.

ret = check_space_cache(root);
err |= !!ret;
if (ret) {
if (btrfs_fs_compat_ro(info, FREE_SPACE_TREE))
error("errors found in free space tree");
else
error("errors found in free space cache");
goto out;
^^^
}

/*
 * We used to have to have these hole extents in between our real
 * extents so if we don't have this flag set we need to make sure there
 * are no gaps in the file extents for inodes, otherwise we can just
 * ignore it when this happens.
 */
no_holes = btrfs_fs_incompat(root->fs_info, NO_HOLES);
ret = do_check_fs_roots(info, &root_cache);
  ^^^
err |= !!ret;
if (ret) {
error("errors found in fs roots");
goto out;
}

-- 
Thanks,
Lu

>> found 63691210752 bytes used, error(s) found
>> total csum bytes: 61339388
>> total tree bytes: 860540928
>> total fs tree bytes: 688816128
>> total extent tree bytes: 84549632
>> btree space waste bytes: 224428280
>> file data blocks allocated: 62890483712
>>  referenced 62828957696
>> 


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Qu Wenruo



On 2018年01月15日 09:47, Johannes Ernst wrote:
> INFO: task systemd-journal:20876 blocked for more than 120 seconds.
> [ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
> [ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" 
> disables this message.
> [ 5037.962616] systemd-journal D0 20876  20860 0x0100
> [ 5037.962622] Call Trace:
> [ 5037.962635]  ? __schedule+0x290/0x890
> [ 5037.962640]  ? __slab_free+0x14e/0x300
> [ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
> [ 5037.962651]  schedule+0x2f/0x90
> [ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]

 Still some tree write is blocking all incoming read.

 BTW, did your tests include any send and relocation operations?

 IIRC there is a bug that makes send and relocation (I'm not sure though)
 will cause similar problem.
>>>
>>> No. I’m doing nothing btrfs-specific at all, it’s just whatever 
>>> systemd-nspawn —ephemeral does.
>>
>> So, only systemd-nspwan --ephemeral, and nothing else at all?
>>
>> Even nothing happened inside the container?
> 
> 
> Gotcha. (Sorry for missing that)
> 
> Inside the container, snapper might be invoked. Here are the snippets:
> 
> snapper -c root create-config -t ubos-default /
> snapper setup-quota
> snapper -c root create --type pre --print-number
> snapper -c root create --type post --pre-number 
> 
> Here is the snapper config:
> 
> # Snapper default configuration for UBOS.
> #
> # subvolume to snapshot
> SUBVOLUME="/"
> 
> # filesystem type
> FSTYPE="btrfs"
> 
> # btrfs qgroup for space aware cleanup algorithms
> QGROUP=""

What about disabling qgroup here?

And since I'm not familiar with snapper, I'm not pretty sure what it
will do.

But what about disabling snapper and try again to see if the btrfs hangs?

Thanks,
Qu

> 
> # fraction of the filesystems space the snapshots may use
> SPACE_LIMIT="0.2"
> 
> # users and groups allowed to work with config
> ALLOW_USERS=""
> ALLOW_GROUPS=""
> 
> # sync users and groups from ALLOW_USERS and ALLOW_GROUPS to .snapshots
> # directory
> SYNC_ACL="no"
> 
> # start comparing pre- and post-snapshot in background after creating
> # post-snapshot
> BACKGROUND_COMPARISON="no"
> 
> # run daily number cleanup
> NUMBER_CLEANUP="yes"
> 
> # limit for number cleanup; unit is seconds
> NUMBER_MIN_AGE="1800"
> NUMBER_LIMIT="50"
> NUMBER_LIMIT_IMPORTANT="10"
> 
> # create hourly snapshots
> TIMELINE_CREATE="no"
> 
> # cleanup hourly snapshots after some time
> TIMELINE_CLEANUP="yes"
> 
> # limits for timeline cleanup; unit is seconds
> TIMELINE_MIN_AGE="1800"
> TIMELINE_LIMIT_HOURLY="12"
> TIMELINE_LIMIT_DAILY="7"
> TIMELINE_LIMIT_WEEKLY="0"
> TIMELINE_LIMIT_MONTHLY="6"
> TIMELINE_LIMIT_YEARLY="0"
> 
> # cleanup empty pre-post-pairs
> EMPTY_PRE_POST_CLEANUP="yes"
> 
> # limits for empty pre-post-pair cleanup
> EMPTY_PRE_POST_MIN_AGE="1800"
> 
> 
> 
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



signature.asc
Description: OpenPGP digital signature

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Johannes Ernst

 INFO: task systemd-journal:20876 blocked for more than 120 seconds.
 [ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
 [ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
 this message.
 [ 5037.962616] systemd-journal D0 20876  20860 0x0100
 [ 5037.962622] Call Trace:
 [ 5037.962635]  ? __schedule+0x290/0x890
 [ 5037.962640]  ? __slab_free+0x14e/0x300
 [ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
 [ 5037.962651]  schedule+0x2f/0x90
 [ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
>>> 
>>> Still some tree write is blocking all incoming read.
>>> 
>>> BTW, did your tests include any send and relocation operations?
>>> 
>>> IIRC there is a bug that makes send and relocation (I'm not sure though)
>>> will cause similar problem.
>> 
>> No. I’m doing nothing btrfs-specific at all, it’s just whatever 
>> systemd-nspawn —ephemeral does.
> 
> So, only systemd-nspwan --ephemeral, and nothing else at all?
> 
> Even nothing happened inside the container?


Gotcha. (Sorry for missing that)

Inside the container, snapper might be invoked. Here are the snippets:

snapper -c root create-config -t ubos-default /
snapper setup-quota
snapper -c root create --type pre --print-number
snapper -c root create --type post --pre-number 

Here is the snapper config:

# Snapper default configuration for UBOS.
#
# subvolume to snapshot
SUBVOLUME="/"

# filesystem type
FSTYPE="btrfs"

# btrfs qgroup for space aware cleanup algorithms
QGROUP=""

# fraction of the filesystems space the snapshots may use
SPACE_LIMIT="0.2"

# users and groups allowed to work with config
ALLOW_USERS=""
ALLOW_GROUPS=""

# sync users and groups from ALLOW_USERS and ALLOW_GROUPS to .snapshots
# directory
SYNC_ACL="no"

# start comparing pre- and post-snapshot in background after creating
# post-snapshot
BACKGROUND_COMPARISON="no"

# run daily number cleanup
NUMBER_CLEANUP="yes"

# limit for number cleanup; unit is seconds
NUMBER_MIN_AGE="1800"
NUMBER_LIMIT="50"
NUMBER_LIMIT_IMPORTANT="10"

# create hourly snapshots
TIMELINE_CREATE="no"

# cleanup hourly snapshots after some time
TIMELINE_CLEANUP="yes"

# limits for timeline cleanup; unit is seconds
TIMELINE_MIN_AGE="1800"
TIMELINE_LIMIT_HOURLY="12"
TIMELINE_LIMIT_DAILY="7"
TIMELINE_LIMIT_WEEKLY="0"
TIMELINE_LIMIT_MONTHLY="6"
TIMELINE_LIMIT_YEARLY="0"

# cleanup empty pre-post-pairs
EMPTY_PRE_POST_CLEANUP="yes"

# limits for empty pre-post-pair cleanup
EMPTY_PRE_POST_MIN_AGE="1800"





--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Partionless Btrfs drives now return PTTYPE="PMBR"

2018-01-14 Thread Qu Wenruo



On 2018年01月15日 00:36, evan d wrote:
> I removed two partitionless Btrfs formatted drives from an Arch Linux
> based PC and installed them in another Linux based PC also running
> Arch.  The drives are independent of one another, each containing its
> own Btrfs filesystem.  Both machines are running recent kernels and
> the original host would be running whatever kernel was current in Arch
> on 1 Jan 2018.
> 
> lsblk shows the two drives as partitionless as I'd expect, but
> attempting to mount them returns an error saying they're not
> recognised and btrfs filesystem show /dev/sdX also returns that the
> devices do not contain a Brtfs partition.  1 Jan 2018 was the last
> time I accessed these drives, which were mounted via the first
> machine's fstab.
> 
> blkid returns PTTYPE="PMBR" for both.
> 
> For the current machine:
> uname -a returns:
> Linux ryzen 4.14.13-1-ARCH #1 SMP PREEMPT Wed Jan 10 11:14:50 UTC 2018
> x86_64 GNU/Linux

Nice to see another Ryzen system. :)

> 
> btrfs --version returns:
> btrfs-progs v4.14
> 
> 
> Any ideas what could have gone wrong here?  I'm  hoping that
> reinserting them in the original machine may allow me to access the
> underlying data again, but that machine is in another country at
> present.

btrfs inspect dump-super -Ffa will help us to determine if it's still a
btrfs filesystem.

And it's also recommended to use grep to find the btrfs magic in the
beginning several MB range to ensure there is no offset screwing up fs
detection:

dd if= of=/tmp/output bs=1M count=128
grep -obUaP "\x5F\x42\x48\x52\x66\x53\x5F\x4D" /tmp/output

Thanks,
Qu

> 
> Any help much appreciated.
> 
> Thanks
> Evan
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



signature.asc
Description: OpenPGP digital signature

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Qu Wenruo



On 2018年01月15日 08:58, Johannes Ernst wrote:
>>> INFO: task systemd-journal:20876 blocked for more than 120 seconds.
>>> [ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
>>> [ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
>>> this message.
>>> [ 5037.962616] systemd-journal D0 20876  20860 0x0100
>>> [ 5037.962622] Call Trace:
>>> [ 5037.962635]  ? __schedule+0x290/0x890
>>> [ 5037.962640]  ? __slab_free+0x14e/0x300
>>> [ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
>>> [ 5037.962651]  schedule+0x2f/0x90
>>> [ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
>>
>> Still some tree write is blocking all incoming read.
>>
>> BTW, did your tests include any send and relocation operations?
>>
>> IIRC there is a bug that makes send and relocation (I'm not sure though)
>> will cause similar problem.
> 
> No. I’m doing nothing btrfs-specific at all, it’s just whatever 
> systemd-nspawn —ephemeral does.

So, only systemd-nspwan --ephemeral, and nothing else at all?

Even nothing happened inside the container?

Thanks,
Qu

> 
> Cheers,
> 
> 
> 
> Johannes.--
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



signature.asc
Description: OpenPGP digital signature

Re: invalid files names, btrfs check can't repair it

2018-01-14 Thread Qu Wenruo



On 2018年01月15日 00:38, Sebastian Andrzej Siewior wrote:
> On 2018-01-14 08:36:41 [+0800], Qu Wenruo wrote:
>> Still needs more. (and maybe even more depending on the output)
>>
>> The original mode doesn't report error clear enough, so it would help if
>> --mode=lowmem can be used.
>>
>> # btrfs check --mode=lowmem /dev/sdb4
> 
> ~# btrfs check --mode=lowmem /dev/sdb4
> Checking filesystem on /dev/sdb4
> UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
> checking extents
> checking free space cache
> Wanted bytes 4096, found 8192 for off 159178047488
> Wanted bytes 1073725440, found 8192 for off 159178047488
> cache appears valid but isn't 159178031104
> ERROR: errors found in free space cache

A new bug is exposed for lowmem mode.
It doesn't check fs tree if space cache has any problem.

> found 63691210752 bytes used, error(s) found
> total csum bytes: 61339388
> total tree bytes: 860540928
> total fs tree bytes: 688816128
> total extent tree bytes: 84549632
> btree space waste bytes: 224428280
> file data blocks allocated: 62890483712
>  referenced 62828957696
> 
>> And the tree dump of the affected dir inode would help too:
>>
>> # btrfs inspect dump-tree /dev/sdb4 | grep -C 20 \(57648595
> 
> ~# btrfs inspect dump-tree /dev/sdb4 | grep -C 20 \(57648595
> generation 88831 type 1 (regular)
> extent data disk byte 193705541632 nr 134217728
> extent data offset 0 nr 134217728 ram 134217728
> extent compression 0 (none)
> item 4 key (57022249 EXTENT_DATA 402653184) itemoff 3769 itemsize 53
> generation 88831 type 1 (regular)
> extent data disk byte 190035439616 nr 60600320
> extent data offset 0 nr 60600320 ram 60600320
> extent compression 0 (none)
> item 5 key (57643659 INODE_ITEM 0) itemoff 3609 itemsize 160
> generation 89044 transid 89423 size 2 nbytes 0
> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> sequence 0 flags 0x150be(none)
> atime 1513266994.264310920 (2017-12-14 16:56:34)
> ctime 1513868332.369178014 (2017-12-21 15:58:52)
> mtime 1513868332.369178014 (2017-12-21 15:58:52)
> otime 1513266994.264310920 (2017-12-14 16:56:34)
> item 6 key (57643659 INODE_REF 58477961) itemoff 3598 itemsize 11
> index 6 namelen 1 name: 4
> item 7 key (57643659 DIR_ITEM 4189400016) itemoff 3567 itemsize 31
> location key (57648595 INODE_ITEM 0) type DIR
> transid 89045 data_len 0 name_len 1
> name: d
> item 8 key (57643659 DIR_INDEX 113) itemoff 3536 itemsize 31
> location key (57648595 INODE_ITEM 0) type DIR
> transid 89045 data_len 0 name_len 1
> name: d
> item 9 key (57643872 INODE_ITEM 0) itemoff 3376 itemsize 160
> generation 89044 transid 89699 size 0 nbytes 0
> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> sequence 0 flags 0x603b3(none)
> atime 1513266995.540343055 (2017-12-14 16:56:35)
> ctime 1515602448.66422211 (2018-01-10 17:40:48)
> mtime 1515602448.66422211 (2018-01-10 17:40:48)
> otime 1513266995.540343055 (2017-12-14 16:56:35)
> item 10 key (57643872 INODE_REF 682) itemoff 3363 itemsize 13
> index 58 namelen 3 name: tmp
> item 11 key (57648595 INODE_ITEM 0) itemoff 3203 itemsize 160
> generation 89045 transid 89423 size 8350 nbytes 0
> block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
> sequence 0 flags 0xc90(none)
> atime 1513267009.164686143 (2017-12-14 16:56:49)
> ctime 1513868329.753150507 (2017-12-21 15:58:49)
> mtime 1513868329.753150507 (2017-12-21 15:58:49)
> otime 1513267009.164686143 (2017-12-14 16:56:49)
> item 12 key (57648595 INODE_REF 57643659) itemoff 3192 itemsize 11
> index 113 namelen 1 name: d
> item 13 key (57648595 DIR_ITEM 3331247447) itemoff 3123 itemsize 69
> location key (58472210 INODE_ITEM 0) type FILE
> transid 89418 data_len 0 name_len 8231
> name: 454bf066ddfbf42e0f3b77ea71c82f-878732.oq
> item 14 key (57648595 DIR_ITEM 3363354030) itemoff 3053 itemsize 70
> location key (57923894 INODE_ITEM 0) type DIR_ITEM.33
> transid 89142 data_len 0 name_len 40
> name: 2f3f379b2a3d7499471edb74869efe-1948311.d
> item 15 key (57648595 DIR_INDEX 435) itemoff 2983 itemsize 70
> location key (57923894 INODE_ITEM 0) type FILE

Please run the following command too:

# btrfs inspect dump-tree  | grep -C20 \(57923894

> transid 89142 data_len 0

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Johannes Ernst

>> INFO: task systemd-journal:20876 blocked for more than 120 seconds.
>> [ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
>> [ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
>> this message.
>> [ 5037.962616] systemd-journal D0 20876  20860 0x0100
>> [ 5037.962622] Call Trace:
>> [ 5037.962635]  ? __schedule+0x290/0x890
>> [ 5037.962640]  ? __slab_free+0x14e/0x300
>> [ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
>> [ 5037.962651]  schedule+0x2f/0x90
>> [ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
> 
> Still some tree write is blocking all incoming read.
> 
> BTW, did your tests include any send and relocation operations?
> 
> IIRC there is a bug that makes send and relocation (I'm not sure though)
> will cause similar problem.

No. I’m doing nothing btrfs-specific at all, it’s just whatever systemd-nspawn 
—ephemeral does.

Cheers,



Johannes.--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Qu Wenruo



On 2018年01月15日 04:17, Johannes Ernst wrote:
>> On Jan 13, 2018, at 18:27, Qu Wenruo  wrote:
>> On 2018年01月14日 09:36, Johannes Ernst wrote:
>>> Summary: frequent “hangs” of the system with dmesg saying:
>>>
>>> task systemd:9 blocked for more than 120 seconds.
>>> [ 2948.928653]   Tainted: G C O4.14.9-1-ARCH #1
>>> [ 2948.928658] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
>>> this message.
>>> [ 2948.928665] systemd D0 9  6 0x0100
>>> [ 2948.928671] Call Trace:
>>> [ 2948.928684]  ? __schedule+0x290/0x890
>>> [ 2948.928690]  schedule+0x2f/0x90
>>> [ 2948.928744]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
>>> [ 2948.928752]  ? wait_woken+0x80/0x80
>>> [ 2948.928799]  find_parent_nodes+0x341/0xfd0 [btrfs]
>>> [ 2948.928827]  ? btrfs_search_slot+0x84c/0x9f0 [btrfs]
>>> [ 2948.928873]  ? btrfs_find_all_roots_safe+0x9e/0x110 [btrfs]
>>> [ 2948.928912]  btrfs_find_all_roots_safe+0x9e/0x110 [btrfs]
>>> [ 2948.928950]  btrfs_find_all_roots+0x45/0x60 [btrfs]
>>> [ 2948.928987]  btrfs_qgroup_trace_extent_post+0x30/0x60 [btrfs]
>>
>> You're using qgroup, and the timing is to find the old_roots of one
>> extent, which will only search commit roots.
>>
>> Normally this shouldn't cause any problem, especially for commit roots.
>>
>> Is there any special operation happening?
> 
> Nope. It appears it happens right when systemd-nspawn begins to run and I am 
> not executing any other commands.
> 
> I did not realize qgroups are involved … all I did is mkfs.btrfs and running 
> systemd-nspawn :-) Perhaps the defaults should be qgroups off? (But I digress)
> 
>>> That works well … but not for long. Often we don’t make it through the test 
>>> suite and the starting of new containers hangs. Other filesystem operations 
>>> also hang. The above stacktrace, or something rather similar shows up in 
>>> dmesg (not in the journal, because that hangs, too!) This is repeated, but 
>>> I don’t see any other relevant messages. Only a reboot seems to allows to 
>>> recover.
>>
>> So Qgroup is used to limit disk usage of each container?
>>
>> Maybe it's related to subvolume deletion?
>>
>> Anyway, if qgroup is not necessary, disabling qgroup should fix your
>> problem.
>>
>> Despite of that, did that really hangs?
>> Qgroup dramatically increase overhead to delete a subvolume or balance
>> the fs.
>> Maybe it's just a little slow?
> 
> I have waited for several hours and the system has not recovered (me walking 
> away from the running tests, returning hours later).
> 
> Update: so I executed "btrfs quota disable” on all relevant file systems. 
> (right?) Running tests again, this morning I’m getting:

So not a bug for quota. (At least I have less thing to worry about)

> 
> INFO: task systemd-journal:20876 blocked for more than 120 seconds.
> [ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
> [ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
> this message.
> [ 5037.962616] systemd-journal D0 20876  20860 0x0100
> [ 5037.962622] Call Trace:
> [ 5037.962635]  ? __schedule+0x290/0x890
> [ 5037.962640]  ? __slab_free+0x14e/0x300
> [ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
> [ 5037.962651]  schedule+0x2f/0x90
> [ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]

Still some tree write is blocking all incoming read.

BTW, did your tests include any send and relocation operations?

IIRC there is a bug that makes send and relocation (I'm not sure though)
will cause similar problem.

Thanks,
Qu

> [ 5037.962713]  ? wait_woken+0x80/0x80
> [ 5037.962739]  btrfs_read_lock_root_node+0x2f/0x40 [btrfs]
> [ 5037.962767]  btrfs_search_slot+0x703/0x9f0 [btrfs]
> [ 5037.962796]  btrfs_insert_empty_items+0x66/0xb0 [btrfs]
> [ 5037.962841]  btrfs_insert_orphan_item+0x66/0xa0 [btrfs]
> [ 5037.962880]  btrfs_orphan_add+0xa1/0x200 [btrfs]
> [ 5037.962919]  btrfs_setattr+0x123/0x3b0 [btrfs]
> [ 5037.962926]  notify_change+0x2fd/0x420
> [ 5037.962933]  do_truncate+0x75/0xc0
> [ 5037.962940]  do_sys_ftruncate.constprop.19+0xe7/0x100
> [ 5037.962947]  do_syscall_64+0x55/0x110
> [ 5037.962952]  entry_SYSCALL64_slow_path+0x25/0x25
> [ 5037.962956] RIP: 0033:0x7fd9423697ba
> [ 5037.962958] RSP: 002b:7fff1179cc18 EFLAGS: 0206 ORIG_RAX: 
> 004d
> [ 5037.962962] RAX: ffda RBX: 55bd48cbe9f0 RCX: 
> 7fd9423697ba
> [ 5037.962965] RDX: 55bd48cbe660 RSI: 0064 RDI: 
> 000f
> [ 5037.962966] RBP: 7fff1179cc50 R08: 55bd48cbc62c R09: 
> 55bd48cbea6c
> [ 5037.962968] R10: 55bd48cbe9f0 R11: 0206 R12: 
> 7fff1179cc48
> [ 5037.962970] R13: 0003 R14: 000562b7234e71a9 R15: 
> 55bd47749ca0
> 
> and doing a simple “touch /build/tmp/foo” never returns. 10+ hours have 
> passed since the previous command was issued.
> 
> 
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Johannes Ernst

> On Jan 13, 2018, at 18:27, Qu Wenruo  wrote:
> On 2018年01月14日 09:36, Johannes Ernst wrote:
>> Summary: frequent “hangs” of the system with dmesg saying:
>> 
>> task systemd:9 blocked for more than 120 seconds.
>> [ 2948.928653]   Tainted: G C O4.14.9-1-ARCH #1
>> [ 2948.928658] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
>> this message.
>> [ 2948.928665] systemd D0 9  6 0x0100
>> [ 2948.928671] Call Trace:
>> [ 2948.928684]  ? __schedule+0x290/0x890
>> [ 2948.928690]  schedule+0x2f/0x90
>> [ 2948.928744]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
>> [ 2948.928752]  ? wait_woken+0x80/0x80
>> [ 2948.928799]  find_parent_nodes+0x341/0xfd0 [btrfs]
>> [ 2948.928827]  ? btrfs_search_slot+0x84c/0x9f0 [btrfs]
>> [ 2948.928873]  ? btrfs_find_all_roots_safe+0x9e/0x110 [btrfs]
>> [ 2948.928912]  btrfs_find_all_roots_safe+0x9e/0x110 [btrfs]
>> [ 2948.928950]  btrfs_find_all_roots+0x45/0x60 [btrfs]
>> [ 2948.928987]  btrfs_qgroup_trace_extent_post+0x30/0x60 [btrfs]
> 
> You're using qgroup, and the timing is to find the old_roots of one
> extent, which will only search commit roots.
> 
> Normally this shouldn't cause any problem, especially for commit roots.
> 
> Is there any special operation happening?

Nope. It appears it happens right when systemd-nspawn begins to run and I am 
not executing any other commands.

I did not realize qgroups are involved … all I did is mkfs.btrfs and running 
systemd-nspawn :-) Perhaps the defaults should be qgroups off? (But I digress)

>> That works well … but not for long. Often we don’t make it through the test 
>> suite and the starting of new containers hangs. Other filesystem operations 
>> also hang. The above stacktrace, or something rather similar shows up in 
>> dmesg (not in the journal, because that hangs, too!) This is repeated, but I 
>> don’t see any other relevant messages. Only a reboot seems to allows to 
>> recover.
> 
> So Qgroup is used to limit disk usage of each container?
> 
> Maybe it's related to subvolume deletion?
> 
> Anyway, if qgroup is not necessary, disabling qgroup should fix your
> problem.
> 
> Despite of that, did that really hangs?
> Qgroup dramatically increase overhead to delete a subvolume or balance
> the fs.
> Maybe it's just a little slow?

I have waited for several hours and the system has not recovered (me walking 
away from the running tests, returning hours later).

Update: so I executed "btrfs quota disable” on all relevant file systems. 
(right?) Running tests again, this morning I’m getting:

INFO: task systemd-journal:20876 blocked for more than 120 seconds.
[ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
[ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
[ 5037.962616] systemd-journal D0 20876  20860 0x0100
[ 5037.962622] Call Trace:
[ 5037.962635]  ? __schedule+0x290/0x890
[ 5037.962640]  ? __slab_free+0x14e/0x300
[ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
[ 5037.962651]  schedule+0x2f/0x90
[ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
[ 5037.962713]  ? wait_woken+0x80/0x80
[ 5037.962739]  btrfs_read_lock_root_node+0x2f/0x40 [btrfs]
[ 5037.962767]  btrfs_search_slot+0x703/0x9f0 [btrfs]
[ 5037.962796]  btrfs_insert_empty_items+0x66/0xb0 [btrfs]
[ 5037.962841]  btrfs_insert_orphan_item+0x66/0xa0 [btrfs]
[ 5037.962880]  btrfs_orphan_add+0xa1/0x200 [btrfs]
[ 5037.962919]  btrfs_setattr+0x123/0x3b0 [btrfs]
[ 5037.962926]  notify_change+0x2fd/0x420
[ 5037.962933]  do_truncate+0x75/0xc0
[ 5037.962940]  do_sys_ftruncate.constprop.19+0xe7/0x100
[ 5037.962947]  do_syscall_64+0x55/0x110
[ 5037.962952]  entry_SYSCALL64_slow_path+0x25/0x25
[ 5037.962956] RIP: 0033:0x7fd9423697ba
[ 5037.962958] RSP: 002b:7fff1179cc18 EFLAGS: 0206 ORIG_RAX: 
004d
[ 5037.962962] RAX: ffda RBX: 55bd48cbe9f0 RCX: 7fd9423697ba
[ 5037.962965] RDX: 55bd48cbe660 RSI: 0064 RDI: 000f
[ 5037.962966] RBP: 7fff1179cc50 R08: 55bd48cbc62c R09: 55bd48cbea6c
[ 5037.962968] R10: 55bd48cbe9f0 R11: 0206 R12: 7fff1179cc48
[ 5037.962970] R13: 0003 R14: 000562b7234e71a9 R15: 55bd47749ca0

and doing a simple “touch /build/tmp/foo” never returns. 10+ hours have passed 
since the previous command was issued.


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Johannes Ernst

> On Jan 13, 2018, at 18:27, Qu Wenruo  wrote:
> On 2018年01月14日 09:36, Johannes Ernst wrote:
>> Summary: frequent “hangs” of the system with dmesg saying:
>> 
>> task systemd:9 blocked for more than 120 seconds.
>> [ 2948.928653]   Tainted: G C O4.14.9-1-ARCH #1
>> [ 2948.928658] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables 
>> this message.
>> [ 2948.928665] systemd D0 9  6 0x0100
>> [ 2948.928671] Call Trace:
>> [ 2948.928684]  ? __schedule+0x290/0x890
>> [ 2948.928690]  schedule+0x2f/0x90
>> [ 2948.928744]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
>> [ 2948.928752]  ? wait_woken+0x80/0x80
>> [ 2948.928799]  find_parent_nodes+0x341/0xfd0 [btrfs]
>> [ 2948.928827]  ? btrfs_search_slot+0x84c/0x9f0 [btrfs]
>> [ 2948.928873]  ? btrfs_find_all_roots_safe+0x9e/0x110 [btrfs]
>> [ 2948.928912]  btrfs_find_all_roots_safe+0x9e/0x110 [btrfs]
>> [ 2948.928950]  btrfs_find_all_roots+0x45/0x60 [btrfs]
>> [ 2948.928987]  btrfs_qgroup_trace_extent_post+0x30/0x60 [btrfs]
> 
> You're using qgroup, and the timing is to find the old_roots of one
> extent, which will only search commit roots.
> 
> Normally this shouldn't cause any problem, especially for commit roots.
> 
> Is there any special operation happening?

Nope. It appears it happens right when systemd-nspawn begins to run and I am 
not executing any other commands.

I did not realize qgroups are involved … all I did is mkfs.btrfs and running 
systemd-nspawn :-) Perhaps the defaults should be qgroups off? (But I digress)

>> That works well … but not for long. Often we don’t make it through the test 
>> suite and the starting of new containers hangs. Other filesystem operations 
>> also hang. The above stacktrace, or something rather similar shows up in 
>> dmesg (not in the journal, because that hangs, too!) This is repeated, but I 
>> don’t see any other relevant messages. Only a reboot seems to allows to 
>> recover.
> 
> So Qgroup is used to limit disk usage of each container?
> 
> Maybe it's related to subvolume deletion?
> 
> Anyway, if qgroup is not necessary, disabling qgroup should fix your
> problem.
> 
> Despite of that, did that really hangs?
> Qgroup dramatically increase overhead to delete a subvolume or balance
> the fs.
> Maybe it's just a little slow?

I have waited for several hours and the system has not recovered (me walking 
away from the running tests, returning hours later).

Update: so I executed "btrfs quota disable” on all relevant file systems. 
(right?) Running tests again, this morning I’m getting:

INFO: task systemd-journal:20876 blocked for more than 120 seconds.
[ 5037.962603]   Tainted: G C O4.14.9-1-ARCH #1
[ 5037.962609] "echo 0 > /proc/sys/kernel/hung_task_timeout_secs" disables this 
message.
[ 5037.962616] systemd-journal D0 20876  20860 0x0100
[ 5037.962622] Call Trace:
[ 5037.962635]  ? __schedule+0x290/0x890
[ 5037.962640]  ? __slab_free+0x14e/0x300
[ 5037.962645]  ? _copy_to_iter+0x8f/0x3d0
[ 5037.962651]  schedule+0x2f/0x90
[ 5037.962704]  btrfs_tree_read_lock+0xb6/0x100 [btrfs]
[ 5037.962713]  ? wait_woken+0x80/0x80
[ 5037.962739]  btrfs_read_lock_root_node+0x2f/0x40 [btrfs]
[ 5037.962767]  btrfs_search_slot+0x703/0x9f0 [btrfs]
[ 5037.962796]  btrfs_insert_empty_items+0x66/0xb0 [btrfs]
[ 5037.962841]  btrfs_insert_orphan_item+0x66/0xa0 [btrfs]
[ 5037.962880]  btrfs_orphan_add+0xa1/0x200 [btrfs]
[ 5037.962919]  btrfs_setattr+0x123/0x3b0 [btrfs]
[ 5037.962926]  notify_change+0x2fd/0x420
[ 5037.962933]  do_truncate+0x75/0xc0
[ 5037.962940]  do_sys_ftruncate.constprop.19+0xe7/0x100
[ 5037.962947]  do_syscall_64+0x55/0x110
[ 5037.962952]  entry_SYSCALL64_slow_path+0x25/0x25
[ 5037.962956] RIP: 0033:0x7fd9423697ba
[ 5037.962958] RSP: 002b:7fff1179cc18 EFLAGS: 0206 ORIG_RAX: 
004d
[ 5037.962962] RAX: ffda RBX: 55bd48cbe9f0 RCX: 7fd9423697ba
[ 5037.962965] RDX: 55bd48cbe660 RSI: 0064 RDI: 000f
[ 5037.962966] RBP: 7fff1179cc50 R08: 55bd48cbc62c R09: 55bd48cbea6c
[ 5037.962968] R10: 55bd48cbe9f0 R11: 0206 R12: 7fff1179cc48
[ 5037.962970] R13: 0003 R14: 000562b7234e71a9 R15: 55bd47749ca0

and doing a simple “touch /build/tmp/foo” never returns. 10+ hours have passed 
since the previous command was issued.


--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: invalid files names, btrfs check can't repair it

2018-01-14 Thread Sebastian Andrzej Siewior

On 2018-01-14 08:36:41 [+0800], Qu Wenruo wrote:
> Still needs more. (and maybe even more depending on the output)
> 
> The original mode doesn't report error clear enough, so it would help if
> --mode=lowmem can be used.
> 
> # btrfs check --mode=lowmem /dev/sdb4

~# btrfs check --mode=lowmem /dev/sdb4
Checking filesystem on /dev/sdb4
UUID: b3bfb56e-d445-4335-93f0-c1fb2d1f6df1
checking extents
checking free space cache
Wanted bytes 4096, found 8192 for off 159178047488
Wanted bytes 1073725440, found 8192 for off 159178047488
cache appears valid but isn't 159178031104
ERROR: errors found in free space cache
found 63691210752 bytes used, error(s) found
total csum bytes: 61339388
total tree bytes: 860540928
total fs tree bytes: 688816128
total extent tree bytes: 84549632
btree space waste bytes: 224428280
file data blocks allocated: 62890483712
 referenced 62828957696

> And the tree dump of the affected dir inode would help too:
> 
> # btrfs inspect dump-tree /dev/sdb4 | grep -C 20 \(57648595

~# btrfs inspect dump-tree /dev/sdb4 | grep -C 20 \(57648595
generation 88831 type 1 (regular)
extent data disk byte 193705541632 nr 134217728
extent data offset 0 nr 134217728 ram 134217728
extent compression 0 (none)
item 4 key (57022249 EXTENT_DATA 402653184) itemoff 3769 itemsize 53
generation 88831 type 1 (regular)
extent data disk byte 190035439616 nr 60600320
extent data offset 0 nr 60600320 ram 60600320
extent compression 0 (none)
item 5 key (57643659 INODE_ITEM 0) itemoff 3609 itemsize 160
generation 89044 transid 89423 size 2 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x150be(none)
atime 1513266994.264310920 (2017-12-14 16:56:34)
ctime 1513868332.369178014 (2017-12-21 15:58:52)
mtime 1513868332.369178014 (2017-12-21 15:58:52)
otime 1513266994.264310920 (2017-12-14 16:56:34)
item 6 key (57643659 INODE_REF 58477961) itemoff 3598 itemsize 11
index 6 namelen 1 name: 4
item 7 key (57643659 DIR_ITEM 4189400016) itemoff 3567 itemsize 31
location key (57648595 INODE_ITEM 0) type DIR
transid 89045 data_len 0 name_len 1
name: d
item 8 key (57643659 DIR_INDEX 113) itemoff 3536 itemsize 31
location key (57648595 INODE_ITEM 0) type DIR
transid 89045 data_len 0 name_len 1
name: d
item 9 key (57643872 INODE_ITEM 0) itemoff 3376 itemsize 160
generation 89044 transid 89699 size 0 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0x603b3(none)
atime 1513266995.540343055 (2017-12-14 16:56:35)
ctime 1515602448.66422211 (2018-01-10 17:40:48)
mtime 1515602448.66422211 (2018-01-10 17:40:48)
otime 1513266995.540343055 (2017-12-14 16:56:35)
item 10 key (57643872 INODE_REF 682) itemoff 3363 itemsize 13
index 58 namelen 3 name: tmp
item 11 key (57648595 INODE_ITEM 0) itemoff 3203 itemsize 160
generation 89045 transid 89423 size 8350 nbytes 0
block group 0 mode 40755 links 1 uid 1000 gid 1000 rdev 0
sequence 0 flags 0xc90(none)
atime 1513267009.164686143 (2017-12-14 16:56:49)
ctime 1513868329.753150507 (2017-12-21 15:58:49)
mtime 1513868329.753150507 (2017-12-21 15:58:49)
otime 1513267009.164686143 (2017-12-14 16:56:49)
item 12 key (57648595 INODE_REF 57643659) itemoff 3192 itemsize 11
index 113 namelen 1 name: d
item 13 key (57648595 DIR_ITEM 3331247447) itemoff 3123 itemsize 69
location key (58472210 INODE_ITEM 0) type FILE
transid 89418 data_len 0 name_len 8231
name: 454bf066ddfbf42e0f3b77ea71c82f-878732.oq
item 14 key (57648595 DIR_ITEM 3363354030) itemoff 3053 itemsize 70
location key (57923894 INODE_ITEM 0) type DIR_ITEM.33
transid 89142 data_len 0 name_len 40
name: 2f3f379b2a3d7499471edb74869efe-1948311.d
item 15 key (57648595 DIR_INDEX 435) itemoff 2983 itemsize 70
location key (57923894 INODE_ITEM 0) type FILE
transid 89142 data_len 0 name_len 40
name: 2f3f379b2a3d7499471edb74869efe-1948311.d
item 16 key (57648595 DIR_INDEX 1137) itemoff 2914 itemsize 69
location key (58472210 INODE_ITEM 0) type FILE
transid 89418 data_len 0 name_len 39
name: 454bf066ddfbf42e0f3b77ea71c82f-878732.o
item 17 key (57923894 INODE_ITEM 0) itemoff 2754 itemsize 160

Partionless Btrfs drives now return PTTYPE="PMBR"

2018-01-14 Thread evan d

I removed two partitionless Btrfs formatted drives from an Arch Linux
based PC and installed them in another Linux based PC also running
Arch.  The drives are independent of one another, each containing its
own Btrfs filesystem.  Both machines are running recent kernels and
the original host would be running whatever kernel was current in Arch
on 1 Jan 2018.

lsblk shows the two drives as partitionless as I'd expect, but
attempting to mount them returns an error saying they're not
recognised and btrfs filesystem show /dev/sdX also returns that the
devices do not contain a Brtfs partition.  1 Jan 2018 was the last
time I accessed these drives, which were mounted via the first
machine's fstab.

blkid returns PTTYPE="PMBR" for both.

For the current machine:
uname -a returns:
Linux ryzen 4.14.13-1-ARCH #1 SMP PREEMPT Wed Jan 10 11:14:50 UTC 2018
x86_64 GNU/Linux

btrfs --version returns:
btrfs-progs v4.14


Any ideas what could have gone wrong here?  I'm  hoping that
reinserting them in the original machine may allow me to access the
underlying data again, but that machine is in another country at
present.

Any help much appreciated.

Thanks
Evan
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Hanging after frequent use of systemd-nspawn --ephemeral

2018-01-14 Thread Duncan

Qu Wenruo posted on Sun, 14 Jan 2018 10:27:40 +0800 as excerpted:

> Despite of that, did that really hangs?
> Qgroup dramatically increase overhead to delete a subvolume or balance
> the fs.
> Maybe it's just a little slow?

Same question about the "hang" here.

Note that btrfs is optimized to make snapshot creation fast, while 
snapshot deletion has to do more work to clean things up.  So even 
without qgroup enabled, deletion can take a bit of time (much longer than 
creation, which should be nearly instantaneous in human terms) if there's 
a lot of relinks and the like to clean up.

And qgroups makes btrfs do much more work to track that as well, so as Qu 
says, that'll make snapshot deletion take even longer, and you probably 
want it disabled unless you actually need the feature for something 
you're doing.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Limit on the number of btrfs snapshots?

2018-01-14 Thread Duncan

Daniel E. Shub posted on Fri, 12 Jan 2018 16:38:30 -0500 as excerpted:

> A couple of years ago I asked a question on the Unix and Linux Stack
> Exchange about the limit on the number of BTRFS snapshots:
> https://unix.stackexchange.com/q/140360/22724
> 
> Basically, I want to use something like snapper to take time based
> snapshots so that I can browse old versions of my data. This would be in
> addition to my current off site backup since a drive failure would wipe
> out the data and the snapshots. Is there a limit to the number of
> snapshots I can take and store? If I have a million snapshots (e.g., a
> snapshot every minute for two years) would that cause havoc, assuming I
> have enough disk space for the data, the changed data, and the meta
> data?
> 
> The answers there provided a link to the wiki:
> https://btrfs.wiki.kernel.org/index.php/
Btrfs_design#Snapshots_and_Subvolumes
> that says: "snapshots are writable, and they can be snapshotted again
> any number of times."
> 
> While I don't doubt that that is technically true, another user
> suggested that the practical limit is around 100 snapshots.
> 
> While I am not convinced that having minute-by-minute versions of my
> data for two years is helpful (how the hell is anyone going to find the
> exact minute they are looking for), if there is no cost then I figure
> why not.
> 
> I guess I am asking is what is the story and where is it documented.

Very logical question. =:^)

The (practical) answer depends to some extent on how you use btrfs.

Btrfs does have scaling issues due to too many snapshots (or actually the 
reflinks snapshots use, dedup using reflinks can trigger the same scaling 
issues), and single to low double-digits of snapshots per snapshotted 
subvolume remains the strong recommendation for that reason.

But the scaling issues primarily affect btrfs maintenance commands 
themselves, balance, check, subvolume delete.  While millions of 
snapshots will make balance for example effectively unworkable (it'll 
sort of work but could take months), normal filesystem operations like 
reading and saving files doesn't tend to be affected, except to the 
extent that fragmentation becomes an issue (tho cow filesystems such as 
btrfs are noted for fragmentation, unless steps like defrag are taken to 
reduce it).

So for home and SOHO type usage where you might for instance want to add 
a device to the filesystem and rebalance to make full use of it, and 
where when a filesystem won't mount you are likely to want to run btrfs 
check to try to fix it, a max of 100 or so snapshots per subvolume is 
indeed a strong recommendation.

But in large business/corporate environments where there's hot-spare 
standbys to fail-over to and three-way offsite backups of the hot-spare 
and onsite backups, it's not such a big issue, because rather than 
balancing or fscking, such usage generally just fail-overs to the backups 
and recycles the previous working filesystem devices, so a balance or a 
check taking three years isn't an issue because they don't tend to run 
those sorts of commands in the first place.

-- 
Duncan - List replies preferred.   No HTML msgs.
"Every nonfree program has a lord, a master --
and if you use the program, he is your master."  Richard Stallman

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-14 Thread Ming Lei

On Sun, Jan 14, 2018 at 11:12 PM, jianchao.wang
 wrote:
>
>
> On 01/13/2018 05:19 AM, Bart Van Assche wrote:
>> Sorry but I only retrieved the blk-mq debugfs several minutes after the hang
>> started so I'm not sure the state information is relevant. Anyway, I have 
>> attached
>> it to this e-mail. The most remarkable part is the following:
>>
>> ./9ddfa913/requeue_list:9646711c {.op=READ, .state=idle, 
>> gen=0x1
>> 18, abort_gen=0x0, .cmd_flags=, .rq_flags=SORTED|1|SOFTBARRIER|IO_STAT, 
>> complete
>> =0, .tag=-1, .internal_tag=217}
>>
>> The hexadecimal number at the start is the request_queue pointer (I modified 
>> the
>> blk-mq-debugfs code such that queues are registered with there address just 
>> after
>> creation and until a name is assigned). This is a dm-mpath queue.
>
> There seems to be something wrong in hctx->nr_active.

Then looks it is same with the issue I saw during starting multipathd, and the
following patch should fix that, if there isn't other issue.

https://marc.info/?l=linux-block&m=151586577400558&w=2

-- 
Ming Lei
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

2018-01-14 Thread jianchao.wang



On 01/13/2018 05:19 AM, Bart Van Assche wrote:
> Sorry but I only retrieved the blk-mq debugfs several minutes after the hang
> started so I'm not sure the state information is relevant. Anyway, I have 
> attached
> it to this e-mail. The most remarkable part is the following:
> 
> ./9ddfa913/requeue_list:9646711c {.op=READ, .state=idle, 
> gen=0x1
> 18, abort_gen=0x0, .cmd_flags=, .rq_flags=SORTED|1|SOFTBARRIER|IO_STAT, 
> complete
> =0, .tag=-1, .internal_tag=217}
> 
> The hexadecimal number at the start is the request_queue pointer (I modified 
> the
> blk-mq-debugfs code such that queues are registered with there address just 
> after
> creation and until a name is assigned). This is a dm-mpath queue.

There seems to be something wrong in hctx->nr_active.
./sde/hctx2/cpu2/completed:2 3
./sde/hctx2/cpu2/merged:0
./sde/hctx2/cpu2/dispatched:2 3
./sde/hctx2/active:5

./sde/hctx1/cpu1/completed:2 38
./sde/hctx1/cpu1/merged:0
./sde/hctx1/cpu1/dispatched:2 38
./sde/hctx1/active:40

./sde/hctx0/cpu0/completed:20 11
./sde/hctx0/cpu0/merged:0
./sde/hctx0/cpu0/dispatched:20 11
./sde/hctx0/active:31
...
./sdc/hctx1/cpu1/completed:14 13
./sdc/hctx1/cpu1/merged:0
./sdc/hctx1/cpu1/dispatched:14 13
./sdc/hctx1/active:21

./sdc/hctx0/cpu0/completed:1 41
./sdc/hctx0/cpu0/merged:0
./sdc/hctx0/cpu0/dispatched:1 41
./sdc/hctx0/active:36

Then hctx_may_queue return false.

Thanks
Jianchao
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Qu Wenruo



On 2018年01月14日 18:32, Ilan Schwarts wrote:
> Thank you for clarification.
> Just 2 quick questions,
> 1. Sub volumes - 2 sub volumes cannot have 2 same inode numbers ?

They can.

So to really locate an inode in btrfs, you need:

fsid (locate the fs) -> subvolume id (locate subvolume) -> inode number.

fsid can be feteched from superblock as mentioned in previous reply.

subvolume id can be get from BTRFS_I(inode)->root.
And normally root is what you need.

If you really want the number, then either
BTRFS_I(inode)->root->objectid or
BTRFS_I(inode)->root->root_key->objectid will give you the u64 subvolume id.

> 2. Why fsInfo fsid return u8 and the traditional file system return
> dev_t, usually 32 integer ?

As far as I found in xfs or ext4, their fsid is still u8[16] or uuid_t,
same as btrfs.

For ext4 it's ext4_super_block->s_uuid[16]
And for xfs, it's xfs_sb->sb_uuid.

I don't know how you get the dev_t parameter.

Thanks,
Qu

> 
> 
> On Sun, Jan 14, 2018 at 12:22 PM, Qu Wenruo  wrote:
>>
>>
>> On 2018年01月14日 18:13, Ilan Schwarts wrote:
>>> both btrfs filesystems will have same fsid ?
>>>
>>>
>>> On Sun, Jan 14, 2018 at 12:06 PM, Ilan Schwarts  wrote:
 But both filesystems will have same fsid?

 On Jan 14, 2018 12:04, "Nikolay Borisov"  wrote:
>
>
>
> On 14.01.2018 12:02, Ilan Schwarts wrote:
>> First of all, Thanks for response !
>> So if i have 2 btrfs file system on the same machine (not your
>> everyday scenario, i know)
>>
>> Not a problem, the 2 filesystems will have 2 different fsid.
>>
>> (And it's my everyday scenario, since fstests neeeds TEST_DEV and
>> SCRATCH_DEV_POOL)
>>
>> Lets say a file is created on device A, the file gets inode number X
>> is it possible on device B to have inode number X also ?
>> or each device has its own Inode number range ?
>>
>> Forget the mess about device.
>>
>> Inode is bounded to a filesystem, not bounded to a device.
>>
>> Just traditional filesytems are normally bounded to a single device.
>> (Although even traditional filesystems can have external journal devices)
>>
>> So there is nothing to do with device at all.
>>
>> And you can have same inode numbers in different filesystems, but
>> BTRFS_I(inode)->root->fs_info will point to different fs_infos, with
>> different fsid.
>>
>> So return to your initial question:
>>> both btrfs filesystems will have same fsid ?
>>
>> No, different filesystems will have different fsid.
>>
>> (Unless you're SUUUPER lucky to have 2 filesystems with
>> same fsid)
>>
>> Thanks,
>> Qu
>>
>>
>
> Of course it is possible. Inodes are guaranteed to be unique only across
> filesystem instances. In your case you are going to have 2 fs instances.
>
>>
>> I need to create unique identifier for a file, I need to understand if
>> the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
>> is enough.
>>
>> Thanks
>>
>>
>>
>>
>>
>> On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo 
>> wrote:
>>>
>>>
>>> On 2018年01月14日 16:33, Ilan Schwarts wrote:
 Hello btrfs developers/users,

 I was wondering regarding to fetching the correct fsid on btrfs from
 the context of a kernel module.
>>>
>>> There are two IDs for btrfs. (in fact more, but you properly won't need
>>> the extra ids)
>>>
>>> FSID: Global one, one fs one FSID.
>>> Device ID: Bonded to device, each device will have one.
>>>
>>> So in case of 2 devices btrfs, each device will has its own device id,
>>> while both of the devices have the same fsid.
>>>
>>> And I think you're talking about the global fsid instead of device id.
>>>
 if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
 do the following:
 convert inode struct to btrfs_inode struct (use btrfsInode =
 BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
 from root i take anon_dev or anon_super.s_dev.
 struct btrfs_inode *btrfsInode;
 btrfsInode = BTRFS_I(inode);
btrfsInode->root->anon_super.s_devor
btrfsInode->root->anon_dev- depend on kernel.
>>>
>>> The most directly method would be:
>>>
>>> btrfs_inode->root->fs_info->fsid.
>>> (For newer kernel, as I'm not familiar with older kernels)
>>>
>>> Or from superblock:
>>> btrfs_inode->root->fs_info->super_copy->fsid.
>>> (The most reliable one, no matter which kernel version you're using, as
>>> long as the super block format didn't change)
>>>
>>> For device id, it's not that commonly used unless you're dealing with
>>> chunk mapping, so I'm assuming you're referring to fsid.
>>>
>>> Thanks,
>>> Qu
>>>

 In kernel 3.12.28-4-default in order to get the fsid, i need to go
 to the inode -> superblock -> device

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Hugo Mills

On Sun, Jan 14, 2018 at 12:32:25PM +0200, Ilan Schwarts wrote:
> Thank you for clarification.
> Just 2 quick questions,
> 1. Sub volumes - 2 sub volumes cannot have 2 same inode numbers ?

   Incorrect. You can have two subvolumes of the same filesystem, and
you can have files with the same inode number in each subvolume. Each
subvolume has its own inode number space. So an inode number on its
own is not enough to uniquely identify a file -- you also need the
subvolid to uniquely identify a specific file in the filesystem.

   Hugo.

> 2. Why fsInfo fsid return u8 and the traditional file system return
> dev_t, usually 32 integer ?
> 
> 
> On Sun, Jan 14, 2018 at 12:22 PM, Qu Wenruo  wrote:
> >
> >
> > On 2018年01月14日 18:13, Ilan Schwarts wrote:
> >> both btrfs filesystems will have same fsid ?
> >>
> >>
> >> On Sun, Jan 14, 2018 at 12:06 PM, Ilan Schwarts  wrote:
> >>> But both filesystems will have same fsid?
> >>>
> >>> On Jan 14, 2018 12:04, "Nikolay Borisov"  wrote:
> 
> 
> 
>  On 14.01.2018 12:02, Ilan Schwarts wrote:
> > First of all, Thanks for response !
> > So if i have 2 btrfs file system on the same machine (not your
> > everyday scenario, i know)
> >
> > Not a problem, the 2 filesystems will have 2 different fsid.
> >
> > (And it's my everyday scenario, since fstests neeeds TEST_DEV and
> > SCRATCH_DEV_POOL)
> >
> > Lets say a file is created on device A, the file gets inode number X
> > is it possible on device B to have inode number X also ?
> > or each device has its own Inode number range ?
> >
> > Forget the mess about device.
> >
> > Inode is bounded to a filesystem, not bounded to a device.
> >
> > Just traditional filesytems are normally bounded to a single device.
> > (Although even traditional filesystems can have external journal devices)
> >
> > So there is nothing to do with device at all.
> >
> > And you can have same inode numbers in different filesystems, but
> > BTRFS_I(inode)->root->fs_info will point to different fs_infos, with
> > different fsid.
> >
> > So return to your initial question:
> >> both btrfs filesystems will have same fsid ?
> >
> > No, different filesystems will have different fsid.
> >
> > (Unless you're SUUUPER lucky to have 2 filesystems with
> > same fsid)
> >
> > Thanks,
> > Qu
> >
> >
> 
>  Of course it is possible. Inodes are guaranteed to be unique only across
>  filesystem instances. In your case you are going to have 2 fs instances.
> 
> >
> > I need to create unique identifier for a file, I need to understand if
> > the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
> > is enough.
> >
> > Thanks
> >
> >
> >
> >
> >
> > On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo 
> > wrote:
> >>
> >>
> >> On 2018年01月14日 16:33, Ilan Schwarts wrote:
> >>> Hello btrfs developers/users,
> >>>
> >>> I was wondering regarding to fetching the correct fsid on btrfs from
> >>> the context of a kernel module.
> >>
> >> There are two IDs for btrfs. (in fact more, but you properly won't need
> >> the extra ids)
> >>
> >> FSID: Global one, one fs one FSID.
> >> Device ID: Bonded to device, each device will have one.
> >>
> >> So in case of 2 devices btrfs, each device will has its own device id,
> >> while both of the devices have the same fsid.
> >>
> >> And I think you're talking about the global fsid instead of device id.
> >>
> >>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
> >>> do the following:
> >>> convert inode struct to btrfs_inode struct (use btrfsInode =
> >>> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
> >>> from root i take anon_dev or anon_super.s_dev.
> >>> struct btrfs_inode *btrfsInode;
> >>> btrfsInode = BTRFS_I(inode);
> >>>btrfsInode->root->anon_super.s_devor
> >>>btrfsInode->root->anon_dev- depend on kernel.
> >>
> >> The most directly method would be:
> >>
> >> btrfs_inode->root->fs_info->fsid.
> >> (For newer kernel, as I'm not familiar with older kernels)
> >>
> >> Or from superblock:
> >> btrfs_inode->root->fs_info->super_copy->fsid.
> >> (The most reliable one, no matter which kernel version you're using, as
> >> long as the super block format didn't change)
> >>
> >> For device id, it's not that commonly used unless you're dealing with
> >> chunk mapping, so I'm assuming you're referring to fsid.
> >>
> >> Thanks,
> >> Qu
> >>
> >>>
> >>> In kernel 3.12.28-4-default in order to get the fsid, i need to go
> >>> to the inode -> superblock -> device id (inode->i_sb->s_dev)
> >>>
> >>> Why is this ? and is there a proper/an official way to get it ?

-- 
Hugo Mills | Gentlemen! You can't fight here! This is the War
h

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Ilan Schwarts

Thank you for clarification.
Just 2 quick questions,
1. Sub volumes - 2 sub volumes cannot have 2 same inode numbers ?
2. Why fsInfo fsid return u8 and the traditional file system return
dev_t, usually 32 integer ?


On Sun, Jan 14, 2018 at 12:22 PM, Qu Wenruo  wrote:
>
>
> On 2018年01月14日 18:13, Ilan Schwarts wrote:
>> both btrfs filesystems will have same fsid ?
>>
>>
>> On Sun, Jan 14, 2018 at 12:06 PM, Ilan Schwarts  wrote:
>>> But both filesystems will have same fsid?
>>>
>>> On Jan 14, 2018 12:04, "Nikolay Borisov"  wrote:



 On 14.01.2018 12:02, Ilan Schwarts wrote:
> First of all, Thanks for response !
> So if i have 2 btrfs file system on the same machine (not your
> everyday scenario, i know)
>
> Not a problem, the 2 filesystems will have 2 different fsid.
>
> (And it's my everyday scenario, since fstests neeeds TEST_DEV and
> SCRATCH_DEV_POOL)
>
> Lets say a file is created on device A, the file gets inode number X
> is it possible on device B to have inode number X also ?
> or each device has its own Inode number range ?
>
> Forget the mess about device.
>
> Inode is bounded to a filesystem, not bounded to a device.
>
> Just traditional filesytems are normally bounded to a single device.
> (Although even traditional filesystems can have external journal devices)
>
> So there is nothing to do with device at all.
>
> And you can have same inode numbers in different filesystems, but
> BTRFS_I(inode)->root->fs_info will point to different fs_infos, with
> different fsid.
>
> So return to your initial question:
>> both btrfs filesystems will have same fsid ?
>
> No, different filesystems will have different fsid.
>
> (Unless you're SUUUPER lucky to have 2 filesystems with
> same fsid)
>
> Thanks,
> Qu
>
>

 Of course it is possible. Inodes are guaranteed to be unique only across
 filesystem instances. In your case you are going to have 2 fs instances.

>
> I need to create unique identifier for a file, I need to understand if
> the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
> is enough.
>
> Thanks
>
>
>
>
>
> On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo 
> wrote:
>>
>>
>> On 2018年01月14日 16:33, Ilan Schwarts wrote:
>>> Hello btrfs developers/users,
>>>
>>> I was wondering regarding to fetching the correct fsid on btrfs from
>>> the context of a kernel module.
>>
>> There are two IDs for btrfs. (in fact more, but you properly won't need
>> the extra ids)
>>
>> FSID: Global one, one fs one FSID.
>> Device ID: Bonded to device, each device will have one.
>>
>> So in case of 2 devices btrfs, each device will has its own device id,
>> while both of the devices have the same fsid.
>>
>> And I think you're talking about the global fsid instead of device id.
>>
>>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
>>> do the following:
>>> convert inode struct to btrfs_inode struct (use btrfsInode =
>>> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
>>> from root i take anon_dev or anon_super.s_dev.
>>> struct btrfs_inode *btrfsInode;
>>> btrfsInode = BTRFS_I(inode);
>>>btrfsInode->root->anon_super.s_devor
>>>btrfsInode->root->anon_dev- depend on kernel.
>>
>> The most directly method would be:
>>
>> btrfs_inode->root->fs_info->fsid.
>> (For newer kernel, as I'm not familiar with older kernels)
>>
>> Or from superblock:
>> btrfs_inode->root->fs_info->super_copy->fsid.
>> (The most reliable one, no matter which kernel version you're using, as
>> long as the super block format didn't change)
>>
>> For device id, it's not that commonly used unless you're dealing with
>> chunk mapping, so I'm assuming you're referring to fsid.
>>
>> Thanks,
>> Qu
>>
>>>
>>> In kernel 3.12.28-4-default in order to get the fsid, i need to go
>>> to the inode -> superblock -> device id (inode->i_sb->s_dev)
>>>
>>> Why is this ? and is there a proper/an official way to get it ?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>>> in
>>> the body of a message to majord...@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
>
>
>
>>
>>
>>
>



-- 


-
Ilan Schwarts
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Qu Wenruo



On 2018年01月14日 18:13, Ilan Schwarts wrote:
> both btrfs filesystems will have same fsid ?
> 
> 
> On Sun, Jan 14, 2018 at 12:06 PM, Ilan Schwarts  wrote:
>> But both filesystems will have same fsid?
>>
>> On Jan 14, 2018 12:04, "Nikolay Borisov"  wrote:
>>>
>>>
>>>
>>> On 14.01.2018 12:02, Ilan Schwarts wrote:
 First of all, Thanks for response !
 So if i have 2 btrfs file system on the same machine (not your
 everyday scenario, i know)

Not a problem, the 2 filesystems will have 2 different fsid.

(And it's my everyday scenario, since fstests neeeds TEST_DEV and
SCRATCH_DEV_POOL)

 Lets say a file is created on device A, the file gets inode number X
 is it possible on device B to have inode number X also ?
 or each device has its own Inode number range ?

Forget the mess about device.

Inode is bounded to a filesystem, not bounded to a device.

Just traditional filesytems are normally bounded to a single device.
(Although even traditional filesystems can have external journal devices)

So there is nothing to do with device at all.

And you can have same inode numbers in different filesystems, but
BTRFS_I(inode)->root->fs_info will point to different fs_infos, with
different fsid.

So return to your initial question:
> both btrfs filesystems will have same fsid ?

No, different filesystems will have different fsid.

(Unless you're SUUUPER lucky to have 2 filesystems with
same fsid)

Thanks,
Qu


>>>
>>> Of course it is possible. Inodes are guaranteed to be unique only across
>>> filesystem instances. In your case you are going to have 2 fs instances.
>>>

 I need to create unique identifier for a file, I need to understand if
 the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
 is enough.

 Thanks





 On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo 
 wrote:
>
>
> On 2018年01月14日 16:33, Ilan Schwarts wrote:
>> Hello btrfs developers/users,
>>
>> I was wondering regarding to fetching the correct fsid on btrfs from
>> the context of a kernel module.
>
> There are two IDs for btrfs. (in fact more, but you properly won't need
> the extra ids)
>
> FSID: Global one, one fs one FSID.
> Device ID: Bonded to device, each device will have one.
>
> So in case of 2 devices btrfs, each device will has its own device id,
> while both of the devices have the same fsid.
>
> And I think you're talking about the global fsid instead of device id.
>
>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
>> do the following:
>> convert inode struct to btrfs_inode struct (use btrfsInode =
>> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
>> from root i take anon_dev or anon_super.s_dev.
>> struct btrfs_inode *btrfsInode;
>> btrfsInode = BTRFS_I(inode);
>>btrfsInode->root->anon_super.s_devor
>>btrfsInode->root->anon_dev- depend on kernel.
>
> The most directly method would be:
>
> btrfs_inode->root->fs_info->fsid.
> (For newer kernel, as I'm not familiar with older kernels)
>
> Or from superblock:
> btrfs_inode->root->fs_info->super_copy->fsid.
> (The most reliable one, no matter which kernel version you're using, as
> long as the super block format didn't change)
>
> For device id, it's not that commonly used unless you're dealing with
> chunk mapping, so I'm assuming you're referring to fsid.
>
> Thanks,
> Qu
>
>>
>> In kernel 3.12.28-4-default in order to get the fsid, i need to go
>> to the inode -> superblock -> device id (inode->i_sb->s_dev)
>>
>> Why is this ? and is there a proper/an official way to get it ?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>> in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>



> 
> 
> 



signature.asc
Description: OpenPGP digital signature

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Ilan Schwarts

both btrfs filesystems will have same fsid ?


On Sun, Jan 14, 2018 at 12:06 PM, Ilan Schwarts  wrote:
> But both filesystems will have same fsid?
>
> On Jan 14, 2018 12:04, "Nikolay Borisov"  wrote:
>>
>>
>>
>> On 14.01.2018 12:02, Ilan Schwarts wrote:
>> > First of all, Thanks for response !
>> > So if i have 2 btrfs file system on the same machine (not your
>> > everyday scenario, i know)
>> > Lets say a file is created on device A, the file gets inode number X
>> > is it possible on device B to have inode number X also ?
>> > or each device has its own Inode number range ?
>>
>> Of course it is possible. Inodes are guaranteed to be unique only across
>> filesystem instances. In your case you are going to have 2 fs instances.
>>
>> >
>> > I need to create unique identifier for a file, I need to understand if
>> > the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
>> > is enough.
>> >
>> > Thanks
>> >
>> >
>> >
>> >
>> >
>> > On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo 
>> > wrote:
>> >>
>> >>
>> >> On 2018年01月14日 16:33, Ilan Schwarts wrote:
>> >>> Hello btrfs developers/users,
>> >>>
>> >>> I was wondering regarding to fetching the correct fsid on btrfs from
>> >>> the context of a kernel module.
>> >>
>> >> There are two IDs for btrfs. (in fact more, but you properly won't need
>> >> the extra ids)
>> >>
>> >> FSID: Global one, one fs one FSID.
>> >> Device ID: Bonded to device, each device will have one.
>> >>
>> >> So in case of 2 devices btrfs, each device will has its own device id,
>> >> while both of the devices have the same fsid.
>> >>
>> >> And I think you're talking about the global fsid instead of device id.
>> >>
>> >>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
>> >>> do the following:
>> >>> convert inode struct to btrfs_inode struct (use btrfsInode =
>> >>> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
>> >>> from root i take anon_dev or anon_super.s_dev.
>> >>> struct btrfs_inode *btrfsInode;
>> >>> btrfsInode = BTRFS_I(inode);
>> >>>btrfsInode->root->anon_super.s_devor
>> >>>btrfsInode->root->anon_dev- depend on kernel.
>> >>
>> >> The most directly method would be:
>> >>
>> >> btrfs_inode->root->fs_info->fsid.
>> >> (For newer kernel, as I'm not familiar with older kernels)
>> >>
>> >> Or from superblock:
>> >> btrfs_inode->root->fs_info->super_copy->fsid.
>> >> (The most reliable one, no matter which kernel version you're using, as
>> >> long as the super block format didn't change)
>> >>
>> >> For device id, it's not that commonly used unless you're dealing with
>> >> chunk mapping, so I'm assuming you're referring to fsid.
>> >>
>> >> Thanks,
>> >> Qu
>> >>
>> >>>
>> >>> In kernel 3.12.28-4-default in order to get the fsid, i need to go
>> >>> to the inode -> superblock -> device id (inode->i_sb->s_dev)
>> >>>
>> >>> Why is this ? and is there a proper/an official way to get it ?
>> >>> --
>> >>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs"
>> >>> in
>> >>> the body of a message to majord...@vger.kernel.org
>> >>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>> >>>
>> >>
>> >
>> >
>> >



-- 


-
Ilan Schwarts
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Nikolay Borisov



On 14.01.2018 12:02, Ilan Schwarts wrote:
> First of all, Thanks for response !
> So if i have 2 btrfs file system on the same machine (not your
> everyday scenario, i know)
> Lets say a file is created on device A, the file gets inode number X
> is it possible on device B to have inode number X also ?
> or each device has its own Inode number range ?

Of course it is possible. Inodes are guaranteed to be unique only across
filesystem instances. In your case you are going to have 2 fs instances.

> 
> I need to create unique identifier for a file, I need to understand if
> the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
> is enough.
> 
> Thanks
> 
> 
> 
> 
> 
> On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo  wrote:
>>
>>
>> On 2018年01月14日 16:33, Ilan Schwarts wrote:
>>> Hello btrfs developers/users,
>>>
>>> I was wondering regarding to fetching the correct fsid on btrfs from
>>> the context of a kernel module.
>>
>> There are two IDs for btrfs. (in fact more, but you properly won't need
>> the extra ids)
>>
>> FSID: Global one, one fs one FSID.
>> Device ID: Bonded to device, each device will have one.
>>
>> So in case of 2 devices btrfs, each device will has its own device id,
>> while both of the devices have the same fsid.
>>
>> And I think you're talking about the global fsid instead of device id.
>>
>>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
>>> do the following:
>>> convert inode struct to btrfs_inode struct (use btrfsInode =
>>> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
>>> from root i take anon_dev or anon_super.s_dev.
>>> struct btrfs_inode *btrfsInode;
>>> btrfsInode = BTRFS_I(inode);
>>>btrfsInode->root->anon_super.s_devor
>>>btrfsInode->root->anon_dev- depend on kernel.
>>
>> The most directly method would be:
>>
>> btrfs_inode->root->fs_info->fsid.
>> (For newer kernel, as I'm not familiar with older kernels)
>>
>> Or from superblock:
>> btrfs_inode->root->fs_info->super_copy->fsid.
>> (The most reliable one, no matter which kernel version you're using, as
>> long as the super block format didn't change)
>>
>> For device id, it's not that commonly used unless you're dealing with
>> chunk mapping, so I'm assuming you're referring to fsid.
>>
>> Thanks,
>> Qu
>>
>>>
>>> In kernel 3.12.28-4-default in order to get the fsid, i need to go
>>> to the inode -> superblock -> device id (inode->i_sb->s_dev)
>>>
>>> Why is this ? and is there a proper/an official way to get it ?
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majord...@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
> 
> 
> 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Ilan Schwarts

First of all, Thanks for response !
So if i have 2 btrfs file system on the same machine (not your
everyday scenario, i know)
Lets say a file is created on device A, the file gets inode number X
is it possible on device B to have inode number X also ?
or each device has its own Inode number range ?

I need to create unique identifier for a file, I need to understand if
the identifier would be: GlobalFSID_DeviceID_Inode or DeviceID_Inode
is enough.

Thanks





On Sun, Jan 14, 2018 at 11:13 AM, Qu Wenruo  wrote:
>
>
> On 2018年01月14日 16:33, Ilan Schwarts wrote:
>> Hello btrfs developers/users,
>>
>> I was wondering regarding to fetching the correct fsid on btrfs from
>> the context of a kernel module.
>
> There are two IDs for btrfs. (in fact more, but you properly won't need
> the extra ids)
>
> FSID: Global one, one fs one FSID.
> Device ID: Bonded to device, each device will have one.
>
> So in case of 2 devices btrfs, each device will has its own device id,
> while both of the devices have the same fsid.
>
> And I think you're talking about the global fsid instead of device id.
>
>> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
>> do the following:
>> convert inode struct to btrfs_inode struct (use btrfsInode =
>> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
>> from root i take anon_dev or anon_super.s_dev.
>> struct btrfs_inode *btrfsInode;
>> btrfsInode = BTRFS_I(inode);
>>btrfsInode->root->anon_super.s_devor
>>btrfsInode->root->anon_dev- depend on kernel.
>
> The most directly method would be:
>
> btrfs_inode->root->fs_info->fsid.
> (For newer kernel, as I'm not familiar with older kernels)
>
> Or from superblock:
> btrfs_inode->root->fs_info->super_copy->fsid.
> (The most reliable one, no matter which kernel version you're using, as
> long as the super block format didn't change)
>
> For device id, it's not that commonly used unless you're dealing with
> chunk mapping, so I'm assuming you're referring to fsid.
>
> Thanks,
> Qu
>
>>
>> In kernel 3.12.28-4-default in order to get the fsid, i need to go
>> to the inode -> superblock -> device id (inode->i_sb->s_dev)
>>
>> Why is this ? and is there a proper/an official way to get it ?
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>



-- 


-
Ilan Schwarts
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Qu Wenruo

On 2018年01月14日 16:33, Ilan Schwarts wrote:
> Hello btrfs developers/users,
> 
> I was wondering regarding to fetching the correct fsid on btrfs from
> the context of a kernel module.

There are two IDs for btrfs. (in fact more, but you properly won't need
the extra ids)

FSID: Global one, one fs one FSID.
Device ID: Bonded to device, each device will have one.

So in case of 2 devices btrfs, each device will has its own device id,
while both of the devices have the same fsid.

And I think you're talking about the global fsid instead of device id.

> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
> do the following:
> convert inode struct to btrfs_inode struct (use btrfsInode =
> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
> from root i take anon_dev or anon_super.s_dev.
> struct btrfs_inode *btrfsInode;
> btrfsInode = BTRFS_I(inode);
>btrfsInode->root->anon_super.s_devor
>btrfsInode->root->anon_dev- depend on kernel.

The most directly method would be:

btrfs_inode->root->fs_info->fsid.
(For newer kernel, as I'm not familiar with older kernels)

Or from superblock:
btrfs_inode->root->fs_info->super_copy->fsid.
(The most reliable one, no matter which kernel version you're using, as
long as the super block format didn't change)

For device id, it's not that commonly used unless you're dealing with
chunk mapping, so I'm assuming you're referring to fsid.

Thanks,
Qu

> 
> In kernel 3.12.28-4-default in order to get the fsid, i need to go
> to the inode -> superblock -> device id (inode->i_sb->s_dev)
> 
> Why is this ? and is there a proper/an official way to get it ?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

signature.asc
Description: OpenPGP digital signature

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Qu Wenruo

On 2018年01月14日 16:33, Ilan Schwarts wrote:
> Hello btrfs developers/users,
> 
> I was wondering regarding to fetching the correct fsid on btrfs from
> the context of a kernel module.

There are two IDs for btrfs. (in fact more, but you properly won't need
the extra ids)

FSID: Global one, one fs one FSID.
Device ID: Bonded to device, each device will have one.

So in case of 2 devices btrfs, each device will has its own device id,
while both of the devices have the same fsid.

And I think you're talking about the global fsid instead of device id.

> if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
> do the following:
> convert inode struct to btrfs_inode struct (use btrfsInode =
> BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
> from root i take anon_dev or anon_super.s_dev.
> struct btrfs_inode *btrfsInode;
> btrfsInode = BTRFS_I(inode);
>btrfsInode->root->anon_super.s_devor
>btrfsInode->root->anon_dev- depend on kernel.

The most directly method would be:

btrfs_inode->root->fs_info->fsid.

Or from superblock:
btrfs_inode->root->fs_info->super_copy->fsid.

For device id, it's not that commonly used unless you're dealing with
chunk mapping, so I'm assuming you're referring to fsid.

Thanks,
Qu

> 
> In kernel 3.12.28-4-default in order to get the fsid, i need to go
> to the inode -> superblock -> device id (inode->i_sb->s_dev)
> 
> Why is this ? and is there a proper/an official way to get it ?
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 

signature.asc
Description: OpenPGP digital signature

Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

2018-01-14 Thread Ilan Schwarts

Hello btrfs developers/users,

I was wondering regarding to fetching the correct fsid on btrfs from
the context of a kernel module.
if on suse11.3 kernel 3.0.101-0.47.71-default in order to get fsid, I
do the following:
convert inode struct to btrfs_inode struct (use btrfsInode =
BTRFS_I(inode)), then from btrfs_inode struct i go to root field, and
from root i take anon_dev or anon_super.s_dev.
struct btrfs_inode *btrfsInode;
btrfsInode = BTRFS_I(inode);
   btrfsInode->root->anon_super.s_devor
   btrfsInode->root->anon_dev- depend on kernel.

In kernel 3.12.28-4-default in order to get the fsid, i need to go
to the inode -> superblock -> device id (inode->i_sb->s_dev)

Why is this ? and is there a proper/an official way to get it ?
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html

Re: [PATCH] fstests: common/rc: fix device still mounted error with SCRATCH_DEV_POOL

[PATCH] btrfs: Add chunk allocation ENOSPC debug message for enospc_debug mount option

[PATCH] btrfs-progs: fsck: Continue check even if we find space cache problem

Re: invalid files names, btrfs check can't repair it

Re: invalid files names, btrfs check can't repair it

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: Partionless Btrfs drives now return PTTYPE="PMBR"

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: invalid files names, btrfs check can't repair it

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: invalid files names, btrfs check can't repair it

Partionless Btrfs drives now return PTTYPE="PMBR"

Re: Hanging after frequent use of systemd-nspawn --ephemeral

Re: Limit on the number of btrfs snapshots?

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

Re: [PATCHSET v5] blk-mq: reimplement timeout handling

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Re: Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

Fwd: Fwd: Question regarding to Btrfs patchwork /2831525

30 matches

Site Navigation

Mail list logo

Footer information