Re: About 'key type for persistent [...] items'

2017-11-24 Thread Qu Wenruo


On 2017年11月25日 09:12, Qu Wenruo wrote:
> 
> 
> On 2017年11月25日 03:16, Hans van Kranenburg wrote:
>> Hi,
>>
>> Last week, when implementing the automatic classifier to dynamically
>> create tree item data objects by key type in python-btrfs, I ran into
>> the following commits in btrfs-progs:
>>
>>   commit 8609c8bad68528f668d9ce564b868aa4828107a0
>>   btrfs-progs: print-tree: factor out temporary_item dump
>> and
>>   commit a4b65f00d53deb1b495728dd58253af44fcf70df
>>   btrfs-progs: print-tree: factor out persistent_item dump
>>
>> ...which are related to kernel...
>>
>>   commit 50c2d5abe64c1726b48d292a2ab04f60e8238933
>>   btrfs: introduce key type for persistent permanent items
>> and
>>   commit 0bbbccb17fea86818e1a058faf5903aefd20b31a
>>   btrfs: introduce key type for persistent temporary items
>>
>> Afaics the goal is to overload types because there can be only 256 in
>> total.
> 
> Personally speaking, to overload types, we can easily make different
> meanings of type based on tree owner.
> 
>> However, I'm missing the design decisions behind the
>> implementation of it. It's not in the commit messages, and it hasn't
>> been on the mailing list.
> 
> Btrfs_tree.h has a good enough description for it.
> 
>>
>> Before, there was an 1:1 mapping from key types to data structures. Now,
>> with the new PERSISTENT_ITEM_KEY and TEMPORARY_ITEM_KEY, it seems items
>> which use this type can be using any data structure they want, so it's
>> some kind of YOLO_ITEM_KEY.
> 
> For PERSISTENT and TEMPORARY item, they use the objectid and type to
> determine the real data structure type.
> 
> Since in case of existing PERSISTENT/TEMPORARY item usage, their
> objectid is not really used for storing data.

PS: PERSISTENT/TEMPORARY keys are not the only keys where only
key->offset is really used.

CHUNK_ITME also has a fixed objecitd, BTRFS_FIRST_CHUNK_TREE_OBJECTID.

And QGROUP_STATUS is even a step further, with 0 as both objectid and
offset.

So from this point, such PERSISTENT/TEMPORARY design doesn't really
follow the existing type design.
It may only help for later expansion, but I doubt if we will use such
schema.

Thanks,
Qu

> 
>>
>> The print-tree code in progs 8609c8b and a4b65f0 seems incomplete. For
>> example, for the PERSISTENT_ITEM_KEY, there's a switch (objectid) with
>> case BTRFS_DEV_STATS_OBJECTID.
>>
>> However, BTRFS_DEV_STATS_OBJECTID is just the value 0. So, that means
>> that if I want to have another tree where BTRFS_MOUTON_OBJECTID is also
>> 0, and I'm storing a btrfs_kebab_item struct indexed at
>> (BTRFS_MOUTON_OBJECTID, PERSISTENT_ITEM_KEY, 31337), then print_tree.c
>> will try to parse the data by calling print_dev_stats?
> 
> In this case, you shouldn't have your "BTRFS_MOUNTON_OBJECTID" assigned
> to 0.
> 
>>
>> What's the idea behind that? Instead of having the key type field define
>> the struct and meaning, we now suddenly need the tuple (tree, objectid,
>> type),
> 
> Not exactly, it's (objectid, type) only to determine the data structure
> type for PERSISTENT/TEMPORARY key type.
> Btrfs doesn't (yet) use root to determine the meaning.
> 
> So current btrfs-progs works fine.
> 
> Thanks,
> Qu
> 
>> and we need all three to determine what's inside the item data?
>> So, the code in print_tree.c would also need to know about the tree
>> number and pass that into the different functions.
>>
>> Am I missing something, or is my observation correct?
>>
>> Thanks,--
>> Hans van Kranenburg
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
> 



signature.asc
Description: OpenPGP digital signature


Re: About 'key type for persistent [...] items'

2017-11-24 Thread Qu Wenruo


On 2017年11月25日 03:16, Hans van Kranenburg wrote:
> Hi,
> 
> Last week, when implementing the automatic classifier to dynamically
> create tree item data objects by key type in python-btrfs, I ran into
> the following commits in btrfs-progs:
> 
>   commit 8609c8bad68528f668d9ce564b868aa4828107a0
>   btrfs-progs: print-tree: factor out temporary_item dump
> and
>   commit a4b65f00d53deb1b495728dd58253af44fcf70df
>   btrfs-progs: print-tree: factor out persistent_item dump
> 
> ...which are related to kernel...
> 
>   commit 50c2d5abe64c1726b48d292a2ab04f60e8238933
>   btrfs: introduce key type for persistent permanent items
> and
>   commit 0bbbccb17fea86818e1a058faf5903aefd20b31a
>   btrfs: introduce key type for persistent temporary items
> 
> Afaics the goal is to overload types because there can be only 256 in
> total.

Personally speaking, to overload types, we can easily make different
meanings of type based on tree owner.

> However, I'm missing the design decisions behind the
> implementation of it. It's not in the commit messages, and it hasn't
> been on the mailing list.

Btrfs_tree.h has a good enough description for it.

> 
> Before, there was an 1:1 mapping from key types to data structures. Now,
> with the new PERSISTENT_ITEM_KEY and TEMPORARY_ITEM_KEY, it seems items
> which use this type can be using any data structure they want, so it's
> some kind of YOLO_ITEM_KEY.

For PERSISTENT and TEMPORARY item, they use the objectid and type to
determine the real data structure type.

Since in case of existing PERSISTENT/TEMPORARY item usage, their
objectid is not really used for storing data.

> 
> The print-tree code in progs 8609c8b and a4b65f0 seems incomplete. For
> example, for the PERSISTENT_ITEM_KEY, there's a switch (objectid) with
> case BTRFS_DEV_STATS_OBJECTID.
> 
> However, BTRFS_DEV_STATS_OBJECTID is just the value 0. So, that means
> that if I want to have another tree where BTRFS_MOUTON_OBJECTID is also
> 0, and I'm storing a btrfs_kebab_item struct indexed at
> (BTRFS_MOUTON_OBJECTID, PERSISTENT_ITEM_KEY, 31337), then print_tree.c
> will try to parse the data by calling print_dev_stats?

In this case, you shouldn't have your "BTRFS_MOUNTON_OBJECTID" assigned
to 0.

> 
> What's the idea behind that? Instead of having the key type field define
> the struct and meaning, we now suddenly need the tuple (tree, objectid,
> type),

Not exactly, it's (objectid, type) only to determine the data structure
type for PERSISTENT/TEMPORARY key type.
Btrfs doesn't (yet) use root to determine the meaning.

So current btrfs-progs works fine.

Thanks,
Qu

> and we need all three to determine what's inside the item data?
> So, the code in print_tree.c would also need to know about the tree
> number and pass that into the different functions.
> 
> Am I missing something, or is my observation correct?
> 
> Thanks,--
> Hans van Kranenburg
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



signature.asc
Description: OpenPGP digital signature


Re: [PATCH] btrfs-progs: dump_tree: remove superfluous _TREE

2017-11-24 Thread Qu Wenruo


On 2017年11月25日 04:26, Hans van Kranenburg wrote:
> -# btrfs inspect-internal dump-tree -t fs /dev/block/device
> ERROR: unrecognized tree id: fs
> 
> Without this fix I can't dump-tree fs, but I can dump-tree fs_tree and
> also fs_tree_tree, which is a bit silly.
> ---
>  cmds-inspect-dump-tree.c | 2 +-
>  1 file changed, 1 insertion(+), 1 deletion(-)
> 
> diff --git a/cmds-inspect-dump-tree.c b/cmds-inspect-dump-tree.c
> index 4e72c08a..df44bb63 100644
> --- a/cmds-inspect-dump-tree.c
> +++ b/cmds-inspect-dump-tree.c
> @@ -143,7 +143,7 @@ static u64 treeid_from_string(const char *str, const char 
> **end)
>   { "CHUNK", BTRFS_CHUNK_TREE_OBJECTID },
>   { "DEVICE", BTRFS_DEV_TREE_OBJECTID },
>   { "DEV", BTRFS_DEV_TREE_OBJECTID },
> - { "FS_TREE", BTRFS_FS_TREE_OBJECTID },
> + { "FS", BTRFS_FS_TREE_OBJECTID },

Considering no other name has the _tree suffix, this looks good to me.

Reviewed-by: Qu Wenruo 

Thanks,
Qu
>   { "CSUM", BTRFS_CSUM_TREE_OBJECTID },
>   { "CHECKSUM", BTRFS_CSUM_TREE_OBJECTID },
>   { "QUOTA", BTRFS_QUOTA_TREE_OBJECTID },
> 



signature.asc
Description: OpenPGP digital signature


[PATCH] btrfs-progs: dump_tree: remove superfluous _TREE

2017-11-24 Thread Hans van Kranenburg
-# btrfs inspect-internal dump-tree -t fs /dev/block/device
ERROR: unrecognized tree id: fs

Without this fix I can't dump-tree fs, but I can dump-tree fs_tree and
also fs_tree_tree, which is a bit silly.
---
 cmds-inspect-dump-tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)

diff --git a/cmds-inspect-dump-tree.c b/cmds-inspect-dump-tree.c
index 4e72c08a..df44bb63 100644
--- a/cmds-inspect-dump-tree.c
+++ b/cmds-inspect-dump-tree.c
@@ -143,7 +143,7 @@ static u64 treeid_from_string(const char *str, const char 
**end)
{ "CHUNK", BTRFS_CHUNK_TREE_OBJECTID },
{ "DEVICE", BTRFS_DEV_TREE_OBJECTID },
{ "DEV", BTRFS_DEV_TREE_OBJECTID },
-   { "FS_TREE", BTRFS_FS_TREE_OBJECTID },
+   { "FS", BTRFS_FS_TREE_OBJECTID },
{ "CSUM", BTRFS_CSUM_TREE_OBJECTID },
{ "CHECKSUM", BTRFS_CSUM_TREE_OBJECTID },
{ "QUOTA", BTRFS_QUOTA_TREE_OBJECTID },
-- 
2.11.0

--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


About 'key type for persistent [...] items'

2017-11-24 Thread Hans van Kranenburg
Hi,

Last week, when implementing the automatic classifier to dynamically
create tree item data objects by key type in python-btrfs, I ran into
the following commits in btrfs-progs:

  commit 8609c8bad68528f668d9ce564b868aa4828107a0
  btrfs-progs: print-tree: factor out temporary_item dump
and
  commit a4b65f00d53deb1b495728dd58253af44fcf70df
  btrfs-progs: print-tree: factor out persistent_item dump

...which are related to kernel...

  commit 50c2d5abe64c1726b48d292a2ab04f60e8238933
  btrfs: introduce key type for persistent permanent items
and
  commit 0bbbccb17fea86818e1a058faf5903aefd20b31a
  btrfs: introduce key type for persistent temporary items

Afaics the goal is to overload types because there can be only 256 in
total. However, I'm missing the design decisions behind the
implementation of it. It's not in the commit messages, and it hasn't
been on the mailing list.

Before, there was an 1:1 mapping from key types to data structures. Now,
with the new PERSISTENT_ITEM_KEY and TEMPORARY_ITEM_KEY, it seems items
which use this type can be using any data structure they want, so it's
some kind of YOLO_ITEM_KEY.

The print-tree code in progs 8609c8b and a4b65f0 seems incomplete. For
example, for the PERSISTENT_ITEM_KEY, there's a switch (objectid) with
case BTRFS_DEV_STATS_OBJECTID.

However, BTRFS_DEV_STATS_OBJECTID is just the value 0. So, that means
that if I want to have another tree where BTRFS_MOUTON_OBJECTID is also
0, and I'm storing a btrfs_kebab_item struct indexed at
(BTRFS_MOUTON_OBJECTID, PERSISTENT_ITEM_KEY, 31337), then print_tree.c
will try to parse the data by calling print_dev_stats?

What's the idea behind that? Instead of having the key type field define
the struct and meaning, we now suddenly need the tuple (tree, objectid,
type), and we need all three to determine what's inside the item data?
So, the code in print_tree.c would also need to know about the tree
number and pass that into the different functions.

Am I missing something, or is my observation correct?

Thanks,--
Hans van Kranenburg
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH 00/11] Lowmem mode btrfs fixes exposed by complex tree

2017-11-24 Thread David Sterba
On Wed, Nov 22, 2017 at 05:03:14PM +0800, Qu Wenruo wrote:
> The patchset can be fetched from github:
> https://github.com/adam900710/btrfs-progs/tree/lowmem_fix
> 
> The patchset is mostly rebased to v4.14, since there is some conflicts
> with lowmem repair enhancement from Su Yue.
> 
> However the lowmem repair enhancement from Su Yue caused one regression,
> and the original lowmem code also has some problem handling tree reloc
> tree.

Thanks for the fixes, patchset added to devel. I've also enabled the
lowmem mode in the CI build tests.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Lakshmipathi.G
>>>
>>> To make it clear, what kernel config and commit are you using when the
>>> test fails?
>>>
>> When I use attached kernel config along with tar from misc-next branch
>> (wget https://github.com/kdave/btrfs-devel/archive/misc-next.zip) this
>> issue happens.
>
> BTW, David seems to update his branch more often in his git.kernel.org
> branch than github.
>
> So it's possible that the branch from github is out dated.

Okay, I didn't know that one, let me checkout his git.kernel.org branch and
check the results there.

>
> For v4.14-rc7, even I enabled all the btrfs related configs, I still
> can't reproduce it.
> --
> $ zcat /proc/config.gz  | grep BTRFS
> CONFIG_BTRFS_FS=m
> CONFIG_BTRFS_FS_POSIX_ACL=y
> CONFIG_BTRFS_FS_CHECK_INTEGRITY=y
> CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y
> CONFIG_BTRFS_DEBUG=y
> CONFIG_BTRFS_ASSERT=y
> --
>
> So there is something strange happened.
>

Yes. something strange going on. I use the same kernel config
for misc-next, 4.14 and 4.14.2 - issue happens only with
misc-next, as you mentioned it may not be up-to-date.



Cheers,
Lakshmipathi.G
http://www.giis.co.in http://www.webminal.org
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Qu Wenruo


On 2017年11月24日 19:42, Lakshmipathi.G wrote:
>>
>> To make it clear, what kernel config and commit are you using when the
>> test fails?
>>
> When I use attached kernel config along with tar from misc-next branch
> (wget https://github.com/kdave/btrfs-devel/archive/misc-next.zip) this
> issue happens.

BTW, David seems to update his branch more often in his git.kernel.org
branch than github.

So it's possible that the branch from github is out dated.

> 
>> In my test, if I use the david/for-4.15 branch, it will fails to umount
>> the fs, due to the known bug if you enabled
>> CONFIG_BTRFS_FS_CHECK_INTEGRITY (you reported it first).
>> And for any upstream kernel version like v4.14-rc7 or v4.14, not problem
>> at all.
>>
> Here's btrfs related config. I checked with 4.14.2 it worked fine there too.
> 
> The issue happens with v4.14-rc7, can you check with above config
> file and zip file(misc-next.zip)?
> 
> $ cat kernel.config | grep "BTRFS"
> CONFIG_BTRFS_FS=y
> CONFIG_BTRFS_FS_POSIX_ACL=y
> CONFIG_BTRFS_FS_CHECK_INTEGRITY=n
> CONFIG_BTRFS_FS_RUN_SANITY_TESTS=n
> CONFIG_BTRFS_DEBUG=y
> CONFIG_BTRFS_ASSERT=y
> CONFIG_BTRFS_FS_REF_VERIFY=y

For v4.14-rc7, even I enabled all the btrfs related configs, I still
can't reproduce it.
--
$ zcat /proc/config.gz  | grep BTRFS
CONFIG_BTRFS_FS=m
CONFIG_BTRFS_FS_POSIX_ACL=y
CONFIG_BTRFS_FS_CHECK_INTEGRITY=y
CONFIG_BTRFS_FS_RUN_SANITY_TESTS=y
CONFIG_BTRFS_DEBUG=y
CONFIG_BTRFS_ASSERT=y
--

So there is something strange happened.

Thanks,
Qu

> 
> thanks!
> 
> 
> Cheers,
> Lakshmipathi.G
> http://www.giis.co.in http://www.webminal.org
> 



signature.asc
Description: OpenPGP digital signature


[PATCH] btrfs: test send for files with multiple hard links renamed

2017-11-24 Thread fdmanana
From: Filipe Manana 

Test that an incremental send operation works if a file that has multiple
hard links has some of its hard links renamed in the send snapshot, with
one of them getting the same path that some other inode had in the send
snapshot.

At the moment this test fails on btrfs and a fix is provived by a linux
kernel patch titled:

  "Btrfs: incremental send, fix wrong unlink path after renaming file"

Signed-off-by: Filipe Manana 
---
 tests/btrfs/155 | 147 
 tests/btrfs/155.out |   6 +++
 tests/btrfs/group   |   1 +
 3 files changed, 154 insertions(+)
 create mode 100755 tests/btrfs/155
 create mode 100644 tests/btrfs/155.out

diff --git a/tests/btrfs/155 b/tests/btrfs/155
new file mode 100755
index ..37c23260
--- /dev/null
+++ b/tests/btrfs/155
@@ -0,0 +1,147 @@
+#! /bin/bash
+# FS QA Test No. btrfs/155
+#
+# Test that an incremental send operation works if a file that has multiple
+# hard links has some of its hard links renamed in the send snapshot, with one
+# of them getting the same path that some other inode had in the send snapshot.
+#
+#---
+#
+# Copyright (C) 2017 SUSE Linux Products GmbH. All Rights Reserved.
+# Author: Filipe Manana 
+#
+# This program is free software; you can redistribute it and/or
+# modify it under the terms of the GNU General Public License as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it would be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License
+# along with this program; if not, write the Free Software Foundation,
+# Inc.,  51 Franklin St, Fifth Floor, Boston, MA  02110-1301  USA
+#---
+#
+
+seq=`basename $0`
+seqres=$RESULT_DIR/$seq
+echo "QA output created by $seq"
+
+tmp=/tmp/$$
+status=1   # failure is the default!
+trap "_cleanup; exit \$status" 0 1 2 3 15
+
+_cleanup()
+{
+   cd /
+   rm -fr $send_files_dir
+   rm -f $tmp.*
+}
+
+# get standard environment, filters and checks
+. ./common/rc
+. ./common/filter
+
+# real QA test starts here
+_supported_fs btrfs
+_supported_os Linux
+_require_test
+_require_scratch
+_require_fssum
+
+send_files_dir=$TEST_DIR/btrfs-test-$seq
+
+rm -f $seqres.full
+rm -fr $send_files_dir
+mkdir $send_files_dir
+
+_scratch_mkfs >>$seqres.full 2>&1
+_scratch_mount
+
+mkdir $SCRATCH_MNT/a
+touch $SCRATCH_MNT/a/f1
+mkdir -p $SCRATCH_MNT/a/b/c
+ln $SCRATCH_MNT/a/f1 $SCRATCH_MNT/a/b/c/f1l1
+touch $SCRATCH_MNT/a/b/f2
+mkdir $SCRATCH_MNT/d
+mv $SCRATCH_MNT/a/b/c/f1l1 $SCRATCH_MNT/d/f1l1_2
+ln $SCRATCH_MNT/a/b/f2 $SCRATCH_MNT/a/f2l1
+ln $SCRATCH_MNT/a/b/f2 $SCRATCH_MNT/d/f2l2
+mv $SCRATCH_MNT/a/f1 $SCRATCH_MNT/d/f1_2
+
+# Filesystem looks like:
+#
+# .  (ino 256)
+# | a/   (ino 257)
+# | | b/ (ino 259)
+# | | | c/   (ino 260)
+# | | | f2   (ino 261)
+# | |
+# | | f2l1   (ino 261)
+# |
+# | d/   (ino 262)
+#   | f1l1_2 (ino 258)
+#   | f2l2   (ino 261)
+#   | f1_2   (ino 258)
+#
+$BTRFS_UTIL_PROG subvolume snapshot -r $SCRATCH_MNT \
+   $SCRATCH_MNT/mysnap1 > /dev/null
+
+$BTRFS_UTIL_PROG send -f $send_files_dir/1.snap \
+   $SCRATCH_MNT/mysnap1 2>&1 1>/dev/null | _filter_scratch
+
+mv $SCRATCH_MNT/d $SCRATCH_MNT/a/b/d2
+mv $SCRATCH_MNT/a/f2l1 $SCRATCH_MNT/d
+mkdir $SCRATCH_MNT/a/f2l1
+ln $SCRATCH_MNT/a/b/d2/f1_2 $SCRATCH_MNT/a/b/f1l2
+mv $SCRATCH_MNT/a/b $SCRATCH_MNT/a/f2l1/b2
+mv $SCRATCH_MNT/a/f2l1/b2/d2 $SCRATCH_MNT/a/f2l1/b2/c/d3
+mv $SCRATCH_MNT/a/f2l1/b2/c/d3/f2l2 $SCRATCH_MNT/a/f2l1/b2/c/d3/f2l2_2
+
+# Filesystem now looks like:
+#
+# .  (ino 256)
+# | a/   (ino 257)
+# | | f2l1/  (ino 263)
+# | | b2/(ino 259)
+# |   | c/   (ino 260)
+# |   | | d3 (ino 262)
+# |   |   | f1l1_2   (ino 258)
+# |   |   | f2l2_2   (ino 261)
+# |   |   | f1_2 (ino 258)
+# |   |
+# |   | f2

[PATCH] Btrfs: incremental send, fix wrong unlink path after renaming file

2017-11-24 Thread fdmanana
From: Filipe Manana 

Under some circumstances, an incremental send operation can issue wrong
paths for unlink commands related to files that have multiple hard links
and some (or all) of those links were renamed between the parent and send
snapshots. Consider the following example:

Parent snapshot

 .  (ino 256)
 | a/   (ino 257)
 | | b/ (ino 259)
 | | | c/   (ino 260)
 | | | f2   (ino 261)
 | |
 | | f2l1   (ino 261)
 |
 | d/   (ino 262)
   | f1l1_2 (ino 258)
   | f2l2   (ino 261)
   | f1_2   (ino 258)

Send snapshot

 .  (ino 256)
 | a/   (ino 257)
 | | f2l1/  (ino 263)
 | | b2/(ino 259)
 |   | c/   (ino 260)
 |   | | d3 (ino 262)
 |   |   | f1l1_2   (ino 258)
 |   |   | f2l2_2   (ino 261)
 |   |   | f1_2 (ino 258)
 |   |
 |   | f2   (ino 261)
 |   | f1l2 (ino 258)
 |
 | d(ino 261)

When computing the incremental send stream the following steps happen:

1) When processing inode 261, a rename operation is issued that renames
   inode 262, which currently as a path of "d", to an orphan name of
   "o262-7-0". This is done because in the send snapshot, inode 261 has
   of its hard links with a path of "d" as well.

2) Two link operations are issued that create the new hard links for
   inode 261, whose names are "d" and "f2l2_2", at paths "/" and
   "o262-7-0/" respectively.

3) Still while processing inode 261, unlink operations are issued to
   remove the old hard links of inode 261, with names "f2l1" and "f2l2",
   at paths "a/" and "d/". However path "d/" does not correspond anymore
   to the directory inode 262 but corresponds instead to a hard link of
   inode 261 (link command issued in the previous step). This makes the
   receiver fail with a ENOTDIR error when attempting the unlink
   operation.

The problem happens because before sending the unlink operation, we failed
to detect that inode 262 was one of ancestors for inode 261 in the parent
snapshot, and therefore we didn't recompute the path for inode 262 before
issuing the unlink operation for the link named "f2l2" of inode 262. The
detection failed because the function "is_ancestor()" only follows the
first hard link it finds for an inode instead of all of its hard links
(as it was originally created for being used with directories only, for
which only one hard link exists). So fix this by making "is_ancestor()"
follow all hard links of the input inode.

A test case for fstests follows soon.

Signed-off-by: Filipe Manana 
---
 fs/btrfs/send.c | 124 
 1 file changed, 106 insertions(+), 18 deletions(-)

diff --git a/fs/btrfs/send.c b/fs/btrfs/send.c
index 8fd195cfe81b..2c35717a3470 100644
--- a/fs/btrfs/send.c
+++ b/fs/btrfs/send.c
@@ -3527,7 +3527,40 @@ static int wait_for_dest_dir_move(struct send_ctx *sctx,
 }
 
 /*
- * Check if ino ino1 is an ancestor of inode ino2 in the given root.
+ * Check if inode ino2, or any of its ancestors, is inode ino1.
+ * Return 1 if true, 0 if false and < 0 on error.
+ */
+static int check_ino_in_path(struct btrfs_root *root,
+const u64 ino1,
+const u64 ino1_gen,
+const u64 ino2,
+const u64 ino2_gen,
+struct fs_path *fs_path)
+{
+   u64 ino = ino2;
+
+   if (ino1 == ino2)
+   return ino1_gen == ino2_gen;
+
+   while (ino > BTRFS_FIRST_FREE_OBJECTID) {
+   u64 parent;
+   u64 parent_gen;
+   int ret;
+
+   fs_path_reset(fs_path);
+   ret = get_first_ref(root, ino, &parent, &parent_gen, fs_path);
+   if (ret < 0)
+   return ret;
+   if (parent == ino1)
+   return parent_gen == ino1_gen;
+   ino = parent;
+   }
+   return 0;
+}
+
+/*
+ * Check if ino ino1 is an ancestor of inode ino2 in the given root for any
+ * possible path (in case ino2 is not a directory and 

Re: [PATCH 1/3] btrfs: Introduce macros to handle bytes and sector conversion

2017-11-24 Thread David Sterba
On Fri, Nov 24, 2017 at 08:53:09AM +0200, Nikolay Borisov wrote:
> On 24.11.2017 06:10, Qu Wenruo wrote:
> > Since block I/O is always done in unit of 512 bytes, add BI_SECTOR_SIZE
> > and BI_SECTOR_SHIFT macros and to_sector() and to_bytes() for later
> > use.
> > 
> > Although the best position to define such things should be bvec.h, and
> > to_sector() to_bytes() are also defined in device-mapper.h, there are
> > a lot of code defining their own SECTOR_SIZE and to_bytes() in
> > device-mapper is not using u64 (unsigned long long) but unsigned long,
> > which doesn't fit btrfs usage.
> > 
> > So define btrfs' own macros and inlined converters.
> > 
> > Signed-off-by: Qu Wenruo 
> > ---
> >  fs/btrfs/ctree.h | 13 +
> >  1 file changed, 13 insertions(+)
> > 
> > diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.h
> > index 8fc690384c58..8f6f57167661 100644
> > --- a/fs/btrfs/ctree.h
> > +++ b/fs/btrfs/ctree.h
> > @@ -3734,4 +3734,17 @@ static inline int btrfs_is_testing(struct 
> > btrfs_fs_info *fs_info)
> >  #endif
> > return 0;
> >  }
> > +
> > +#define BI_SECTOR_SHIFT(9)
> > +#define BI_SECTOR_SIZE (1 << BI_SECTOR_SHIFT)
> > +
> > +static inline u64 to_bytes(sector_t n)
> > +{
> > +   return ((u64)n << BI_SECTOR_SHIFT);
> > +}
> > +
> > +static inline sector_t to_sector(u64 n)
> > +{
> > +   return (n >> BI_SECTOR_SHIFT);
> > +}
> 
> nit: I don't like the naming, I'd rather have something similar to what
> xfs has for its conversion macros i.e:
> 
> XFS_FSS_TO_BB - filesystem sector to basic block (bb in this case is 512
> bytes)
> 
> so why not - BTRFS_B_TO_BB or BTRFS_B_TO_S and BTRFS_S_TO_B. Or
> something like that but just to_bytes and to_sector look a bit silly.
> It's not a big paint point but more of a "let's get the good ideas from
> other fsses"

I don't like the naming in the patch either, but I don't think we should
copy the xfs naming. There are differences in the building objects,
thers's nothing like a 'basic block' and thus BB in btrfs does not refer
to anything we know. It's fine for xfs as this has been there since the
beginning. The wordy_naming in btrfs is similar in that way and has been
part of the format (names of the on-disk structures etc), so we're past
the time to switch to something completely different.
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Qu Wenruo


On 2017年11月24日 18:15, Lakshmipathi.G wrote:
>>
>> No, it's not about btrfs-image itself, the flag is introduced long long
>> long way ago.
>>
> Okay.
> 
>> It's about kernel change which doesn't allow image with such flag to be
>> mounted AFAIK.
> 
> No luck yet! It works on 4.14.0

The same dmesg result as I, so I didn't see any problem here.

To make it clear, what kernel config and commit are you using when the
test fails?

In my test, if I use the david/for-4.15 branch, it will fails to umount
the fs, due to the known bug if you enabled
CONFIG_BTRFS_FS_CHECK_INTEGRITY (you reported it first).
And for any upstream kernel version like v4.14-rc7 or v4.14, not problem
at all.

BTW, the btrfs-progs version should not affect, since all the problem is
about kernel failing to mount.

Despite that (either disable that option or apply the patch I submitted)
I can't reproduce the bug you reported (which fails to mount the fs).

Thanks,
Qu
> -
> -bash-4.4$ sudo make TEST=021\* test-misc
> [TEST]   misc-tests.sh
> [TEST/misc]   021-image-multi-devices
> -bash-4.4$ sudo dmesg
> [  162.635824] BTRFS: device fsid f0c92f1b-c1ba-4576-8a07-8ce4f7b5bacd
> devid 1 transid 5 /dev/loop0
> [  162.672952] BTRFS: device fsid f0c92f1b-c1ba-4576-8a07-8ce4f7b5bacd
> devid 2 transid 5 /dev/loop1
> [  162.692848] BTRFS info (device loop1): disk space caching is enabled
> [  162.695725] BTRFS info (device loop1): has skinny extents
> [  162.698003] BTRFS info (device loop1): flagging fs with big metadata 
> feature
> [  162.701643] BTRFS info (device loop1): creating UUID tree
> [  163.792851] BTRFS warning (device loop0): unrecognized super flag:
> 17179869184
> [  163.796350] BTRFS info (device loop0): disk space caching is enabled
> [  163.807644] BTRFS info (device loop0): has skinny extents
> [  163.811613] BTRFS info (device loop0): checking UUID tree
> [  163.841625] BTRFS error (device loop0): 1 errors while writing supers
> [  163.845457] BTRFS: error (device loop0) in write_all_supers:3709:
> errno=-5 IO failure (1 errors while writing supers)
> [  163.851520] BTRFS info (device loop0): forced readonly
> [  163.855184] BTRFS warning (device loop0): Skipping commit of
> aborted transaction.
> [  163.859121] BTRFS: error (device loop0) in
> cleanup_transaction:1873: errno=-5 IO failure
> [  163.862577] BTRFS info (device loop0): delayed_refs has NO entry
> [  163.865623] BTRFS error (device loop0): commit super ret -5
> [  163.868303] BTRFS error (device loop0): cleaner transaction attach
> returned -30
> -bash-4.4$ uname -a
> Linux ip-172-31-33-196.us-west-2.compute.internal 4.14.0 #1 SMP Fri
> Nov 24 09:48:11 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
> -
> 
> Cheers,
> Lakshmipathi.G
> http://www.giis.co.in http://www.webminal.org
> 



signature.asc
Description: OpenPGP digital signature


[PATCH 5/5] btrfs-progs: return error to caller instead of BUG_ON

2017-11-24 Thread Gu Jinxiang
The following test fails when deal with corrupt block below in fs tree.
key (3742682168 UNKNOWN.0 51539611375) block 256 (0) gen 298824472660

And when try to repair this fs tree, it fails in repair_btree.
Because in repair_btree, it attempt to balance, but can not
find the following item in extent tree.
key (34084861431808 UNKNOWN.0 256) block 131072 (32) gen 36283884678912

Since from returned value, we can know the result of a procedure,
so using return error instead of BUG_ON.

Here comes the error message:
$ sudo TEST=003\* make test-fuzz
ctree.c:197: update_ref_for_cow: BUG_ON `ret` triggered, value -5
/home/adam/btrfs/btrfs-progs/btrfs[0x40b5d4]
/home/adam/btrfs/btrfs-progs/btrfs[0x40b6c2]
/home/adam/btrfs/btrfs-progs/btrfs[0x40c727]
/home/adam/btrfs/btrfs-progs/btrfs(__btrfs_cow_block+0x2cf)[0x40cda4]
/home/adam/btrfs/btrfs-progs/btrfs(btrfs_cow_block+0x105)[0x40d0fc]
/home/adam/btrfs/btrfs-progs/btrfs[0x40dd8a]
/home/adam/btrfs/btrfs-progs/btrfs(btrfs_search_slot+0x355)[0x40f14f]
/home/adam/btrfs/btrfs-progs/btrfs[0x477546]
/home/adam/btrfs/btrfs-progs/btrfs[0x477c19]
/home/adam/btrfs/btrfs-progs/btrfs[0x477ff1]
/home/adam/btrfs/btrfs-progs/btrfs[0x47cd3f]
/home/adam/btrfs/btrfs-progs/btrfs(cmd_check+0xd6b)[0x48fc86]
/home/adam/btrfs/btrfs-progs/btrfs(main+0x127)[0x40b49d]
/lib64/libc.so.6(__libc_start_main+0xea)[0x7fe14bf1803a]
/home/adam/btrfs/btrfs-progs/btrfs(_start+0x2a)[0x40ad9a]
failed (ignored, ret=134): /home/adam/btrfs/btrfs-progs/btrfs check 
--init-csum-tree 
/home/adam/btrfs/btrfs-progs/tests/fuzz-tests/images/bko-172811.raw.restored
mayfail: returned code 134 (SIGABRT), not ignored

Signed-off-by: Gu Jinxiang 
---
 ctree.c | 6 --
 1 file changed, 4 insertions(+), 2 deletions(-)

diff --git a/ctree.c b/ctree.c
index 4fc33b14..f4cf006b 100644
--- a/ctree.c
+++ b/ctree.c
@@ -194,8 +194,10 @@ static noinline int update_ref_for_cow(struct 
btrfs_trans_handle *trans,
ret = btrfs_lookup_extent_info(trans, root, buf->start,
   btrfs_header_level(buf), 1,
   &refs, &flags);
-   BUG_ON(ret);
-   BUG_ON(refs == 0);
+   if (refs == 0)
+   ret = -EIO;
+   if (ret)
+   return ret;
} else {
refs = 1;
if (root->root_key.objectid == BTRFS_TREE_RELOC_OBJECTID ||
-- 
2.14.3



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 2/5] btrfs-progs: Add logic to judge the level between parent and child

2017-11-24 Thread Gu Jinxiang
The following test failed becasuse the level of child node is not
correct. In repair mode, when update parent's ptr in
btrfs_search_slot->btrfs_cow_block->__btrfs_cow_block,
it use path[level+1] to get the parent, and not judge whether
path[level+1] is NULL.

$sudo TEST=003\* make test-fuzz
failed (ignored, ret=139): /home/adam/btrfs/btrfs-progs/btrfs check 
--init-csum-tree 
/home/adam/btrfs/btrfs-progs/tests/fuzz-tests/images/bko-161821.raw.restored
mayfail: returned code 139 (SEGFAULT), not ignored

Signed-off-by: Gu Jinxiang 
---
 cmds-check.c | 19 +++
 1 file changed, 19 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 71b15de4..ac0375e5 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -2085,6 +2085,7 @@ static void reada_walk_down(struct btrfs_root *root,
  *in parent.
  * 2. block in parent node should match the child node/leaf.
  * 3. generation of parent node and child's header should be consistent.
+ * 4. level of parent node should not smaller than the child node/leaf.
  *
  * Or the child node/leaf pointed by the key in parent is not valid.
  *
@@ -2125,6 +2126,17 @@ static int check_child_node(struct extent_buffer 
*parent, int slot,
btrfs_header_generation(child),
btrfs_node_ptr_generation(parent, slot));
}
+   /* If level of child is not correct, it will lead to
+* Segmentation fault when repair parent.
+* Because when update parent's ptr in
+* btrfs_search_slot->btrfs_cow_block->__btrfs_cow_block,
+* it use path[level+1] to get the parent
+*/
+   if (btrfs_header_level(parent) <= btrfs_header_level(child)) {
+   ret = -EFAULT;
+   fprintf(stderr, "Wrong level of child node/leaf, wanted: %d, 
have: %d\n",
+   btrfs_header_level(parent)-1, 
btrfs_header_level(child));
+   }
return ret;
 }
 
@@ -4067,6 +4079,12 @@ static int check_fs_root(struct btrfs_root *root,
wret = walk_down_tree(root, &path, wc, &level, &nrefs);
if (wret < 0)
ret = wret;
+   /* When return value is -EFAULT, it indicate that level
+* in path is incorrect. And it will lead to Segmentation fault
+* in repair mod.
+*/
+   if (wret == -EFAULT && repair)
+   goto out;
if (wret != 0)
break;
 
@@ -4123,6 +4141,7 @@ skip_walking:
if (!ret)
ret = err;
 
+out:
free_corrupt_blocks_tree(&corrupt_blocks);
root->fs_info->corrupt_blocks = NULL;
free_orphan_data_extents(&root->orphan_data_extents);
-- 
2.14.3



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 4/5] btrfs-progs: check null pointer before use it

2017-11-24 Thread Gu Jinxiang
The following test failed when trying to check tree below.
item 9 key (TREE_RELOC ROOT_ITEM 0) itemoff 1135 itemsize 439
Since it has a inconsistent level in root and root->node, Segment fault
accures when use btrfs_node_key after btrfs_search_slot.

So add null pointer check before use btrfs_node_key.

Here comes the error message:
$ sudo TEST=003\* make test-fuzz
failed (ignored, ret=139): /home/adam/btrfs/btrfs-progs/btrfs check 
--init-csum-tree 
/home/adam/btrfs/btrfs-progs/tests/fuzz-tests/images/bko-172811.raw.restored
mayfail: returned code 139 (SEGFAULT), not ignored
test failed for case 003-multi-check-unmounted

Signed-off-by: Gu Jinxiang 
---
 cmds-check.c | 2 ++
 1 file changed, 2 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 49b0792b..3f4244a2 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -4070,6 +4070,8 @@ static int check_fs_root(struct btrfs_root *root,
wret = btrfs_search_slot(NULL, root, &key, &path, 0, 0);
if (wret < 0)
goto skip_walking;
+   if (!path.nodes[level])
+   goto skip_walking;
btrfs_node_key(path.nodes[level], &found_key,
path.slots[level]);
WARN_ON(memcmp(&found_key, &root_item->drop_progress,
-- 
2.14.3



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 3/5] btrfs-progs: return error to upper caller instead of BUG_ON

2017-11-24 Thread Gu Jinxiang
Return error to upper caller instead of BUG_ON.

The following test failed when trying to repair fs root,
because the bytenr of item below in fs tree is smaller than sectorsize.
key (256 INODE_ITEM 0) block 3835 (0) gen 6052837899185946625
It fails in function read_tree_block.

Here comes the call stack:
at disk-io.c:324
at ctree.c:652
p=0x721ec0, ins_len=185, cow=1) at ctree.c:1173
path=0x721ec0, cpu_key=0x7fffdc90, data_size=0x7fffdbcc, nr=1) at 
ctree.c:2485
key=0x7fffdc90, data_size=160) at ctree.h:2603
cpu_key=0x7fffdc90, data=0x7fffdce0, data_size=160) at ctree.c:2584
inode_item=0x7fffdce0) at inode-item.c:155
at utils.c:397
at cmds-check.c:3925

Here comes the error message:
$ sudo TEST=003\* make test-fuzz
cmds-check.c:3938: check_inode_recs: BUG_ON `ret` triggered, value -5
/home/adam/btrfs/btrfs-progs/btrfs[0x46aba7]
/home/adam/btrfs/btrfs-progs/btrfs[0x46ac95]
/home/adam/btrfs/btrfs-progs/btrfs[0x4760ee]
/home/adam/btrfs/btrfs-progs/btrfs[0x477d01]
/home/adam/btrfs/btrfs-progs/btrfs[0x477ff1]
/home/adam/btrfs/btrfs-progs/btrfs[0x47cd3f]
/home/adam/btrfs/btrfs-progs/btrfs(cmd_check+0xd6b)[0x48fc86]
/home/adam/btrfs/btrfs-progs/btrfs(main+0x127)[0x40b49d]
/lib64/libc.so.6(__libc_start_main+0xea)[0x7fc7005a603a]
/home/adam/btrfs/btrfs-progs/btrfs(_start+0x2a)[0x40ad9a]
failed (ignored, ret=134): /home/adam/btrfs/btrfs-progs/btrfs check 
--init-csum-tree 
/home/adam/btrfs/btrfs-progs/tests/fuzz-tests/images/bko-172811.raw.restored

Signed-off-by: Gu Jinxiang 
---
 cmds-check.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)

diff --git a/cmds-check.c b/cmds-check.c
index ac0375e5..49b0792b 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -3489,9 +3489,10 @@ static int check_inode_recs(struct btrfs_root *root,
(unsigned long long)root->objectid);
 
ret = btrfs_make_root_dir(trans, root, root_dirid);
-   BUG_ON(ret);
 
btrfs_commit_transaction(trans, root);
+   if (ret)
+   return ret;
return -EAGAIN;
}
 
-- 
2.14.3



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


[PATCH 1/5] btrfs-progs: Add location check when process share_data_ref item

2017-11-24 Thread Gu Jinxiang
The following test failed becasuse share_data_ref be added into
extent_cache when deal with root tree node.

$sudo TEST=003\* make test-fuzz
cmds-check.c:5660: check_owner_ref: BUG_ON `rec->is_root` triggered, value 1
/home/adam/btrfs/btrfs-progs/btrfs[0x46a43b]
/home/adam/btrfs/btrfs-progs/btrfs[0x46a529]
/home/adam/btrfs/btrfs-progs/btrfs[0x479e55]
/home/adam/btrfs/btrfs-progs/btrfs[0x47af81]
/home/adam/btrfs/btrfs-progs/btrfs[0x47f69e]
/home/adam/btrfs/btrfs-progs/btrfs[0x484680]
/home/adam/btrfs/btrfs-progs/btrfs[0x484cb9]
/home/adam/btrfs/btrfs-progs/btrfs[0x4884f9]
/home/adam/btrfs/btrfs-progs/btrfs(cmd_check+0xb53)[0x48b7c3]
/home/adam/btrfs/btrfs-progs/btrfs(main+0x127)[0x40b39d]
/lib64/libc.so.6(__libc_start_main+0xea)[0x7f9ac76db03a]
/home/adam/btrfs/btrfs-progs/btrfs(_start+0x2a)[0x40ac9a]
failed (ignored, ret=134): /home/adam/btrfs/btrfs-progs/btrfs check 
--check-data-csum 
/home/adam/btrfs/btrfs-progs/tests/fuzz-tests/images/bko-156731.raw.restored
mayfail: returned code 134 (SIGABRT), not ignored
test failed for case 003-multi-check-unmounted

Signed-off-by: Gu Jinxiang 
---
 cmds-check.c | 8 
 1 file changed, 8 insertions(+)

diff --git a/cmds-check.c b/cmds-check.c
index 5c822b84..71b15de4 100644
--- a/cmds-check.c
+++ b/cmds-check.c
@@ -8018,6 +8018,14 @@ static int run_next_block(struct btrfs_root *root,
}
if (key.type == BTRFS_SHARED_DATA_REF_KEY) {
struct btrfs_shared_data_ref *ref;
+   if (root->root_key.objectid !=
+   BTRFS_EXTENT_TREE_OBJECTID) {
+   error(
+   "invaild location of share_data_ref [%d %d] 
root %d",
+   key.objectid, key.offset,
+   root->root_key.objectid);
+   continue;
+   }
ref = btrfs_item_ptr(buf, i,
struct btrfs_shared_data_ref);
add_data_backref(extent_cache,
-- 
2.14.3



--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Lakshmipathi.G
>
> No, it's not about btrfs-image itself, the flag is introduced long long
> long way ago.
>
Okay.

> It's about kernel change which doesn't allow image with such flag to be
> mounted AFAIK.

No luck yet! It works on 4.14.0
-
-bash-4.4$ sudo make TEST=021\* test-misc
[TEST]   misc-tests.sh
[TEST/misc]   021-image-multi-devices
-bash-4.4$ sudo dmesg
[  162.635824] BTRFS: device fsid f0c92f1b-c1ba-4576-8a07-8ce4f7b5bacd
devid 1 transid 5 /dev/loop0
[  162.672952] BTRFS: device fsid f0c92f1b-c1ba-4576-8a07-8ce4f7b5bacd
devid 2 transid 5 /dev/loop1
[  162.692848] BTRFS info (device loop1): disk space caching is enabled
[  162.695725] BTRFS info (device loop1): has skinny extents
[  162.698003] BTRFS info (device loop1): flagging fs with big metadata feature
[  162.701643] BTRFS info (device loop1): creating UUID tree
[  163.792851] BTRFS warning (device loop0): unrecognized super flag:
17179869184
[  163.796350] BTRFS info (device loop0): disk space caching is enabled
[  163.807644] BTRFS info (device loop0): has skinny extents
[  163.811613] BTRFS info (device loop0): checking UUID tree
[  163.841625] BTRFS error (device loop0): 1 errors while writing supers
[  163.845457] BTRFS: error (device loop0) in write_all_supers:3709:
errno=-5 IO failure (1 errors while writing supers)
[  163.851520] BTRFS info (device loop0): forced readonly
[  163.855184] BTRFS warning (device loop0): Skipping commit of
aborted transaction.
[  163.859121] BTRFS: error (device loop0) in
cleanup_transaction:1873: errno=-5 IO failure
[  163.862577] BTRFS info (device loop0): delayed_refs has NO entry
[  163.865623] BTRFS error (device loop0): commit super ret -5
[  163.868303] BTRFS error (device loop0): cleaner transaction attach
returned -30
-bash-4.4$ uname -a
Linux ip-172-31-33-196.us-west-2.compute.internal 4.14.0 #1 SMP Fri
Nov 24 09:48:11 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
-

Cheers,
Lakshmipathi.G
http://www.giis.co.in http://www.webminal.org
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: [PATCH v7 3/5] bpf: add a bpf_override_function helper

2017-11-24 Thread Daniel Borkmann
On 11/22/2017 10:23 PM, Josef Bacik wrote:
> From: Josef Bacik 
> 
> Error injection is sloppy and very ad-hoc.  BPF could fill this niche
> perfectly with it's kprobe functionality.  We could make sure errors are
> only triggered in specific call chains that we care about with very
> specific situations.  Accomplish this with the bpf_override_funciton
> helper.  This will modify the probe'd callers return value to the
> specified value and set the PC to an override function that simply
> returns, bypassing the originally probed function.  This gives us a nice
> clean way to implement systematic error injection for all of our code
> paths.
> 
> Acked-by: Alexei Starovoitov 
> Acked-by: Ingo Molnar 
> Signed-off-by: Josef Bacik 

Series looks good to me as well; BPF bits:

Acked-by: Daniel Borkmann 
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Qu Wenruo


On 2017年11月24日 16:20, Lakshmipathi.G wrote:
> Okay thanks for the details on btrfs-image. I was about to try with
> previous version of
> btrfs-image.

No, it's not about btrfs-image itself, the flag is introduced long long
long way ago.

It's about kernel change which doesn't allow image with such flag to be
mounted AFAIK.

Thanks,
Qu

> 
> I'll use v4.14 
> (https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.14.tar.xz)
> and check
> again.
> 
> 
> Cheers,
> Lakshmipathi.G
> http://www.giis.co.in http://www.webminal.org
> 
> 
> On Fri, Nov 24, 2017 at 1:36 PM, Qu Wenruo  wrote:
>>
>>
>> On 2017年11月24日 15:47, Lakshmipathi.G wrote:
>>> Hi Qu,
>>>
>>> I'm using btrfs-progs devel branch. mount command is failing with misc/021.
>>> When I tried to manually mount it. dmesg shows  "unrecognized super
>>> flag: 17179869184"
>>> and interestingly "df -h" shows 16E as available!
>>
>> That flag is meta dump flag, any image recovered from btrfs-image should
>> have that flag.
>>
>> 16E may also related to that, but not a big problem as btrfs-image
>> result should only be used by developers, and all its data is wiped out.
>> (Maybe one day we will not allow RW mount for such image)
>>
>>
>> The problem happens when you try to mount the image, there may be some
>> new commits doing more restrict check for rw mount in devel branch.
>>
>> Would you please try v4.14 kernel?
>>
>> Thanks,
>> Qu
>>
>>>
>>> https://asciinema.org/a/4JGYg7YGQz1PHtdBRJ6AVw0fh
>>>
>>> Let me check further.
>>> 
>>> Cheers,
>>> Lakshmipathi.G
>>> http://www.giis.co.in http://www.webminal.org
>>>
>>>
>>> On Fri, Nov 24, 2017 at 9:55 AM, Lakshmipathi.G
>>>  wrote:
 Hi Qu,

 I'm using the same instance. Let me check again with and without the
 commits list on Nov-23
 (https://github.com/kdave/btrfs-devel/commits/misc-next) and get back
 with results.

 
 Cheers,
 Lakshmipathi.G
 http://www.giis.co.in http://www.webminal.org
>>> --
>>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>>> the body of a message to majord...@vger.kernel.org
>>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>>
>>
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



signature.asc
Description: OpenPGP digital signature


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Lakshmipathi.G
Okay thanks for the details on btrfs-image. I was about to try with
previous version of
btrfs-image.

I'll use v4.14 (https://cdn.kernel.org/pub/linux/kernel/v4.x/linux-4.14.tar.xz)
and check
again.


Cheers,
Lakshmipathi.G
http://www.giis.co.in http://www.webminal.org


On Fri, Nov 24, 2017 at 1:36 PM, Qu Wenruo  wrote:
>
>
> On 2017年11月24日 15:47, Lakshmipathi.G wrote:
>> Hi Qu,
>>
>> I'm using btrfs-progs devel branch. mount command is failing with misc/021.
>> When I tried to manually mount it. dmesg shows  "unrecognized super
>> flag: 17179869184"
>> and interestingly "df -h" shows 16E as available!
>
> That flag is meta dump flag, any image recovered from btrfs-image should
> have that flag.
>
> 16E may also related to that, but not a big problem as btrfs-image
> result should only be used by developers, and all its data is wiped out.
> (Maybe one day we will not allow RW mount for such image)
>
>
> The problem happens when you try to mount the image, there may be some
> new commits doing more restrict check for rw mount in devel branch.
>
> Would you please try v4.14 kernel?
>
> Thanks,
> Qu
>
>>
>> https://asciinema.org/a/4JGYg7YGQz1PHtdBRJ6AVw0fh
>>
>> Let me check further.
>> 
>> Cheers,
>> Lakshmipathi.G
>> http://www.giis.co.in http://www.webminal.org
>>
>>
>> On Fri, Nov 24, 2017 at 9:55 AM, Lakshmipathi.G
>>  wrote:
>>> Hi Qu,
>>>
>>> I'm using the same instance. Let me check again with and without the
>>> commits list on Nov-23
>>> (https://github.com/kdave/btrfs-devel/commits/misc-next) and get back
>>> with results.
>>>
>>> 
>>> Cheers,
>>> Lakshmipathi.G
>>> http://www.giis.co.in http://www.webminal.org
>> --
>> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
>> the body of a message to majord...@vger.kernel.org
>> More majordomo info at  http://vger.kernel.org/majordomo-info.html
>>
>
--
To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
the body of a message to majord...@vger.kernel.org
More majordomo info at  http://vger.kernel.org/majordomo-info.html


Re: misc/021-image-multi-devices fails on latest btrfs-devel/misc-next

2017-11-24 Thread Qu Wenruo


On 2017年11月24日 15:47, Lakshmipathi.G wrote:
> Hi Qu,
> 
> I'm using btrfs-progs devel branch. mount command is failing with misc/021.
> When I tried to manually mount it. dmesg shows  "unrecognized super
> flag: 17179869184"
> and interestingly "df -h" shows 16E as available!

That flag is meta dump flag, any image recovered from btrfs-image should
have that flag.

16E may also related to that, but not a big problem as btrfs-image
result should only be used by developers, and all its data is wiped out.
(Maybe one day we will not allow RW mount for such image)


The problem happens when you try to mount the image, there may be some
new commits doing more restrict check for rw mount in devel branch.

Would you please try v4.14 kernel?

Thanks,
Qu

> 
> https://asciinema.org/a/4JGYg7YGQz1PHtdBRJ6AVw0fh
> 
> Let me check further.
> 
> Cheers,
> Lakshmipathi.G
> http://www.giis.co.in http://www.webminal.org
> 
> 
> On Fri, Nov 24, 2017 at 9:55 AM, Lakshmipathi.G
>  wrote:
>> Hi Qu,
>>
>> I'm using the same instance. Let me check again with and without the
>> commits list on Nov-23
>> (https://github.com/kdave/btrfs-devel/commits/misc-next) and get back
>> with results.
>>
>> 
>> Cheers,
>> Lakshmipathi.G
>> http://www.giis.co.in http://www.webminal.org
> --
> To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in
> the body of a message to majord...@vger.kernel.org
> More majordomo info at  http://vger.kernel.org/majordomo-info.html
> 



signature.asc
Description: OpenPGP digital signature