Re: [PATCH 3/4] btrfs: sysfs: export supported checksums

2019-10-07 Thread Johannes Thumshirn
On 07/10/2019 17:46, David Sterba wrote: [...] >> nit: This function is used only once and the ARRAY_SIZE() macro is >> descriptive enough, why not just remove it and opencoude the call to >> array_size > > Agreed, ARRAY_SIZE in loops is fine, it's a compile-time constant. Nope, btrfs_csums[] is

Re: [PATCH v3 0/5] btrfs: fix issues due to alien device

2019-10-07 Thread Anand Jain
On 10/8/19 1:36 AM, David Sterba wrote: On Mon, Oct 07, 2019 at 05:45:10PM +0800, Anand Jain wrote: v3: Fix alien device is due to wipefs in Patch4. Fix a nit in Patch3. Patches are reordered. Alien device is a device in fs_devices list having a different fsid than the expected fsid o

[PATCH v2 1/7] btrfs-progs: Refactor excluded extent functions to use fs_info

2019-10-07 Thread Qu Wenruo
The following functions are just using @root to reach fs_info: - exclude_super_stripes - free_excluded_extents - add_excluded_extent Refactor them to use fs_info directly. Signed-off-by: Qu Wenruo --- check/main.c | 4 ++-- ctree.h | 4 ++-- extent-tree.c | 20 ++-- 3 f

[PATCH v2 0/7] btrfs-progs: Support for BG_TREE feature

2019-10-07 Thread Qu Wenruo
This patchset can be fetched from github: https://github.com/adam900710/btrfs-progs/tree/bg_tree Which is based on v5.2.2 tag. This patchset provides the needed user space infrastructure for BG_TREE feature. Since it's an new incompatible feature, unlike SKINNY_METADATA, btrfs-progs is needed to

[PATCH v2 3/7] btrfs-progs: Enable read-write ability for 'bg_tree' feature

2019-10-07 Thread Qu Wenruo
Allow btrfs-progs to open, read and write 'bg_tree' enabled fs. The modification itself is not large, as block groups items are only used at 4 timing: 1) open_ctree() We only need to populate fs_info->bg_root and read block group items from fs_info->bg_root. The obvious change is, we don'

[PATCH v2 5/7] btrfs-progs: dump-tree/dump-super: Introduce support for bg tree

2019-10-07 Thread Qu Wenruo
Just a new tree called BLOCK_GROUP_TREE. Signed-off-by: Qu Wenruo --- cmds/inspect-dump-super.c | 3 ++- cmds/inspect-dump-tree.c | 5 + print-tree.c | 3 +++ 3 files changed, 10 insertions(+), 1 deletion(-) diff --git a/cmds/inspect-dump-super.c b/cmds/inspect-dump-super.c in

[PATCH v2 4/7] btrfs-progs: mkfs: Introduce -O bg-tree

2019-10-07 Thread Qu Wenruo
This allow mkfs.btrfs to create a btrfs with bg-tree feature. This patch introduce a global function, btrfs_convert_to_bg_tree() in extent-tree.c, to do the work. The workflow is pretty simple: - Create a new tree block for bg tree - Set the BG_TREE feature for superblock - Set the fs_info->conve

[PATCH v2 3/3] btrfs: Introduce new incompat feature, BG_TREE, to speed up mount time

2019-10-07 Thread Qu Wenruo
The overall idea of the new BG_TREE is pretty simple: Put BLOCK_GROUP_ITEMS into a separate tree. This brings one obvious enhancement: - Reduce mount time of large fs Although it could be possible to accept BLOCK_GROUP_ITEMS in either trees (extent root or bg root), I'll leave that kernel convert

[PATCH v2 0/3] btrfs: Introduce new incompat feature BG_TREE to hugely reduce mount time

2019-10-07 Thread Qu Wenruo
This patchset can be fetched from: https://github.com/adam900710/linux/tree/bg_tree Which is based on v5.4-rc1 tag. This patchset will hugely reduce mount time of large fs by putting all block group items into its own tree. The old behavior will try to read out all block group items at mount time

[PATCH v2 2/3] btrfs: disk-io: Remove unnecessary check before freeing chunk root

2019-10-07 Thread Qu Wenruo
In free_root_pointers(), free_root_extent_buffers() has checked if the @root parameter is NULL. So there is no need checking chunk_root before freeing it. Signed-off-by: Qu Wenruo --- fs/btrfs/disk-io.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs

[PATCH v2 7/7] btrfs-progs: btrfstune: Allow to enable bg-tree feature offline

2019-10-07 Thread Qu Wenruo
Add a new option '-b' for btrfstune, to enable bg-tree feature for a unmounted fs. This feature will convert all BLOCK_GROUP_ITEMs in extent tree to bg tree, by reusing the existing btrfs_convert_to_bg_tree() function. Signed-off-by: Qu Wenruo --- Documentation/btrfstune.asciidoc | 6 + bt

[PATCH v2 2/7] btrfs-progs: Refactor btrfs_read_block_groups()

2019-10-07 Thread Qu Wenruo
This patch does the following refactor: - Refactor parameter from @root to @fs_info - Refactor the large loop body into another function Now we have a helper function, read_one_block_group(), to handle block group cache and space info related routine. - Refactor the return value Even we hav

[PATCH v2 1/3] btrfs: block-group: Refactor btrfs_read_block_groups()

2019-10-07 Thread Qu Wenruo
Refactor the work inside the loop of btrfs_read_block_groups() into one separate function, read_one_block_group(). This allows read_one_block_group to be reused for later BG_TREE feature. Signed-off-by: Qu Wenruo --- fs/btrfs/block-group.c | 214 + 1 file

[PATCH v2 6/7] btrfs-progs: check: Introduce support for bg-tree feature

2019-10-07 Thread Qu Wenruo
Just some minor modification. - original mode: * Block group item can occur in extent tree and bg tree. - lowmem mode: * search block group items in bg tree if BG_TREE feature is set. Signed-off-by: Qu Wenruo --- check/main.c| 3 ++- check/mode-lowmem.c | 9 +++-- 2 files change

Re: [PATCH 4/5] btrfs: remove identified alien device in open_fs_devices

2019-10-07 Thread Anand Jain
On 10/8/19 1:03 AM, David Sterba wrote: On Mon, Oct 07, 2019 at 09:37:49PM +0800, Qu Wenruo wrote: On 2019/10/7 下午9:30, Nikolay Borisov wrote: On 7.10.19 г. 12:45 ч., Anand Jain wrote: Following test case explains it all, even though the degraded mount is successful the btrfs-progs fails t

Re: [PATCH 1/3] btrfs: remove unnecessary hash_init()

2019-10-07 Thread Chengguang Xu
在 星期一, 2019-10-07 23:44:45 David Sterba 撰写 > On Sat, Oct 05, 2019 at 01:17:34PM +0800, Chengguang Xu wrote: > > hash_init() is not necessary in btrfs_props_init(), > > so remove it. > > The part that explains why it's not necessary is missing in the > changelo. And looking what h

Re: [PATCH v3] btrfs-progs: add verbose option to btrfs device scan

2019-10-07 Thread Anand Jain
On 10/8/19 1:41 AM, David Sterba wrote: On Wed, Oct 02, 2019 at 12:11:52PM +0800, Anand Jain wrote: To help debug device scan issues, add verbose option to btrfs device scan. The common options like --verbose are going to be added into the global command so I'd rather avoid adding them to new

[PATCH] btrfs: block-group: Rework documentation of check_system_chunk function

2019-10-07 Thread Marcos Paulo de Souza
Commit 4617ea3a52cf (" Btrfs: fix necessary chunk tree space calculation when allocating a chunk") removed the is_allocation argument from check_system_chunk, since the formula for reserving the necessary space for allocation or removing a chunk would be the same. So, rework the comment by removin

Re: [PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-07 Thread Dennis Zhou
On Mon, Oct 07, 2019 at 04:37:28PM -0400, Josef Bacik wrote: > On Mon, Oct 07, 2019 at 04:17:34PM -0400, Dennis Zhou wrote: > > Async discard will use the free space cache as backing knowledge for > > which extents to discard. This patch plumbs knowledge about which > > extents need to be discarded

Re: [PATCH 01/19] bitmap: genericize percpu bitmap region iterators

2019-10-07 Thread Dennis Zhou
On Mon, Oct 07, 2019 at 04:26:13PM -0400, Josef Bacik wrote: > On Mon, Oct 07, 2019 at 04:17:32PM -0400, Dennis Zhou wrote: > > Bitmaps are fairly popular for their space efficiency, but we don't have > > generic iterators available. Make percpu's bitmap region iterators > > available to everyone.

RE:PERSONAL LETTER FROM MRS RASHIA AMIRA

2019-10-07 Thread Mr Barrister Hans Erich
Greetings My name is Barrister Hans Erich. I have a client who is interested to invest in your country, she is a well known politician in her country and deserve a lucrative investment partnership with you outside her country without any delay Please can you manage such investment please Kin

Re: [PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-07 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:34PM -0400, Dennis Zhou wrote: > Async discard will use the free space cache as backing knowledge for > which extents to discard. This patch plumbs knowledge about which > extents need to be discarded into the free space cache from > unpin_extent_range(). > > An untri

Re: [PATCH 02/19] btrfs: rename DISCARD opt to DISCARD_SYNC

2019-10-07 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:33PM -0400, Dennis Zhou wrote: > This series introduces async discard which will use the flag > DISCARD_ASYNC, so rename the original flag to DISCARD_SYNC as it is > synchronously done in transaction commit. > > Signed-off-by: Dennis Zhou Reviewed-by: Josef Bacik

Re: [PATCH 01/19] bitmap: genericize percpu bitmap region iterators

2019-10-07 Thread Josef Bacik
On Mon, Oct 07, 2019 at 04:17:32PM -0400, Dennis Zhou wrote: > Bitmaps are fairly popular for their space efficiency, but we don't have > generic iterators available. Make percpu's bitmap region iterators > available to everyone. > > Signed-off-by: Dennis Zhou > --- > include/linux/bitmap.h | 35

[PATCH 16/19] btrfs: keep track of discard reuse stats

2019-10-07 Thread Dennis Zhou
Keep track of how much we are discarding and how often we are reusing with async discard. Signed-off-by: Dennis Zhou --- fs/btrfs/ctree.h| 3 +++ fs/btrfs/discard.c | 5 + fs/btrfs/free-space-cache.c | 10 ++ fs/btrfs/sysfs.c| 36 +++

[PATCH 17/19] btrfs: add async discard header

2019-10-07 Thread Dennis Zhou
Give a brief overview for how async discard is implemented. Signed-off-by: Dennis Zhou --- fs/btrfs/discard.c | 34 ++ 1 file changed, 34 insertions(+) diff --git a/fs/btrfs/discard.c b/fs/btrfs/discard.c index f0088ca19d28..61e341685acd 100644 --- a/fs/btrfs/dis

[PATCH 18/19] btrfs: increase the metadata allowance for the free_space_cache

2019-10-07 Thread Dennis Zhou
Currently, there is no way for the free space cache to recover from being serviced by purely bitmaps because the extent threshold is set to 0 in recalculate_thresholds() when we surpass the metadata allowance. This adds a recovery mechanism by keeping large extents out of the bitmaps and increases

[PATCH 19/19] btrfs: make smaller extents more likely to go into bitmaps

2019-10-07 Thread Dennis Zhou
It's less than ideal for small extents to eat into our extent budget, so force extents <= 32KB into the bitmaps save for the first handful. Signed-off-by: Dennis Zhou --- fs/btrfs/free-space-cache.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/free-space-cache

[PATCH 08/19] btrfs: track discardable extents for asnyc discard

2019-10-07 Thread Dennis Zhou
The number of discardable extents will serve as the rate limiting metric for how often we should discard. This keeps track of discardable extents in the free space caches by maintaining deltas and propagating them to the global count. This also setups up a discard directory in btrfs sysfs and expo

[PATCH 09/19] btrfs: keep track of discardable_bytes

2019-10-07 Thread Dennis Zhou
Keep track of this metric so that we can understand how ahead or behind we are in discarding rate. Signed-off-by: Dennis Zhou --- fs/btrfs/ctree.h| 1 + fs/btrfs/discard.c | 1 + fs/btrfs/discard.h | 7 +++ fs/btrfs/free-space-cache.c | 32 +++

[PATCH 10/19] btrfs: calculate discard delay based on number of extents

2019-10-07 Thread Dennis Zhou
Use the number of discardable extents to help guide our discard delay interval. This value is reevaluated every transaction commit. Signed-off-by: Dennis Zhou --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/discard.c | 31 +-- fs/btrfs/discard.h | 3 +++ fs/btrf

[PATCH 06/19] btrfs: handle empty block_group removal

2019-10-07 Thread Dennis Zhou
block_group removal is a little tricky. It can race with the extent allocator, the cleaner thread, and balancing. The current path is for a block_group to be added to the unused_bgs list. Then, when the cleaner thread comes around, it starts a transaction and then proceeds with removing the block_g

[PATCH 07/19] btrfs: discard one region at a time in async discard

2019-10-07 Thread Dennis Zhou
The prior two patches added discarding via a background workqueue. This just piggybacked off of the fstrim code to trim the whole block at once. Well inevitably this is worse performance wise and will aggressively overtrim. But it was nice to plumb the other infrastructure to keep the patches easie

[PATCH 05/19] btrfs: add the beginning of async discard, discard workqueue

2019-10-07 Thread Dennis Zhou
When discard is enabled, everytime a pinned extent is released back to the block_group's free space cache, a discard is issued for the extent. This is an overeager approach when it comes to discarding and helping the SSD maintain enough free space to prevent severe garbage collection situations. T

[PATCH 11/19] btrfs: add bps discard rate limit

2019-10-07 Thread Dennis Zhou
Provide an ability to rate limit based on mbps in addition to the iops delay calculated from number of discardable extents. Signed-off-by: Dennis Zhou --- fs/btrfs/ctree.h | 2 ++ fs/btrfs/discard.c | 11 +++ fs/btrfs/sysfs.c | 30 ++ 3 files changed, 43

[PATCH 15/19] btrfs: load block_groups into discard_list on mount

2019-10-07 Thread Dennis Zhou
Async discard doesn't remember the discard state of a block_group when unmounting or when we crash. So, any block_group that is not fully used may have undiscarded regions. However, free space caches are read in on demand. Let the discard worker read in the free space cache so we can proceed with d

[PATCH 13/19] btrfs: have multiple discard lists

2019-10-07 Thread Dennis Zhou
Non-block group destruction discarding currently only had a single list with no minimum discard length. This can lead to caravaning more meaningful discards behind a heavily fragmented block group. This adds support for multiple lists with minimum discard lengths to prevent the caravan effect. We

[PATCH 12/19] btrfs: limit max discard size for async discard

2019-10-07 Thread Dennis Zhou
Throttle the maximum size of a discard so that we can provide an upper bound for the rate of async discard. While the block layer is able to split discards into the appropriate sized discards, we want to be able to account more accurately the rate at which we are consuming ncq slots as well as limi

[PATCH 14/19] btrfs: only keep track of data extents for async discard

2019-10-07 Thread Dennis Zhou
As mentioned earlier, discarding data can be done either by issuing an explicit discard or implicitly by reusing the LBA. Metadata chunks see much more frequent reuse due to well it being metadata. So instead of explicitly discarding metadata blocks, just leave them be and let the latter implicit d

[PATCH 04/19] btrfs: keep track of cleanliness of the bitmap

2019-10-07 Thread Dennis Zhou
There is a cap in btrfs in the amount of free extents that a block group can have. When it surpasses that threshold, future extents are placed into bitmaps. Instead of keeping track of if a certain bit is trimmed or not in a second bitmap, keep track of the relative state of the bitmap. With async

[PATCH 01/19] bitmap: genericize percpu bitmap region iterators

2019-10-07 Thread Dennis Zhou
Bitmaps are fairly popular for their space efficiency, but we don't have generic iterators available. Make percpu's bitmap region iterators available to everyone. Signed-off-by: Dennis Zhou --- include/linux/bitmap.h | 35 mm/percpu.c| 61 +++-

[PATCH 03/19] btrfs: keep track of which extents have been discarded

2019-10-07 Thread Dennis Zhou
Async discard will use the free space cache as backing knowledge for which extents to discard. This patch plumbs knowledge about which extents need to be discarded into the free space cache from unpin_extent_range(). An untrimmed extent can merge with everything as this is a new region. Absorbing

[PATCH 02/19] btrfs: rename DISCARD opt to DISCARD_SYNC

2019-10-07 Thread Dennis Zhou
This series introduces async discard which will use the flag DISCARD_ASYNC, so rename the original flag to DISCARD_SYNC as it is synchronously done in transaction commit. Signed-off-by: Dennis Zhou --- fs/btrfs/block-group.c | 2 +- fs/btrfs/ctree.h | 2 +- fs/btrfs/extent-tree.c | 4 ++--

[RFC PATCH 00/19] btrfs: async discard support

2019-10-07 Thread Dennis Zhou
Hello, Discard is an operation that allows for the filesystem to communicate with underlying ssds that a lba region is no longer needed. This gives the drive the more information as it tries to manage the available free space to minimize write amplification. However, discard hasn't been given the

[PATCH 4/5] btrfs: remove extent_map::bdev

2019-10-07 Thread David Sterba
We can now remove the bdev from extent_map. Previous patches made sure that bio_set_dev is correctly in all places and that we don't need to grab it from latest_bdev or pass it around inside the extent map. Signed-off-by: David Sterba --- fs/btrfs/disk-io.c| 3 --- fs/btrfs/extent_io.c |

[PATCH 3/5] btrfs: drop bio_set_dev where not needed

2019-10-07 Thread David Sterba
bio_set_dev sets a bdev to a bio and is not only setting a pointer bug also changing some state bits if there was a different bdev set before. This is one thing that's not needed. Another thing is that setting a bdev at bio allocation time is too early and actually does not work with plain redunda

[PATCH 5/5] btrfs: drop bdev argument from submit_extent_page

2019-10-07 Thread David Sterba
After previous patches removing bdev being passed around to set it to bio, it has become unused in submit_extent_page. So it now has "only" 13 parameters. Signed-off-by: David Sterba --- fs/btrfs/extent_io.c | 11 +++ 1 file changed, 3 insertions(+), 8 deletions(-) diff --git a/fs/btrfs

[PATCH 2/5] btrfs: get bdev from latest_dev for dio bh_result

2019-10-07 Thread David Sterba
To remove use of extent_map::bdev we need to find a replacement, and the latest_bdev is the only one we can use here, because inode::i_bdev and superblock::s_bdev are NULL. The only thing that DIO code uses from the bdev is the blocksize to perform alignment checks in do_blockdev_direct_IO, but we

[PATCH 1/5] btrfs: assert extent_map bdevs and lookup_map and split

2019-10-07 Thread David Sterba
This is a preparatory patch for removing extent_map::bdev. There's some history behind the code so this is only precaution to catch if things break before the actual removal happens. Logically, comparing a raw low-level block device (bdev) does not make sense for extent maps (high-level objects).

[PATCH 0/5] Remove extent_map::bdev

2019-10-07 Thread David Sterba
The extent_map::bdev is unused and and can be removed, but it's not straightforward so there are several steps. The first patch decouples bdev from map_lookup. The remaining patches clean up use of the bdev, removing a few bio_set_dev on the way. In the end, submit_extent_page is one parameter ligh

[PATCH][v3] btrfs/194: add a test for multi-subvolume fsyncing

2019-10-07 Thread Josef Bacik
I discovered a problem in btrfs where we'd end up pointing at a block we hadn't written out yet. This is triggered by a race when two different files on two different subvolumes fsync. This test exercises this path with dm-log-writes, and then replays the log at every FUA to verify the file syste

Re: [PATCH] Setup GitLab-CI for btrfs-progs

2019-10-07 Thread David Sterba
On Mon, Sep 30, 2019 at 10:26:54PM +0530, Lakshmipathi.G wrote: > Make use of GitLab-CI nested virutal environment to start QEMU instance > inside containers > and perform btrfs-progs build, execute unit test cases and save the logs. This looks good, thanks! > More details can be found at https:

Re: [PATCH v3] btrfs-progs: add verbose option to btrfs device scan

2019-10-07 Thread David Sterba
On Wed, Oct 02, 2019 at 12:11:52PM +0800, Anand Jain wrote: > To help debug device scan issues, add verbose option to btrfs device scan. The common options like --verbose are going to be added into the global command so I'd rather avoid adding them to new subcommands as this would become unnecessa

Re: [PATCH v3 0/5] btrfs: fix issues due to alien device

2019-10-07 Thread David Sterba
On Mon, Oct 07, 2019 at 05:45:10PM +0800, Anand Jain wrote: > v3: Fix alien device is due to wipefs in Patch4. > Fix a nit in Patch3. > Patches are reordered. > > Alien device is a device in fs_devices list having a different fsid than > the expected fsid or no btrfs_magic. This patch set

Re: [PATCH 4/5] btrfs: remove identified alien device in open_fs_devices

2019-10-07 Thread David Sterba
On Mon, Oct 07, 2019 at 09:37:49PM +0800, Qu Wenruo wrote: > > > On 2019/10/7 下午9:30, Nikolay Borisov wrote: > > > > > > On 7.10.19 г. 12:45 ч., Anand Jain wrote: > >> Following test case explains it all, even though the degraded mount is > >> successful the btrfs-progs fails to report the missin

Re: [PATCH 0/3] btrfs: tree-checker: False alerts fixes for log trees

2019-10-07 Thread David Sterba
On Fri, Oct 04, 2019 at 05:31:30PM +0800, Qu Wenruo wrote: > There is a false alerts of tree-checker when running fstests/btrfs/063 > in a loop. > > The bug is caused by commit 59b0d030fb30 ("btrfs: tree-checker: Try to detect > missing INODE_ITEM"). > For the full error analyse, please check the

Re: [PATCH] btrfs-progs: add sha256 as supported checksumming algorithm

2019-10-07 Thread David Sterba
On Mon, Oct 07, 2019 at 01:13:26PM +0200, Johannes Thumshirn wrote: > Signed-off-by: Johannes Thumshirn Please write even a short changelog. The patch adds the definition and allows the new hash in mkfs, that's beyond what I'd consider trivial that a subject would be sufficient. I wrote something

Re: [PATCH 3/4] btrfs: sysfs: export supported checksums

2019-10-07 Thread David Sterba
On Mon, Oct 07, 2019 at 06:36:14PM +0300, Nikolay Borisov wrote: > > > On 7.10.19 г. 12:11 ч., Johannes Thumshirn wrote: > > From: David Sterba > > > > Export supported checksum algorithms via sysfs. > > > > Co-developed-by: David Sterba > > Signed-off-by: Johannes Thumshirn > > --- > > fs

Re: [PATCH 1/3] btrfs: remove unnecessary hash_init()

2019-10-07 Thread David Sterba
On Sat, Oct 05, 2019 at 01:17:34PM +0800, Chengguang Xu wrote: > hash_init() is not necessary in btrfs_props_init(), > so remove it. The part that explains why it's not necessary is missing in the changelo. And looking what hash_init and plain DEFINE_HASHTABLE does I don't think that removing hash

Re: [PATCH v6] btrfs-progs: add xxhash64 to mkfs

2019-10-07 Thread David Sterba
On Mon, Oct 07, 2019 at 07:11:17PM +0800, Qu Wenruo wrote: > On 2019/9/26 下午6:11, Johannes Thumshirn wrote: > > Signed-off-by: Johannes Thumshirn > > --- > > Not related to the patchset itself, but it would be pretty nice if we > check the sysfs interface to guess if we can mount the fs. > > And

Re: [PATCH 3/4] btrfs: sysfs: export supported checksums

2019-10-07 Thread Nikolay Borisov
On 7.10.19 г. 12:11 ч., Johannes Thumshirn wrote: > From: David Sterba > > Export supported checksum algorithms via sysfs. > > Co-developed-by: David Sterba > Signed-off-by: Johannes Thumshirn > --- > fs/btrfs/ctree.c | 5 + > fs/btrfs/ctree.h | 2 ++ > fs/btrfs/sysfs.c | 33 +++

Re: [PATCH 1/3] btrfs: tree-checker: Fix false alerts on log trees

2019-10-07 Thread David Sterba
On Fri, Oct 04, 2019 at 03:15:51PM +0100, Filipe Manana wrote: > On Fri, Oct 4, 2019 at 11:27 AM Qu Wenruo wrote: > > Reported-by: David Sterba > > Fixes: 59b0d030fb30 ("btrfs: tree-checker: Try to detect missing > > INODE_ITEM") > > So this is bogus, since that commit is not in Linus' tree, an

Re: [PATCH 0/3] Relocation/backref cache cleanups

2019-10-07 Thread David Sterba
On Wed, Oct 02, 2019 at 02:58:55PM +0200, David Sterba wrote: > On Fri, Sep 06, 2019 at 10:15:30AM -0700, Mark Fasheh wrote: > > Hi, > > > > Relocation caches extent backrefs in an rbtree (the 'backref cache'). The > > following patches move the backref cache code out of relocation.c and into > >

Re: [PATCH] btrfs: fix Wmaybe-uninitialized warning

2019-10-07 Thread David Sterba
On Tue, Sep 03, 2019 at 12:30:19PM +0900, Austin Kim wrote: > gcc throws warning message as below: > > ‘clone_src_i_size’ may be used uninitialized in this function > [-Wmaybe-uninitialized] > #define IS_ALIGNED(x, a) (((x) & ((typeof(x))(a) - 1)) == 0) >^ > fs/btrfs/send

Re: [PATCH 0/4] Add xxhash64 and sha256 as possible new checksums

2019-10-07 Thread Nikolay Borisov
On 7.10.19 г. 12:11 ч., Johannes Thumshirn wrote: > This series adds support for two additional checksum algorithms to btrfs. > These > algorithms are xxhash64[1] and sha256[2]. > > xxhash64 is a fast non-cryptographic hash function with good collision > resistance. > It has a constant output

Re: [PATCH 2/3] btrfs: code cleanup for compression type

2019-10-07 Thread Chengguang Xu
在 星期一, 2019-10-07 07:28:32 David Sterba 撰写 > On Sat, Oct 05, 2019 at 01:17:35PM +0800, Chengguang Xu wrote: > > Let BTRFS_COMPRESS_TYPES represents the total number > > of cmpressoin types and fix related calling places. > > It will be more safe when adding new compression type > >

Re: [PATCH 4/5] btrfs: remove identified alien device in open_fs_devices

2019-10-07 Thread Qu Wenruo
On 2019/10/7 下午9:30, Nikolay Borisov wrote: > > > On 7.10.19 г. 12:45 ч., Anand Jain wrote: >> Following test case explains it all, even though the degraded mount is >> successful the btrfs-progs fails to report the missing device. >> >> mkfs.btrfs -fq -draid1 -mraid1 /dev/sdc /dev/sdd && \ >>

Re: [PATCH 4/5] btrfs: remove identified alien device in open_fs_devices

2019-10-07 Thread Nikolay Borisov
On 7.10.19 г. 12:45 ч., Anand Jain wrote: > Following test case explains it all, even though the degraded mount is > successful the btrfs-progs fails to report the missing device. > > mkfs.btrfs -fq -draid1 -mraid1 /dev/sdc /dev/sdd && \ > wipefs -a /dev/sdd && mount -o degraded /dev/sdc /btr

[PATCH] btrfs-progs: add sha256 as supported checksumming algorithm

2019-10-07 Thread Johannes Thumshirn
Signed-off-by: Johannes Thumshirn --- Makefile | 2 +- cmds/inspect-dump-super.c | 1 + crypto/hash.c | 12 crypto/hash.h | 1 + crypto/sha224-256.c | 4 ++-- ctree.c | 1 + ctree.h | 1 + disk-i

Re: [PATCH v6] btrfs-progs: add xxhash64 to mkfs

2019-10-07 Thread Qu Wenruo
On 2019/9/26 下午6:11, Johannes Thumshirn wrote: > Signed-off-by: Johannes Thumshirn > --- Not related to the patchset itself, but it would be pretty nice if we check the sysfs interface to guess if we can mount the fs. And if not supported, a warning (at stdout) will not hurt. Thanks, Qu > >

[PATCH 4/5] btrfs: remove identified alien device in open_fs_devices

2019-10-07 Thread Anand Jain
Following test case explains it all, even though the degraded mount is successful the btrfs-progs fails to report the missing device. mkfs.btrfs -fq -draid1 -mraid1 /dev/sdc /dev/sdd && \ wipefs -a /dev/sdd && mount -o degraded /dev/sdc /btrfs && \ btrfs fi show -m /btrfs Label: none uuid: 2

[PATCH 2/5] btrfs: include non-missing as a qualifier for the latest_bdev

2019-10-07 Thread Anand Jain
btrfs_free_extra_devids() reorgs fs_devices::latest_bdev to point to the bdev with greatest device::generation number. For a typical-missing device the generation number is zero so fs_devices::latest_bdev will never point to it. But if the missing device is due to alienating [1], then device::gene

[PATCH v3 3/5] btrfs: remove identified alien btrfs device in open_fs_devices

2019-10-07 Thread Anand Jain
In open_fs_devices() we identify alien device but we don't reset its the device::name. So progs device list does not show the device missing as shown in the script below. mkfs.btrfs -fq /dev/sdd && mount /dev/sdd /btrfs mkfs.btrfs -fq -draid1 -mraid1 /dev/sdc /dev/sdb sleep 3 # avoid racing with u

[PATCH 1/5] btrfs: drop useless goto in open_fs_devices

2019-10-07 Thread Anand Jain
There is no need of goto out in open_fs_devices() as there is nothing special done at %out:. So refactor it. Signed-off-by: Anand Jain Reviewed-by: Nikolay Borisov --- fs/btrfs/volumes.c | 12 +--- 1 file changed, 5 insertions(+), 7 deletions(-) diff --git a/fs/btrfs/volumes.c b/fs/btr

[PATCH v3 0/5] btrfs: fix issues due to alien device

2019-10-07 Thread Anand Jain
v3: Fix alien device is due to wipefs in Patch4. Fix a nit in Patch3. Patches are reordered. Alien device is a device in fs_devices list having a different fsid than the expected fsid or no btrfs_magic. This patch set fixes issues found due to the same. Patch1: is a cleanup patch, not rel

[PATCH 5/5] btrfs: free alien device due to device add

2019-10-07 Thread Anand Jain
When the old device has new fsid through btrfs device add -f our fs_devices list has an alien device in one of the fs_devices. By having an alien device in fs_devices, we have two issues so far 1. missing device is not shows as missing in the userland Which is due to cracks in the function btrf

Re: [PATCH 0/4] btrfs: fix issues due to alien device

2019-10-07 Thread Anand Jain
On 4/10/19 4:19 PM, Nikolay Borisov wrote: On 4.10.19 г. 10:49 ч., Anand Jain wrote: Alien device is a device in fs_devices list having a different fsid than the expected fsid. This patch set fixes issues found due to the same. Are you going to submit an fstests patch for this bug ? Yes.

[PATCH 2/2] fstest: btrfs/197: test for alien devices

2019-10-07 Thread Anand Jain
Test if btrfs.ko sucessfully identifies and reports the missing device, if the missed device contians no btrfs magic string. Signed-off-by: Anand Jain --- tests/btrfs/197 | 79 + tests/btrfs/197.out | 25 + tests/btrfs/group

[PATCH 1/2] fstest: btrfs/196: test for alien btrfs-devices

2019-10-07 Thread Anand Jain
Test if btrfs.ko sucessfully identifies and reports the missing device, if the missed device contians someother btrfs. Signed-off-by: Anand Jain --- tests/btrfs/196 | 77 + tests/btrfs/196.out | 25 + tests/btrfs/group |

[PATCH 4/4] btrfs: show used checksum driver per filesystem in sysfs

2019-10-07 Thread Johannes Thumshirn
Show the used driver for the checksum algorithm for the filesystem in sysfs. Signed-off-by: Johannes Thumshirn --- fs/btrfs/sysfs.c | 15 +++ 1 file changed, 15 insertions(+) diff --git a/fs/btrfs/sysfs.c b/fs/btrfs/sysfs.c index aeebbdfe1a98..11a3cf7f563e 100644 --- a/fs/btrfs/sysf

[PATCH 1/4] btrfs: add xxhash64 to checksumming algorithms

2019-10-07 Thread Johannes Thumshirn
Add xxhash64 to the list of possible checksumming algorithms used by BTRFS. Signed-off-by: Johannes Thumshirn --- fs/btrfs/Kconfig| 1 + fs/btrfs/ctree.c| 1 + fs/btrfs/disk-io.c | 1 + fs/btrfs/super.c| 1 + include/uapi/linux/btrfs_t

[PATCH 0/4] Add xxhash64 and sha256 as possible new checksums

2019-10-07 Thread Johannes Thumshirn
This series adds support for two additional checksum algorithms to btrfs. These algorithms are xxhash64[1] and sha256[2]. xxhash64 is a fast non-cryptographic hash function with good collision resistance. It has a constant output length of 8 Byte (64 Bit), it provides a good trade-off between col

[PATCH 2/4] btrfs: add sha256 to checksumming algorithms

2019-10-07 Thread Johannes Thumshirn
Add sha256 to the list of possible checksumming algorithms used by BTRFS. Signed-off-by: Johannes Thumshirn --- fs/btrfs/Kconfig| 1 + fs/btrfs/ctree.c| 1 + fs/btrfs/disk-io.c | 1 + fs/btrfs/super.c| 1 + include/uapi/linux/btrfs_tre

[PATCH 3/4] btrfs: sysfs: export supported checksums

2019-10-07 Thread Johannes Thumshirn
From: David Sterba Export supported checksum algorithms via sysfs. Co-developed-by: David Sterba Signed-off-by: Johannes Thumshirn --- fs/btrfs/ctree.c | 5 + fs/btrfs/ctree.h | 2 ++ fs/btrfs/sysfs.c | 33 + 3 files changed, 40 insertions(+) diff --git