When the delayed refs for a head are all run, eventually
cleanup_ref_head is called which (in case of deletion) obtains a
reference for the relevant btrfs_space_info struct by querying the bg
for the range. This is problematic because when the last extent of a
bg is deleted a race window emerges be
On 2018年04月11日 15:03, Su Yue wrote:
> This tests is most similar to xfstests generic/405.
> It calls device mapper to create a thin provision device with small
> backing size and big virtual size. mkfs.btrfs should fail on such
> devices.
>
> This test should pass after commit e805b143a4fe
> ("b
On Wed, Apr 11, 2018 at 10:04:49AM +0300, Nikolay Borisov wrote:
> When the delayed refs for a head are all run, eventually
> cleanup_ref_head is called which (in case of deletion) obtains a
> reference for the relevant btrfs_space_info struct by querying the bg
> for the range. This is problematic
On 11.04.2018 10:09, Omar Sandoval wrote:
> On Wed, Apr 11, 2018 at 10:04:49AM +0300, Nikolay Borisov wrote:
>> When the delayed refs for a head are all run, eventually
>> cleanup_ref_head is called which (in case of deletion) obtains a
>> reference for the relevant btrfs_space_info struct by que
On 04/11/2018 03:05 PM, Qu Wenruo wrote:
On 2018年04月11日 15:03, Su Yue wrote:
This tests is most similar to xfstests generic/405.
It calls device mapper to create a thin provision device with small
backing size and big virtual size. mkfs.btrfs should fail on such
devices.
This test should pa
On 10.04.2018 18:49, David Sterba wrote:
> On Thu, Feb 22, 2018 at 06:12:13PM +0200, Nikolay Borisov wrote:
>> Currently this function handles both the READ and WRITE dio cases. This
>> is facilitated by a bunch of 'if' statements, a goto short-circuit
>> statement and a very perverse aliasing of
On 2018年04月11日 15:28, Su Yue wrote:
>
>
> On 04/11/2018 03:05 PM, Qu Wenruo wrote:
>>
>>
>> On 2018年04月11日 15:03, Su Yue wrote:
>>> This tests is most similar to xfstests generic/405.
>>> It calls device mapper to create a thin provision device with small
>>> backing size and big virtual size.
The old flag OPEN_CTREE_FS_PARTIAL is in fact quite easy to be confused
with OPEN_CTREE_PARTIAL, which allow btrfs-progs to open damaged
filesystem (like corrupted extent/csum tree).
However OPEN_CTREE_FS_PARTIAL, unlike its name, is just allowing
btrfs-progs to open fs with temporary superblocks
Corrupted extent tree (either the root node or leaf) can normally block
us from open the fs.
As normally open_ctree() has the following call chain:
__open_ctree_fd()
|- btrfs_setup_all_roots()
|- btrfs_read_block_groups()
And we will search block group items in extent tree.
And considerin
On 11.04.2018 10:29, Qu Wenruo wrote:
>
>
> On 2018年04月11日 15:28, Su Yue wrote:
>>
>>
>> On 04/11/2018 03:05 PM, Qu Wenruo wrote:
>>>
>>>
>>> On 2018年04月11日 15:03, Su Yue wrote:
This tests is most similar to xfstests generic/405.
It calls device mapper to create a thin provision devic
On 2018年04月11日 15:37, Nikolay Borisov wrote:
>
>
> On 11.04.2018 10:29, Qu Wenruo wrote:
>>
>>
>> On 2018年04月11日 15:28, Su Yue wrote:
>>>
>>>
>>> On 04/11/2018 03:05 PM, Qu Wenruo wrote:
On 2018年04月11日 15:03, Su Yue wrote:
> This tests is most similar to xfstests generic/405.
Currently this function handles both the READ and WRITE dio cases. This
is facilitated by a bunch of 'if' statements, a goto short-circuit
statement and a very perverse aliasing of "!created"(READ) case
by setting lockstart = lockend and checking for lockstart < lockend for
detecting the write. Let
Now that the read side is extracted into its own function, do the same
to the write side. This leaves btrfs_get_blocks_direct_write with the
sole purpose of handling common locking required. Also flip the
condition in btrfs_get_blocks_direct_write so that the write case
comes first and we check for
Corrupted extent tree (either the root node or leaf) can normally block
us from open the fs.
As normally open_ctree() has the following call chain:
__open_ctree_fd()
|- btrfs_setup_all_roots()
|- btrfs_read_block_groups()
And we will search block group items in extent tree.
And considerin
When the delayed refs for a head are all run, eventually
cleanup_ref_head is called which (in case of deletion) obtains a
reference for the relevant btrfs_space_info struct by querying the bg
for the range. This is problematic because when the last extent of a
bg is deleted a race window emerges be
On 11.04.2018 10:59, Nikolay Borisov wrote:
> When the delayed refs for a head are all run, eventually
> cleanup_ref_head is called which (in case of deletion) obtains a
> reference for the relevant btrfs_space_info struct by querying the bg
> for the range. This is problematic because when the l
Currently __endio_write_update_ordered uses labels to implement
what is essentially a simple while loop. This makes the code more
cumbersome to follow than it actually has to be. No functional
changes. No xfstest regressions were found during testing.
Signed-off-by: Nikolay Borisov
---
fs/btrfs/
Locks should generally be released in the oppposite order they are
acquired. Generally lock acquisiton ordering is used to ensure
deadlocks don't happen. However, as becomes more complicated it's
best to also maintain proper unlock order so as to avoid possible dead
locks. This was found by code in
The second if is really a subcase of ret being less than 0. So
introduce a generic if (ret < 0) check, and inside have another if
which explicitly handles the -ENOSPC and any other errors. No
functional changes.
Signed-off-by: Nikolay Borisov
---
fs/btrfs/extent-tree.c | 12 +++-
1 file
do_chunk_alloc implements logic to detect whether there is currently
pending chunk allocation (by means of space_info->chunk_alloc being
set) and if so it loops around to the 'again' label. Additionally,
based on the state of the space_info (e.g. whether it's full or not)
and the return value of s
Here's a collection of various cleanups I've done while working on other
things.
All of these should be pretty low-risk, have soaked for a while on my test
branch and survived multiple xfstest runs.
They strive to mostly untangle the code and make it more readable.
Nikolay Borisov (4):
btr
AC_PATH_PROG won't fail even if it fails to find command path.
xmlto is required for document build and we should report error
if it doesn't exist at configure time.
Signed-off-by: Tomohiro Misono
---
configure.ac | 6 +-
1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/configure
On 2018年04月11日 16:29, Misono Tomohiro wrote:
> AC_PATH_PROG won't fail even if it fails to find command path.
> xmlto is required for document build and we should report error
> if it doesn't exist at configure time.
>
> Signed-off-by: Tomohiro Misono
Looks good.
Reviewed-by: Qu Wenruo
Than
From: Omar Sandoval
We have a build system internally which only needs to build the
libraries out of a repository, not any binaries. I looked at how this
works with other projects, and the best example was util-linux, which
makes it possible to enable or disable everything individually. This is
n
From: Omar Sandoval
These don't build anymore and don't appear to be used for anything.
Signed-off-by: Omar Sandoval
---
Makefile | 10 +-
dir-test.c | 518 ---
quick-test.c | 226 --
3 files changed, 1 insertion(+), 75
This patch enhance the following things:
- tree block header
* add generation and owner output for node and leaf
- node pointer generation output
- allow btrfs_print_tree() to not follow nodes
* just like btrfs-progs
Please note that, although function btrfs_print_tree() is not called by
anyo
It's pretty handy if we can get debug output for locking status of an
extent buffer, specially for race related debugging.
So add the following output for btrfs_print_tree() and
btrfs_print_leaf():
- refs
- write_locks (as w:%u)
- read_locks (as r:%u)
- blocking_writers (as bw:%u)
- blocking_reade
Hi
While trying to compile my kernel module on suse 12.2 kernel
4.4.103-92.53-default
I recieve the following warning: error: redefinition of 'struct
btrfs_ioctl_defrag_range_args
I see that struct is defined in 2 places:
/lib/modules/4.4.103-92.53-default/source/fs/btrfs/ctree.h:1985:8
/usr/src/li
On Wed, Apr 11, 2018 at 05:29:00PM +0900, Misono Tomohiro wrote:
> AC_PATH_PROG won't fail even if it fails to find command path.
> xmlto is required for document build and we should report error
> if it doesn't exist at configure time.
>
> Signed-off-by: Tomohiro Misono
Applied, thanks.
--
To u
On Tue, Apr 10, 2018 at 11:03:41PM +0300, Baruch Siach wrote:
> Add libbtrfsutil objects to btrfs.static link command. This fixes static
> build failure:
>
> utils.static.o: In function `parse_qgroupid':
> utils.c:(.text.parse_qgroupid+0xb0): undefined reference to
> `btrfs_util_is_subvolume'
> p
On Wed, Apr 11, 2018 at 03:29:35PM +0800, Qu Wenruo wrote:
> The old flag OPEN_CTREE_FS_PARTIAL is in fact quite easy to be confused
> with OPEN_CTREE_PARTIAL, which allow btrfs-progs to open damaged
> filesystem (like corrupted extent/csum tree).
>
> However OPEN_CTREE_FS_PARTIAL, unlike its name
My btrfs laptop had a power outage and failed to boot with "parent
transid verify failed..." errors. (I have backups).
I couldn't rw mount on a live disk, but could ro mount. I tried btrfs
scrub and then btrfs check --repair to no avail. However, btrfs rescue
zero-log _did_ work; the drive can be
On Wed, Apr 11, 2018 at 03:47:35PM +0800, Qu Wenruo wrote:
> What about using lvm other than plain dmsetup to create thin provision
> devices?
> IIRC this should be much easier to implement, without all the linear
> target hassles.
> >>>
> >>> Yes, doing it by LVM is more conven
On Wed, Apr 11, 2018 at 03:03:30PM +0800, Su Yue wrote:
> This tests is most similar to xfstests generic/405.
> It calls device mapper to create a thin provision device with small
> backing size and big virtual size. mkfs.btrfs should fail on such
> devices.
>
> This test should pass after commit
On 04/11/2018 02:32 AM, Qu Wenruo wrote:
[...]
so to get rid of generate_tab_indent and indent_str
>>>
>>> And we need to call such functions in each helper macros, with
>>> duplicated codes.
>>
>> Please look at the asm generated: even if the "source generated" by the
>> expansion of the ma
On 11.04.2018 20:15, Goffredo Baroncelli wrote:
> On 04/11/2018 02:32 AM, Qu Wenruo wrote:
> [...]
> so to get rid of generate_tab_indent and indent_str
And we need to call such functions in each helper macros, with
duplicated codes.
>>>
>>> Please look at the asm generated: e
On 2018年04月12日 01:15, Goffredo Baroncelli wrote:
> On 04/11/2018 02:32 AM, Qu Wenruo wrote:
> [...]
> so to get rid of generate_tab_indent and indent_str
And we need to call such functions in each helper macros, with
duplicated codes.
>>>
>>> Please look at the asm generated:
On 2018年04月11日 23:33, Tom Vincent wrote:
> My btrfs laptop had a power outage and failed to boot with "parent
> transid verify failed..." errors. (I have backups).
Metadata corruption, again.
I'm curious about what's the underlying disk?
Is it plain physical device? Or have other layers like bc
From: Omar Sandoval
This test creates a 256 MB filesystem and then writes a 200 MB file.
With separate data and metadata, Btrfs will run out of data space since
it needs to allocate some metadata space. Use mixed mode, which is the
recommendation for smaller filesystems.
Signed-off-by: Omar Sand
From: Omar Sandoval
generic/427 fails on Btrfs with a cryptic "pread: Success" message. This
is because an aio pwrite fails with ENOSPC, so the file isn't as long as
we expect it to be. Make sure we check the result of the aio writes and
also print a more explicit message for short reads (which a
On Wed, Apr 04, 2018 at 08:17:22PM +0800, Qu Wenruo wrote:
>
>
> On 2018年04月04日 16:53, Nikolay Borisov wrote:
> >
> >
> > On 3.04.2018 10:30, Qu Wenruo wrote:
> >> Hi David,
> >>
> >> I didn't see this patch merged in your misc-next branch but only the
> >> remaining patches.
> >>
> >> However
On Wed, Apr 11, 2018 at 04:47:30PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
>
> This test creates a 256 MB filesystem and then writes a 200 MB file.
> With separate data and metadata, Btrfs will run out of data space since
> it needs to allocate some metadata space. Use mixed mode, which
On Thu, Apr 12, 2018 at 10:56:30AM +1000, Dave Chinner wrote:
> On Wed, Apr 11, 2018 at 04:47:30PM -0700, Omar Sandoval wrote:
> > From: Omar Sandoval
> >
> > This test creates a 256 MB filesystem and then writes a 200 MB file.
> > With separate data and metadata, Btrfs will run out of data space
This tests is most similar to xfstests generic/405.
It calls device mapper to create a thin provision device with small
backing size and big virtual size. mkfs.btrfs should fail on such
devices.
This test should pass after commit e805b143a4fe
("btrfs-progs: mkfs: return nozero value on thin provis
On 04/11/2018 11:41 PM, David Sterba wrote:
On Wed, Apr 11, 2018 at 03:03:30PM +0800, Su Yue wrote:
This tests is most similar to xfstests generic/405.
It calls device mapper to create a thin provision device with small
backing size and big virtual size. mkfs.btrfs should fail on such
devices.
On Wed, Apr 11, 2018 at 12:54 AM, Nikolay Borisov wrote:
> Currently this function handles both the READ and WRITE dio cases. This
> is facilitated by a bunch of 'if' statements, a goto short-circuit
> statement and a very perverse aliasing of "!created"(READ) case
> by setting lockstart = lockend
On Tue, Apr 10, 2018 at 5:12 AM, David Sterba wrote:
> On Mon, Apr 09, 2018 at 06:23:14PM -0700, Liu Bo wrote:
>> >>> As maybe_insert_hole is only called by btrfs_cont_expand here, which
>> >>> means it's a really hole, I don't expect drop_extents would drop
>> >>> anything, we can remove this dro
uuid_mutex lock is not a per-fs lock but a global lock. The main aim of
this patch-set is to critically review the usage of this lock, and delete
the unnecessary once. By doing this we improve the concurrency of
device operations across multiple btrfs filesystems is in the system.
patch 1: Was sen
btrfs_dev_replace_finish() updates devices (soruce and target) which
are within the btrfs_fs_devices::devices or withint the cloned seed
devices (btrfs_fs_devices::seed::devices), so we don't need the global
uuid_mutex.
Signed-off-by: Anand Jain
---
fs/btrfs/dev-replace.c | 3 ---
1 file changed
read_chunk_tree() calls read_one_dev(), but for seed device we have
to search the fs_uuids list, so we need the uuid_mutex. Add a comment
comment, so that we can improve this part.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 4
1 file changed, 4 insertions(+)
diff --git a/fs/btrfs/v
__btrfs_close_devices() is un-exported, drop the __ prefix and rename it
to close_fs_devices().
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 12 ++--
1 file changed, 6 insertions(+), 6 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 793471eb3f51..ef8578bf6b
__btrfs_open_devices() is un-exported drop __ prefix and rename it to
open_fs_devices().
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 7 +++
1 file changed, 3 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index ef8578bf6b99..9ed670ffa6a5 100644
---
Declare a local btrfs_fs_devices pointer to access the same, as there are
more than one access to the pointer.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 12 +++-
1 file changed, 7 insertions(+), 5 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 1f694083c
This function uses fs_info::fs_devices number of time, however we declare
and use it only at the end, instead do it in the beginning of the function
and use it.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 13 ++---
1 file changed, 6 insertions(+), 7 deletions(-)
diff --git a/fs/b
btrfs_free_extra_devids() frees the orphan fsid::devid but its search is
limited to btrfs_fs_devices::devices, so we dont need uuid_mutex.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 3 ---
1 file changed, 3 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 8996a1df
Delete the uuid_mutex lock here as this thread access the
btrfs_fs_devices::devices only. And the device_list_mutex lock is already
in place.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 2 --
1 file changed, 2 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 4c2921
btrfs_fs_devices::list is the list of BTRFS fsid in the kernel, a generic
name 'list' makes it's search very difficult, rename it to fs_list.
Signed-off-by: Anand Jain
---
fs/btrfs/sysfs.c | 2 +-
fs/btrfs/volumes.c | 16
fs/btrfs/volumes.h | 2 +-
3 files changed, 10 insert
close_fs_devices() closes devices of a given fsid, and it is limited
to all the devices of a fsid, so we don't have to hold the global
uuid_mutex, instead we need the device_list_mutex as the device state is
being changed.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 9 -
1 file ch
Instead of de-referencing the device->fs_devices use cur_devices
which points to the same fs_devices.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 13 +
1 file changed, 9 insertions(+), 4 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index db077c505fe0..297
Move uuid_mutex closer to the exclusion section.
Signed-off-by: Anand Jain
Reviewed-by: David Sterba
---
fs/btrfs/volumes.c | 12 +---
1 file changed, 5 insertions(+), 7 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 1d9f20fc3392..7ccd1a38c634 100644
--- a/fs/b
find_device() declares struct list_head *head pointer and used only once,
instead just use it directly.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
index 9ed670ffa6a5..5714cdaf9c03
__btrfs_open_devices() declares struct list_head *head, however head is
used only once, instead use btrfs_fs_devices::devices directly.
Signed-off-by: Anand Jain
---
fs/btrfs/volumes.c | 3 +--
1 file changed, 1 insertion(+), 2 deletions(-)
diff --git a/fs/btrfs/volumes.c b/fs/btrfs/volumes.c
i
btrfs_open_devices() is using uuid_mutex, but as the btrfs_open_devices()
is just limited to open all the devices under a given fsid, so we don't
need uuid_mutex.
Instead it should hold the device_list_mutex as it updated the status
of the btrfs_fs_devices and btrfs_device of a fsid.
Signed-off-b
From: Omar Sandoval
generic/427 fails on Btrfs with a cryptic "pread: Success" message. This
is because an aio pwrite fails with ENOSPC, so the file isn't as long as
we expect it to be. Make sure we check the result of the aio writes and
also print a more explicit message for short reads (which a
From: Omar Sandoval
generic/427 creates a 256 MB filesystem and then writes a 200 MB file,
which fails on Btrfs if mixed mode is not enabled. Raise the threshold
to 1GB, which is where we typically recommend mixed mode.
Signed-off-by: Omar Sandoval
---
common/rc | 2 +-
1 file changed, 1 inser
On Wed, Apr 11, 2018 at 10:07:29PM -0700, Omar Sandoval wrote:
> From: Omar Sandoval
>
> generic/427 creates a 256 MB filesystem and then writes a 200 MB file,
> which fails on Btrfs if mixed mode is not enabled. Raise the threshold
> to 1GB, which is where we typically recommend mixed mode.
>
>
On 12.04.2018 04:33, Liu Bo wrote:
> On Wed, Apr 11, 2018 at 12:54 AM, Nikolay Borisov wrote:
>> Currently this function handles both the READ and WRITE dio cases. This
>> is facilitated by a bunch of 'if' statements, a goto short-circuit
>> statement and a very perverse aliasing of "!created"(R
On 04/10/2018 10:48 PM, David Sterba wrote:
On Fri, Mar 30, 2018 at 08:09:16AM +0800, Anand Jain wrote:
v3->v4:
Update changelog and signoff.
Reintroduce explicit check for '-EUCLEAN'
at Patch 2/8 and 5/8.
v2->v3:
Squash
4/8 btrfs: make btrfs_check_super_csum() non static
to
68 matches
Mail list logo