On 27.11.18 г. 9:30 ч., Qu Wenruo wrote:
>
>
> On 2018/11/27 下午3:28, Nikolay Borisov wrote:
>>
>>
>> On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
>>> With existing dev extents cleaned up, now we can rebuild dev extents
>>> using the correct chunk tree.
>>>
>>> Since new dev extents are all rebuild
On 2018/11/27 下午3:29, Nikolay Borisov wrote:
>
>
> On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
>> Test case misc/021 is testing if we could mount a single disk btrfs
>> image recovered from multi disk fs.
>>
>> The problem is, current kernel has extra check for block group, chunk
>> and dev exten
On 2018/11/27 下午3:28, Nikolay Borisov wrote:
>
>
> On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
>> With existing dev extents cleaned up, now we can rebuild dev extents
>> using the correct chunk tree.
>>
>> Since new dev extents are all rebuild from scratch, even we're restoring
>> image from mult
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
> Test case misc/021 is testing if we could mount a single disk btrfs
> image recovered from multi disk fs.
>
> The problem is, current kernel has extra check for block group, chunk
> and dev extent.
> This means any image can't pass btrfs check for chun
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
> With existing dev extents cleaned up, now we can rebuild dev extents
> using the correct chunk tree.
>
> Since new dev extents are all rebuild from scratch, even we're restoring
> image from multi-device fs to single disk, we won't have any problem
> r
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
> This patch will remove all existing dev extents for later rebuild.
>
> Signed-off-by: Qu Wenruo
> ---
> image/main.c | 68
> 1 file changed, 68 insertions(+)
>
> diff --git a/image/main.c b/image/
Alexander Fieroch posted on Mon, 26 Nov 2018 11:23:00 +0100 as excerpted:
> Am 26.11.18 um 09:13 schrieb Qu Wenruo:
>> The corruption itself looks like some disk error, not some btrfs error
>> like transid error.
>
> You're right! SMART has an increased value for one harddisk on
> reallocated sec
On 2018/11/27 下午3:15, Nikolay Borisov wrote:
>
>
> On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
>> Since btrfs-image is just restoring tree blocks without really check if
>> that tree block contents makes sense, for multi-device image, block
>> group items will keep that incorrect block group flag
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
> Since btrfs-image is just restoring tree blocks without really check if
> that tree block contents makes sense, for multi-device image, block
> group items will keep that incorrect block group flags.
>
> For example, for a metadata RAID1 data RAID0 btr
On 2018/11/27 下午3:13, Nikolay Borisov wrote:
>
>
> On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
>> Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM
>> size.
>> Later we need to do more fixup works, so change the name to
>> fixup_chunks_and_devices() and refactor the original d
On 27.11.18 г. 4:33 ч., Qu Wenruo wrote:
> Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM
> size.
> Later we need to do more fixup works, so change the name to
> fixup_chunks_and_devices() and refactor the original device size fixup
> to fixup_device_size().
>
> Signed-of
Signed-off-by: Misono Tomohiro
Signed-off-by: David Sterba
---
tests/cli-tests/009-subvolume-list/test.sh | 130 +
1 file changed, 130 insertions(+)
create mode 100755 tests/cli-tests/009-subvolume-list/test.sh
diff --git a/tests/cli-tests/009-subvolume-list/test.sh
b/test
Allow non-privileged user to call subvolume show if new ioctls
(BTRFS_IOC_GET_SUBVOL_INFO/BTRFS_IOC_GET_SUBVOL_ROOTREF,
BTRFS_IOC_INO_LOOKUP_USER, from kernel 4.18) are available.
Non-privileged user still cannot use -r or -u option.
The behavior for root user is the same as before.
There are som
This is a copy of non-merged following patch originally written
by Omar Sandoval:
btrfs-progs: use libbtrfsutil for subvolume list
expect this commit keeps libbtrfs implementation which above commit
tries to remove (therefore this adds suffix _v2 for struct/function).
Original Author: Omar Sando
O_NOATIME flag requires effective UID of process matches file's owner
or has CAP_FOWNER capabilities. Fallback to open without O_NOATIME flag
so that non-privileged user can also call find_mount_root().
This is a preparation work to allow non-privileged user to call
"subvolume show".
Signed-off-b
Hello,
This is basically the resend of
"[PATCH v2 00/20] btrfs-progs: Rework of "subvolume list/show" and relax the
root privileges of them" [1]
which I submitted in June. The aim of this series is to allow non-privileged
user
to use basic subvolume functionality (create/list/snapshot/
Add helper function to check that if user 'nobody' exists or not.
Note that 'nobody' should not have root privileges as it will be used
to test the behavior of non-privileged user.
Signed-off-by: Misono Tomohiro
Signed-off-by: David Sterba
---
tests/common | 10 ++
1 file changed, 10 in
Currently, -a option add filter and change subvolume path as follows:
- If a subvolume is a child of the specified path, nothing changes
- otherwise, adds to head
This is rather meaningless, so let's remove this filter.
As a result, the behavior of -a option becomes the same as
default behav
Change the default behavior of "subvolume list" and allow non-privileged
user to call it as well.
>From this commit, by default it only lists subvolumes under the specified
path (incl. the path itself except top-level subvolume. the path needs
to be a subvolume). Also, if kernel supports new ioctl
No functional changes.
This is a preparation work for reworking "subvolume list".
Signed-off-by: Misono Tomohiro
Signed-off-by: David Sterba
---
cmds-subvolume.c | 50 +---
1 file changed, 30 insertions(+), 20 deletions(-)
diff --git a/cmds-subvolume
> -Original Message-
> From: David Sterba [mailto:dste...@suse.cz]
> Sent: Tuesday, November 27, 2018 1:18 AM
> To: Omar Sandoval
> Cc: linux-btrfs@vger.kernel.org; kernel-t...@fb.com; Misono, Tomohiro
>
> Subject: Re: [PATCH 00/10] btrfs-progs: my libbtrfsutil patch queue
>
> On Tue, No
> On Nov 19, 2018, at 12:34 PM, Daniel Kiper wrote:
>
> On Mon, Nov 19, 2018 at 11:20:06AM -0800, Nick Terrell wrote:
>> Hi all,
>>
>> This patch set imports the upstream zstd library, adds zstd support to the
>> btrfs module, and adds a test case. I've also tested the patch set by storing
>>
Test case misc/021 is testing if we could mount a single disk btrfs
image recovered from multi disk fs.
The problem is, current kernel has extra check for block group, chunk
and dev extent.
This means any image can't pass btrfs check for chunk tree will not
mount.
So do extra btrfs check before m
This patch will remove all existing dev extents for later rebuild.
Signed-off-by: Qu Wenruo
---
image/main.c | 68
1 file changed, 68 insertions(+)
diff --git a/image/main.c b/image/main.c
index 9060f6b1b665..707568f22e01 100644
--- a/image/m
Since btrfs-image is just restoring tree blocks without really check if
that tree block contents makes sense, for multi-device image, block
group items will keep that incorrect block group flags.
For example, for a metadata RAID1 data RAID0 btrfs recovered to a single
disk, its chunk tree will loo
With existing dev extents cleaned up, now we can rebuild dev extents
using the correct chunk tree.
Since new dev extents are all rebuild from scratch, even we're restoring
image from multi-device fs to single disk, we won't have any problem
reported by btrfs check.
Signed-off-by: Qu Wenruo
---
Current fixup_devices() will only remove DEV_ITEMs and reset DEV_ITEM
size.
Later we need to do more fixup works, so change the name to
fixup_chunks_and_devices() and refactor the original device size fixup
to fixup_device_size().
Signed-off-by: Qu Wenruo
---
image/main.c | 52 ++
This patchset can be fetched from github:
https://github.com/adam900710/btrfs-progs/tree/image_recover
The base commit is as usual, the latest stable tag, v4.19.
Test case misc/021 will fail if using latest upstream kernel.
This is due to the enhanced code in kernel to check block group <->
chu
On Mon, Nov 26, 2018 at 01:58:42PM +0100, Christoph Hellwig wrote:
> > + phys_addr_t vec_end_addr = page_to_phys(bv->bv_page) +
> > + bv->bv_offset + bv->bv_len;
>
> The name is a little confusing, as the real end addr would be -1. Maybe
> throw the -1 in here, and adj
On 2018/11/26 下午11:01, Andrea Gelmini wrote:
> Hi everybody,
>and thanks a lot for your work.
>
>I'm using BTRFS over LVM over cryptsetup, over Samsung SSD 860 EVO (latest
> git of btrfs-progs).
>Usually I run kernel in development, because I know BTRFS is young and
> there are sti
On Mon, Nov 26, 2018 at 10:17:16AM +0800, Ming Lei wrote:
> This patch pulls the trigger for multi-page bvecs.
Reviewed-by: Omar Sandoval
> Signed-off-by: Ming Lei
> ---
> block/bio.c | 22 +++---
> fs/iomap.c | 4 ++--
> fs/xfs/xfs_aops.c | 4 ++--
> inclu
On Mon, Nov 26, 2018 at 10:17:02AM +0800, Ming Lei wrote:
> From: Christoph Hellwig
>
> bio_readpage_error currently uses bi_vcnt to decide if it is worth
> retrying an I/O. But the vector count is mostly an implementation
> artifact - it really should figure out if there is more than a
> single
On Mon, Nov 26, 2018 at 10:17:17AM +0800, Ming Lei wrote:
> Now multi-page bvec can cover CONFIG_THP_SWAP, so we don't need to
> increase BIO_MAX_PAGES for it.
>
> CONFIG_THP_SWAP needs to split one THP into normal pages and adds
> them all to one bio. With multipage-bvec, it just takes one bvec t
On Mon, Nov 26, 2018 at 10:17:18AM +0800, Ming Lei wrote:
> Now multi-page bvec is supported, some helpers may return page by
> page, meantime some may return segment by segment, this patch
> documents the usage.
Reviewed-by: Omar Sandoval
> Signed-off-by: Ming Lei
> ---
> Documentation/block/
On Mon, Nov 26, 2018 at 10:17:15AM +0800, Ming Lei wrote:
> This patch introduces one extra iterator variable to
> bio_for_each_segment_all(),
> then we can allow bio_for_each_segment_all() to iterate over multi-page bvec.
>
> Given it is just one mechannical & simple change on all
> bio_for_eac
On Mon, Nov 26, 2018 at 10:17:14AM +0800, Ming Lei wrote:
> bch_bio_alloc_pages() is always called on one new bio, so it is safe
> to access the bvec table directly. Given it is the only kind of this
> case, open code the bvec table access since bio_for_each_segment_all()
> will be changed to suppo
On Mon, Nov 26, 2018 at 10:17:13AM +0800, Ming Lei wrote:
> iov_iter is implemented on bvec itererator helpers, so it is safe to pass
> multi-page bvec to it, and this way is much more efficient than passing one
> page in each bvec.
>
> Reviewed-by: Christoph Hellwig
Reviewed-by: Omar Sandoval
On Mon, Nov 26, 2018 at 10:17:09AM +0800, Ming Lei wrote:
> First it is more efficient to use bio_for_each_bvec() in both
> blk_bio_segment_split() and __blk_recalc_rq_segments() to compute how
> many multi-page bvecs there are in the bio.
>
> Secondly once bio_for_each_bvec() is used, the bvec ma
On Mon, Nov 26, 2018 at 10:17:01AM +0800, Ming Lei wrote:
> From: Christoph Hellwig
>
> The btrfs write path passes a bio_offset argument through some deep
> callchains including async offloading. In the end this is easily
> calculatable using page_offset plus the bvec offset for the first
> pag
On Mon, Nov 26, 2018 at 10:17:06AM +0800, Ming Lei wrote:
> We will support multi-page bvec soon, and have to deal with
> single-page vs multi-page bvec. This patch follows Christoph's
> suggestion to rename all the following helpers:
>
> for_each_bvec
> bvec_iter_bvec
> bvec_ite
On Mon, Nov 26, 2018 at 10:17:05AM +0800, Ming Lei wrote:
> Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes
> bio_rewind_iter(), then no one uses bvec_iter_rewind() any more,
> so remove it.
Reviewed-by: Omar Sandoval
> Signed-off-by: Ming Lei
> ---
> include/linux/bvec.h | 24
On Mon, Nov 26, 2018 at 10:17:04AM +0800, Ming Lei wrote:
> It is wrong to use bio->bi_vcnt to figure out how many segments
> there are in the bio even though CLONED flag isn't set on this bio,
> because this bio may be splitted or advanced.
>
> So always use bio_segments() in blk_recount_segments
On Mon, Nov 26, 2018 at 10:17:03AM +0800, Ming Lei wrote:
> From: Christoph Hellwig
>
> The cluster flag implements some very old SCSI behavior. As far as I
> can tell the original intent was to enable or disable any kind of
> segment merging. But the actually visible effect to the LLDD is that
On Mon, Nov 26, 2018 at 6:17 PM David Sterba wrote:
>
> On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrote:
> > From: Filipe Manana
> >
> > When a transaction commit starts, it attempts to pause scrub and it blocks
> > until the scrub is paused. So while the transaction is blocke
From: Filipe Manana
When a transaction commit starts, it attempts to pause scrub and it blocks
until the scrub is paused. So while the transaction is blocked waiting for
scrub to pause, we can not do memory allocation with GFP_KERNEL from scrub,
otherwise we risk getting into a deadlock with recl
Hello,
My name is ms. Reem Al-Hashimi. The UAE minister of state for international
cooparation. I got your contact from an email database from your country. I
have a financial transaction i would like to discuss with you. Please reply to
reem2...@daum.net, for more details if you are interested
I started having a host freeze randomly when running a 4.18 kernel. The
host was stable when running 4.17.12.
At first, it appeared that it was only IO that was frozen since I could
run common commands that were likely cached in RAM and that did not
touch storage. Anything that did touch stora
On Fri, Nov 23, 2018 at 09:42:43AM +0100, Johannes Thumshirn wrote:
> err holds the return value of either btrfs_del_root_ref() or
> btrfs_del_inode_ref() but it hasn't been checked since it's
> introduction with commit fe66a05a0679 (Btrfs: improve error handling
> for btrfs_insert_dir_item callers
On Fri, Nov 23, 2018 at 06:25:40PM +, fdman...@kernel.org wrote:
> From: Filipe Manana
>
> When a transaction commit starts, it attempts to pause scrub and it blocks
> until the scrub is paused. So while the transaction is blocked waiting for
> scrub to pause, we can not do memory allocation
On Mon, Nov 26, 2018 at 05:18:12PM +0100, David Sterba wrote:
> On Tue, Nov 13, 2018 at 11:46:55PM -0800, Omar Sandoval wrote:
> > From: Omar Sandoval
> >
> > Hi,
> >
> > This series contains my backlog of libbtrfsutil changes which I've been
> > collecting over the past few weeks.
> >
> > Patc
On Fri, Nov 16, 2018 at 11:09 AM wrote:
>
> From: Filipe Manana
>
> Commit d7396f07358a ("Btrfs: optimize key searches in btrfs_search_slot"),
> dated from August 2013, introduced an optimization to search for keys in a
> node/leaf to both btrfs_search_slot() and btrfs_search_old_slot(). For the
On Tue, Nov 13, 2018 at 11:46:55PM -0800, Omar Sandoval wrote:
> From: Omar Sandoval
>
> Hi,
>
> This series contains my backlog of libbtrfsutil changes which I've been
> collecting over the past few weeks.
>
> Patches 1-4 are fixes. Patches 5-6 add functionality to the unit tests
> which is ne
On Thu, Nov 22, 2018 at 10:17:49AM +0200, Nikolay Borisov wrote:
> For data inodes this hook does nothing but to return -EAGAIN which is
> used to signal to the endio routines that this bio belongs to a data
> inode. If this is the case the actual retrying is handled by
> bio_readpage_error. Altern
Hi everybody,
and thanks a lot for your work.
I'm using BTRFS over LVM over cryptsetup, over Samsung SSD 860 EVO (latest
git of btrfs-progs).
Usually I run kernel in development, because I know BTRFS is young and there
are still lots of bugs and corner case to fix.
Anyway, I just wa
On 21.11.18 г. 21:09 ч., Josef Bacik wrote:
> Delayed iputs means we can have final iputs of deleted inodes in the
> queue, which could potentially generate a lot of pinned space that could
> be free'd. So before we decide to commit the transaction for ENOPSC
> reasons, run the delayed iputs so
On Thu, Nov 22, 2018 at 10:17:33AM +0200, Nikolay Borisov wrote:
> The first step fo the rebalance process, ensuring there is 1mb free on
> each device. This number seems rather small. And in fact when talking
> to the original authors their opinions were:
>
> "man that's a little bonkers"
> "i do
Looks good,
Reviewed-by: Christoph Hellwig
Looks fine:
Reviewed-by: Christoph Hellwig
Nitpick below:
> +{
> + unsigned len = bv->bv_len;
> + unsigned total_len = 0;
> + unsigned new_nsegs = 0, seg_size = 0;
> +
> + /*
> + * Multipage bvec may be too big to hold in one segment,
> + * so the current bvec has to b
> + phys_addr_t vec_end_addr = page_to_phys(bv->bv_page) +
> + bv->bv_offset + bv->bv_len;
The name is a little confusing, as the real end addr would be -1. Maybe
throw the -1 in here, and adjust for it in the contigous check below?
On Mon, Nov 26, 2018 at 10:17:18AM +0800, Ming Lei wrote:
> Now multi-page bvec is supported, some helpers may return page by
> page, meantime some may return segment by segment, this patch
> documents the usage.
>
> Signed-off-by: Ming Lei
Looks fine,
Reviewed-by: Christoph Hellwig
On Mon, Nov 26, 2018 at 10:17:08AM +0800, Ming Lei wrote:
> bio_for_each_bvec() is used for iterating over multi-page bvec for bio
> split & merge code.
>
> rq_for_each_bvec() can be used for drivers which may handle the
> multi-page bvec directly, so far loop is one perfect use case.
>
> Reviewe
Looks fine,
Reviewed-by: Christoph Hellwig
On Mon, Nov 26, 2018 at 10:17:06AM +0800, Ming Lei wrote:
> We will support multi-page bvec soon, and have to deal with
> single-page vs multi-page bvec. This patch follows Christoph's
> suggestion to rename all the following helpers:
>
> for_each_bvec
> bvec_iter_bvec
> bvec_ite
On Mon, Nov 26, 2018 at 10:17:05AM +0800, Ming Lei wrote:
> Commit 7759eb23fd980 ("block: remove bio_rewind_iter()") removes
> bio_rewind_iter(), then no one uses bvec_iter_rewind() any more,
> so remove it.
>
> Signed-off-by: Ming Lei
Looks good,
Reviewed-by: Christoph Hellwig
On 26.11.18 г. 14:41 ч., Nikolay Borisov wrote:
>
>
> On 21.11.18 г. 21:03 ч., Josef Bacik wrote:
>> For FLUSH_LIMIT flushers we really can only allocate chunks and flush
>> delayed inode items, everything else is problematic. I added a bunch of
>> new states and it lead to weirdness in the F
On 21.11.18 г. 21:03 ч., Josef Bacik wrote:
> For FLUSH_LIMIT flushers we really can only allocate chunks and flush
> delayed inode items, everything else is problematic. I added a bunch of
> new states and it lead to weirdness in the FLUSH_LIMIT case because I
> forgot about how it worked. So
On 21.11.18 г. 21:03 ч., Josef Bacik wrote:
> With the introduction of the per-inode block_rsv it became possible to
> have really really large reservation requests made because of data
> fragmentation. Since the ticket stuff assumed that we'd always have
> relatively small reservation requests
On 21.11.18 г. 21:03 ч., Josef Bacik wrote:
> With my change to no longer take into account the global reserve for
> metadata allocation chunks we have this side-effect for mixed block
> group fs'es where we are no longer allocating enough chunks for the
> data/metadata requirements. To deal wi
On 21.11.18 г. 21:03 ч., Josef Bacik wrote:
> We've done this forever because of the voodoo around knowing how much
> space we have. However we have better ways of doing this now, and on
> normal file systems we'll easily have a global reserve of 512MiB, and
> since metadata chunks are usually
On 21.11.18 г. 21:03 ч., Josef Bacik wrote:
> may_commit_transaction will skip committing the transaction if we don't
> have enough pinned space or if we're trying to find space for a SYSTEM
> chunk. However if we have pending free block groups in this transaction
> we still want to commit as w
Am 26.11.18 um 09:13 schrieb Qu Wenruo:
The corruption itself looks like some disk error, not some btrfs error
like transid error.
You're right! SMART has an increased value for one harddisk on
reallocated sector count. Sorry, I missed to check this first...
I'll try to salvage my data...
T
On 26.11.18 г. 11:07 ч., Anand Jain wrote:
> Circular locking dependency check reports warning[1], that's because
> the btrfs_scrub_dev() calls the stack #0 below with, the
> fs_info::scrub_lock held. The test case leading to this warning..
>
> mkfs.btrfs -fq /dev/sdb && mount /dev/sdb /btrfs
On 26.11.18 г. 11:07 ч., Anand Jain wrote:
> The fs_info::device_list_mutex and fs_info::scrub_lock creates a
> nested locks in btrfs_scrub_dev(). During the lock acquire the
> hierarchy is fs_info::device_list_mutex and then fs_info::scrub_lock,
> so following the same reverse order during unlo
On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> We have a bunch of magic to make sure we're throttling delayed refs when
> truncating a file. Now that we have a delayed refs rsv and a mechanism
> for refilling that reserve simply use that instead of all of this magic.
>
> Signed-off-by: Josef Ba
On 21.11.18 г. 20:59 ч., Josef Bacik wrote:
> From: Josef Bacik
>
> Traditionally we've had voodoo in btrfs to account for the space that
> delayed refs may take up by having a global_block_rsv. This works most
> of the time, except when it doesn't. We've had issues reported and seen
> in pr
Circular locking dependency check reports warning[1], that's because
the btrfs_scrub_dev() calls the stack #0 below with, the
fs_info::scrub_lock held. The test case leading to this warning..
mkfs.btrfs -fq /dev/sdb && mount /dev/sdb /btrfs
btrfs scrub start -B /btrfs
In fact we have fs_info:
The fs_info::device_list_mutex and fs_info::scrub_lock creates a
nested locks in btrfs_scrub_dev(). During the lock acquire the
hierarchy is fs_info::device_list_mutex and then fs_info::scrub_lock,
so following the same reverse order during unlock, that is
fs_info::scrub_lock and then fs_info::devi
Anand Jain (2):
btrfs: scrub: maintain the unlock order in scrub thread
btrfs: scrub: fix circular locking dependency warning
fs/btrfs/scrub.c | 19 ++-
1 file changed, 10 insertions(+), 9 deletions(-)
--
1.8.3.1
On Mon, Nov 26, 2018 at 3:20 AM Ming Lei wrote:
>
> We will support multi-page bvec soon, and have to deal with
> single-page vs multi-page bvec. This patch follows Christoph's
> suggestion to rename all the following helpers:
>
> for_each_bvec
> bvec_iter_bvec
> bvec_iter_
On 2018/11/26 下午3:19, Alexander Fieroch wrote:
> Hi,
>
> My data partition with btrfs RAID 0 (/dev/sdc0 and /dev/sdd0) shows
> errors in syslog:
>
> BTRFS error (device sdc): cleaner transaction attach returned -30
> BTRFS info (device sdc): disk space caching is enabled
> BTRFS info (device sd
80 matches
Mail list logo
