Btrfs: cut down on loops through the allocator - a5e681d9bd641c4f0677e87d3a0c92a8f4f16293

2019-03-07 Thread Alex Lyakas
Hi Josef, This commit added the following code in find_free_extent: ret = do_chunk_alloc(trans, root, flags, CHUNK_ALLOC_FORCE); + + /* +* If we can't allocate a new chunk we've alrea

Re: [GIT PULL] Btrfs fixes for 4.15-rc2

2019-02-18 Thread Alex Lyakas
Hi David, > Btrfs: incremental send, fix wrong unlink path after renaming file > (2017-11-28 17:15:30 +0100) > > > David Sterba (2): > btrfs: add missing device::flush_bio puts Is there a reason that this one should not be t

Re: [PATCH 0/7] eb reference count cleanups

2019-02-06 Thread Alex Lyakas
Hi Nikolay, In my kernel (4.14.x) the flag is called EXTENT_BUFFER_DUMMY, and indeed I see that there is an extra dec-ref for them in free_extent_buffer(). Thanks for clearing that up, Alex. On Wed, Feb 6, 2019 at 4:36 PM Nikolay Borisov wrote: > > > > On 6.02.19 г. 16:26 ч.,

Re: [PATCH 0/7] eb reference count cleanups

2019-02-06 Thread Alex Lyakas
Hi Nikolay, David, Isn't patch 5 (btrfs: Remove extra reference count bumps in btrfs_compare_trees) fixing a memory leak, and hence should be tagged as "stable"? I am specifically interested in 4.14.x. The comment says "remove redundant calls to extent_buffer_get since they don't really add any v

Re: [PATCH 2/2] Btrfs: fix unprotected deletion from pending_chunks list

2019-01-22 Thread Alex Lyakas
t_empty(&fs_info->pinned_chunks)) { struct extent_map *em; em = list_first_entry(&fs_info->pinned_chunks, struct extent_map, list); list_del_init(&em->list); free_extent_map(em); } Can you please comment on that? Thanks, Alex. O

Re: [PATCH 2/2] Btrfs: fix unprotected deletion from pending_chunks list

2019-01-21 Thread Alex Lyakas
Hi Filipe, On Tue, Dec 2, 2014 at 8:08 PM Filipe Manana wrote: > > On block group remove if the corresponding extent map was on the > transaction->pending_chunks list, we were deleting the extent map > from that list, through remove_extent_mapping(), without any > synchronization with chunk alloc

Re: [PATCH RFC] btrfs: introduce a separate mutex for caching_block_groups list

2017-05-13 Thread Alex Lyakas
Hi Liu, On Wed, Mar 22, 2017 at 1:40 AM, Liu Bo wrote: > On Sun, Mar 19, 2017 at 07:18:59PM +0200, Alex Lyakas wrote: >> We have a commit_root_sem, which is a read-write semaphore that protects the >> commit roots. >> But it is also used to protect the list of caching blo

Re: include linux kernel headers for btrfs filesystem

2017-03-20 Thread Alex Lyakas
Ilan, On Mon, Mar 20, 2017 at 10:33 AM, Ilan Schwarts wrote: > I need to cast struct inode to struct btrfs_inode. > in order to do it, i looked at implementation of btrfs_getattr. > > the code is simple: > struct btrfs_inode *btrfsInode; > btrfsInode = BTRFS_I(inode); > > in order to compile i mu

[PATCH RFC] btrfs: introduce a separate mutex for caching_block_groups list

2017-03-19 Thread Alex Lyakas
d on one of the latest kernels, because here btrfs is part of the larger system, and upgrading the kernel is a significant effort. Hence marking the patch as RFC. Hopefully, this patch still has some value to the community. Signed-off-by: Alex Lyakas diff --git a/fs/btrfs/ctree.h b/fs/btrfs/ctree.

Re: [PATCH] Btrfs: deal with unexpected return value in flush_space

2016-10-01 Thread Alex Lyakas
David, Holger, Thank you for picking up that old patch of mine. Alex. On Fri, Jul 29, 2016 at 8:53 PM, Liu Bo wrote: > On Fri, Jul 29, 2016 at 07:01:50PM +0200, David Sterba wrote: >> On Thu, Jul 28, 2016 at 11:49:14AM -0700, Liu Bo wrote: >> > > For reviewers - this came up before here: >> >

[RCF - PATCH] btrfs: do not ignore errors from primary superblock

2016-05-17 Thread Alex Lyakas
RFC: This patch not for merging, but only for review and discussion. When mounting, we consider only the primary superblock on each device. But when writing the superblocks, we might silently ignore errors from the primary superblock, if we succeeded to write secondary superblocks. In such case,

Re: [PATCH v4 6/9] Btrfs: implement the free space B-tree

2016-04-22 Thread Alex Lyakas
Hi Omar, Chris, I have reviewed the free-space-tree code. It is a very nice feature. However, I have a basic understanding question. Let's say we are running a delayed ref which inserts a new EXTENT_ITEM into the extent tree, e.g., we are in alloc_reserved_file_extent. At this point we call remo

Re: [PATCH v8 00/27][For 4.7] Btrfs: Add inband (write time) de-duplication framework

2016-04-03 Thread Alex Lyakas
Hello Qu, Wang, On Wed, Mar 30, 2016 at 2:34 AM, Qu Wenruo wrote: > > > Alex Lyakas wrote on 2016/03/29 19:22 +0200: >> >> Greetings Qu Wenruo, >> >> I have reviewed the dedup patchset found in the github account you >> mentioned. I have several questions

Re: [PATCH v8 00/27][For 4.7] Btrfs: Add inband (write time) de-duplication framework

2016-03-30 Thread Alex Lyakas
Thanks for your comments, Qu. Alex. On Wed, Mar 30, 2016 at 2:34 AM, Qu Wenruo wrote: > > > Alex Lyakas wrote on 2016/03/29 19:22 +0200: >> >> Greetings Qu Wenruo, >> >> I have reviewed the dedup patchset found in the github account you >> mentioned. I h

Re: [PATCH v8 11/27] btrfs: dedupe: Introduce interfaces to resume and cleanup dedupe info

2016-03-29 Thread Alex Lyakas
Hi Qu, Wang, On Tue, Mar 22, 2016 at 3:35 AM, Qu Wenruo wrote: > Since we will introduce a new on-disk based dedupe method, introduce new > interfaces to resume previous dedupe setup. > > And since we introduce a new tree for status, also add disable handler > for it. > > Signed-off-by: Wang Xiao

Re: [PATCH v8 00/27][For 4.7] Btrfs: Add inband (write time) de-duplication framework

2016-03-29 Thread Alex Lyakas
Greetings Qu Wenruo, I have reviewed the dedup patchset found in the github account you mentioned. I have several questions. Please note that by all means I am not criticizing your design or code. I just want to make sure that my understanding of the code is proper. 1) You mentioned in several em

Re: [PATCH 2/2] btrfs: do not write corrupted metadata blocks to disk

2016-03-13 Thread Alex Lyakas
Nicholas, On Sat, Mar 12, 2016 at 12:19 AM, Nicholas D Steeves wrote: > On 10 March 2016 at 06:10, Alex Lyakas wrote: >> csum_dirty_buffer was issuing a warning in case the extent buffer >> did not look alright, but was still returning success. >> Let's return error

Re: [RFC - PATCH] btrfs: do not write corrupted metadata blocks to disk

2016-03-10 Thread Alex Lyakas
/file/d/0B9rmyUifdvMLbHBuSWU5dlVKNWc. Due to this I did not include the chunk tree UUID check. Hoping very much that fs UUID should always be valid for all tree blocks)) Thanks, Alex. On Mon, Feb 22, 2016 at 12:28 PM, Filipe Manana wrote: > On Mon, Feb 22, 2016 at 9:46 AM, Alex Lyakas wr

[PATCH 1/2] btrfs: csum_tree_block: return proper errno value

2016-03-10 Thread Alex Lyakas
Signed-off-by: Alex Lyakas --- fs/btrfs/disk-io.c | 13 + 1 file changed, 5 insertions(+), 8 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 4545e2e..4420ab2 100644 --- a/fs/btrfs/disk-io.c +++ b/fs/btrfs/disk-io.c @@ -296,52 +296,52 @@ static int

[PATCH 2/2] btrfs: do not write corrupted metadata blocks to disk

2016-03-10 Thread Alex Lyakas
e_bio will, but it is better than to have a silent metadata corruption on disk. Signed-off-by: Alex Lyakas --- fs/btrfs/disk-io.c | 15 +-- 1 file changed, 13 insertions(+), 2 deletions(-) diff --git a/fs/btrfs/disk-io.c b/fs/btrfs/disk-io.c index 4420ab2..cf85714 100644 --- a/fs/

Re: [RFC - PATCH] btrfs: do not write corrupted metadata blocks to disk

2016-02-22 Thread Alex Lyakas
Thank you, Filipe, for your review. On Mon, Feb 22, 2016 at 3:05 AM, Filipe Manana wrote: > On Sun, Feb 21, 2016 at 3:36 PM, Alex Lyakas wrote: >> csum_dirty_buffer was issuing a warning in case the extent buffer >> did not look alright, but was still returning success. >>

[RFC - PATCH] btrfs: do not write corrupted metadata blocks to disk

2016-02-21 Thread Alex Lyakas
t the extent buffer doesn't look alright. The caller up the chain may BUG_ON on this, for example flush_epd_write_bio will, but it is better than to have a silent metadata corruption on disk. Note: this patch has been properly tested on 3.18 kernel only. Signed-off-by: Alex Lyakas --- fs/

Re: [PATCH] Btrfs: fix quick exhaustion of the system array in the superblock

2015-12-13 Thread Alex Lyakas
wrote: > On Sun, Dec 13, 2015 at 10:29 AM, Alex Lyakas wrote: >> Hi Filipe Manana, >> >> Can't the call to btrfs_create_pending_block_groups() cause a >> deadlock, like in >> http://www.spinics.net/lists/linux-btrfs/msg48744.html? Because this >> call up

Re: [PATCH 1/2 v3] Btrfs: fix regression when running delayed references

2015-12-13 Thread Alex Lyakas
Hi Filipe, Thank you for the explanation. On Sun, Dec 13, 2015 at 5:43 PM, Filipe Manana wrote: > On Sun, Dec 13, 2015 at 10:51 AM, Alex Lyakas wrote: >> Hi Filipe Manana, >> >> My understanding of selecting delayed refs to run or merging them is >> far from compl

Re: [PATCH 1/2 v3] Btrfs: fix regression when running delayed references

2015-12-13 Thread Alex Lyakas
Hi Filipe Manana, My understanding of selecting delayed refs to run or merging them is far from complete. Can you please explain what will happen in the following scenario: 1) Ref1 is created, as you explain 2) Somebody calls __btrfs_run_delayed_refs() and runs Ref1, and we end up with an EXTENT_

Re: [PATCH] Btrfs: fix quick exhaustion of the system array in the superblock

2015-12-13 Thread Alex Lyakas
Hi Filipe Manana, Can't the call to btrfs_create_pending_block_groups() cause a deadlock, like in http://www.spinics.net/lists/linux-btrfs/msg48744.html? Because this call updates the device tree, and we may be calling do_chunk_alloc() from find_free_extent() when holding a lock on the device tree

Re: [PATCH V2] Btrfs: find_free_extent: Do not erroneously skip LOOP_CACHING_WAIT state

2015-12-13 Thread Alex Lyakas
[Resending in plain text, apologies.] Hi Chandan, Josef, Chris, I am not sure I understand the fix to the problem. It may happen that when updating the device tree, we need to allocate a new chunk via do_chunk_alloc (while we are holding the device tree root node locked). This is a legitimate th

Re: [RFC PATCH] btrfs: flush_space: treat return value of do_chunk_alloc properly

2015-12-06 Thread Alex Lyakas
So it will return -ENOSPC. Signed-off-by: Alex Lyakas Reviewed-by: Josef Bacik diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c index 4b89680..1ba3f0d 100644 --- a/fs/btrfs/extent-tree.c +++ b/fs/btrfs/extent-tree.c @@ -4727,7 +4727,7 @@ static int flush_space(struct btrfs_

Re: [RFC PATCH] btrfs: flush_space: treat return value of do_chunk_alloc properly

2015-12-06 Thread Alex Lyakas
Thu, Dec 3, 2015 at 8:14 PM, Liu Bo wrote: > On Thu, Dec 03, 2015 at 06:51:03PM +0200, Alex Lyakas wrote: >> do_chunk_alloc returns 1 when it succeeds to allocate a new chunk. >> But flush_space will not convert this to 0, and will also return 1. >> As a result, reserve_metadata_by

[RFC PATCH] btrfs: flush_space: treat return value of do_chunk_alloc properly

2015-12-03 Thread Alex Lyakas
do_chunk_alloc returns 1 when it succeeds to allocate a new chunk. But flush_space will not convert this to 0, and will also return 1. As a result, reserve_metadata_bytes will think that flush_space failed, and may potentially return this value "1" to the caller (depends how reserve_metadata_bytes

Re: [PATCH] btrfs: clear bio reference after submit_one_bio()

2015-11-07 Thread Alex Lyakas
and comment on patches. Alex. On Thu, Nov 5, 2015 at 3:08 PM, Holger Hoffstätte wrote: > On 10/11/15 20:09, Alex Lyakas wrote: >> Hi Naota, >> >> What happens if btrfs_bio_alloc() in submit_extent_page fails? Then we >> return -ENOMEM to the caller, but we do not set *bio_re

Re: [PATCH] Btrfs: throttle delayed refs better

2015-10-14 Thread Alex Lyakas
Hi Josef, Looking at the latest Linus tree, I still see: if (actual_count > 0) { u64 runtime = ktime_to_ns(ktime_sub(ktime_get(), start)); ... avg = fs_info->avg_delayed_ref_runtime * 3 + runtime; avg = div64_u64(avg, 4); So we need to divide "runtime" by "actual_count" before acco

Re: [PATCH] btrfs: clear bio reference after submit_one_bio()

2015-10-11 Thread Alex Lyakas
Hi Naota, What happens if btrfs_bio_alloc() in submit_extent_page fails? Then we return -ENOMEM to the caller, but we do not set *bio_ret to NULL. And if *bio_ret was non-NULL upon entry into submit_extent_page, then we had submitted this bio before getting to btrfs_bio_alloc(). So should btrfs_bi

Re: [PATCH] Btrfs: check pending chunks when shrinking fs to avoid corruption

2015-09-30 Thread Alex Lyakas
Hi Filipe, Looking the code of this patch, I see that if we discover a pending chunk, we unlock the chunk mutex, commit the transaction (which completes the allocation of all pending chunks and inserts relevant items into the device tree and chunk tree), and retry the search. However, after we un

Re: [PATCH v5 04/18] btrfs: Add threshold workqueue based on kernel workqueue

2015-08-19 Thread Alex Lyakas
Hi Qu, On Fri, Feb 28, 2014 at 4:46 AM, Qu Wenruo wrote: > The original btrfs_workers has thresholding functions to dynamically > create or destroy kthreads. > > Though there is no such function in kernel workqueue because the worker > is not created manually, we can still use the workqueue_set_

Re: question about should_cow_block() and BTRFS_HEADER_FLAG_WRITTEN

2015-07-21 Thread Alex Lyakas
hey are not in the page cache. Also need to keep in mind that every COW of eb requires a new slot in the page cache, because we index by "bytenr" that we receive from the free-space cache, which is a "logical" coordinate by which EXTENT_ITEMs are sorted in the extent tree.

Re: question about should_cow_block() and BTRFS_HEADER_FLAG_WRITTEN

2015-07-13 Thread Alex Lyakas
rote: > On Sun, Jul 12, 2015 at 6:15 PM, Alex Lyakas wrote: >> Greetings, >> Looking at the code of should_cow_block(), I see: >> >> if (btrfs_header_generation(buf) == trans->transid && >>!btrfs_header_flag(buf, BTRFS_HEADER_FLAG_WRITTEN) &&

question about should_cow_block() and BTRFS_HEADER_FLAG_WRITTEN

2015-07-12 Thread Alex Lyakas
Greetings, Looking at the code of should_cow_block(), I see: if (btrfs_header_generation(buf) == trans->transid && !btrfs_header_flag(buf, BTRFS_HEADER_FLAG_WRITTEN) && ... So if the extent buffer has been written to disk, and now is changed again in the same transaction, we insist on COW'ing

Re: [PATCH] btrfs-progs: rebuild missing block group during chunk recovery if possible

2014-12-24 Thread Alex Lyakas
Hi Qu, On Wed, Dec 24, 2014 at 3:09 AM, Qu Wenruo wrote: > > Original Message > Subject: Re: [PATCH] btrfs-progs: rebuild missing block group during chunk > recovery if possible > From: Alex Lyakas > To: Qu Wenruo > Date: 2014年12月24日 00:49 >> >

Re: [PATCH] btrfs-progs: rebuild missing block group during chunk recovery if possible

2014-12-23 Thread Alex Lyakas
Hi Qu, On Thu, Oct 30, 2014 at 4:54 AM, Qu Wenruo wrote: > Before the patch, chunk will be considered bad if the corresponding > block group is missing, even the only uncertain data is the 'used' > member of the block group. > > This patch will try to recalculate the 'used' value of the block gro

Re: How btrfs-find-root knows that the block is actually a root?

2014-12-23 Thread Alex Lyakas
Hi Qu, On Tue, Dec 23, 2014 at 7:27 AM, Qu Wenruo wrote: > > Original Message > Subject: How btrfs-find-root knows that the block is actually a root? > From: Alex Lyakas > To: linux-btrfs > Date: 2014年12月22日 22:57 >> >> Greetings, >

How btrfs-find-root knows that the block is actually a root?

2014-12-22 Thread Alex Lyakas
Greetings, I am looking at the code of search_iobuf() in btrfs-find-root.c.(3.17.3) I see that we probe nodesize blocks one by one, and for each block we check: - its owner is what we are looking for - its header->bytenr is what we are looking at currently - its level is not too small - it has va

Re: [PATCH] Btrfs: update commit root on snapshot creation after orphan cleanup

2014-08-02 Thread Alex Lyakas
. Unless we crash in the middle, like you pointed. I will also look at the new patch. Thanks! Alex. On Thu, Jul 31, 2014 at 3:41 PM, Filipe David Manana wrote: > On Mon, Jul 28, 2014 at 6:31 PM, Filipe David Manana > wrote: >> On Sat, Jul 19, 2014 at 8:11 PM, Alex Lyakas >&

Re: [PATCH] Btrfs: update commit root on snapshot creation after orphan cleanup

2014-07-19 Thread Alex Lyakas
Hi Filipe, It's quite possible I don't fully understand the issue. It seems that we are creating a read-only snapshot, commit a transaction, and then go and modify the snapshot once again, by deleting all the ORPHAN_ITEMs we have in its file tree (btrfs_orphan_cleanup). Shouldn't all this be part o

Re: safe/necessary to balance system chunks?

2014-06-19 Thread Alex Lyakas
On Fri, Apr 25, 2014 at 10:14 PM, Hugo Mills wrote: > On Fri, Apr 25, 2014 at 02:12:17PM -0400, Austin S Hemmelgarn wrote: >> On 2014-04-25 13:24, Chris Murphy wrote: >> > >> > On Apr 25, 2014, at 8:57 AM, Steve Leung wrote: >> > >> >> >> >> Hi list, >> >> >> >> I've got a 3-device RAID1 btrfs fi

Re: Snapshot aware defrag and qgroups thoughts

2014-06-19 Thread Alex Lyakas
Hi Josef, thanks for the detailed description of how the extent tree works! When I was digging through that in the past, I made some slides to remember all the call chains. Maybe somebody finds that useful to accompany your notes. https://drive.google.com/file/d/0ByBy89zr3kJNNmM5OG5wXzQ3LUE/edit?us

Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv

2014-06-18 Thread Alex Lyakas
013 at 3:19 PM, Filipe David Manana wrote: > On Mon, Nov 4, 2013 at 12:16 PM, Alex Lyakas > wrote: >> Hi Filipe, >> any luck with this patch?:) > > Hey Alex, > > I haven't digged further, but I remember I couldn't reproduce your > issue (with latest btrfs-ne

Re: snapshot send with parent question

2014-05-31 Thread Alex Lyakas
Michael, btrfs-send doesn't really know or care how did you manage to get from a to c. It is able to compare any two RO subvolumes (not necessarily related by snapshot operations), and produce a stream of commands that transfer a-content into c-content. Send assumes that at a receive side, you hav

Re: [PATCH] Btrfs: fix deadlock with nested trans handles

2014-05-03 Thread Alex Lyakas
Hi Josef, this problem could not happen when find_free_extent() was receiving a transaction handle (which was changed in "Btrfs: avoid starting a transaction in the write path"), correct? Because it would have used the passed transaction handle to do the chunk allocation, and thus would not need to

Re: [PATCH] Btrfs: abort the transaction when we don't find our extent ref

2014-05-03 Thread Alex Lyakas
Hi Josef, how about aborting the transaction also in place that you print: "umm, got %d back from search, was looking for %llu" You abort in case ret<0, otherwise the code just proceeds with extent_slot = path->slots[0]; which can't be right in that case. Thanks, Alex. On Mon, Mar 17, 2014 at 3:5

Re: [PATCH] Btrfs: correctly determine if blocks are shared in btrfs_compare_trees

2014-04-05 Thread Alex Lyakas
Hi Filipe, Can you please explain more what is the scenario you are worried about. Let's say we have two FS trees (subvolumes) subv1 and subv2, subv2 being a RO snapshot of subv1. And they have a shared subtree at logical==X. Now we change subv1, so its subtree is COW'ed and some other logical add

Re: [PATCH] Btrfs: attach delayed ref updates to delayed ref heads

2014-03-30 Thread Alex Lyakas
Hi Josef, I have a question about update_existing_head_ref() logic. The question is not specific to the rework that you have done. You have a code like this: if (ref->must_insert_reserved) { /* if the extent was freed and then * reallocated before the delayed ref * en

Re: [PATCH] Btrfs: fix memory leak in btrfs_create_tree()

2014-03-27 Thread Alex Lyakas
Hi Tsutomu Itoh, On Thu, Mar 21, 2013 at 6:32 AM, Tsutomu Itoh wrote: > We should free leaf and root before returning from the error > handling code. > > Signed-off-by: Tsutomu Itoh > --- > fs/btrfs/disk-io.c | 12 +--- > 1 file changed, 9 insertions(+), 3 deletions(-) > > diff --git a/

Re: [RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-03-08 Thread Alex Lyakas
transaction commits and we crash, then yes, the allocation is lost. Alex. On Tue, Mar 4, 2014 at 8:04 AM, Miao Xie wrote: > On sat, 1 Mar 2014 20:05:01 +0200, Alex Lyakas wrote: >> Hi Miao, >> >> On Wed, Jan 15, 2014 at 2:00 PM, Miao Xie wrote: >>> When we mounted

Re: [RFC PATCH 5/5] Btrfs: fix broken free space cache after the system crashed

2014-03-01 Thread Alex Lyakas
Hi Miao, On Wed, Jan 15, 2014 at 2:00 PM, Miao Xie wrote: > When we mounted the filesystem after the crash, we got the following > message: > BTRFS error (device xxx): block group 4315938816 has wrong amount of free > space > BTRFS error (device xxx): failed to load free space cache for bloc

Re: [PATCH] Btrfs: fix a deadlock on chunk mutex

2014-02-18 Thread Alex Lyakas
: SHA1 > > > > On 02/18/2014 10:47 AM, Alex Lyakas wrote: >> Hello Josef, >> >> On Tue, Dec 18, 2012 at 3:52 PM, Josef Bacik >> wrote: >>> On Wed, Dec 12, 2012 at 06:52:37PM -0700, Liu Bo wrote: >>>> An user reported that he has hit a

Re: [PATCH] Btrfs: fix a deadlock on chunk mutex

2014-02-18 Thread Alex Lyakas
Hello Josef, On Tue, Dec 18, 2012 at 3:52 PM, Josef Bacik wrote: > On Wed, Dec 12, 2012 at 06:52:37PM -0700, Liu Bo wrote: >> An user reported that he has hit an annoying deadlock while playing with >> ceph based on btrfs. >> >> Current updating device tree requires space from METADATA chunk, >>

Re: [PATCH] btrfs: Fix 32/64-bit problem with BTRFS_SET_RECEIVED_SUBVOL ioctl

2014-02-15 Thread Alex Lyakas
Hello Hugo, Is this issue specific to the receive ioctl? Because what you are describing applies to any user-kernel ioctl-based interface, when you compile the user-space as 32-bit, which the kernel space has been compiled as 64-bit. For that purpose, I believe, there exists the "compat_ioctl" ca

Re: [PATCH] Btrfs: return ENOSPC when target space is full

2014-01-19 Thread Alex Lyakas
Hi Filipe, I think in the context of do_chunk_alloc(), 0 doesn't mean "success". 0 means "allocation was not attempted". While 1 means "allocation was attempted and succeeded". -ENOSPC means "allocation was attempted but failed". Any other errno deserves transaction abort. Anyways, the callers are

Re: [PATCH v2] Btrfs-progs: avoid using btrfs internal subvolume path to send

2014-01-11 Thread Alex Lyakas
Hi Miguel, On Sat, Nov 30, 2013 at 1:43 PM, Miguel Negrão wrote: > Em 29-11-2013 16:37, Wang Shilong escreveu: >> From: Wang Shilong >> >> Steps to reproduce: >> # mkfs.btrfs -f /dev/sda >> # mount /dev/sda /mnt >> # btrfs subvolume create /mnt/foo >> # umount /mnt >>

Re: [PATCH v7 04/13] Btrfs: disable qgroups accounting when quata_enable is 0

2013-12-03 Thread Alex Lyakas
x5f0 [btrfs] [] btrfs_ioctl_snap_create_transid+0x10f/0x1b0 [btrfs] On Tue, Dec 3, 2013 at 7:13 PM, Alex Lyakas wrote: > Hi Liu, Jan, > > What happens to "struct qgroup_update"s sitting in > trans->qgroup_ref_list in case the transaction aborts? It seems that >

Re: [PATCH v7 04/13] Btrfs: disable qgroups accounting when quata_enable is 0

2013-12-03 Thread Alex Lyakas
Hi Liu, Jan, What happens to "struct qgroup_update"s sitting in trans->qgroup_ref_list in case the transaction aborts? It seems that they are not freed. For example, if we are in btrfs_commit_transaction() and: - call create_pending_snapshots() - call btrfs_run_delayed_items() and this fails So w

Re: [PATCH v2] btrfs-progs: calculate disk space that a subvol could free

2013-11-28 Thread Alex Lyakas
Hello Anand, I have sent a similar comment to your email thread in http://www.spinics.net/lists/linux-btrfs/msg27547.html I believe this approach of calculating freeable space is incorrect. Try this: - create a fresh btrfs - create a regular file - write some data into it in such a way, that it wa

Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv

2013-11-04 Thread Alex Lyakas
Hi Filipe, any luck with this patch?:) Alex. On Wed, Oct 23, 2013 at 5:26 PM, Filipe David Manana wrote: > On Wed, Oct 23, 2013 at 3:14 PM, Alex Lyakas > wrote: >> Hello, >> >> On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana >> wrote: >>> On We

Re: [PATCH] btrfs-progs: calculate disk space that a subvol could free upon delete

2013-10-26 Thread Alex Lyakas
only. This is accomplished by _btrfs_mod_ref() code, called from btrfs_copy_root() during snapshot creation flow. Snapshot deletion code is the one that is smart enough to properly "unshare" shared tree blocks with such "implicit" references. What do you think? Alex. On Sat,

Re: [PATCH] btrfs-progs: calculate disk space that a subvol could free upon delete

2013-10-26 Thread Alex Lyakas
Hi Anand, 1) so let's say I have a subvolume and a snapshot of this subvolume. So in this case, I will see "Sole space = 0" for both of them, correct? Because all extents (except inline ones) are shared. 2) How is this in terms of responsiveness? On a huge subvolume, we need to iterate all the EX

Re: [patch 3/7] btrfs: Add per-super attributes to sysfs

2013-10-26 Thread Alex Lyakas
Hi Jeff, On Tue, Sep 10, 2013 at 7:24 AM, Jeff Mahoney wrote: > This patch adds per-super attributes to sysfs. > > It doesn't publish any attributes yet, but does the proper lifetime > handling as well as the basic infrastructure to add new attributes. > > Signed-off-by: Jeff Mahoney > --- > fs

Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv

2013-10-23 Thread Alex Lyakas
Hello, On Wed, Oct 23, 2013 at 4:35 PM, Filipe David Manana wrote: > On Wed, Oct 23, 2013 at 2:33 PM, Alex Lyakas > wrote: >> Hi Filipe, >> >> >> On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana >> wrote: >>> >>> This issue is

Re: [PATCH v6] Btrfs: fix memory leak of orphan block rsv

2013-10-23 Thread Alex Lyakas
Hi Filipe, On Tue, Aug 20, 2013 at 2:52 AM, Filipe David Borba Manana wrote: > > This issue is simple to reproduce and observe if kmemleak is enabled. > Two simple ways to reproduce it: > > ** 1 > > $ mkfs.btrfs -f /dev/loop0 > $ mount /dev/loop0 /mnt/btrfs > $ btrfs balance start /mnt/btrfs > $

Re: [PATCH] btrfs: commit transaction after deleting a subvolume

2013-10-20 Thread Alex Lyakas
nalty incured by the change, but deleting a > subvolume is not a frequent operation and the tradeoff seems justified > by getting the guarantee stated above. > > CC: Alex Lyakas > CC: Josef Bacik > Signed-off-by: David Sterba > --- > fs/btrfs/ioctl.c |2 +- > 1 files c

Re: [PATCH 2/2] Btrfs: stop caching thread if extent_commit_sem is contended

2013-10-17 Thread Alex Lyakas
Thanks for addressing this issue, Josef! On Thu, Sep 26, 2013 at 4:26 PM, Josef Bacik wrote: > We can starve out the transaction commit with a bunch of caching threads all > running at the same time. This is because we will only drop the > extent_commit_sem if we need_resched(), which isn't like

Re: btrfs:async-thread: atomic_start_pending=1 is set, but it's too late

2013-08-29 Thread Alex Lyakas
Thanks, Chris, Josef, for confirming! On Thu, Aug 29, 2013 at 11:08 PM, Chris Mason wrote: > Quoting Josef Bacik (2013-08-29 16:03:06) >> On Mon, Aug 26, 2013 at 05:16:42PM +0300, Alex Lyakas wrote: >> > Greetings all, >> > I see a following issue with spawning ne

Re: [PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Alex Lyakas
On Thu, Aug 29, 2013 at 10:55 PM, Josef Bacik wrote: > On Thu, Aug 29, 2013 at 10:09:29PM +0300, Alex Lyakas wrote: >> Hi Josef, >> >> On Thu, Aug 29, 2013 at 5:38 PM, Josef Bacik wrote: >> > On Thu, Aug 29, 2013 at 01:31:05PM +0300, Alex Lyakas wrote: >> &g

[PATCH] Notify caching_thread()s to give up on extent_commit_sem when needed.

2013-08-29 Thread Alex Lyakas
this problem happens, and whether the fix goes in the right direction. Signed-off-by: Alex Lyakas --- fs/btrfs/ctree.h |7 +++ fs/btrfs/disk-io.c |1 + fs/btrfs/extent-tree.c |9 + fs/btrfs/transaction.c |2 +- 4 files changed, 14 insertions(+), 5 deletions

Re: [PATCH] Btrfs: handle errors when doing slow caching

2013-08-27 Thread Alex Lyakas
Hi Josef, thanks for addressing this. On Mon, Aug 5, 2013 at 6:19 PM, Josef Bacik wrote: > Alex Lyakas reported a bug where wait_block_group_cache_progress() would wait > forever if a drive failed. This is because we just bail out if there is an > error while trying to cache a block

btrfs:async-thread: atomic_start_pending=1 is set, but it's too late

2013-08-26 Thread Alex Lyakas
Greetings all, I see a following issue with spawning new threads for btrfs_workers that have atomic_worker_start set: # I have BTRFS that has 24Gb of total metadata, out of which extent tree takes 11Gb; space_cache option is not used. # After mouting, cache_block_group() triggers ~250 work items t

Re: [PATCH] Btrfs: stop all workers before cleaning up roots

2013-08-01 Thread Alex Lyakas
Hi Josef, On Thu, May 30, 2013 at 11:58 PM, Josef Bacik wrote: > Dave reported a panic because the extent_root->commit_root was NULL in the > caching kthread. That is because we just unset it in free_root_pointers, > which > is not the correct thing to do, we have to either wait for the caching

Re: [PATCH] Btrfs: fix all callers of read_tree_block

2013-07-30 Thread Alex Lyakas
Hi Josef, On Tue, Apr 23, 2013 at 9:20 PM, Josef Bacik wrote: > We kept leaking extent buffers when mounting a broken file system and it turns > out it's because not everybody uses read_tree_block properly. You need to > check > and make sure the extent_buffer is uptodate before you use it. Th

Re: [PATCH] Btrfs: update drop progress before stopping snapshot dropping

2013-07-30 Thread Alex Lyakas
it the snapshot dropping without updating the drop > progress. So the next time we go to resume we will BUG_ON() because we > can't > find the extent we left off at because we never updated it. This patch > fixes > the problem. > > Cc: sta...@vger.kernel.org > Reported-b

Re: [PATCH] Btrfs: fix lock leak when resuming snapshot deletion

2013-07-16 Thread Alex Lyakas
e evicted the extent buffer > from cache. Thanks, > > Reported-by: Alex Lyakas > Signed-off-by: Josef Bacik > --- > fs/btrfs/extent-tree.c |2 ++ > 1 files changed, 2 insertions(+), 0 deletions(-) > > diff --git a/fs/btrfs/extent-tree.c b/fs/btrfs/extent-tree.c > ind

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-07-14 Thread Alex Lyakas
Hi, On Thu, Jul 4, 2013 at 10:52 PM, Alex Lyakas wrote: > Hi David, > > On Thu, Jul 4, 2013 at 8:03 PM, David Sterba wrote: >> On Thu, Jul 04, 2013 at 06:29:23PM +0300, Alex Lyakas wrote: >>> > @@ -7363,6 +7365,12 @@ int btrfs_drop_snapshot(struct btrfs_root *root, &g

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-07-04 Thread Alex Lyakas
Hi David, On Thu, Jul 4, 2013 at 8:03 PM, David Sterba wrote: > On Thu, Jul 04, 2013 at 06:29:23PM +0300, Alex Lyakas wrote: >> > @@ -7363,6 +7365,12 @@ int btrfs_drop_snapshot(struct btrfs_root *root, >> > wc->reada_count = BTRFS_NODEPTRS_PER_BLOCK(root); &g

Re: [PATCH v3] btrfs: clean snapshots one by one

2013-07-04 Thread Alex Lyakas
Hi David, I believe this patch has the following problem: On Tue, Mar 12, 2013 at 5:13 PM, David Sterba wrote: > Each time pick one dead root from the list and let the caller know if > it's needed to continue. This should improve responsiveness during > umount and balance which at some point wait

Re: question about transaction-abort-related commits

2013-07-02 Thread Alex Lyakas
On Sun, Jun 30, 2013 at 2:36 PM, Josef Bacik wrote: > On Sun, Jun 30, 2013 at 11:29:14AM +0300, Alex Lyakas wrote: >> Hi Josef, >> >> On Wed, Jun 26, 2013 at 5:16 PM, Alex Lyakas >> wrote: >> > Hi Josef, >> > Can you please help me with another q

Re: question about transaction-abort-related commits

2013-06-30 Thread Alex Lyakas
Hi Josef, On Wed, Jun 26, 2013 at 5:16 PM, Alex Lyakas wrote: > Hi Josef, > Can you please help me with another question. > > I am looking at your patch: > Btrfs: fix chunk allocation error handling > https://git.kernel.org/cgit/linux/kernel/git/torvalds/li

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-26 Thread Alex Lyakas
Hi Miao, On Mon, Jun 17, 2013 at 4:51 AM, Miao Xie wrote: > On sun, 16 Jun 2013 13:38:42 +0300, Alex Lyakas wrote: >> Hi Miao, >> >> On Thu, Jun 13, 2013 at 6:08 AM, Miao Xie wrote: >>> On wed, 12 Jun 2013 23:11:02 +0300, Alex Lyakas wrote: >>

Re: question about transaction-abort-related commits

2013-06-26 Thread Alex Lyakas
btrfs] [] __btrfs_inc_extent_ref+0x9f/0x240 [btrfs] [] run_clustered_refs+0x971/0xd00 [btrfs] [] btrfs_run_delayed_refs+0xd0/0x330 [btrfs] [] __btrfs_end_transaction+0xf7/0x440 [btrfs] [] btrfs_end_transaction+0x10/0x20 [btrfs] On Mon, Jun 24, 2013 at 9:56 PM, Alex Lyakas wrote: > > Thanks for commenti

Re: question about transaction-abort-related commits

2013-06-24 Thread Alex Lyakas
anks, Alex. On Mon, Jun 24, 2013 at 7:24 PM, Josef Bacik wrote: > On Sun, Jun 23, 2013 at 09:52:14PM +0300, Alex Lyakas wrote: >> Hello Josef, Liu, >> I am reviewing commits in the mainline tree: >> >> e4a2bcaca9643e7430207810653222fc5187f2be Btrfs: if we aren't >

Re: [PATCH] Btrfs: make delayed ref lock logic more readable

2013-06-24 Thread Alex Lyakas
Hi Miao, I believe the name of this patch is misleading. The significant contribution of this patch IMO is the btrfs_release_ref_cluster(), which is being called when btrfs_run_delayed_refs() aborts the transaction. Without this fix, btrfs_destroy_delayed_refs() was crashing when it was doing: list

question about transaction-abort-related commits

2013-06-23 Thread Alex Lyakas
Hello Josef, Liu, I am reviewing commits in the mainline tree: e4a2bcaca9643e7430207810653222fc5187f2be Btrfs: if we aren't committing just end the transaction if we error out (call end_transaction() instead of goto cleanup_transaction) - Josef f094ac32aba3a51c00e970a2ea029339af2ca048 Btrfs: fix

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-16 Thread Alex Lyakas
Hi Miao, On Thu, Jun 13, 2013 at 6:08 AM, Miao Xie wrote: > On wed, 12 Jun 2013 23:11:02 +0300, Alex Lyakas wrote: >> I reviewed the code starting from: >> 69aef69a1bc154 Btrfs: don't wait for all the writers circularly during >> the transaction commit >> until

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-06-12 Thread Alex Lyakas
# Can I apply this fix to 3.8.x kernel (manually, of course)? Or some additional things are needed that are missing in this kernel? Thanks, Alex. > > Thanks > Miao > > On Wed, 10 Apr 2013 21:45:43 +0300, Alex Lyakas wrote: >> Hi Miao, >> I attempted to fix the issue b

wait_block_group_cache_progress() waits forever in case of drive failure

2013-06-04 Thread Alex Lyakas
Greetings all, when testing drive failures, I occasionally hit the following hang: # Block group is being cached-in by caching_thread() # caching_thread() experiences an error, e.g., in btrfs_search_slot, because of drive failure: ret = btrfs_search_slot(NULL, extent_root, &key, path, 0, 0

[no subject]

2013-05-28 Thread Alex Lyakas
Hello all, I have the following unresponsive btrfs: btrfs_end_transaction() is called and is stuck in btrfs_tree_lock(): May 27 16:13:55 vc kernel: [ 7130.421159] kworker/u:85D 0 19859 2 0x May 27 16:13:55 vc kernel: [ 7130.421159] 880095335568 0

Re: [PATCH] Btrfs: clear received_uuid field for new writable snapshots

2013-05-22 Thread Alex Lyakas
Hi Stephan, I fully understand the first part of your fix, and I believe it's quite critical. Indeed, a writable snapshot should have no evidence that it has an ancestor that was once "received". Can you pls let me know that I understand the second part of your fix. In btrfs-progs the following co

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-04-10 Thread Alex Lyakas
its that take like 80 seconds, out of which like 50 seconds are spent in the do-while loop of btrfs_commit_transaction. Thanks, Alex. On Mon, Mar 25, 2013 at 11:11 AM, Alex Lyakas wrote: > Hi Miao, > > On Mon, Mar 25, 2013 at 3:51 AM, Miao Xie wrote: >> On Sun, 24 Mar 2013 13:13

Re: Backup Options

2013-04-09 Thread Alex Lyakas
Hi David, maybe my old patch http://www.spinics.net/lists/linux-btrfs/msg19739.html can help this issue? Thanks, Alex. On Wed, Apr 3, 2013 at 8:23 PM, David Sterba wrote: > On Wed, Apr 03, 2013 at 04:33:22AM +0200, Harald Glatt wrote: >> However what I actually did was: >> # cd /mnt/restore >>

Re: [PATCH v2] Btrfs: fix locking on ROOT_REPLACE operations in tree mod log

2013-04-02 Thread Alex Lyakas
operations). > > This fixes the brand-new version of xfstest 276 as of commit cfe73f71. > > Signed-off-by: Jan Schmidt > --- > Has probably been Reported-by: Alex Lyakas > (unconfirmed). > > Chages for v2: > - use the correct base (current cmason/for-linus) > &g

Re: btrfs "stuck" on

2013-04-02 Thread Alex Lyakas
Hi David, On Fri, Mar 29, 2013 at 8:12 PM, David Sterba wrote: > On Thu, Mar 21, 2013 at 11:56:37AM -0700, Ask Bjørn Hansen wrote: >> A few weeks ago I replaced a ZFS backup system with one backed by >> btrfs. A script loops over a bunch of hosts rsyncing them to each >> their own subvolume. Aft

Re: [PATCH 2/3] Btrfs: fix the deadlock between the transaction start/attach and commit

2013-03-25 Thread Alex Lyakas
Hi Miao, On Mon, Mar 25, 2013 at 3:51 AM, Miao Xie wrote: > On Sun, 24 Mar 2013 13:13:22 +0200, Alex Lyakas wrote: >> Hi Miao, >> I am seeing another issue. Your fix prevents from TRANS_START to get >> in the way of a committing transaction. But it does not prevent from

  1   2   3   >