Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
At 06/03/2016 10:02 PM, Josef Bacik wrote: On 06/01/2016 01:48 AM, Lu Fengqi wrote: Only in the case of different root_id or different object_id, check_shared identified extent as the shared. However, If a extent was referred by different offset of same file, it should also be identified as shared. In addition, check_shared's loop scale is at least n^3, so if a extent has too many references, even causes soft hang up. First, add all delayed_ref to the ref_tree and calculate the unqiue_refs, if the unique_refs is greater than one, return BACKREF_FOUND_SHARED. Then individually add the on-disk reference(inline/keyed) to the ref_tree and calculate the unique_refs of the ref_tree to check if the unique_refs is greater than one.Because once there are two references to return SHARED, so the time complexity is close to the constant. Reported-by: Tsutomu Itoh Signed-off-by: Lu Fengqi This is a lot of work for just wanting to know if something is shared. Instead lets adjust this slightly. Instead of passing down a root_objectid/inum and noticing this and returned shared, add a new way to iterate refs. Currently we go gather all the refs and then do the iterate dance, which is what takes so long. So instead add another helper that calls the provided function every time it has a match, and then we can pass in whatever context we want, and we return when something matches. This way we don't have all this extra accounting, and we're no longer passing root_objectid/inum around and testing for some magic scenario. Thanks, Josef With this patch, we can quickly find extent that has more than one reference(delayed, inline and keyed) and return SHARED immediately. However, for indirect refs, we have to continue to resolve indirect refs to their parent bytenr, and check if this parent bytenr is shared. So, the original function is necessary. The original refs list reduces the efficiency of search, so maybe we can use rb_tree to replace it for optimize the original function in the furture. And, we just want to solve the problem of check_shared now. -- Thanks, Lu -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On 06/01/2016 01:48 AM, Lu Fengqi wrote: Only in the case of different root_id or different object_id, check_shared identified extent as the shared. However, If a extent was referred by different offset of same file, it should also be identified as shared. In addition, check_shared's loop scale is at least n^3, so if a extent has too many references, even causes soft hang up. First, add all delayed_ref to the ref_tree and calculate the unqiue_refs, if the unique_refs is greater than one, return BACKREF_FOUND_SHARED. Then individually add the on-disk reference(inline/keyed) to the ref_tree and calculate the unique_refs of the ref_tree to check if the unique_refs is greater than one.Because once there are two references to return SHARED, so the time complexity is close to the constant. Reported-by: Tsutomu Itoh Signed-off-by: Lu Fengqi This is a lot of work for just wanting to know if something is shared. Instead lets adjust this slightly. Instead of passing down a root_objectid/inum and noticing this and returned shared, add a new way to iterate refs. Currently we go gather all the refs and then do the iterate dance, which is what takes so long. So instead add another helper that calls the provided function every time it has a match, and then we can pass in whatever context we want, and we return when something matches. This way we don't have all this extra accounting, and we're no longer passing root_objectid/inum around and testing for some magic scenario. Thanks, Josef -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On Thu, Jun 02, 2016 at 02:17:40PM -0700, Mark Fasheh wrote: > On Thu, Jun 02, 2016 at 04:56:06PM -0400, Jeff Mahoney wrote: > > On 6/2/16 3:08 PM, Mark Fasheh wrote: > > > On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: > > >> On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > +/* dynamically allocate and initialize a ref_root */ > > +static struct ref_root *ref_root_alloc(void) > > +{ > > + struct ref_root *ref_tree; > > + > > + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); > > >>> > > >>> I'm pretty sure we want GFP_NOFS here. > > >> > > >> Then please explain to me why/where the reasoning below is wrong: > > > > > > The general reasoning of when to use GFP_NOFS below is fine, I don't > > > disagree with that at all. If there is no way a recursion back into btrfs > > > can't happen at that allocation site then we can use GFP_KERNEL. > > > > > > That said, have you closely audited this path? Does the allocation happen > > > completely outside any locks that might be shared with the writeout path? > > > What happens if we have to do writeout of the inode being fiemapped in > > > order > > > to allocate space? If the answer to all my questions is "there is no way > > > this can deadlock" then by all means, we should use GFP_KERNEL. Otherwise > > > GFP_NOFS is a sensible guard against possible future deadlocks. > > > > This is exactly the situation we discussed at LSF/MM this year. The MM > > folks are pushing back because the fs folks tend to use GFP_NOFS as a > > talisman. The audit needs to happen, otherwise that last sentence is > > another talisman. > > There's nothing here I disagree with. I'm not seeing a strong technical > justification, which is what I want (being called from an ioctl means > nothing in this case). A small amount of searching shows me that extent_fiemap() does lock_extent_bits() and writepage_delalloc() also calls lock_extent_bits() (via find_lock_delalloc_range()). I'm no expert on the extent locking but that seems pretty deadlocky to me. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On Thu, Jun 02, 2016 at 04:56:06PM -0400, Jeff Mahoney wrote: > On 6/2/16 3:08 PM, Mark Fasheh wrote: > > On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: > >> On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > +/* dynamically allocate and initialize a ref_root */ > +static struct ref_root *ref_root_alloc(void) > +{ > +struct ref_root *ref_tree; > + > +ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); > >>> > >>> I'm pretty sure we want GFP_NOFS here. > >> > >> Then please explain to me why/where the reasoning below is wrong: > > > > The general reasoning of when to use GFP_NOFS below is fine, I don't > > disagree with that at all. If there is no way a recursion back into btrfs > > can't happen at that allocation site then we can use GFP_KERNEL. > > > > That said, have you closely audited this path? Does the allocation happen > > completely outside any locks that might be shared with the writeout path? > > What happens if we have to do writeout of the inode being fiemapped in order > > to allocate space? If the answer to all my questions is "there is no way > > this can deadlock" then by all means, we should use GFP_KERNEL. Otherwise > > GFP_NOFS is a sensible guard against possible future deadlocks. > > This is exactly the situation we discussed at LSF/MM this year. The MM > folks are pushing back because the fs folks tend to use GFP_NOFS as a > talisman. The audit needs to happen, otherwise that last sentence is > another talisman. There's nothing here I disagree with. I'm not seeing a strong technical justification, which is what I want (being called from an ioctl means nothing in this case). --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On 6/2/16 3:08 PM, Mark Fasheh wrote: > On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: >> On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: +/* dynamically allocate and initialize a ref_root */ +static struct ref_root *ref_root_alloc(void) +{ + struct ref_root *ref_tree; + + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); >>> >>> I'm pretty sure we want GFP_NOFS here. >> >> Then please explain to me why/where the reasoning below is wrong: > > The general reasoning of when to use GFP_NOFS below is fine, I don't > disagree with that at all. If there is no way a recursion back into btrfs > can't happen at that allocation site then we can use GFP_KERNEL. > > That said, have you closely audited this path? Does the allocation happen > completely outside any locks that might be shared with the writeout path? > What happens if we have to do writeout of the inode being fiemapped in order > to allocate space? If the answer to all my questions is "there is no way > this can deadlock" then by all means, we should use GFP_KERNEL. Otherwise > GFP_NOFS is a sensible guard against possible future deadlocks. This is exactly the situation we discussed at LSF/MM this year. The MM folks are pushing back because the fs folks tend to use GFP_NOFS as a talisman. The audit needs to happen, otherwise that last sentence is another talisman. -Jeff -- Jeff Mahoney SUSE Labs signature.asc Description: OpenPGP digital signature
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On Thu, Jun 02, 2016 at 01:46:27PM +0800, Lu Fengqi wrote: > > At 06/02/2016 05:15 AM, Mark Fasheh wrote: > >Thanks for trying to fix this problem, comments below. > > > >On Wed, Jun 01, 2016 at 01:48:05PM +0800, Lu Fengqi wrote: > >>Only in the case of different root_id or different object_id, check_shared > >>identified extent as the shared. However, If a extent was referred by > >>different offset of same file, it should also be identified as shared. > >>In addition, check_shared's loop scale is at least n^3, so if a extent > >>has too many references, even causes soft hang up. > >> > >>First, add all delayed_ref to the ref_tree and calculate the unqiue_refs, > >>if the unique_refs is greater than one, return BACKREF_FOUND_SHARED. > >>Then individually add the on-disk reference(inline/keyed) to the ref_tree > >>and calculate the unique_refs of the ref_tree to check if the unique_refs > >>is greater than one.Because once there are two references to return > >>SHARED, so the time complexity is close to the constant. > >Constant time in the best case, but still n^3 in the worst case right? I'm > >not complaining btw, I just want to be sure we're not over promising :) > Only in case of a large number of delayed_ref, the worst case time > complexity will be n^2*logn. Otherwise, it will be constant even if > there are many on-disk references. Ahh ok so it's driven more by the # of delayed refs. That makes sense, thanks. > >>@@ -34,6 +35,253 @@ struct extent_inode_elem { > >>struct extent_inode_elem *next; > >> }; > >> > >>+/* > >>+ * ref_root is used as the root of the ref tree that hold a collection > >>+ * of unique references. > >>+ */ > >>+struct ref_root { > >>+ struct rb_root rb_root; > >>+ > >>+ /* > >>+* the unique_refs represents the number of ref_nodes with a positive > >>+* count stored in the tree. Even if a ref_node(the count is greater > >>+* than one) is added, the unique_refs will only increase one. > >>+*/ > >>+ unsigned int unique_refs; > >>+}; > >>+ > >>+/* ref_node is used to store a unique reference to the ref tree. */ > >>+struct ref_node { > >>+ struct rb_node rb_node; > >>+ > >>+ /* for NORMAL_REF, otherwise all these fields should be set to 0 */ > >>+ u64 root_id; > >>+ u64 object_id; > >>+ u64 offset; > >>+ > >>+ /* for SHARED_REF, otherwise parent field should be set to 0 */ > >>+ u64 parent; > >>+ > >>+ /* ref to the ref_mod of btrfs_delayed_ref_node(delayed-ref.h) */ > >>+ int ref_mod; > >>+}; > >Why are we mirroring so much of the backref structures here? It seems like > >we're just throwing layers on top of layers. Can't we modify the backref > >structures and code to handle whatever small amount of unique accounting you > >must do? > The original structure(struct __prelim_ref) store reference in list, > and I have to perform many search operations that not suitable for > list. However, if I modify the original structure, it would require > a lot of rework. So I just want to fix fiemap with this patch. If > necessary, we can use this structure to replace the original > structure later. Well there's room for an rb_node on that structure so we can solve the 'it only uses a list' problem trivially. I definitely understand your reluctance to modify the backref code, but to me that just sounds like we need someone who is familiar with that code to review your work and provide advice when needed. Otherwise, I believe my point holds. If there's some technical reason why this is a bad idea, that's a different story. So far though this just seems like a situation where we need some extra review from the primary developers. I cc'd Josef in the hopes he could shed some light for us. > >>+/* dynamically allocate and initialize a ref_root */ > >>+static struct ref_root *ref_root_alloc(void) > >>+{ > >>+ struct ref_root *ref_tree; > >>+ > >>+ ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); > >I'm pretty sure we want GFP_NOFS here. > Yes, perhaps you're right. > >Because there's no need to narrow the allocation constraints. GFP_NOFS > >is necessary when the caller is on a critical path that must not recurse > >back to the filesystem through the allocation (ie. if the allocator > >decides to free some memory and tries tro write dirty data). FIEMAP is > >called from an ioctl. > But David seems to have a different point of view with you, so I > would like to ask for his advice again. Sounds good, hopefully David and I can figure it out :) Thanks again Lu, --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On Thu, Jun 02, 2016 at 07:07:32PM +0200, David Sterba wrote: > On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > > +/* dynamically allocate and initialize a ref_root */ > > > +static struct ref_root *ref_root_alloc(void) > > > +{ > > > + struct ref_root *ref_tree; > > > + > > > + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); > > > > I'm pretty sure we want GFP_NOFS here. > > Then please explain to me why/where the reasoning below is wrong: The general reasoning of when to use GFP_NOFS below is fine, I don't disagree with that at all. If there is no way a recursion back into btrfs can't happen at that allocation site then we can use GFP_KERNEL. That said, have you closely audited this path? Does the allocation happen completely outside any locks that might be shared with the writeout path? What happens if we have to do writeout of the inode being fiemapped in order to allocate space? If the answer to all my questions is "there is no way this can deadlock" then by all means, we should use GFP_KERNEL. Otherwise GFP_NOFS is a sensible guard against possible future deadlocks. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > +/* dynamically allocate and initialize a ref_root */ > > +static struct ref_root *ref_root_alloc(void) > > +{ > > + struct ref_root *ref_tree; > > + > > + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); > > I'm pretty sure we want GFP_NOFS here. Then please explain to me why/where the reasoning below is wrong: > > Because there's no need to narrow the allocation constraints. GFP_NOFS > > is necessary when the caller is on a critical path that must not recurse > > back to the filesystem through the allocation (ie. if the allocator > > decides to free some memory and tries tro write dirty data). FIEMAP is > > called from an ioctl. -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
At 06/02/2016 05:15 AM, Mark Fasheh wrote: Thanks for trying to fix this problem, comments below. On Wed, Jun 01, 2016 at 01:48:05PM +0800, Lu Fengqi wrote: Only in the case of different root_id or different object_id, check_shared identified extent as the shared. However, If a extent was referred by different offset of same file, it should also be identified as shared. In addition, check_shared's loop scale is at least n^3, so if a extent has too many references, even causes soft hang up. First, add all delayed_ref to the ref_tree and calculate the unqiue_refs, if the unique_refs is greater than one, return BACKREF_FOUND_SHARED. Then individually add the on-disk reference(inline/keyed) to the ref_tree and calculate the unique_refs of the ref_tree to check if the unique_refs is greater than one.Because once there are two references to return SHARED, so the time complexity is close to the constant. Constant time in the best case, but still n^3 in the worst case right? I'm not complaining btw, I just want to be sure we're not over promising :) Only in case of a large number of delayed_ref, the worst case time complexity will be n^2*logn. Otherwise, it will be constant even if there are many on-disk references. @@ -34,6 +35,253 @@ struct extent_inode_elem { struct extent_inode_elem *next; }; +/* + * ref_root is used as the root of the ref tree that hold a collection + * of unique references. + */ +struct ref_root { + struct rb_root rb_root; + + /* +* the unique_refs represents the number of ref_nodes with a positive +* count stored in the tree. Even if a ref_node(the count is greater +* than one) is added, the unique_refs will only increase one. +*/ + unsigned int unique_refs; +}; + +/* ref_node is used to store a unique reference to the ref tree. */ +struct ref_node { + struct rb_node rb_node; + + /* for NORMAL_REF, otherwise all these fields should be set to 0 */ + u64 root_id; + u64 object_id; + u64 offset; + + /* for SHARED_REF, otherwise parent field should be set to 0 */ + u64 parent; + + /* ref to the ref_mod of btrfs_delayed_ref_node(delayed-ref.h) */ + int ref_mod; +}; Why are we mirroring so much of the backref structures here? It seems like we're just throwing layers on top of layers. Can't we modify the backref structures and code to handle whatever small amount of unique accounting you must do? The original structure(struct __prelim_ref) store reference in list, and I have to perform many search operations that not suitable for list. However, if I modify the original structure, it would require a lot of rework. So I just want to fix fiemap with this patch. If necessary, we can use this structure to replace the original structure later. +/* dynamically allocate and initialize a ref_root */ +static struct ref_root *ref_root_alloc(void) +{ + struct ref_root *ref_tree; + + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); I'm pretty sure we want GFP_NOFS here. Yes, perhaps you're right. Because there's no need to narrow the allocation constraints. GFP_NOFS is necessary when the caller is on a critical path that must not recurse back to the filesystem through the allocation (ie. if the allocator decides to free some memory and tries tro write dirty data). FIEMAP is called from an ioctl. But David seems to have a different point of view with you, so I would like to ask for his advice again. + +/* + * search ref_node with (root_id, object_id, offset, parent) in the tree + * + * if found, the pointer of the ref_node will be returned; + * if not found, NULL will be returned and pos will point to the rb_node for + * insert, pos_parent will point to pos'parent for insert; +*/ +static struct ref_node *__ref_tree_search(struct ref_root *ref_tree, + struct rb_node ***pos, + struct rb_node **pos_parent, + u64 root_id, u64 object_id, + u64 offset, u64 parent) +{ + struct ref_node *cur = NULL; + + *pos = &ref_tree->rb_root.rb_node; + + while (**pos) { + *pos_parent = **pos; + cur = rb_entry(*pos_parent, struct ref_node, rb_node); + + if (cur->root_id < root_id) { + *pos = &(**pos)->rb_right; + continue; + } else if (cur->root_id > root_id) { + *pos = &(**pos)->rb_left; + continue; + } + + if (cur->object_id < object_id) { + *pos = &(**pos)->rb_right; + continue; + } else if (cur->object_id > object_id) { + *pos = &(**pos)->rb_left; + continue; + } + + if (cur->offset < offset) { +
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
On Wed, Jun 01, 2016 at 02:15:22PM -0700, Mark Fasheh wrote: > > +static int ref_tree_add(struct ref_root *ref_tree, u64 root_id, u64 > > object_id, > > + u64 offset, u64 parent, int count) > > +{ > > + struct ref_node *node = NULL; > > + struct rb_node **pos = NULL; > > + struct rb_node *pos_parent = NULL; > > + int origin_count; > > + int ret; > > + > > + if (!count) > > + return 0; > > + > > + node = __ref_tree_search(ref_tree, &pos, &pos_parent, root_id, > > +object_id, offset, parent); > > + if (node == NULL) { > > + node = kmalloc(sizeof(*node), GFP_KERNEL); > > + if (!node) > > + return -ENOMEM; > > + > > + node->root_id = root_id; > > + node->object_id = object_id; > > + node->offset = offset; > > + node->parent = parent; > > + node->ref_mod = count; > > + > > + ret = ref_tree_insert(ref_tree, pos, pos_parent, node); > > + ASSERT(!ret); > > + if (ret) { > > + kfree(node); > > + return ret; > > + } > > If you put the open coded comparisons into their own function, then it > should be trivial to call that here and we can have a 'standard' looking > rbtree insert instead of this custom version. See the rbtree.h header for an > example. oops, I meant Documentation/rbtree.txt instead of include/linux/rbtree.h. --Mark -- Mark Fasheh -- To unsubscribe from this list: send the line "unsubscribe linux-btrfs" in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH v2] btrfs: fix check_shared for fiemap ioctl
Thanks for trying to fix this problem, comments below. On Wed, Jun 01, 2016 at 01:48:05PM +0800, Lu Fengqi wrote: > Only in the case of different root_id or different object_id, check_shared > identified extent as the shared. However, If a extent was referred by > different offset of same file, it should also be identified as shared. > In addition, check_shared's loop scale is at least n^3, so if a extent > has too many references, even causes soft hang up. > > First, add all delayed_ref to the ref_tree and calculate the unqiue_refs, > if the unique_refs is greater than one, return BACKREF_FOUND_SHARED. > Then individually add the on-disk reference(inline/keyed) to the ref_tree > and calculate the unique_refs of the ref_tree to check if the unique_refs > is greater than one.Because once there are two references to return > SHARED, so the time complexity is close to the constant. Constant time in the best case, but still n^3 in the worst case right? I'm not complaining btw, I just want to be sure we're not over promising :) > @@ -34,6 +35,253 @@ struct extent_inode_elem { > struct extent_inode_elem *next; > }; > > +/* > + * ref_root is used as the root of the ref tree that hold a collection > + * of unique references. > + */ > +struct ref_root { > + struct rb_root rb_root; > + > + /* > + * the unique_refs represents the number of ref_nodes with a positive > + * count stored in the tree. Even if a ref_node(the count is greater > + * than one) is added, the unique_refs will only increase one. > + */ > + unsigned int unique_refs; > +}; > + > +/* ref_node is used to store a unique reference to the ref tree. */ > +struct ref_node { > + struct rb_node rb_node; > + > + /* for NORMAL_REF, otherwise all these fields should be set to 0 */ > + u64 root_id; > + u64 object_id; > + u64 offset; > + > + /* for SHARED_REF, otherwise parent field should be set to 0 */ > + u64 parent; > + > + /* ref to the ref_mod of btrfs_delayed_ref_node(delayed-ref.h) */ > + int ref_mod; > +}; Why are we mirroring so much of the backref structures here? It seems like we're just throwing layers on top of layers. Can't we modify the backref structures and code to handle whatever small amount of unique accounting you must do? > +/* dynamically allocate and initialize a ref_root */ > +static struct ref_root *ref_root_alloc(void) > +{ > + struct ref_root *ref_tree; > + > + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); I'm pretty sure we want GFP_NOFS here. > + > +/* > + * search ref_node with (root_id, object_id, offset, parent) in the tree > + * > + * if found, the pointer of the ref_node will be returned; > + * if not found, NULL will be returned and pos will point to the rb_node for > + * insert, pos_parent will point to pos'parent for insert; > +*/ > +static struct ref_node *__ref_tree_search(struct ref_root *ref_tree, > + struct rb_node ***pos, > + struct rb_node **pos_parent, > + u64 root_id, u64 object_id, > + u64 offset, u64 parent) > +{ > + struct ref_node *cur = NULL; > + > + *pos = &ref_tree->rb_root.rb_node; > + > + while (**pos) { > + *pos_parent = **pos; > + cur = rb_entry(*pos_parent, struct ref_node, rb_node); > + > + if (cur->root_id < root_id) { > + *pos = &(**pos)->rb_right; > + continue; > + } else if (cur->root_id > root_id) { > + *pos = &(**pos)->rb_left; > + continue; > + } > + > + if (cur->object_id < object_id) { > + *pos = &(**pos)->rb_right; > + continue; > + } else if (cur->object_id > object_id) { > + *pos = &(**pos)->rb_left; > + continue; > + } > + > + if (cur->offset < offset) { > + *pos = &(**pos)->rb_right; > + continue; > + } else if (cur->offset > offset) { > + *pos = &(**pos)->rb_left; > + continue; > + } > + > + if (cur->parent < parent) { > + *pos = &(**pos)->rb_right; > + continue; > + } else if (cur->parent > parent) { > + *pos = &(**pos)->rb_left; > + continue; > + } These 4 comparisons need to be in their own cmp() function - we want to avoid open coded comparisons with an rbtree search. This is a pretty standard way to handle it. > + > + return cur; > + } > + > + return NULL; > +} > + > +/* > + * insert a ref_node to the ref tree > + * @pos used for specifiy the position to insert > + * @pos_parent for specifiy pos's parent > + * > + *
[PATCH v2] btrfs: fix check_shared for fiemap ioctl
Only in the case of different root_id or different object_id, check_shared identified extent as the shared. However, If a extent was referred by different offset of same file, it should also be identified as shared. In addition, check_shared's loop scale is at least n^3, so if a extent has too many references, even causes soft hang up. First, add all delayed_ref to the ref_tree and calculate the unqiue_refs, if the unique_refs is greater than one, return BACKREF_FOUND_SHARED. Then individually add the on-disk reference(inline/keyed) to the ref_tree and calculate the unique_refs of the ref_tree to check if the unique_refs is greater than one.Because once there are two references to return SHARED, so the time complexity is close to the constant. Reported-by: Tsutomu Itoh Signed-off-by: Lu Fengqi --- fs/btrfs/backref.c | 349 +-- fs/btrfs/extent_io.c | 18 ++- 2 files changed, 357 insertions(+), 10 deletions(-) diff --git a/fs/btrfs/backref.c b/fs/btrfs/backref.c index 80e8472..325a144 100644 --- a/fs/btrfs/backref.c +++ b/fs/btrfs/backref.c @@ -17,6 +17,7 @@ */ #include +#include #include "ctree.h" #include "disk-io.h" #include "backref.h" @@ -34,6 +35,253 @@ struct extent_inode_elem { struct extent_inode_elem *next; }; +/* + * ref_root is used as the root of the ref tree that hold a collection + * of unique references. + */ +struct ref_root { + struct rb_root rb_root; + + /* +* the unique_refs represents the number of ref_nodes with a positive +* count stored in the tree. Even if a ref_node(the count is greater +* than one) is added, the unique_refs will only increase one. +*/ + unsigned int unique_refs; +}; + +/* ref_node is used to store a unique reference to the ref tree. */ +struct ref_node { + struct rb_node rb_node; + + /* for NORMAL_REF, otherwise all these fields should be set to 0 */ + u64 root_id; + u64 object_id; + u64 offset; + + /* for SHARED_REF, otherwise parent field should be set to 0 */ + u64 parent; + + /* ref to the ref_mod of btrfs_delayed_ref_node(delayed-ref.h) */ + int ref_mod; +}; + +/* dynamically allocate and initialize a ref_root */ +static struct ref_root *ref_root_alloc(void) +{ + struct ref_root *ref_tree; + + ref_tree = kmalloc(sizeof(*ref_tree), GFP_KERNEL); + if (!ref_tree) + return NULL; + + ref_tree->rb_root = RB_ROOT; + ref_tree->unique_refs = 0; + + return ref_tree; +} + +/* free all nodes in the ref tree, and reinit ref_root */ +static void ref_root_fini(struct ref_root *ref_tree) +{ + struct ref_node *node; + struct rb_node *next; + + while ((next = rb_first(&ref_tree->rb_root)) != NULL) { + node = rb_entry(next, struct ref_node, rb_node); + rb_erase(next, &ref_tree->rb_root); + kfree(node); + } + + ref_tree->rb_root = RB_ROOT; + ref_tree->unique_refs = 0; +} + +/* free dynamically allocated ref_root */ +static void ref_root_free(struct ref_root *ref_tree) +{ + if (!ref_tree) + return; + + ref_root_fini(ref_tree); + kfree(ref_tree); +} + +/* + * search ref_node with (root_id, object_id, offset, parent) in the tree + * + * if found, the pointer of the ref_node will be returned; + * if not found, NULL will be returned and pos will point to the rb_node for + * insert, pos_parent will point to pos'parent for insert; +*/ +static struct ref_node *__ref_tree_search(struct ref_root *ref_tree, + struct rb_node ***pos, + struct rb_node **pos_parent, + u64 root_id, u64 object_id, + u64 offset, u64 parent) +{ + struct ref_node *cur = NULL; + + *pos = &ref_tree->rb_root.rb_node; + + while (**pos) { + *pos_parent = **pos; + cur = rb_entry(*pos_parent, struct ref_node, rb_node); + + if (cur->root_id < root_id) { + *pos = &(**pos)->rb_right; + continue; + } else if (cur->root_id > root_id) { + *pos = &(**pos)->rb_left; + continue; + } + + if (cur->object_id < object_id) { + *pos = &(**pos)->rb_right; + continue; + } else if (cur->object_id > object_id) { + *pos = &(**pos)->rb_left; + continue; + } + + if (cur->offset < offset) { + *pos = &(**pos)->rb_right; + continue; + } else if (cur->offset > offset) { + *pos = &(**pos)->rb_left; + continue; + } + +