Re: [PATCH 0/3] flush delalloc by multi-task
Hi, Josef Please drop this patchset from your btrfs-next tree because it may cause the performance regression in some cases. I'll improve it later. Thanks Miao On thu, 25 Oct 2012 17:20:29 +0800, Miao Xie wrote: This patchset introduce multi-task delalloc flush, it can make the delalloc flush more faster. And besides that, it also can fix the problem that we join the same transaction handler more than 2 times. Implementation: - Create a new worker pool. - Queue the inode with pending delalloc into the work queue of the worker pool when we want to force them into the disk, and then we will wait till all the works we submit are done. - The ordered extents also can be queued into this work queue. The process is similar to the second one. Miao Xie (3): Btrfs: make delalloc inodes be flushed by multi-task Btrfs: make ordered operations be handled by multi-task Btrfs: make ordered extent be flushed by multi-task fs/btrfs/ctree.h| 14 +++ fs/btrfs/disk-io.c |7 fs/btrfs/inode.c| 78 ++--- fs/btrfs/ordered-data.c | 87 ++- fs/btrfs/ordered-data.h |7 +++- fs/btrfs/relocation.c |6 +++- fs/btrfs/transaction.c | 24 ++--- 7 files changed, 185 insertions(+), 38 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] flush delalloc by multi-task
On 10/26/2012 11:25 AM, Miao Xie wrote: On Fri, 26 Oct 2012 10:05:55 +0800, Liu Bo wrote: On 10/26/2012 09:56 AM, Miao Xie wrote: I can see the potential improvements brought by flushing inodes this way. But I don't think it makes much sense by making waiting process multi-task, since even we spread wait order extents into different cpus, they just occpied the cpu and went on waiting and scheduled then, I mean, the bottleneck is on what we're waiting for. Thanks for your comment, I think only btrfs_run_ordered_operations(root, 0) needn't wait for the works, the others must wait. The first reason is to avoid changing the semantic of those tree function. The second reason is we have to wait for the completion of all works, if not, the file data in snapshots may be different with the source suvolumes because the flush may not end before the snapshot creation. Yes, it's right that they must wait for all workers to finish. But I don't mean that(sorry for my confusing words). IMO we don't need to let *btrfs_wait_ordered_extents()* run as multi-task. It also need to be done by multi-task because btrfs_wait_ordered_extents() doesn't imply that all the dirty pages in the ordered extent have been written into the disk, that is it also need do lots of things before waiting for the event - BTRFS_ORDERED_COMPLETE, so the multi-task process is useful, I think. Well, I missed the flushing part. Anyway, we need test to validate it. Thanks Miao thanks, liubo -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
[PATCH 0/3] flush delalloc by multi-task
This patchset introduce multi-task delalloc flush, it can make the delalloc flush more faster. And besides that, it also can fix the problem that we join the same transaction handler more than 2 times. Implementation: - Create a new worker pool. - Queue the inode with pending delalloc into the work queue of the worker pool when we want to force them into the disk, and then we will wait till all the works we submit are done. - The ordered extents also can be queued into this work queue. The process is similar to the second one. Miao Xie (3): Btrfs: make delalloc inodes be flushed by multi-task Btrfs: make ordered operations be handled by multi-task Btrfs: make ordered extent be flushed by multi-task fs/btrfs/ctree.h| 14 +++ fs/btrfs/disk-io.c |7 fs/btrfs/inode.c| 78 ++--- fs/btrfs/ordered-data.c | 87 ++- fs/btrfs/ordered-data.h |7 +++- fs/btrfs/relocation.c |6 +++- fs/btrfs/transaction.c | 24 ++--- 7 files changed, 185 insertions(+), 38 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] flush delalloc by multi-task
On 10/25/2012 05:20 PM, Miao Xie wrote: This patchset introduce multi-task delalloc flush, it can make the delalloc flush more faster. And besides that, it also can fix the problem that we join the same transaction handler more than 2 times. Implementation: - Create a new worker pool. - Queue the inode with pending delalloc into the work queue of the worker pool when we want to force them into the disk, and then we will wait till all the works we submit are done. - The ordered extents also can be queued into this work queue. The process is similar to the second one. I can see the potential improvements brought by flushing inodes this way. But I don't think it makes much sense by making waiting process multi-task, since even we spread wait order extents into different cpus, they just occpied the cpu and went on waiting and scheduled then, I mean, the bottleneck is on what we're waiting for. Besides, considering that this patchset is about to getting us better performance, I'm expecting any performance numbers (I'm a little worried about context switches overhead). btw, cool ideas indeed. thanks, liubo Miao Xie (3): Btrfs: make delalloc inodes be flushed by multi-task Btrfs: make ordered operations be handled by multi-task Btrfs: make ordered extent be flushed by multi-task fs/btrfs/ctree.h| 14 +++ fs/btrfs/disk-io.c |7 fs/btrfs/inode.c| 78 ++--- fs/btrfs/ordered-data.c | 87 ++- fs/btrfs/ordered-data.h |7 +++- fs/btrfs/relocation.c |6 +++- fs/btrfs/transaction.c | 24 ++--- 7 files changed, 185 insertions(+), 38 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] flush delalloc by multi-task
On thu, 25 Oct 2012 19:53:05 +0800, Liu Bo wrote: On 10/25/2012 05:20 PM, Miao Xie wrote: This patchset introduce multi-task delalloc flush, it can make the delalloc flush more faster. And besides that, it also can fix the problem that we join the same transaction handler more than 2 times. Implementation: - Create a new worker pool. - Queue the inode with pending delalloc into the work queue of the worker pool when we want to force them into the disk, and then we will wait till all the works we submit are done. - The ordered extents also can be queued into this work queue. The process is similar to the second one. I can see the potential improvements brought by flushing inodes this way. But I don't think it makes much sense by making waiting process multi-task, since even we spread wait order extents into different cpus, they just occpied the cpu and went on waiting and scheduled then, I mean, the bottleneck is on what we're waiting for. Thanks for your comment, I think only btrfs_run_ordered_operations(root, 0) needn't wait for the works, the others must wait. The first reason is to avoid changing the semantic of those tree function. The second reason is we have to wait for the completion of all works, if not, the file data in snapshots may be different with the source suvolumes because the flush may not end before the snapshot creation. Besides, considering that this patchset is about to getting us better performance, I'm expecting any performance numbers (I'm a little worried about context switches overhead). OK, I'll send it out later. Thanks Miao btw, cool ideas indeed. thanks, liubo Miao Xie (3): Btrfs: make delalloc inodes be flushed by multi-task Btrfs: make ordered operations be handled by multi-task Btrfs: make ordered extent be flushed by multi-task fs/btrfs/ctree.h| 14 +++ fs/btrfs/disk-io.c |7 fs/btrfs/inode.c| 78 ++--- fs/btrfs/ordered-data.c | 87 ++- fs/btrfs/ordered-data.h |7 +++- fs/btrfs/relocation.c |6 +++- fs/btrfs/transaction.c | 24 ++--- 7 files changed, 185 insertions(+), 38 deletions(-) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] flush delalloc by multi-task
On 10/26/2012 09:56 AM, Miao Xie wrote: I can see the potential improvements brought by flushing inodes this way. But I don't think it makes much sense by making waiting process multi-task, since even we spread wait order extents into different cpus, they just occpied the cpu and went on waiting and scheduled then, I mean, the bottleneck is on what we're waiting for. Thanks for your comment, I think only btrfs_run_ordered_operations(root, 0) needn't wait for the works, the others must wait. The first reason is to avoid changing the semantic of those tree function. The second reason is we have to wait for the completion of all works, if not, the file data in snapshots may be different with the source suvolumes because the flush may not end before the snapshot creation. Yes, it's right that they must wait for all workers to finish. But I don't mean that(sorry for my confusing words). IMO we don't need to let *btrfs_wait_ordered_extents()* run as multi-task. thanks, liubo -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
Re: [PATCH 0/3] flush delalloc by multi-task
On Fri, 26 Oct 2012 10:05:55 +0800, Liu Bo wrote: On 10/26/2012 09:56 AM, Miao Xie wrote: I can see the potential improvements brought by flushing inodes this way. But I don't think it makes much sense by making waiting process multi-task, since even we spread wait order extents into different cpus, they just occpied the cpu and went on waiting and scheduled then, I mean, the bottleneck is on what we're waiting for. Thanks for your comment, I think only btrfs_run_ordered_operations(root, 0) needn't wait for the works, the others must wait. The first reason is to avoid changing the semantic of those tree function. The second reason is we have to wait for the completion of all works, if not, the file data in snapshots may be different with the source suvolumes because the flush may not end before the snapshot creation. Yes, it's right that they must wait for all workers to finish. But I don't mean that(sorry for my confusing words). IMO we don't need to let *btrfs_wait_ordered_extents()* run as multi-task. It also need to be done by multi-task because btrfs_wait_ordered_extents() doesn't imply that all the dirty pages in the ordered extent have been written into the disk, that is it also need do lots of things before waiting for the event - BTRFS_ORDERED_COMPLETE, so the multi-task process is useful, I think. Anyway, we need test to validate it. Thanks Miao thanks, liubo -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body of a message to majord...@vger.kernel.org More majordomo info at http://vger.kernel.org/majordomo-info.html
