[sheepdog] [PATCH v4 RESEND 8/8] sheepdev: add Makefile.in and modify configure.ac to adopt sheepdev

2013-01-23 Thread levin li
From: levin li Signed-off-by: levin li --- Makefile.am | 4 configure.ac | 9 + sheepdev/Makefile.in | 8 3 files changed, 21 insertions(+) create mode 100644 sheepdev/Makefile.in diff --git a/Makefile.am b/Makefile.am index 53d18b9..4db40bf 100644

[sheepdog] [PATCH v4 RESEND 7/8] sheepdev: make it safe to open a snapshot VDI

2013-01-23 Thread levin li
From: levin li In case we open a snapshot VDI to make it as a block device, we immediately create a new writable VDI for this snapshot just as what QEMU does, then we can safely write to this VDI Signed-off-by: levin li --- sheepdev/sheep.c | 60

[sheepdog] [PATCH v4 RESEND 6/8] sheepdev: create kernel threads to process the IO request in the driver

2013-01-23 Thread levin li
From: levin li This patch creates two kernel threads 'sheep_req' and 'sheep_fin' to process the IO requests and sheep response respectively, sheep_req just fetches IO request from the pending list, and forward the request to sheep, sheep_fin waits for sheep response, by

[sheepdog] [PATCH v4 RESEND 3/8] sheepdev: add connect.c for network operations

2013-01-23 Thread levin li
From: levin li Signed-off-by: levin li --- sheepdev/Kbuild | 3 +- sheepdev/connect.c | 178 sheepdev/sheepdev.h | 7 +++ 3 files changed, 187 insertions(+), 1 deletion(-) create mode 100644 sheepdev/connect.c diff --git a

[sheepdog] [PATCH v4 RESEND 5/8] sheepdev: make it possible to add or remove a sheep device

2013-01-23 Thread levin li
From: levin li This patch provides functions to create a sheep device by writing to the proc entry, but it hasn't processed the IO requests, it just ends the requests with -EIO. It creates sheep devices named /dev/sheepX, but until now, it can not handle IO request, any read/write reque

[sheepdog] [PATCH v4 RESEND 4/8] sheepdev: add sheep.c to process the sheepdog protocol

2013-01-23 Thread levin li
From: levin li Signed-off-by: levin li --- sheepdev/Kbuild | 3 +- sheepdev/sheep.c| 186 sheepdev/sheepdev.h | 8 +++ 3 files changed, 196 insertions(+), 1 deletion(-) create mode 100644 sheepdev/sheep.c diff --git a/sheepdev

[sheepdog] [PATCH v4 RESEND 2/8] sheepdev: use proc entry to control adding/removing devices

2013-01-23 Thread levin li
From: levin li Add a proc entry /proc/sheep for sheep device controlling, by writing to this entry to add or remove device For example: Add a sheep VDI as a block device: echo "add 127.0.0.1 linux:tag" > /proc/sheep Remove a sheep device: echo "del linux:tag" >

[sheepdog] [PATCH v4 RESEND 1/8] sheep: modify sheepdog_proto.h to make it usable in linux kernel module

2013-01-23 Thread levin li
From: levin li Signed-off-by: levin li --- include/sheepdog_proto.h | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h index 865f451..fb35f3a 100644 --- a/include/sheepdog_proto.h +++ b/include/sheepdog_proto.h

[sheepdog] [PATCH v4 RESEND 0/8] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-23 Thread levin li
From: levin li v3 -- > v4: 1. Refactor the code in proc.c and device.c 2. Split the big patch into some small patches 3. Fixed coding style to pass checkpatch.pl check v2 -- > v3: 1. Make it Ok for QEMU to run VDIs from the driver 2. Code refactor 3. Use the format 'del vdina

[sheepdog] [PATCH v4 8/8] sheepdev: add Makefile.in and modify configure.ac to adopt sheepdev

2013-01-22 Thread levin li
From: levin li Signed-off-by: levin li --- Makefile.am | 4 configure.ac | 9 + 2 files changed, 13 insertions(+) diff --git a/Makefile.am b/Makefile.am index 53d18b9..4db40bf 100644 --- a/Makefile.am +++ b/Makefile.am @@ -23,6 +23,10 @@ if BUILD_SHEEPFS SUBDIRS

[sheepdog] [PATCH v4 7/8] sheepdev: make it safe to open a snapshot VDI

2013-01-22 Thread levin li
From: levin li In case we open a snapshot VDI to make it as a block device, we immediately create a new writable VDI for this snapshot just as what QEMU does, then we can safely write to this VDI Signed-off-by: levin li --- sheepdev/sheep.c | 60

[sheepdog] [PATCH v4 6/8] sheepdev: create kernel threads to process the IO request in the driver

2013-01-22 Thread levin li
From: levin li This patch creates two kernel threads 'sheep_req' and 'sheep_fin' to process the IO requests and sheep response respectively, sheep_req just fetches IO request from the pending list, and forward the request to sheep, sheep_fin waits for sheep response, by

[sheepdog] [PATCH v4 5/8] sheepdev: make it possible to add or remove a sheep device

2013-01-22 Thread levin li
From: levin li This patch provides functions to create a sheep device by writing to the proc entry, but it hasn't processed the IO requests, it just ends the requests with -EIO. It creates sheep devices named /dev/sheepX, but until now, it can not handle IO request, any read/write reque

[sheepdog] [PATCH v4 4/8] sheepdev: add sheep.c to process the sheepdog protocol

2013-01-22 Thread levin li
From: levin li Signed-off-by: levin li --- sheepdev/Kbuild | 3 +- sheepdev/sheep.c| 186 sheepdev/sheepdev.h | 8 +++ 3 files changed, 196 insertions(+), 1 deletion(-) create mode 100644 sheepdev/sheep.c diff --git a/sheepdev

[sheepdog] [PATCH v4 3/8] sheepdev: add connect.c for network operations

2013-01-22 Thread levin li
From: levin li Signed-off-by: levin li --- sheepdev/Kbuild | 3 +- sheepdev/connect.c | 178 sheepdev/sheepdev.h | 7 +++ 3 files changed, 187 insertions(+), 1 deletion(-) create mode 100644 sheepdev/connect.c diff --git a

[sheepdog] [PATCH v4 2/8] sheepdev: use proc entry to control adding/removing devices

2013-01-22 Thread levin li
From: levin li Add a proc entry /proc/sheep for sheep device controlling, by writing to this entry to add or remove device For example: Add a sheep VDI as a block device: echo "add 127.0.0.1 linux:tag" > /proc/sheep Remove a sheep device: echo "del linux:tag" >

[sheepdog] [PATCH v4 1/8] sheep: modify sheepdog_proto.h to make it usable in linux kernel module

2013-01-22 Thread levin li
From: levin li Signed-off-by: levin li --- include/sheepdog_proto.h | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h index 865f451..fb35f3a 100644 --- a/include/sheepdog_proto.h +++ b/include/sheepdog_proto.h

[sheepdog] [PATCH v4 0/8] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-22 Thread levin li
From: levin li v3 -- > v4: 1. Refactor the code in proc.c and device.c 2. Split the big patch into some small patches 3. Fixed coding style to pass checkpatch.pl check v2 -- > v3: 1. Make it Ok for QEMU to run VDIs from the driver 2. Code refactor 3. Use the format 'del vdina

Re: [sheepdog] [PATCH v3 2/4] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-22 Thread levin li
On 01/22/2013 01:20 PM, Liu Yuan wrote: On 01/22/2013 11:16 AM, levin li wrote: sheepdev/connect.c | 178 ++ sheepdev/device.c | 985 As a general rule, if the proper code of one functions exceed 60 lines, try hard to rethink

[sheepdog] [PATCH v3 4/4] sheepdev: add kbuild and Makefile.in

2013-01-21 Thread levin li
From: levin li Signed-off-by: levin li --- Makefile.am | 4 configure.ac | 9 + sheepdev/Kbuild | 13 + sheepdev/Makefile.in | 8 4 files changed, 34 insertions(+) create mode 100644 sheepdev/Kbuild create mode 100644 sheepdev

[sheepdog] [PATCH v3 2/4] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-21 Thread levin li
From: levin li This module provides function for users to take sheepdog VDIs as block devices in linux, users can register a VDI to its kernel space, it just like that a new hard disk is added to the computer, users can create partitions for the disk, format the disk or mount the disk, it

[sheepdog] [PATCH v3 3/4] sheepdev: make it safe to open a snapshot VDI

2013-01-21 Thread levin li
From: levin li In case we open a snapshot VDI to make it as a block device, we immediately create a new writable VDI for this snapshot just as what QEMU does, then we can safely write to this VDI Signed-off-by: levin li --- sheepdev/sheep.c | 60

[sheepdog] [PATCH v3 1/4] sheep: modify sheepdog_proto.h to make it usable in linux kernel module

2013-01-21 Thread levin li
From: levin li Signed-off-by: levin li --- include/sheepdog_proto.h | 11 --- 1 file changed, 8 insertions(+), 3 deletions(-) diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h index 865f451..fb35f3a 100644 --- a/include/sheepdog_proto.h +++ b/include/sheepdog_proto.h

[sheepdog] [PATCH v3 0/4] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-21 Thread levin li
From: levin li v2 -- > v3: 1. Make it Ok for QEMU to run VDIs from the driver 2. Code refactor 3. Use the format 'del vdiname:snap_tag[snap_id]' to delete a device v1 -- > v2: 1. Rename module.c to device.c 2. Split proc operations from device.c to proc.c 3. Remove sheep

[sheepdog] [PATCH v2 4/4] sheepdev: add kbuild and Makefile.in

2013-01-06 Thread levin li
From: levin li Signed-off-by: levin li --- Makefile.am |4 configure.ac |9 + sheepdev/Kbuild | 13 + sheepdev/Makefile.in |8 4 files changed, 34 insertions(+), 0 deletions(-) create mode 100644 sheepdev/Kbuild create

[sheepdog] [PATCH v2 2/4] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-06 Thread levin li
From: levin li This module provides function for users to take sheepdog VDIs as block devices in linux, users can register a VDI to its kernel space, it just like that a new hard disk is added to the computer, users can create partitions for the disk, format the disk or mount the disk, it

[sheepdog] [PATCH v2 3/4] sheepdev: make it safe to open a snapshot VDI

2013-01-06 Thread levin li
From: levin li In case we open a snapshot VDI to make it as a block device, we immediately create a new writable VDI for this snapshot just as what QEMU does, then we can safely write to this VDI Signed-off-by: levin li --- sheepdev/sheep.c | 59

[sheepdog] [PATCH v2 1/4] sheep: modify sheepdog_proto.h to make it usable in linux kernel module

2013-01-06 Thread levin li
From: levin li Signed-off-by: levin li --- include/sheepdog_proto.h | 11 --- 1 files changed, 8 insertions(+), 3 deletions(-) diff --git a/include/sheepdog_proto.h b/include/sheepdog_proto.h index 865f451..fb35f3a 100644 --- a/include/sheepdog_proto.h +++ b/include

[sheepdog] [PATCH v2 0/4] sheepdev: linux kernel module of block device driver for sheepdog

2013-01-06 Thread levin li
From: levin li v1 -- > v2: 1. Rename module.c to device.c 2. Split proc operations from device.c to proc.c 3. Remove sheepdog_proto.h from sheepdev, use the one in include directory 4. Supprt write snapshot VDI 5. Kill the compile error in linux kernel 3.7 6. Modify some functions

Re: [sheepdog] [PATCH RFC 1/2] sheepdev: linux kernel module of block device driver for sheepdog

2012-12-27 Thread levin li
On 12/27/2012 11:57 PM, Hitoshi Mitake wrote: > At Thu, 27 Dec 2012 22:00:41 +0800, > levin li wrote: >> >> From: levin li >> >> This module provides function for users to take sheepdog VDIs as block >> devices in linux, users can register a VDI to its kern

[sheepdog] [PATCH RFC 2/2] sheepdev: add kbuild and Makefile.in

2012-12-27 Thread levin li
From: levin li Signed-off-by: levin li --- Makefile.am | 4 configure.ac | 9 + sheepdev/Kbuild | 12 sheepdev/Makefile.in | 8 4 files changed, 33 insertions(+) create mode 100644 sheepdev/Kbuild create mode 100644 sheepdev

[sheepdog] [PATCH RFC 1/2] sheepdev: linux kernel module of block device driver for sheepdog

2012-12-27 Thread levin li
From: levin li This module provides function for users to take sheepdog VDIs as block devices in linux, users can register a VDI to its kernel space, it just like that a new hard disk is added to the computer, users can create partitions for the disk, format the disk or mount the disk, it

[sheepdog] [PATCH RFC 0/2] sheepdev: linux kernel module of block device driver for sheepdog

2012-12-27 Thread levin li
From: levin li This module provides function for users to take sheepdog VDIs as block devices in linux, users can register a VDI to its kernel space, it just like that a new hard disk is added to the computer, users can create partitions for the disk, format the disk or mount the disk, it

[sheepdog] [PATCH UPDATE] collie: add 'collie vdi cache flush' to flush dirty cache

2012-12-17 Thread levin li
From: levin li This command can flush cache of an VDI in specified node, which is useful when QEMU terminates accidentally, left the dirty cache data unflushed, this command helps us to flush and delete this cache into the cluster. Signed-off-by: levin li --- collie/vdi.c | 51

[sheepdog] [PATCH 2/2] collie: add 'collie vdi cache delete' to delete cache

2012-12-15 Thread levin li
From: levin li When VMs are migrated or terminated accidentally, the cache of the VMs won't be deleted at once, by this command we can delete the cache of an specified VDI in all nodes of the cluster. Signed-off-by: levin li --- collie/vdi.c |

[sheepdog] [PATCH 1/2] collie: add 'collie vdi cache flush' to flush dirty cache

2012-12-15 Thread levin li
From: levin li This command can flush cache of an VDI in specified node, which is useful when QEMU terminates accidentally, left the dirty cache data unflushed, this command helps us to flush and delete this cache into the cluster. Signed-off-by: levin li --- collie/vdi.c | 51

[sheepdog] [PATCH] collie: fix collie to pass test 042

2012-12-05 Thread levin li
From: levin li We use GET_VDI_COPIES to get vdi list before in parse_vdi() before, since we send vdi copies number before vdi creation now, then we should not get vdi list by GET_VDI_COPIES, but to use READ_VDIS. Signed-off-by: levin li --- collie/common.c | 30

[sheepdog] [PATCH UPDATE] sheep: no need to wait for nr_objs of an object

2012-11-29 Thread levin li
From: levin li Since we've notified the nr_objs of a newly creately VDI before the VDI objects are created, we can always get the nr_objs by get_obj_copy_number(), I leave the error message undeleted for later debug in case there's an error we haven't noticed. Signed-o

Re: [sheepdog] [PATCH] sheep: no need to wait for nr_objs of an object

2012-11-29 Thread levin li
On 2012年11月29日 17:24, Liu Yuan wrote: On 11/29/2012 05:17 PM, levin li wrote: + dprintf("ERROR: can not find copy number for object %" + PRIx64 "\n", oids[i]); maybe use eprintf is better which will show us mess

[sheepdog] [PATCH] sheep: no need to wait for nr_objs of an object

2012-11-29 Thread levin li
From: levin li Since we've notified the nr_objs of a newly creately VDI before the VDI objects are created, we can always get the nr_objs by get_obj_copy_number(), I leave the error message undeleted for later debug in case there's an error we haven't noticed. Signed-o

[sheepdog] [PATCH] sheep: notify vdi copies number before we create the vdi

2012-11-27 Thread levin li
From: levin li We can not notify the vdi copies number to the cluster after creating the vdi, because if a recovery happens in the time window after cluster_vdi_add() finishes, and just before post_cluster_vdi_add() notifies the copies number of the new vdi to all the nodes in the cluster, it

Re: [sheepdog] [PATCH] sheep: fix a dead-lock bug in screen_object_list()

2012-11-25 Thread levin li
On 2012年11月26日 13:55, Liu Yuan wrote: On 11/26/2012 01:48 PM, levin li wrote: screen_object_list() is called in recovery, if a VDI creation is in progress, then the VDI creation operation is blocked by gateway to wait for completion of recovery, and if we make screen_object_list() to retry

[sheepdog] [PATCH] sheep: fix a dead-lock bug in screen_object_list()

2012-11-25 Thread levin li
From: levin li The bug is introduced in the patch 97ccd87ea15e606b6ec9fecb54f5de453f9c5c1f which causes the cluster hangs in recovery, this patch is like a revert patch except that I add comment to explain why we should not ignore the objects whose nr_objs are zero in screen_object_list

[sheepdog] [PATCH UPDATE] sheep: read inode directly from backend in VDI deletion work

2012-09-05 Thread levin li
From: levin li This patch is preparation for next patch that deletes object cache when deleting VDI, if object cache deletion work is processed in notify, then it may causes race between cache deletion and next VDI deletion work, because VDI deletion work may needs to read its parent VDI inode

[sheepdog] [PATCH 2/2] object cache: try to delete cache entris in every node when deleting a VDI

2012-09-05 Thread levin li
From: levin li Every node that starts VM in writeback mode can create cache entries, so we can not only remove cache entries in the node that executes the VDI deletion work, but should notify to try to delete cache entries on every node Signed-off-by: levin li --- sheep/object_cache.c |2

[sheepdog] [PATCH 1/2] sheep: read inode directly from backend in VDI deletion work

2012-09-05 Thread levin li
From: levin li This patch is preparation for next patch that deletes object cache when deleting VDI, if object cache deletion work is processed in notify, then it may causes race between cache deletion and next VDI deletion work, because VDI deletion work may needs to read its parent VDI inode

[sheepdog] [PATCH 0/2] object cache: try to delete cache entris in every node when deleting a VDI

2012-09-05 Thread levin li
From: levin li levin li (2): sheep: read inode directly from backend in VDI deletion work object cache: try to delete cache entris in every node when deleting a VDI sheep/object_cache.c |2 +- sheep/ops.c | 45 ++--- sheep

[sheepdog] [PATCH] object cache: fix crash when deleting object cache

2012-09-04 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/object_cache.c |2 ++ 1 files changed, 2 insertions(+), 0 deletions(-) diff --git a/sheep/object_cache.c b/sheep/object_cache.c index 431fe26..5bd7060 100644 --- a/sheep/object_cache.c +++ b/sheep/object_cache.c @@ -602,6 +602,7

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-31 Thread levin li
On 08/31/2012 05:37 PM, MORITA Kazutaka wrote: > At Fri, 31 Aug 2012 17:09:22 +0800, > levin li wrote: >> >> >> I still don't understand what's the use of your rollback of a snapshot, it >> seems > > A rollback operation is supported by many syste

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-31 Thread levin li
On 08/31/2012 04:25 PM, MORITA Kazutaka wrote: > Here is a draft patch which implements my example but doesn't support > resetting clone vdis. I hope it would help you. > > == > commit 138511431d4f87642572b0e2a51d869240b5ac83 > Author: MORITA Kazutaka > Date: Fri Aug 31 17:21:38 2012 +0900 >

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-31 Thread levin li
On 08/31/2012 03:03 PM, MORITA Kazutaka wrote: > At Fri, 31 Aug 2012 14:59:43 +0800, > levin li wrote: >> >> On 08/31/2012 02:38 PM, MORITA Kazutaka wrote: >>> At Fri, 31 Aug 2012 14:25:04 +0800, >>> levin li wrote: >>>> >>>> On 08/31/201

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-30 Thread levin li
On 08/31/2012 02:38 PM, MORITA Kazutaka wrote: > At Fri, 31 Aug 2012 14:25:04 +0800, > levin li wrote: >> >> On 08/31/2012 02:10 PM, MORITA Kazutaka wrote: >>> At Fri, 31 Aug 2012 13:53:29 +0800, >>> levin li wrote: >>>> >>>> On 08/31/201

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-30 Thread levin li
On 08/31/2012 02:10 PM, MORITA Kazutaka wrote: > At Fri, 31 Aug 2012 13:53:29 +0800, > levin li wrote: >> >> On 08/31/2012 12:17 PM, MORITA Kazutaka wrote: >>> At Thu, 30 Aug 2012 13:59:54 +0800, >>> levin li wrote: >>>> >>>> From: levin

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-30 Thread levin li
On 08/31/2012 01:55 PM, Liu Yuan wrote: > On 08/31/2012 01:53 PM, levin li wrote: >> Maybe I misunderstood you, but this patch can do more than you expected, >> it's no harm for users to rollback a VDI to any snapshot if they choose to. > > So this implementation is w

Re: [sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-30 Thread levin li
On 08/31/2012 12:17 PM, MORITA Kazutaka wrote: > At Thu, 30 Aug 2012 13:59:54 +0800, > levin li wrote: >> >> From: levin li >> >> With this command we can rollback a cloned VDI to any specified >> snapshot, or easily reset to its parent VDI, the usage is ea

[sheepdog] [PATCH 2/2] sheep: add VDI rollback function support

2012-08-29 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/ops.c| 13 sheep/sheep_priv.h |1 + sheep/vdi.c| 83 3 files changed, 97 insertions(+), 0 deletions(-) diff --git a/sheep/ops.c b/sheep/ops.c index ccb1c5e

[sheepdog] [PATCH 1/2] collie: add command 'collie vdi rollback'

2012-08-29 Thread levin li
From: levin li With this command we can rollback a cloned VDI to any specified snapshot, or easily reset to its parent VDI, the usage is easy. To rollback to a specified snapshot: $ collie vdi rollback -s 1 base_vdi clone_vdi To reset to its parent: $ collie vdi rollback clone_vdi Signed

[sheepdog] [PATCH] collie: don't fetch node list if not use

2012-08-29 Thread levin li
From: levin li Signed-off-by: levin li --- collie/cluster.c |5 + collie/common.c |3 --- collie/node.c|4 ++-- collie/vdi.c | 27 +-- 4 files changed, 16 insertions(+), 23 deletions(-) diff --git a/collie/cluster.c b/collie/cluster.c index

Re: [sheepdog] [PATCH] collie: add command 'collie vdi reset' to reset a cloned VDI

2012-08-29 Thread levin li
On 2012年08月29日 15:27, MORITA Kazutaka wrote: > At Wed, 29 Aug 2012 15:18:12 +0800, > levin li wrote: >> >> On 2012年08月29日 15:08, MORITA Kazutaka wrote: >>> At Wed, 29 Aug 2012 13:57:57 +0800, >>> levin li wrote: >>>> >>>> From: levin l

[sheepdog] [PATCH UPDATE] sheep: initialize reclaim work queue only when cache is enabled

2012-08-29 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/sheep.c |8 ++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/sheep/sheep.c b/sheep/sheep.c index ea25a05..36e059a 100644 --- a/sheep/sheep.c +++ b/sheep/sheep.c @@ -373,9 +373,13 @@ int main(int argc, char **argv

Re: [sheepdog] [PATCH] collie: add command 'collie vdi reset' to reset a cloned VDI

2012-08-29 Thread levin li
On 2012年08月29日 15:08, MORITA Kazutaka wrote: > At Wed, 29 Aug 2012 13:57:57 +0800, > levin li wrote: >> >> From: levin li >> >> With this command, we can reset a cloned VDI to its initial state as >> it's just cloned from a snapshot, the usage is simple:

Re: [sheepdog] [PATCH] sheep: initialize recovery work queue only when cache is enabled

2012-08-29 Thread levin li
On 2012年08月29日 15:10, MORITA Kazutaka wrote: > At Wed, 29 Aug 2012 14:47:57 +0800, > levin li wrote: >> >> From: levin li >> >> >> Signed-off-by: levin li >> --- >> sheep/sheep.c |8 ++-- >> 1 files changed, 6 insertions(+), 2 de

[sheepdog] [PATCH] sheep: initialize recovery work queue only when cache is enabled

2012-08-28 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/sheep.c |8 ++-- 1 files changed, 6 insertions(+), 2 deletions(-) diff --git a/sheep/sheep.c b/sheep/sheep.c index ea25a05..b23a312 100644 --- a/sheep/sheep.c +++ b/sheep/sheep.c @@ -373,9 +373,13 @@ int main(int argc, char **argv

[sheepdog] [PATCH] sheep: use short thread for recovery

2012-08-28 Thread levin li
From: levin li Most time sheep isn't in recovery state, so the two recovery work queue recovery_wqueue/recovery_notify_wqueue stay there useless, since we have short thread, we can save resource by creating the thread when recovery needs Signed-off-by: levin li --- sheep/recovery.c |

[sheepdog] [PATCH] collie: add command 'collie vdi reset' to reset a cloned VDI

2012-08-28 Thread levin li
From: levin li With this command, we can reset a cloned VDI to its initial state as it's just cloned from a snapshot, the usage is simple: $ collie vdi reset cloned_vdi_name Signed-off-by: levin li --- collie/collie.h |1 + collie/common.c | 35 ++ c

Re: [sheepdog] [PATCH] sheep: no need to set SD_FLAG_CMD_WRITE for local request

2012-08-28 Thread levin li
On 08/28/2012 05:41 PM, MORITA Kazutaka wrote: > At Tue, 28 Aug 2012 16:43:50 +0800, > levin li wrote: >> >> On 08/28/2012 04:38 PM, Liu Yuan wrote: >>> On 08/28/2012 04:14 PM, levin li wrote: >>>> - hdr.flags = SD_FLAG_CMD_WRITE; >>>>hdr.da

Re: [sheepdog] [PATCH] sheep: no need to set SD_FLAG_CMD_WRITE for local request

2012-08-28 Thread levin li
On 08/28/2012 04:38 PM, Liu Yuan wrote: > On 08/28/2012 04:14 PM, levin li wrote: >> -hdr.flags = SD_FLAG_CMD_WRITE; >> hdr.data_length = sizeof(sys->this_node); > > Why not? We do write &sys->this_node as data to transfer. > > Thanks, > Yu

[sheepdog] [PATCH] sheep: no need to set SD_FLAG_CMD_WRITE for local request

2012-08-28 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/recovery.c |1 - 1 file changed, 1 deletion(-) diff --git a/sheep/recovery.c b/sheep/recovery.c index 59ac9d6..0033c6b 100644 --- a/sheep/recovery.c +++ b/sheep/recovery.c @@ -346,7 +346,6 @@ static void notify_recovery_completion_work

[sheepdog] [PATCH] collie: fix error output of 'collie node info'

2012-08-27 Thread levin li
From: levin li Signed-off-by: levin li --- collie/node.c |1 + sheep/ops.c |5 - 2 files changed, 5 insertions(+), 1 deletions(-) diff --git a/collie/node.c b/collie/node.c index a5a57a3..63bb27a 100644 --- a/collie/node.c +++ b/collie/node.c @@ -79,6 +79,7 @@ static int

[sheepdog] [PATCH] sheep: fix a bug that hangs the cluster during recovery

2012-08-27 Thread levin li
From: levin li During recovery, a VDI creation request may waits for recovery to complete, and VDI creation request is a cluster request which prevent other cluster requests being processed, when recovery comes to notify_recovery_completion_work, it issues another cluster request with

[sheepdog] [PATCH v3 2/2] collie: make `collie vdi list` not read vdi_inuse bitmap

2012-08-24 Thread levin li
From: levin li Signed-off-by: levin li --- collie/common.c| 32 include/sheep.h|5 + sheep/sheep_priv.h |5 - 3 files changed, 21 insertions(+), 21 deletions(-) diff --git a/collie/common.c b/collie/common.c index 364e36c..ce8dcf7

[sheepdog] [PATCH v3 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-24 Thread levin li
From: levin li Since we already send vdi copy list in get_vdis(), there's no need to send the vdi_inuse bitmap any more, without it we can send less data as the length of vdi copy list is variable and it contains the vid just as vdi_inuse does. Signed-off-by: levin li --- in

Re: [sheepdog] [PATCH v2 2/2] collie: make `collie vdi list` not read vdi_inuse bitmap

2012-08-23 Thread levin li
On 2012年08月24日 14:41, MORITA Kazutaka wrote: > At Fri, 24 Aug 2012 14:30:42 +0800, > levin li wrote: >> >> I think SD_NR_VDIS is big enough, it can contain up to 2097152 >> vdi copy entries, 2097152 VDIs are more than enough for our cluster, >> and by this we don'

Re: [sheepdog] [PATCH v2 2/2] collie: make `collie vdi list` not read vdi_inuse bitmap

2012-08-23 Thread levin li
On 2012年08月24日 14:11, MORITA Kazutaka wrote: > At Fri, 24 Aug 2012 13:36:47 +0800, > levin li wrote: >> >> From: levin li >> >> >> Signed-off-by: levin li >> --- >> collie/common.c| 32 >> include

Re: [sheepdog] [PATCH v2 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-23 Thread levin li
On 2012年08月24日 14:09, MORITA Kazutaka wrote: > At Fri, 24 Aug 2012 13:36:46 +0800, > levin li wrote: >> >> From: levin li >> >> Since we already send vdi copy list in get_vdis(), there's no >> need to send the vdi_inuse bitmap any more, without it we c

[sheepdog] [PATCH v2 2/2] collie: make `collie vdi list` not read vdi_inuse bitmap

2012-08-23 Thread levin li
From: levin li Signed-off-by: levin li --- collie/common.c| 32 include/sheep.h|5 + sheep/sheep_priv.h |5 - 3 files changed, 21 insertions(+), 21 deletions(-) diff --git a/collie/common.c b/collie/common.c index 364e36c..c03cab7

[sheepdog] [PATCH v2 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-23 Thread levin li
From: levin li Since we already send vdi copy list in get_vdis(), there's no need to send the vdi_inuse bitmap any more, without it we can send less data as the length of vdi copy list is variable and it contains the vid just as vdi_inuse does. Signed-off-by: levin li --- in

Re: [sheepdog] [PATCH 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-23 Thread levin li
On 2012年08月24日 12:38, MORITA Kazutaka wrote: > At Fri, 24 Aug 2012 10:56:35 +0800, > levin li wrote: >> >> From: levin li >> >> Since we already send vdi copy list in get_vdis(), there's no >> need to send the vdi_inuse bitmap any more, without it we c

[sheepdog] [PATCH 2/2] collie: make `collie vdi list` not read vdi_inuse bitmap

2012-08-23 Thread levin li
From: levin li Signed-off-by: levin li --- collie/common.c| 30 +++--- include/sheep.h|5 + sheep/sheep_priv.h |5 - 3 files changed, 20 insertions(+), 20 deletions(-) diff --git a/collie/common.c b/collie/common.c index 364e36c..6e047a3

[sheepdog] [PATCH 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-23 Thread levin li
From: levin li Since we already send vdi copy list in get_vdis(), there's no need to send the vdi_inuse bitmap any more, without it we can send less data as the length of vdi copy list is variable and it contains the vid just as vdi_inuse does. Signed-off-by: levin li --- sheep/gr

[sheepdog] [PATCH 3/3] test: add a test case for different VDI redundancy level

2012-08-23 Thread levin li
From: levin li Signed-off-by: levin li --- tests/029 | 52 tests/029.out | 33 + tests/group |1 + 3 files changed, 86 insertions(+), 0 deletions(-) create mode 100755 tests/029 create mode

[sheepdog] [PATCH 2/3] sheep: fix cloning VDI fail when cloned VDI has greater nr_copies than the base

2012-08-23 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/vdi.c | 10 -- 1 files changed, 4 insertions(+), 6 deletions(-) diff --git a/sheep/vdi.c b/sheep/vdi.c index 5c9bf68..aa5134b 100644 --- a/sheep/vdi.c +++ b/sheep/vdi.c @@ -225,7 +225,7 @@ static int create_vdi_obj(struct vdi_iocb

[sheepdog] [PATCH 1/3] sheep: return the copy number of the request object in peer_read_obj

2012-08-23 Thread levin li
From: levin li collie uses this return value in the output of 'collie vdi object', without this, we can not pass test 028 Signed-off-by: levin li --- sheep/ops.c |4 1 files changed, 4 insertions(+), 0 deletions(-) diff --git a/sheep/ops.c b/sheep/ops.c index 42b99ef..aaa4

[sheepdog] [PATCH v6 8/8] collie: show copy number in the output of 'collie vdi list'

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- collie/vdi.c | 10 +- tests/016.out |2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/collie/vdi.c b/collie/vdi.c index 52015ed..b354619 100644 --- a/collie/vdi.c +++ b/collie/vdi.c @@ -112,14 +112,14 @@ static void

[sheepdog] [PATCH v6 7/8] sheep: remove some unused copies related functions

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/group.c | 36 +++- sheep/ops.c |3 +-- 2 files changed, 12 insertions(+), 27 deletions(-) diff --git a/sheep/group.c b/sheep/group.c index 11ea5ff..4510ab7 100644 --- a/sheep/group.c +++ b/sheep/group.c

[sheepdog] [PATCH v6 6/8] sheep: use the specified copies number for IO requests

2012-08-22 Thread levin li
From: levin li In gateway_read{write,create_and_write}_obj, and read{write}_object, we should use different copies number for different requests and objects, instead of using the global copies number sys->nr_copies or calculated from vnodes Signed-off-by: levin li --- sheep/farm/trun

[sheepdog] [PATCH v6 5/8] sheep: fetch vdi copy list after sheep joins the cluster

2012-08-22 Thread levin li
From: levin li The new joined node doesn't have the vdi copy list, or have incomplete vdi copy list, so we need to fetch the copy list data from other nodes Signed-off-by: levin li --- collie/common.c| 15 +++-- sheep/farm/farm.c | 31 +++- sheep/gr

[sheepdog] [PATCH v6 4/8] sheep: add vdi_copy tree to store copies number for every VDI

2012-08-22 Thread levin li
From: levin li The normal requests from QEMU include the copies number in its header, QEMU stores the nr_copies the first time it reads the inode data, but many other local requests such as read_copy_from_replica and recover_object_from_replica don't know the copies number for every objec

[sheepdog] [PATCH v6 3/8] sheep: rename ctime to create_time in vdi.c

2012-08-22 Thread levin li
From: levin li 'ctime' shadows a variable in pthread.h, in order to use pthread in vdi.c, so we need to rename ctime to get rid of the compile warning Signed-off-by: levin li --- collie/vdi.c |6 +++--- include/sheepdog_proto.h |2 +- sheep/vdi.c

[sheepdog] [PATCH v6 2/8] collie: add -c for 'collie vdi create' to specify redundancy level for per-vdi

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- collie/vdi.c | 25 +++-- sheep/ops.c |9 ++--- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/collie/vdi.c b/collie/vdi.c index 5781e75..d8fe029 100644 --- a/collie/vdi.c +++ b/collie/vdi.c @@ -23,6

[sheepdog] [PATCH v6 1/8] sheep: use struct vdi_iocb to simplify the vdi_create api

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/ops.c| 12 -- sheep/sheep_priv.h | 12 +- sheep/vdi.c| 55 +-- 3 files changed, 46 insertions(+), 33 deletions(-) diff --git a/sheep/ops.c b/sheep/ops.c

[sheepdog] [PATCH v6 0/8] different redundancy level for different VDIs

2012-08-22 Thread levin li
From: levin li v5 ---> v6 1. rename add_vdi_copies to add_vdi_copy_number 2. remove the state of recovery waiting for copy list 3. fix a bug that loses inode data in recovery - With this patch set, we don't need to specify copies number when forma

[sheepdog] [PATCH v5 8/8] collie: show copy number in the output of 'collie vdi list'

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- collie/vdi.c | 10 +- tests/016.out |2 +- 2 files changed, 6 insertions(+), 6 deletions(-) diff --git a/collie/vdi.c b/collie/vdi.c index 52015ed..b354619 100644 --- a/collie/vdi.c +++ b/collie/vdi.c @@ -112,14 +112,14 @@ static void

[sheepdog] [PATCH v5 7/8] sheep: remove some unused copies related functions

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/group.c | 36 +++- sheep/ops.c |3 +-- 2 files changed, 12 insertions(+), 27 deletions(-) diff --git a/sheep/group.c b/sheep/group.c index e935218..c02b1fe 100644 --- a/sheep/group.c +++ b/sheep/group.c

[sheepdog] [PATCH v5 6/8] sheep: use the specified copies number for IO requests

2012-08-22 Thread levin li
From: levin li In gateway_read{write,create_and_write}_obj, and read{write}_object, we should use different copies number for different requests and objects, instead of using the global copies number sys->nr_copies or calculated from vnodes Signed-off-by: levin li --- sheep/farm/trun

[sheepdog] [PATCH v5 5/8] sheep: fetch vdi copy list after sheep joins the cluster

2012-08-22 Thread levin li
From: levin li The new joined node doesn't have the vdi copy list, or have incomplete vdi copy list, so we need to fetch the copy list data from other nodes Signed-off-by: levin li --- collie/common.c| 15 +-- sheep/farm/farm.c | 31 +++- sheep/group.c |

[sheepdog] [PATCH v5 4/8] sheep: add vdi_copy tree to store copies number for every VDI

2012-08-22 Thread levin li
From: levin li The normal requests from QEMU include the copies number in its header, QEMU stores the nr_copies the first time it reads the inode data, but many other local requests such as read_copy_from_replica and recover_object_from_replica don't know the copies number for every objec

[sheepdog] [PATCH v5 2/8] collie: add -c for 'collie vdi create' to specify redundancy level for per-vdi

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- collie/vdi.c | 25 +++-- sheep/ops.c |9 ++--- 2 files changed, 25 insertions(+), 9 deletions(-) diff --git a/collie/vdi.c b/collie/vdi.c index 5781e75..d8fe029 100644 --- a/collie/vdi.c +++ b/collie/vdi.c @@ -23,6

[sheepdog] [PATCH v5 3/8] sheep: rename ctime to create_time in vdi.c

2012-08-22 Thread levin li
From: levin li 'ctime' shadows a variable in pthread.h, in order to use pthread in vdi.c, so we need to rename ctime to get rid of the compile warning Signed-off-by: levin li --- collie/vdi.c |6 +++--- include/sheepdog_proto.h |2 +- sheep/vdi.c

[sheepdog] [PATCH v5 1/8] sheep: use struct vdi_iocb to simplify the vdi_create api

2012-08-22 Thread levin li
From: levin li Signed-off-by: levin li --- sheep/ops.c| 12 -- sheep/sheep_priv.h | 12 +- sheep/vdi.c| 55 +-- 3 files changed, 46 insertions(+), 33 deletions(-) diff --git a/sheep/ops.c b/sheep/ops.c

  1   2   3   4   5   >