Re: [sheepdog] [PATCH v2] sheep/recovery: multi-threading recovery process

On Wed, Jan 29, 2014 at 03:32:34PM +0800, Liu Yuan wrote: > On Wed, Jan 29, 2014 at 04:28:35PM +0900, Hitoshi Mitake wrote: > > At Tue, 28 Jan 2014 18:01:42 +0800, > > Liu Yuan wrote: > > > > > > Rationale for multi-threaded recovery: > > > > > > 1. If one node is added, we find that all the VMs

[sheepdog] [PATCH] dog: print new VID of current VDI when snapshot is created

Especially new VID of current VDI is useful information. So this patch lets dog print both of the IDs. Signed-off-by: Hitoshi Mitake --- dog/vdi.c |9 + 1 file changed, 5 insertions(+), 4 deletions(-) diff --git a/dog/vdi.c b/dog/vdi.c index 1e175b3..9df0985 100644 --- a/dog/vdi.c +

[sheepdog] [PATCH v2 10/12] sheep/nfs: add mkdir support

Now we support 'mkdir' command and gain hierarchical structure support. TODO: - make dentry operation operate on top of 'file' operation - add hash or btree based dentries management Signed-off-by: Liu Yuan --- sheep/nfs/fs.c | 41 ++--- sheep/nfs/fs.h | 1 + sheep/

[sheepdog] [PATCH v2 11/12] sheep: make stat_data_objs a generic function

and rename it as sd_inode_stat() Signed-off-by: Liu Yuan --- dog/vdi.c| 96 +--- include/internal_proto.h | 3 ++ lib/sd_inode.c | 95 +++ 3 files changed, 99 insertions(+), 95 dele

[sheepdog] [PATCH v2 09/12] sheep/nfs: implement read/write nfs operation

Now we support more bash command such as cp, mv, echo, cat ... Signed-off-by: Liu Yuan --- include/util.h | 8 + sheep/Makefile.am | 5 +++ sheep/nfs/fs.c | 41 + sheep/nfs/fs.h | 2 ++ sheep/nfs/nfs.c | 87 +

[sheepdog] [PATCH v2 12/12] sheep/nfs: implement nfs3_fsstat operation

Now we support 'df' command to get file system stats. Signed-off-by: Liu Yuan --- sheep/nfs/nfs.c | 39 +-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/sheep/nfs/nfs.c b/sheep/nfs/nfs.c index 8d0e11f..afc0cf8 100644 --- a/sheep/nfs/nfs.c +++

[sheepdog] [PATCH v2 08/12] sheep/nfs: support lookup/create/setattr nfs operation

Now we support 'touch' command. Signed-off-by: Liu Yuan --- sheep/nfs/fs.c | 51 + sheep/nfs/fs.h | 2 + sheep/nfs/nfs.c | 140 ++-- sheep/nfs/nfs.h | 26 +++ sheep/nfs/xdr.c | 12 ++--- 5 files changed, 202 i

[sheepdog] [PATCH v2 06/12] sheep/nfs: add basic file system framework

Add basic inode, dentry structures and just support 'ls' /mntpoint operation. For inode management, we use the hash scheme as in http implementation. This will save us a lots of code on inode allocation and deallocation and management. We also make use the simiar concepts found in traditional UNI

[sheepdog] [PATCH v2 05/12] util: add PTR_ERR helpers

Sometimes we need to return ERROR code from pointers Signed-off-by: Liu Yuan --- include/compiler.h | 2 ++ include/util.h | 30 ++ 2 files changed, 32 insertions(+) diff --git a/include/compiler.h b/include/compiler.h index 4a59b92..2fbd2a4 100644 --- a/include

[sheepdog] [PATCH v2 07/12] sheep: clean up sd_inode btree error code

These error codes are used internally, not for client communication. Signed-off-by: Liu Yuan --- include/internal_proto.h | 1 + include/sheepdog_proto.h | 3 --- lib/sd_inode.c | 22 +++--- 3 files changed, 12 insertions(+), 14 deletions(-) diff --git a/include/inte

[sheepdog] [PATCH v2 01/12] sheep: fix some compile errors

Both MIN/MAX and roundup are provided by glibc and would cause compile error when we #include . Just use GNU version. Signed-off-by: Liu Yuan --- include/util.h | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/include/util.h b/include/util.h index 83f3c08..1932fe3 100644

[sheepdog] [PATCH v2 04/12] dog: add nfs command

We support allocate as many nfs backend storage as you want. usage: dog nfs create $name # create a nfs backend named $name dog nfs delete $name # delete a nfs backend named $name $name is the name for users to mount at client in $IP:$NAME Signed-off-by: Liu Yuan --- dog/Makefile.am

[sheepdog] [PATCH v2 03/12] sheep/nfs: add mount protocol stub

This is a preparation patch Signed-off-by: Liu Yuan --- sheep/Makefile.am | 2 +- sheep/nfs/mount.c | 42 +++ sheep/nfs/nfs.h | 99 sheep/nfs/nfsd.c | 75 + sheep/nfs/xdr.c | 122 ++

[sheepdog] [PATCH v2 00/12] introduce basic NFS server support

v2: Address Hithoshi's comment - add NFS macro to sheep/ops.c - add -lrt - move sd_inode_stat into lib/sd_inode.c This patch set mainly introduce NFS server building blocks, that is - NFS v3 transport by glibc's SUNRPC - preliminary NFS server NFS protocol is seen as the most established sha

Re: [sheepdog] [PATCH v2] sheep/recovery: multi-threading recovery process

On Wed, Jan 29, 2014 at 04:28:35PM +0900, Hitoshi Mitake wrote: > At Tue, 28 Jan 2014 18:01:42 +0800, > Liu Yuan wrote: > > > > Rationale for multi-threaded recovery: > > > > 1. If one node is added, we find that all the VMs on other nodes will get > >noticeably affected until 50% data is tra

Re: [sheepdog] [PATCH v2] sheep/recovery: multi-threading recovery process

At Tue, 28 Jan 2014 18:01:42 +0800, Liu Yuan wrote: > > Rationale for multi-threaded recovery: > > 1. If one node is added, we find that all the VMs on other nodes will get >noticeably affected until 50% data is transferred to the new node. > > 2. For node failure, we might not have problems

Re: [sheepdog] [PATCH 00/12] introduce basic NFS server support

At Wed, 29 Jan 2014 14:17:27 +0800, Liu Yuan wrote: > > On Wed, Jan 29, 2014 at 11:41:59AM +0900, Hitoshi Mitake wrote: > > At Wed, 29 Jan 2014 04:19:00 +0800, > > Liu Yuan wrote: > > > > > > This patch set mainly introduce NFS server building blocks, that is > > > > > > - NFS v3 transport by gl

Re: [sheepdog] [PATCH 00/12] introduce basic NFS server support

On Wed, Jan 29, 2014 at 11:41:59AM +0900, Hitoshi Mitake wrote: > At Wed, 29 Jan 2014 04:19:00 +0800, > Liu Yuan wrote: > > > > This patch set mainly introduce NFS server building blocks, that is > > > > - NFS v3 transport by glibc's SUNRPC > > - preliminary NFS server > > > > NFS protocol is se

Re: [sheepdog] [PATCH 2/2] sheep: cleaning vdi deletion process, round 2

At Tue, 28 Jan 2014 19:30:39 +0900, Hitoshi Mitake wrote: > > At Tue, 28 Jan 2014 17:04:14 +0900, > MORITA Kazutaka wrote: > > > > At Tue, 28 Jan 2014 15:26:52 +0900, > > Hitoshi Mitake wrote: > > > > > > > > > > > Only Kazutaka knows the reason. I guess it was planned to be paralle. I > > > >

Re: [sheepdog] [PATCH stable-0.7 3/3] logger: add a unified log destination option and syslog support

At Tue, 28 Jan 2014 16:53:56 +0900, MORITA Kazutaka wrote: > > At Tue, 28 Jan 2014 11:34:22 +0800, > Liu Yuan wrote: > > > > On Mon, Jan 27, 2014 at 07:54:35PM +0900, Hitoshi Mitake wrote: > > > This patch adds a new parameter "dst=" to the "-l" option for > > > specifying logging destination. Po

Re: [sheepdog] [PATCH 00/12] introduce basic NFS server support

At Wed, 29 Jan 2014 04:19:00 +0800, Liu Yuan wrote: > > This patch set mainly introduce NFS server building blocks, that is > > - NFS v3 transport by glibc's SUNRPC > - preliminary NFS server > > NFS protocol is seen as the most established shared storage protocol to > provide > POSIX file sema

Re: [sheepdog] [PATCH 04/12] dog: add nfs command

At Wed, 29 Jan 2014 04:19:04 +0800, Liu Yuan wrote: > diff --git a/sheep/ops.c b/sheep/ops.c > index 1e9bc1e..ad8847a 100644 > --- a/sheep/ops.c > +++ b/sheep/ops.c > @@ -998,7 +998,16 @@ static int local_set_loglevel(struct request *req) > set_loglevel(new_level); > > return SD_RES_S

Re: [sheepdog] [PATCH 11/12] sheep: make stat_data_objs a generic function

On Wed, Jan 29, 2014 at 10:45:05AM +0900, Hitoshi Mitake wrote: > At Wed, 29 Jan 2014 04:19:11 +0800, > Liu Yuan wrote: > > > > and rename it as sd_inode_stat() > > > > Signed-off-by: Liu Yuan > > --- > > dog/vdi.c| 96 +--- > > include/inter

Re: [sheepdog] [PATCH 09/12] sheep/nfs: implement read/write nfs operation

At Wed, 29 Jan 2014 04:19:09 +0800, Liu Yuan wrote: > > Now we support more bash command such as cp, mv, echo, cat ... > > Signed-off-by: Liu Yuan > --- > include/util.h | 8 + > sheep/nfs/fs.c | 41 + > sheep/nfs/fs.h | 2 ++ > sheep/nfs/nfs.c |

Re: [sheepdog] [PATCH 11/12] sheep: make stat_data_objs a generic function

At Wed, 29 Jan 2014 04:19:11 +0800, Liu Yuan wrote: > > and rename it as sd_inode_stat() > > Signed-off-by: Liu Yuan > --- > dog/vdi.c| 96 +--- > include/internal_proto.h | 3 ++ > lib/Makefile.am | 3 +- > lib/sd.c

[sheepdog] Jenkins build is back to normal : sheepdog-build #252

See -- sheepdog mailing list sheepdog@lists.wpkg.org http://lists.wpkg.org/mailman/listinfo/sheepdog

[sheepdog] [PATCH 11/12] sheep: make stat_data_objs a generic function

and rename it as sd_inode_stat() Signed-off-by: Liu Yuan --- dog/vdi.c| 96 +--- include/internal_proto.h | 3 ++ lib/Makefile.am | 3 +- lib/sd.c | 112 +++ 4 files chan

[sheepdog] [PATCH 10/12] sheep/nfs: add mkdir support

Now we support 'mkdir' command and gain hierarchical structure support. TODO: - make dentry operation operate on top of 'file' operation - add hash or btree based dentries management Signed-off-by: Liu Yuan --- sheep/nfs/fs.c | 41 ++--- sheep/nfs/fs.h | 1 + sheep/

[sheepdog] [PATCH 08/12] sheep/nfs: support lookup/create/setattr nfs operation

Now we support 'touch' command. Signed-off-by: Liu Yuan --- sheep/nfs/fs.c | 51 + sheep/nfs/fs.h | 2 + sheep/nfs/nfs.c | 140 ++-- sheep/nfs/nfs.h | 26 +++ sheep/nfs/xdr.c | 12 ++--- 5 files changed, 202 i

[sheepdog] [PATCH 12/12] sheep/nfs: implement nfs3_fsstat operation

Now we support 'df' command to get file system stats. Signed-off-by: Liu Yuan --- sheep/nfs/nfs.c | 39 +-- 1 file changed, 37 insertions(+), 2 deletions(-) diff --git a/sheep/nfs/nfs.c b/sheep/nfs/nfs.c index 8d0e11f..afc0cf8 100644 --- a/sheep/nfs/nfs.c +++

[sheepdog] [PATCH 09/12] sheep/nfs: implement read/write nfs operation

Now we support more bash command such as cp, mv, echo, cat ... Signed-off-by: Liu Yuan --- include/util.h | 8 + sheep/nfs/fs.c | 41 + sheep/nfs/fs.h | 2 ++ sheep/nfs/nfs.c | 87 +++-- sheep/nfs/nf

[sheepdog] [PATCH 03/12] sheep/nfs: add mount protocol stub

This is a preparation patch Signed-off-by: Liu Yuan --- sheep/Makefile.am | 2 +- sheep/nfs/mount.c | 42 +++ sheep/nfs/nfs.h | 99 sheep/nfs/nfsd.c | 75 + sheep/nfs/xdr.c | 122 ++

[sheepdog] [PATCH 07/12] sheep: clean up sd_inode btree error code

These error codes are used internally, not for client communication. Signed-off-by: Liu Yuan --- include/internal_proto.h | 1 + include/sheepdog_proto.h | 3 --- lib/sd_inode.c | 22 +++--- 3 files changed, 12 insertions(+), 14 deletions(-) diff --git a/include/inte

[sheepdog] [PATCH 06/12] sheep/nfs: add basic file system framework

Add basic inode, dentry structures and just support 'ls' /mntpoint operation. For inode management, we use the hash scheme as in http implementation. This will save us a lots of code on inode allocation and deallocation and management. We also make use the simiar concepts found in traditional UNI

[sheepdog] [PATCH 04/12] dog: add nfs command

We support allocate as many nfs backend storage as you want. usage: dog nfs create $name # create a nfs backend named $name dog nfs delete $name # delete a nfs backend named $name $name is the name for users to mount at client in $IP:$NAME Signed-off-by: Liu Yuan --- dog/Makefile.am

[sheepdog] [PATCH 05/12] util: add PTR_ERR helpers

Sometimes we need to return ERROR code from pointers Signed-off-by: Liu Yuan --- include/compiler.h | 2 ++ include/util.h | 30 ++ 2 files changed, 32 insertions(+) diff --git a/include/compiler.h b/include/compiler.h index 4a59b92..2fbd2a4 100644 --- a/include

[sheepdog] [PATCH 00/12] introduce basic NFS server support

This patch set mainly introduce NFS server building blocks, that is - NFS v3 transport by glibc's SUNRPC - preliminary NFS server NFS protocol is seen as the most established shared storage protocol to provide POSIX file semantics and has rich client and server features. It was introduced to part

[sheepdog] [PATCH 01/12] sheep: fix some compile errors

Both MIN/MAX and roundup are provided by glibc and would cause compile error when we #include . Just use GNU version. Signed-off-by: Liu Yuan --- include/util.h | 6 ++ 1 file changed, 2 insertions(+), 4 deletions(-) diff --git a/include/util.h b/include/util.h index 83f3c08..1932fe3 100644

Re: [sheepdog] [PATCH 2/2] sheep: cleaning vdi deletion process, round 2

At Tue, 28 Jan 2014 17:04:14 +0900, MORITA Kazutaka wrote: > > At Tue, 28 Jan 2014 15:26:52 +0900, > Hitoshi Mitake wrote: > > > > > > > > Only Kazutaka knows the reason. I guess it was planned to be paralle. I > > > think > > > deletion code is really dirty and tricky and hard to maintain, nee

[sheepdog] [PATCH v2] sheep/recovery: multi-threading recovery process

Rationale for multi-threaded recovery: 1. If one node is added, we find that all the VMs on other nodes will get noticeably affected until 50% data is transferred to the new node. 2. For node failure, we might not have problems of running VM but the recovery process boost will benefit IO op

[sheepdog] [PATCH] sheep/recovery: multi-threading recovery process

Rationale for multi-threaded recovery: 1. If one node is added, we find that all the VMs on other nodes will get noticeably affected until 50% data is transferred to the new node. 2. For node failure, we might not have problems of running VM but the recovery process boost will benefit IO op

Re: [sheepdog] [PATCH] tests/func/058: fix supurious failure

Reviewed-by: Robin Dong 2014-01-28 Liu Yuan > Signed-off-by: Liu Yuan > --- > dog/vdi.c | 2 +- > 1 file changed, 1 insertion(+), 1 deletion(-) > > diff --git a/dog/vdi.c b/dog/vdi.c > index 1e175b3..10b0307 100644 > --- a/dog/vdi.c > +++ b/dog/vdi.c > @@ -1048,7 +1048,7 @@ static int vdi_ob

[sheepdog] [PATCH] tests/func/058: fix supurious failure

Signed-off-by: Liu Yuan --- dog/vdi.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/dog/vdi.c b/dog/vdi.c index 1e175b3..10b0307 100644 --- a/dog/vdi.c +++ b/dog/vdi.c @@ -1048,7 +1048,7 @@ static int vdi_object_location(int argc, char **argv)

Re: [sheepdog] [PATCH] sheepfs: fix complie error when http isn't enabled

At Tue, 28 Jan 2014 15:02:02 +0900, Hitoshi Mitake wrote: > > At Tue, 28 Jan 2014 11:13:34 +0800, > Liu Yuan wrote: > > > > Forgot to remove -lcurl in my last patch, remove it. > > > > Signed-off-by: Liu Yuan > > --- > > sheepfs/Makefile.am | 2 +- > > 1 file changed, 1 insertion(+), 1 deletio

Re: [sheepdog] [PATCH 2/2] sheep: cleaning vdi deletion process, round 2

At Tue, 28 Jan 2014 15:26:52 +0900, Hitoshi Mitake wrote: > > > > > Only Kazutaka knows the reason. I guess it was planned to be paralle. I > > think > > deletion code is really dirty and tricky and hard to maintain, need > > refactor and > > redesign. > > Of course I agree so I'm writing this