Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: fix overlapping metadata update

2015-08-02 Thread Liu Yuan
On Sun, Aug 02, 2015 at 02:52:08PM +0300, Vasiliy Tolstov wrote: 2015-07-31 15:08 GMT+03:00 Vasiliy Tolstov v.tols...@selfip.ru: Please wait to performance comparison. As i see Liu's patch may be more slow then Hitoshi. I'm switch to local cluster driver to test only local ssd and not

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: fix overlapping metadata update

2015-08-01 Thread Liu Yuan
On Fri, Jul 31, 2015 at 03:08:09PM +0300, Vasiliy Tolstov wrote: 2015-07-31 14:55 GMT+03:00 Vasiliy Tolstov v.tols...@selfip.ru: Liu's patch also works for me. But also like in Hitoshi patch breaks when using discards in qemu =(. Please wait to performance comparison. As i see Liu's

Re: [sheepdog] [PATCH] sheepdog: fix overlapping metadata update

2015-07-29 Thread Liu Yuan
On Wed, Jul 29, 2015 at 02:04:55PM +0900, Hitoshi Mitake wrote: At Wed, 29 Jul 2015 12:02:35 +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching the updates. But there is subtle problem

Re: [sheepdog] what is sheepdog-ng

2015-07-29 Thread Liu Yuan
On Wed, Jul 29, 2015 at 10:17:45AM +0800, 陈李粮 wrote: what is sheepdog-ng? It is a forked of Sheepdog due to unstability and unfriendliness of open source project. Sheepdog, in my eyes, now become a NTT's project, so Kazutaka and I quit the maintainership of sheepdog and moved to sheepdog-ng.

Re: [sheepdog] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: Current sheepdog driver only serializes create requests in oid unit. This mechanism isn't enough for handling requests to overwrapping area spanning multiple oids, so it can result bugs like below:

Re: [sheepdog] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Tue, Jul 28, 2015 at 04:50:08PM +0800, Liu Yuan wrote: On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: Current sheepdog driver only serializes create requests in oid unit. This mechanism isn't enough for handling requests to overwrapping area spanning multiple oids, so

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Mon, Jul 27, 2015 at 11:23:02AM -0400, Jeff Cody wrote: On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: Current sheepdog driver only serializes create requests in oid unit. This mechanism isn't enough for handling requests to overwrapping area spanning multiple oids, so

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Tue, Jul 28, 2015 at 10:31:32PM +0800, Liu Yuan wrote: On Mon, Jul 27, 2015 at 11:23:02AM -0400, Jeff Cody wrote: On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: Current sheepdog driver only serializes create requests in oid unit. This mechanism isn't enough

[sheepdog] [PATCH] sheepdog: fix overlapping metadata update

2015-07-28 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching the updates. But there is subtle problem by determining min_idx and max_idx: for a single create request, min_idx == max_idx, so actually we just update one one bit

Re: [sheepdog] The magic number of 12

2015-07-07 Thread Liu Yuan
On Wed, Jul 08, 2015 at 11:10:47AM +0800, Yu Yang wrote: There is a struct node_id which saves the sheep node address and port as follows in internal_proto.h: struct node_id { uint8_t addr[16]; uint16_t port; uint8_t io_addr[16]; uint16_t io_port; uint8_t

Re: [sheepdog] VDI r/w performance comparison test

2015-05-27 Thread Liu Yuan
On Wed, May 27, 2015 at 04:24:29PM +0800, yuyang wrote: Hello everyone, Recently we need copy one VDI from sheepdog to another storage system(such as sheepdog or RAID). There are 2 ways to choose, 1. dog vdi r | dog vdi w,(only for sheepdog, or temp file is needed) 2. lib client, which is

Re: [sheepdog] [PATCH] lib/sheep.c: fix return value of sd_run_sdreq()

2015-05-17 Thread Liu Yuan
On Thu, May 14, 2015 at 03:12:35PM +0800, Yu Yang wrote: From: Yu Yang yuy...@cmss.chinamobile.com The return value should be SD_RES_XXX, so return SD_RES_SYSTEM_ERROR when sheep_submit_sdreq() failed. Signed-off-by: Yu Yang yuy...@cmss.chinamobile.com --- lib/shared/sheep.c |2 +-

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Thu, May 14, 2015 at 10:31:47PM +0300, Vasiliy Tolstov wrote: Per volume redundancy useful for hostings or openstack based solutions. So users may need to pay for stability or increase speed.. 14 мая 2015 г. 13:21 пользователь Valerio Pachera siri...@gmail.com написал: IMHO, per volume

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Mon, May 18, 2015 at 09:52:03AM +0900, Hitoshi Mitake wrote: At Thu, 14 May 2015 00:57:24 +0800, Liu Yuan wrote: Hi y'all, Based on recent frustrating[1] debug on our test cluster, I'd like propsose, which might looks very radical to you, that we should remove struct vdi_state

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Mon, May 18, 2015 at 01:42:59PM +0900, Hitoshi Mitake wrote: At Mon, 18 May 2015 09:46:25 +0800, Liu Yuan wrote: On Mon, May 18, 2015 at 09:52:03AM +0900, Hitoshi Mitake wrote: At Thu, 14 May 2015 00:57:24 +0800, Liu Yuan wrote: Hi y'all, Based on recent

[sheepdog] A radical rethinking on struct vdi_state

2015-05-13 Thread Liu Yuan
Hi y'all, Based on recent frustrating[1] debug on our test cluster, I'd like propsose, which might looks very radical to you, that we should remove struct vdi_state completely from sheepdog code. Let me show you the background picture how it was introduced. It was introduced by Leven Li by the

[sheepdog] [PATCH] sbd: use kernel_setsockopt helper

2015-05-04 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- sbd/sheep.c | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/sbd/sheep.c b/sbd/sheep.c index bfc935b..8b74b59 100644 --- a/sbd/sheep.c +++ b/sbd/sheep.c @@ -34,7

[sheepdog] [PATCH 1/2] rename libsheepdog.a as libsd.a for internal use

2015-05-04 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch for creating user statically linked libsheepdog.a. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- dog/Makefile.am | 4 ++-- include/common.h | 2 +- lib/Makefile.am | 14

[sheepdog] [PATCH 2/2] create static library for user linking

2015-05-04 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Some use case might want to statically linked to the sheepdog library. Howto: $ gcc your_prog.c -lpthread /path/to/libsheepdog.a Note, 'make install' will install libsheepdog.a into /usr/lib/ along with libsheepdog.so. Signed-off-by: Liu Yuan liuy

Re: [sheepdog] [PATCH 1/4] sheep: rename files for store_driver

2015-04-26 Thread Liu Yuan
On Mon, Apr 27, 2015 at 12:54:22PM +0900, Hitoshi Mitake wrote: From: Masaki Saeki saeki.mas...@po.ntts.co.jp This change is a preparation patch for add store_driver. Put files together to the new folder. Signed-off-by: Masaki Saeki saeki.mas...@po.ntts.co.jp Applied this series, thanks.

[sheepdog] [PATCH 2/3] shared lib: add the function description

2015-04-23 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- lib/shared/sheepdog.h | 70 +-- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/lib/shared/sheepdog.h b/lib/shared/sheepdog.h index

[sheepdog] [PATCH 3/3] shared lib: don't export unnecessary structures to user

2015-04-23 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com internal.h is dedicated to hold the non-user functions and structures for shared lib. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- lib/shared/internal.h | 28 lib/shared/sheep.c| 1 + lib/shared/sheepdog.h

[sheepdog] [PATCH v2 1/3] shared lib: return SD_RES_XXX instead of mixing std errno and sd error code

2015-04-23 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Have all the sd_xxx functions return SD_RES_XXX, we will have a unified error handling for user program and more importantly, std error can't cover all the error cases that is specific to the sheepdog cluster, such as SD_RES_NO_VDI and SD_RES_NO_OBJ

[sheepdog] [PATCH v2 3/3] shared lib: don't export unnecessary structures to user

2015-04-23 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com internal.h is dedicated to hold the non-user functions and structures for shared lib. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- lib/shared/internal.h | 28 lib/shared/sheep.c| 1 + lib/shared/sheepdog.h

[sheepdog] [PATCH v2 2/3] shared lib: add the function description

2015-04-23 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- lib/shared/sheepdog.h | 70 +-- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/lib/shared/sheepdog.h b/lib/shared/sheepdog.h index

Re: [sheepdog] [PATCH v2] fix systemd service to run sheep in foreground and with defaults

2015-04-13 Thread Liu Yuan
On Mon, Apr 13, 2015 at 01:32:56PM +0300, Vasiliy Tolstov wrote: sheepdog now able to run in foreground, so use this in systemd service, also remove running shell script to get defaults, because it provided in service file now. Signed-off-by: Vasiliy Tolstov v.tols...@selfip.ru ---

Re: [sheepdog] [PATCH] lib/work: fix compiler warning

2015-04-13 Thread Liu Yuan
On Mon, Apr 13, 2015 at 02:35:02PM +0800, Ruoyu wrote: This patch fixes the compiler warning message as below. warning: implicit declaration of function ‘install_sighandler’ [-Wimplicit-function-declaration] Signed-off-by: Ruoyu lian...@ucweb.com --- lib/work.c | 1 + 1 file changed, 1

Re: [sheepdog] sheepdog cluster data area node with bonding mode 4 with (Bonding Mode: IEEE 802.3ad Dynamic link aggregation) can't join gateway node

2015-04-12 Thread Liu Yuan
On Sun, Apr 12, 2015 at 07:38:09PM +0800, passedwind wrote: today i change my sheepdog cluster data area node network with bonding mode 4 with (Bonding Mode: IEEE 802.3ad Dynamic link aggregation).gateway node with normal network interface. gateway node to dataArea node network link no

Re: [sheepdog] [PATCH] sheep journal:fix the parameter type of 'size' passed to 'int prealloc(int fd, uint32_t size)'

2015-04-12 Thread Liu Yuan
On Thu, Apr 09, 2015 at 03:30:21PM +0800, 张灿群 wrote: dear, When i start sheep daemon with journal,the daemon couldn't start. command: sheep -j dir=/sdac,size=160G,skip=1 -l dir=/var/log/,level=err,format=server /sdad/ -n -D -y 10.64.0.38 -p 7000 -c

Re: [sheepdog] [PATCH v6] try to set RLIMIT_NOFILE to recommended value

2015-04-08 Thread Liu Yuan
On Wed, Apr 08, 2015 at 09:14:29AM +0300, Vasiliy Tolstov wrote: Signed-off-by: Vasiliy Tolstov v.tols...@selfip.ru --- sheep/sheep.c | 26 +++-- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/sheep/sheep.c b/sheep/sheep.c index 86444b0..38ffed0

Re: [sheepdog] [PATCH v1] fix systemd service to run sheep in foreground and with defaults

2015-04-08 Thread Liu Yuan
On Wed, Apr 08, 2015 at 09:26:05AM +0300, Vasiliy Tolstov wrote: sheepdog now able to run in foreground, so use this in systemd service, also remove running shell script to get defaults, because it provided in service file now. Signed-off-by: Vasiliy Tolstov v.tols...@selfip.ru ---

Re: [sheepdog] [PATCH v7] try to set RLIMIT_NOFILE to recommended value

2015-04-08 Thread Liu Yuan
On Wed, Apr 08, 2015 at 09:43:10AM +0300, Vasiliy Tolstov wrote: Signed-off-by: Vasiliy Tolstov v.tols...@selfip.ru --- sheep/sheep.c | 26 +++-- 1 file changed, 14 insertions(+), 12 deletions(-) diff --git a/sheep/sheep.c b/sheep/sheep.c index 86444b0..4f6a662

Re: [sheepdog] [PATCH 2/8] move functions used by libsheepdog.a out of util.c

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 01:21:05AM +0900, Hitoshi Mitake wrote: At Fri, 3 Apr 2015 11:20:48 +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com This is a prepation patch to make util.c self-contained and then can be used outside sheep. Signed-off-by: Liu Yuan

Re: [sheepdog] [PATCH 8/8] shared lib: kick it run

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 01:19:20AM +0900, Hitoshi Mitake wrote: At Fri, 3 Apr 2015 11:20:54 +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com Add the final makefile stuff Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- configure.ac| 3

[sheepdog] [PATCH v2 2/8] move functions used by libsheepdog.a out of util.c

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a prepation patch to make util.c self-contained and then can be used outside sheep. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- dog/common.c| 25 + dog/dog.h | 5 + include/Makefile.am | 3 +- include

[sheepdog] [PATCH v2 1/8] add sd_assert()

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch to dissociate util.h and utli.c from sheepdog, in order to be used by shared lib. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- configure.ac | 11 --- include/logger.h | 7

[sheepdog] [PATCH v2 6/8] shared lib: add the low level request handling framework

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com The core idea is the same as sbd's framework. User request | | aio control block (struct aiocb) V +++ ||| r1 r2r3 ||| VVV

[sheepdog] [PATCH v2 0/8] introduce libsheepdog.so

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Finally, we kick started, though it is far away from a complete implemention. This patch set introduces a shared library and for now it only supports simple vdi operation. The vdi read/write operation would be the most difficult part of shared library

[sheepdog] [PATCH v2 4/8] move sheep specific headers out of sheepdog_proto.h

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is preparation patch for shared lib. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- include/sheep.h | 13 + include/sheepdog_proto.h | 16 +--- lib/sd_inode.c | 1 + 3 files changed, 15

[sheepdog] [PATCH v2 5/8] unrelate list.h, util.h to compiler.h

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch. list.h and util.h will be used by shared library and exported to the use. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- include/compiler.h | 9 - include/list.h | 2 -- include/util.h | 14

[sheepdog] [PATCH v2 3/8] sheep: dissociate utils.c from logger.c

2015-04-07 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch for shared lib. shared lib can't call sd logger. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- dog/dog.h| 1 + dog/treeview.c | 1 + include/fec.h| 1 + include/logger.h | 6 -- include

Re: [sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-03 Thread Liu Yuan
On Fri, Apr 03, 2015 at 03:59:18PM +0900, Hitoshi Mitake wrote: At Fri, 3 Apr 2015 12:06:56 +0800, Liu Yuan wrote: On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote: At Fri, 3 Apr 2015 11:20:46 +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com

[sheepdog] [PATCH] lib: clean up lib/*.o files for 'make clean'

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This will kill a hidden build problem because of stale .o in libsheepdog.a Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- lib/Makefile.am | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/Makefile.am b/lib/Makefile.am

[sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Finally, we kick started, though it is far away from a complete implemention. This patch set introduces a shared library and for now it only supports simple vdi operation. The vdi read/write operation would be the most difficult part of shared library

[sheepdog] [PATCH 6/8] shared lib: add the low level request handling framework

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com The core idea is the same as sbd's framework. User request | | aio control block (struct aiocb) V +++ ||| r1 r2r3 ||| VVV

[sheepdog] [PATCH 2/8] move functions used by libsheepdog.a out of util.c

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a prepation patch to make util.c self-contained and then can be used outside sheep. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- dog/common.c| 25 + dog/dog.h | 5 + include/Makefile.am | 3 +- include

[sheepdog] [PATCH 1/8] add sd_assert()

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch to dissociate util.h and utli.c from sheepdog, in order to be used by shared lib. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- configure.ac | 11 --- include/logger.h | 7

[sheepdog] [PATCH 4/8] move sheep specific headers out of sheepdog_proto.h

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is preparation patch for shared lib. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- include/sheep.h | 13 + include/sheepdog_proto.h | 16 +--- lib/sd_inode.c | 1 + 3 files changed, 15

[sheepdog] [PATCH 3/8] sheep: dissociate utils.c from logger.c

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch for shared lib. shared lib can't call sd logger. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- dog/dog.h| 1 + dog/treeview.c | 1 + include/fec.h| 1 + include/logger.h | 6 -- include

[sheepdog] [PATCH 5/8] unralate list.h, util.h to compiler.h

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This is a preparation patch. list.h and util.h will be used by shared library and exported to the use. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- include/compiler.h | 9 - include/list.h | 2 -- include/util.h | 14

[sheepdog] [PATCH 8/8] shared lib: kick it run

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Add the final makefile stuff Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- configure.ac| 3 ++- lib/Makefile.am | 14 ++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index

[sheepdog] [PATCH 7/8] shared lib: implement vdi_{open, read, write, close}

2015-04-02 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com Nothing intersting, these functions are basically wrappers of sheep.c. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- lib/shared/sheep.c| 3 +- lib/shared/sheepdog.h | 10 ++- lib/shared/vdi.c | 185

Re: [sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-02 Thread Liu Yuan
On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote: At Fri, 3 Apr 2015 11:20:46 +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com Finally, we kick started, though it is far away from a complete implemention. This patch set introduces a shared

Re: [sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-02 Thread Liu Yuan
On Fri, Apr 03, 2015 at 12:06:56PM +0800, Liu Yuan wrote: On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote: At Fri, 3 Apr 2015 11:20:46 +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com Finally, we kick started, though it is far away from

Re: [sheepdog] [PATCH v2] add foreground option

2015-03-25 Thread Liu Yuan
On Wed, Mar 25, 2015 at 11:33:17AM +0300, Vasiliy Tolstov wrote: 2015-03-25 5:13 GMT+03:00 Liu Yuan namei.u...@gmail.com: If you just want to see log in stdout, you can try -l dst=stdout'. Any other reason to bring this back? firstly under systemd forking not recommended, also you

Re: [sheepdog] [PATCH v4] try to set RLIMIT_NOFILE to recommended value

2015-03-25 Thread Liu Yuan
On Tue, Mar 17, 2015 at 05:30:47PM +0300, Vasiliy Tolstov wrote: Signed-off-by: Vasiliy Tolstov v.tols...@selfip.ru --- sheep/sheep.c | 29 + 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/sheep/sheep.c b/sheep/sheep.c index fc53ec9..cad8293

Re: [sheepdog] [PATCH v2] add foreground option

2015-03-24 Thread Liu Yuan
with -l dst=stdout. Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp Signed-off-by: Liu Yuan namei.u...@gmail.com --- If you just want to see log in stdout, you can try -l dst=stdout'. Any other reason to bring

Re: [sheepdog] [PATCH v4] try to set RLIMIT_NOFILE to recommended value

2015-03-24 Thread Liu Yuan
On Tue, Mar 17, 2015 at 05:30:47PM +0300, Vasiliy Tolstov wrote: Signed-off-by: Vasiliy Tolstov v.tols...@selfip.ru --- sheep/sheep.c | 29 + 1 file changed, 17 insertions(+), 12 deletions(-) diff --git a/sheep/sheep.c b/sheep/sheep.c index fc53ec9..cad8293

Re: [sheepdog] [PATCH 3/4 v3] sheep : add new strage type tree

2015-03-22 Thread Liu Yuan
On Mon, Mar 23, 2015 at 02:18:24PM +0900, Saeki Masaki wrote: On 2015/03/21 21:02, Liu Yuan wrote: On Fri, Mar 20, 2015 at 06:39:08PM +0900, Saeki Masaki wrote: Current sheepdog stores whole objects in single directory like /var/lib/sheepdog/obj This mechanism is difficult to handle massive

Re: [sheepdog] [PATCH 3/4 v3] sheep : add new strage type tree

2015-03-21 Thread Liu Yuan
On Fri, Mar 20, 2015 at 06:39:08PM +0900, Saeki Masaki wrote: Current sheepdog stores whole objects in single directory like /var/lib/sheepdog/obj This mechanism is difficult to handle massive files when increasing cluster volume. In particular, inode object having special informations

Re: [sheepdog] [PATCH 1/4 v3] sheep: rename files for store_driver

2015-03-21 Thread Liu Yuan
sheep_SOURCES+= http/http.c http/kv.c http/s3.c http/swift.c \ diff --git a/sheep/md.c b/sheep/md.c deleted file mode 100644 index 87ab759..000 --- a/sheep/md.c +++ /dev/null @@ -1,878 +0,0 @@ -/* - * Copyright (C) 2013 Taobao Inc. - * - * Liu Yuan namei.u...@gmail.com

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-17 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:51:32PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:40:51 +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-17 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:59:12PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:54:16 +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:40:51PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-17 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:59:12PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:54:16 +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:40:51PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 04:44:46AM +0900, MORITA Kazutaka wrote: At Mon, 16 Mar 2015 21:13:29 +0800, Liu Yuan wrote: How about make 'dog vdi clone --no-share' as the default clone operation? And we can add dog vdi clone --share to keep old behavior as optional. By this manner

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 11:42:01AM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 10:03:53 +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 04:44:46AM +0900, MORITA Kazutaka wrote: At Mon, 16 Mar 2015 21:13:29 +0800, Liu Yuan wrote: How about make 'dog vdi clone --no-share

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 10:03:53AM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 04:44:46AM +0900, MORITA Kazutaka wrote: At Mon, 16 Mar 2015 21:13:29 +0800, Liu Yuan wrote: How about make 'dog vdi clone --no-share' as the default clone operation? And we can add dog vdi clone

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 09:37:45PM +0900, Hitoshi Mitake wrote: On Mon, Mar 16, 2015 at 4:39 PM, Liu Yuan namei.u...@gmail.com wrote: On Mon, Mar 16, 2015 at 03:55:43PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 14:48:47 +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 03:39:17PM

[sheepdog] [PATCH 2/6] Revert sheep: construct data structures for representing family tree

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This reverts commit b30374cf653c6bf93c91e14fe0954c58e4097637. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- sheep/group.c | 7 ++- sheep/plain_store.c | 2 +- sheep/sheep_priv.h | 2 - sheep/vdi.c | 146

[sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This patch fixes following problem: $ dog vdi create test 100M $ dog vdi delete test $ dog vdi create test 200M $ dog vdi list # expect show test, but nothing shows out. Which was brought by *commit f68feab7edc0ded86701a2e902d85616b24942ab *Author

[sheepdog] [PATCH 3/6] Revert sheep: let vdi state have parent VID

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This reverts commit fcb91648b1a1f97c55c2a42fff1ec92b93bfea44. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- include/internal_proto.h | 1 - sheep/group.c| 3 +-- sheep/ops.c | 7 +++ sheep/plain_store.c

[sheepdog] [PATCH 5/6] Revert sheep: don't recycle VDI ID

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This reverts commit 21549a1bd4981fabcc09d062a647162127fe0637. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- sheep/vdi.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/sheep/vdi.c b/sheep/vdi.c index dd01a20..af96548 100644

[sheepdog] [PATCH 4/6] Revert tests/functional: update test scripts for VID GC feature

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This reverts commit 5d005d41862647f6c7cdb7977f2611e61abc104b. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- tests/functional/016.out | 12 ++-- tests/functional/044.out | 12 ++-- tests/functional/046.out | 6 +++--- tests

[sheepdog] [PATCH 1/6] Revert sheep: garbage collect needless VIDs

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This reverts commit 58f0d6dcdfa370df4456ed08cd32983731fc0c4e. Signed-off-by: Liu Yuan liuy...@cmss.chinamobile.com --- sheep/ops.c| 1 - sheep/sheep_priv.h | 1 - sheep/vdi.c| 110

[sheepdog] [PATCH 0/6] fix vdi recycling

2015-03-16 Thread Liu Yuan
From: Liu Yuan liuy...@cmss.chinamobile.com This patch set mainly revert the patch set that broke the old vdi recycling algorithm that allow vdi to be recycled if it is deleted, though we keep deleted inodes in the storage because of vdi allocation algorithm assumes it. The old vid recycle work

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 09:39:35PM +0900, Hitoshi Mitake wrote: On Mon, Mar 16, 2015 at 7:42 PM, Liu Yuan namei.u...@gmail.com wrote: On Mon, Mar 16, 2015 at 03:57:40PM +0900, Hitoshi Mitake wrote: This patch adds a new option -R to dog cluster format. If user specifies the option during

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 08:49:34PM +0800, Liu Yuan wrote: From: Liu Yuan liuy...@cmss.chinamobile.com This patch fixes following problem: $ dog vdi create test 100M $ dog vdi delete test $ dog vdi create test 200M $ dog vdi list # expect show test, but nothing shows out. Which

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 03:55:56AM +, morita...@nttdata.co.jp wrote: I'm still not sure which code in vdi_lookup() is a problem. The problem happens even when we disable VID garbage collection? vdi_lookup() becomes a problem if Hitoshi's patch is enabled after he fixes a

Re: [sheepdog] 答复: [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 12:08:39PM +0800, redtone wrote: Many people want do daily snapshot (delete the old one and create a new one with the save name) This is not the case we talked about. Your case will work fine. I don't think you will delete all the snapshots since day 1 very frequently.

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:58:08 +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:51:48 +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 12:42:35 +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 08:49:34PM

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:58:08 +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:51:48 +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 12:42:35 +0800

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:58:08 +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:51:48 +0900

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:40:51PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 13:58:08 +0900, Hitoshi

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:33:58PM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 11:06:34 +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 11:42:01AM +0900, Hitoshi Mitake wrote: At Tue, 17 Mar 2015 10:03:53 +0800, Liu Yuan wrote: On Tue, Mar 17, 2015 at 04:44:46AM +0900

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 02:36:57PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 10:21:50 +0800, Liu Yuan wrote: On Thu, Mar 12, 2015 at 08:14:33PM +0900, Hitoshi Mitake wrote: At Thu, 12 Mar 2015 14:41:56 +0800, Liu Yuan wrote: On Tue, Jan 13, 2015 at 10:37:40AM +0900

Re: [sheepdog] [PATCH 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:24:31PM +0900, Hitoshi Mitake wrote: This patch adds a new option -R to dog cluster format. If user specifies the option during cluster format, the cluster will enable recycling VID (disabled in default). Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:55:43PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 14:48:47 +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 03:39:17PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 14:31:44 +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 02:08:25PM +0800

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:39:17PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 14:31:44 +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 02:08:25PM +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 02:36:57PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 10:21:50 +0800, Liu

Re: [sheepdog] [PATCH 1/2] sheep : add new strage type tree

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:54:30PM +0900, Saeki Masaki wrote: On 2015/03/16 11:06, Liu Yuan wrote: On Thu, Mar 12, 2015 at 01:50:26PM +0900, Saeki Masaki wrote: Current sheepdog stores whole objects in single directory like /var/lib/sheepdog/obj This mechanism is difficult to handle massive

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 02:08:25PM +0800, Liu Yuan wrote: On Mon, Mar 16, 2015 at 02:36:57PM +0900, Hitoshi Mitake wrote: At Mon, 16 Mar 2015 10:21:50 +0800, Liu Yuan wrote: On Thu, Mar 12, 2015 at 08:14:33PM +0900, Hitoshi Mitake wrote: At Thu, 12 Mar 2015 14:41:56 +0800, Liu

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:57:40PM +0900, Hitoshi Mitake wrote: This patch adds a new option -R to dog cluster format. If user specifies the option during cluster format, the cluster will enable recycling VID (disabled in default). Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp

Re: [sheepdog] [PATCH] tools/zk_control: support multi-cluster

2015-03-16 Thread Liu Yuan
On Thu, Mar 12, 2015 at 02:08:13PM +0800, Meng Lingkun wrote: From: Meng Lingkun mengling...@cmss.chinamobile.com Add option -c/--cluster to specify the cluster which zk_control will handle. Note: Using malloc rather than xmalloc because it can't be linked under current makefile. Maybe fix

Re: [sheepdog] rlimit nofile

2015-03-15 Thread Liu Yuan
On Thu, Mar 12, 2015 at 03:17:35PM +0300, Vasiliy Tolstov wrote: Why sheepdog complains about maximum open files limit is too small and not increase it via http://man7.org/linux/man-pages/man2/getrlimit.2.html ? Thanks, we should give it a try, could you post a patch for it and verify it can

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-15 Thread Liu Yuan
On Thu, Mar 12, 2015 at 08:14:33PM +0900, Hitoshi Mitake wrote: At Thu, 12 Mar 2015 14:41:56 +0800, Liu Yuan wrote: On Tue, Jan 13, 2015 at 10:37:40AM +0900, Hitoshi Mitake wrote: Current sheepdog never recycles VIDs. But it will cause problems e.g. VID space exhaustion, too much

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-12 Thread Liu Yuan
On Tue, Jan 13, 2015 at 10:37:40AM +0900, Hitoshi Mitake wrote: Current sheepdog never recycles VIDs. But it will cause problems e.g. VID space exhaustion, too much garbage inode objects. Keeping deleted inode objects is required because living inodes (snapshots or clones) can point objects

Re: [sheepdog] [SHEEPDOG-UTILS][PATCH] Set different list of packages for debian 8

2015-03-12 Thread Liu Yuan
On Thu, Mar 12, 2015 at 08:27:02AM +0100, Valerio Pachera wrote: 2015-03-12 2:51 GMT+01:00 Liu Yuan namei.u...@gmail.com: install.sh is generated by autoreconf, I guess, controled by configure.ac. You can verify this by rm install.sh and run ./autogen.sh. So the fix should go

Re: [sheepdog] [SHEEPDOG-UTILS][PATCH] Set different list of packages for debian 8

2015-03-11 Thread Liu Yuan
On Wed, Mar 11, 2015 at 04:05:25PM +0100, Valerio Pachera wrote: From: Valerio Pachera siri...@gmail.conf This patch add the package yasm to be installed in both debian 7 and debian 8. liburcu2 will be insalled instead of liburcu1 in debian 8. It also fixed an error message that didn't get

Re: [sheepdog] [PATCH v4 1/3] sheep: handle VID overflow correctly

2015-03-11 Thread Liu Yuan
On Sun, Mar 08, 2015 at 01:08:33PM +0900, Hitoshi Mitake wrote: Current sheep cannot handle a case of VID overflow correctly. The main cause of this problem is invalid usage of fill_vdi_info_range() by fill_vdi_info() in a case of overflow. This patch fixes this problem. Could you please

Re: [sheepdog] [PATCH] automake: work-around for not expanding variables in _SOURCES.

2015-03-10 Thread Liu Yuan
On Mon, Mar 09, 2015 at 07:30:40PM -0700, Alexander Guy wrote: Common versions of automake have a bug[1] related to enabling subdir-objects while using variables inside *_SOURCES declarations. In order to work around these problems with the unit tests, copy source dependencies locally for

Re: [sheepdog] writing cluster driver

2015-03-10 Thread Liu Yuan
On Tue, Mar 10, 2015 at 12:07:30PM +0300, Vasiliy Tolstov wrote: Hello. I'm ready to write etcd cluster driver, but i have some absent info. For example sheperd cluster driver have struct: static struct cluster_driver cdrv_shepherd = { .name = shepherd, .init

  1   2   3   4   5   6   7   8   9   10   >