Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: fix overlapping metadata update

2015-08-02 Thread Liu Yuan
On Thu, Jul 30, 2015 at 09:27:44AM -0400, Jeff Cody wrote: > On Thu, Jul 30, 2015 at 09:41:08AM +0300, Vasiliy Tolstov wrote: > > 2015-07-29 12:31 GMT+03:00 Liu Yuan : > > > Technically, it won't affect the performance because index updates are > > > not rang

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: fix overlapping metadata update

2015-08-02 Thread Liu Yuan
On Sun, Aug 02, 2015 at 02:52:08PM +0300, Vasiliy Tolstov wrote: > 2015-07-31 15:08 GMT+03:00 Vasiliy Tolstov : > > Please wait to performance comparison. As i see Liu's patch may be > > more slow then Hitoshi. > > > I'm switch to local cluster driver to test only local ssd and not > network over

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: fix overlapping metadata update

2015-08-01 Thread Liu Yuan
On Fri, Jul 31, 2015 at 03:08:09PM +0300, Vasiliy Tolstov wrote: > 2015-07-31 14:55 GMT+03:00 Vasiliy Tolstov : > > Liu's patch also works for me. But also like in Hitoshi patch breaks > > when using discards in qemu =(. > > > Please wait to performance comparison. As i see Liu's patch may be > m

Re: [sheepdog] [PATCH] sheepdog: fix overlapping metadata update

2015-07-29 Thread Liu Yuan
On Wed, Jul 29, 2015 at 02:04:55PM +0900, Hitoshi Mitake wrote: > At Wed, 29 Jul 2015 12:02:35 +0800, > Liu Yuan wrote: > > > > From: Liu Yuan > > > > Current sheepdog driver use a range update_inode(min_idx, max_idx) for > > batching > > the updates.

Re: [sheepdog] what is sheepdog-ng

2015-07-29 Thread Liu Yuan
On Wed, Jul 29, 2015 at 10:17:45AM +0800, 陈李粮 wrote: > what is sheepdog-ng? > It is a forked of Sheepdog due to unstability and unfriendliness of open source project. Sheepdog, in my eyes, now become a NTT's project, so Kazutaka and I quit the maintainership of sheepdog and moved to sheepdog-ng.

[sheepdog] [PATCH] sheepdog: fix overlapping metadata update

2015-07-28 Thread Liu Yuan
From: Liu Yuan Current sheepdog driver use a range update_inode(min_idx, max_idx) for batching the updates. But there is subtle problem by determining min_idx and max_idx: for a single create request, min_idx == max_idx, so actually we just update one one bit as expected. Suppose we have 2

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Tue, Jul 28, 2015 at 10:31:32PM +0800, Liu Yuan wrote: > On Mon, Jul 27, 2015 at 11:23:02AM -0400, Jeff Cody wrote: > > On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: > > > Current sheepdog driver only serializes create requests in oid > > > unit.

Re: [sheepdog] [Qemu-devel] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Mon, Jul 27, 2015 at 11:23:02AM -0400, Jeff Cody wrote: > On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: > > Current sheepdog driver only serializes create requests in oid > > unit. This mechanism isn't enough for handling requests to > > overwrapping area spanning multiple oids

Re: [sheepdog] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Tue, Jul 28, 2015 at 04:50:08PM +0800, Liu Yuan wrote: > On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: > > Current sheepdog driver only serializes create requests in oid > > unit. This mechanism isn't enough for handling requests to > > overwrapping a

Re: [sheepdog] [PATCH] sheepdog: serialize requests to overwrapping area

2015-07-28 Thread Liu Yuan
On Sat, Jul 18, 2015 at 01:44:24AM +0900, Hitoshi Mitake wrote: > Current sheepdog driver only serializes create requests in oid > unit. This mechanism isn't enough for handling requests to > overwrapping area spanning multiple oids, so it can result bugs like > below: > https://bugs.launchpad.net/

Re: [sheepdog] The magic number of 12

2015-07-07 Thread Liu Yuan
On Wed, Jul 08, 2015 at 11:10:47AM +0800, Yu Yang wrote: > There is a struct node_id which saves the sheep node address and port as > follows > in internal_proto.h: > > struct node_id { > uint8_t addr[16]; > uint16_t port; > uint8_t io_addr[16]; > uint16_t io_port; >

Re: [sheepdog] VDI r/w performance comparison test

2015-05-27 Thread Liu Yuan
On Wed, May 27, 2015 at 04:24:29PM +0800, yuyang wrote: > Hello everyone, > > Recently we need copy one VDI from sheepdog to another storage > system(such as sheepdog or RAID). There are 2 ways to choose, > 1. dog vdi r | dog vdi w,(only for sheepdog, or temp file is needed) > 2. lib client, whic

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Mon, May 18, 2015 at 09:52:03AM +0900, Hitoshi Mitake wrote: > At Thu, 14 May 2015 00:57:24 +0800, > Liu Yuan wrote: > > > > Hi y'all, > > > > Based on recent frustrating[1] debug on our test cluster, I'd like propsose, > > which might looks ver

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Mon, May 18, 2015 at 01:42:59PM +0900, Hitoshi Mitake wrote: > At Mon, 18 May 2015 09:46:25 +0800, > Liu Yuan wrote: > > > > On Mon, May 18, 2015 at 09:52:03AM +0900, Hitoshi Mitake wrote: > > > At Thu, 14 May 2015 00:57:24 +0800, > > > Liu Y

Re: [sheepdog] [PATCH] lib/sheep.c: fix return value of sd_run_sdreq()

2015-05-17 Thread Liu Yuan
On Thu, May 14, 2015 at 03:12:35PM +0800, Yu Yang wrote: > From: Yu Yang > > The return value should be SD_RES_XXX, so return SD_RES_SYSTEM_ERROR > when sheep_submit_sdreq() failed. > > Signed-off-by: Yu Yang > --- > lib/shared/sheep.c |2 +- > 1 file changed, 1 insertion(+), 1 deletion(-)

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Mon, May 18, 2015 at 09:52:03AM +0900, Hitoshi Mitake wrote: > At Thu, 14 May 2015 00:57:24 +0800, > Liu Yuan wrote: > > > > Hi y'all, > > > > Based on recent frustrating[1] debug on our test cluster, I'd like propsose, > > which might looks ver

Re: [sheepdog] A radical rethinking on struct vdi_state

2015-05-17 Thread Liu Yuan
On Thu, May 14, 2015 at 10:31:47PM +0300, Vasiliy Tolstov wrote: > Per volume redundancy useful for hostings or openstack based solutions. So > users may need to pay for stability or increase speed.. > 14 мая 2015 г. 13:21 пользователь "Valerio Pachera" > написал: > > > IMHO, per volume redundanc

[sheepdog] A radical rethinking on struct vdi_state

2015-05-13 Thread Liu Yuan
Hi y'all, Based on recent frustrating[1] debug on our test cluster, I'd like propsose, which might looks very radical to you, that we should remove struct vdi_state completely from sheepdog code. Let me show you the background picture how it was introduced. It was introduced by Leven Li by the id

[sheepdog] [PATCH 1/2] rename libsheepdog.a as libsd.a for internal use

2015-05-04 Thread Liu Yuan
From: Liu Yuan This is a preparation patch for creating user statically linked libsheepdog.a. Signed-off-by: Liu Yuan --- dog/Makefile.am | 4 ++-- include/common.h | 2 +- lib/Makefile.am | 14 +++--- lib/common.c | 2 +- sheep

[sheepdog] [PATCH 2/2] create static library for user linking

2015-05-04 Thread Liu Yuan
From: Liu Yuan Some use case might want to statically linked to the sheepdog library. Howto: $ gcc your_prog.c -lpthread /path/to/libsheepdog.a Note, 'make install' will install libsheepdog.a into /usr/lib/ along with libsheepdog.so. Signed-off-by: Liu Yuan --- lib/Makef

[sheepdog] [PATCH] sbd: use kernel_setsockopt helper

2015-05-04 Thread Liu Yuan
From: Liu Yuan Signed-off-by: Liu Yuan --- sbd/sheep.c | 13 - 1 file changed, 4 insertions(+), 9 deletions(-) diff --git a/sbd/sheep.c b/sbd/sheep.c index bfc935b..8b74b59 100644 --- a/sbd/sheep.c +++ b/sbd/sheep.c @@ -34,7 +34,6 @@ static struct sbd_device *sheep_aiocb_to_device

Re: [sheepdog] [PATCH 1/4] sheep: rename files for store_driver

2015-04-26 Thread Liu Yuan
On Mon, Apr 27, 2015 at 12:54:22PM +0900, Hitoshi Mitake wrote: > From: Masaki Saeki > > This change is a preparation patch for add store_driver. > Put files together to the new folder. > > Signed-off-by: Masaki Saeki Applied this series, thanks. Yuan -- sheepdog mailing list sheepdog@lists.

[sheepdog] [PATCH v2 3/3] shared lib: don't export unnecessary structures to user

2015-04-23 Thread Liu Yuan
From: Liu Yuan internal.h is dedicated to hold the non-user functions and structures for shared lib. Signed-off-by: Liu Yuan --- lib/shared/internal.h | 28 lib/shared/sheep.c| 1 + lib/shared/sheepdog.h | 11 --- lib/shared/vdi.c | 1 + 4 files

[sheepdog] [PATCH v2 2/3] shared lib: add the function description

2015-04-23 Thread Liu Yuan
From: Liu Yuan Signed-off-by: Liu Yuan --- lib/shared/sheepdog.h | 70 +-- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/lib/shared/sheepdog.h b/lib/shared/sheepdog.h index f0be852..04e2ed6 100644 --- a/lib/shared/sheepdog.h

[sheepdog] [PATCH v2 1/3] shared lib: return SD_RES_XXX instead of mixing std errno and sd error code

2015-04-23 Thread Liu Yuan
From: Liu Yuan Have all the sd_xxx functions return SD_RES_XXX, we will have a unified error handling for user program and more importantly, std error can't cover all the error cases that is specific to the sheepdog cluster, such as SD_RES_NO_VDI and SD_RES_NO_OBJ. Signed-off-by: Liu

[sheepdog] [PATCH 3/3] shared lib: don't export unnecessary structures to user

2015-04-23 Thread Liu Yuan
From: Liu Yuan internal.h is dedicated to hold the non-user functions and structures for shared lib. Signed-off-by: Liu Yuan --- lib/shared/internal.h | 28 lib/shared/sheep.c| 1 + lib/shared/sheepdog.h | 11 --- lib/shared/vdi.c | 1 + 4 files

[sheepdog] [PATCH 1/3] shared lib: return SD_RES_XXX instead of mixing std errno and sd error code

2015-04-23 Thread Liu Yuan
From: Liu Yuan Have all the sd_xxx functions return SD_RES_XXX, we will have a unified error handling for user program and more importantly, std error can't cover all the error cases that is specific to the sheepdog cluster, such as SD_RES_NO_VDI and SD_RES_NO_OBJ. Signed-off-by: Liu

[sheepdog] [PATCH 2/3] shared lib: add the function description

2015-04-23 Thread Liu Yuan
From: Liu Yuan Signed-off-by: Liu Yuan --- lib/shared/sheepdog.h | 70 +-- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/lib/shared/sheepdog.h b/lib/shared/sheepdog.h index f0be852..daa16e7 100644 --- a/lib/shared/sheepdog.h

Re: [sheepdog] [PATCH v2] fix systemd service to run sheep in foreground and with defaults

2015-04-13 Thread Liu Yuan
On Mon, Apr 13, 2015 at 01:32:56PM +0300, Vasiliy Tolstov wrote: > sheepdog now able to run in foreground, so use this in systemd service, > also remove running shell script to get defaults, because it provided > in service file now. > > Signed-off-by: Vasiliy Tolstov > --- > script/sheepdog.ser

Re: [sheepdog] [PATCH] lib/work: fix compiler warning

2015-04-13 Thread Liu Yuan
On Mon, Apr 13, 2015 at 02:35:02PM +0800, Ruoyu wrote: > This patch fixes the compiler warning message as below. > > warning: implicit declaration of function ‘install_sighandler’ > [-Wimplicit-function-declaration] > > Signed-off-by: Ruoyu > --- > lib/work.c | 1 + > 1 file changed, 1 insertio

Re: [sheepdog] sheepdog cluster data area node with bonding mode 4 with (Bonding Mode: IEEE 802.3ad Dynamic link aggregation) can't join gateway node

2015-04-12 Thread Liu Yuan
On Sun, Apr 12, 2015 at 07:38:09PM +0800, passedwind wrote: > today i change my sheepdog cluster data area node network with bonding mode > 4 with (Bonding Mode: IEEE 802.3ad Dynamic link aggregation).gateway node > with normal network interface. gateway node to dataArea node network link no >

Re: [sheepdog] [PATCH] sheep journal:fix the parameter type of 'size' passed to 'int prealloc(int fd, uint32_t size)'

2015-04-12 Thread Liu Yuan
On Thu, Apr 09, 2015 at 03:30:21PM +0800, 张灿群 wrote: > dear, > When i start sheep daemon with journal,the daemon couldn't start. > command: > sheep -j dir=/sdac,size=160G,skip=1 -l dir=/var/log/,level=err,format=server > /sdad/ -n -D -y 10.64.0.38 -p 7000 -c > zookeeper:host-0-31:2181,host-0-32

Re: [sheepdog] [PATCH v7] try to set RLIMIT_NOFILE to recommended value

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 09:43:10AM +0300, Vasiliy Tolstov wrote: > Signed-off-by: Vasiliy Tolstov > --- > sheep/sheep.c | 26 +++-- > 1 file changed, 14 insertions(+), 12 deletions(-) > > diff --git a/sheep/sheep.c b/sheep/sheep.c > index 86444b0..4f6a662 100644 > --- a/s

Re: [sheepdog] [PATCH v1] fix systemd service to run sheep in foreground and with defaults

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 09:26:05AM +0300, Vasiliy Tolstov wrote: > sheepdog now able to run in foreground, so use this in systemd service, > also remove running shell script to get defaults, because it provided > in service file now. > > Signed-off-by: Vasiliy Tolstov > --- > script/sheepdog.ser

Re: [sheepdog] [PATCH v6] try to set RLIMIT_NOFILE to recommended value

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 09:14:29AM +0300, Vasiliy Tolstov wrote: > Signed-off-by: Vasiliy Tolstov > --- > sheep/sheep.c | 26 +++-- > 1 file changed, 14 insertions(+), 12 deletions(-) > > diff --git a/sheep/sheep.c b/sheep/sheep.c > index 86444b0..38ffed0 100644 > --- a/s

[sheepdog] [PATCH v2 8/8] shared lib: kick it run

2015-04-07 Thread Liu Yuan
From: Liu Yuan Add the final makefile stuff Signed-off-by: Liu Yuan --- configure.ac| 4 +++- lib/Makefile.am | 14 ++ 2 files changed, 17 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 3635dc8..cae4771 100644 --- a/configure.ac +++ b

[sheepdog] [PATCH v2 7/8] shared lib: implement vdi_{open, read, write, close}

2015-04-07 Thread Liu Yuan
From: Liu Yuan Nothing intersting, these functions are basically wrappers of sheep.c. Signed-off-by: Liu Yuan --- lib/shared/sheep.c| 3 +- lib/shared/sheepdog.h | 10 ++- lib/shared/vdi.c | 185 ++ 3 files changed, 195 insertions

[sheepdog] [PATCH v2 6/8] shared lib: add the low level request handling framework

2015-04-07 Thread Liu Yuan
From: Liu Yuan The core idea is the same as sbd's framework. User request | | aio control block (struct aiocb) V +++ ||| r1 r2r3 ||| VVV obj obj obj

[sheepdog] [PATCH v2 5/8] unrelate list.h, util.h to compiler.h

2015-04-07 Thread Liu Yuan
From: Liu Yuan This is a preparation patch. list.h and util.h will be used by shared library and exported to the use. Signed-off-by: Liu Yuan --- include/compiler.h | 9 - include/list.h | 2 -- include/util.h | 14 +- lib/util.c | 3 +++ 4 files changed

[sheepdog] [PATCH v2 3/8] sheep: dissociate utils.c from logger.c

2015-04-07 Thread Liu Yuan
From: Liu Yuan This is a preparation patch for shared lib. shared lib can't call sd logger. Signed-off-by: Liu Yuan --- dog/dog.h| 1 + dog/treeview.c | 1 + include/fec.h| 1 + include/logger.h | 6 -- include/sheep.h | 1 + include/util.h

[sheepdog] [PATCH v2 4/8] move sheep specific headers out of sheepdog_proto.h

2015-04-07 Thread Liu Yuan
From: Liu Yuan This is preparation patch for shared lib. Signed-off-by: Liu Yuan --- include/sheep.h | 13 + include/sheepdog_proto.h | 16 +--- lib/sd_inode.c | 1 + 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/include/sheep.h b

[sheepdog] [PATCH v2 2/8] move functions used by libsheepdog.a out of util.c

2015-04-07 Thread Liu Yuan
From: Liu Yuan This is a prepation patch to make util.c self-contained and then can be used outside sheep. Signed-off-by: Liu Yuan --- dog/common.c| 25 + dog/dog.h | 5 + include/Makefile.am | 3 +- include/common.h| 33 ++ include/util.h | 18

[sheepdog] [PATCH v2 1/8] add sd_assert()

2015-04-07 Thread Liu Yuan
From: Liu Yuan This is a preparation patch to dissociate util.h and utli.c from sheepdog, in order to be used by shared lib. Signed-off-by: Liu Yuan --- configure.ac | 11 --- include/logger.h | 7 +++ include/util.h| 17 + lib

[sheepdog] [PATCH v2 0/8] introduce libsheepdog.so

2015-04-07 Thread Liu Yuan
From: Liu Yuan Finally, we kick started, though it is far away from a complete implemention. This patch set introduces a shared library and for now it only supports simple vdi operation. The vdi read/write operation would be the most difficult part of shared library, which require a high

Re: [sheepdog] [PATCH 2/8] move functions used by libsheepdog.a out of util.c

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 01:21:05AM +0900, Hitoshi Mitake wrote: > At Fri, 3 Apr 2015 11:20:48 +0800, > Liu Yuan wrote: > > > > From: Liu Yuan > > > > This is a prepation patch to make util.c self-contained and then can be used > > outside she

Re: [sheepdog] [PATCH 8/8] shared lib: kick it run

2015-04-07 Thread Liu Yuan
On Wed, Apr 08, 2015 at 01:19:20AM +0900, Hitoshi Mitake wrote: > At Fri, 3 Apr 2015 11:20:54 +0800, > Liu Yuan wrote: > > > > From: Liu Yuan > > > > Add the final makefile stuff > > > > Signed-off-by: Liu Yuan > > --- > &g

Re: [sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-03 Thread Liu Yuan
On Fri, Apr 03, 2015 at 03:59:18PM +0900, Hitoshi Mitake wrote: > At Fri, 3 Apr 2015 12:06:56 +0800, > Liu Yuan wrote: > > > > On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote: > > > At Fri, 3 Apr 2015 11:20:46 +0800, > > > Liu Yuan wr

Re: [sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-02 Thread Liu Yuan
On Fri, Apr 03, 2015 at 12:06:56PM +0800, Liu Yuan wrote: > On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote: > > At Fri, 3 Apr 2015 11:20:46 +0800, > > Liu Yuan wrote: > > > > > > From: Liu Yuan > > > > > > Finally, we

Re: [sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-02 Thread Liu Yuan
On Fri, Apr 03, 2015 at 12:49:31PM +0900, Hitoshi Mitake wrote: > At Fri, 3 Apr 2015 11:20:46 +0800, > Liu Yuan wrote: > > > > From: Liu Yuan > > > > Finally, we kick started, though it is far away from a complete > > implemention. > > > > Th

[sheepdog] [PATCH 7/8] shared lib: implement vdi_{open, read, write, close}

2015-04-02 Thread Liu Yuan
From: Liu Yuan Nothing intersting, these functions are basically wrappers of sheep.c. Signed-off-by: Liu Yuan --- lib/shared/sheep.c| 3 +- lib/shared/sheepdog.h | 10 ++- lib/shared/vdi.c | 185 ++ 3 files changed, 195 insertions

[sheepdog] [PATCH 8/8] shared lib: kick it run

2015-04-02 Thread Liu Yuan
From: Liu Yuan Add the final makefile stuff Signed-off-by: Liu Yuan --- configure.ac| 3 ++- lib/Makefile.am | 14 ++ 2 files changed, 16 insertions(+), 1 deletion(-) diff --git a/configure.ac b/configure.ac index 3635dc8..14fbc7a 100644 --- a/configure.ac +++ b/configure.ac

[sheepdog] [PATCH 0/9] introduce libsheepdog.so

2015-04-02 Thread Liu Yuan
From: Liu Yuan Finally, we kick started, though it is far away from a complete implemention. This patch set introduces a shared library and for now it only supports simple vdi operation. The vdi read/write operation would be the most difficult part of shared library, which require a high

[sheepdog] [PATCH 6/8] shared lib: add the low level request handling framework

2015-04-02 Thread Liu Yuan
From: Liu Yuan The core idea is the same as sbd's framework. User request | | aio control block (struct aiocb) V +++ ||| r1 r2r3 ||| VVV obj obj obj

[sheepdog] [PATCH 2/8] move functions used by libsheepdog.a out of util.c

2015-04-02 Thread Liu Yuan
From: Liu Yuan This is a prepation patch to make util.c self-contained and then can be used outside sheep. Signed-off-by: Liu Yuan --- dog/common.c| 25 + dog/dog.h | 5 + include/Makefile.am | 3 +- include/util.h | 18 lib/Makefile.am | 3 +- lib

[sheepdog] [PATCH 1/8] add sd_assert()

2015-04-02 Thread Liu Yuan
From: Liu Yuan This is a preparation patch to dissociate util.h and utli.c from sheepdog, in order to be used by shared lib. Signed-off-by: Liu Yuan --- configure.ac | 11 --- include/logger.h | 7 +++ include/util.h| 17 + lib

[sheepdog] [PATCH 4/8] move sheep specific headers out of sheepdog_proto.h

2015-04-02 Thread Liu Yuan
From: Liu Yuan This is preparation patch for shared lib. Signed-off-by: Liu Yuan --- include/sheep.h | 13 + include/sheepdog_proto.h | 16 +--- lib/sd_inode.c | 1 + 3 files changed, 15 insertions(+), 15 deletions(-) diff --git a/include/sheep.h b

[sheepdog] [PATCH 3/8] sheep: dissociate utils.c from logger.c

2015-04-02 Thread Liu Yuan
From: Liu Yuan This is a preparation patch for shared lib. shared lib can't call sd logger. Signed-off-by: Liu Yuan --- dog/dog.h| 1 + dog/treeview.c | 1 + include/fec.h| 1 + include/logger.h | 6 -- include/sheep.h | 1 + include/util.h

[sheepdog] [PATCH 5/8] unralate list.h, util.h to compiler.h

2015-04-02 Thread Liu Yuan
From: Liu Yuan This is a preparation patch. list.h and util.h will be used by shared library and exported to the use. Signed-off-by: Liu Yuan --- include/compiler.h | 9 - include/list.h | 2 -- include/util.h | 14 +- lib/util.c | 3 +++ 4 files changed

[sheepdog] [PATCH] lib: clean up lib/*.o files for 'make clean'

2015-04-02 Thread Liu Yuan
From: Liu Yuan This will kill a hidden build problem because of stale .o in libsheepdog.a Signed-off-by: Liu Yuan --- lib/Makefile.am | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-) diff --git a/lib/Makefile.am b/lib/Makefile.am index e1dfb1b..a630977 100644 --- a/lib/Makefile.am

Re: [sheepdog] [PATCH v4] try to set RLIMIT_NOFILE to recommended value

2015-03-25 Thread Liu Yuan
On Tue, Mar 17, 2015 at 05:30:47PM +0300, Vasiliy Tolstov wrote: > Signed-off-by: Vasiliy Tolstov > --- > sheep/sheep.c | 29 + > 1 file changed, 17 insertions(+), 12 deletions(-) > > diff --git a/sheep/sheep.c b/sheep/sheep.c > index fc53ec9..cad8293 100644 > --- a/s

Re: [sheepdog] [PATCH v2] add foreground option

2015-03-25 Thread Liu Yuan
On Wed, Mar 25, 2015 at 11:33:17AM +0300, Vasiliy Tolstov wrote: > 2015-03-25 5:13 GMT+03:00 Liu Yuan : > > If you just want to see log in stdout, you can try -l dst=stdout'. Any other > > reason to bring this back? > > > firstly under systemd forking not recomme

Re: [sheepdog] [PATCH v4] try to set RLIMIT_NOFILE to recommended value

2015-03-24 Thread Liu Yuan
On Tue, Mar 17, 2015 at 05:30:47PM +0300, Vasiliy Tolstov wrote: > Signed-off-by: Vasiliy Tolstov > --- > sheep/sheep.c | 29 + > 1 file changed, 17 insertions(+), 12 deletions(-) > > diff --git a/sheep/sheep.c b/sheep/sheep.c > index fc53ec9..cad8293 100644 > --- a/s

Re: [sheepdog] [PATCH v2] add foreground option

2015-03-24 Thread Liu Yuan
nd syslog support This patch adds a new parameter "dst=" to the "-l" option for specifying logging destination. Possible parameters are below: - default: a dedicated file under sheep's directory - syslog: standard syslog - stdout: standard output

Re: [sheepdog] [PATCH 3/4 v3] sheep : add new strage type "tree"

2015-03-22 Thread Liu Yuan
On Mon, Mar 23, 2015 at 02:18:24PM +0900, Saeki Masaki wrote: > On 2015/03/21 21:02, Liu Yuan wrote: > >On Fri, Mar 20, 2015 at 06:39:08PM +0900, Saeki Masaki wrote: > >>Current sheepdog stores whole objects in single directory like > >>"/var/lib/sheepdog/obj&quo

Re: [sheepdog] [PATCH 3/4 v3] sheep : add new strage type "tree"

2015-03-21 Thread Liu Yuan
On Fri, Mar 20, 2015 at 06:39:08PM +0900, Saeki Masaki wrote: > Current sheepdog stores whole objects in single directory like > "/var/lib/sheepdog/obj" > This mechanism is difficult to handle massive files when increasing cluster > volume. > > In particular, inode object having special informat

Re: [sheepdog] [PATCH 1/4 v3] sheep: rename files for store_driver

2015-03-21 Thread Liu Yuan
re/md.c \ > + store/plain_store.c \ > + config.c migrate.c > if BUILD_HTTP > sheep_SOURCES+= http/http.c http/kv.c http/s3.c http/swift.c > \ > diff --git a/sheep/md.c b/sheep/md.c > deleted file mode 100644 > index 87a

Re: [sheepdog] [PATCH 1/2] sheep: rename files for store_driver

2015-03-17 Thread Liu Yuan
diff --git a/sheep/md.c b/sheep/md.c > deleted file mode 100644 > index c00d7a5..000 > --- a/sheep/md.c > +++ /dev/null > @@ -1,878 +0,0 @@ > -/* > - * Copyright (C) 2013 Taobao Inc. > - * > - * Liu Yuan > - * > - * This program is free software; you can redistribute it and/

Re: [sheepdog] [PATCH 1/2 v2] sheep : add new strage type "tree"

2015-03-17 Thread Liu Yuan
On Tue, Mar 17, 2015 at 06:03:26PM +0900, Saeki Masaki wrote: > Current sheepdog stores whole objects in single directory like > "/var/lib/sheepdog/obj" > This mechanism is difficult to handle massive files when increasing cluster > volume. > > In particular, inode object having special informat

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:59:12PM +0900, Hitoshi Mitake wrote: > At Tue, 17 Mar 2015 13:54:16 +0800, > Liu Yuan wrote: > > > > On Tue, Mar 17, 2015 at 01:40:51PM +0800, Liu Yuan wrote: > > > On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: > > >

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:59:12PM +0900, Hitoshi Mitake wrote: > At Tue, 17 Mar 2015 13:54:16 +0800, > Liu Yuan wrote: > > > > On Tue, Mar 17, 2015 at 01:40:51PM +0800, Liu Yuan wrote: > > > On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: > > >

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:51:32PM +0900, Hitoshi Mitake wrote: > At Tue, 17 Mar 2015 13:40:51 +0800, > Liu Yuan wrote: > > > > On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: > > > On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: > > >

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:40:51PM +0800, Liu Yuan wrote: > On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: > > On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: > > > On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: > > > > At T

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:32:44PM +0800, Liu Yuan wrote: > On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: > > On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: > > > At Tue, 17 Mar 2015 13:58:08 +0900, > > > Hitoshi Mitake wrote: > > >

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:57:40PM +0900, Hitoshi Mitake wrote: > This patch adds a new option -R to "dog cluster format". If user > specifies the option during cluster format, the cluster will enable > recycling VID (disabled in default). > > Signed-off-by: Hitoshi Mitake > --- > dog/cluster.c

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:16:00PM +0800, Liu Yuan wrote: > On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: > > At Tue, 17 Mar 2015 13:58:08 +0900, > > Hitoshi Mitake wrote: > > > > > > At Tue, 17 Mar 2015 13:51:48 +0900, > > > Hitoshi Mit

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 02:03:03PM +0900, Hitoshi Mitake wrote: > At Tue, 17 Mar 2015 13:58:08 +0900, > Hitoshi Mitake wrote: > > > > At Tue, 17 Mar 2015 13:51:48 +0900, > > Hitoshi Mitake wrote: > > > > > > At Tue, 17 Mar 2015 12:42:35 +0800, > >

Re: [sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 08:49:34PM +0800, Liu Yuan wrote: > From: Liu Yuan > > This patch fixes following problem: > > $ dog vdi create test 100M > $ dog vdi delete test > $ dog vdi create test 200M > $ dog vdi list # expect show test, but nothing shows out. &

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 01:33:58PM +0900, Hitoshi Mitake wrote: > At Tue, 17 Mar 2015 11:06:34 +0800, > Liu Yuan wrote: > > > > On Tue, Mar 17, 2015 at 11:42:01AM +0900, Hitoshi Mitake wrote: > > > At Tue, 17 Mar 2015 10:03:53 +0800, > > > Liu Yuan wrote: >

Re: [sheepdog] 答复: [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 12:08:39PM +0800, redtone wrote: > Many people want do daily snapshot (delete the old one and create a new one > with the save name) This is not the case we talked about. Your case will work fine. I don't think you will delete all the snapshots since day 1 very frequently.

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 03:55:56AM +, morita...@nttdata.co.jp wrote: > > > > I'm still not sure which code in vdi_lookup() is a problem. The > > > > problem happens even when we disable VID garbage collection? > > > > > > vdi_lookup() becomes a problem if Hitoshi's patch is enabled after he >

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 10:03:53AM +0800, Liu Yuan wrote: > On Tue, Mar 17, 2015 at 04:44:46AM +0900, MORITA Kazutaka wrote: > > At Mon, 16 Mar 2015 21:13:29 +0800, > > Liu Yuan wrote: > > > > > > How about make 'dog vdi clone --no-share' as the defau

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 11:42:01AM +0900, Hitoshi Mitake wrote: > At Tue, 17 Mar 2015 10:03:53 +0800, > Liu Yuan wrote: > > > > On Tue, Mar 17, 2015 at 04:44:46AM +0900, MORITA Kazutaka wrote: > > > At Mon, 16 Mar 2015 21:13:29 +0800, > > > Liu Yuan wrote: >

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Tue, Mar 17, 2015 at 04:44:46AM +0900, MORITA Kazutaka wrote: > At Mon, 16 Mar 2015 21:13:29 +0800, > Liu Yuan wrote: > > > > How about make 'dog vdi clone --no-share' as the default clone operation? > > And > > we can add dog vdi clone --share t

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 09:37:45PM +0900, Hitoshi Mitake wrote: > On Mon, Mar 16, 2015 at 4:39 PM, Liu Yuan wrote: > > On Mon, Mar 16, 2015 at 03:55:43PM +0900, Hitoshi Mitake wrote: > >> At Mon, 16 Mar 2015 14:48:47 +0800, > >> Liu Yuan wrote: > >> > &g

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 09:39:35PM +0900, Hitoshi Mitake wrote: > On Mon, Mar 16, 2015 at 7:42 PM, Liu Yuan wrote: > > On Mon, Mar 16, 2015 at 03:57:40PM +0900, Hitoshi Mitake wrote: > >> This patch adds a new option -R to "dog cluster format". If user > >>

[sheepdog] [PATCH 6/6] sheep: clear bit in vdi_deleted if vdi is created

2015-03-16 Thread Liu Yuan
From: Liu Yuan This patch fixes following problem: $ dog vdi create test 100M $ dog vdi delete test $ dog vdi create test 200M $ dog vdi list # expect show test, but nothing shows out. Which was brought by *commit f68feab7edc0ded86701a2e902d85616b24942ab *Author: Saeki Masaki *Date: Wed

[sheepdog] [PATCH 4/6] Revert "tests/functional: update test scripts for VID GC feature"

2015-03-16 Thread Liu Yuan
From: Liu Yuan This reverts commit 5d005d41862647f6c7cdb7977f2611e61abc104b. Signed-off-by: Liu Yuan --- tests/functional/016.out | 12 ++-- tests/functional/044.out | 12 ++-- tests/functional/046.out | 6 +++--- tests/functional/086.out | 6 +++--- tests/functional/092.out

[sheepdog] [PATCH 5/6] Revert "sheep: don't recycle VDI ID"

2015-03-16 Thread Liu Yuan
From: Liu Yuan This reverts commit 21549a1bd4981fabcc09d062a647162127fe0637. Signed-off-by: Liu Yuan --- sheep/vdi.c | 7 +++ 1 file changed, 7 insertions(+) diff --git a/sheep/vdi.c b/sheep/vdi.c index dd01a20..af96548 100644 --- a/sheep/vdi.c +++ b/sheep/vdi.c @@ -1236,6 +1236,13

[sheepdog] [PATCH 1/6] Revert "sheep: garbage collect needless VIDs"

2015-03-16 Thread Liu Yuan
From: Liu Yuan This reverts commit 58f0d6dcdfa370df4456ed08cd32983731fc0c4e. Signed-off-by: Liu Yuan --- sheep/ops.c| 1 - sheep/sheep_priv.h | 1 - sheep/vdi.c| 110 - 3 files changed, 112 deletions(-) diff --git a

[sheepdog] [PATCH 3/6] Revert "sheep: let vdi state have parent VID"

2015-03-16 Thread Liu Yuan
From: Liu Yuan This reverts commit fcb91648b1a1f97c55c2a42fff1ec92b93bfea44. Signed-off-by: Liu Yuan --- include/internal_proto.h | 1 - sheep/group.c| 3 +-- sheep/ops.c | 7 +++ sheep/plain_store.c | 2 +- sheep/sheep_priv.h | 2 +- sheep/vdi.c

[sheepdog] [PATCH 2/6] Revert "sheep: construct data structures for representing family tree"

2015-03-16 Thread Liu Yuan
From: Liu Yuan This reverts commit b30374cf653c6bf93c91e14fe0954c58e4097637. Signed-off-by: Liu Yuan --- sheep/group.c | 7 ++- sheep/plain_store.c | 2 +- sheep/sheep_priv.h | 2 - sheep/vdi.c | 146 ++-- 4 files changed, 9

[sheepdog] [PATCH 0/6] fix vdi recycling

2015-03-16 Thread Liu Yuan
From: Liu Yuan This patch set mainly revert the patch set that broke the old vdi recycling algorithm that allow vdi to be recycled if it is deleted, though we keep deleted inodes in the storage because of vdi allocation algorithm assumes it. The old vid recycle work as following: 1. for non

Re: [sheepdog] [PATCH v2 1/2] sheep, dog: make recycling VID selectable

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:57:40PM +0900, Hitoshi Mitake wrote: > This patch adds a new option -R to "dog cluster format". If user > specifies the option during cluster format, the cluster will enable > recycling VID (disabled in default). > > Signed-off-by: Hitoshi Mitake > --- > dog/cluster.c

Re: [sheepdog] [PATCH] tools/zk_control: support multi-cluster

2015-03-16 Thread Liu Yuan
On Thu, Mar 12, 2015 at 02:08:13PM +0800, Meng Lingkun wrote: > From: Meng Lingkun > > Add option -c/--cluster to specify the cluster which zk_control > will handle. > Note: Using malloc rather than xmalloc because it can't be linked > under current makefile. Maybe fix it later. > > Signed-off-b

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:55:43PM +0900, Hitoshi Mitake wrote: > At Mon, 16 Mar 2015 14:48:47 +0800, > Liu Yuan wrote: > > > > On Mon, Mar 16, 2015 at 03:39:17PM +0900, Hitoshi Mitake wrote: > > > At Mon, 16 Mar 2015 14:31:44 +0800, > > > Liu Yuan wrote: >

Re: [sheepdog] [PATCH 1/2] sheep : add new strage type "tree"

2015-03-16 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:54:30PM +0900, Saeki Masaki wrote: > On 2015/03/16 11:06, Liu Yuan wrote: > >On Thu, Mar 12, 2015 at 01:50:26PM +0900, Saeki Masaki wrote: > >>Current sheepdog stores whole objects in single directory like > >>"/var/lib/sheepdog/obj&quo

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-15 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:39:17PM +0900, Hitoshi Mitake wrote: > At Mon, 16 Mar 2015 14:31:44 +0800, > Liu Yuan wrote: > > > > On Mon, Mar 16, 2015 at 02:08:25PM +0800, Liu Yuan wrote: > > > On Mon, Mar 16, 2015 at 02:36:57PM +0900, Hitoshi Mitake wrote: > >

Re: [sheepdog] [PATCH 1/2] sheep, dog: make recycling VID selectable

2015-03-15 Thread Liu Yuan
On Mon, Mar 16, 2015 at 03:24:31PM +0900, Hitoshi Mitake wrote: > This patch adds a new option -R to "dog cluster format". If user > specifies the option during cluster format, the cluster will enable > recycling VID (disabled in default). > > Signed-off-by: Hitoshi Mitake > --- > dog/cluster.c

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-15 Thread Liu Yuan
On Mon, Mar 16, 2015 at 02:08:25PM +0800, Liu Yuan wrote: > On Mon, Mar 16, 2015 at 02:36:57PM +0900, Hitoshi Mitake wrote: > > At Mon, 16 Mar 2015 10:21:50 +0800, > > Liu Yuan wrote: > > > > > > On Thu, Mar 12, 2015 at 08:14:33PM +0900, Hitoshi Mitake wrote: >

Re: [sheepdog] [PATCH v2 0/5] garbage collect needless VIDs and inode objects

2015-03-15 Thread Liu Yuan
On Mon, Mar 16, 2015 at 02:36:57PM +0900, Hitoshi Mitake wrote: > At Mon, 16 Mar 2015 10:21:50 +0800, > Liu Yuan wrote: > > > > On Thu, Mar 12, 2015 at 08:14:33PM +0900, Hitoshi Mitake wrote: > > > At Thu, 12 Mar 2015 14:41:56 +0800, > > > Liu Yuan wrote: >

  1   2   3   4   5   6   7   8   9   10   >