Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/24/2012 01:37 PM, MORITA Kazutaka wrote: > It is a good idea to refine the farm backend store too, but simple > store should be added as another driver, I think. The simple driver > would become a good tutorial to introduce new drivers. Okay, I'd suggest use default_xxx as prefix instead of

Re: [sheepdog] snapshot rollback ?

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 08:35:25 +0200 (CEST), Alexandre DERUMIER wrote: > > >>If you run the following command, you can rollback the snapshot again. > >> > >>$ qemu sheepdog:image1:snap1 > >> > >>I think the 'collie vdi tree' command would help you. > > Thanks, But I don't understand how It can w

[sheepdog] [PATCH v3 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-24 Thread levin li
From: levin li Since we already send vdi copy list in get_vdis(), there's no need to send the vdi_inuse bitmap any more, without it we can send less data as the length of vdi copy list is variable and it contains the vid just as vdi_inuse does. Signed-off-by: levin li --- include/internal_prot

[sheepdog] [PATCH v3 2/2] collie: make `collie vdi list` not read vdi_inuse bitmap

2012-08-24 Thread levin li
From: levin li Signed-off-by: levin li --- collie/common.c| 32 include/sheep.h|5 + sheep/sheep_priv.h |5 - 3 files changed, 21 insertions(+), 21 deletions(-) diff --git a/collie/common.c b/collie/common.c index 364e36c..ce8dcf7 100

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 15:04:56 +0800, Liu Yuan wrote: > > On 08/24/2012 01:37 PM, MORITA Kazutaka wrote: > > It is a good idea to refine the farm backend store too, but simple > > store should be added as another driver, I think. The simple driver > > would become a good tutorial to introduce new d

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/24/2012 03:15 PM, MORITA Kazutaka wrote: > At Fri, 24 Aug 2012 15:04:56 +0800, > Liu Yuan wrote: >> >> On 08/24/2012 01:37 PM, MORITA Kazutaka wrote: >>> It is a good idea to refine the farm backend store too, but simple >>> store should be added as another driver, I think. The simple driver

Re: [sheepdog] snapshot rollback ?

2012-08-24 Thread Alexandre DERUMIER
Thanks for the explain, I'll do tests this week. Regards, Alexandre - Mail original - De: "MORITA Kazutaka" À: "Alexandre DERUMIER" Cc: "MORITA Kazutaka" , sheepdog@lists.wpkg.org Envoyé: Vendredi 24 Août 2012 09:08:51 Objet: Re: [sheepdog] snapshot rollback ? At Fri, 24 Aug 20

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/24/2012 03:28 PM, Liu Yuan wrote: > which is also relevant to which is also not relevant to -- sheepdog mailing list sheepdog@lists.wpkg.org http://lists.wpkg.org/mailman/listinfo/sheepdog

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/24/2012 03:28 PM, Liu Yuan wrote: >>> >> >>> >> I'll go to update the farm's io path to use defaul_xxx functions, I >>> >> think this simple store implementation can be nicely applied to farm. >> > >> > The update will change the data layout of farm. How about introducing >> > 'farm2'? I g

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 15:28:07 +0800, Liu Yuan wrote: > > I think simple_store.c itself won't cause confusion. 'default store' > will instead cause greater confusion. The reason I think 'simple' is not good is that we had a 'simple' driver before which had a completely different data layout. There

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/24/2012 03:37 PM, MORITA Kazutaka wrote: > The reason I think 'simple' is not good is that we had a 'simple' > driver before which had a completely different data layout. There may > be some users who still use a simple driver with old sheepdog version. > If they upgrade to the latest versio

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 15:40:14 +0800, Liu Yuan wrote: > > On 08/24/2012 03:37 PM, MORITA Kazutaka wrote: > > The reason I think 'simple' is not good is that we had a 'simple' > > driver before which had a completely different data layout. There may > > be some users who still use a simple driver wi

[sheepdog] [PATCH] sheep: add plain store driver

2012-08-24 Thread MORITA Kazutaka
This introduces a storage driver 'plain' based on the current storage interface. The design of the plain store is similar to one of the farm driver. The main difference is that the current farm uses the sha1 based backend store for stale objects, but simple_store uses a flat directory for them.

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/23/2012 10:58 PM, MORITA Kazutaka wrote: > +static int simple_store_read_from_path(uint64_t oid, char *path, > +struct siocb *iocb) > +{ > + int flags = def_open_flags, fd, ret = SD_RES_SUCCESS; > + ssize_t size; > + > + if (!is_data_obj(oid)) >

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 15:57:37 +0800, Liu Yuan wrote: > > On 08/23/2012 10:58 PM, MORITA Kazutaka wrote: > > +static int simple_store_read_from_path(uint64_t oid, char *path, > > + struct siocb *iocb) > > +{ > > + int flags = def_open_flags, fd, ret = SD_RES_SUCCES

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread Liu Yuan
On 08/24/2012 04:08 PM, MORITA Kazutaka wrote: > I think both farm and plain don't need it. Can you explain a scenario > where it can cause a problem? Qemu can issue multiple requests on the same oid, doesn't it? If there is some kind of write barrier to prevent reading stale oid or actually no p

Re: [sheepdog] [PATCH] sheep: add simple_store driver again

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 16:13:51 +0800, Liu Yuan wrote: > > On 08/24/2012 04:08 PM, MORITA Kazutaka wrote: > > I think both farm and plain don't need it. Can you explain a scenario > > where it can cause a problem? > > Qemu can issue multiple requests on the same oid, doesn't it? If there Yes, but

Re: [sheepdog] [PATCH] sheep: add plain store driver

2012-08-24 Thread Liu Yuan
On 08/24/2012 03:55 PM, MORITA Kazutaka wrote: > This introduces a storage driver 'plain' based on the current storage > interface. The design of the plain store is similar to one of the > farm driver. The main difference is that the current farm uses the > sha1 based backend store for stale obje

Re: [sheepdog] [PATCH v3 1/2] sheep: don't send vdi_inuse in get_vdis

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 15:11:37 +0800, levin li wrote: > > From: levin li > > Since we already send vdi copy list in get_vdis(), there's no > need to send the vdi_inuse bitmap any more, without it we can > send less data as the length of vdi copy list is variable and > it contains the vid just as v

[sheepdog] [PATCH 0/6] rework farm store

2012-08-24 Thread Liu Yuan
From: Liu Yuan Use plain implementation for core IO path. This greately reduce the complexity of farm recovery handling. With this patch, Farm will operate *entirely* the same as plain store for IO and recovery handling. Farm and Plain store will share core IO functions completely. Liu Yuan (6)

[sheepdog] [PATCH 2/6] plain: fix init_objlist_and_vdi_bitmap()

2012-08-24 Thread Liu Yuan
From: Liu Yuan We should add vdi copy number to copy list. Signed-off-by: Liu Yuan --- sheep/plain_store.c | 30 +- 1 file changed, 29 insertions(+), 1 deletion(-) diff --git a/sheep/plain_store.c b/sheep/plain_store.c index 02b69a5..4b322b4 100644 --- a/sheep/pl

[sheepdog] [PATCH 1/6] farm: use plain implementation for core IO path

2012-08-24 Thread Liu Yuan
From: Liu Yuan This greately reduce the complexity of farm recovery handling. With this patch, Farm will operate entirely the same as plain store for IO and recovery handling. Farm and Plain store will share core IO functions completely. Signed-off-by: Liu Yuan --- collie/cluster.c|2 +

[sheepdog] [PATCH 3/6] plain: clean up stale directory at startup

2012-08-24 Thread Liu Yuan
From: Liu Yuan When we start up, the objects in .stale is useless. This patch also reworks the ->cleanup() interface to reflect the current implementation. Signed-off-by: Liu Yuan --- sheep/ops.c |4 +--- sheep/plain_store.c |5 - sheep/sheep_priv.h |4 ++-- 3 files ch

[sheepdog] [PATCH 4/6] farm: use default_format for formatting

2012-08-24 Thread Liu Yuan
From: Liu Yuan - refine the interface to get name string directly Signed-off-by: Liu Yuan --- sheep/farm/farm.c | 26 +- sheep/ops.c |2 +- sheep/plain_store.c |3 +-- sheep/sheep_priv.h |4 ++-- 4 files changed, 5 insertions(+), 30 deletions(-)

[sheepdog] [PATCH 5/6] farm: rework trunk logic

2012-08-24 Thread Liu Yuan
From: Liu Yuan Since we move recovery handling out of farm backend, we don't need track IO requests on object, this leave most of functions in trunk.c useless for now. Signed-off-by: Liu Yuan --- sheep/farm/farm.c |5 +- sheep/farm/farm.h | 14 +- sheep/farm/trunk.c | 362 ++---

[sheepdog] [PATCH 6/6] test: add a test for cluster wide snapshot

2012-08-24 Thread Liu Yuan
From: Liu Yuan Signed-off-by: Liu Yuan --- tests/030 | 46 ++ tests/030.out |7 +++ tests/group |1 + 3 files changed, 54 insertions(+) create mode 100755 tests/030 create mode 100644 tests/030.out diff --git a/tests/030 b/tests

Re: [sheepdog] [PATCH 3/6] plain: clean up stale directory at startup

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 20:10:14 +0800, Liu Yuan wrote: > > From: Liu Yuan > > When we start up, the objects in .stale is useless. This patch also reworks > the > ->cleanup() interface to reflect the current implementation. > > Signed-off-by: Liu Yuan > --- > sheep/ops.c |4 +--- > s

Re: [sheepdog] [PATCH 3/6] plain: clean up stale directory at startup

2012-08-24 Thread Liu Yuan
在 2012-8-25 AM8:26,"MORITA Kazutaka" 写道: > > At Fri, 24 Aug 2012 20:10:14 +0800, > Liu Yuan wrote: > > > > From: Liu Yuan > > > > When we start up, the objects in .stale is useless. This patch also reworks the > > ->cleanup() interface to reflect the current implementation. > > > > Signed-off-by:

Re: [sheepdog] [PATCH 0/6] rework farm store

2012-08-24 Thread MORITA Kazutaka
At Fri, 24 Aug 2012 20:10:11 +0800, Liu Yuan wrote: > > From: Liu Yuan > > Use plain implementation for core IO path. This greately reduce the complexity > of farm recovery handling. With this patch, Farm will operate *entirely* the > same > as plain store for IO and recovery handling. Farm and

[sheepdog] [PATCH] test: consolidate 011 to support different filesystem type

2012-08-24 Thread Liu Yuan
From: Liu Yuan Signed-off-by: Liu Yuan --- tests/011 | 10 +- 1 files changed, 9 insertions(+), 1 deletions(-) diff --git a/tests/011 b/tests/011 index 781c4fb..b214c3c 100755 --- a/tests/011 +++ b/tests/011 @@ -31,8 +31,16 @@ dd if=/dev/zero of=$STORE/0.img seek=$((2 * 1024 ** 3 -

Re: [sheepdog] [PATCH] test: consolidate 011 to support different filesystem type

2012-08-24 Thread MORITA Kazutaka
At Sat, 25 Aug 2012 09:54:11 +0800, Liu Yuan wrote: > > From: Liu Yuan > > > Signed-off-by: Liu Yuan > --- > tests/011 | 10 +- > 1 files changed, 9 insertions(+), 1 deletions(-) Applied, thanks! Kazutaka -- sheepdog mailing list sheepdog@lists.wpkg.org http://lists.wpkg.org/mail

[sheepdog] [PATCH 1/2] tests: add test to check vdi list after cluster startup

2012-08-24 Thread MORITA Kazutaka
Signed-off-by: MORITA Kazutaka --- tests/031 | 35 +++ tests/031.out | 10 ++ tests/group |1 + 3 files changed, 46 insertions(+), 0 deletions(-) create mode 100755 tests/031 create mode 100644 tests/031.out diff --git a/tests/031 b/tests/0

[sheepdog] [PATCH 0/2] fix vdi disappear problem after cluster restart

2012-08-24 Thread MORITA Kazutaka
MORITA Kazutaka (2): tests: add test to check vdi list after cluster startup store: use O_DIRECT only for data objects sheep/plain_store.c | 31 --- tests/031 | 35 +++ tests/031.out | 10 ++ tests/group

[sheepdog] [PATCH 2/2] store: use O_DIRECT only for data objects

2012-08-24 Thread MORITA Kazutaka
init_vdi_copy_number() uses O_DIRECT for vdi objects too. This patch introduces get_open_flags to avoid making this kind of mistake any more. Signed-off-by: MORITA Kazutaka --- sheep/plain_store.c | 31 --- 1 files changed, 16 insertions(+), 15 deletions(-) diff -