From: Hitoshi Mitake mitake.hito...@gmail.com
Because we cannot utilize the journaling mechanism for config, this
patch implements atomic creation and writing for it.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
v2: use access(2) for checking remained temporal file
From: Hitoshi Mitake mitake.hito...@gmail.com
The journaling support for config cannot work well because it causes
deadlock in the initialization sequence of sheep.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
sheep/config.c | 14 +-
sheep/journal.c | 17
At Tue, 16 Apr 2013 11:17:04 +0800,
Liu Yuan wrote:
On 04/15/2013 11:16 PM, Hitoshi Mitake wrote:
+static void check_tmp_config(void)
+{
+ int fd;
+
+ fd = open(tmp_config_path, O_RDONLY);
+ if (fd 0 errno == ENOENT)
+ return;
+
FD leak.
+
On 04/16/2013 02:39 PM, Hitoshi Mitake wrote:
sd_iprintf(temporal file for config exists);
It is better rephrased as remove temporal config file and this is
nothing interesting for common users, so sd_dprintf() would be better.
+#define TMP_CONFIG_PATH /config.tmp
Add a global pathname isn't
We have to manipulate next_rw atomically because next_rw can be
accessed by both main and worker threads.
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
include/util.h | 8
sheep/recovery.c | 37 +++--
2 files changed, 27
We have some variables that can be accessed only in the main thread.
This patch introduces the macro 'thrad_unsafe' to guard it with
assert().
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
sheep/sheep_priv.h | 16
1 file changed, 16 insertions(+)
diff --git
We can call this function in the worker threads because urcu atomics
guard vnode_info.
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
sheep/group.c | 6 +-
1 file changed, 1 insertion(+), 5 deletions(-)
diff --git a/sheep/group.c b/sheep/group.c
index 98b916f..ee23588
This series makes sheep assert when it runs in the inappropriate
thread, and fixes problems detected by the assertion.
The first 3 patches are preparation and trivial fixes. The next 5
ones fix problems that we are accessing thread unsafe variables
outside the main thread. The last 3 ones fix
We usually call force operation even after the cluster starts up, so
this patch gives a chance for it to access vnode_info. With this
patch, get_vnode_info() returns NULL if cluster is not started yet.
This prepares for the next patch.
Signed-off-by: MORITA Kazutaka
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
sheep/work.c | 22 --
sheep/work.h | 1 +
2 files changed, 13 insertions(+), 10 deletions(-)
diff --git a/sheep/work.c b/sheep/work.c
index 2e1c8f0..495bd14 100644
--- a/sheep/work.c
+++ b/sheep/work.c
@@
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
sheep/sheep_priv.h | 10 ++
1 file changed, 10 insertions(+)
diff --git a/sheep/sheep_priv.h b/sheep/sheep_priv.h
index 4a3cd2e..6f19fc8 100644
--- a/sheep/sheep_priv.h
+++ b/sheep/sheep_priv.h
@@ -209,6 +209,16 @@ extern
We cannot call exec_req in cluster_force_recover (in the main thread).
With this patch, sheep gets epoch info in the worker thread, and
notifies it to all the node.
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
collie/cluster.c | 13 ---
sheep/group.c | 4 ++--
Currently, we assume that process_main sets the response data when
SD_FLAG_CMD_WRITE is not set. This also allows process_work to set
the data and notify it to all the nodes.
This prepares for the next patch.
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
sheep/group.c | 8
Calling these functions in the main thread lead to dead-lock and must
be avoided.
Signed-off-by: MORITA Kazutaka morita.kazut...@lab.ntt.co.jp
---
sheep/request.c | 2 ++
sheep/sockfd_cache.c | 2 ++
2 files changed, 4 insertions(+)
diff --git a/sheep/request.c b/sheep/request.c
index
On Tue, Apr 16, 2013 at 12:15:04AM +0800, Liu Yuan wrote:
@@ -727,6 +730,20 @@ static void coroutine_fn aio_read_response(void *opaque)
rsp.result = SD_RES_SUCCESS;
}
break;
+case AIOCB_DISCARD_OBJ:
+switch (rsp.result) {
+case
At Tue, 16 Apr 2013 14:52:14 +0800,
Liu Yuan wrote:
On 04/16/2013 02:39 PM, Hitoshi Mitake wrote:
sd_iprintf(temporal file for config exists);
It is better rephrased as remove temporal config file and this is
nothing interesting for common users, so sd_dprintf() would be better.
I agree
From: Hitoshi Mitake mitake.hito...@gmail.com
Because we cannot utilize the journaling mechanism for config, this
patch implements atomic creation and writing for it.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
v3:
- rephrase the message of check_tmp_config()
- remove a
From: Hitoshi Mitake mitake.hito...@gmail.com
The journaling support for config cannot work well because it causes
deadlock in the initialization sequence of sheep.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
sheep/config.c | 14 +-
sheep/journal.c | 17
From: Liu Yuan tailai...@taobao.com
wq_state isn't used at all. It is non-sense to keep it in the code.
Signed-off-by: Liu Yuan tailai...@taobao.com
---
sheep/work.c | 11 +--
1 file changed, 1 insertion(+), 10 deletions(-)
diff --git a/sheep/work.c b/sheep/work.c
index
From: Liu Yuan tailai...@taobao.com
Signed-off-by: Liu Yuan tailai...@taobao.com
---
sheep/trace/trace.c | 148 ++-
sheep/trace/trace.h | 10 ++--
sheep/work.c|6 +--
sheep/work.h|7 +--
4 files changed, 62 insertions(+),
From: Liu Yuan tailai...@taobao.com
Signed-off-by: Liu Yuan tailai...@taobao.com
---
collie/debug.c |3 +++
1 file changed, 3 insertions(+)
diff --git a/collie/debug.c b/collie/debug.c
index ace5737..83dbf8f 100644
--- a/collie/debug.c
+++ b/collie/debug.c
@@ -168,6 +168,9 @@ static int
From: Liu Yuan tailai...@taobao.com
Signed-off-by: Liu Yuan tailai...@taobao.com
---
collie/debug.c | 10 +-
include/event.h |2 +-
2 files changed, 2 insertions(+), 10 deletions(-)
diff --git a/collie/debug.c b/collie/debug.c
index 83dbf8f..34f2a69 100644
--- a/collie/debug.c
On 04/16/2013 04:28 PM, Hitoshi Mitake wrote:
+ snprintf(tmp_config_path, len, %s CONFIG_PATH .tmp, base_path);
Why not remove this tmp_config_path too? We should always use global
variable as less as much. Only write_config() and check_tmp_config()
need it, and both of them can deduce
Am 16.04.2013 um 10:18 hat Stefan Hajnoczi geschrieben:
On Tue, Apr 16, 2013 at 12:15:04AM +0800, Liu Yuan wrote:
@@ -727,6 +730,20 @@ static void coroutine_fn aio_read_response(void
*opaque)
rsp.result = SD_RES_SUCCESS;
}
break;
+case
At Tue, 16 Apr 2013 16:42:46 +0800,
Liu Yuan wrote:
On 04/16/2013 04:28 PM, Hitoshi Mitake wrote:
+ snprintf(tmp_config_path, len, %s CONFIG_PATH .tmp, base_path);
Why not remove this tmp_config_path too? We should always use global
variable as less as much. Only write_config() and
From: Hitoshi Mitake mitake.hito...@gmail.com
The journaling support for config cannot work well because it causes
deadlock in the initialization sequence of sheep.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
sheep/config.c | 14 +-
sheep/journal.c | 17
On 04/16/2013 04:47 PM, Kevin Wolf wrote:
Am 16.04.2013 um 10:18 hat Stefan Hajnoczi geschrieben:
On Tue, Apr 16, 2013 at 12:15:04AM +0800, Liu Yuan wrote:
@@ -727,6 +730,20 @@ static void coroutine_fn aio_read_response(void
*opaque)
rsp.result = SD_RES_SUCCESS;
On 04/16/2013 05:04 PM, Hitoshi Mitake wrote:
+ ret = rename(tmp_config_path, config_path);
+ if (ret 0) {
+ sd_eprintf(failed to rename, %m);
+ ret = SD_RES_EIO;
+ }
+
+ unlink(tmp_config_path);
rename already implies unlink() the old dentry.
On 04/16/2013 03:24 PM, MORITA Kazutaka wrote:
@@ -509,6 +509,8 @@ int sheep_exec_req(const struct node_id *nid, struct
sd_req *hdr, void *buf)
struct sockfd *sfd;
int ret;
+ assert(is_worker_thread());
+
sfd = sheep_get_sockfd(nid);
if (!sfd)
On 04/16/2013 03:24 PM, MORITA Kazutaka wrote:
+/* uatomic_xchg for pointers */
Need document what this function returns. And xchg isn't programmer
friendly. I'd suggest following comment:
Swaps the old value stored at location p with new value given by val.
Returns old value.
Thanks,
Yuan
--
At Wed, 17 Apr 2013 11:15:08 +0800,
Liu Yuan wrote:
On 04/16/2013 03:24 PM, MORITA Kazutaka wrote:
@@ -509,6 +509,8 @@ int sheep_exec_req(const struct node_id *nid, struct
sd_req *hdr, void *buf)
struct sockfd *sfd;
int ret;
+ assert(is_worker_thread());
+
sfd =
At Wed, 17 Apr 2013 11:05:26 +0800,
Liu Yuan wrote:
On 04/16/2013 05:04 PM, Hitoshi Mitake wrote:
+ ret = rename(tmp_config_path, config_path);
+ if (ret 0) {
+ sd_eprintf(failed to rename, %m);
+ ret = SD_RES_EIO;
+ }
+
+ unlink(tmp_config_path);
From: Hitoshi Mitake mitake.hito...@gmail.com
Because we cannot utilize the journaling mechanism for config, this
patch implements atomic creation and writing for it.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
v5: remove the unnecessary global unlink()
v4: remove the
From: Hitoshi Mitake mitake.hito...@gmail.com
The journaling support for config cannot work well because it causes
deadlock in the initialization sequence of sheep.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
sheep/config.c | 14 +-
sheep/journal.c | 17
On 04/16/2013 05:04 PM, Hitoshi Mitake wrote:
patch implements atomic creation and writing for it.
BTW, Could you abstract atomic_create_and_write(char *buf, size_t len)
in util.c?
I am cooking a patch which will use this too, so after applying your
patch, I can simply call it.
Thanks,
Yuan
--
On 04/17/2013 01:08 PM, Liu Yuan wrote:
BTW, Could you abstract atomic_create_and_write(char *buf, size_t len)
in util.c?
the prototype should be:
int atomic_create_and_write(char *path, char *buf, size_t len);
Thanks,
Yuan
--
sheepdog mailing list
sheepdog@lists.wpkg.org
At Wed, 17 Apr 2013 13:08:40 +0800,
Liu Yuan wrote:
On 04/16/2013 05:04 PM, Hitoshi Mitake wrote:
patch implements atomic creation and writing for it.
BTW, Could you abstract atomic_create_and_write(char *buf, size_t len)
in util.c?
I am cooking a patch which will use this too, so
On 04/17/2013 01:14 PM, Hitoshi Mitake wrote:
I agree with writing the function if there are two or more users. But
the abstraction will be a little bit difficult because of its nature:
we have to clean up temporal files during the initialization sequence.
Do you have an idea about this
At Wed, 17 Apr 2013 13:17:10 +0800,
Liu Yuan wrote:
On 04/17/2013 01:14 PM, Hitoshi Mitake wrote:
I agree with writing the function if there are two or more users. But
the abstraction will be a little bit difficult because of its nature:
we have to clean up temporal files during the
From: Hitoshi Mitake mitake.hito...@gmail.com
The journaling support for config cannot work well because it causes
deadlock in the initialization sequence of sheep.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
sheep/config.c | 14 +-
sheep/journal.c | 17
This patch adds a new API atomic_create_and_write(). As the name
implies, the API creates and writes data to a file.
atomic_create_and_write() creates a file path.tmp (path is a
parameter) internally. During the initialization process of sheep,
every user of this API should detect and clean
Because config cannot employ the journaling mechanism, this patchset
adds a new API, atomic_create_and_write() and let the config subsystem
use it for safe update of its data.
Hitoshi Mitake (3):
journal: remove journaling support for config
util:add a new API atomic_create_and_write()
From: Hitoshi Mitake mitake.hito...@gmail.com
Because config of sheep cannot use the journaling mechanism, it should
use atomic_create_and_write() for safe update of its data.
Signed-off-by: Hitoshi Mitake mitake.hito...@lab.ntt.co.jp
---
sheep/config.c | 32
On 04/17/2013 01:45 PM, Hitoshi Mitake wrote:
+ if (!access(tmp_path, F_OK))
+ panic(temporal file: %s exists, this implies invalid usage of
+ atomic_create_and_write() of sheepdog, tmp_path);
panic() is too stringent and not necessary. open(2) will return
On 04/17/2013 01:45 PM, Hitoshi Mitake wrote:
+ if (!access(tmp_config_path, F_OK))
+ return;
+
+ sd_iprintf(remove temporal config file);
+ unlink(tmp_config_path);
No need to access(tmp_config_path), just call unlink(tmp_config_path) is
sufficient.
Thanks,
Yuan
--
45 matches
Mail list logo
