On 2016/11/28 14:00, Changlong Xie wrote:
On 11/28/2016 01:13 PM, Hailiang Zhang wrote:

On 2016/10/25 17:03, Changlong Xie wrote:
On 10/20/2016 09:57 PM, zhanghailiang wrote:
Introuduce the scenario of shared-disk block replication
and how to use it.

Signed-off-by: zhanghailiang <zhang.zhanghaili...@huawei.com>
Signed-off-by: Wen Congyang <we...@cn.fujitsu.com>
Signed-off-by: Zhang Chen <zhangchen.f...@cn.fujitsu.com>
---
    docs/block-replication.txt | 131
+++++++++++++++++++++++++++++++++++++++++++--
    1 file changed, 127 insertions(+), 4 deletions(-)

diff --git a/docs/block-replication.txt b/docs/block-replication.txt
index 6bde673..97fcfc1 100644
--- a/docs/block-replication.txt
+++ b/docs/block-replication.txt
@@ -24,7 +24,7 @@ only dropped at next checkpoint time. To reduce the
network transportation
    effort during a vmstate checkpoint, the disk modification
operations of
    the Primary disk are asynchronously forwarded to the Secondary node.

-== Workflow ==
+== Non-shared disk workflow ==
    The following is the image of block replication workflow:

            +----------------------+
+------------------------+
@@ -57,7 +57,7 @@ The following is the image of block replication
workflow:
        4) Secondary write requests will be buffered in the Disk
buffer and it
           will overwrite the existing sector content in the buffer.

-== Architecture ==
+== None-shared disk architecture ==

s/None-shared/Non-shared/g


    We are going to implement block replication from many basic
    blocks that are already in QEMU.

@@ -106,6 +106,74 @@ any state that would otherwise be lost by the
speculative write-through
    of the NBD server into the secondary disk. So before block
replication,
    the primary disk and secondary disk should contain the same data.

+== Shared Disk Mode Workflow ==
+The following is the image of block replication workflow:
+
+        +----------------------+            +------------------------+
+        |Primary Write Requests|            |Secondary Write Requests|
+        +----------------------+            +------------------------+
+                  |                                       |
+                  |                                      (4)
+                  |                                       V
+                  |                              /-------------\
+                  | (2)Forward and write through |             |
+                  | +--------------------------> | Disk Buffer |
+                  | |                            |             |
+                  | |                            \-------------/
+                  | |(1)read                           |
+                  | |                                  |
+       (3)write   | |                                  | backing file
+                  V |                                  |
+                 +-----------------------------+       |
+                 | Shared Disk                 | <-----+
+                 +-----------------------------+
+
+    1) Primary writes will read original data and forward it to
Secondary
+       QEMU.
+    2) Before Primary write requests are written to Shared disk, the
+       original sector content will be read from Shared disk and
+       forwarded and buffered in the Disk buffer on the secondary site,
+       but it will not overwrite the existing

extra spaces at the end of line


+       sector content(it could be from either "Secondary Write
Requests" or

Need a space before "(" for better style.


+       previous COW of "Primary Write Requests") in the Disk buffer.
+    3) Primary write requests will be written to Shared disk.
+    4) Secondary write requests will be buffered in the Disk buffer
and it
+       will overwrite the existing sector content in the buffer.
+
+== Shared Disk Mode Architecture ==
+We are going to implement block replication from many basic
+blocks that are already in QEMU.
+         virtio-blk
||                               .----------
+             /
||                               | Secondary
+            /
||                               '----------
+           /
||                                 virtio-blk
+          /
||                                      |
+          |
||                               replication(5)
+          |                    NBD  -------->   NBD
(2)                       |
+          |                  client     ||    server ---> hidden
disk <-- active disk(4)
+          |                     ^       ||                      |
+          |              replication(1) ||                      |
+          |                     |       ||                      |
+          |   +-----------------'       ||                      |
+         (3)  |drive-backup sync=none   ||                      |
+--------. |   +-----------------+       ||                      |
+Primary | |                     |       ||           backing    |
+--------' |                     |       ||                      |
+          V                     |                               |
+       +-------------------------------------------+            |
+       |               shared disk                 | <----------+
+       +-------------------------------------------+
+
+
+    1) Primary writes will read original data and forward it to
Secondary
+       QEMU.
+    2) The hidden-disk buffers the original content that is modified
by the
+       primary VM. It should also be an empty disk, and

extra spaces at end of line


+       the driver supports bdrv_make_empty() and backing file.
+    3) Primary write requests will be written to Shared disk.
+    4) Secondary write requests will be buffered in the active disk
and it
+       will overwrite the existing sector content in the buffer.
+
    == Failure Handling ==
    There are 7 internal errors when block replication is running:
    1. I/O error on primary disk
@@ -145,7 +213,7 @@ d. replication_stop_all()
       things except failover. The caller must hold the I/O mutex lock
if it is
       in migration/checkpoint thread.

-== Usage ==
+== Non-shared disk usage ==
    Primary:
      -drive
if=xxx,driver=quorum,read-pattern=fifo,id=colo1,vote-threshold=1,\
             children.0.file.filename=1.raw,\
@@ -234,6 +302,61 @@ Secondary:
      The primary host is down, so we should do the following thing:
      { 'execute': 'nbd-server-stop' }

+== Shared disk usage ==

Keep the some coding style with "== Non-shared disk usage ==" part is
good to me.


+Primary:
+ -drive if=virtio,id=primary_disk0,file.filename=1.raw,driver=raw
+
+Issue qmp command:
+ {'execute': 'human-monitor-command',

two space indentation for the whole "{...}" part

+    'arguments': {
+        'command-line': 'drive_add-nbuddydriver=replication,

missing spaces

+        mode=primary,
+        file.driver=nbd,
+        file.host=9.42.3.17,
+        file.port=9998,
+        file.export=hidden_disk0,
+        shared-disk-id=primary_disk0,
+        shared-disk=on,
+        node-name=rep'


Keep the whole commands after "command-line" in one line, or you can
execute it correctly. IIRC


Hmm, i will change this hmp command to qmp 'blockdev-add' command in next
version, because it is supported now, though it is ready for production.


It's a good start, but i'm not sure here.

http://lists.nongnu.org/archive/html/qemu-devel/2016-11/msg01062.html


Yes, i noticed that, but for COLO, it is not ready for production either.
So I think it is OK to use it here ...

Thanks
        -Xie
+    }
+ }

Secondary:

+ -drive
if=none,driver=qcow2,file.filename=/mnt/ramfs/hidden_disk.img,id=hidden_disk0,\

+        backing.driver=raw,backing.file.filename=1.raw \
+ -drive if=virtio,id=active-disk0,driver=replication,mode=secondary,\
+        file.driver=qcow2,top-id=active-disk0,\
+        file.file.filename=/mnt/ramfs/active_disk.img,\
+        file.backing=hidden_disk0,shared-disk=on
+
+Issue qmp command:
+1. {'execute': 'nbd-server-start',
+    'arguments': {
+        'addr': {
+            'type': 'inet',
+            'data': {
+                'host': '0',

s/0/9.42.3.17/g, since you use designated ip address above


+                'port': '9998'
+            }
+        }
+    }
+   }
+2. {
+    'execute': 'nbd-server-add',
+    'arguments': {
+        'device': 'hidden_disk0',
+        'writable': true
+    }
+  }
+
+After Failover:
+Primary:
+{'execute': 'human-monitor-command',
+    'arguments': {
+        'command-line': 'drive_delrep'

drive_del rep


I'll use the qmp command instead here.

+    }
+}
+
+Secondary:
+  {'execute': 'nbd-server-stop' }
+
    TODO:
    1. Continuous block replication
-2. Shared disk



I will fix all the above problems in next version, thanks.



.




.




.



Reply via email to