:Thanks a lot for your reply Matt :-)
:
:Could you please tell how mirror-stream actually works?
:Does it work sequentially or as changes happen?
:
:By sequentially I mean If there are 100 files in the master then
:mirror-stream copies from file 1 to 100 to the slave so if mirror has
:passed from 1 to 50 and a change happens in file 25 it will be copied
:to the slave only after the mirror-stream has completed copying from
:50 till 100 and come back to file 25 from the begining.
:
:Again what happens if a file in the master and the corresponding file
:in the slave are identical? Does mirror-stream copy it again or leave
:it as it is?
:
:thanks
:
:Siju
Mirror-stream works on HAMMER's B-Tree elements. Whenever a HAMMER
flush occurs (every 30-60 seconds automatically or if you sync) any
B-Tree elements which are deleted, updated, or inserted are sent across
the wire to the slave.
This works transactionally. Mirror-stream keeps track of the highest
transaction id (TID) that it had successfully transfered and sends any
B-Tree elements with a higher TID across the wire. When it gets an
acknowledgement from the other side that they are on the slave's media
it updates its notion of the highest transaction id and repeats.
This is an incremental process. If a change is made to a file, such
as new data appended (say a log file), only the changes are sent across
the wire because only the B-Tree elements related to the new appends and
the inode update will have been updated. So, for example, this will
track a multi-gigabyte log file incrementally.
Since mirror-stream only operates on records flushed to the media on
the master one typically sees a little burst of activity every time
the HAMMER filesystem is flushed (30-60 seconds, or sooner if there is
a lot of activity). There is also a bandwidth-limiting option (-b)
that you can set to put a ceiling on the network bandwidth the
mirror-stream uses. And other parameters.
Also, the mirroring code uses a non-queued implementation. There are
no queue files involved which means the mirroring stream can be
interrupted at any time and restarted at any time, even if it is days
or months later, without losing track of its place. One can also mirror
to multiple slaves or chain-mirror between slaves, and one can implement
a different snapshot policy on the slaves then was on the master if
one desires.
You can use the verbose option (-v, -vv, -vvv, etc) to get more detail
of the streaming operation or the quiet option (-q) to make it not
output tracking info to stdout for use in a script.
If you use a script to maintain the mirror-stream you typically either
use cron to check if it is still running (using the lockf program
to prevent duplication), or you create a while (1) loop in the script
along with a long sleep (like sleep 300) to deal with any network
interruptions which terminate the mirror-stream operation.
-Matt
Matthew Dillon
<dil...@backplane.com>
