On 2015/2/25 17:08, Dr. David Alan Gilbert wrote:
* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
On 2015/2/16 20:03, Dr. David Alan Gilbert wrote:
* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
Implement colo nic device interface configure()
add a script to configure nic devices:
${QEMU_SCRIPT_DIR}/colo-proxy-script.sh
Do you have some more documentation of the new colo-proxy? I've
Yes, gaofeng is writing it now...
Great.
been reading the kernel module source and I can see that it's
a nice idea to do the sequence number adjustment on the host,
that reduces the need to modify the guest kernel; I was trying to
figure out how you synchronise the master/slave idea of sequence numbers -
is that purely from the 'ack' that's duplicated back to the secondary?
Yes, you've got it :)
If you were unlucky and the 'ack' packet was lost on the duplicated
link from the primary to secondary how would you recover?
The 'ack' packet will be consider to be lost, because the primary will not
respond to this 'ack' packet until it got secondary's response,
and client will resend it ('ack' packet).
What about TCP connections setup before colo was activated?
Actually, now, we only support activate colo before guest is startup (for test
procedure,
'-S' is needed for qemu command line).
Consider this:
1) Start primary
2) Start secondary
3) Start the colo pairing
4) Primary fails
5) Colo failover to secondary
Now we have only the old secondary running; we'd really like to get back to
having a pair of fault-tolerant hosts, so it would be good to be able to:
6) Make the old secondary the new primary
7) Add a new secondary
8) Start colo-pairing to the new secondary
Er, what you described is continuous FT, yes, it is in our TODO list.
You could theoretically do this with colo-agent, but not with colo-proxy.
We have decided using colo-proxy, it has more advantages.
The other thought is that passing the 'sec_dev' as a module parameter
gives you an artificial limitation; it forces all of the pairs
to be between the same pair of hosts. If the 'sec_dev' was a parameter
to the connection then you could have different slaves associated with
each guest on the primary host.
Hmm, do you mean we should pass this 'sec_dev' as a parameter from qemu to
proxy module by
maybe ioctl ?
Yes, ioctl or tc or whatever; and make it per-guest.
OK. Thanks.
Yes, it is ugly to pass this 'sec_dev' directly to module as parameter.
We will consider this, thanks ;)
Thanks!
Dave
P.S. You probably need to clean the debug messages up in the kernel module!
OK, will do that.
Thanks.
Dave
Signed-off-by: zhanghailiang <zhang.zhanghaili...@huawei.com>
Signed-off-by: Gao feng <gaof...@cn.fujitsu.com>
Signed-off-by: Li Zhijian <lizhij...@cn.fujitsu.com>
---
net/colo-nic.c | 56 +++++++++++++++++++++++++++-
scripts/colo-proxy-script.sh | 88 ++++++++++++++++++++++++++++++++++++++++++++
2 files changed, 143 insertions(+), 1 deletion(-)
create mode 100755 scripts/colo-proxy-script.sh
diff --git a/net/colo-nic.c b/net/colo-nic.c
index 965af49..f8fc35d 100644
--- a/net/colo-nic.c
+++ b/net/colo-nic.c
@@ -39,12 +39,66 @@ static bool colo_nic_support(NetClientState *nc)
return nc && nc->colo_script[0] && nc->colo_nicname[0];
}
+static int launch_colo_script(char *argv[])
+{
+ int pid, status;
+ char *script = argv[0];
+
+ /* try to launch network script */
+ pid = fork();
+ if (pid == 0) {
+ execv(script, argv);
+ _exit(1);
+ } else if (pid > 0) {
+ while (waitpid(pid, &status, 0) != pid) {
+ /* loop */
+ }
+
+ if (WIFEXITED(status) && WEXITSTATUS(status) == 0) {
+ return 0;
+ }
+ }
+ return -1;
+}
+
+static int colo_nic_configure(NetClientState *nc,
+ bool up, int side, int index)
+{
+ int i, argc = 6;
+ char *argv[7], index_str[32];
+ char **parg;
+
+ if (!nc && index <= 0) {
+ error_report("Can not parse colo_script or colo_nicname");
+ return -1;
+ }
+
+ parg = argv;
+ *parg++ = nc->colo_script;
+ *parg++ = (char *)(side == COLO_SECONDARY_MODE ? "slave" : "master");
+ *parg++ = (char *)(up ? "install" : "uninstall");
+ *parg++ = nc->colo_nicname;
+ *parg++ = nc->ifname;
+ sprintf(index_str, "%d", index);
+ *parg++ = index_str;
+ *parg = NULL;
+
+ for (i = 0; i < argc; i++) {
+ if (!argv[i][0]) {
+ error_report("Can not get colo_script argument");
+ return -1;
+ }
+ }
+
+ return launch_colo_script(argv);
+}
+
void colo_add_nic_devices(NetClientState *nc)
{
struct nic_device *nic = g_malloc0(sizeof(*nic));
nic->support_colo = colo_nic_support;
- nic->configure = NULL;
+ nic->configure = colo_nic_configure;
/*
* TODO
* only support "-netdev tap,colo_scripte..." options
diff --git a/scripts/colo-proxy-script.sh b/scripts/colo-proxy-script.sh
new file mode 100755
index 0000000..c7aa53f
--- /dev/null
+++ b/scripts/colo-proxy-script.sh
@@ -0,0 +1,88 @@
+#!/bin/sh
+#usage: ./colo-proxy-script.sh master/slave install/uninstall phy_if virt_if
index
+#.e.g ./colo-proxy-script.sh master install eth2 tap0 1
+
+side=$1
+action=$2
+phy_if=$3
+virt_if=$4
+index=$5
+br=br1
+failover_br=br0
+
+script_usage()
+{
+ echo -n "usage: ./colo-proxy-script.sh master/slave "
+ echo -e "install/uninstall phy_if virt_if index\n"
+}
+
+master_install()
+{
+ tc qdisc add dev $virt_if root handle 1: prio
+ tc filter add dev $virt_if parent 1: protocol ip prio 10 u32 match u32 0 0
flowid 1:2 action mirred egress mirror dev $phy_if
+ tc filter add dev $virt_if parent 1: protocol arp prio 11 u32 match u32 0
0 flowid 1:2 action mirred egress mirror dev $phy_if
+ tc filter add dev $virt_if parent 1: protocol ipv6 prio 12 u32 match u32 0
0 flowid 1:2 action mirred egress mirror dev $phy_if
+
+ modprobe nf_conntrack_ipv4
+ modprobe xt_PMYCOLO sec_dev=$phy_if
+
+ /usr/local/sbin/iptables -t mangle -I PREROUTING -m physdev --physdev-in
$virt_if -j PMYCOLO --index $index
+ /usr/local/sbin/ip6tables -t mangle -I PREROUTING -m physdev --physdev-in
$virt_if -j PMYCOLO --index $index
+ /usr/local/sbin/arptables -I INPUT -i $phy_if -j MARK --set-mark $index
+}
+
+master_uninstall()
+{
+ tc filter del dev $virt_if parent 1: protocol ip prio 10 u32 match u32 0 0
flowid 1:2 action mirred egress mirror dev $phy_if
+ tc filter del dev $virt_if parent 1: protocol arp prio 11 u32 match u32 0
0 flowid 1:2 action mirred egress mirror dev $phy_if
+ tc filter del dev $virt_if parent 1: protocol ipv6 prio 12 u32 match u32 0
0 flowid 1:2 action mirred egress mirror dev $phy_if
+ tc qdisc del dev $virt_if root handle 1: prio
+
+ /usr/local/sbin/iptables -t mangle -F
+ /usr/local/sbin/ip6tables -t mangle -F
+ /usr/local/sbin/arptables -F
+ rmmod xt_PMYCOLO
+}
+
+slave_install()
+{
+ brctl addif $br $phy_if
+ modprobe xt_SECCOLO
+
+ /usr/local/sbin/iptables -t mangle -I PREROUTING -m physdev --physdev-in
$virt_if -j SECCOLO --index $index
+ /usr/local/sbin/ip6tables -t mangle -I PREROUTING -m physdev --physdev-in
$virt_if -j SECCOLO --index $index
+}
+
+
+slave_uninstall()
+{
+ brctl delif $br $phy_if
+ brctl delif $br $virt_if
+ brctl addif $failover_br $virt_if
+
+ /usr/local/sbin/iptables -t mangle -F
+ /usr/local/sbin/ip6tables -t mangle -F
+ rmmod xt_SECCOLO
+}
+
+if [ $# -ne 5 ]; then
+ script_usage
+ exit 1
+fi
+
+if [ "x$side" != "xmaster" ] && [ "x$side" != "xslave" ]; then
+ script_usage
+ exit 2
+fi
+
+if [ "x$action" != "xinstall" ] && [ "x$action" != "xuninstall" ]; then
+ script_usage
+ exit 3
+fi
+
+if [ $index -lt 0 ] || [ $index -gt 100 ]; then
+ echo "index overflow"
+ exit 4
+fi
+
+${side}_${action}
--
1.7.12.4
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
.
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
.
