* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
> On 2015/2/25 17:08, Dr. David Alan Gilbert wrote:
> >* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
> >>On 2015/2/16 20:03, Dr. David Alan Gilbert wrote:
> >>>* zhanghailiang (zhang.zhanghaili...@huawei.com) wrote:
> >>>>Implement colo nic device interface configure()
> >>>>add a script to configure nic devices:
> >>>>${QEMU_SCRIPT_DIR}/colo-proxy-script.sh
> >>>
> >>>Do you have some more documentation of the new colo-proxy? I've
> >>
> >>Yes, gaofeng is writing it now...
> >
> >Great.
> >
> >>>been reading the kernel module source and I can see that it's
> >>>a nice idea to do the sequence number adjustment on the host,
> >>>that reduces the need to modify the guest kernel; I was trying to
> >>>figure out how you synchronise the master/slave idea of sequence numbers -
> >>>is that purely from the 'ack' that's duplicated back to the secondary?
> >>
> >>Yes, you've got it :)
> >>
> >>>If you were unlucky and the 'ack' packet was lost on the duplicated
> >>>link from the primary to secondary how would you recover?
> >>
> >>The 'ack' packet will be consider to be lost, because the primary will not
> >>respond to this 'ack' packet until it got secondary's response,
> >>and client will resend it ('ack' packet).
> >>
> >>>What about TCP connections setup before colo was activated?
> >>>
> >>
> >>Actually, now, we only support activate colo before guest is startup (for
> >>test procedure,
> >>'-S' is needed for qemu command line).
> >
> >Consider this:
> > 1) Start primary
> > 2) Start secondary
> > 3) Start the colo pairing
> > 4) Primary fails
> > 5) Colo failover to secondary
> >
> >Now we have only the old secondary running; we'd really like to get back to
> >having a pair of fault-tolerant hosts, so it would be good to be able to:
> >
> > 6) Make the old secondary the new primary
> > 7) Add a new secondary
> > 8) Start colo-pairing to the new secondary
> >
>
> Er, what you described is continuous FT, yes, it is in our TODO list.
Ah, OK, I wondered what that meant.
Dave
> >You could theoretically do this with colo-agent, but not with colo-proxy.
> >
>
> We have decided using colo-proxy, it has more advantages.
>
> >>>The other thought is that passing the 'sec_dev' as a module parameter
> >>>gives you an artificial limitation; it forces all of the pairs
> >>>to be between the same pair of hosts. If the 'sec_dev' was a parameter
> >>>to the connection then you could have different slaves associated with
> >>>each guest on the primary host.
> >>>
> >>
> >>Hmm, do you mean we should pass this 'sec_dev' as a parameter from qemu to
> >>proxy module by
> >>maybe ioctl ?
> >
> >Yes, ioctl or tc or whatever; and make it per-guest.
> >
>
> OK. Thanks.
>
> >>Yes, it is ugly to pass this 'sec_dev' directly to module as parameter.
> >>We will consider this, thanks ;)
> >
> >Thanks!
> >
> >>>Dave
> >>>P.S. You probably need to clean the debug messages up in the kernel module!
> >>>
> >>
> >>OK, will do that.
> >
> >Thanks.
> >
> >Dave
> >
> >>
> >>>>Signed-off-by: zhanghailiang <zhang.zhanghaili...@huawei.com>
> >>>>Signed-off-by: Gao feng <gaof...@cn.fujitsu.com>
> >>>>Signed-off-by: Li Zhijian <lizhij...@cn.fujitsu.com>
> >>>>---
> >>>> net/colo-nic.c | 56 +++++++++++++++++++++++++++-
> >>>> scripts/colo-proxy-script.sh | 88
> >>>> ++++++++++++++++++++++++++++++++++++++++++++
> >>>> 2 files changed, 143 insertions(+), 1 deletion(-)
> >>>> create mode 100755 scripts/colo-proxy-script.sh
> >>>>
> >>>>diff --git a/net/colo-nic.c b/net/colo-nic.c
> >>>>index 965af49..f8fc35d 100644
> >>>>--- a/net/colo-nic.c
> >>>>+++ b/net/colo-nic.c
> >>>>@@ -39,12 +39,66 @@ static bool colo_nic_support(NetClientState *nc)
> >>>> return nc && nc->colo_script[0] && nc->colo_nicname[0];
> >>>> }
> >>>>
> >>>>+static int launch_colo_script(char *argv[])
> >>>>+{
> >>>>+ int pid, status;
> >>>>+ char *script = argv[0];
> >>>>+
> >>>>+ /* try to launch network script */
> >>>>+ pid = fork();
> >>>>+ if (pid == 0) {
> >>>>+ execv(script, argv);
> >>>>+ _exit(1);
> >>>>+ } else if (pid > 0) {
> >>>>+ while (waitpid(pid, &status, 0) != pid) {
> >>>>+ /* loop */
> >>>>+ }
> >>>>+
> >>>>+ if (WIFEXITED(status) && WEXITSTATUS(status) == 0) {
> >>>>+ return 0;
> >>>>+ }
> >>>>+ }
> >>>>+ return -1;
> >>>>+}
> >>>>+
> >>>>+static int colo_nic_configure(NetClientState *nc,
> >>>>+ bool up, int side, int index)
> >>>>+{
> >>>>+ int i, argc = 6;
> >>>>+ char *argv[7], index_str[32];
> >>>>+ char **parg;
> >>>>+
> >>>>+ if (!nc && index <= 0) {
> >>>>+ error_report("Can not parse colo_script or colo_nicname");
> >>>>+ return -1;
> >>>>+ }
> >>>>+
> >>>>+ parg = argv;
> >>>>+ *parg++ = nc->colo_script;
> >>>>+ *parg++ = (char *)(side == COLO_SECONDARY_MODE ? "slave" : "master");
> >>>>+ *parg++ = (char *)(up ? "install" : "uninstall");
> >>>>+ *parg++ = nc->colo_nicname;
> >>>>+ *parg++ = nc->ifname;
> >>>>+ sprintf(index_str, "%d", index);
> >>>>+ *parg++ = index_str;
> >>>>+ *parg = NULL;
> >>>>+
> >>>>+ for (i = 0; i < argc; i++) {
> >>>>+ if (!argv[i][0]) {
> >>>>+ error_report("Can not get colo_script argument");
> >>>>+ return -1;
> >>>>+ }
> >>>>+ }
> >>>>+
> >>>>+ return launch_colo_script(argv);
> >>>>+}
> >>>>+
> >>>> void colo_add_nic_devices(NetClientState *nc)
> >>>> {
> >>>> struct nic_device *nic = g_malloc0(sizeof(*nic));
> >>>>
> >>>> nic->support_colo = colo_nic_support;
> >>>>- nic->configure = NULL;
> >>>>+ nic->configure = colo_nic_configure;
> >>>> /*
> >>>> * TODO
> >>>> * only support "-netdev tap,colo_scripte..." options
> >>>>diff --git a/scripts/colo-proxy-script.sh b/scripts/colo-proxy-script.sh
> >>>>new file mode 100755
> >>>>index 0000000..c7aa53f
> >>>>--- /dev/null
> >>>>+++ b/scripts/colo-proxy-script.sh
> >>>>@@ -0,0 +1,88 @@
> >>>>+#!/bin/sh
> >>>>+#usage: ./colo-proxy-script.sh master/slave install/uninstall phy_if
> >>>>virt_if index
> >>>>+#.e.g ./colo-proxy-script.sh master install eth2 tap0 1
> >>>>+
> >>>>+side=$1
> >>>>+action=$2
> >>>>+phy_if=$3
> >>>>+virt_if=$4
> >>>>+index=$5
> >>>>+br=br1
> >>>>+failover_br=br0
> >>>>+
> >>>>+script_usage()
> >>>>+{
> >>>>+ echo -n "usage: ./colo-proxy-script.sh master/slave "
> >>>>+ echo -e "install/uninstall phy_if virt_if index\n"
> >>>>+}
> >>>>+
> >>>>+master_install()
> >>>>+{
> >>>>+ tc qdisc add dev $virt_if root handle 1: prio
> >>>>+ tc filter add dev $virt_if parent 1: protocol ip prio 10 u32 match
> >>>>u32 0 0 flowid 1:2 action mirred egress mirror dev $phy_if
> >>>>+ tc filter add dev $virt_if parent 1: protocol arp prio 11 u32 match
> >>>>u32 0 0 flowid 1:2 action mirred egress mirror dev $phy_if
> >>>>+ tc filter add dev $virt_if parent 1: protocol ipv6 prio 12 u32 match
> >>>>u32 0 0 flowid 1:2 action mirred egress mirror dev $phy_if
> >>>>+
> >>>>+ modprobe nf_conntrack_ipv4
> >>>>+ modprobe xt_PMYCOLO sec_dev=$phy_if
> >>>>+
> >>>>+ /usr/local/sbin/iptables -t mangle -I PREROUTING -m physdev
> >>>>--physdev-in $virt_if -j PMYCOLO --index $index
> >>>>+ /usr/local/sbin/ip6tables -t mangle -I PREROUTING -m physdev
> >>>>--physdev-in $virt_if -j PMYCOLO --index $index
> >>>>+ /usr/local/sbin/arptables -I INPUT -i $phy_if -j MARK --set-mark
> >>>>$index
> >>>>+}
> >>>>+
> >>>>+master_uninstall()
> >>>>+{
> >>>>+ tc filter del dev $virt_if parent 1: protocol ip prio 10 u32 match
> >>>>u32 0 0 flowid 1:2 action mirred egress mirror dev $phy_if
> >>>>+ tc filter del dev $virt_if parent 1: protocol arp prio 11 u32 match
> >>>>u32 0 0 flowid 1:2 action mirred egress mirror dev $phy_if
> >>>>+ tc filter del dev $virt_if parent 1: protocol ipv6 prio 12 u32 match
> >>>>u32 0 0 flowid 1:2 action mirred egress mirror dev $phy_if
> >>>>+ tc qdisc del dev $virt_if root handle 1: prio
> >>>>+
> >>>>+ /usr/local/sbin/iptables -t mangle -F
> >>>>+ /usr/local/sbin/ip6tables -t mangle -F
> >>>>+ /usr/local/sbin/arptables -F
> >>>>+ rmmod xt_PMYCOLO
> >>>>+}
> >>>>+
> >>>>+slave_install()
> >>>>+{
> >>>>+ brctl addif $br $phy_if
> >>>>+ modprobe xt_SECCOLO
> >>>>+
> >>>>+ /usr/local/sbin/iptables -t mangle -I PREROUTING -m physdev
> >>>>--physdev-in $virt_if -j SECCOLO --index $index
> >>>>+ /usr/local/sbin/ip6tables -t mangle -I PREROUTING -m physdev
> >>>>--physdev-in $virt_if -j SECCOLO --index $index
> >>>>+}
> >>>>+
> >>>>+
> >>>>+slave_uninstall()
> >>>>+{
> >>>>+ brctl delif $br $phy_if
> >>>>+ brctl delif $br $virt_if
> >>>>+ brctl addif $failover_br $virt_if
> >>>>+
> >>>>+ /usr/local/sbin/iptables -t mangle -F
> >>>>+ /usr/local/sbin/ip6tables -t mangle -F
> >>>>+ rmmod xt_SECCOLO
> >>>>+}
> >>>>+
> >>>>+if [ $# -ne 5 ]; then
> >>>>+ script_usage
> >>>>+ exit 1
> >>>>+fi
> >>>>+
> >>>>+if [ "x$side" != "xmaster" ] && [ "x$side" != "xslave" ]; then
> >>>>+ script_usage
> >>>>+ exit 2
> >>>>+fi
> >>>>+
> >>>>+if [ "x$action" != "xinstall" ] && [ "x$action" != "xuninstall" ]; then
> >>>>+ script_usage
> >>>>+ exit 3
> >>>>+fi
> >>>>+
> >>>>+if [ $index -lt 0 ] || [ $index -gt 100 ]; then
> >>>>+ echo "index overflow"
> >>>>+ exit 4
> >>>>+fi
> >>>>+
> >>>>+${side}_${action}
> >>>>--
> >>>>1.7.12.4
> >>>>
> >>>>
> >>>--
> >>>Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> >>>
> >>>.
> >>>
> >>
> >>
> >--
> >Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
> >
> >.
> >
>
>
--
Dr. David Alan Gilbert / dgilb...@redhat.com / Manchester, UK
