On 01/18/2016 03:05 PM, Zhang Chen wrote: > > > On 01/06/2016 01:16 PM, Jason Wang wrote: >> >> On 01/04/2016 07:17 PM, Zhang Chen wrote: >>> >>> On 01/04/2016 05:46 PM, Jason Wang wrote: >>>> On 01/04/2016 04:16 PM, Zhang Chen wrote: >>>>> On 01/04/2016 01:37 PM, Jason Wang wrote: >>>>>> On 12/31/2015 04:40 PM, Zhang Chen wrote: >>>>>>> On 12/31/2015 10:36 AM, Jason Wang wrote: >>>>>>>> On 12/22/2015 06:42 PM, Zhang Chen wrote: >>>>>>>>> From: zhangchen <zhangchen.f...@cn.fujitsu.com> >>>>>>>>> >>>>>>>>> Hi,all >>>>>>>>> >>>>>>>>> This patch add an colo-proxy object, COLO-Proxy is a part of >>>>>>>>> COLO, >>>>>>>>> based on qemu netfilter and it's a plugin for qemu netfilter. the >>>>>>>>> function >>>>>>>>> keep Secondary VM connect normal to Primary VM and compare >>>>>>>>> packets >>>>>>>>> sent by PVM to sent by SVM.if the packet difference,notify >>>>>>>>> COLO do >>>>>>>>> checkpoint and send all primary packet has queued. >>>>>>>> Thanks for the work. I don't object this method but still not >>>>>>>> convinced >>>>>>>> that qemu is the best place for this. >>>>>>>> >>>>>>>> As been raised in the past discussion, it's almost impossible to >>>>>>>> cooperate with vhost backends. If we want this to be used in >>>>>>>> production >>>>>>>> environment, need to think of a solution for vhost. There's no >>>>>>>> such >>>>>>>> worry if we decouple this from qemu. >>>>>>>> >>>>>>>>> You can also get the series from: >>>>>>>>> >>>>>>>>> https://github.com/zhangckid/qemu/tree/colo-v2.2-periodic-mode-with-colo-proxyV2 >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> >>>>>>>>> Usage: >>>>>>>>> >>>>>>>>> primary: >>>>>>>>> -netdev tap,id=bn0 -device e1000,netdev=bn0 >>>>>>>>> -object >>>>>>>>> colo-proxy,id=f0,netdev=bn0,queue=all,mode=primary,addr=host:port >>>>>>>>> >>>>>>>>> secondary: >>>>>>>>> -netdev tap,id=bn0 -device e1000,netdev=bn0 >>>>>>>>> -object >>>>>>>>> colo-proxy,id=f0,netdev=bn0,queue=all,mode=secondary,addr=host:port >>>>>>>>> >>>>>>>> Have a quick glance at how secondary mode work. What it does is >>>>>>>> just >>>>>>>> forwarding packets between a nic and a socket, qemu socket >>>>>>>> backend did >>>>>>>> exact the same job. You could even use socket in primary node and >>>>>>>> let >>>>>>>> packet compare module talk to both primary and secondary node. >>>>>>> If we use qemu socket backend , the same netdev will used by qemu >>>>>>> socket and >>>>>>> qemu netfilter. this will against qemu net design. and then, when >>>>>>> colo >>>>>>> do failover, >>>>>>> secondary do not have backend to use. that's the real problem. >>>>>> Then, maybe it's time to implement changing the netdev of a nic. The >>>>>> point here is that what secondary mode did is in fact a netdev >>>>>> backend >>>>>> instead of a filter ... >>>>> Currently, you are right. in colo-proxy V2 code, I just compare IP >>>>> packet to >>>>> decide whether to do checkpoint. >>>>> But, in colo-proxy V3 I will compare tcp,icmp,udp packet to decide >>>>> it. >>>>> because that can reduce frequency of checkpoint and improve >>>>> performance. To keep tcp connection well, colo secondary need to >>>>> record >>>>> primary guest's init seq and adjust secondary guest's ack. if colo do >>>>> failover, >>>>> secondary also need do this to old tcp connection. qemu socket >>>>> can't do this job. >>>> So a question here: is it a must to do things (e.g TCP analysis >>>> stuffs) >>>> at secondary? Looks like we could do this at primary node. And I saw >>>> you're doing packet comparing in primary node, any advantages of doing >>>> this in primary instead of secondary? >>> We think must to do this in secondary, because if colo do >>> failover,secondary >>> must continues do TCP analysis stuffs to before tcp connection(if not, >>> tcp connection >>> will disconnect in that time), in this time primary already down or >>> disconnect to >>> secondary.so we can't make primary do this TCP analysis stuffs.it can >>> not ensure >>> FT function. >>> >>> Thanks >>> zhangchen >> Makes sense. >> >> Thanks > > Hi~, Jason. > No news for a week. > Can you give me some comments for code. > Let's make colo-proxy work well.
Sure. Two main comments/suggestions: - TCP analysis is missed in current version, maybe you point a git tree (or another version of RFC) to me for a better understanding of the design. (Just a skeleton for TCP should be sufficient to discuss). - I prefer to make the code as reusable as possible. So it's better to split/decouple the reusable parts from the codes. So a vague idea is: 1) Decouple the packet comparing from the netfilter. You've achieved this 99% since the work has been done in a thread. Just let the thread poll sockets directly, then the comparing have the possibility to be reused by other kinds of dataplane. 2) Implement traffic mirror/redirector as filter. 3) Implement TCP seq rewriting as a filter. Then, in primary node, you need just a traffic mirror, which did: - mirror ingress traffic to secondary node - mirror outgress traffic to packet comparing thread And in secondadry node, you need two filters: - A TCP seq rewriter which adjust tcp sequence number. - A traffic redirector which redirect packet from a socket as ingress traffic, and redirect outgress traffic to the socket which could be polled by remote packet comparing thread. Thoughts? Thanks > > Thanks > zhangchen