Update: DAS, CEP and MB are the three products that currently we have identified, which need point to point TCP connections for HA deployment.
*DAS and CEP:* This is required for event synchronization [1,2] using thrift transport. Had a offline chat with Sinthuja on this. As per the ThriftDataReceiver code [3], the thrift event receiver actually picks up the bind hostname from repository/conf/data-bridge/data-bridge-config.xml file. In that case, we can simply use the local IP for Thrift event receiver, and use the mapped host ip and port for the event synchronization configuration. *MB:* Required for slot co-ordination [4]. As the solution, we would need to define a bind ip + port and a separate public ip + port to advertise. Had a discussion regarding this with Ramith. If MB is using the same thrift data receiver [5], we might be able to use data-bridge-config.xml to define a bind ip. @Ramith: Can you please clarify this? [1]. https://docs.wso2.com/display/CLUSTER44x/Minimum+High+Availability+Deployment+-+DAS+3.0.1 [2]. https://docs.wso2.com/display/CLUSTER44x/Clustering+CEP+4.0.0#ClusteringCEP4.0.0-Highavailabilitydeployment [3]. https://github.com/wso2/carbon-analytics-common/blob/master/components/data-bridge/org.wso2.carbon.databridge.receiver.thrift/src/main/java/org/wso2/carbon/databridge/receiver/thrift/conf/ThriftDataReceiverConfiguration.java [4]. https://docs.wso2.com/display/CLUSTER44x/Configuring+the+Broker+Nodes [5]. https://github.com/wso2/carbon-analytics-common/blob/f3ff09d9655bf599d45436f0630b6413fc3d1b60/components/data-bridge/org.wso2.carbon.databridge.receiver.thrift/src/main/java/org/wso2/carbon/databridge/receiver/thrift/ThriftDataReceiver.java On Tue, Jul 12, 2016 at 1:52 PM, Udara Liyanage <ud...@wso2.com> wrote: > > > On Tue, Jul 12, 2016 at 12:56 PM, Isuru Haththotuwa <isu...@wso2.com> > wrote: > >> Hi all, >> >> This is to discuss about $subject. >> >> By default Mesos-Marathon [1] does not have an overlay network. Therefore >> when deploying wso2 products in distributed/HA manner in mesos, the >> products are unable to create point to point TCP connections directly from >> container to container. >> >> In Carbon products, p2p connections are created by Hazelcast, within >> cluster members. Since this is just for communication purposes, its >> possible to do a workaround by querying the API for the IPs of the host >> machine. This is done in the Mesos membership scheme [2]. >> >> There are few places where we need to create TCP Listeners, for an >> example in CEP HA/Distributed setup [3]. For this, we can't just bind to >> the host node IP since the public IP is not visible to the container; and >> can't use the local container IP since container to container communication >> is not possible (due to the lack of an overlay network). >> >> One possible option is such scenarios is to use the HOST networking mode >> of Docker [4]. Then the container will be using the host IP and ports >> directly. One issue with this approach is since we are mapping container >> ports to host ports directly without assigning random ports from the host >> side, only one container can run in one host node. But, in a way this >> defeats the purpose of containerization. >> > Using host mode would be a good alternative. I don't think limiting one > container per Mesos node (Mesos HOST=UNIQUE constrain) in this scenario > since a HA Mesos deployment will have multiple Mesos nodes. > Another advantage of running in host mode is performance efficiency rather > than bridge mode. Network efficiency would be useful in thrift scenario > since it is assumed to be fast. > Agreed. However, IMHO we should consider this as the last option if nothing else works out. > >> Please share your thoughts on this. >> >> [1]. https://github.com/mesosphere/marathon >> >> [2]. >> https://github.com/wso2/mesos-artifacts/tree/master/common/mesos-membership-scheme >> >> [3]. >> https://docs.wso2.com/display/CLUSTER44x/Clustering+CEP+4.0.0#ClusteringCEP4.0.0-Highavailabilitydeployment >> >> [4]. http://www.dasblinkenlichten.com/docker-networking-101-host-mode/ >> >> >> -- >> Thanks and Regards, >> >> Isuru H. >> +94 716 358 048 >> >> >> >> _______________________________________________ >> Dev mailing list >> Dev@wso2.org >> http://wso2.org/cgi-bin/mailman/listinfo/dev >> >> > > > -- > > Udara Liyanage > Software Engineer > WSO2, Inc.: http://wso2.com > lean. enterprise. middleware > > web: http://udaraliyanage.wordpress.com > phone: +94 71 443 6897 > -- Thanks and Regards, Isuru H. +94 716 358 048* <http://wso2.com/>*
_______________________________________________ Dev mailing list Dev@wso2.org http://wso2.org/cgi-bin/mailman/listinfo/dev
