Thank you so much for your reply. This was very helpful to understand overall network partition scenario.
Best, On Tue, Dec 9, 2008 at 7:18 PM, Emmanuel Cecchet <[email protected]>wrote: > Hi Bogon, > > Well what happens depends on your timeout settings. It the failure last > less than the timeout, nothing will happen (the query will just wait for the > network to come back up). > If the timeout expires, the group communication will report that the other > controllers are dead and that you are now the only one in the world. Note > that if the same NIC is used on the controller for all communications > (database backends, clients and group communication), then connectivity will > be lost with all of them and the backends will be automatically disabled. > If only the group communication failed, when the network connectivity comes > back, the controllers gets notified of the new controllers in the group. > They will then compare their recovery log and find out if they were involved > in a partition. If no writes happened during the partition they will recover > smoothly otherwise only the first controller in the group will survive and > all others will shutdown. > > Hope this helps, > Emmanuel > > Thank you for your reply. >> >> If you have a time, could you tell me what happens in Sequoia when a >> controller suffers intermittent communication failure in network devices >> such as a switch or NIC? For example, for a couple of minutes, an NIC is not >> responded. I am not sure if this is possible failure scenario in real, >> though. >> Thank you for the paper list. >> >> >> On Mon, Dec 8, 2008 at 12:52 PM, Emmanuel Cecchet >> <[email protected]<mailto: >> [email protected]>> wrote: >> >> Hi, >> >> There is no solution to the partitioning problem if you need >> consistency. The first impossibility proof appeared in Brian A. >> Coan, Brian M. Oki, and Elliot K. Kolodner. "Limitations on >> Database Availability when Networks Partition." Proceedings of the >> Fifth Annual ACM Symposium on Principles of Distributed Computing >> (1986), pp. 187–194. >> If your machines are connected to the same network switch, they >> will all lose connectivity simultaneously if the switch goes down. >> You can then have 2 networks with something like heartbeat (the HA >> tool) for failover from one network to the other. >> If someone artificially introduces a network partition by >> misconfiguring the network (bad firewall or VPN setup) the you >> will have to do reconciliation (which cannot be done >> automatically). Some people have been thinking about the problem: >> Asplund, M. and Nadjm-Tehrani, S. 2006. Post-partition >> reconciliation protocols for maintaining consistency. In >> /Proceedings of the 2006 ACM Symposium on Applied Computing/ >> (Dijon, France, April 23 - 27, 2006). SAC '06. ACM, New York, NY, >> 710-717. >> >> In all cases, you have to try to avoid partitions by designing >> carefully your network configuration. You will only be able to >> detect a partition when it is too late and what you want to do for >> reconciliation is application specific. >> >> Hope this helps, >> Emmanuel >> >> I am running two controllers with RAIDB-1 scheme. But, the >> problem is when network is partitioned because of a switch >> failure. >> In this scenario, two controllers will receive user's requests >> from each. So, it will make two backends inconsistent. >> >From the Bianca's papers I reallized that this network >> partition very often can happen. >> >> When I look at this mailing-list, Emmanuel suggested to unify >> database communication path and user request path in network >> topology. >> But, in our case, databases are already communicating via >> gigabit network, seperated from outside communication channel. >> >> And, I read the "C-JDBC Horizontal Scalability: A controller >> replication user guide", and understood I need to make a JMX >> client which >> listens from JMX notification of Sequoia controllers. I think >> that this process will need to stop one controller, when two >> controllers are >> partitioned, or switch a backend database to the read-only mode. >> >> At this point, I have a quick question. I wonder if there is >> another better solution dealing with this sort of network >> partition in Sequoia. >> Thank you for your reading. >> >> >> Best, >> >> >> >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Sequoia mailing list >> [email protected] >> <mailto:[email protected]> >> https://forge.continuent.org/mailman/listinfo/sequoia >> >> >> >> -- Emmanuel Cecchet >> FTO @ Frog Thinker Open Source Development & Consulting >> -- >> Web: http://www.frogthinker.org >> email: [email protected] <mailto:[email protected]> >> Skype: emmanuel_cecchet >> >> _______________________________________________ >> Sequoia mailing list >> [email protected] >> <mailto:[email protected]> >> https://forge.continuent.org/mailman/listinfo/sequoia >> >> >> >> >> -- >> 여호와는 네게 복을 주시고 너를 지키시기를 원하며 >> 여호와는 그 얼굴을 네게 비추사 은혜 베푸시기를 원하며 >> 여호와는 그 얼굴을 네게로 향하여 드사 평강 주시기를 원하노라 >> (민수기 6:24-26) >> ------------------------------------------------------------------------ >> >> _______________________________________________ >> Sequoia mailing list >> [email protected] >> https://forge.continuent.org/mailman/listinfo/sequoia >> > > > -- > Emmanuel Cecchet > FTO @ Frog Thinker Open Source Development & Consulting > -- > Web: http://www.frogthinker.org > email: [email protected] > Skype: emmanuel_cecchet > > _______________________________________________ > Sequoia mailing list > [email protected] > https://forge.continuent.org/mailman/listinfo/sequoia > -- 여호와는 네게 복을 주시고 너를 지키시기를 원하며 여호와는 그 얼굴을 네게 비추사 은혜 베푸시기를 원하며 여호와는 그 얼굴을 네게로 향하여 드사 평강 주시기를 원하노라 (민수기 6:24-26)
_______________________________________________ Sequoia mailing list [email protected] https://forge.continuent.org/mailman/listinfo/sequoia
