Jamie Wilkinson <j...@spacepants.org> writes:
> On 14 July 2010 23:14, Daniel Pittman <dan...@rimspace.net> wrote:
>
>> Sadly, I can't right now advise a better solution than these, however,
>> since it is the main problem I face in trying to bridge two data-centers
>> and provide coherent and sensible file access.
>
> I think you're going to be out of luck without some fat short pipes to
> satisfy fast atomic commits to both sides. The cloud way is to have your
> applications understand there's a replication delay and know how to deal
> with conflict resolutions, drop the atomicity and integrity constraints to
> gain some speed.
*nod* Factors I am well aware of, but thank you for being explicit about
them.
> I suspect from your mention of "file access" then you're not dealing with
> *an* application, but *all of them* and your storage layer API is just
> POSIX, in which case I wish you well in your pipe procurement endeavours.
Our needs vary wildly; in some cases we do want POSIX style file access,
with a remote mirror for read-mostly speed-of-access purposes, or where we
have a "write-mostly" application inside the one data center, with fail-over
to the other.
In others we are quite happy with an eventual-consistency. I mostly focused
on files here because suggesting that the OP investigate Reak or Cassandra
probably wouldn't fly when they wanted Microsoft Office to access it. ;)
> Random tangential brainstorm: if your application knew that your POSIX
> filesystem was being slowly replicated between two DCs, and knew to look in
> *both* for the same data, and was robust enough to handle the loss of one
> DC, then it ought to be able to pick up where it left off in the other DC
> modulo some journalling. Again I suspect this isn't going to help you in
> the slightest, not knowing anything about your app :)
Actually, that is pretty much the model I expect we will use for the "most
legacy" application in our stack, in which we are probably stuck for the next
year or so with "nothing but POSIX".
For the more agile applications my hope is that we can avoid that by, indeed,
having the applications aware of the replication issues and all, and using a
simple eventual-consistency or last-update-wins vector-clock approach.
Daniel
Also, lots of different apps, so I might well end up with multiple
solutions. A good distributed POSIX FS with replication, eventual
consistency, some sensible conflict resolution model, and data center
awareness would have been easy enough to use though.
If I could have my pony. ;)
--
✣ Daniel Pittman ✉ dan...@rimspace.net ☎ +61 401 155 707
♽ made with 100 percent post-consumer electrons
--
SLUG - Sydney Linux User's Group Mailing List - http://slug.org.au/
Subscription info and FAQs: http://slug.org.au/faq/mailinglists.html
