sorry they should be private mails 2011/4/5 dormando <dorma...@rydia.net>: > (First; Roberto I swear if you do that thing where you spam three e-mails > in a row one more time I'm blocking you from the list. To be honest I do > that to occasionally, but I limit myself to two responses and I try a lot > harder to be useful.) > > Mohit; Sorry for the confusion here. I hope you can see what happened and > that it was without malice: > > - User comes in with a vague request (I didn't even understand what "httpd > to cache" meant until just now) > - People who have solved this problem themselves answer with what works. I > can vouch for these people that there's a reason why they give these > answers. What you're suggesting isn't something you can innovate around, > it's something you can either hack or fail at. > > If you're willing I'd like to try one more time with a bit less vagueness, > so I've picked up your message here: > >> Currently there is no memcache. I was thinking of using memcache to do >> store user session per site info and redirect if user came to wrong >> site, but memcached seems to be not meant to be master/master or >> master/slave either. Generally industry practice is such use cases is >> to use cookies but we can't even use that because most of our apps are >> non-browser based app. Flow is something like this for eg: >> >> desktop client -> service X -> our service in either data center. >> >> So essentially client calls the server which then calls us. > > I'm going to take some shots in the dark, but it would be useful to have > even more information about the behavior of this cilent, so: > > You say that you want to prevent a client from hitting the wrong DC once > they're supposed to be "sticky". Are you saying that the desktop client > has to guess every single time it needs to make a request? ie; request > goes to random DC, then *redirects* to the correct one if it guessed > wrong? > > Is there any sense of keepalive? ie; if one request succeeds, is the > connection held open for further requests? Or do you have to do this > discovery every single time? > > Your description of non-browser-based thing is fuzzy as well. How much > control do you have over the service calling you? Cookies do in fact work > anywhere you bother to parse them, they're not particular to browsers :P > > So correct me if I'm wrong, but it looks like you're running a service > where "some client" makes a single HTTP request to you, but will honor > HTTP redirects, and you have zero ability to change the way this guy makes > requests? Or can you actually add some control here? > > So some shots in the dark: > > 1) You might even be able to do this with redirects and no client smarts. > The workflow would be something like: > > - Client sends WRITE req to http://happytimes.com/api/whatever > - RR DNS sends them to 192.168.5.1 > - 192.168.5.1 has no record of talking to the client before. It creates a > local session and stores a note about seeing this recent request, then it > redirects to the other DC (192.168.6.1) > - 192.168.6.1 does the same, and redirects back again (since it hasn't > seen this client before). It also notes that the original request had been > redirected once, so in the future it should bounce all requests back to > 192.168.5.1 > - 192.168.5.1 gets the request again, has a session which says it's seen > it before, and that this DC was the originator of the request, so then > serves back the request. > - Client sends second WRITE req in > - RR DNS sends them to 192.168.6.1 > - 192.168.6.1 has a note about that client, knows it's not the master, and > redirects back to 192.168.5.1, who then serves the request. > > * This *requires* a method of uniquely identifying the remote client > * This works best with RR dns. If you want to make a decision about > whether or not to handle the incoming client, the exchange gets trickier.. > If a DC has received a request it hasn't seen before, but doesn't want to > handle it, it has to redirect to a special url telling the other side to > become master and handle the request. > * This *requires* that if one DC has never seen a client before, both > *must* see the request before *either* serves it. > > * It's also a huge hack, and the extra redirects will cause annoying lag. > > 2) Use GeoDNS. > > I'm not sure why you said you couldn't. Something something 70% of users > in one area, 30% in another? I always thought it'd be a cute trick to mess > with a GeoIP database a bit. If 50% of my users are in california but my > servers are in texas and chicago, I can just hack the GeoIP database to > redistribute what areas get what responses (norcal to chicago, socal to > texas). So far as I know nothing stops you from dicking with the data. > > Some users will get worse latency, but if you wanted to serve all the > users without extra latency I'd *hopefully* assume your business has taken > this into account before deciding it's better to split traffic 50/50 and > doubling your chances of catastrophic downtime. > > 3) Add a teeny tiny bit of app smarts. When they send a write, read back > the cookie and use it :P But given my assumption about your client above, > I'm guessing this isn't possible. If business has some shitty reason for > not doing it, I would push harder unless you have a good way to dodge when > the hack put in place ultimately fails. > > 4) Assuming that the client hits a *random datacenter* *every single > time*, ALL OTHER OPTIONS, which use asynchronous replication, will have a > race condition failure. If you want to implement this, you *must* have the > source datacenter block the client's response until it has written the > session note to the remote datacenter. Perhaps you only need to do this > once per hour. > > 5) None of this will work if the client can make multiple requests at a > time, or if your service makes decisions based on *any* data that isn't > uniquely paired to that original client (like a feed list or twitter > timeline) > > 6) I can think of more variations of #1 while using backhauls, but tbh > they're all super gross. > > n' stuff. > -Dormando >
-- Roberto Spadim Spadim Technology / SPAEmpresarial
