On Tue, 2008-01-29 at 23:33 -0600, James Bennett wrote: > On Jan 29, 2008 11:18 PM, Mark Green <[EMAIL PROTECTED]> wrote: > > I agree on the loadbalancer front but the overhead for all > > those TCP connections (and pgpool managing them) worries me a bit. > > I've used pgpool in production with great success, so I'm not really > sure what overhead you're talking about. > > > Furthermore, and much more serious, I see no way to ensure > > graceful degration in case of overload. > > And here you completely change the topic of discussion from persistent > pooling of connections to failover when a database reaches its maximum > connection level, so I'm not really sure what it has to do with > anything...
Well, "Build for failure". Temporary overload can happen at any time and I'd expect django to behave exceptionally bad in that case as it is. The problem is that under load it will start displaying errors to users instead of just slowing down. Not nice, imho. There is no "buffer" (i.e. timeout, connection retries) between django and the [next hop to the] database. Disclaimer: I haven't actually tested this behaviour but I've seen it in JDBC apps before we added pooling and don't know why django should be different. These apps would basically "chop off" (i.e. return errors for) the excess percentile of requests. Naturally the affected users would use their "reload"-button and there we have a nice death spiral... > > So, long story short, I see no way out of this without > > proper connection pooling built right into django. > > Or am I missing something? > > You're missing the fact that you've switched from asking about pooling > to asking about failover. Hm, I wouldn't say this is about failover. It's about behaving gracefully under load. > Also, your solution would mean that: > > 1. Django must have its own configuration for the number of > connections it's allowed to use, how long to keep them alive and how > often to retry them in case of failure, and this must be updated if > and when use patterns change. Yup, connection pooling. > 2. Django must have its own configuration for being notified of what > every other client application of the same database is doing, and this > must be updated if and when use patterns change. > > 3. Every other client application of the same database must have > similar dual configuration to know what it's allowed to do and what > everybody else is doing, and these must be updated if and when use > patterns change. Not really. My desire is to make each individual django instance play well when things get crowded. Making them aware of each other, or even making all database clients aware of each other, sounds like an interesting project but is not what I'm after here. > Or you could just use a single external utility to manage database > connections, thus keeping all that essentially infrastructural cruft > out of the application layer while giving you a single place to > configure it and a single place to make changes when you need them. Well, there is a point where a single instance of the external utility doesn't cut it anymore. The only way to go seems to be one pgpool instance per django instance (for performance and to avoid the single point of failure). So there you have your "n configurations" again, only outside of django and without really solving the overload-problem. Maybe I'm blowing all this out of proportion but I wonder if any of the high-traffic, multi-server django sites ever ran into it? -mark --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
