Don Dwiggins <d...@dondwiggins.net> writes: > Doing some exploration on the intersection of Twisted and SA, I came > across a message by David Bolen in February of 2007, describing a > "simple database class that contained a background thread for > execution". I'd like to know if that work, or some successor of it, has > proven useful and may be worth adopting.
Well, the server using it, in a slightly modified version from that message, remains in production, and has been continuously since July of 2007. So it's certainly worked for its intended purpose for me - that is, offloading the SA database I/O to a dedicated thread to avoid any possible blocking in the main Twisted reactor thread. In my application, the Twisted code handles web requests (public and internal) as well as PB connections from an internal client application (on OSX also using Twisted). Probably the biggest change between my message and putting it in production was to stop using the ORM. I found it too fragile to try to leverage ORM features while ensuring I didn't accidentally generate database I/O from the Twisted thread. It worked, but unless I eager loaded the world, if I then forgot and tried to reference a relation, bad things would happen :-) SA wasn't doing anything wrong, it was just a headache to manage while trying to use ORM features. And it wasn't giving me a ton of benefit anyway - most of my model objects were based on those I communicated across the Twisted PB connection anyway, so I already had to convert the SA ORM objects, so I just went back to using only the SA SQL layer instead. As comes up periodically on the Twisted list, I still don't think there's any good generalized solution for incorporating the ORM into Twisted - there's too much magic that happens inline during attribute access that invokes database layer I/O. I always considered it a solution to a very specific usage pattern that worked for my particular case, but not necessarily something generalized enough for inclusion elsewhere. It's pretty brute force; no ORM support (well, or be very careful if using the ORM), and each operation queued to the database thread needs to be self-contained (since ops from different twisted deferred chains could interleave over the same database connection). But within those constraints it works just fine. As you can see from that older message, there's not too much involved. It's basically a standard worker thread pattern using queues, with only a single worker thread, some knowledge about the SA engine and the use of Twisted Deferreds to return the result. -- David --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~----------~----~----~----~------~----~------~--~---
