On Wed, Aug 5, 2009 at 10:12 PM, wam<wamc...@gmail.com> wrote: > > I am using the Django ORM to build and manipulate models from outside > of a web context and within a long running daemonized application. I'm > using the django ORM talking to a postgresql db for this processing > since the results of this external processing are then used by a > Django based web reporting application and it was exceptionally > convenient to have a library that acted on these objects in a > consistent way regardless of whether it was called from the daemon vs > from a web request. > > I've been seeing more and more database errors occurring within the > external processor. For example, I get a lot of these: > InternalError: current transaction is aborted, commands ignored > until end of transaction block > > I get a few of these: > InternalError: SET TRANSACTION ISOLATION LEVEL must be called before > any query > > The postgresql logs are not very descriptive as to what the root > problem is: > 2009-08-04 14:04:08 EDT STATEMENT: BEGIN; SET TRANSACTION ISOLATION > LEVEL READ COMMITTED > 2009-08-04 14:04:14 EDT ERROR: current transaction is aborted, > commands ignored until end of transaction block > > The research I've seen so far leads me to think the problem is that > the ORM is building transactions that ordinarily would be closed at > the end of a view, but are being kept open in my case since I really > don't have a 'view' life-cycle. I've tried decorating my db-hitting > routines in the daemon code with django.db.transaction.commit_manually > along with appropriate calls to commit() and rollback() in case of > error, yet the problem persists. I've read the docs (and some of the > code) associated with transactions, yet the focus (naturally) is on > views and functions that have a typical request/response lifecycle. > I'm wondering what suggestions people would have for using the ORM in > a long running daemon (e.g. should I be closing the db connection > periodically? how would you suggest managing the transactions?). > > The basic flow of my code is: > daemonize my process > while True: > listen for an external event (e.g. from the net) > process that event into model based objects > save the objects > > Any help would be appreciated.
What you have described shouldn't require any particularly special handling. I have code in production that does exactly the same thing - a daemonized process that responds to requests on a port, does some processing, and saves the result to the database. You shouldn't need to do anything exotic with connection or transaction handling. As for the cause of your problem - my immediate thought is that your processing+save steps aren't as simple as you think they are. In particular, if you're getting: > InternalError: current transaction is aborted, commands ignored > until end of transaction block then something is going wrong in your code - somewhere, an error isn't being handled properly. However, without specifics, it's hard to narrow down the problem. Time to crack open the debugger to see exactly what sequence of events is leading to errors. The only other alarm bell that is ringing for me is that your errors are related to SET ISOLATION LEVEL requests, and you're using Postgres. Out of interest - are you using the autocommit feature added to Django in v1.1? If so, you might want to investigate the implications of ticket #10509 [1]. I'm not saying that this is your problem, but it might be worth look. It's also possible that you have found a new problem - database-level autocommit is a very new feature, so it hasn't been as thoroughly exercised as other parts of Django. [1] http://code.djangoproject.com/ticket/10509 Yours, Russ Magee %-) --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/django-users?hl=en -~----------~----~----~----~------~----~------~--~---
