Re: Django, Postgres and recovering from database constraints
ok, thanks a lot.
savepoints saved my day :p
It just worked out of the box while the other methods didn't seem to work.
Back on a more serious topic, could someone confirm or not that the transaction
rollback
(http://docs.djangoproject.com/en/1.2/topics/db/transactions/#transaction-rollback)
doesn't need extra setting ?
This part doesn't say it requires an explicit transaction management, so I'd
like to know whether the documentation would benefit a minor update or if it's
a bug due to multi database.
Xavier.
Le 1 févr. 2011 à 20:15, Xavier Ordoquy a écrit :
> thanks,
>
> I'll give it a try tomorrow and let you know.
>
> Xavier.
>
> Le 1 févr. 2011 à 19:57, Casey S. Greene a écrit :
>
>> Also (sorry for the follow-up spam but I just remembered this) if you are
>> hoping to use the database level autocommit (postgres 8.2 or later), you
>> need to configure it:
>>
>> Autocommit mode
>> New in Django 1.1: Please, see the release notes
>>
>> If your application is particularly read-heavy and doesn’t make many
>> database writes, the overhead of a constantly open transaction can sometimes
>> be noticeable. For those situations, if you’re using the postgresql_psycopg2
>> backend, you can configure Django to use “autocommit” behavior for the
>> connection, meaning that each database operation will normally be in its own
>> transaction, rather than having the transaction extend over multiple
>> operations. In this case, you can still manually start a transaction if
>> you’re doing something that requires consistency across multiple database
>> operations. The autocommit behavior is enabled by setting the autocommit key
>> in the OPTIONS part of your database configuration in DATABASES:
>>
>> 'OPTIONS': {
>> 'autocommit': True,
>> }
>>
>> from
>> http://docs.djangoproject.com/en/1.2/ref/databases/
>> --
>> Casey
>>
>> On 02/01/2011 01:52 PM, Casey S. Greene wrote:
>>> Here is some code pulled from my (using postgres) django application
>>> that recovers fine. Perhaps this is helpful to you. I am storing the
>>> non-unique values and dealing with them later (pulling from an external
>>> source that is supposed to have unique IDs assigned but they don't
>>> always pan out somehow...).
>>>
>>> from django.db import IntegrityError, transaction
>>>
>>> #Other Code
>>> ...
>>> #relevant code
>>> try:
>>> sid = transaction.savepoint()
>>> foo = FooModel(fooarg=fooarg, ...)
>>> foo.save()
>>> except IntegrityError:
>>> transaction.savepoint_rollback(sid)
>>> non_unique_foo.append((FooDB, FooID))
>>>
>>> Hope this helps!
>>> Casey
>>>
>>> On 02/01/2011 11:48 AM, Xavier Ordoquy wrote:
Le 1 févr. 2011 à 15:59, bruno desthuilliers wrote :
> On 1 fév, 15:24, Xavier Ordoquy wrote:
>> Hi all,
>>
>> I got a project which sometime outputs database integrity errors
>> (key violates unique constraint).
>> It is fine since I'm keeping an events table which may have several
>> creation messages for the same object.
>
> Hmmm... Either I misunderstood you or you have a design problem here.
You probably miss the big picture (ie, several legacy databases, each
with their own models plus synchronization between them and a central
django database).
> But anyway:
>
>> Until now, I was just catching the exception and forgot about it.
>>
>> Today, I'm not sure this is the right way.
>> If the exception in itself is correctly processed, I noticed that
>> any database operation behind raises another exception saying that
>> postgresql hates me and won't perform any other operation (current
>> transation is aborted, commands ignored until end of transaction
>> block.).
>
>> I already tried to end the transaction, but django will let me know
>> I'm not within a transaction
>
> Hu ?
>
> I assume you did read this:
> http://docs.djangoproject.com/en/dev/topics/db/transactions
>
> and specially this part:
> http://docs.djangoproject.com/en/dev/topics/db/transactions/#handling-exceptions-within-postgresql-transactions
>
Actually, I posted because this just doesn't seem to work.
If I try to rollback the transaction, I just get a nice error that
I'not within a transaction.
I haven't tested savepoints, but transaction rollback and autocommit
doesn't seem to work here.
I'll probably write a sample project this week to point the issue.
Regards,
Xavier.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Django users" group.
>> To post to this group, send email to django-users@googlegroups.com.
>> To unsubscribe from this group, send email to
>> django-users+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/django-users?hl=en.
>>
>
> --
> You received this message because you are subscribe
Re: Django, Postgres and recovering from database constraints
thanks,
I'll give it a try tomorrow and let you know.
Xavier.
Le 1 févr. 2011 à 19:57, Casey S. Greene a écrit :
> Also (sorry for the follow-up spam but I just remembered this) if you are
> hoping to use the database level autocommit (postgres 8.2 or later), you need
> to configure it:
>
> Autocommit mode
> New in Django 1.1: Please, see the release notes
>
> If your application is particularly read-heavy and doesn’t make many database
> writes, the overhead of a constantly open transaction can sometimes be
> noticeable. For those situations, if you’re using the postgresql_psycopg2
> backend, you can configure Django to use “autocommit” behavior for the
> connection, meaning that each database operation will normally be in its own
> transaction, rather than having the transaction extend over multiple
> operations. In this case, you can still manually start a transaction if
> you’re doing something that requires consistency across multiple database
> operations. The autocommit behavior is enabled by setting the autocommit key
> in the OPTIONS part of your database configuration in DATABASES:
>
> 'OPTIONS': {
>'autocommit': True,
> }
>
> from
> http://docs.djangoproject.com/en/1.2/ref/databases/
> --
> Casey
>
> On 02/01/2011 01:52 PM, Casey S. Greene wrote:
>> Here is some code pulled from my (using postgres) django application
>> that recovers fine. Perhaps this is helpful to you. I am storing the
>> non-unique values and dealing with them later (pulling from an external
>> source that is supposed to have unique IDs assigned but they don't
>> always pan out somehow...).
>>
>> from django.db import IntegrityError, transaction
>>
>> #Other Code
>> ...
>> #relevant code
>> try:
>> sid = transaction.savepoint()
>> foo = FooModel(fooarg=fooarg, ...)
>> foo.save()
>> except IntegrityError:
>> transaction.savepoint_rollback(sid)
>> non_unique_foo.append((FooDB, FooID))
>>
>> Hope this helps!
>> Casey
>>
>> On 02/01/2011 11:48 AM, Xavier Ordoquy wrote:
>>>
>>> Le 1 févr. 2011 à 15:59, bruno desthuilliers wrote :
On 1 fév, 15:24, Xavier Ordoquy wrote:
> Hi all,
>
> I got a project which sometime outputs database integrity errors
> (key violates unique constraint).
> It is fine since I'm keeping an events table which may have several
> creation messages for the same object.
Hmmm... Either I misunderstood you or you have a design problem here.
>>>
>>> You probably miss the big picture (ie, several legacy databases, each
>>> with their own models plus synchronization between them and a central
>>> django database).
>>>
But anyway:
> Until now, I was just catching the exception and forgot about it.
>
> Today, I'm not sure this is the right way.
> If the exception in itself is correctly processed, I noticed that
> any database operation behind raises another exception saying that
> postgresql hates me and won't perform any other operation (current
> transation is aborted, commands ignored until end of transaction
> block.).
> I already tried to end the transaction, but django will let me know
> I'm not within a transaction
Hu ?
I assume you did read this:
http://docs.djangoproject.com/en/dev/topics/db/transactions
and specially this part:
http://docs.djangoproject.com/en/dev/topics/db/transactions/#handling-exceptions-within-postgresql-transactions
>>>
>>> Actually, I posted because this just doesn't seem to work.
>>> If I try to rollback the transaction, I just get a nice error that
>>> I'not within a transaction.
>>>
>>> I haven't tested savepoints, but transaction rollback and autocommit
>>> doesn't seem to work here.
>>> I'll probably write a sample project this week to point the issue.
>>>
>>> Regards,
>>> Xavier.
>>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Django users" group.
> To post to this group, send email to django-users@googlegroups.com.
> To unsubscribe from this group, send email to
> django-users+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/django-users?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Django users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: Django, Postgres and recovering from database constraints
Also (sorry for the follow-up spam but I just remembered this) if you
are hoping to use the database level autocommit (postgres 8.2 or later),
you need to configure it:
Autocommit mode
New in Django 1.1: Please, see the release notes
If your application is particularly read-heavy and doesn’t make many
database writes, the overhead of a constantly open transaction can
sometimes be noticeable. For those situations, if you’re using the
postgresql_psycopg2 backend, you can configure Django to use
“autocommit” behavior for the connection, meaning that each database
operation will normally be in its own transaction, rather than having
the transaction extend over multiple operations. In this case, you can
still manually start a transaction if you’re doing something that
requires consistency across multiple database operations. The autocommit
behavior is enabled by setting the autocommit key in the OPTIONS part of
your database configuration in DATABASES:
'OPTIONS': {
'autocommit': True,
}
from
http://docs.djangoproject.com/en/1.2/ref/databases/
--
Casey
On 02/01/2011 01:52 PM, Casey S. Greene wrote:
Here is some code pulled from my (using postgres) django application
that recovers fine. Perhaps this is helpful to you. I am storing the
non-unique values and dealing with them later (pulling from an external
source that is supposed to have unique IDs assigned but they don't
always pan out somehow...).
from django.db import IntegrityError, transaction
#Other Code
...
#relevant code
try:
sid = transaction.savepoint()
foo = FooModel(fooarg=fooarg, ...)
foo.save()
except IntegrityError:
transaction.savepoint_rollback(sid)
non_unique_foo.append((FooDB, FooID))
Hope this helps!
Casey
On 02/01/2011 11:48 AM, Xavier Ordoquy wrote:
Le 1 févr. 2011 à 15:59, bruno desthuilliers wrote :
On 1 fév, 15:24, Xavier Ordoquy wrote:
Hi all,
I got a project which sometime outputs database integrity errors
(key violates unique constraint).
It is fine since I'm keeping an events table which may have several
creation messages for the same object.
Hmmm... Either I misunderstood you or you have a design problem here.
You probably miss the big picture (ie, several legacy databases, each
with their own models plus synchronization between them and a central
django database).
But anyway:
Until now, I was just catching the exception and forgot about it.
Today, I'm not sure this is the right way.
If the exception in itself is correctly processed, I noticed that
any database operation behind raises another exception saying that
postgresql hates me and won't perform any other operation (current
transation is aborted, commands ignored until end of transaction
block.).
I already tried to end the transaction, but django will let me know
I'm not within a transaction
Hu ?
I assume you did read this:
http://docs.djangoproject.com/en/dev/topics/db/transactions
and specially this part:
http://docs.djangoproject.com/en/dev/topics/db/transactions/#handling-exceptions-within-postgresql-transactions
Actually, I posted because this just doesn't seem to work.
If I try to rollback the transaction, I just get a nice error that
I'not within a transaction.
I haven't tested savepoints, but transaction rollback and autocommit
doesn't seem to work here.
I'll probably write a sample project this week to point the issue.
Regards,
Xavier.
--
You received this message because you are subscribed to the Google Groups "Django
users" group.
To post to this group, send email to django-users@googlegroups.com.
To unsubscribe from this group, send email to
django-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/django-users?hl=en.
Re: Django, Postgres and recovering from database constraints
Here is some code pulled from my (using postgres) django application that recovers fine. Perhaps this is helpful to you. I am storing the non-unique values and dealing with them later (pulling from an external source that is supposed to have unique IDs assigned but they don't always pan out somehow...). from django.db import IntegrityError, transaction #Other Code ... #relevant code try: sid = transaction.savepoint() foo = FooModel(fooarg=fooarg, ...) foo.save() except IntegrityError: transaction.savepoint_rollback(sid) non_unique_foo.append((FooDB, FooID)) Hope this helps! Casey On 02/01/2011 11:48 AM, Xavier Ordoquy wrote: Le 1 févr. 2011 à 15:59, bruno desthuilliers wrote : On 1 fév, 15:24, Xavier Ordoquy wrote: Hi all, I got a project which sometime outputs database integrity errors (key violates unique constraint). It is fine since I'm keeping an events table which may have several creation messages for the same object. Hmmm... Either I misunderstood you or you have a design problem here. You probably miss the big picture (ie, several legacy databases, each with their own models plus synchronization between them and a central django database). But anyway: Until now, I was just catching the exception and forgot about it. Today, I'm not sure this is the right way. If the exception in itself is correctly processed, I noticed that any database operation behind raises another exception saying that postgresql hates me and won't perform any other operation (current transation is aborted, commands ignored until end of transaction block.). I already tried to end the transaction, but django will let me know I'm not within a transaction Hu ? I assume you did read this: http://docs.djangoproject.com/en/dev/topics/db/transactions and specially this part: http://docs.djangoproject.com/en/dev/topics/db/transactions/#handling-exceptions-within-postgresql-transactions Actually, I posted because this just doesn't seem to work. If I try to rollback the transaction, I just get a nice error that I'not within a transaction. I haven't tested savepoints, but transaction rollback and autocommit doesn't seem to work here. I'll probably write a sample project this week to point the issue. Regards, Xavier. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Django, Postgres and recovering from database constraints
Le 1 févr. 2011 à 15:59, bruno desthuilliers wrote : > On 1 fév, 15:24, Xavier Ordoquy wrote: >> Hi all, >> >> I got a project which sometime outputs database integrity errors (key >> violates unique constraint). >> It is fine since I'm keeping an events table which may have several creation >> messages for the same object. > > Hmmm... Either I misunderstood you or you have a design problem here. You probably miss the big picture (ie, several legacy databases, each with their own models plus synchronization between them and a central django database). > But anyway: > >> Until now, I was just catching the exception and forgot about it. >> >> Today, I'm not sure this is the right way. >> If the exception in itself is correctly processed, I noticed that any >> database operation behind raises another exception saying that postgresql >> hates me and won't perform any other operation (current transation is >> aborted, commands ignored until end of transaction block.). > >> I already tried to end the transaction, but django will let me know I'm not >> within a transaction > > Hu ? > > I assume you did read this: > http://docs.djangoproject.com/en/dev/topics/db/transactions > > and specially this part: > http://docs.djangoproject.com/en/dev/topics/db/transactions/#handling-exceptions-within-postgresql-transactions Actually, I posted because this just doesn't seem to work. If I try to rollback the transaction, I just get a nice error that I'not within a transaction. I haven't tested savepoints, but transaction rollback and autocommit doesn't seem to work here. I'll probably write a sample project this week to point the issue. Regards, Xavier. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Re: Django, Postgres and recovering from database constraints
On 1 fév, 15:24, Xavier Ordoquy wrote: > Hi all, > > I got a project which sometime outputs database integrity errors (key > violates unique constraint). > It is fine since I'm keeping an events table which may have several creation > messages for the same object. Hmmm... Either I misunderstood you or you have a design problem here. But anyway: > Until now, I was just catching the exception and forgot about it. > > Today, I'm not sure this is the right way. > If the exception in itself is correctly processed, I noticed that any > database operation behind raises another exception saying that postgresql > hates me and won't perform any other operation (current transation is > aborted, commands ignored until end of transaction block.). > I already tried to end the transaction, but django will let me know I'm not > within a transaction Hu ? I assume you did read this: http://docs.djangoproject.com/en/dev/topics/db/transactions and specially this part: http://docs.djangoproject.com/en/dev/topics/db/transactions/#handling-exceptions-within-postgresql-transactions -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
Django, Postgres and recovering from database constraints
Hi all, I got a project which sometime outputs database integrity errors (key violates unique constraint). It is fine since I'm keeping an events table which may have several creation messages for the same object. Until now, I was just catching the exception and forgot about it. Today, I'm not sure this is the right way. If the exception in itself is correctly processed, I noticed that any database operation behind raises another exception saying that postgresql hates me and won't perform any other operation (current transation is aborted, commands ignored until end of transaction block.). I already tried to end the transaction, but django will let me know I'm not within a transaction I have been pretty unlucky in my search on the net. The only solution I have seen is an advice to close the connection but it sounds a bit like "reboot to make it work" So, has anyone a way to gracefully handle such errors ? Xavier. -- You received this message because you are subscribed to the Google Groups "Django users" group. To post to this group, send email to django-users@googlegroups.com. To unsubscribe from this group, send email to django-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/django-users?hl=en.
