Dear Rick, Thank you for your reply. I understand, but is there not a better way than doing a lot of single commits in case of a commit exception? In other words, is there a way to tell SQLAlchemy to throw an exception on the Session.add if there's a duplicate as opposed to on the Session.commit? Or else, is it not better to keep an indexed collection on the side to check that an id was inserted before?
Thanks! On Apr 20, 12:05 pm, Richard Harding <rhard...@mitechie.com> wrote: > What I tend to do in cases like this is to break things into commit chunks. > For instance, I've got an import script that goes through and processes 10 > at a time and performs a commit every 10. This is tweakable via a config > setting, but so far 10 works for my needs. > > As for the duplicates, If they're exact and you don't need to processes > changes between one record and another of the same pk, just try: except and > catch the pk error on the second insert. With my import script, if I get an > exception, I then run through all 10 committing one at a time and finding > the 'bad egg' so that I can log out that this one failed because of the > exception. Sure, for that batch of 10 I'm doing a bunch of single commits, > but more often than not I'm running in my batch mode. > > I'd just the db/exceptions tell me a record exists vs trying to query the > server for each one to check first. Good ole case of 'better to ask for > forgiveness than permission'. > > Rick -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
