[sqlalchemy] transaction spanning multiple threads

2015-02-17 Thread Christian Lang
Hi, I have a question regarding multi-threading and transactions in the non-ORM case. Assume, we have a global Connection c (with transaction), on which a number of insert/update queries are executed and some insert/update queries are executed in threads: Main thread:

[sqlalchemy] Re: transaction spanning multiple threads

2015-02-17 Thread Laurence Rowe
I'd recommend keeping the writes to a single thread if you can as that will be easiest. If you can't do that then you could tie them together using the transaction module's two phase commit and zope.sqlalchemy. As you want the same transaction shared across threads you'll want to create your

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Javier Pajuelo
Just to give you an idea of the code: Basically the code scrapes some websites and gets info into objects. Client class with client info, event class with event info, and so on. There are foreign keys to tie event and client classes. Sometimes even when I scrape the web and try to upload

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Javier Pajuelo
Hi Chris, Thanks for trying to help. The code is as follow: #== # CompassCandidate() #== class CompassCandidate(Base): '''

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Jonathan Vanasco
The simplest way using your code would be: for _candidate_data in candidates: _existing = session.query(CompassCandidate).filter(CompassCandidate.id == _candidate_data.id).first() if not _existing: # add the new item session.add(_candidate_data) else: # translate the attributes from the new item

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Jonathan Vanasco
This looks to be your problem: id = Column(String(64), primary_key=True) `id` is a primary key, which means it must be unique. But you're not setting it to a unique value. Most people will use a Serial column for this. -- You received this message because you are subscribed to the

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Javier Pajuelo
Could you give an example of using a serial column? I believe that whatever comes into the first argument of column is a unique value from another DB. Also, I thought that the person who wrote the code missed to use relationships between candidates and jobs. e.g.:class User(Base): #

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Jonathan Vanasco
A serial would usually be an integer that is tied to a sequence (postgres, oracle) or auto_increment (mysql). I see in your example that you have a unicode string -- `ORaE9+aCdP0` If that is coming from another db, then you've already migrated that candidate. you'll should do a search for

Re: [sqlalchemy] SQL Alchemy on Insert to DB

2015-02-17 Thread Chris Withers
What does the code look like that generates 'candidates'? Chris On 17/02/2015 00:48, Javier Pajuelo wrote: I get the following error: | sqlalchemy.exc.IntegrityError: (IntegrityError) column id is not unique u'INSERT INTO candidates (id, timeStamp, name, link) VALUES (?, ?, ?, ?)'