I'm trying to setup fast insertions using sqlalchemy/sqlite and it
seems to be taking an unnecessarily long amount of time to do many of
the inserts.
It looks like the main call that inserts into a few tables takes about
1-2 seconds each for the first few calls, but then drops down to
around 50ms, which is more acceptable.
Do I have something in my setup that significantly slows down the
earlier calls?
This is what I use to create all the tables
engine = create_engine('sqlite:///' + statsFile,
echo=False)
conn = engine.connect()
metadata = MetaData(engine)
self.createStatsTables(metadata)
metadata.create_all(engine) # save tables
# functions like this are called several times
def addUniqueValue(table, key, value, returnKey):
try:
table.insert().execute(...)
except IntegrityError, error: #probably duplicate
if 'not unique' not in str(error):
raise error
rows = table.select().execute() # get the primaryId for the thing I
just inserted using python to filter
...
return primaryId
I'm not using mappers because I'm normalizing my database and sqlite
doesn't support foreign keys directly. Basically:
1) I have an object come in
2) normalized attributes are stored in multiple different tables
3) I read their primary ids back
4) I insert the object into a table replacing the normalized
attributes with their respective primary id
It all needs to happen pretty fast and can be done in one transaction,
I guess, but from what I understand, sessions and transactions only
apply well to mappers.
--~--~---------~--~----~------------~-------~--~----~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~----------~----~----~----~------~----~------~--~---
