Re: [sqlalchemy] Help with optimizing
On May 28, 2010, at 5:18 PM, Jason Baker wrote:
> On Fri, May 28, 2010 at 3:48 PM, Michael Bayer
> wrote:
>
> This is the ORM affecting 124K statements so must be a very large data
> persist (it seems like a persist heavy operation, i see 287K objects total
> treated as part of units of work).
>
> It seems like you are calling commit() a very large number of times. So the
> most obvious enhancement would be to call this a whole lot less - the commit
> necessitates a flush, and also by default fully expires the session (unless
> you turn off expire_on_commit), meaning all rows have to be fully reloaded,
> which is probably making the number of statements executed much larger than
> it needs to be.
>
> Unfortunately committing less isn't doable. :-(
>
> We need to have a transaction for each record we process otherwise, we'd have
> to go back and clean up if something failed. I will try turning off
> expire_on_commit and see if that helps though.
>
> A good deal of time is spent in compiling constructs into SQL strings here,
> there is a feature whereby this can be cut down dramatically for similar
> statements executed many times which is the "compiled_cache" execution
> option. The ORM uses this a bit internally now though it might be a nice
> feature for you to be able to switch it on for a given session, and have all
> SQL statement compilation cached for the life of that session. This feature
> can be approximated right now though I'd want to modify _save_obj to not
> overwrite the cache with its own, which defeats the usage of a
> session-spanning compilation cache.
>
> That would be a *huge* help. Would this involve a custom Query subclass?
you can use it right now like this (assuming usage of a scoped session):
connection = engine.connect()
compiled_cache = {}
session = MyScopedSession()
session.bind = connection.execution_options(compiled_cache=compiled_cache)
# do many things with the session. every SQL construct would store its
"compiled" form in that dictionary
MyScopedSession.remove()
so that will work now, *except* that most mapper._save_obj() calls are going to
replace the cache with its own that is local to the _save_obj() call for the
duration of its call, so it doesn't get to take advantage of that cache. The
adjustment would be that save_obj checks for a cache in place already. I'd
probably add an execution_options() method to Session. A patch is attached
which should achieve this (not tested):
session.execution_options(compiled_cache={})
# good to go
>
> --
> Jason Baker
> Developer
> ZeOmega
> 3010 Gaylord Parkway, Suite 210
> Frisco, TX 75034
> O: 214-618-9880 ext 8024
> jba...@zeomega.com
> www.ZeOmega.com
> Proven. Progressive. Partner.
>
> --
> You received this message because you are subscribed to the Google Groups
> "sqlalchemy" group.
> To post to this group, send email to sqlalch...@googlegroups.com.
> To unsubscribe from this group, send email to
> sqlalchemy+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/sqlalchemy?hl=en.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
-- Jason BakerDeveloperZeOmega3010 Gaylord Parkway, Suite 210Frisco, TX 75034O: 214-618-9880 ext 8024
jba...@zeomega.comwww.ZeOmega.comProven. Progressive. Partner.
--
You received this message because you are subscribed to the Google Groups "sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
session_compiled_cache.patch
Description: Binary data
Re: [sqlalchemy] Help with optimizing
On Fri, May 28, 2010 at 3:48 PM, Michael Bayer wrote: > > This is the ORM affecting 124K statements so must be a very large data > persist (it seems like a persist heavy operation, i see 287K objects total > treated as part of units of work). > > It seems like you are calling commit() a very large number of times. So > the most obvious enhancement would be to call this a whole lot less - the > commit necessitates a flush, and also by default fully expires the session > (unless you turn off expire_on_commit), meaning all rows have to be fully > reloaded, which is probably making the number of statements executed much > larger than it needs to be. > Unfortunately committing less isn't doable. :-( We need to have a transaction for each record we process otherwise, we'd have to go back and clean up if something failed. I will try turning off expire_on_commit and see if that helps though. > A good deal of time is spent in compiling constructs into SQL strings here, > there is a feature whereby this can be cut down dramatically for similar > statements executed many times which is the "compiled_cache" execution > option. The ORM uses this a bit internally now though it might be a nice > feature for you to be able to switch it on for a given session, and have all > SQL statement compilation cached for the life of that session. This > feature can be approximated right now though I'd want to modify _save_obj to > not overwrite the cache with its own, which defeats the usage of a > session-spanning compilation cache. > That would be a *huge* help. Would this involve a custom Query subclass? -- Jason Baker Developer ZeOmega 3010 Gaylord Parkway, Suite 210 Frisco, TX 75034 O: 214-618-9880 ext 8024 jba...@zeomega.com www.ZeOmega.com Proven. Progressive. Partner. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Help with optimizing
This is the ORM affecting 124K statements so must be a very large data persist (it seems like a persist heavy operation, i see 287K objects total treated as part of units of work). It seems like you are calling commit() a very large number of times. So the most obvious enhancement would be to call this a whole lot less - the commit necessitates a flush, and also by default fully expires the session (unless you turn off expire_on_commit), meaning all rows have to be fully reloaded, which is probably making the number of statements executed much larger than it needs to be. You might want to establish better control over when flush() is called. Sometimes turning off autoflush and having it flush the whole thing just once, or at some pre-determined interval within a transaction, spans out the work more efficiently. A good deal of time is spent in compiling constructs into SQL strings here, there is a feature whereby this can be cut down dramatically for similar statements executed many times which is the "compiled_cache" execution option. The ORM uses this a bit internally now though it might be a nice feature for you to be able to switch it on for a given session, and have all SQL statement compilation cached for the life of that session. This feature can be approximated right now though I'd want to modify _save_obj to not overwrite the cache with its own, which defeats the usage of a session-spanning compilation cache. the last thing there is the only further potential optimization to SQLA that is apparent, and would probably cut at least a few million function calls out of the run you have here. Other than that you're already working with the most optimized version of the ORM :). On May 28, 2010, at 4:14 PM, Jason Baker wrote: > I'm attaching output from cProfile from our application. I'm mainly sending > this to you for two reasons: > > 1. In case profiling data is helpful to you guys for optimizing SQLAlchemy. > 2. Because I'm looking for ways to speed my application up, and I am not > sure how to interpret the SQLAlchemy portions. > > It looks like the biggest timesink is Session.commit. Does anyone have any > general advice on making commit go faster? I can answer any questions that > aren't answered by the profiling data, but I'm not sure how much actual > source code I can give out. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
On 5/28/2010 10:08 AM, Michael Bayer wrote: Is the pattern that you want to keep re-issuing a savepoint repeatedly using the same name ? Does that have some different usage of resources versus issuing/closing distinct savepoints with different names ? As an aside, since oracle apparently has no 'release savepoint', I imagine the answer to your second question might be: "yes, when using oracle with thousands of potential savepoints."I think I've worked around this for now, so I'm not asking you to necessarily do anything about it, but it might be somewhat of a minor enhancement at some point. Thanks again. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
On May 28, 2010, at 1:24 PM, Kent wrote: > > > On May 27, 6:39 pm, Michael Bayer wrote: > >> commit() releases the savepoint, if thats whats going on contextually. It >> doesnt actually commit the outer transaction if you've last called >> begin_nested(). >> > > > In a SessionExtension, 'before_commit' is called for nested > transactions, which do not actually perform a database commit, as well > as the RootTransaction, which does issue a COMMIT. > > I believe the zope guys overlooked this when they wrote their 'def > before_commit()' method of their SessionExtension, and I am going to > attempt to fix this. > >def before_commit(self, session): >assert zope_transaction.get().status == 'Committing', > "Transaction must be committed using the transaction manager" > > So my question is, first, do you call before_commit() on > SessionExtensions intentionally *even for nested transactions* which > are not performing a database commit? I assume the answer is "yes, > that is by design". it is by design, yes. you ultimately need to look at the transaction itself if you want to determine if its the "real" commit. the entire session conversation could be nested inside of a larger begin/commit external to the session, for example (I just illustrated a unit test recipe that uses this technique if you search around yesterday's messages). > > Which leads to this question: Inside a SessionExtension's > before_commit() method, how can I work out whether this is a nested > transaction vs. the root transaction? you'd look at session.transaction for details. to get the engine.Transaction object you'd need to dig into its _connections dictionary, but for most purposes you can probably just look at transaction.nested. > > Thanks, > Kent > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
Is the answer to second questoin "session.transaction.nested"? On May 28, 1:24 pm, Kent wrote: > On May 27, 6:39 pm, Michael Bayer wrote: > > > commit() releases the savepoint, if thats whats going on contextually. It > > doesnt actually commit the outer transaction if you've last called > > begin_nested(). > > In a SessionExtension, 'before_commit' is called for nested > transactions, which do not actually perform a database commit, as well > as the RootTransaction, which does issue a COMMIT. > > I believe the zope guys overlooked this when they wrote their 'def > before_commit()' method of their SessionExtension, and I am going to > attempt to fix this. > > def before_commit(self, session): > assert zope_transaction.get().status == 'Committing', > "Transaction must be committed using the transaction manager" > > So my question is, first, do you call before_commit() on > SessionExtensions intentionally *even for nested transactions* which > are not performing a database commit? I assume the answer is "yes, > that is by design". > > Which leads to this question: Inside a SessionExtension's > before_commit() method, how can I work out whether this is a nested > transaction vs. the root transaction? > > Thanks, > Kent -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
On May 27, 6:39 pm, Michael Bayer wrote: > commit() releases the savepoint, if thats whats going on contextually. It > doesnt actually commit the outer transaction if you've last called > begin_nested(). > In a SessionExtension, 'before_commit' is called for nested transactions, which do not actually perform a database commit, as well as the RootTransaction, which does issue a COMMIT. I believe the zope guys overlooked this when they wrote their 'def before_commit()' method of their SessionExtension, and I am going to attempt to fix this. def before_commit(self, session): assert zope_transaction.get().status == 'Committing', "Transaction must be committed using the transaction manager" So my question is, first, do you call before_commit() on SessionExtensions intentionally *even for nested transactions* which are not performing a database commit? I assume the answer is "yes, that is by design". Which leads to this question: Inside a SessionExtension's before_commit() method, how can I work out whether this is a nested transaction vs. the root transaction? Thanks, Kent -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] mysql vs sqlite for testing
Michael Bayer wrote: How do people get around this? What's best practice in this area? your test suite ideally wouldn't be tearing down and building up tables many times. Correct ;-) For an application where the testing is against a fixed set of tables (i.e. not at all like SQLA's own unit tests), you would run all your tests in transactions that get rolled back when the test is complete. This may show my poor RDB knowledge; if you need to test selects against stuff that would have normally been committed to the DB, can you do that without committing a transaction? I use setup/teardowns like this for this purpose (assume scoped_session, which yes you should probably use all the time so that the session is accessed by a single reference): Where do you actually drop/create the tables? def setup_for_rollback(): Session.remove() The docs on this are a little brief ;-) http://www.sqlalchemy.org/docs/reference/orm/sessions.html#sqlalchemy.orm.scoping.ScopedSession.remove This seems key to your suggested strategy. What does it do? sess = Session() c = sess.bind.connect() global transaction Maybe set as an attibute of a TestCase rather than asa global? transaction = c.begin() sess.bind = c def teardown_for_rollback(): transaction.rollback() Session.remove() above, "transaction" is the "real" transaction. All begin/commits inside don't actually commit anything. Again, this last bit escapes me, how come Session().commit() or begin doesn't do anything? cheers, Chris -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
From a quick reading of the 'transaction' package source, it looks like
you should be able to create savepoints and roll them back something
like this:
savepoint = transaction.savepoint()
try:
# ...
except:
savepoint.rollback()
raise
Thanks for the interest in helping. I had already gone down that road
and abandoned it when I realized it wasn't playing nicely with
sqlalchemy session:
>>> a=TranCode()
>>> a.transactioncodeid='PMT'
>>> sp=transaction.savepoint()
>>> DBSession.add(a)
>>> DBSession.flush()
15:53:52,852 INFO [sqlalchemy.engine.base.Engine.0x...88d0] SAVEPOINT
sa_savepoint_1
15:53:52,853 INFO [sqlalchemy.engine.base.Engine.0x...88d0] {}
/home/rarch/tg2env/lib/python2.6/site-packages/SQLAlchemy-0.6.0.2-py2.6-linux-x86_64.egg/sqlalchemy/engine/default.py:500:
SAWarning: Unicode type received non-unicode bind param value 'PMT'
param[key.encode(encoding)] = processors[key](compiled_params[key])
15:53:52,855 INFO [sqlalchemy.engine.base.Engine.0x...88d0] INSERT INTO
trancodes (transactioncodeid, description, debitorcredit, posttogl,
userdefined, codetype) VALUES (:transactioncodeid, :description,
:debitorcredit, :posttogl, :userdefined, :codetype)
15:53:52,855 INFO [sqlalchemy.engine.base.Engine.0x...88d0]
{'description': None, 'userdefined': None, 'transactioncodeid': 'PMT',
'codetype': None, 'debitorcredit': None, 'posttogl': None}
after flush!!
>>> sp.rollback()
15:53:52,859 INFO [sqlalchemy.engine.base.Engine.0x...88d0] ROLLBACK TO
SAVEPOINT sa_savepoint_1
15:53:52,860 INFO [sqlalchemy.engine.base.Engine.0x...88d0] {}
Traceback (most recent call last):
File "", line 1, in
File
"/home/rarch/tg2env/lib/python2.6/site-packages/transaction-1.0a1-py2.6.egg/transaction/_transaction.py",
line 662, in rollback
transaction._saveAndRaiseCommitishError() # reraises!
File
"/home/rarch/tg2env/lib/python2.6/site-packages/transaction-1.0a1-py2.6.egg/transaction/_transaction.py",
line 659, in rollback
savepoint.rollback()
File
"/home/rarch/tg2env/lib/python2.6/site-packages/zope.sqlalchemy-0.4-py2.6.egg/zope/sqlalchemy/datamanager.py",
line 147, in rollback
self.session.clear() # remove when Session.rollback does an
attribute_manager.rollback
AttributeError: 'Session' object has no attribute 'clear'
>>>
Apparently I need to look into whether zope has a SQLA 0.6.0 compatible release
yet...looks to be trying to call session.clear
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] postgresql text search
On Fri, May 28, 2010 at 3:12 PM, Michael Bayer wrote: > we support the to_tsquery() syntax through the match() operator: > > http://www.sqlalchemy.org/trac/browser/test/dialect/test_postgresql.py#L1748 Thank you very much. -- Eric Lemoine Camptocamp France SAS Savoie Technolac, BP 352 73377 Le Bourget du Lac, Cedex Tel : 00 33 4 79 44 44 96 Mail : eric.lemo...@camptocamp.com http://www.camptocamp.com -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
RE: [sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
Kent wrote: [SNIP] > I'm fine with how SQLA is designed, it isn't really a SQLA > issue, I was > just appealing to you to see if you could think of a workaround I > believe the problem is in the framework tools we are using, > whether it > is Zope or TG. (I've posted to zope group now to see if they > intended > to support savepoints and how, etc.). > Since that framework won't allow me to issue the command > session.commit(), I cannot release the savepoints until the zope > transaction commits, and by then I'm getting a python max recursion > problem because there are so many outstanding savepoints for > it to release. >From a quick reading of the 'transaction' package source, it looks like you should be able to create savepoints and roll them back something like this: savepoint = transaction.savepoint() try: # ... except: savepoint.rollback() raise I've no idea if that will confuse SA's accounting mechanisms though - I imagine you'd probably need to clean up some objects in the session. If you have a look at the tg.configuration module, the transaction middleware is added based on the config.use_transaction_manager value, so you could set that to False and implement whatever transaction management features you want in your own middleware. Hope that helps, Simon -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
* Is there a way besides session.commit() to free the savepoint resource? Is there a way to provide the savepoint name, so I can use the same name over? Lastly, if you aren't the expert, where would you point me, zope group or TG group? you could issue the SAVEPOINT instructions manually if you don't like SQLA's abstraction of it, although then you don't get the Session's object management behavior around those savepoints (though it can be approximated). Yeah, I've considered it. I don't really understand what you're trying to achieve, such that SQLA is "forcing" you to create thousands of savepoints when you only need one. Is the pattern that you want to keep re-issuing a savepoint repeatedly using the same name ? Does that have some different usage of resources versus issuing/closing distinct savepoints with different names ? I'm fine with how SQLA is designed, it isn't really a SQLA issue, I was just appealing to you to see if you could think of a workaround I believe the problem is in the framework tools we are using, whether it is Zope or TG. (I've posted to zope group now to see if they intended to support savepoints and how, etc.). Since that framework won't allow me to issue the command session.commit(), I cannot release the savepoints until the zope transaction commits, and by then I'm getting a python max recursion problem because there are so many outstanding savepoints for it to release. So where I thought you could help is if there is another (even non-public) way to release the savepoint without calling session.commit()? Ultimately I understand now that this is not a SQLA issue. Thanks -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] postgresql text search
we support the to_tsquery() syntax through the match() operator: http://www.sqlalchemy.org/trac/browser/test/dialect/test_postgresql.py#L1748 On May 28, 2010, at 9:53 AM, Eric Lemoine wrote: > Hi > > Are there examples of using PostgreSQL's full-text search with > SQLAlchemy? I'm interested in any kind of information about that. > Thanks, > > -- > Eric Lemoine > > Camptocamp France SAS > Savoie Technolac, BP 352 > 73377 Le Bourget du Lac, Cedex > > Tel : 00 33 4 79 44 44 96 > Mail : eric.lemo...@camptocamp.com > http://www.camptocamp.com > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
On May 28, 2010, at 8:47 AM, Kent wrote: > Say I have this: > > session().begin_nested() > try: >session.add(obj) >session.flush() >session.commit() > except: >session.rollback() > ... > transaction.commit() > > 2 questions: > > * I assume that the session.rollback() undoes the add(obj) - obj is > again transient? yes > > Also, this seems to give me the behavior I'd like, except that the > session.commit() pukes due to zope, saying: > > """ > File "/home/rarch/tg2env/lib/python2.6/site-packages/ > zope.sqlalchemy-0.4-py2.6.egg/zope/sqlalchemy/datamanager.py", line > 201, in before_commit >assert zope_transaction.get().status == 'Committing', "Transaction > must be committed using the transaction manager" > AssertionError: Transaction must be committed using the transaction > manager. > """ *shrugs* > > Later, the transaction.commit() seems to free the savepoint: I don't know what "transaction" here is, again if thats zope/tg/whatever it might be automating that process, sure. > * Is there a way besides session.commit() to free the savepoint > resource? Is there a way to provide the savepoint name, so I can use > the same name over? Lastly, if you aren't the expert, where would you > point me, zope group or TG group? you could issue the SAVEPOINT instructions manually if you don't like SQLA's abstraction of it, although then you don't get the Session's object management behavior around those savepoints (though it can be approximated). I don't really understand what you're trying to achieve, such that SQLA is "forcing" you to create thousands of savepoints when you only need one. Is the pattern that you want to keep re-issuing a savepoint repeatedly using the same name ? Does that have some different usage of resources versus issuing/closing distinct savepoints with different names ? -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] postgresql text search
Hi Are there examples of using PostgreSQL's full-text search with SQLAlchemy? I'm interested in any kind of information about that. Thanks, -- Eric Lemoine Camptocamp France SAS Savoie Technolac, BP 352 73377 Le Bourget du Lac, Cedex Tel : 00 33 4 79 44 44 96 Mail : eric.lemo...@camptocamp.com http://www.camptocamp.com -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Creating and implementing a dialect
to intercept and act upon SQL strings, I would look into ConnectionProxy: http://www.sqlalchemy.org/docs/reference/sqlalchemy/interfaces.html?highlight=connectionproxy#sqlalchemy.interfaces.ConnectionProxy otherwise, if its a mediating layer to a DBAPI that is supported by SQLA, you could make a wrapper around it and pass it to create_engine() using the "module" argument. On May 28, 2010, at 2:12 AM, Paul Hemans wrote: > I am very new to SA and I need to create a dialect for a legacy db on > windows. The problem is that the resulting SQL strings need to pass > into another application to be actioned. That is, not through any sort > of ODBC/OLEDB... connector. > So the way I see it, I have a few issues to deal with. Create a > dialect, trap the generated SQL strings, action them and return a > result set to SA. Don't mind the work but I wouldn't know where to > begin, so if anyone could give me some pointers that would be great. > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To post to this group, send email to sqlalch...@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: For each begin_nested() call, a corresponding rollback() or commit() must be issued.
>
> >> session.begin_nested()
> >> try:
> >> ...
> >> session.flush()
> >> session.commit()
> >> except:
> >> session.rollback()
>
Say I have this:
session().begin_nested()
try:
session.add(obj)
session.flush()
session.commit()
except:
session.rollback()
...
transaction.commit()
2 questions:
* I assume that the session.rollback() undoes the add(obj) - obj is
again transient?
Also, this seems to give me the behavior I'd like, except that the
session.commit() pukes due to zope, saying:
"""
File "/home/rarch/tg2env/lib/python2.6/site-packages/
zope.sqlalchemy-0.4-py2.6.egg/zope/sqlalchemy/datamanager.py", line
201, in before_commit
assert zope_transaction.get().status == 'Committing', "Transaction
must be committed using the transaction manager"
AssertionError: Transaction must be committed using the transaction
manager.
"""
Later, the transaction.commit() seems to free the savepoint:
>>> transaction.commit()
13:41:41,039 INFO [sqlalchemy.engine.base.Engine.0x...2c10] RELEASE
SAVEPOINT sa_savepoint_2
13:41:41,039 INFO [sqlalchemy.engine.base.Engine.0x...2c10] {}
13:41:41,040 INFO [sqlalchemy.engine.base.Engine.0x...2c10] COMMIT
>>>
However, by then I'll have hundreds/thousands of savepoints to release
since this is being called as part of data-takeon. I'd rather not
take up thousands of savepoint resources when I only need one.
So the second question(s) is:
* Is there a way besides session.commit() to free the savepoint
resource? Is there a way to provide the savepoint name, so I can use
the same name over? Lastly, if you aren't the expert, where would you
point me, zope group or TG group?
Thanks in advance.
--
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
