On 10/11/2016 09:59 PM, Alfred Perlstein wrote:
Hello folks, we are using SqlA in a pretty large flask app. One issue we have is that sometimes due to orm, serializers (marshmallow) or other complex code paths we find ourselves doing many more queries than we expected and this causes us a lot of headache. We need to be able to track down the code paths causing the extra SQL. In addition because the codebase is large we can't turn on logging globally otherwise we would drown in noise. The solution we thought would be simple would be to be able to temporarily turn on logging in a single thread of execution or on a single session object... however that seems to be an elusive thing to be able to do. What we have found is that while sqlA offers the ability to *create* an engine with "echo=True", there doesn't seem to be a sane way to *set* echo=True with an existing session.
Well first off, SQL logging is a function of the Core, not the ORM, so the Session wouldn't know anything about it.
One could try to set session.engine.echo=True, however it appears that the connection object hung from the engine object grabs the value of "echo" when instantiated and never checks to see if the engine has been reconfigured.
this is actually documented in the green note at the end of http://docs.sqlalchemy.org/en/rel_1_1/core/engines.html#configuring-logging
Surely a better way to do this would be to have a method
to pass down to the Connection object to enable echo.
That's not how Python logging works. Logging in Python is organized as a "cross-cutting" concern, meaning a logger presents itself as an open stream in which log messages are unconditionally dropped. The configuration of what messages to log, when, and where, is accomplished within the logging configuration, not within the system that is emitting messages to logging. SQLAlchemy's "echo" flag is merely a quick shortcut to a very simplistic log output, and additionally it adds a small "block" to remove the overhead of emitting messages to the log object if we've detected they aren't being used (this occurs independently of the "echo" flag itself).
Any non-trivial use of logging should be using Python's logging system directly. The loggers in use are described at http://docs.sqlalchemy.org/en/rel_1_1/core/engines.html#configuring-logging.
In this case I'd in the simple case just add "threadid" to the log format (https://docs.python.org/2/library/logging.html#logrecord-attributes) and use grep against the stream. More ambitiously I'd write a simple logging.Filter that only allows messages from a certain thread (some example filters such as https://docs.python.org/2/howto/logging-cookbook.html#using-filters-to-impart-contextual-information).
I've done quite a bit of looking through google and the docs and have simply been unable to find a way do to the following: .... lots of code previous to this line... old_enable = session.log_enable() # turn on logging, return the previous value of logging (so this can be nested) ... do some stuff and see which SQL queries are being made ... session.log_set(old_enable) # restore logging to prior to it being set. ... lots more code without any logging happening. If I were to create a method on engine that would then reconfigure its connection object as well to remedy this, would that be an acceptable patch to the project? Or is there some other trick I am missing? Thank you! -Alfred -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com <mailto:sqlalchemy+unsubscr...@googlegroups.com>. To post to this group, send email to sqlalchemy@googlegroups.com <mailto:sqlalchemy@googlegroups.com>. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
-- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.
