[sqlalchemy] Re: Odd behaviour of echo flag
Forgot to mention - this happens with SQLAlchemy versions 0.6.7 and 0.6.8. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Odd behaviour of echo flag
Thanks for the detailed explanation, Mike. It makes complete sense - I missed the fact that a new Connection was being created under the hood at different times in the two different scenarios. Regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Odd behaviour of echo flag
The following short script behaves differently if run with no arguments or with
arguments:
from elixir import *
import os
from sqlalchemy import Unicode, select, and_, __version__
import sys
class School(Entity):
name = Field(Unicode(50), required=True)
class Subject(Entity):
name = Field(Unicode(50), required=True)
teachers = ManyToMany('Teacher',
local_colname='course_id',
remote_colname='prof_id')
school = ManyToOne('School')
class Teacher(Entity):
name = Field(Unicode(50), required=True)
def main():
if os.path.exists('/tmp/schools.db'):
os.remove('/tmp/schools.db')
metadata.bind = sqlite:tmp/schools.db
setup_all(create_tables=True)
school = School(name='Rydell High')
session.commit()
r = Subject._descriptor.find_relationship('teachers')
t = r.table
if len(sys.argv) 1:
c = Subject.school == school
else:
c = Subject.school_id == school.id
s = select([Teacher.name],
and_(c, t.c.prof_id == Teacher.id, t.c.course_id == Subject.id))
metadata.bind.echo = True
print('Executing query using SQLAlchemy', __version__)
session.execute(s)
if __name__ == '__main__':
main()
The script chooses between two comparator conditions:
Subject.school == school
or
Subject.school_id == school.id
before running a SQLA select query using the chosen comparator. In one case the
generated SQL is printed, and in the other it's not, despite echo being set to
True. In the real program of which this is a cut-down example, the query ran
correctly in either case, so it's just the engine.echo behaviour which is odd.
Sorry this is an Elixir script - I'm unable to reproduce using just SQLAlchemy,
but it seems unlikely to be an Elixir issue. It seems to be a subtle enough bug
(if bug it is) that removing Elixir from the picture causes the bug to disappear
from view.
Why should the two cases behave differently?
Regards,
Vinay Sajip
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Odd behaviour of echo flag
Thanks for the detailed explanation, Mike. I missed the fact that the Connection gets created at different times in the two scenarios. As expected, moving the echo=True line to just after the commit gives the same result in both scenarios. Regards, Vinay Sajip P.S. I'm reposting this, Google seems to have swallowed my earlier attempt to respond - trying via gmane now. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Unexpected SessionTransaction behaviour
On Jun 15, 6:59 pm, Michael Bayer mike...@zzzcomputing.com wrote: So no, calling gc.collect() is not necessary under normal circumstances, it is only needed here to for the purposes of the method of testing. Ok, thanks. I've done some more digging and find that there *are* some references knocking around, but not sure of an easy way of getting rid of them. The SessionTransaction which is created in SessionTransaction.close() hangs around because it is referred to by an orm.session.Session instance, which in turn is referred to by various orm.query.Query objects which are used to populate comboboxes earlier in the request processing. These comboboxes are generated using some widget classes which hold references to the queries which populate them - and I want these widget-query connections to persist across requests because they codify e.g. specific orderings for the combobox entries. So it seems that I should break the connection between these query objects and the orm.session.Sessions they use, in the HTTP request cleanup code where I am doing the scoped_session.remove() calls. Of course at that point I don't have a clear view of what widgets/queries were used in rendering the response - so what would be a reasonably easy way of ensuring everything's cleaned up? I don't believe that what I'm doing is a particularly uncommon pattern. Thanks regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Unexpected SessionTransaction behaviour
On Jun 14, 11:19 pm, Michael Bayer mike...@zzzcomputing.com wrote:
the new SessionTransaction that occurs in close() does not request any
connection resources, and is discarded immediately along with the session
that is the subject of remove(). I don't see how it could be affected by
any previous requests. Each request should have a totally new Session
object, with a totally new SessionTransaction object. The old one is gone as
soon as remove() completes.
Here's my example short test case:
import gc
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker, scoped_session
from sqlalchemy.orm.session import SessionTransaction
from sqlalchemy.pool import NullPool
engine = create_engine('sqlite://', poolclass=NullPool)
sessions = []
# To simulate multi-db setup, create two sessions
for i in range(2):
sessions.append(scoped_session(sessionmaker(bind=engine)))
for i in range(1, 3):
print 'Looping: iteration (request) #%d' % i
tolist = [c for c in gc.get_objects()
if isinstance(c, SessionTransaction)]
assert len(tolist) == 0
# Apparently scoped_sessions can be used without
# instantiating, so do that
# Just one of the databases will be used in the
# processing
s = sessions[0]
s.execute('select 1')
tolist = [c for c in gc.get_objects()
if isinstance(c, SessionTransaction)]
assert len(tolist) == 1
for s in sessions:
#s.commit() # Not needed, and makes no difference
s.remove()
# There should be no SessionTransactions remaining
tolist = [c for c in gc.get_objects()
if isinstance(c, SessionTransaction)]
# On my system, the next line raises AssertionError
assert len(tolist) == 0, 'No SessionTransactions hanging around'
output:
Looping: iteration (request) #1
Traceback (most recent call last):
File sqlatest.py, line 35, in module
assert len(tolist) == 0, 'No SessionTransactions hanging around'
AssertionError: No SessionTransactions hanging around
BTW I'm using SQLA trunk on Ubuntu Karmic.
Regards,
Vinay Sajip
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Unexpected SessionTransaction behaviour
On Jun 15, 4:30 pm, Michael Bayer mike...@zzzcomputing.com wrote: err, no, your test is incorrect. You are maintaining a reference to the SessionTransaction in tolist. Whoops, you're right. However, should I really have to do a gc.collect() after the session.remove() calls? Without it, I still get the AssertionError. With it, I don't - but doesn't a requirement to call gc.collect() mean that there could be non-deterministic, potentially long pauses during request handling, because of other stuff needing to be garbage-collected which has no connection to the SQLA stuff? Regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Unexpected SessionTransaction behaviour
object at 0x9a3a7a0 checked out from pool [sqlalchemy.pool.SingletonThreadPool.0x...338c:387] DEBUG _ConnectionFairy.checkout: counter = 1 So far, everything is as before. Now, we make additional database calls, exactly as before, to populate the comboboxes. [ersatz.db:904] DEBUG DBMultiSelectField: get_choices: authors Now comes the first unexpected occurrence. The SessionTransaction with id 0x9c814cc referenced below is not the one which was created above, 0x9c8448c. Although there is a logging statement in SessionTransaction.__init__ which produces the logging output indicating an instance was created, this has not happened for ID 0x9c814cc - how/where/why is this SessionTransaction created? SQLA has taken a different path. [sqlalchemy.orm.session:310] DEBUG SessionTransaction._connection_for_bind: not in connections: SessionTransaction/0x9c814cc, Engine(sqlite:tmp/demo1.db), [] [sqlalchemy.orm.session:323] DEBUG SessionTransaction._connection_for_bind: calling bind.contextual_connect [sqlalchemy.pool.SingletonThreadPool.0x...338c:387] DEBUG _ConnectionFairy.checkout: counter = 2 The existing connection is used but its refcount is bumped up to 2 (reasonably). However, this will cause the connection to not be returned to the pool during cleanup. The remaining combobox-population calls use this connection. [ersatz.db:879] DEBUG DBSelectField: get_choices: publisher [ersatz.db:904] DEBUG DBMultiSelectField: get_choices: tags End of request, and cleanup time: [ersatz.core:747] DEBUG 0.053: time for http://localhost:5000/admin/book/add/ [sqlalchemy.orm.scoping:61] DEBUG ScopedSession.remove registry has, closing: sqlalchemy.orm.session.Session object at 0x9c841ec [sqlalchemy.orm.session:762] DEBUG Session.close [sqlalchemy.orm.session:766] DEBUG Session.close: closing transaction: sqlalchemy.orm.session.SessionTransaction object at 0x9c8448c [sqlalchemy.pool.SingletonThreadPool.0x...338c:432] DEBUG _ConnectionFairy.close: counter = 2 As a result of the counter not becoming zero when decremented, the connection is not returned to the pool, even though it should be. Is this a bug, or is there some explanation which escapes me? This doesn't appear to be related to the connection being to a SQLite database. Regards, Vinay Sajip [1] http://groups.google.com/group/sqlalchemy/browse_frm/thread/5e08e49fa1722f91 -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Unexpected SessionTransaction behaviour
On Jun 14, 3:14 pm, Michael Bayer mike...@zzzcomputing.com wrote: On Jun 14, 2010, at 9:56 AM, Vinay Sajip wrote: There's really no way to tell what's happening in your app without an example that illustrates the issue, such that if unusual behavior were observed, one could issue a pdb and step around to see the state of things as they progress. You can check the source code of Session.remove() yourself - it issues a close() on the session which detaches all objects, and then the whole thing is just thrown away. Its all Python GC accomplishing the work here, and after remove() nothing on the SQLA side is referencing that Session. Its not just SQLite that would have issues if Session.remove() left connection resources lying around - the connection would still have open transactional resources on it, which would cause locking issues with databases such as Postgresql and MS-SQL. My advice would be, assuming you cannot isolate the issue inside a small test case, to trap the application at the point at which it appears to be opening a mystery SessionTransaction, dropping into pdb, and checking around why there's more than one Session in memory, as well as checking what gc.collect() accomplishes here. That's fine, I'll keep on digging - I was just hoping for some guidance on where to dig most productively. The difficulty with making a small test case is that the memory footprint would be quite different, and the problem could fail to manifest - but I completely understand that from your point of view, it's hard to comment further without the small test case. About the mystery SessionTransaction, for example - having put a log statement in SessionTransaction.__init__, I would expect every SessionTransaction creation to be logged. I couldn't easily see how that SessionTransaction was created, though perhaps some more digging through the source would enlighten me. Regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Unexpected SessionTransaction behaviour
On Jun 14, 3:14 pm, Michael Bayer mike...@zzzcomputing.com wrote: My advice would be, assuming you cannot isolate the issue inside a small test case, to trap the application at the point at which it appears to be opening a mystery SessionTransaction, dropping into pdb, and checking around why there's more than one Session in memory, as well as checking what gc.collect() accomplishes here. Okay, I found the phantom transaction - it's being created in the session.remove() call in the cleanup of the previous request, in the following code in orm/session.py, in SessionTransaction.close(): if not self.session.autocommit: self.session.begin() I hadn't expected the SessionTransaction.close() to create a brand new SessionTransaction, sorry I missed that. But this means that SQLA starting state is different for two consecutive, identical requests - the session.remove() for the first request leaves a state which causes a different path to be taken by SQLA on the second request. How do I completely clean up after each request so that the next identical request is processed in the same way? Regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: mod_wsgi, SQLite3 and a threading anomaly because connection is not returned to pool by session.remove()
On Jun 10, 10:08 pm, Michael Bayer mike...@zzzcomputing.com wrote: First off, the absolute recommended behavior for SQLite if a file-based database is to not use pooling. I would suggest you use a NullPool to eliminate any connection pooling. Some detail on this athttp://www.sqlalchemy.org/docs/reference/dialects/sqlite.html#threadi Seems like I mention it in the linked thread as well. Indeed. I understand about NullPool, and the setup I'm using is a test setup for multi-db operation. I would expect the session.remove() to clean up any SQLA connections used during the request, but it's not happening some of the time - I was hoping to find the cause; of course if I switched to NullPool the problem would go away. You then need to ensure no sqlalchemy-session-bound objects in your web application are shared between threads. When you put things into your HTTP session, assuming its an in-memory HTTP session, make sure objects that go in there arent attached to a SQLAlchemy session. Use expunge() or similar for this. Calling remove() at the end of the request will work, except for the fact that a concurrent thread might be accessing the HTTP session before you get that far. In the failing case, requests are not actually happening concurrently - mod_wsgi just happens to use different threads for servicing the sequential requests, which are a second or two apart. The problem is caused by a connection sometimes not being found when session.remove() is called. The connection is always created in response to a query to get a User object from an integer ID stored in the session. While the User object is a SQLA mapped object, it's not stored in the session, only the ID is. Note that the problem occurs during clean up via session.remove() (connection not found to return to pool) but manifests itself on the next request (a second or two later). I'm wondering if there's some reason why an opened connection might sometimes not be registered properly [ has() returning False ] which is why it's not returned to the pool. Concurrent thread access to the session isn't (I believe) happening in this case as the server is on my local test machine with no other load than me accessing via Firefox locally. There are concurrent requests but only for static files. I would recommend using cookie based HTTP sessions in any case (see Beaker for this functionality). I presume you mean where all session state is stored in the cookie. I can do this but I'm working on a framework component which also needs to support server-side sessions. That thread regarded someone using an extremely rare tool called PyISAPIe, which had threading bugs in it. That doesn't apply to a basic mod_wsgi configuration. I see now. It was a clutching-at-straws kind of thing :-) Thanks regards, Vinay Sajip -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalch...@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] mod_wsgi, SQLite3 and a threading anomaly because connection is not returned to pool by session.remove()
/-1249039504}
DEBUG Connection sqlite3.Connection object at 0x2202f4a0 checked out
from pool
[sqlalchemy.pool.SingletonThreadPool.0x...c44c:236]{5648/-1249039504}
INFO Invalidate connection sqlite3.Connection object at 0x2202f5a0
(reason: ProgrammingError:SQLite objects created in a thread can only
be used in that same thread.The object was created in thread id
-1238549648 and this is thread id -1249039504)
[sqlalchemy.pool.SingletonThreadPool.0x...c44c:263]{5648/-1249039504}
DEBUG Closing connection sqlite3.Connection object at 0x2202f5a0
[sqlalchemy.pool.SingletonThreadPool.0x...c44c:269]{5648/-1249039504}
DEBUG Connection sqlite3.Connection object at 0x2202f5a0 threw an
error on close: SQLite objects created in a thread can only be used in
that same thread.The object was created in thread id -1238549648 and
this is thread id -1249039504
And here we have the error.
[sqlalchemy.pool.SingletonThreadPool.0x...c44c:310]{5648/-1249039504}
DEBUG Connection None being returned to pool
[ersatz.core:738]{5648/-1249039504} ERROR Error while handling
request /admin/book/add/: (ProgrammingError) SQLite objects created in
a thread can only be used in that same thread.The object was created
in thread id -1238549648 and this is thread id -1249039504
Traceback (most recent call last): part snipped
File /usr/lib/python2.6/dist-packages/ersatz/db.py, line 902, in
get_choices
for o in sorted(self.query.all(), key=get_label):
File /usr/lib/python2.6/dist-packages/sqlalchemy/orm/query.py,
line 1422, in all
return list(self)
File /usr/lib/python2.6/dist-packages/sqlalchemy/orm/query.py,
line 1534, in __iter__
return self._execute_and_instances(context)
File /usr/lib/python2.6/dist-packages/sqlalchemy/orm/query.py,
line 1539, in _execute_and_instances
mapper=self._mapper_zero_or_none())
File /usr/lib/python2.6/dist-packages/sqlalchemy/orm/session.py,
line 735, in execute
clause, params or {})
File /usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py,
line 1157, in execute
params)
File /usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py,
line 1235, in _execute_clauseelement
parameters=params
File /usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py,
line 1348, in __create_execution_context
None, None)
File /usr/lib/python2.6/dist-packages/sqlalchemy/engine/base.py,
line 1343, in __create_execution_context
connection=self, **kwargs)
File /usr/lib/python2.6/dist-packages/sqlalchemy/engine/
default.py, line 370, in __init__
self.cursor = self.create_cursor()
File /usr/lib/python2.6/dist-packages/sqlalchemy/engine/
default.py, line 511, in create_cursor
return self._connection.connection.cursor()
File /usr/lib/python2.6/dist-packages/sqlalchemy/pool.py, line
378, in cursor
c = self.connection.cursor(*args, **kwargs)
ProgrammingError: (ProgrammingError) SQLite objects created in a
thread can only be used in that same thread.The object was created in
thread id -1238549648 and this is thread id -1249039504 None [{}]
[ersatz.core:746]{5648/-1249039504} DEBUG 0.121: time for
http://localhost/admin/book/add/
[ersatz.db:328]{5648/-1249039504} DEBUG session cleanup starting
[ersatz.db:336]{5648/-1249039504} DEBUG Calling session.remove() for
db1 (sqlalchemy.orm.scoping.ScopedSession object at 0x21b6190c)
[ersatz.db:336]{5648/-1249039504} DEBUG Calling session.remove() for
db3 (sqlalchemy.orm.scoping.ScopedSession object at 0x21b61aec)
[ersatz.db:336]{5648/-1249039504} DEBUG Calling session.remove() for
db2 (sqlalchemy.orm.scoping.ScopedSession object at 0x21b61ccc)
[sqlalchemy.pool.SingletonThreadPool.0x...c44c:310]{5648/-1249039504}
DEBUG Connection sqlite3.Connection object at 0x2202f4a0 being
returned to pool
[ersatz.db:338]{5648/-1249039504} DEBUG session cleanup done
Notice that here again, the connection which was checked out of the
pool is returned to it at the end, as expected.
Using Mike Bayer's _threading_local patch in thread [1] made no
difference.
It seems as if under some circumstances the link between the
connection and the session is lost in some way, so that
session.remove() doesn't return the connection to the pool.
Can anyone help? Is there any particular place I could add more
logging or debugging to see what's going on? Have I missed something
really obvious?
Regards,
Vinay Sajip
[1]
http://groups.google.com/group/sqlalchemy/browse_thread/thread/fbca1399020f6a2e/
[2] http://paste.pocoo.org/show/223990/
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] A portable way of resetting sequences?
Is there a portable way of resetting sequences on platforms where they
are used? I am initializing tables (for unit tests) - which have an
autoincrementing id column as primary key - by uploading data from
fixture files on disk into a database. On SQLite everything works
fine, but on Postgres the sequences get initialized to 1 and need to
be set to reflect the id values which have been inserted, otherwise I
get duplicate-key errors when inserting.
What database-independent mechanism can I use to do this? I couldn't
find sufficient detail in the docs/on this list to help me figure it
out. Here's a couple of posts I looked at:
http://groups.google.com/group/sqlalchemy/msg/bd14c45a3cad801f
to which Mike Bayer said,
after populating the tables with your CSV data, manually increment
the sequence to the next available id.
Yes, that's what I'd like to do, but without any direct execution of
Postgres-specific SQL from my code. Following this post:
http://groups.google.com/group/sqlalchemy/msg/c1942b1e38f282c9
which asks about how to operate on sequences, Mike Bayer makes the
comment:
table reflection does get a value for sequence defaults in PG, and
SQLA
then knows how to execute the sequence. there is an issue
specifically
when the sequence name has been changed in that PG no longer provides
consistent access to the sequence name (theres a trac ticket for that
issue), but it works for the typical use case.
I think my use case is pretty typical, but I could use a few pointers
on how to get SQLA to execute
select setval('XXX_id_seq', max(id)) from XXX
I'm happy doing table and column reflection to decide when to make
these calls, but I want to stay at the SQLA layer so that my code
works unchanged on all mainstream databases.
Thanks for any pointers,
Vinay Sajip
--
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalch...@googlegroups.com.
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] SQLAlchemy and logging
Hi, I've just been discussing Python's logging package with Armin Ronacher of Pocoo over on stdlib-sig: http://mail.python.org/pipermail/stdlib-sig/2009-September/000671.html In that post, Armin says that the SQLAlchemy development team had a lot of problems with Python logging and had to do an Incredible dance relating to loggers for temporary database connections. I like dancing as much as the next guy, and the more incredible the better, but *unnecessary* incredible dancing is a different thing altogether ;-) Seriously, as the author of the logging package, I'm sorry if you've had any problems with logging, and I'm reasonably responsive to issues raised on Python's bug tracker. I don't recall seeing anything from the SQLAlchemy developers,but I could have missed it. Can someone enlighten me as to the specifics of these problems, and what I can do to help? Thanks and regards, Vinay Sajip P.S. I think SQLAlchemy is great! --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: SQLAlchemy and logging
On Sep 18, 3:49 pm, Michael Bayer mike...@zzzcomputing.com wrote: Vinay Sajip wrote: hmmm OK I think what he may be referring to is that we have this flag called echo which works like this: engine = create_engine(url, echo=True) what echo does is a shortcut to setting up logging and enabling INFO logging for the logger named sqlalchemy.engine. In order to ensure output, it does a basicConfig() to standard out. What happens though, particularly when people use Pylons or similar, is that they have a logging configuration in their conf file, which already sets up handlers for logging. then they'd like to use the echo flag for debugging, but it ends up setting up a second handler, so in the case of already logging to stdout you get double log messages. It would be nice if there were some APIish way we could check that handlers have already been configured, so that the echo flag wouldn't do a second handler configuration. If you are literally calling basicConfig, that checks for handlers added to the root logger (as it's intended to configure the root logger for casual/novice usage, though of course anyone can use it). If the root logger already has loggers configured, then it doesn't do anything. If you're using basicConfig(), then it's probably just an ordering problem - your code gets called first, configures the root logger, then Pylons or other framework adds another handler explicitly. This can perhaps be handled by documentation, as it's (IMO) mainly an interaction between SQLA and the other framework. I'll take a peek at SQLA code sometime soon and make suggestions if I think any changes are needed, if that's OK. the echo issue is not really a big deal. the other issue we have is one I'm not sure much can be done about. We have logging calls which we would like to make within extremely performance critical sections, like when fetching results. However if you look at our source code we jump through hoops to avoid an unnecessary method call, like: if self._should_log_debug: self.log_debug(...) otherwise the log_debug() would result in about three method calls per row even when logging is disabled. A bad side effect of this is that in many cases _should_log_debug is determined when an engine or session is first created (based on logging.isDebugEnabled() etc.) - so an application has to take care to set up their logging config before the rest of their SQLAlchemy objects are generated. Okay, let me think about that for a bit, and look at your source to see exactly what those methods do. Those are pretty much the only two things we have, thanks for the interest ! So - Armin's statement SQLAlchemy for example does an incredible dance to get separate loggers for temporary database connections. doesn't ring any bells? Also - do you have any performance numbers or even rough metrics as to the overhead from logging in your tight loops? Regards, Vinay Sajip --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Connecting to MSSQL with a System DSN on Windows
On Jan 16, 7:38 pm, Paul Johnston [EMAIL PROTECTED] wrote: Hi, 'mssql://DSN=MyDSN'. How can I get this to work without specifying You were nearly there... mssql://?dsn=mydsn Paul Thanks, Paul, For some reason it didn't work for me (sqlalchemy 0.4.2p3). I tried mssql://?dsn=MyDSN, and also mssql://?DSN=MyDSN. Oh well. Vinay --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] Re: Connecting to MSSQL with a System DSN on Windows
On Jan 17, 5:59 pm, Paul Johnston [EMAIL PROTECTED] wrote:
Hi,
For some reason it didn't work for me (sqlalchemy 0.4.2p3). I tried
mssql://?dsn=MyDSN, and also mssql://?DSN=MyDSN. Oh well.
Sorry, it's
mssql:///?dsn=mydsn
This definitely works, just tested it
eng = sqlalchemy.create_engine('mssql:///?dsn=bob')
eng.execute('select 1')
sqlalchemy.engine.base.ResultProxy object at 0x00D17CB0
Paul
Oh, I see... trying it with the extra / ... It works for me, too.
Thanks!
Vinay
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Connecting to MSSQL with a System DSN on Windows
I have a Windows setup in which I need to use Integrated
Authentication with pyodbc and MSSQL. I have set up a System DSN using
the Control Panel and pyodbc connects OK using this DSN - I can run
queries etc. without problems. I use the connection string 'DSN=MyDSN'
for pyodbc, where MyDSN is the System DSN I have set up.
However, this doesn't work if I pass it to create_engine() - e.g.
'mssql://DSN=MyDSN'. How can I get this to work without specifying
username, password or database in the URL?
Sorry if this has come up before. I saw an earlier (Sep 2007) thread
which seems to imply that you have to the username password in the
URL, and an earlier (June 2007) thread about using Driver={} on Unix.
Is there an easy way to just pass the pyodbc connection to SQLAlchemy?
I couldn't spot anything obvious in the docs.
Thanks in advance,
Vinay Sajip
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: Connecting to MSSQL with a System DSN on Windows [Solved]
On 16 Jan, 18:10, Vinay Sajip [EMAIL PROTECTED] wrote:
I have a Windows setup in which I need to use Integrated
Authentication with pyodbc and MSSQL. I have set up a System DSN using
the Control Panel and pyodbc connects OK using this DSN - I can run
queries etc. without problems. I use the connection string 'DSN=MyDSN'
for pyodbc, where MyDSN is the System DSN I have set up.
However, this doesn't work if I pass it to create_engine() - e.g.
'mssql://DSN=MyDSN'. How can I get this to work without specifying
username, password or database in the URL?
Sorry if this has come up before. I saw an earlier (Sep 2007) thread
which seems to imply that you have to the username password in the
URL, and an earlier (June 2007) thread about using Driver={} on Unix.
Is there an easy way to just pass the pyodbc connection to SQLAlchemy?
I couldn't spot anything obvious in the docs.
Problem solved. I had to use r'machinename\sqlexpress' as the host
because it's an embedded version of MSSQL. Sorry for the noise.
Vinay Sajip
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
sqlalchemy group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to [EMAIL PROTECTED]
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
