Re: [sqlalchemy] SQLAlchemy Engine cleanup
On Jan 11, 2013, at 12:42 AM, Anoop K wrote:
> I am not holding engine or using event.listen anywhere. Code is as simple as
>
> def create_sn(url):
> engine = create_engine(url)
> return sessionmaker(bind=engine)(expire_on_commit=False)
>
> sn = create_sn(url)
> try:
>sn...
> finally:
> sn.close()
> sn.bind.dispose()
>
>
> In engine/strategies.py DefaultEngineStrategy => create
>
> I see
>
> event.listen(pool, 'first_connect', ...)
> event.listen(pool, 'connect', ...)
>
> But didn't find an equivalent event.remove ??
there's no remove for events implemented right now. When the host object of
an event is garbage collected, so are its listeners that are not referred to
elsewhere.
Here is a test, confirms the Engine is gone using your code sample:
from sqlalchemy import create_engine
from sqlalchemy.orm import sessionmaker
import weakref
import gc
the_weak_ref = None
engine_is_removed = False
def engine_removed(ref):
global engine_is_removed
engine_is_removed = True
print "Engine was GCed!"
def create_sn(url):
global the_weak_ref
engine = create_engine(url)
the_weak_ref = weakref.ref(engine, engine_removed)
return sessionmaker(bind=engine)(expire_on_commit=False)
sn = create_sn("sqlite://")
sn.execute("select 1")
sn.close()
print "about to delete sn..."
del sn
print "about to gc.collect()..."
gc.collect() # this is always needed to make sure cycles are cleared in python
assert engine_is_removed
output (using CPython, results will differ with pypy):
about to delete sn...
about to gc.collect()...
Engine was GCed!
>
> Anoop
>
>
>
>
>
> On Friday, 11 January 2013 11:03:57 UTC+5:30, Michael Bayer wrote:
> this is all about whatever you've done specifically. If you have the Session
> instance still around, where my_session.bind still points to your Engine,
> then that Engine is still in memory - you're holding a reference to it.
> Similarly, if you've assigned an event to the Session class whose callable
> refers to the Engine in question, then the Engine is still referenced as
> well, such as:
>
> def make_evt(engine):
> @event.listens_for(Session, "before_flush")
> def go(session, ctx):
> # do something with engine
>
> make_evt(my_engine)
>
> the above code will associate "my_engine" with a class-level Session listener
> and cannot be dereferenced without unloading all of sqlalchemy.orm.
>
> dispose() and NullPool have *nothing to do* with the Engine itself, only the
> connections inside the pool contained within. Your Engine will remain
> present as long as its reachable via references, just like any other Python
> object.
>
>
>
> On Jan 10, 2013, at 11:55 PM, Anoop K wrote:
>
>> I tried session.bind.dispose() and NullPool. But engine is still not
>> cleared. It still seems to have some reference from SessionEventsDispatch =>
>> sqlalchemy.event._DispatchDescriptor.
>>
>>
>>
>> On Friday, 11 January 2013 05:24:19 UTC+5:30, Michael Bayer wrote:
>>
>> On Jan 10, 2013, at 8:25 AM, Anoop K wrote:
>>
>>> It seems engine created using create_engine is not freed after cleanup of a
>>> session. Object graph indicates that event.listen for pool is holding
>>> reference to engine even after session is garbage collected.
>>> What is the right way/api to delete an engine ?
>>>
>>> UseCase
>>> As there are lot of databases in multiple boxes whose ip/user/password can
>>> change I would like to destroy session and engine after use. Cost of
>>> creating/closing connection can be ignored for my use case.
>>>
>>>
>>
>> engines can be disposed using engine.dispose() to close connections referred
>> to by the pool, and then removing all references to that engine.
>>
>> if you aren't concerned about connection pooling (which seems to be the case
>> here if you're looking to dispose engines regularly) then just use NullPool
>> with create_engine(..., pool_class=NullPool). Then dispose() isn't really
>> needed, just lose references to the engine.
>>
>>
>>
>>
>>
>>
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups
>>> "sqlalchemy" group.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msg/sqlalchemy/-/bTtgyK4eAy4J.
>>> To post to this group, send email to sqlal...@googlegroups.com.
>>> To unsubscribe from this group, send email to
>>> sqlalchemy+...@googlegroups.com.
>>> For more options, visit this group at
>>> http://groups.google.com/group/sqlalchemy?hl=en.
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "sqlalchemy" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/sqlalchemy/-/xtb_18kV1ZUJ.
>> To post to this group, send email to sqlal...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> sqlalchemy+...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/gr
Re: [sqlalchemy] SQLAlchemy Engine cleanup
I am not holding engine or using event.listen anywhere. Code is as simple as def create_sn(url): engine = create_engine(url) return sessionmaker(bind=engine)(expire_on_commit=False) sn = create_sn(url) try: sn... finally: sn.close() sn.bind.dispose() In engine/strategies.py DefaultEngineStrategy => create I see event.listen(pool, 'first_connect', ...) event.listen(pool, 'connect', ...) But didn't find an equivalent event.remove ?? Anoop On Friday, 11 January 2013 11:03:57 UTC+5:30, Michael Bayer wrote: > > this is all about whatever you've done specifically. If you have the > Session instance still around, where my_session.bind still points to your > Engine, then that Engine is still in memory - you're holding a reference to > it. Similarly, if you've assigned an event to the Session class whose > callable refers to the Engine in question, then the Engine is still > referenced as well, such as: > > def make_evt(engine): > @event.listens_for(Session, "before_flush") > def go(session, ctx): > # do something with engine > > make_evt(my_engine) > > the above code will associate "my_engine" with a class-level Session > listener and cannot be dereferenced without unloading all of sqlalchemy.orm. > > dispose() and NullPool have *nothing to do* with the Engine itself, only > the connections inside the pool contained within. Your Engine will remain > present as long as its reachable via references, just like any other Python > object. > > > > On Jan 10, 2013, at 11:55 PM, Anoop K wrote: > > I tried session.bind.dispose() and NullPool. But engine *is still not > cleared*. It still seems to have some reference from > SessionEventsDispatch => sqlalchemy.event._DispatchDescriptor. > > > > On Friday, 11 January 2013 05:24:19 UTC+5:30, Michael Bayer wrote: >> >> >> On Jan 10, 2013, at 8:25 AM, Anoop K wrote: >> >> It seems engine created using *create_engine* is not freed after cleanup >> of a session. Object graph indicates that event.listen for pool is holding >> reference to engine even after session is garbage collected. >> *What is the right way/api to delete an engine ?* >> >> *UseCase* >> As there are lot of databases in multiple boxes whose ip/user/password >> can change I would like to destroy *session and engine* after use. Cost >> of creating/closing connection can be ignored for my use case. >> >> >> >> engines can be disposed using engine.dispose() to close connections >> referred to by the pool, and then removing all references to that engine. >> >> if you aren't concerned about connection pooling (which seems to be the >> case here if you're looking to dispose engines regularly) then just use >> NullPool with create_engine(..., pool_class=NullPool). Then dispose() >> isn't really needed, just lose references to the engine. >> >> >> >> >> >> >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/sqlalchemy/-/bTtgyK4eAy4J. >> To post to this group, send email to sqlal...@googlegroups.com. >> To unsubscribe from this group, send email to >> sqlalchemy+...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/sqlalchemy?hl=en. >> >> >> > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/sqlalchemy/-/xtb_18kV1ZUJ. > To post to this group, send email to sqlal...@googlegroups.com > . > To unsubscribe from this group, send email to > sqlalchemy+...@googlegroups.com . > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/N65z6tSD6IsJ. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] SQLAlchemy Engine cleanup
this is all about whatever you've done specifically. If you have the Session instance still around, where my_session.bind still points to your Engine, then that Engine is still in memory - you're holding a reference to it. Similarly, if you've assigned an event to the Session class whose callable refers to the Engine in question, then the Engine is still referenced as well, such as: def make_evt(engine): @event.listens_for(Session, "before_flush") def go(session, ctx): # do something with engine make_evt(my_engine) the above code will associate "my_engine" with a class-level Session listener and cannot be dereferenced without unloading all of sqlalchemy.orm. dispose() and NullPool have *nothing to do* with the Engine itself, only the connections inside the pool contained within. Your Engine will remain present as long as its reachable via references, just like any other Python object. On Jan 10, 2013, at 11:55 PM, Anoop K wrote: > I tried session.bind.dispose() and NullPool. But engine is still not cleared. > It still seems to have some reference from SessionEventsDispatch => > sqlalchemy.event._DispatchDescriptor. > > > > On Friday, 11 January 2013 05:24:19 UTC+5:30, Michael Bayer wrote: > > On Jan 10, 2013, at 8:25 AM, Anoop K wrote: > >> It seems engine created using create_engine is not freed after cleanup of a >> session. Object graph indicates that event.listen for pool is holding >> reference to engine even after session is garbage collected. >> What is the right way/api to delete an engine ? >> >> UseCase >> As there are lot of databases in multiple boxes whose ip/user/password can >> change I would like to destroy session and engine after use. Cost of >> creating/closing connection can be ignored for my use case. >> >> > > engines can be disposed using engine.dispose() to close connections referred > to by the pool, and then removing all references to that engine. > > if you aren't concerned about connection pooling (which seems to be the case > here if you're looking to dispose engines regularly) then just use NullPool > with create_engine(..., pool_class=NullPool). Then dispose() isn't really > needed, just lose references to the engine. > > > > > > >> >> >> >> -- >> You received this message because you are subscribed to the Google Groups >> "sqlalchemy" group. >> To view this discussion on the web visit >> https://groups.google.com/d/msg/sqlalchemy/-/bTtgyK4eAy4J. >> To post to this group, send email to sqlal...@googlegroups.com. >> To unsubscribe from this group, send email to >> sqlalchemy+...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/sqlalchemy?hl=en. > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/sqlalchemy/-/xtb_18kV1ZUJ. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] SQLAlchemy Engine cleanup
I tried session.bind.dispose() and NullPool. But engine *is still not cleared*. It still seems to have some reference from SessionEventsDispatch => sqlalchemy.event._DispatchDescriptor. On Friday, 11 January 2013 05:24:19 UTC+5:30, Michael Bayer wrote: > > > On Jan 10, 2013, at 8:25 AM, Anoop K wrote: > > It seems engine created using *create_engine* is not freed after cleanup > of a session. Object graph indicates that event.listen for pool is holding > reference to engine even after session is garbage collected. > *What is the right way/api to delete an engine ?* > > *UseCase* > As there are lot of databases in multiple boxes whose ip/user/password can > change I would like to destroy *session and engine* after use. Cost of > creating/closing connection can be ignored for my use case. > > > > engines can be disposed using engine.dispose() to close connections > referred to by the pool, and then removing all references to that engine. > > if you aren't concerned about connection pooling (which seems to be the > case here if you're looking to dispose engines regularly) then just use > NullPool with create_engine(..., pool_class=NullPool). Then dispose() > isn't really needed, just lose references to the engine. > > > > > > > > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/sqlalchemy/-/bTtgyK4eAy4J. > To post to this group, send email to sqlal...@googlegroups.com > . > To unsubscribe from this group, send email to > sqlalchemy+...@googlegroups.com . > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/xtb_18kV1ZUJ. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] SQLAlchemy Engine cleanup
On Jan 10, 2013, at 8:25 AM, Anoop K wrote: > It seems engine created using create_engine is not freed after cleanup of a > session. Object graph indicates that event.listen for pool is holding > reference to engine even after session is garbage collected. > What is the right way/api to delete an engine ? > > UseCase > As there are lot of databases in multiple boxes whose ip/user/password can > change I would like to destroy session and engine after use. Cost of > creating/closing connection can be ignored for my use case. > > engines can be disposed using engine.dispose() to close connections referred to by the pool, and then removing all references to that engine. if you aren't concerned about connection pooling (which seems to be the case here if you're looking to dispose engines regularly) then just use NullPool with create_engine(..., pool_class=NullPool). Then dispose() isn't really needed, just lose references to the engine. > > > > -- > You received this message because you are subscribed to the Google Groups > "sqlalchemy" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/sqlalchemy/-/bTtgyK4eAy4J. > To post to this group, send email to sqlalchemy@googlegroups.com. > To unsubscribe from this group, send email to > sqlalchemy+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] SQLAlchemy Engine cleanup
It seems engine created using *create_engine* is not freed after cleanup of a session. Object graph indicates that event.listen for pool is holding reference to engine even after session is garbage collected. *What is the right way/api to delete an engine ?* *UseCase* As there are lot of databases in multiple boxes whose ip/user/password can change I would like to destroy *session and engine* after use. Cost of creating/closing connection can be ignored for my use case. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To view this discussion on the web visit https://groups.google.com/d/msg/sqlalchemy/-/bTtgyK4eAy4J. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
