On 12/16/2015 04:07 PM, Kai Groner wrote: > Hi, > > I work on a project that uses dependency injection. We want to log > queries in the context of a particular instantiation of the injector. > This means that when we register an event listener, that listener has > references to a unique logger configuration and we don't want that > listener to leak into other contexts. > > I think the way to do this is to use a separate Engine for each > injector, but I still want to have a shared connection pool. My first > attempt looked similar to this: > > _pool = None > @Injector.factory('db_engine') > def engine_factory(): > nonlocal _pool > engine = create_engine(dburi, pool=_pool) > if _pool is None: > _pool = engine.pool > return engine
oh, OK. This is, a little bit brittle. Maybe not as much as in earlier releases but an engine.dispose() call is still going to make you sad. > > This worked. Some of the engine events are actually pool events, so > those couldn't be isolated, but I was able to avoid using those. This > also leaks memory (I think it's related to the new dialect object each > time we create an engine). Perhaps this isn't how connection pools were > meant to be used. hmmm, memory leaks, not sure, but then again no, we assume the pool is owned by a single Engine. > > Now I'm considering another way to isolate the engine and it's event > listeners. It looks like this: > > _engine = None > @Injector.factory('db_engine') > def engine_factory(): > nonlocal _engine > if _engine is None: > _engine = create_engine(dburi) > engine = copy.copy(_engine) > engine.dispatch = copy.copy(engine.dispatch) > return engine that looks like something that would totally break. the event dispatch system is extremely meta / dynamic / weak referenced / special and a blunt tool like copy(), I can't imagine what that would do. > > This works ok, and it's leaking less memory than it was before. OK well it's not like we're trying to seal with duct tape, any leak means, "it's broke" :) But, > I've made some assumptions that I probably shouldn't be making. > > So, I'm looking for comments or advice on whether this is a terrible > idea, or if there's another way I should be approaching this. You can make event-independent "proxies" of engines using the engine.execution_options() feature. You can define whatever keywords you want there (or not send any), and you can make event listeners that are local to that engine and alternatively can peek into the _execution_options dictionary (which means you could really just have one listener that dispatches based on something in the dictionary). from sqlalchemy import create_engine, event e = create_engine("sqlite://", echo=True) def listen_for_thing(keyword, engine): @event.listens_for(engine, "before_cursor_execute") def before_cursor_execute( conn, cursor, statement, parameters, context, executemany): print("statement for engine %s / %s: '%s'" % ( keyword, conn._execution_options['our_engine_name'], statement)) e1 = e.execution_options(our_engine_name='engine1') e2 = e.execution_options(our_engine_name='engine2') listen_for_thing('engine1', e1) listen_for_thing('engine2', e2) e1.execute("select 'engine1'") e2.execute("select 'engine2'") > > > Thanks, > Kai > > -- > You received this message because you are subscribed to the Google > Groups "sqlalchemy" group. > To unsubscribe from this group and stop receiving emails from it, send > an email to sqlalchemy+unsubscr...@googlegroups.com > <mailto:sqlalchemy+unsubscr...@googlegroups.com>. > To post to this group, send email to sqlalchemy@googlegroups.com > <mailto:sqlalchemy@googlegroups.com>. > Visit this group at https://groups.google.com/group/sqlalchemy. > For more options, visit https://groups.google.com/d/optout. -- You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To unsubscribe from this group and stop receiving emails from it, send an email to sqlalchemy+unsubscr...@googlegroups.com. To post to this group, send email to sqlalchemy@googlegroups.com. Visit this group at https://groups.google.com/group/sqlalchemy. For more options, visit https://groups.google.com/d/optout.