Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-27 Thread Mike Bayer
On 02/27/2016 08:48 AM, Will Angenent wrote: The code was just to reproduce the exact SQLA error by doing what alembic and SQLA are doing and additionally determining when the garbage collector runs. The theory is that garbage gets collected somewhere in between list(_mapper_registry) in

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-27 Thread Will Angenent
The code was just to reproduce the exact SQLA error by doing what alembic and SQLA are doing and additionally determining when the garbage collector runs. The theory is that garbage gets collected somewhere in between list(_mapper_registry) in configure_mappers and what happens in

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-23 Thread Mike Bayer
On 02/23/2016 03:38 PM, Will Angenent wrote: Hi, I think I’ve finally got to the bottom of it. This triggers the problem: - Load a module with some model classes using imp.load_source - Delete the module from sys.modules - Add a reference to the model classes using _mapper_registry.keys() so

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-22 Thread Will Angenent
Hi, You’re definitely on to something when you say it’s related to garbage collection and weak references. If I add this: import gc gc.disable() the problem vanishes. I’ve been spending the last couple of hours trying to reduce the code to the smallest possible case. I’ve reached the point

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-21 Thread Mike Bayer
Well what makes no sense is why these mappers and classes would be present at all. These classes are strictly defined within an alembic migration, and make **no** callouts to any other mapped structures in the application, correct? If so, when the module is garbage collected, all of the

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-21 Thread Will Angenent
Hi Mike, Thanks for your quick response yet again! Here’s the stack trace. tests/integration/test_database.py:14: in test_database_is_up_to_date create_test_db(session) __init__.py:111: in create_test_db pd_utils.do_import(dtype='locations', ifile=yaml_file)

Re: [sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-21 Thread Mike Bayer
Hi there - Can you post a stack trace, and also is your test suite making use of clear_mappers() ? The sys.modules activity is not really the primary cause, it's that alembic makes use of a module object in a temporary way. > On Feb 21, 2016, at 1:48 PM, Will Angenent

[sqlalchemy] alembic del sys.modules[module_id] leading to orm mapper error

2016-02-21 Thread Will Angenent
Hi, We had this interesting issue recently, and I've been trying to figure out if we deserve this, if this is simply unavoidable, or whether it can be considered a bug. We're using python 2.7.6, sqlalchemy 1.0.12 and alembic 0.8.4. Summary: This statement in