[sqlalchemy] Re: detached, pickled sqlalchemy collection object failing after server restart
Thanks for the compile_mappers() call - that is a lot easier than calling compile() on all the individual mappers. What is very strange is that I can see at the point of my first calling on the object from the beaker session, I retrieve it and merge it successfully into the SQLAlchemy Session. I have put debug statements in and they get logged before the throwing of the exception which in my mind proves that the mappers are initialized successfully prior to the error. I'm really not sure what beaker is up to. Another hint that may help is that after the throwing of the exception after server restart, if I just reload the page (its a web app) everything resolves itself. It is only on initial initialization of which it recovers successfully on a subsequent call. On Oct 22, 7:35 am, "Michael Bayer" wrote: > rootsmith wrote: > > > Well, it appears I have answered my own question and, yes, it was a > > SQLAlchemy issue. I resolved the problem by explicitly compiling the > > mapping at the end of the model initialization: > > > manager_mapper = orm.mapper(Manager, manager_table, ... etc. > > ... (all other dependent mappers) > > > manager_mapper.compile() > > > My extremely vague understanding of what is going on is that if you > > are explicitly using the SQLAlchemy API to retrieve the pickled object > > then compiling is done implicitly but if the API is not used then the > > compiling is not done and you run into troubles. I am not sure how > > the beaker session is trying to interact with the object that causes > > the problem versus my own testing of interacting with the pickled > > object being successful but compiling is the answer. > > > If someone has a more robust explanation I would be very interested to > > hear it. > > that seems like a tricky error to reproduce. When your application > starts, assuming it sets up mappers immediately, as soon as they are used > all mappers are compiled. pickling/depickling wouldn't have any > compilation related issues. Additionally your error seems to occur > during *serialization*, which is very strange. Its not occurring to me > how your application would have a hold of a mapped instance without your > mappers having been compiled...unless that object was pulled from a cache > previously. > > In any case if you throw a compile_mappers() at the end of environment.py > load_environment() you shouldn't have this kind of issue. but i would be > curious how you're getting it to do that. > > > > > On Oct 21, 10:08 pm, rootsmith wrote: > >> Hi, > > >> I have a pylons application using file based beaker sessions (not to > >> be confused with SQLAlchemy Sessions.) I retrieve a mapped object > >> from the database, detach it from the SQLAlchemy Session like so: > > >> Session.expunge(object) > > >> and save it to the beaker session (basically pickling it to a file.) > >> like so: > > >> session['user'] = user > >> session.save() > > >> If I restart the server and retrieve the pickled object from the > >> session, merge it back into the SQLAlchemy Session everything seems to > >> be fine. > > >> user = Session.merge(session['user'], dont_load=True) > > >> But then somewhere in the beaker session middleware something goes > >> awry before it completes the call to the controller. > > >> Now I know this sounds like a beaker session problem but it is failing > >> when it is making a call to the SQLAlchemy object. Here is the stack > >> trace: > > >> URL:http://127.0.0.1:5000/ > >> File '/Users/kevin/mydevenv/lib/python2.6/site-packages/ > >> WebError-0.10.1-py2.6.egg/weberror/evalexception.py', line 431 in > >> respond > >> app_iter = self.application(environ, detect_start_response) > >> File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Paste-1.7.2- > >> py2.6.egg/paste/recursive.py', line 80 in __call__ > >> return self.application(environ, start_response) > >> File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > >> py2.6.egg/beaker/middleware.py', line 73 in __call__ > >> return self.app(environ, start_response) > >> File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > >> py2.6.egg/beaker/middleware.py', line 152 in __call__ > >> return self.wrap_app(environ, session_start_response) > >> File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Routes-1.10.3- > >> py2.6.egg/routes/middleware.py', line 130 in __call__
[sqlalchemy] Re: detached, pickled sqlalchemy collection object failing after server restart
Well, it appears I have answered my own question and, yes, it was a SQLAlchemy issue. I resolved the problem by explicitly compiling the mapping at the end of the model initialization: manager_mapper = orm.mapper(Manager, manager_table, ... etc. ... (all other dependent mappers) manager_mapper.compile() My extremely vague understanding of what is going on is that if you are explicitly using the SQLAlchemy API to retrieve the pickled object then compiling is done implicitly but if the API is not used then the compiling is not done and you run into troubles. I am not sure how the beaker session is trying to interact with the object that causes the problem versus my own testing of interacting with the pickled object being successful but compiling is the answer. If someone has a more robust explanation I would be very interested to hear it. On Oct 21, 10:08 pm, rootsmith wrote: > Hi, > > I have a pylons application using file based beaker sessions (not to > be confused with SQLAlchemy Sessions.) I retrieve a mapped object > from the database, detach it from the SQLAlchemy Session like so: > > Session.expunge(object) > > and save it to the beaker session (basically pickling it to a file.) > like so: > > session['user'] = user > session.save() > > If I restart the server and retrieve the pickled object from the > session, merge it back into the SQLAlchemy Session everything seems to > be fine. > > user = Session.merge(session['user'], dont_load=True) > > But then somewhere in the beaker session middleware something goes > awry before it completes the call to the controller. > > Now I know this sounds like a beaker session problem but it is failing > when it is making a call to the SQLAlchemy object. Here is the stack > trace: > > URL:http://127.0.0.1:5000/ > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/ > WebError-0.10.1-py2.6.egg/weberror/evalexception.py', line 431 in > respond > app_iter = self.application(environ, detect_start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Paste-1.7.2- > py2.6.egg/paste/recursive.py', line 80 in __call__ > return self.application(environ, start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/middleware.py', line 73 in __call__ > return self.app(environ, start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/middleware.py', line 152 in __call__ > return self.wrap_app(environ, session_start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Routes-1.10.3- > py2.6.egg/routes/middleware.py', line 130 in __call__ > response = self.app(environ, start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Pylons-0.9.7- > py2.6.egg/pylons/wsgiapp.py', line 125 in __call__ > response = self.dispatch(controller, environ, start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Pylons-0.9.7- > py2.6.egg/pylons/wsgiapp.py', line 324 in dispatch > return controller(environ, start_response) > File '/Users/kevin/Documents/workspace/ryzoe/ryzoe/lib/base.py', line > 54 in __call__ > return WSGIController.__call__(self, environ, start_response) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Pylons-0.9.7- > py2.6.egg/pylons/controllers/core.py', line 284 in __call__ > return response(environ, self.start_response) > File 'build/bdist.macosx-10.6-universal/egg/webob/__init__.py', line > 2182 in __call__ > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/middleware.py', line 146 in session_start_response > session.persist() > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/session.py', line 600 in persist > self._session().save() > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/session.py', line 293 in save > self.namespace.release_write_lock() > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/container.py', line 149 in release_write_lock > self.close(checkcount=True) > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/container.py', line 172 in close > self.do_close() > File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1- > py2.6.egg/beaker/container.py', line 536 in do_close > cPickle.dump(self.hash, fh) > File '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/ > python2.6/copy_reg.py', line 84 in _re
[sqlalchemy] detached, pickled sqlalchemy collection object failing after server restart
Hi,
I have a pylons application using file based beaker sessions (not to
be confused with SQLAlchemy Sessions.) I retrieve a mapped object
from the database, detach it from the SQLAlchemy Session like so:
Session.expunge(object)
and save it to the beaker session (basically pickling it to a file.)
like so:
session['user'] = user
session.save()
If I restart the server and retrieve the pickled object from the
session, merge it back into the SQLAlchemy Session everything seems to
be fine.
user = Session.merge(session['user'], dont_load=True)
But then somewhere in the beaker session middleware something goes
awry before it completes the call to the controller.
Now I know this sounds like a beaker session problem but it is failing
when it is making a call to the SQLAlchemy object. Here is the stack
trace:
URL: http://127.0.0.1:5000/
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/
WebError-0.10.1-py2.6.egg/weberror/evalexception.py', line 431 in
respond
app_iter = self.application(environ, detect_start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Paste-1.7.2-
py2.6.egg/paste/recursive.py', line 80 in __call__
return self.application(environ, start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/middleware.py', line 73 in __call__
return self.app(environ, start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/middleware.py', line 152 in __call__
return self.wrap_app(environ, session_start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Routes-1.10.3-
py2.6.egg/routes/middleware.py', line 130 in __call__
response = self.app(environ, start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Pylons-0.9.7-
py2.6.egg/pylons/wsgiapp.py', line 125 in __call__
response = self.dispatch(controller, environ, start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Pylons-0.9.7-
py2.6.egg/pylons/wsgiapp.py', line 324 in dispatch
return controller(environ, start_response)
File '/Users/kevin/Documents/workspace/ryzoe/ryzoe/lib/base.py', line
54 in __call__
return WSGIController.__call__(self, environ, start_response)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Pylons-0.9.7-
py2.6.egg/pylons/controllers/core.py', line 284 in __call__
return response(environ, self.start_response)
File 'build/bdist.macosx-10.6-universal/egg/webob/__init__.py', line
2182 in __call__
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/middleware.py', line 146 in session_start_response
session.persist()
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/session.py', line 600 in persist
self._session().save()
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/session.py', line 293 in save
self.namespace.release_write_lock()
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/container.py', line 149 in release_write_lock
self.close(checkcount=True)
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/container.py', line 172 in close
self.do_close()
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/Beaker-1.4.1-
py2.6.egg/beaker/container.py', line 536 in do_close
cPickle.dump(self.hash, fh)
File '/System/Library/Frameworks/Python.framework/Versions/2.6/lib/
python2.6/copy_reg.py', line 84 in _reduce_ex
dict = getstate()
File '/Users/kevin/mydevenv/lib/python2.6/site-packages/
SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/collections.py', line 607 in
__getstate__
return {'key': self.attr.key,
AttributeError: 'NoneType' object has no attribute 'key'
Here is the mapper of the object that is pickled:
orm.mapper(Manager, manager_table, inherits=User,
polymorphic_identity='manager',
properties={'residences': relation(Residence,
lazy=False,
back_populates='manager')})
In the end, I can always just flush the cache (delete the files) on a
server restart but something is striking me odd as to why this is
failing. I want to make sure I don't have some flawed understanding
of something somewhere.
Any help would be greatly appreciated.
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
"sqlalchemy" group.
To post to this group, send email to sqlalchemy@googlegroups.com
To unsubscribe from this group, send email to
sqlalchemy+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/sqlalchemy?hl=en
-~--~~~~--~~--~--~---
[sqlalchemy] Re: table name in foreign key in wrong case on Mac OS X
Just a quick update ... Mac OS X is case preserving but case insensitive so a query directly within mysql for table "USER" using "user" will result correctly. So this just deepens the mystery more - on UNIX either MySQLdb or SQLAlchemy is preserving the case of the table name as "USER" but on Mac OS X it is not preserving it, which should work, but whatever routine that underlies the search for the foreign key in the example I have given is not working. On Oct 1, 10:27 pm, rootsmith wrote: > Previously I was doing development of a project on an Ubuntu 9.04 > machine with MySQL 5.0 and just recently switched to doing development > on a MacBook Pro. I installed identical versions of MySQL on the > MacBook. I used the exact same source files and installed the same > versions of all libraries for the project, however, when I try to > initialize the model on the MacBook I get the following stack trace: > > File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ > SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/__init__.py", line 751, in > mapper > return Mapper(class_, local_table, *args, **params) > File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ > SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/mapper.py", line 194, in > __init__ > self._configure_inheritance() > File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ > SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/mapper.py", line 239, in > _configure_inheritance > self.inherit_condition = sqlutil.join_condition > (self.inherits.local_table, self.local_table) > File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ > SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/sql/util.py", line 135, in > join_condition > "between '%s' and '%s'" % (a.description, b.description)) > sqlalchemy.exc.ArgumentError: Can't find any foreign key relationships > between 'USER' and 'MANAGER' > > This is despite the fact that there is a foreign relation between the > two as follows on the MANAGER table: > > CONSTRAINT FK_MANAGER_USER FOREIGN KEY (user_id) > REFERENCES `USER`(user_id) > > I have traced the problem so far to the fact that on Ubuntu, the > ForeignKey in SQLAlchemy is built as ForeignKey(u'USER.user_id') and > on the Mac it is ForeignKey(u'user.user_id'). This has been confirmed > by tracing the code into sql.util.join_condition where the call > fk.get_referant() is being made. > > Note again, this is the EXACT SAME code running on both machines. I > don't know if it is a MySQL setting (unlikely), MySQLdb, or SQLAlchemy > that is causing the incorrect case issue on the table name. Does > anyone know off hand what might be going on here? > > Just to confirm, the table in the database is called "USER" and > therefore any search for "user" will not yield any results. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
[sqlalchemy] table name in foreign key in wrong case on Mac OS X
Previously I was doing development of a project on an Ubuntu 9.04 machine with MySQL 5.0 and just recently switched to doing development on a MacBook Pro. I installed identical versions of MySQL on the MacBook. I used the exact same source files and installed the same versions of all libraries for the project, however, when I try to initialize the model on the MacBook I get the following stack trace: File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/__init__.py", line 751, in mapper return Mapper(class_, local_table, *args, **params) File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/mapper.py", line 194, in __init__ self._configure_inheritance() File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/orm/mapper.py", line 239, in _configure_inheritance self.inherit_condition = sqlutil.join_condition (self.inherits.local_table, self.local_table) File "/Users/kevin/mydevenv/lib/python2.6/site-packages/ SQLAlchemy-0.5.6-py2.6.egg/sqlalchemy/sql/util.py", line 135, in join_condition "between '%s' and '%s'" % (a.description, b.description)) sqlalchemy.exc.ArgumentError: Can't find any foreign key relationships between 'USER' and 'MANAGER' This is despite the fact that there is a foreign relation between the two as follows on the MANAGER table: CONSTRAINT FK_MANAGER_USER FOREIGN KEY (user_id) REFERENCES `USER`(user_id) I have traced the problem so far to the fact that on Ubuntu, the ForeignKey in SQLAlchemy is built as ForeignKey(u'USER.user_id') and on the Mac it is ForeignKey(u'user.user_id'). This has been confirmed by tracing the code into sql.util.join_condition where the call fk.get_referant() is being made. Note again, this is the EXACT SAME code running on both machines. I don't know if it is a MySQL setting (unlikely), MySQLdb, or SQLAlchemy that is causing the incorrect case issue on the table name. Does anyone know off hand what might be going on here? Just to confirm, the table in the database is called "USER" and therefore any search for "user" will not yield any results. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups "sqlalchemy" group. To post to this group, send email to sqlalchemy@googlegroups.com To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en -~--~~~~--~~--~--~---
