Re: [sqlalchemy] Re: Conflicting state in identity map?
On May 20, 2011, at 11:30 PM, Adrian wrote: Ok, I'll definitely do some quality debugging... Just to be clear -- I **don't** have to worry about closing my sessions in each controller? the best practice we recommend is that in a post-request event handler , you call remove() on your scoped_session() to remove all state and connection resources.turbogears should be handling this from what I know. On May 20, 6:08 pm, Michael Bayer mike...@zzzcomputing.com wrote: On May 20, 2011, at 4:12 PM, Adrian wrote: So if it is the latter, that the Session is being shared amongst threads, what is the correct way to handle the sessions from within Turbogears? What I do now is create a scoped_session in the model, and import it into each controller. For a while, I made use of special functions _before() and _after() to create the session in _before the page loads, and close it in _after, but then I was getting DetatchedInstanceErrors when I tried to access columns from objects returned to the template. I'm not sure how familiar you are with Turbogears, so I apologize if this is too much of a TG question...but I asked this on their mailing list and their answer was that what I was doing was correct -- obviously it's not if I'm getting these AssertionErrors... Thanks for the quick reply, as usual! So scoped_session() will ensure that sessions are associated with threads. Its not a total guarantee of thread safety if for example you're placing objects in some kind of in-memory cache, then using them in other threads without detaching them first from their original session. You really need to figure out what the catalyst for the issue was - short of locating the actual cause of the bug, that would produce the most clues towards what it is. Or, this is a really harsh approach that I had to do once when a deeply nested call to a defunct Google service started crashing the site, I had to disable all pages on the site, then slowly turn one page on after the next to isolate which one was the cause of the issue. Clearly that isn't an option in lots of cases but it depends on if you can reproduce the issue locally, perhaps when load testing with Apache ab and such. If its something I had seen before that would help but I've never seen anyone hitting that assertion before. On May 20, 9:59 am, Michael Bayer mike...@zzzcomputing.com wrote: On May 20, 2011, at 9:45 AM, Adrian wrote: I have a Turbogears server that uses sqlalchemy to interface with a postgres database. Today, I noticed the server was down, so I tried restarting it. Now my turbogears log is full of errors like: AssertionError: A conflicting state is already present in the identity map for key (class 'dr8db.ModelClasses.FITSHeaderKeyword', (1045,)) and Exception KeyError: KeyError((class 'dr8db.ModelClasses.MaskbitsType', (61,)),) in bound method InstanceState._cleanup of sqlalchemy.orm.state.InstanceState object at 0x4445c90 ignored I tried googling this stuff, but found nothing... Basically it lets me start the paster (Turbogears) server, but after ~5-10 minutes the server dies and there are hundreds of these errors in the log -- help!! I need to get this server back up ASAP! That's an assertion that is generally unreachable from within the Session. The only ways I think you could get there would be via direct manipulation of session.identity_map, or if the Session is being shared among concurrent threads, which is not supported. The main thing you'd be looking for here is, at what point did this server begin to fail and what event precluded that happening ?Either a code update, or perhaps the app was never tested against its current load, are the two possibilities. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Conflicting state in identity map?
So if it is the latter, that the Session is being shared amongst threads, what is the correct way to handle the sessions from within Turbogears? What I do now is create a scoped_session in the model, and import it into each controller. For a while, I made use of special functions _before() and _after() to create the session in _before the page loads, and close it in _after, but then I was getting DetatchedInstanceErrors when I tried to access columns from objects returned to the template. I'm not sure how familiar you are with Turbogears, so I apologize if this is too much of a TG question...but I asked this on their mailing list and their answer was that what I was doing was correct -- obviously it's not if I'm getting these AssertionErrors... Thanks for the quick reply, as usual! On May 20, 9:59 am, Michael Bayer mike...@zzzcomputing.com wrote: On May 20, 2011, at 9:45 AM, Adrian wrote: I have a Turbogears server that uses sqlalchemy to interface with a postgres database. Today, I noticed the server was down, so I tried restarting it. Now my turbogears log is full of errors like: AssertionError: A conflicting state is already present in the identity map for key (class 'dr8db.ModelClasses.FITSHeaderKeyword', (1045,)) and Exception KeyError: KeyError((class 'dr8db.ModelClasses.MaskbitsType', (61,)),) in bound method InstanceState._cleanup of sqlalchemy.orm.state.InstanceState object at 0x4445c90 ignored I tried googling this stuff, but found nothing... Basically it lets me start the paster (Turbogears) server, but after ~5-10 minutes the server dies and there are hundreds of these errors in the log -- help!! I need to get this server back up ASAP! That's an assertion that is generally unreachable from within the Session. The only ways I think you could get there would be via direct manipulation of session.identity_map, or if the Session is being shared among concurrent threads, which is not supported. The main thing you'd be looking for here is, at what point did this server begin to fail and what event precluded that happening ? Either a code update, or perhaps the app was never tested against its current load, are the two possibilities. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
Re: [sqlalchemy] Re: Conflicting state in identity map?
On May 20, 2011, at 4:12 PM, Adrian wrote: So if it is the latter, that the Session is being shared amongst threads, what is the correct way to handle the sessions from within Turbogears? What I do now is create a scoped_session in the model, and import it into each controller. For a while, I made use of special functions _before() and _after() to create the session in _before the page loads, and close it in _after, but then I was getting DetatchedInstanceErrors when I tried to access columns from objects returned to the template. I'm not sure how familiar you are with Turbogears, so I apologize if this is too much of a TG question...but I asked this on their mailing list and their answer was that what I was doing was correct -- obviously it's not if I'm getting these AssertionErrors... Thanks for the quick reply, as usual! So scoped_session() will ensure that sessions are associated with threads. Its not a total guarantee of thread safety if for example you're placing objects in some kind of in-memory cache, then using them in other threads without detaching them first from their original session. You really need to figure out what the catalyst for the issue was - short of locating the actual cause of the bug, that would produce the most clues towards what it is. Or, this is a really harsh approach that I had to do once when a deeply nested call to a defunct Google service started crashing the site, I had to disable all pages on the site, then slowly turn one page on after the next to isolate which one was the cause of the issue. Clearly that isn't an option in lots of cases but it depends on if you can reproduce the issue locally, perhaps when load testing with Apache ab and such. If its something I had seen before that would help but I've never seen anyone hitting that assertion before. On May 20, 9:59 am, Michael Bayer mike...@zzzcomputing.com wrote: On May 20, 2011, at 9:45 AM, Adrian wrote: I have a Turbogears server that uses sqlalchemy to interface with a postgres database. Today, I noticed the server was down, so I tried restarting it. Now my turbogears log is full of errors like: AssertionError: A conflicting state is already present in the identity map for key (class 'dr8db.ModelClasses.FITSHeaderKeyword', (1045,)) and Exception KeyError: KeyError((class 'dr8db.ModelClasses.MaskbitsType', (61,)),) in bound method InstanceState._cleanup of sqlalchemy.orm.state.InstanceState object at 0x4445c90 ignored I tried googling this stuff, but found nothing... Basically it lets me start the paster (Turbogears) server, but after ~5-10 minutes the server dies and there are hundreds of these errors in the log -- help!! I need to get this server back up ASAP! That's an assertion that is generally unreachable from within the Session. The only ways I think you could get there would be via direct manipulation of session.identity_map, or if the Session is being shared among concurrent threads, which is not supported. The main thing you'd be looking for here is, at what point did this server begin to fail and what event precluded that happening ?Either a code update, or perhaps the app was never tested against its current load, are the two possibilities. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
[sqlalchemy] Re: Conflicting state in identity map?
Ok, I'll definitely do some quality debugging... Just to be clear -- I **don't** have to worry about closing my sessions in each controller? On May 20, 6:08 pm, Michael Bayer mike...@zzzcomputing.com wrote: On May 20, 2011, at 4:12 PM, Adrian wrote: So if it is the latter, that the Session is being shared amongst threads, what is the correct way to handle the sessions from within Turbogears? What I do now is create a scoped_session in the model, and import it into each controller. For a while, I made use of special functions _before() and _after() to create the session in _before the page loads, and close it in _after, but then I was getting DetatchedInstanceErrors when I tried to access columns from objects returned to the template. I'm not sure how familiar you are with Turbogears, so I apologize if this is too much of a TG question...but I asked this on their mailing list and their answer was that what I was doing was correct -- obviously it's not if I'm getting these AssertionErrors... Thanks for the quick reply, as usual! So scoped_session() will ensure that sessions are associated with threads. Its not a total guarantee of thread safety if for example you're placing objects in some kind of in-memory cache, then using them in other threads without detaching them first from their original session. You really need to figure out what the catalyst for the issue was - short of locating the actual cause of the bug, that would produce the most clues towards what it is. Or, this is a really harsh approach that I had to do once when a deeply nested call to a defunct Google service started crashing the site, I had to disable all pages on the site, then slowly turn one page on after the next to isolate which one was the cause of the issue. Clearly that isn't an option in lots of cases but it depends on if you can reproduce the issue locally, perhaps when load testing with Apache ab and such. If its something I had seen before that would help but I've never seen anyone hitting that assertion before. On May 20, 9:59 am, Michael Bayer mike...@zzzcomputing.com wrote: On May 20, 2011, at 9:45 AM, Adrian wrote: I have a Turbogears server that uses sqlalchemy to interface with a postgres database. Today, I noticed the server was down, so I tried restarting it. Now my turbogears log is full of errors like: AssertionError: A conflicting state is already present in the identity map for key (class 'dr8db.ModelClasses.FITSHeaderKeyword', (1045,)) and Exception KeyError: KeyError((class 'dr8db.ModelClasses.MaskbitsType', (61,)),) in bound method InstanceState._cleanup of sqlalchemy.orm.state.InstanceState object at 0x4445c90 ignored I tried googling this stuff, but found nothing... Basically it lets me start the paster (Turbogears) server, but after ~5-10 minutes the server dies and there are hundreds of these errors in the log -- help!! I need to get this server back up ASAP! That's an assertion that is generally unreachable from within the Session. The only ways I think you could get there would be via direct manipulation of session.identity_map, or if the Session is being shared among concurrent threads, which is not supported. The main thing you'd be looking for here is, at what point did this server begin to fail and what event precluded that happening ? Either a code update, or perhaps the app was never tested against its current load, are the two possibilities. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/sqlalchemy?hl=en. -- You received this message because you are subscribed to the Google Groups sqlalchemy group. To post to this group, send email to sqlalchemy@googlegroups.com. To unsubscribe from this group, send email to sqlalchemy+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/sqlalchemy?hl=en.
