Re: help please: mongo/mgo panic

2014-07-30 Thread John Meinel
Could this be something where we are getting transactions faster than we can finalize them? John =:- On Wed, Jul 30, 2014 at 9:17 AM, Gustavo Niemeyer gust...@niemeyer.net wrote: We've got a database dump yesterday, which gives me something to investigate. I'll spend some time on this

Re: help please: mongo/mgo panic

2014-07-30 Thread Gustavo Niemeyer
Alright, the guess last night was correct, and the candidate fix as well. I've managed to reproduce the problem by stressing out the scenario described with 4 concurrent runners running the following two operations, meanwhile the chaos mechanism injects random slowdowns in various critical points:

help please: mongo/mgo panic

2014-07-29 Thread Menno Smits
All, Various people have been seeing the machine agents panic with the following message: panic: rescanned document misses transaction in queue The error message comes from mgo but the actual cause is unknown. There's plenty of detail in the comments for the LP bug that's tracking this. If

Re: help please: mongo/mgo panic

2014-07-29 Thread Gustavo Niemeyer
We've got a database dump yesterday, which gives me something to investigate. I'll spend some time on this tomorrow (today) and report back. On Wed, Jul 30, 2014 at 1:34 AM, Menno Smits menno.sm...@canonical.com wrote: All, Various people have been seeing the machine agents panic with the