Re: How to investigate "Backlog too deep"? (follow-up)
This mailing list is one of the reasons Heroku is so awesome. The fact that Heroku participates and listens to this list just makes the Heroku service that much better. Thank you to the Heroku team and this community. Happy Holidays! Carl On Tue, Dec 29, 2009 at 12:08 PM, Oren Teich wrote: > Hi guys, > yes, this is a (now) known issue - during the slug spin up process we have > found an edge case that isn't so edge, causing this error. We have > hot-patched the servers to fix the bug, and are working on implementing a > cleaned up code path. No apps should be experiencing this, and we hope to > have prevented it from happening on the future as well. > Sorry for the lack of updates - while the technical team has had full > coverage over the holidays and has been working on this, we haven't dont a > great job giving you guys updates. > Service stability hasn't been meeting our targets the past two months. > We've found a lot of areas that we've improved, and this looks to be one of > the last outstanding major fixes. I'm working on a summary of the work > we've done for you guys so you can know what the status is, and what we've > done to make sure we just get better in the future. > Oren > On Tue, Dec 29, 2009 at 8:57 AM, Trevor Turk wrote: >> >> On Dec 29, 9:48 am, Casper Fabricius >> wrote: >> > I am in the same boat as Tim. Despite very low traffic and two active >> > dynos, my site tends to go down with a permanent "Backlog too deep" and no >> > other resolution than rebooting the app. It is very frustrating to have the >> > app go down like this from time to other, with no logs and no way to debug >> > it. >> > >> > I have no long running processes in the dynos - everything like that >> > goes to Delayed Job, even sending out emails. >> >> I've noticed this problem as well with a few of my applications. The >> sites will go down and the only way to get them started again is to >> commit something meaningless and re-push the application. Honestly, I >> expected to hear something from Heroku about this by now. >> >> Is this a known issue? Is there anything we can do to provide help in >> solving the problem? Next time an application of mine fails with this >> message, I'll create a support ticket and try to help troubleshoot. >> >> Thanks, >> - Trevor >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "Heroku" group. >> To post to this group, send email to her...@googlegroups.com. >> To unsubscribe from this group, send email to >> heroku+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/heroku?hl=en. >> >> > > -- > > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to investigate "Backlog too deep"? (follow-up)
On Dec 29, 11:08 am, Oren Teich wrote: > Service stability hasn't been meeting our targets the past two months. > We've found a lot of areas that we've improved, and this looks to be one of > the last outstanding major fixes. I'm working on a summary of the work > we've done for you guys so you can know what the status is, and what we've > done to make sure we just get better in the future. Thanks very much for the update - this is great to hear. - Trevor -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to investigate "Backlog too deep"? (follow-up)
Hi guys, yes, this is a (now) known issue - during the slug spin up process we have found an edge case that isn't so edge, causing this error. We have hot-patched the servers to fix the bug, and are working on implementing a cleaned up code path. No apps should be experiencing this, and we hope to have prevented it from happening on the future as well. Sorry for the lack of updates - while the technical team has had full coverage over the holidays and has been working on this, we haven't dont a great job giving you guys updates. Service stability hasn't been meeting our targets the past two months. We've found a lot of areas that we've improved, and this looks to be one of the last outstanding major fixes. I'm working on a summary of the work we've done for you guys so you can know what the status is, and what we've done to make sure we just get better in the future. Oren On Tue, Dec 29, 2009 at 8:57 AM, Trevor Turk wrote: > On Dec 29, 9:48 am, Casper Fabricius > wrote: > > I am in the same boat as Tim. Despite very low traffic and two active > dynos, my site tends to go down with a permanent "Backlog too deep" and no > other resolution than rebooting the app. It is very frustrating to have the > app go down like this from time to other, with no logs and no way to debug > it. > > > > I have no long running processes in the dynos - everything like that goes > to Delayed Job, even sending out emails. > > I've noticed this problem as well with a few of my applications. The > sites will go down and the only way to get them started again is to > commit something meaningless and re-push the application. Honestly, I > expected to hear something from Heroku about this by now. > > Is this a known issue? Is there anything we can do to provide help in > solving the problem? Next time an application of mine fails with this > message, I'll create a support ticket and try to help troubleshoot. > > Thanks, > - Trevor > > -- > > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to investigate "Backlog too deep"? (follow-up)
To contribute to this discussion, I myself found my low-traffic app experiencing seemingly inexplicable "backlog too deep" errors late last week, but I didn't notice any accompanying error reports on status.heroku.com. A quick 'heroku restart' fixed it. Can an overly-large slug size contribute to this problem? - Matt Buck On Tue, Dec 29, 2009 at 9:48 AM, Casper Fabricius < casper.fabric...@gmail.com> wrote: > I am in the same boat as Tim. Despite very low traffic and two active > dynos, my site tends to go down with a permanent "Backlog too deep" and no > other resolution than rebooting the app. It is very frustrating to have the > app go down like this from time to other, with no logs and no way to debug > it. > > I have no long running processes in the dynos - everything like that goes > to Delayed Job, even sending out emails. > > Cheers, > Casper Fabricius > > On 27/12/2009, at 19.45, Tim wrote: > > > There was a thread a few months ago regarding "backlog too deep" that > > seems to be closed now: > > > > > http://groups.google.com/group/heroku/search?group=heroku&q=backlog+too+deep++&qt_g=Search+this+group > > > > I had the same issue this morning. My small Heroku-hosted app went > > down showing the "backlog too deep" error. However, checking Google > > Analytics and New Relic I can't find anything that would point to > > increased traffic (only ~25 visits/day) or a problem with the app > > (performance indicators all seem to be within the New Relic default > > margins). > > > > Is there anywhere else I should check? Rebooting the app brought the > > site back up, but I'd like to investigate and find out what caused > > this, if possible. > > > > Thanks, > > Tim > > > > -- > > > > You received this message because you are subscribed to the Google Groups > "Heroku" group. > > To post to this group, send email to her...@googlegroups.com. > > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com > . > > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > > > > > -- > > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com > . > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to investigate "Backlog too deep"? (follow-up)
On Dec 29, 9:48 am, Casper Fabricius wrote: > I am in the same boat as Tim. Despite very low traffic and two active dynos, > my site tends to go down with a permanent "Backlog too deep" and no other > resolution than rebooting the app. It is very frustrating to have the app go > down like this from time to other, with no logs and no way to debug it. > > I have no long running processes in the dynos - everything like that goes to > Delayed Job, even sending out emails. I've noticed this problem as well with a few of my applications. The sites will go down and the only way to get them started again is to commit something meaningless and re-push the application. Honestly, I expected to hear something from Heroku about this by now. Is this a known issue? Is there anything we can do to provide help in solving the problem? Next time an application of mine fails with this message, I'll create a support ticket and try to help troubleshoot. Thanks, - Trevor -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
Re: How to investigate "Backlog too deep"? (follow-up)
I am in the same boat as Tim. Despite very low traffic and two active dynos, my site tends to go down with a permanent "Backlog too deep" and no other resolution than rebooting the app. It is very frustrating to have the app go down like this from time to other, with no logs and no way to debug it. I have no long running processes in the dynos - everything like that goes to Delayed Job, even sending out emails. Cheers, Casper Fabricius On 27/12/2009, at 19.45, Tim wrote: > There was a thread a few months ago regarding "backlog too deep" that > seems to be closed now: > > http://groups.google.com/group/heroku/search?group=heroku&q=backlog+too+deep++&qt_g=Search+this+group > > I had the same issue this morning. My small Heroku-hosted app went > down showing the "backlog too deep" error. However, checking Google > Analytics and New Relic I can't find anything that would point to > increased traffic (only ~25 visits/day) or a problem with the app > (performance indicators all seem to be within the New Relic default > margins). > > Is there anywhere else I should check? Rebooting the app brought the > site back up, but I'd like to investigate and find out what caused > this, if possible. > > Thanks, > Tim > > -- > > You received this message because you are subscribed to the Google Groups > "Heroku" group. > To post to this group, send email to her...@googlegroups.com. > To unsubscribe from this group, send email to > heroku+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/heroku?hl=en. > > -- You received this message because you are subscribed to the Google Groups "Heroku" group. To post to this group, send email to her...@googlegroups.com. To unsubscribe from this group, send email to heroku+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/heroku?hl=en.
