I wonder what happened, subscribed to this topic to be updated, hope
someone explains what happened and additionally hope this never happens to
me (python) :)
On Wednesday, September 12, 2012 11:00:49 PM UTC+3, Jeff Schnitzer wrote:
>
> We are back up and running now after 2 hrs of downtime.
>
> To whoever fixed it: THANK YOU!!!
>
> To whoever broke it in the first place: SPANKINGS!!!
>
> Jeff
>
> On Wed, Sep 12, 2012 at 12:11 PM, Jeff Schnitzer
> <je...@infohazard.org<javascript:>>
> wrote:
> > More information:
> >
> > * The failure began at 10:54am (pacific).
> > * Same app on different appid has the same problem.
> >
> > (as listed in the stacktraces, the appid is voost0)
> >
> > Jeff
> >
> > On Wed, Sep 12, 2012 at 11:53 AM, Jeff Schnitzer
> > <je...@infohazard.org<javascript:>>
> wrote:
> >> HEEEEEEEEEEEEELP!
> >>
> >> We have tried everything at this point. Shut down instances, tried to
> >> deploy a new version, even tried old versions. We've reported a
> >> production issue. Something is broken inside of GAE. The Guice error
> >> must be a symptom; the smoking gun seems to be:
> >>
> >> java.lang.ClassNotFoundException: java.io.FileOutputStream
> >>
> >> We've been down for 30 mins and getting complaints from our clients.
> >> This looks really, really bad. It's my nightmare scenario - an outage
> >> in GAE that is small enough not to raise major alarm bells, but
> >> nevertheless cripples my business. It is not the first time this has
> >> happened. It is shaking my faith in GAE.
> >>
> >> Jeff
> >>
> >> On Wed, Sep 12, 2012 at 11:11 AM, Jeff Schnitzer
> >> <je...@infohazard.org<javascript:>>
> wrote:
> >>> Our app has been running fine on the same version, no code deploys
> >>> (our versions are timestamped so this is 100% certain), for two days.
> >>> All of a sudden (10 mins ago) our app stopped running. Every attempt
> >>> to run a request produces this cryptic message:
> >>>
> >>> --------
> >>>
> >>> 2012-09-12 10:56:21.791
> >>> com.google.inject.servlet.GuiceFilter setPipeline: Multiple Servlet
> >>> injectors detected. This is a warning indicating that you have more
> >>> than one GuiceFilter running in your web application. If this is
> >>> deliberate, you may safely ignore this message. If this is NOT
> >>> deliberate however, your application may not work as expected.
> >>> D 2012-09-12 10:56:21.792
> >>> st.voo.tick.util.cambridge.CambridgeSetup <init>: Establishing
> >>> cambridge view resolver
> >>> I 2012-09-12 10:56:21.792
> >>> st.voo.tick.GuiceConfig contextInitialized: Guice initialization took
> 514 millis
> >>> W 2012-09-12 10:56:21.888
> >>> Failed startup of context
> >>>
> com.google.apphosting.utils.jetty.RuntimeAppEngineWebAppContext@1479784{/,/base/data/home/apps/s~voost0/2012-09-10-1715.361669184733923098}
>
>
> >>> java.lang.RuntimeException: java.lang.RuntimeException: Unable to
> >>> instantiate MessageBodyReader
> >>> at
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:35)
>
>
> >>> at
> org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:211)
>
> >>> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:67)
>
>
> >>> at
> org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39)
>
>
> >>> at
> st.voo.tick.util.GuiceResteasyFilterDispatcher.init(GuiceResteasyFilterDispatcher.java:48)
>
>
> >>> at
> com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114)
> >>> at
> com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98)
>
>
> >>> at
> com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172)
> >>> at
> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
> >>> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >>> at
> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662)
>
> >>> at
> org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
> >>> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
>
> >>> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
> >>> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
> >>> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >>> at
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:219)
>
>
> >>> at
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194)
>
>
> >>> at
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
>
>
> >>> at
> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
>
>
> >>> at
> com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
> >>> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
>
>
> >>> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
>
>
> >>> at
> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
>
>
> >>> at java.lang.Thread.run(Thread.java:679)
> >>> Caused by: java.lang.RuntimeException: Unable to instantiate
> MessageBodyReader
> >>> at
> org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:761)
>
>
> >>> at
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.registerProviders(RegisterBuiltin.java:70)
>
>
> >>> at
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:31)
>
>
> >>> ... 27 more
> >>> Caused by: java.lang.SecurityException: Unable to get members for
> >>> class org.jboss.resteasy.plugins.providers.DataSourceProvider
> >>> at
> com.google.appengine.runtime.Request.process-b6ca2b194d66ed23(Request.java)
> >>> at java.lang.Class.getConstructors(Class.java:291)
> >>> at
> org.jboss.resteasy.util.PickConstructor.pickSingletonConstructor(PickConstructor.java:27)
>
>
> >>> at
> org.jboss.resteasy.spi.ResteasyProviderFactory.getProviderInstance(ResteasyProviderFactory.java:1032)
>
>
> >>> at
> org.jboss.resteasy.spi.ResteasyProviderFactory.addMessageBodyReader(ResteasyProviderFactory.java:478)
>
>
> >>> at
> org.jboss.resteasy.spi.ResteasyProviderFactory.registerProvider(ResteasyProviderFactory.java:757)
>
>
> >>> at
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.registerProviders(RegisterBuiltin.java:70)
>
>
> >>> at
> org.jboss.resteasy.plugins.providers.RegisterBuiltin.register(RegisterBuiltin.java:31)
>
>
> >>> at
> org.jboss.resteasy.spi.ResteasyDeployment.start(ResteasyDeployment.java:211)
>
> >>> at
> org.jboss.resteasy.plugins.server.servlet.ServletContainerDispatcher.init(ServletContainerDispatcher.java:67)
>
>
> >>> at
> org.jboss.resteasy.plugins.server.servlet.FilterDispatcher.init(FilterDispatcher.java:39)
>
>
> >>> at
> st.voo.tick.util.GuiceResteasyFilterDispatcher.init(GuiceResteasyFilterDispatcher.java:48)
>
>
> >>> at
> com.google.inject.servlet.FilterDefinition.init(FilterDefinition.java:114)
> >>> at
> com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:98)
>
>
> >>> at
> com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:172)
> >>> at
> org.mortbay.jetty.servlet.FilterHolder.doStart(FilterHolder.java:97)
> >>> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >>> at
> org.mortbay.jetty.servlet.ServletHandler.initialize(ServletHandler.java:662)
>
> >>> at
> org.mortbay.jetty.servlet.Context.startContext(Context.java:140)
> >>> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1250)
>
> >>> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:517)
> >>> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:467)
> >>> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
>
>
> >>> at
> com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
> >>> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
>
>
> >>> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
>
>
> >>> ... 1 more
> >>> Caused by: java.lang.reflect.InvocationTargetException
> >>> at sun.reflect.GeneratedMethodAccessor11.invoke(Unknown
> Source)
> >>> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>
>
> >>> at java.lang.reflect.Method.invoke(Method.java:616)
> >>> ... 29 more
> >>> Caused by: java.lang.NoClassDefFoundError: java/io/FileOutputStream
> >>> at java.lang.Class.getDeclaredConstructors0(Native Method)
> >>> at
> java.lang.Class.privateGetDeclaredConstructors(Class.java:2406)
> >>> ... 29 more
> >>> Caused by: java.lang.ClassNotFoundException: java.io.FileOutputStream
> >>> ... 29 more
> >>> C 2012-09-12 10:56:21.889
> >>> Uncaught exception from servlet
> >>> javax.servlet.UnavailableException: Initialization failed.
> >>> at
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.createHandler(AppVersionHandlerMap.java:228)
>
>
> >>> at
> com.google.apphosting.runtime.jetty.AppVersionHandlerMap.getHandler(AppVersionHandlerMap.java:194)
>
>
> >>> at
> com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:134)
>
>
> >>> at
> com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:447)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:452)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:459)
>
>
> >>> at
> com.google.tracing.TraceContext.runInContext(TraceContext.java:701)
> >>> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:336)
>
>
> >>> at
> com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:328)
>
>
> >>> at
> com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:456)
>
>
> >>> at
> com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
>
>
> >>> at java.lang.Thread.run(Thread.java:679)
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/EJUrxhiFMp4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
