Hey guys,
I also seem to have found a couple more.
1. HeapOnWrite: Java heap space while waiting for outstanding writes in the
GCSOutputChannel. My GCSOutputChannel is wrapped by a print writer. Every
100 records (~25k size) I'm calling waitForOutstandingWrites and then
printWriter.flush.
This does not appear to actually be clearing the buffer on subsequent
calls. I also don't think I'm anywhere NEAR URLFetch's 32mb limit when this
happens.
2. TimeoutException: I thought to myself, "Hey, let's try waiting for write
every 10 records." This hit me with the IOException while writing almost
instantly.
The thing that kills me is that the errors aren't consistent operating over
the same data set. Anyone have any recommendations?
Appreciate you guys looking!
Thanks,
E
On Wednesday, March 12, 2014 7:14:54 AM UTC-4, Evan Ruff wrote:
>
> Hey Vinnie,
>
> The 202 is the "exit code" from my process. I've attached the three major
> exceptions that I've been getting.
>
> 1. Mystery202LogMessage - This one seems to happen ~usually on the first
> run after a new deploy, but can pop up at other times. The entire doPost
> method has been wrapped in a try/catch block, and nothing is ever caught
> related to this exception. I'm really unclear as to what's happening. I do
> know that it does make it pretty far into the process, as it updates some
> datastore entries.
>
> 2. HeapSpaceOnLog - This happens occasionally when writing out a static
> log message. It seems unlikely that my heap was
> THIIIIIIIIIIIIIIIIIIIIIS99.999999999% full that the log message would push
> it over the top, but hey who knows.
>
> 3. ApiDeadlineExceededException - I'm trying to get a cursor from the
> iterator. I have a 27 second window set on all queries. Once it hits that,
> it gets the cursor (this error), reloads the query and sets the iterator to
> the itr of the new query. This exception is caught and logged.
>
> Thanks for any insight! (Oh, I'm using the Java SDK)
>
> E
>
>
> On Wednesday, March 12, 2014 3:48:35 AM UTC-4, Vinny P wrote:
>>
>> When you say you're getting a 202 error, is that a HTTP status code or
>> the App Engine internal error code that you're receiving?
>>
>> Can you also post the stack trace and any exception message connected
>> with the ApiDeadlineException?
>>
>>
>> -----------------
>> -Vinny P
>> Technology & Media Advisor
>> Chicago, IL
>>
>> App Engine Code Samples: http://www.learntogoogleit.com
>>
>>
>>
>>
>> On Tue, Mar 11, 2014 at 9:14 PM, Evan Ruff <evan...@gmail.com> wrote:
>>
>>> I've been working on a Task that exports a bunch of datastore entries
>>> out to GCS and am getting some wacky behavior. Sometimes, the thing fails
>>> in <30 seconds with a 202 error in the log that I can't catch in the code.
>>> Sometimes, I get an ApiDeadlineException when trying to get the cursor from
>>> my iterator after 30 seconds. Other times, I'll get a Java heap space error
>>> after three or four cursor pulls. There is nothing in my logs that is out
>>> of the ordinary and I'm not keeping a reference to anything anywhere.
>>>
>>> What is causing this completely unpredictable behavior? Is something
>>> going on with Datastore?
>>>
>>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine.
For more options, visit https://groups.google.com/d/optout.
2014-03-12 07:09:28.362 /twitter_csv 500 75025ms 0kb AppEngine-Google;
(+http://code.google.com/appengine) module=default version=connect
0.1.0.2 - - [12/Mar/2014:04:09:28 -0700] "POST /twitter_csv HTTP/1.1" 500 0
"http://connect-dot-fivexfiveapp.appspot.com/twitter_csv"; "AppEngine-Google;
(+http://code.google.com/appengine)" "connect-dot-fivexfiveapp.appspot.com"
ms=75025 cpu_ms=52108 queue_name=bigquery-export task_name=6732810365160847048
exit_code=104 app_engine_release=1.9.0
instance=00c61b117c53e3ca57b71b73bbdfb6fc666a7e
I 2014-03-12 07:08:13.962
com.fivexfive.social.service.twitter.ExportTweetToCSVTask doPost: Starting
ExportTweetToCSVTask for 1394622000000
I 2014-03-12 07:08:14.123
com.fivexfive.social.service.twitter.ExportTweetToCSVTask
exportAndImportBigQuery: Creating Tweet Export File for Wed, Mar 12, 2014 @
11:00AM
I 2014-03-12 07:08:41.072
com.fivexfive.social.service.twitter.ExportTweetToCSVTask
exportAndImportBigQuery: Getting new iterator for query,
I 2014-03-12 07:09:07.269
com.fivexfive.social.service.twitter.ExportTweetToCSVTask
exportAndImportBigQuery: Getting new iterator for query,
W 2014-03-12 07:09:28.173
Error for /twitter_csv
java.lang.OutOfMemoryError: Java heap space
at
com.google.appengine.repackaged.com.google.protobuf.ByteString.copyFrom(ByteString.java:178)
at
com.google.appengine.repackaged.com.google.protobuf.ByteString.copyFrom(ByteString.java:190)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertToPb(URLFetchServiceImpl.java:160)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetchAsync(URLFetchServiceImpl.java:69)
at
com.google.appengine.tools.cloudstorage.oauth.AbstractOAuthURLFetchService.fetchAsync(AbstractOAuthURLFetchService.java:52)
at
com.google.appengine.tools.cloudstorage.oauth.OauthRawGcsService.putAsync(OauthRawGcsService.java:322)
at
com.google.appengine.tools.cloudstorage.oauth.OauthRawGcsService.continueObjectCreationAsync(OauthRawGcsService.java:224)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForOutstandingWrites(GcsOutputChannelImpl.java:296)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.exportAndImportBigQuery(ExportTweetToCSVTask.java:227)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.doPost(ExportTweetToCSVTask.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
C 2014-03-12 07:09:28.272
Uncaught exception from servlet
java.lang.OutOfMemoryError: Java heap space
at
com.google.appengine.repackaged.com.google.protobuf.ByteString.copyFrom(ByteString.java:178)
at
com.google.appengine.repackaged.com.google.protobuf.ByteString.copyFrom(ByteString.java:190)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertToPb(URLFetchServiceImpl.java:160)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.fetchAsync(URLFetchServiceImpl.java:69)
at
com.google.appengine.tools.cloudstorage.oauth.AbstractOAuthURLFetchService.fetchAsync(AbstractOAuthURLFetchService.java:52)
at
com.google.appengine.tools.cloudstorage.oauth.OauthRawGcsService.putAsync(OauthRawGcsService.java:322)
at
com.google.appengine.tools.cloudstorage.oauth.OauthRawGcsService.continueObjectCreationAsync(OauthRawGcsService.java:224)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForOutstandingWrites(GcsOutputChannelImpl.java:296)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.exportAndImportBigQuery(ExportTweetToCSVTask.java:227)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.doPost(ExportTweetToCSVTask.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
W 2014-03-12 07:09:28.283
A problem was encountered with the process that handled this request, causing
it to exit. This is likely to cause a new process to be used for the next
request to your application. If you see this message frequently, you may be
throwing exceptions during the initialization of your application. (Error code
104)2014-03-12 07:23:53.038 /twitter_csv active start=2014-03-12,04:22:50.415
AppEngine-Google; (+http://code.google.com/appengine) module=default
version=connect
0.1.0.2 - - [12/Mar/2014:04:23:53 -0700] "POST /twitter_csv HTTP/1.1" 0 0
"http://connect-dot-fivexfiveapp.appspot.com/twitter_csv"; "AppEngine-Google;
(+http://code.google.com/appengine)" "connect-dot-fivexfiveapp.appspot.com"
ms=62624 cpu_ms=0 queue_name=bigquery-export task_name=8814563130256392566
finished=0
I 2014-03-12 07:22:50.885
com.fivexfive.social.service.twitter.ExportTweetToCSVTask doPost: Starting
ExportTweetToCSVTask for 1394622900000
I 2014-03-12 07:22:53.460
com.fivexfive.social.service.twitter.ExportTweetToCSVTask
exportAndImportBigQuery: Creating Tweet Export File for Wed, Mar 12, 2014 @
11:15AM
I 2014-03-12 07:23:20.868
com.fivexfive.social.service.twitter.ExportTweetToCSVTask
exportAndImportBigQuery: Getting new iterator for query,
W 2014-03-12 07:23:43.571
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2 call:
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2@18791e5:
IOException writing block
java.net.SocketTimeoutException: Timeout while fetching URL:
https://storage.googleapis.com/raw_csv/export%2Ftweet%2Fperiod_1394622900000.csv?upload_id=AEnB2UpoVwRTU2YOTWJoHoC9moWJhsg_aawXRwY1vtyWmgM-ciK9HljQ23SKszmVNsgQlfUo-3w6WExVX8q7BCThb5SJNpd9WQ
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:143)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.access$100(URLFetchServiceImpl.java:24)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl$1.convertException(URLFetchServiceImpl.java:92)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:94)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:86)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2.call(GcsOutputChannelImpl.java:230)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2.call(GcsOutputChannelImpl.java:222)
at
com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:75)
at
com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:129)
at
com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:123)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForNextToken(GcsOutputChannelImpl.java:222)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForOutstandingWrites(GcsOutputChannelImpl.java:298)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.exportAndImportBigQuery(ExportTweetToCSVTask.java:224)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.doPost(ExportTweetToCSVTask.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at
com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:146)
at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:446)
at
com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
at
com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
at
com.google.tracing.CurrentContext.runInContext(CurrentContext.java:188)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
at
com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
at
com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
at java.lang.Thread.run(Thread.java:724)
W 2014-03-12 07:23:43.574
com.google.appengine.tools.cloudstorage.RetryHelper doRetry: RetryHelper(10.41
s, 1 attempts,
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2@18791e5):
Attempt 1 failed, sleeping for 54 ms
java.net.SocketTimeoutException: Timeout while fetching URL:
https://storage.googleapis.com/raw_csv/export%2Ftweet%2Fperiod_1394622900000.csv?upload_id=AEnB2UpoVwRTU2YOTWJoHoC9moWJhsg_aawXRwY1vtyWmgM-ciK9HljQ23SKszmVNsgQlfUo-3w6WExVX8q7BCThb5SJNpd9WQ
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:143)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.access$100(URLFetchServiceImpl.java:24)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl$1.convertException(URLFetchServiceImpl.java:92)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:94)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:86)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2.call(GcsOutputChannelImpl.java:230)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2.call(GcsOutputChannelImpl.java:222)
at
com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:75)
at
com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:129)
at
com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:123)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForNextToken(GcsOutputChannelImpl.java:222)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForOutstandingWrites(GcsOutputChannelImpl.java:298)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.exportAndImportBigQuery(ExportTweetToCSVTask.java:224)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.doPost(ExportTweetToCSVTask.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at
com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:146)
at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:446)
at
com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
at
com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
at
com.google.tracing.CurrentContext.runInContext(CurrentContext.java:188)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
at
com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
at
com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
at java.lang.Thread.run(Thread.java:724)
W 2014-03-12 07:23:53.033
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2 call:
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2@18791e5:
IOException writing block
java.net.SocketTimeoutException: Timeout while fetching URL:
https://storage.googleapis.com/raw_csv/export%2Ftweet%2Fperiod_1394622900000.csv?upload_id=AEnB2UpoVwRTU2YOTWJoHoC9moWJhsg_aawXRwY1vtyWmgM-ciK9HljQ23SKszmVNsgQlfUo-3w6WExVX8q7BCThb5SJNpd9WQ
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.convertApplicationException(URLFetchServiceImpl.java:143)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl.access$100(URLFetchServiceImpl.java:24)
at
com.google.appengine.api.urlfetch.URLFetchServiceImpl$1.convertException(URLFetchServiceImpl.java:92)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:94)
at
com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java:86)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2.call(GcsOutputChannelImpl.java:230)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl$2.call(GcsOutputChannelImpl.java:222)
at
com.google.appengine.tools.cloudstorage.RetryHelper.doRetry(RetryHelper.java:75)
at
com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:129)
at
com.google.appengine.tools.cloudstorage.RetryHelper.runWithRetries(RetryHelper.java:123)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForNextToken(GcsOutputChannelImpl.java:222)
at
com.google.appengine.tools.cloudstorage.GcsOutputChannelImpl.waitForOutstandingWrites(GcsOutputChannelImpl.java:298)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.exportAndImportBigQuery(ExportTweetToCSVTask.java:224)
at
com.fivexfive.social.service.twitter.ExportTweetToCSVTask.doPost(ExportTweetToCSVTask.java:151)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:637)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
at
com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:263)
at
com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:178)
at
com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
at
com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:62)
at
com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:118)
at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:113)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java:125)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.JdbcMySqlConnectionCleanupFilter.doFilter(JdbcMySqlConnectionCleanupFilter.java:60)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43)
at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157)
at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388)
at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
at
org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418)
at
com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:266)
at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
at org.mortbay.jetty.Server.handle(Server.java:326)
at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:923)
at
com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java:76)
at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
at
com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java:146)
at
com.google.apphosting.runtime.JavaRuntime$RequestRunnable.run(JavaRuntime.java:446)
at
com.google.tracing.TraceContext$TraceContextRunnable.runInContext(TraceContext.java:437)
at
com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java:444)
at
com.google.tracing.CurrentContext.runInContext(CurrentContext.java:188)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContextNoUnref(TraceContext.java:308)
at
com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedContext(TraceContext.java:300)
at
com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:441)
at
com.google.apphosting.runtime.ThreadGroupPool$PoolEntry.run(ThreadGroupPool.java:251)
at java.lang.Thread.run(Thread.java:724)
