[google-appengine] latency and pending_time in RequestLog
I posted this question to stackoverflow[1], but there seems to be more activity here, so I'm going to repost it. There are a few timing fields in the App Engine RequestLog[2] class. How are they defined? My guess is this: latency = end_time - start_time total request time = pending_time + latency In other words, latency, start_time, end_time all have to do with processing the request, while pending_time is the wait before processing starts. Is this correct? Also, the App Engine dashboard logs, there is a timing field. Is that latency or latency + pending_time? Thanks! --Alex [1] http://stackoverflow.com/questions/15047078/how-do-latency-and-pending-time-relate-in-app-engine-request-logs [2] https://developers.google.com/appengine/docs/python/logs/requestlogclass -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Authorization error
I got error as blew when I run application on appengine with oauth2.
Uncaught exception from servlet
com.google.api.client.auth.oauth2.TokenResponseException: 400
{
error : invalid_scope
}
at
com.google.api.client.auth.oauth2.TokenResponseException.from(TokenResponseException.java:103)
at
com.google.api.client.auth.oauth2.TokenRequest.executeUnparsed(TokenRequest.java:303)
at
com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeTokenRequest.execute(GoogleAuthorizationCodeTokenRequest.java:175)
at
com.google.api.client.googleapis.auth.oauth2.GoogleAuthorizationCodeTokenRequest.execute(GoogleAuthorizationCodeTokenRequest.java:78)
at
com.google.api.client.extensions.servlet.auth.oauth2.AbstractAuthorizationCodeCallbackServlet.doGet(AbstractAuthorizationCodeCallbackServlet.java:129)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:617)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:717)
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Error number of requests display for Top 25 visitor IP address in Console
I'm having the the same problem, Since 2-3 weeks my blacklist is full of 1000k entries, I don't see anything suspicious neither in the dashboard nor in the logs but Google Bot started reporting http error code 503 (Service unavailable) on my web pages I really don't know how to deal with this, is there any Google representative that could give us some suggestion about this? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Error: Server Error
Error: Server Error The server encountered an error and could not complete your request. If the problem persists, please reporthttp://code.google.com/appengine/community.htmlyour problem and mention this error message and the query that caused it. 这是什么问题？ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Blobstore filename created in MapReduce job too long to create BlobKey
I've also been having this problem.
Did you manage to find any workarounds?
On Saturday, March 2, 2013 4:25:17 AM UTC, Jamie Niemasik wrote:
I've been receiving intermittent errors from MapReduce jobs. I'm running
Python 2.7.
The specific error is BadValueError: name must be under 500 bytes which
is raised when calling datastore.Key.from_path() within
blobstore.get_blob_key(); the filename being provided is way too long to
make a key from.
This all occurs within the code in the mapreduce package… nothing in my
code seems to affect it.
Some of the filenames are 288 bytes long, while some are 992. The M/R spec
name and id in each case is nearly the same and is very short; I don't see
where this variance comes from.
The sequence of events is this:
mapreduce.output_writers.init_job() creates a reasonable, short filename
and passes it to files.blobstore.create()
create() calls files.file._create('blobstore', …, filename)
_create() sets up an rpc with that filename and calls _make_call('Create',
...)
And that call sometimes returns a filename that's 288 bytes, sometimes
992. I have no idea why or how to work around this — any help would be
appreciated.
Thanks,
Jamie
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Blobstore filename created in MapReduce job too long to create BlobKey
I've been having a similar problem. As far as I know, we didn't run into this problem until 1st March. Before then, I am fairly certain all filenames were 288 characters in length and since then have been 992 characters. A solution (at least in Java, presumably Python has something similar) is to use a google.appengine.api.datastore.Text object, which will work with the longer filenames. This isn't the ideal solution, at least for us because of backwards compatibility issues with our datastore filled with string values. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Nullpointerexception throws when inserting entity using Auto-generated Classendpoint insert method
Hi All,
I am using JPA for saving entity to datastore.
I am using final version of the GPE and using its auto-generated endpoint
class to store an entity to datastore.
Here is my code which throws exception.
fooEndpoint.insertFoo(foo); // throws nullpointer exception
java.lang.NullPointerException
at org.datanucleus.api.jpa.JPAEntityManager.find(JPAEntityManager.java:318)
at org.datanucleus.api.jpa.JPAEntityManager.find(JPAEntityManager.java:256)
at com.FooEndpoint.containsFoo(FooEndpoint.java:150)
at com.FooEndpoint.insertFoo(FooEndpoint.java:96)
My entity class is similar with the given example at this
source:
https://developers.google.com/appengine/docs/java/datastore/jpa/overview
Here is the my entity:
@Entity
public class Foo {
@Id
@GeneratedValue(strategy=GenerationType.IDENTITY)
private Key ID;
... // other fields and setters and getters
}
The exception says that ID is null. Okay, It is true. But, I expect the
ID value will assign by app-engine. On the otherside, When I trying to save
same object using entity manager persist method. It is working. Because, it
does not check contains or not.
Why does the auto-generated endpoint class insert method not work? Do I
need to initialize the key?
Thank for your time.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To unsubscribe from this group and stop receiving emails from it, send an email
to google-appengine+unsubscr...@googlegroups.com.
To post to this group, send email to google-appengine@googlegroups.com.
Visit this group at http://groups.google.com/group/google-appengine?hl=en.
For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Can anyone confirm or deny that round about 1st March, GoogleAppEngine trippled the length of Blobstore filenames from around 300 chars up to around 900 chars?
Can anyone confirm or deny that round about 1st March, GoogleAppEngine trippled the length of Blobstore filenames from around 300 chars up to around 900 chars? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Can anyone confirm or deny that round about 1st March, GoogleAppEngine trippled the length of Blobstore filenames from around 300 chars up to around 900 chars?
Can't confirm or deny but, I might be wrong, was there a 500 char StringProperty limit and could this cause any problems if the filename is stored in a StringProperty? On Tuesday, March 5, 2013 4:39:44 PM UTC+2, Andrew Bindon wrote: Can anyone confirm or deny that round about 1st March, GoogleAppEngine trippled the length of Blobstore filenames from around 300 chars up to around 900 chars? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Blobstore filename created in MapReduce job too long to create BlobKey
On Friday, March 1, 2013 11:25:17 PM UTC-5, Jamie Niemasik wrote: Some of the filenames are 288 bytes long, while some are 992. The M/R spec name and id in each case is nearly the same and is very short; I don't see where this variance comes from. Have you noticed if the long file names contain the word 'writable' at the beginning? If so, it might similar to an issue that I had (my issue was with google storage, not blobstore, but their APIs are similar): https://groups.google.com/forum/?fromgroups=#!topic/app-engine-pipeline-api/vfWakN0NKSw It seems that when a file is writable, its in a special state with a filename that is very long. When the MR job finishes, it finalizes the file and gets another filename that is shorter. My issue had to do with the MR job not finishing properly. Some of my code was throwing exceptions but it wasn't causing the job to finalize properly and therefore not getting the shorter filename. I would take a look at your logs to see if there are any errors. They may be causing the MR job to not finish properly and then return unfinalized filenames. --Alex -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Instances keep shutting down
Star this issue http://code.google.com/p/googleappengine/issues/detail?id=8931 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Blobstore filename created in MapReduce job too long to create BlobKey
Thanks Alex. Yes, they are writable; however, I can't find any errors in the logs, other than the name errors themselves (and those occur right after Final result for job … is 'success'). The mapreduce code gets through files.finalize(filename), but blows up in get_blob_key because the filename is no shorter than before. Ben, I wish I could do that, the mapreduce lib is creating a __BlobFileIndex__ datastore key using this filename as the id, so I don't know what sort of change I could make there. Unfortunately it's not something I'm storing as a property on my own model. Jamie On Tue, Mar 5, 2013 at 4:48 PM, Alex Burgel abur...@gmail.com wrote: On Friday, March 1, 2013 11:25:17 PM UTC-5, Jamie Niemasik wrote: Some of the filenames are 288 bytes long, while some are 992. The M/R spec name and id in each case is nearly the same and is very short; I don't see where this variance comes from. Have you noticed if the long file names contain the word 'writable' at the beginning? If so, it might similar to an issue that I had (my issue was with google storage, not blobstore, but their APIs are similar): https://groups.google.com/forum/?fromgroups=#!topic/app-engine-pipeline-api/vfWakN0NKSw It seems that when a file is writable, its in a special state with a filename that is very long. When the MR job finishes, it finalizes the file and gets another filename that is shorter. My issue had to do with the MR job not finishing properly. Some of my code was throwing exceptions but it wasn't causing the job to finalize properly and therefore not getting the shorter filename. I would take a look at your logs to see if there are any errors. They may be causing the MR job to not finish properly and then return unfinalized filenames. --Alex -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/I0pXHW1poWU/unsubscribe?hl=en . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Instances keep shutting down
And never, ever, feed your unicorns Starburst candy. David On Tuesday, March 5, 2013 4:12:08 PM UTC-7, vlad wrote: Star this issue http://code.google.com/p/googleappengine/issues/detail?id=8931 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Many cold start problems recently.
There is a resident instance obviously there. But the GAE scheduler often ignores it totally and always creates a new instance then the let the new instacne to handle the new request. What is the functionality of the resident instance? If the scheduler always creates a new instance, the why put a resident instance there? It really make the user experience very bad. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] our site went down frequently recently
our app id is: worldvision-tw we bumped into 10+ site-down recently. here is the stack trace: 1. / 500 63345ms 0kb Mozilla/5.0 (Macintosh; Intel Mac OS X 10_7_4) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22 2. E2013-03-06 10:19:32.744 javax.servlet.ServletContext log: unable to create shared application instance org.jruby.rack.RackInitializationException: load error: builtin/javasup 3. E2013-03-06 10:19:32.744 continued from previous message at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.se 4. E2013-03-06 10:19:32.746 javax.servlet.ServletContext log: Application Error org.jruby.rack.RackInitializationException: unable to create shared application instance at org.j 5. E2013-03-06 10:19:32.746 continued from previous message at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphostin 6. W2013-03-06 10:19:33.864 Error for / com.google.apphosting.runtime.HardDeadlineExceededError: This request (46c1f83c11fb2bd0) started at 2013/03/06 02:18:33.110 UTC and was st 7. W2013-03-06 10:19:33.864 continued from previous message at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.se 8. C2013-03-06 10:19:33.870 Uncaught exception from servlet com.google.apphosting.runtime.HardDeadlineExceededError: This request (46c1f83c11fb2bd0) started at 2013/03/06 02:18:3 9. I2013-03-06 10:19:33.926 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This requ 10. W2013-03-06 10:19:33.926 A problem was encountered with the process that handled this request, causing it to exit. This is likely to can Thanks, -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
