[google-appengine] App is encountering errors after maintenance period ended
My app, eticketexpress, never recovered after the maintenance period ended. When I request any page from my site, I get Error: Server Error pages. Please fix this asap! Thank you, Brent Washburne -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: App is encountering errors after maintenance period ended
Whats the error details in the log ? On Friday, May 24, 2013 2:22:14 PM UTC+8, Iron Mountain Foundry wrote: My app, eticketexpress, never recovered after the maintenance period ended. When I request any page from my site, I get Error: Server Error pages. Please fix this asap! Thank you, Brent Washburne -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Bad news for GAE/Java from Google I/O
Cold startup issues shouldn't be an excuse for reinventing the wheel. Anyone knows if spring will ever be capable to do it's initialization stuff at compilation time? Something like android resources? At least the heavy mapping stuff?! thanks On Thu, May 23, 2013 at 4:33 PM, Nick naoku...@gmail.com wrote: Shameless plug: http://3wks.github.io/thundr/ thundr is our lighweight web-mvc - we built it specifically for use on appengine. - No classpath scanning - Direct control of configuration (in testable code) - DI - Basic interceptor pattern for controllers - Date binding of webrequests into controllers - Out of the box works with jsp, json, exception handling - Modular, so it can be easily extended (for example, we have modules for handlebars, google-prediction, bigquery, async http connectivity, google analytics, cloud storage, mailgun, webpurify and its super easy to publish your own) We're currently working on changes to make it more restful friendly - right now its perfect for web apps that mix in services, but could be a little stronger for pure restful apps. A new version will be coming out soon. Alas, it can't fix your cold start issues. View our sample app here: https://github.com/3wks/thundr-sample On Thursday, May 23, 2013 7:02:56 PM UTC+10, Rafael Sanches wrote: The same here, Coding java-web without spring-mvc just feels like December 1973. Anyone has an actual solution? Maybe a more lightweight solution that I don't know? thanks rafa On Wed, May 22, 2013 at 9:00 PM, Carl Schroeder schroede...@gmail.comwrote: The Java cold starts issue was severe enough that we rewrote our entire app in Go and Python (for the API bits that are not yet supported in Go). GAE/Java worked well when we started with it, then the instance cold start issue appeared and that was a deal breaker. Let me tell you how awesome it was to demo the application to VC's and have the REST requests take 20 seconds each because GAE thinks that cycling instances on and off is the right thing to do. It nearly ruined us. Now that we are on Go, REST calls to cold instances take 100ms to complete. I don't recommend that everyone port to Go. We had no choice. The GAE/Java dashboard is missing one check-box: Never send a request to an instance that has not returned from _ah/warmup. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@**googlegroups.com. To post to this group, send email to google-a...@googlegroups.**com. Visit this group at http://groups.google.com/** group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Accumulated lots of data - deleting is is too expensive
I agree, for some Kinds we accumulated and do no longer use, we just don't delete them as doing so would cost more than keeping them for more than a year (or sometimes several years for light entities with several indexed properties), that is close to the anticipated remaining lifetime of the app. Removing stuff has little added value so paying a big check for it is not appealing. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] ereporter in python 2.7, threadsafe?
Yes! Thanks so much! On May 23, 2013, at 6:22 PM, Bryce Cutt pandas...@gmail.com wrote: Does this work for you? handlers: - url: /_ereporter.* script: google.appengine.ext.ereporter.report_generator.application login: admin On Thursday, May 23, 2013 6:32:17 AM UTC-7, Joshua Smith wrote: Is it possible to use ereporter in a python 2.7, threadsafe application? I can't find any docs or SO answers on how to do it. The source code down in google_appengine/google/appengine/ext/ereporter/ appears to only have old wsgi hooks. Alternately, is there a replacement for ereporter to go with the latest GAE? -Joshua -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: App is encountering errors after maintenance period ended
Good news, it was fixed overnight. No reply from Google on the cause or the fix. Mostly it had errors like this: 1. I2013-05-23 22:36:05.033 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application. 2. W2013-05-23 22:36:05.033 A problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. (Error code 121) On Thu, May 23, 2013 at 11:30 PM, timh zutes...@gmail.com wrote: Whats the error details in the log ? On Friday, May 24, 2013 2:22:14 PM UTC+8, Iron Mountain Foundry wrote: My app, eticketexpress, never recovered after the maintenance period ended. When I request any page from my site, I get Error: Server Error pages. Please fix this asap! Thank you, Brent Washburne -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/2W_uoS40N2I/unsubscribe?hl=en . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Bad news for GAE/Java from Google I/O
Regarding Spring wiring itself ahead of time: https://code.google.com/p/reflections/ and especially: https://code.google.com/p/reflections/wiki/ReflectionsSpring Note that I have not tried this at all. Spring is still initializing itself at startup, but the scanning is done at compile time, as I understand it. On Fri, May 24, 2013 at 2:44 AM, Rafael mufumb...@gmail.com wrote: Cold startup issues shouldn't be an excuse for reinventing the wheel. Anyone knows if spring will ever be capable to do it's initialization stuff at compilation time? Something like android resources? At least the heavy mapping stuff?! thanks On Thu, May 23, 2013 at 4:33 PM, Nick naoku...@gmail.com wrote: Shameless plug: http://3wks.github.io/thundr/ thundr is our lighweight web-mvc - we built it specifically for use on appengine. - No classpath scanning - Direct control of configuration (in testable code) - DI - Basic interceptor pattern for controllers - Date binding of webrequests into controllers - Out of the box works with jsp, json, exception handling - Modular, so it can be easily extended (for example, we have modules for handlebars, google-prediction, bigquery, async http connectivity, google analytics, cloud storage, mailgun, webpurify and its super easy to publish your own) We're currently working on changes to make it more restful friendly - right now its perfect for web apps that mix in services, but could be a little stronger for pure restful apps. A new version will be coming out soon. Alas, it can't fix your cold start issues. View our sample app here: https://github.com/3wks/thundr-sample On Thursday, May 23, 2013 7:02:56 PM UTC+10, Rafael Sanches wrote: The same here, Coding java-web without spring-mvc just feels like December 1973. Anyone has an actual solution? Maybe a more lightweight solution that I don't know? thanks rafa On Wed, May 22, 2013 at 9:00 PM, Carl Schroeder schroede...@gmail.comwrote: The Java cold starts issue was severe enough that we rewrote our entire app in Go and Python (for the API bits that are not yet supported in Go). GAE/Java worked well when we started with it, then the instance cold start issue appeared and that was a deal breaker. Let me tell you how awesome it was to demo the application to VC's and have the REST requests take 20 seconds each because GAE thinks that cycling instances on and off is the right thing to do. It nearly ruined us. Now that we are on Go, REST calls to cold instances take 100ms to complete. I don't recommend that everyone port to Go. We had no choice. The GAE/Java dashboard is missing one check-box: Never send a request to an instance that has not returned from _ah/warmup. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@**googlegroups.com. To post to this group, send email to google-a...@googlegroups.**com. Visit this group at http://groups.google.com/** group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/Nz4Yt8V6PB0/unsubscribe?hl=en . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- Jon Sawyer j...@jsawyer.net -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] MapReduce Failures
I'm seeing shards abruptly fail in my MR jobs for no apparent reason and without retrying: task_name=appengine-mrshard-1581047187783C3601732-14-2-retry-0 app_engine_release=1.8.0 instance=00c61b117c53a40e120ac864168a3fe51c2ce Shard 1581047187783C3601732-14 failed permanently. Is there some adjustment I can make to my queue parameters to avoid or reduce these issues? Recently I had been having problems with MR jobs throwing UnknownErrors and ApplicationError followed by RetrySliceErrors, and setting the min_backoff_seconds to 1 seemed to help with reducing the retry errors. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Frustrated by logs
I'm having a devil of a time using the logs to debug my application. There's a basic pattern of access that I'm trying to troubleshoot. In order to do this, I would like to analyze correlations between the following properties of requests: - Request path - Request status - Instance ID used to serve the request If I could filter on the first two properties in the admin console, I would probably be able to diagnose the issue. But that never seems to work properly; the admin console seems to give me some random subset of the relevant logs - sometimes none, sometimes a few, but never all of them. So I tried downloading the logs using appcfg.sh, and then filtering the logs myself. That works - except that the amount of data per request that you get from downloading logs doesn't include the instance ID used to serve the request! I'm getting enormously frustrated by this. Am I missing something about the admin console, or is its search function really this broken? Is there some way to get more verbose logging from the logs-download feature of appcfg.sh? Any insights or help would be greatly appreciated. Thanks, - Kris -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Bad news for GAE/Java from Google I/O
I don't think so. You just put the X requests into the queues for the N instances already running. This gives you additional latency for the X requests, of course - it's not free - but if the latency added is less than the instance startup time, then it's a better solution than what we have. In my own case, the queue length for existing instances is never greater than 1 or 2, and the average latency per request is about 200ms, while instance startup is 10s. The math in my case is pretty simple: it's hardly ever really worth spinning up a 2nd instance, but GAE does it pretty frequently. Someone should craft a Google Code Jam problem based on this issue. - Kris On Thursday, May 23, 2013 11:55:11 AM UTC-7, Tom wrote: I'm not that experienced with GAE, but I'm wondering whether it really is that simple. If an instance of my app takes 10s to start up, and my app can, at peak, receive X requests in a 10s period, then wouldn't I require close to X idle instances running all the time to satisfy your criteria. And wouldn't that be prohibitively expensive for most apps? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Bad news for GAE/Java from Google I/O
Oh, also: Google uses GAE internally. I've always assumed that the primary motivation for establishing Go support came from inside Google. (Not necessarily a bad thing - Google is on the cutting edge of web tech in a lot of ways, so if Google internally sees value in it, there's probably a good reason.) - Kris On Thursday, May 23, 2013 11:55:11 AM UTC-7, Tom wrote: One final point - I'm baffled at Google's decision to support Go on GAE. When I looked at Go and wondered about design decisions that seemed odd, the answer I always got was that it was designed to reduce build times for enormous systems. How does that primary design criteria make it suitable for GAE? If they had instead added a runtime for something like LLVM then people like myself (and the vast number of other devs who know C/C++/OC) could take advantage of the near-instantaneous startup times that you are talking about, with our existing skills and libraries. Tom On Thursday, May 23, 2013 2:07:18 PM UTC-4, Ryan Chazen wrote: And it's possible to not use app engine at all, and then not have to worry about how fast your app starts. As a bonus, it's far cheaper as well! Possible is never a useful word to throw around here - it's possible to rewrite your java webapp in Go, it's possible to convince people they don't even need a webapp and just use pen and paper. The point is, GAE/J pointing user requests to servers that have not finished loading is broken, and Google needs to fix it if they want to compete with Heroku/Azure/etc who all seem to have this issue sorted out. This isn't some kind of technical impossibility here - every other hosting provider of this type manages to solve this issue. And GAE is the most expensive of the lot in terms of compute resources. GAE should be offering the best solution, not a jump through hoops and pray it works until the next GAE release where everything crawls at half speed for a day solution. On Wed, May 22, 2013 at 11:53 AM, Bediako George bediako...@lucidtechnics.com wrote: So our Airlift application framework starts up pretty much immediately on App Engine for Java. It runs on the Java runtime, but developers write request handlers in JavaScript via Rhino. Even though it is production ready (we do have several customers using it already) it is not ready for public consumption (we are looking to release it at the end the summer). I am not mentioning this as an alternative to what you are doing. Instead, I want to make the point that it is possible to create a framework in Java that plays well with App Engine. https://github.com/LucidTechnics/Airlift Bediako On Friday, May 17, 2013 2:20:53 AM UTC-4, Marcel Overdijk wrote: This is an interesting question indeed. I don't believe startup times for Java will become better on GAE; it's also very typical for in Java land that startup times take 30s for medium to large apps or depending on the frameworks chosen. This is no problem when you up front spin up the number of required instances like on a VPS or CloudFoundry PaaS. One of the great things about GAE is the programming API and the autoscaling part, but this autoscaling (and thus spinning up and killing instances) bites back unfortunately. Couldn't Google prohibit cold requests being served to users? In many cases it would better to have latency when a requested is routed to a warm instance. I think this would solve many problems. What do you think? To bad Google is not participating in these discussions. But with great technologies and frameworks being delivered by Google (like Angular JS) Google cannot be taken seriously when they say just go back 10 years and use servlets, static factories, hard coded configs etc. This is a real development nightmare. ^M On Friday, May 17, 2013 1:21:22 AM UTC+2, jeffrey_t_b wrote: Jeff, I believe that you had asked on this list, a while ago: In what circumstance is it _ever_ good for user requests to see cold starts? Did you ever get an answer to that? That is the part that puzzles me still. Is it just too hard? Maybe they don't have a _scalable_ algorithm for directing requests to already-existing instances? Anyway, with all of the focus on the Compute Engine side, I wonder if improvements to App Engine are going to deprioritized. On Wednesday, May 15, 2013 4:52:51 PM UTC-7, Jeff Schnitzer wrote: I attended the Autoscaling Java session at Google I/O. In summary, the advice is: * Don't use dependency injection. * Don't use AOP. * Hardcode configuration values as much as possible. In other words, go back to Java circa 2002. There was no discussion of changing routing so that user requests don't see cold starts. I asked about this in person - apparently they're still talking about it and nothing has been done about it. I am sad. Jeff -- You received this message because you are subscribed to
[google-appengine] Re: Accumulated lots of data - deleting is is too expensive
I have never let it happens on my apps, i just build a cron to daily clean upp old data. But once you haven't done this on past, it would be possible writing a cron job that erase just dome of data every day. Depending on your apps access, you could do this using only with the free quota. Is this an option? Goog Lucky, Renzo Nuccitelli On Thursday, May 23, 2013 8:56:43 PM UTC-3, John Wheeler wrote: Been having trouble posting. I posted this but it appears it was deleted - I might have inadvertently deleted it. I created an application two years ago and didn't know too much of what I was doing with indexes and such when I created it. For example, I indexed a string property that turned out in practice to be large, and while I thought it would be useful to have it indexed, it turned out it never really was. Fast forward 2 years and 18 million entities later, I've started experimenting with cleaning up some of this data, and cost feel high. For example, I deleted about 1/2 million entities and it cost me over $100. It was also extremely slow. I'd imagine this probably cost Google on the order of a few cents in actuality. The mismatch, mentally is that these entities, all 18M, take up near a 1/2 terabyte of storage, which itself costs less than $100 nowadays. It's hard for me to understand paying $3600 to delete them all, and even though its an expensive lesson I'll not repeat in the future (letting entities accumulate), I don't think it should be this expensive in general to delete data. I'm hoping one of the app engine product managers reads this and takes it into account that deleting unused big data is just too expensive on app engine. Instead, what you're forced to do, is migrate bits of your data you want to keep and shutdown the old app, which seems like a lot of overhead to impose on developers when deletes, and datastore writes in general, could probably be much cheaper. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] ereporter in python 2.7, threadsafe?
No problem. On May 24, 2013, at 6:12 AM, Joshua Smith joshuaesm...@charter.net wrote: Yes! Thanks so much! On May 23, 2013, at 6:22 PM, Bryce Cutt pandas...@gmail.com wrote: Does this work for you? handlers: - url: /_ereporter.* script: google.appengine.ext.ereporter.report_generator.application login: admin On Thursday, May 23, 2013 6:32:17 AM UTC-7, Joshua Smith wrote: Is it possible to use ereporter in a python 2.7, threadsafe application? I can't find any docs or SO answers on how to do it. The source code down in google_appengine/google/appengine/ext/ereporter/ appears to only have old wsgi hooks. Alternately, is there a replacement for ereporter to go with the latest GAE? -Joshua -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/0v9KqB5Q8sQ/unsubscribe?hl=en. To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Channel disconnects after message sent
I am having the same issue and been banging my head for hours! Channel disconnects right after connecting. All was working fine couple days ago, and that's on localhost using 1.8.0 Did you find what was causing this. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Accumulated lots of data - deleting is is too expensive
@Renzo - Deleting data as it comes is not a real solution as it costs you the same amount as deleting it all at once. Just spreading the financial pain in time. The point is deleting data gets very expensive and it is a hidden cost most developers fail to consider. Perhaps GAE should hold an amnesty day once a year when developers can whack their data for free? In terms of customer loyalty that action alone would be worth GAE more than 100s of free developer days which they do around the globe. On Friday, May 24, 2013 9:47:49 AM UTC-7, Renzo Nuccitelli wrote: I have never let it happens on my apps, i just build a cron to daily clean upp old data. But once you haven't done this on past, it would be possible writing a cron job that erase just dome of data every day. Depending on your apps access, you could do this using only with the free quota. Is this an option? Goog Lucky, Renzo Nuccitelli On Thursday, May 23, 2013 8:56:43 PM UTC-3, John Wheeler wrote: Been having trouble posting. I posted this but it appears it was deleted - I might have inadvertently deleted it. I created an application two years ago and didn't know too much of what I was doing with indexes and such when I created it. For example, I indexed a string property that turned out in practice to be large, and while I thought it would be useful to have it indexed, it turned out it never really was. Fast forward 2 years and 18 million entities later, I've started experimenting with cleaning up some of this data, and cost feel high. For example, I deleted about 1/2 million entities and it cost me over $100. It was also extremely slow. I'd imagine this probably cost Google on the order of a few cents in actuality. The mismatch, mentally is that these entities, all 18M, take up near a 1/2 terabyte of storage, which itself costs less than $100 nowadays. It's hard for me to understand paying $3600 to delete them all, and even though its an expensive lesson I'll not repeat in the future (letting entities accumulate), I don't think it should be this expensive in general to delete data. I'm hoping one of the app engine product managers reads this and takes it into account that deleting unused big data is just too expensive on app engine. Instead, what you're forced to do, is migrate bits of your data you want to keep and shutdown the old app, which seems like a lot of overhead to impose on developers when deletes, and datastore writes in general, could probably be much cheaper. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Frustrated by logs
Hi Kristopher, On Fri, May 24, 2013 at 8:51 AM, Kristopher Giesing kris.gies...@gmail.comwrote: I'm having a devil of a time using the logs to debug my application. There's a basic pattern of access that I'm trying to troubleshoot. In order to do this, I would like to analyze correlations between the following properties of requests: - Request path - Request status - Instance ID used to serve the request If I could filter on the first two properties in the admin console, I would probably be able to diagnose the issue. But that never seems to work properly; the admin console seems to give me some random subset of the relevant logs - sometimes none, sometimes a few, but never all of them. Yes, the process stops searching when it hits a certain amount of time limit. However, if you click 'Next Page' on the result, you can navigate through the result. I hope it works for you. So I tried downloading the logs using appcfg.sh, and then filtering the logs myself. That works - except that the amount of data per request that you get from downloading logs doesn't include the instance ID used to serve the request! I'm getting enormously frustrated by this. Am I missing something about the admin console, or is its search function really this broken? Is there some way to get more verbose logging from the logs-download feature of appcfg.sh? The instance id is available via 'INSTANCE_ID' environment variable. How about to write the instance_id to application logs, so that you can download them with --severity option? But yes, I understand you saying there should be a way to easily include the instance ids in the downloaded logs. Can you file a feature request for that? -- Takashi Any insights or help would be greatly appreciated. Thanks, - Kris -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- Takashi Matsuo | Developers Programs Engineer | tmat...@google.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Accumulated lots of data - deleting is is too expensive
@vlad - I was thinking the same exact thing about the amnesty days idea. Ha, it would be funny, but great if we had 'Free delete Fridays' or something like that :-) It would be best if deletes were extremely cheap - give developers the ability to do them at cost so they're not afraid to experiment on App Engine On Fri, May 24, 2013 at 10:55 AM, vlad vlad.troyan...@gmail.com wrote: @Renzo - Deleting data as it comes is not a real solution as it costs you the same amount as deleting it all at once. Just spreading the financial pain in time. The point is deleting data gets very expensive and it is a hidden cost most developers fail to consider. Perhaps GAE should hold an amnesty day once a year when developers can whack their data for free? In terms of customer loyalty that action alone would be worth GAE more than 100s of free developer days which they do around the globe. On Friday, May 24, 2013 9:47:49 AM UTC-7, Renzo Nuccitelli wrote: I have never let it happens on my apps, i just build a cron to daily clean upp old data. But once you haven't done this on past, it would be possible writing a cron job that erase just dome of data every day. Depending on your apps access, you could do this using only with the free quota. Is this an option? Goog Lucky, Renzo Nuccitelli On Thursday, May 23, 2013 8:56:43 PM UTC-3, John Wheeler wrote: Been having trouble posting. I posted this but it appears it was deleted - I might have inadvertently deleted it. I created an application two years ago and didn't know too much of what I was doing with indexes and such when I created it. For example, I indexed a string property that turned out in practice to be large, and while I thought it would be useful to have it indexed, it turned out it never really was. Fast forward 2 years and 18 million entities later, I've started experimenting with cleaning up some of this data, and cost feel high. For example, I deleted about 1/2 million entities and it cost me over $100. It was also extremely slow. I'd imagine this probably cost Google on the order of a few cents in actuality. The mismatch, mentally is that these entities, all 18M, take up near a 1/2 terabyte of storage, which itself costs less than $100 nowadays. It's hard for me to understand paying $3600 to delete them all, and even though its an expensive lesson I'll not repeat in the future (letting entities accumulate), I don't think it should be this expensive in general to delete data. I'm hoping one of the app engine product managers reads this and takes it into account that deleting unused big data is just too expensive on app engine. Instead, what you're forced to do, is migrate bits of your data you want to keep and shutdown the old app, which seems like a lot of overhead to impose on developers when deletes, and datastore writes in general, could probably be much cheaper. -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/STcgkmRyDQ4/unsubscribe?hl=en . To unsubscribe from this group and all its topics, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Frustrated by logs
Hello Kristopher, On Friday, May 24, 2013 10:51:23 AM UTC-5, Kristopher Giesing wrote: I'm getting enormously frustrated by this. Am I missing something about the admin console, or is its search function really this broken? Is there some way to get more verbose logging from the logs-download feature of appcfg.sh? Yup, you're correct. Searching logs can be a PITA. Personally, I use a combination of a homebuilt logging inspector (a backend using the Logging API to inspect error logs) and Google BigQuery to analyze my logs. Streak wrote up a blog entry about using Google BQ to inspect logging, you can read it here: http://blog.streak.com/2012/07/export-your-google-app-engine-logs-to.html . - -Vinny P Technology Media Advisor Chicago, IL My Go side project: http://invalidmail.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Enable suspended Google Apps user used for transactional email
Hi guys, I built a website that's been running in autopilot for months (billing enabled). Recently I realize that the Google Apps user used for transactional email (info@MYAPP) has been suspended for abuse (according to Google Admin Control Panel). I can't even login (This account has been disabled). According to http://support.google.com/a/bin/answer.py?hl=enanswer=33326, Administrators can contact Support for more information, but apparently both phone and email support are *not* available for Free Google Apps (I have this long before they close it). My website is a dating website. I never use info@MYAPP other than notification email for signups in LaunchRock (we've launched, register here), welcome email (please confirm your email) and new message email (somebody sent you a message). I have SPF and DKIM properly set up. I suspect the reason for suspension is because some of my users register with invalid email account, so those transactional emails bounced (I know from implementing Email Bounce Handler recently). What should I do? The website is still not making money (I don't put ads, and everything there is free), so I'm really hesitant to upgrade to Google Apps for Business just to raise support, esp. if it's not guaranteed to enable the user. I'm also not ready to use dedicated 3rd party email service, as I want to keep cost to minimum (not to mention MailChimp doesn't accept dating websites). Has anyone had this problem? How did you solve this? What's the next thing forward to avoid this again? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Accumulated lots of data - deleting is is too expensive
On Fri, May 24, 2013 at 7:33 PM, John Wheeler wrote: It would be best if deletes were extremely cheap - give developers the ability to do them at cost so they're not afraid to experiment on App Engine What makes you think they are not close to 'cost' already? You seem to be assuming deletes are absurdly 'marked up' - for what ever reason. Why would Google doe that? Deleting at 'scale' is not cheap. Your data is replicated around. All those copies have to be found and 'deleted'. The indexes are seperate and have to be deleted too. There may be many. In fact most of the time the data isn't actually deleted. Just Tombstoned. Marked as deleted, so the space is not actually reclaimed right away (to be sold again). Would be to much work to remove the 'holes' all the time. The space will probably be reclaimed eventually, when the tablets are compacted. But not right away In fact when a Application is deleted, wouldnt be surprised if Google don't jsut absorb the storage cost, and not actully bother deleting the data. Deletions will be relativly rare, and few will leave large amounts of data lying around. Will just be orphaned and ignored. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Problem with google app engine using class java.net
Hello Pablo, On Wednesday, May 22, 2013 8:48:03 PM UTC-5, Pablo Brunetti wrote: good night I have a problem to use the java.net class. I'm trying to use the POST method to send an argument and have an answer. It's happening the following error: Exception in thread main java.lang.RuntimeException: Failed: HTTP error code: 500 Could anyone help me? Your code is fine. The error is not coming from Java or the java.net class. The error is coming from the site you're trying to query. I took the URL and parameters from your source code and tried them myself. You can see the results here: http://www.hurl.it/hurls/61a66228eec8ee5cef53a0dfbad584217e90275c/102e05f1bd47fa7dec0e45936e20d97f446c6869 . In short: the site itself is down. It's giving a 500 error, which means that something is wrong with the hosting server. You'll have to contact the site owner and ask him to fix it, or query a different web service. - -Vinny P Technology Media Advisor Chicago, IL My Go side project: http://invalidmail.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Accumulated lots of data - deleting is is too expensive
While it might be true that data deletion is expensive. The reality is Google is losing customers over that! It is obvious that whoever started this thread is not going to fork over $3600 for the privilege. His only way out right now is to cancel his credit card and abandon the account. Since I doubt GAE will let him just stop billing on that app. This is a sad situation and a flaw in GAE's business model. On Friday, May 24, 2013 11:53:59 AM UTC-7, barryhunter wrote: On Fri, May 24, 2013 at 7:33 PM, John Wheeler wrote: It would be best if deletes were extremely cheap - give developers the ability to do them at cost so they're not afraid to experiment on App Engine What makes you think they are not close to 'cost' already? You seem to be assuming deletes are absurdly 'marked up' - for what ever reason. Why would Google doe that? Deleting at 'scale' is not cheap. Your data is replicated around. All those copies have to be found and 'deleted'. The indexes are seperate and have to be deleted too. There may be many. In fact most of the time the data isn't actually deleted. Just Tombstoned. Marked as deleted, so the space is not actually reclaimed right away (to be sold again). Would be to much work to remove the 'holes' all the time. The space will probably be reclaimed eventually, when the tablets are compacted. But not right away In fact when a Application is deleted, wouldnt be surprised if Google don't jsut absorb the storage cost, and not actully bother deleting the data. Deletions will be relativly rare, and few will leave large amounts of data lying around. Will just be orphaned and ignored. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Management of Id Allocation
Ok Thank you very much guys...all is clear. On Wednesday, 15 May 2013 15:43:34 UTC+2, Christophe Pruvost wrote: Hi all, I deploy my application then I use Id Allocation and get a range from Club(1) To Club(10) I redeployed my application then I use Id Allocation and get a range from Club(1001) To Club(1010) So my question : what is the expected behavior for id allocation ? please give some details Each time I redeploy the I increase the sequence by 1000 ? I want to flush the Club Data so I would like to get a new Sequence beginning from Club(1)...is it possible to do that ? What kind of action could I have on this sequence generator ...I do not find information on that ? Thank you for all Christophe. PS : My code : KeyRange keyRangeClub = datastore.allocateIds(Club, 10); -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Frustrated by logs
As Vinny said, certainly BigQuery is something you can consider. Here is another project for ingesting App Engine logs into BigQuery: https://code.google.com/p/log2bq -- Takashi On Fri, May 24, 2013 at 11:49 AM, Vinny P vinny...@gmail.com wrote: Hello Kristopher, On Friday, May 24, 2013 10:51:23 AM UTC-5, Kristopher Giesing wrote: I'm getting enormously frustrated by this. Am I missing something about the admin console, or is its search function really this broken? Is there some way to get more verbose logging from the logs-download feature of appcfg.sh? Yup, you're correct. Searching logs can be a PITA. Personally, I use a combination of a homebuilt logging inspector (a backend using the Logging API to inspect error logs) and Google BigQuery to analyze my logs. Streak wrote up a blog entry about using Google BQ to inspect logging, you can read it here: http://blog.streak.com/2012/07/export-your-google-app-engine-logs-to.html . - -Vinny P Technology Media Advisor Chicago, IL My Go side project: http://invalidmail.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- Takashi Matsuo | Developers Programs Engineer | tmat...@google.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Accumulated lots of data - deleting is is too expensive
I too suspect that deletion is a truly expensive operation and that is directly reflected in pricing. Or worse, that the tablets remain forever fragmented and the space is never actually reused (as previously suggested on this thread). I've often advocated for a way for me to mark an entity as for deletion and allow Google to come around in some kind of batch operation to clean it up. It would be ideal if it were immediately removed from indexes (i.e., from sight) and I would be willing to pay for it until the background cleanup comes around (e.g., maybe at least every X days) - as long as the wait+background-cleanup costs were some fraction of just outright deleting it. Even without the immediate index removal, we have lots of use cases where the data could actually remain indexed because our particular use case naturally avoids these orphaned rows (e.g., think of all the blog posts and comments and +1's for a deleted account). I'm sure this is pretty common. So to be able to mark all these entities as for deletion or reclaimable would let a background process clean them up for little or no cost (apart from datastore storage while holding them during the wait period). j On Friday, 24 May 2013 13:09:43 UTC-6, vlad wrote: While it might be true that data deletion is expensive. The reality is Google is losing customers over that! It is obvious that whoever started this thread is not going to fork over $3600 for the privilege. His only way out right now is to cancel his credit card and abandon the account. Since I doubt GAE will let him just stop billing on that app. This is a sad situation and a flaw in GAE's business model. On Friday, May 24, 2013 11:53:59 AM UTC-7, barryhunter wrote: On Fri, May 24, 2013 at 7:33 PM, John Wheeler wrote: It would be best if deletes were extremely cheap - give developers the ability to do them at cost so they're not afraid to experiment on App Engine What makes you think they are not close to 'cost' already? You seem to be assuming deletes are absurdly 'marked up' - for what ever reason. Why would Google doe that? Deleting at 'scale' is not cheap. Your data is replicated around. All those copies have to be found and 'deleted'. The indexes are seperate and have to be deleted too. There may be many. In fact most of the time the data isn't actually deleted. Just Tombstoned. Marked as deleted, so the space is not actually reclaimed right away (to be sold again). Would be to much work to remove the 'holes' all the time. The space will probably be reclaimed eventually, when the tablets are compacted. But not right away In fact when a Application is deleted, wouldnt be surprised if Google don't jsut absorb the storage cost, and not actully bother deleting the data. Deletions will be relativly rare, and few will leave large amounts of data lying around. Will just be orphaned and ignored. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Issue with GAE and Spring Security
Found the solution from Huseyin OZVEREN: Simply replace .html files to .jsp Refer to: http://www.javablog.fr/springgwtgae-spring-security-on-google-cloud-jetty-tomcat.html#comment-165 For full description of solution On Thursday, May 23, 2013 5:40:40 PM UTC-5, Joey Wilkinson wrote: I have been working on implementing Spring Security into my GWT web application, but having issues with basic Spring Security login redirects. The redirects to the login page stops working if I enable GAE in eclipse in my project settings). I get the same issue when setup a very basic spring security on top of the default GWT GAE sample project when you click new GWT project in eclipse: http auto-config=true intercept-url pattern=/** access=ROLE_USER / form-login/ /http Has anyone experienced this kind of issue before? Or try to replicate issue? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Enable suspended Google Apps user used for transactional email
It was probably an automated mistake. When this happened to me, I logged directly into the mail account of the affected user to automatically remove the suspension. http://mail.google.com/a/domain.com/ If this doesn't work, you can sign up for a free trial of Apps for Business to get support. You might have been suspended for quite some time already. App Engine only started alerting about this recently. https://code.google.com/p/googleappengine/issues/detail?id=6181 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Frustrated by logs
The instance is available today in the logs you can download. I recently updated logparser.py to not fail when it shows up in the logs. https://code.google.com/p/google-app-engine-samples/source/browse/trunk/logparser/logparser.py (Looks like I didn't update the docstring for this, but the fundamentals are correct.) On Friday, May 24, 2013 12:24:24 PM UTC-7, Takashi Matsuo (Google) wrote: As Vinny said, certainly BigQuery is something you can consider. Here is another project for ingesting App Engine logs into BigQuery: https://code.google.com/p/log2bq -- Takashi On Fri, May 24, 2013 at 11:49 AM, Vinny P vinn...@gmail.com javascript: wrote: Hello Kristopher, On Friday, May 24, 2013 10:51:23 AM UTC-5, Kristopher Giesing wrote: I'm getting enormously frustrated by this. Am I missing something about the admin console, or is its search function really this broken? Is there some way to get more verbose logging from the logs-download feature of appcfg.sh? Yup, you're correct. Searching logs can be a PITA. Personally, I use a combination of a homebuilt logging inspector (a backend using the Logging API to inspect error logs) and Google BigQuery to analyze my logs. Streak wrote up a blog entry about using Google BQ to inspect logging, you can read it here: http://blog.streak.com/2012/07/export-your-google-app-engine-logs-to.html . - -Vinny P Technology Media Advisor Chicago, IL My Go side project: http://invalidmail.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@googlegroups.com javascript:. To post to this group, send email to google-a...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/groups/opt_out. -- Takashi Matsuo | Developers Programs Engineer | tma...@google.comjavascript: -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: JSP compile error with 1.7.4 at deploy time
Are you using Windows, Mac OS X, or Linux? On Windows you can navigate to the Add/Remove Programs screen in the Control Panel. You'll see the JRE listed in the pane. Select it and hit the Remove/Uninstall button. After that, remember to install the latest JDK version. - -Vinny P Technology Media Advisor Chicago, IL My Go side project: http://invalidmail.com/ On Wednesday, May 22, 2013 7:29:50 AM UTC-5, Dinakara Sandeep wrote: How to uninstall jre On Thursday, December 13, 2012 4:58:06 AM UTC+5:30, Sekhar wrote: I just updated GAE to 1.7.4, and I'm getting the JDK missing error (see below) at deploy time. The JSPs compile fine in Eclipse and I can test OK on the dev server...the problem comes only at deploy time. Was working fine in 1.7.3. Anyone else? I reverted to 1.7.3 to get around this. gae java.lang.RuntimeException: Cannot get the System Java Compiler. Please use a JDK, not a JRE. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Frustrated by logs
Thanks Matthew, I confirmed that you can download logs with instance_id by running the Python appcfg.py with the '--include_all' option as follows: $ appcfg.py request_logs --oauth2 -A APP_ID --version VERSION --include_all LOG_FILENAME Unfortunately, it doesn't seem Java appcfg have that option yet though. On Fri, May 24, 2013 at 2:00 PM, Matthew Blain matthew.bla...@google.comwrote: The instance is available today in the logs you can download. I recently updated logparser.py to not fail when it shows up in the logs. https://code.google.com/p/google-app-engine-samples/source/browse/trunk/logparser/logparser.py (Looks like I didn't update the docstring for this, but the fundamentals are correct.) On Friday, May 24, 2013 12:24:24 PM UTC-7, Takashi Matsuo (Google) wrote: As Vinny said, certainly BigQuery is something you can consider. Here is another project for ingesting App Engine logs into BigQuery: https://code.google.com/p/**log2bq https://code.google.com/p/log2bq -- Takashi On Fri, May 24, 2013 at 11:49 AM, Vinny P vinn...@gmail.com wrote: Hello Kristopher, On Friday, May 24, 2013 10:51:23 AM UTC-5, Kristopher Giesing wrote: I'm getting enormously frustrated by this. Am I missing something about the admin console, or is its search function really this broken? Is there some way to get more verbose logging from the logs-download feature of appcfg.sh? Yup, you're correct. Searching logs can be a PITA. Personally, I use a combination of a homebuilt logging inspector (a backend using the Logging API to inspect error logs) and Google BigQuery to analyze my logs. Streak wrote up a blog entry about using Google BQ to inspect logging, you can read it here: http://blog.streak.com/** 2012/07/export-your-google-**app-engine-logs-to.htmlhttp://blog.streak.com/2012/07/export-your-google-app-engine-logs-to.html . - -Vinny P Technology Media Advisor Chicago, IL My Go side project: http://invalidmail.com/ -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengi...@**googlegroups.com. To post to this group, send email to google-a...@googlegroups.**com. Visit this group at http://groups.google.com/** group/google-appengine?hl=enhttp://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/**groups/opt_outhttps://groups.google.com/groups/opt_out . -- Takashi Matsuo | Developers Programs Engineer | tma...@google.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- Takashi Matsuo | Developers Programs Engineer | tmat...@google.com -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: App is encountering errors after maintenance period ended
Had the same issue. Seemed to subside after about an hour and then was performing very slowly for a while longer. Seems fine now. On Friday, May 24, 2013 7:12:57 AM UTC-7, Iron Mountain Foundry wrote: Good news, it was fixed overnight. No reply from Google on the cause or the fix. Mostly it had errors like this: 1. I2013-05-23 22:36:05.033 This request caused a new process to be started for your application, and thus caused your application code to be loaded for the first time. This request may thus take longer and use more CPU than a typical request for your application. 2. W2013-05-23 22:36:05.033 A problem was encountered with the process that handled this request, causing it to exit. This is likely to cause a new process to be used for the next request to your application. (Error code 121) On Thu, May 23, 2013 at 11:30 PM, timh zute...@gmail.com javascript:wrote: Whats the error details in the log ? On Friday, May 24, 2013 2:22:14 PM UTC+8, Iron Mountain Foundry wrote: My app, eticketexpress, never recovered after the maintenance period ended. When I request any page from my site, I get Error: Server Error pages. Please fix this asap! Thank you, Brent Washburne -- You received this message because you are subscribed to a topic in the Google Groups Google App Engine group. To unsubscribe from this topic, visit https://groups.google.com/d/topic/google-appengine/2W_uoS40N2I/unsubscribe?hl=en . To unsubscribe from this group and all its topics, send an email to google-appengi...@googlegroups.com javascript:. To post to this group, send email to google-a...@googlegroups.comjavascript: . Visit this group at http://groups.google.com/group/google-appengine?hl=en . For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Upcoming Change to Datastore Auto IDs
Hello App Engine folks, In the upcoming 1.8.1 release, the Datastore default auto ID policy in production will switch to scattered IDs to improve performance. This change will take effect for all versions of your app uploaded with the 1.8.1 SDK. You can preview the new behavior in the dev server, where scattered auto IDs are the default. These IDs are large, well-distributed integers, but are guaranteed to be small enough to be completely represented as 64-bit floats. If you still need legacy ids for some entities (e.g. because you want smaller numbers for user-facing ids), we recommend you use the allocateIds() API, which will continue to behave as before. You can also override the default auto id policy by setting the new auto_id_policy option in your app.yaml/appengine-web.xml to legacy, but please note that this option will be deprecated in a future release and will eventually be removed. If you have any questions don't hesitate to follow-up here or drop me a line directly. -- Chris Product Manager, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] Re: Upcoming Change to Datastore Auto IDs
In short: if any part of your applications use 32 bit integers (which is still the default for many environments) to store any IDs of entities, your application will break once this change is deployed. On Saturday, May 25, 2013 8:04:08 AM UTC+8, Chris Ramsdale wrote: Hello App Engine folks, In the upcoming 1.8.1 release, the Datastore default auto ID policy in production will switch to scattered IDs to improve performance. This change will take effect for all versions of your app uploaded with the 1.8.1 SDK. You can preview the new behavior in the dev server, where scattered auto IDs are the default. These IDs are large, well-distributed integers, but are guaranteed to be small enough to be completely represented as 64-bit floats. If you still need legacy ids for some entities (e.g. because you want smaller numbers for user-facing ids), we recommend you use the allocateIds() API, which will continue to behave as before. You can also override the default auto id policy by setting the new auto_id_policy option in your app.yaml/appengine-web.xml to legacy, but please note that this option will be deprecated in a future release and will eventually be removed. If you have any questions don't hesitate to follow-up here or drop me a line directly. -- Chris Product Manager, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
Re: [google-appengine] Re: Accumulated lots of data - deleting is is too expensive
The economist in me thinks that Google should just double the price of writes and make delete free. Jeff On Fri, May 24, 2013 at 12:57 PM, Jason Collins jason.a.coll...@gmail.comwrote: I too suspect that deletion is a truly expensive operation and that is directly reflected in pricing. Or worse, that the tablets remain forever fragmented and the space is never actually reused (as previously suggested on this thread). I've often advocated for a way for me to mark an entity as for deletion and allow Google to come around in some kind of batch operation to clean it up. It would be ideal if it were immediately removed from indexes (i.e., from sight) and I would be willing to pay for it until the background cleanup comes around (e.g., maybe at least every X days) - as long as the wait+background-cleanup costs were some fraction of just outright deleting it. Even without the immediate index removal, we have lots of use cases where the data could actually remain indexed because our particular use case naturally avoids these orphaned rows (e.g., think of all the blog posts and comments and +1's for a deleted account). I'm sure this is pretty common. So to be able to mark all these entities as for deletion or reclaimable would let a background process clean them up for little or no cost (apart from datastore storage while holding them during the wait period). j On Friday, 24 May 2013 13:09:43 UTC-6, vlad wrote: While it might be true that data deletion is expensive. The reality is Google is losing customers over that! It is obvious that whoever started this thread is not going to fork over $3600 for the privilege. His only way out right now is to cancel his credit card and abandon the account. Since I doubt GAE will let him just stop billing on that app. This is a sad situation and a flaw in GAE's business model. On Friday, May 24, 2013 11:53:59 AM UTC-7, barryhunter wrote: On Fri, May 24, 2013 at 7:33 PM, John Wheeler wrote: It would be best if deletes were extremely cheap - give developers the ability to do them at cost so they're not afraid to experiment on App Engine What makes you think they are not close to 'cost' already? You seem to be assuming deletes are absurdly 'marked up' - for what ever reason. Why would Google doe that? Deleting at 'scale' is not cheap. Your data is replicated around. All those copies have to be found and 'deleted'. The indexes are seperate and have to be deleted too. There may be many. In fact most of the time the data isn't actually deleted. Just Tombstoned. Marked as deleted, so the space is not actually reclaimed right away (to be sold again). Would be to much work to remove the 'holes' all the time. The space will probably be reclaimed eventually, when the tablets are compacted. But not right away In fact when a Application is deleted, wouldnt be surprised if Google don't jsut absorb the storage cost, and not actully bother deleting the data. Deletions will be relativly rare, and few will leave large amounts of data lying around. Will just be orphaned and ignored. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
[google-appengine] What am I missing about dev_appserver2 and push task queues?
Ever since the 1.7.6 dev_appserver (aka dev_appserver2) upgrade, I see multiprocessing for requests. That is, if I define a simple app like this: import logging import time from google.appengine.api import taskqueue from google.appengine.ext import webapp class MyTaskHandler(webapp.RequestHandler): def post(self): logging.info(howdy, this is a tasky!) time.sleep(5) class MyHandler(webapp.RequestHandler): def get(self): logging.info(howdy, this is test!) for i in range(20): taskqueue.add(url=/tasky) application = webapp.WSGIApplication([(/test, MyHandler), (/tasky, MyTaskHandler)]) and then run a zillion *curl http://localhost:8080/test* request simultaneously, I can see multiple instances created, they run in parallel, the new admin interfaces shows them to me, it's just like it runs in the sky. That's great! But in the code, above, I'm also creating 20 tasks that run on /tasky for each hit to /test. I can see those enqueued on the default queue. Because of my sleep, they take a few seconds to run. But they're getting run serially. So it takes ~100s to run those 20 tasks, rather than 5s. Even if I go into the admin interface and hit the Run Now button on each one, they still are pulled off the queue serially. One lucky instance handles all of the task queue request. That's not so great, because it's making local runs of our test suite take a lot longer than they do in the sky. *How do I tell dev_appserver to run those tasks in parallel?* If this isn't possible, can someone help me understand why? I can easily POST directly to /tasky, and get the effect I want I thought multiprocessing support was a design goal for dev_appserver2 - why isn't taskqueue doing this for me? Thanks! -ckhan -- You received this message because you are subscribed to the Google Groups Google App Engine group. To unsubscribe from this group and stop receiving emails from it, send an email to google-appengine+unsubscr...@googlegroups.com. To post to this group, send email to google-appengine@googlegroups.com. Visit this group at http://groups.google.com/group/google-appengine?hl=en. For more options, visit https://groups.google.com/groups/opt_out.
