Re: [google-appengine] Article about tuning Java apps for AppEngine
I know, and it makes sense, but what exactly is the limit? How long is a query allowed to run? I've been searching in the docs and billing settings but can't find it. Again, this is only because I want to have the correct numbers, I'll still advice to use alternative approaches. On Wed, Dec 15, 2010 at 6:13 AM, Robert Kluin robert.kl...@gmail.comwrote: Hi Paul, The overall execution time has been increased, but the limits on datastore queries have not. Use cursors and a loop. Robert On Tue, Dec 14, 2010 at 16:42, Paul Bakker paul.bakker...@gmail.com wrote: Hi Ikai, It's mostly CPU time that it's using now just generating the test data. The costs are lower than I expected though so no problem (and interesting to note in the article). I can't find any up to date information about the datastore timeout limits. In 1.4 the background task timeout is 10 minutes, but long before that I get a datastore timeout for a select all query with a dataset of a few million rows. I know you shouldn't normally do this, but it's just to test and get some numbers. Any idea about this? Paul On Mon, Dec 13, 2010 at 7:28 PM, Ikai Lan (Google) ikai.l+gro...@google.com ikai.l%2bgro...@google.com wrote: Hi Paul, You can enable billing for this. I doubt you'll end up paying a lot for it. Storage charges are based on snapshots and averaged for the month. Good luck! -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Dec 13, 2010 at 4:54 AM, Paul Bakker paul.bakker...@gmail.com wrote: Hi Ikai, Glad you like it. The article gets quite some hits so it seems there's a lot of interest in this topic. I got some questions relating large volumes of data. I'm working on another article that goes into the details of this. To get some real numbers I need to run some benchmarks with larger amounts of data (max 10 million rows), but I run into free quota limits. Is there any way this limit could be stretched for a few days so I can finish the article without having to pay a lot for it? I'm trying to advocate the use of GAE after all :-) My app id is gae-benchmarks. Great work on the latest releases (most specifically no more 1000 row limit in 1.3.1 and always on in 1.4). Takes away a lot of the reasons why not to use GAE! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com . To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Article about tuning Java apps for AppEngine
Hi Ian, It's mostly CPU time that it's using now just generating the test data. The costs are lower than I expected though so no problem (and interesting to note in the article). I can't find any up to date information about the datastore timeout limits. In 1.4 the background task timeout is 10 minutes, but long before that I get a datastore timeout for a select all query with a dataset of a few million rows. I know you shouldn't normally do this, but it's just to test and get some numbers. Any idea about this? Paul On Mon, Dec 13, 2010 at 7:28 PM, Ikai Lan (Google) ikai.l+gro...@google.comikai.l%2bgro...@google.com wrote: Hi Paul, You can enable billing for this. I doubt you'll end up paying a lot for it. Storage charges are based on snapshots and averaged for the month. Good luck! -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Dec 13, 2010 at 4:54 AM, Paul Bakker paul.bakker...@gmail.comwrote: Hi Ikai, Glad you like it. The article gets quite some hits so it seems there's a lot of interest in this topic. I got some questions relating large volumes of data. I'm working on another article that goes into the details of this. To get some real numbers I need to run some benchmarks with larger amounts of data (max 10 million rows), but I run into free quota limits. Is there any way this limit could be stretched for a few days so I can finish the article without having to pay a lot for it? I'm trying to advocate the use of GAE after all :-) My app id is gae-benchmarkshttps://appengine.google.com/dashboard?app_id=gae-benchmarks . Great work on the latest releases (most specifically no more 1000 row limit in 1.3.1 and always on in 1.4). Takes away a lot of the reasons why not to use GAE! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Article about tuning Java apps for AppEngine
Hi Ikai, It's mostly CPU time that it's using now just generating the test data. The costs are lower than I expected though so no problem (and interesting to note in the article). I can't find any up to date information about the datastore timeout limits. In 1.4 the background task timeout is 10 minutes, but long before that I get a datastore timeout for a select all query with a dataset of a few million rows. I know you shouldn't normally do this, but it's just to test and get some numbers. Any idea about this? Paul On Mon, Dec 13, 2010 at 7:28 PM, Ikai Lan (Google) ikai.l+gro...@google.comikai.l%2bgro...@google.com wrote: Hi Paul, You can enable billing for this. I doubt you'll end up paying a lot for it. Storage charges are based on snapshots and averaged for the month. Good luck! -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Dec 13, 2010 at 4:54 AM, Paul Bakker paul.bakker...@gmail.comwrote: Hi Ikai, Glad you like it. The article gets quite some hits so it seems there's a lot of interest in this topic. I got some questions relating large volumes of data. I'm working on another article that goes into the details of this. To get some real numbers I need to run some benchmarks with larger amounts of data (max 10 million rows), but I run into free quota limits. Is there any way this limit could be stretched for a few days so I can finish the article without having to pay a lot for it? I'm trying to advocate the use of GAE after all :-) My app id is gae-benchmarkshttps://appengine.google.com/dashboard?app_id=gae-benchmarks . Great work on the latest releases (most specifically no more 1000 row limit in 1.3.1 and always on in 1.4). Takes away a lot of the reasons why not to use GAE! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Article about tuning Java apps for AppEngine
Hi Paul, The overall execution time has been increased, but the limits on datastore queries have not. Use cursors and a loop. Robert On Tue, Dec 14, 2010 at 16:42, Paul Bakker paul.bakker...@gmail.com wrote: Hi Ikai, It's mostly CPU time that it's using now just generating the test data. The costs are lower than I expected though so no problem (and interesting to note in the article). I can't find any up to date information about the datastore timeout limits. In 1.4 the background task timeout is 10 minutes, but long before that I get a datastore timeout for a select all query with a dataset of a few million rows. I know you shouldn't normally do this, but it's just to test and get some numbers. Any idea about this? Paul On Mon, Dec 13, 2010 at 7:28 PM, Ikai Lan (Google) ikai.l+gro...@google.com wrote: Hi Paul, You can enable billing for this. I doubt you'll end up paying a lot for it. Storage charges are based on snapshots and averaged for the month. Good luck! -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Dec 13, 2010 at 4:54 AM, Paul Bakker paul.bakker...@gmail.com wrote: Hi Ikai, Glad you like it. The article gets quite some hits so it seems there's a lot of interest in this topic. I got some questions relating large volumes of data. I'm working on another article that goes into the details of this. To get some real numbers I need to run some benchmarks with larger amounts of data (max 10 million rows), but I run into free quota limits. Is there any way this limit could be stretched for a few days so I can finish the article without having to pay a lot for it? I'm trying to advocate the use of GAE after all :-) My app id is gae-benchmarks. Great work on the latest releases (most specifically no more 1000 row limit in 1.3.1 and always on in 1.4). Takes away a lot of the reasons why not to use GAE! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Article about tuning Java apps for AppEngine
Hi Ikai, Glad you like it. The article gets quite some hits so it seems there's a lot of interest in this topic. I got some questions relating large volumes of data. I'm working on another article that goes into the details of this. To get some real numbers I need to run some benchmarks with larger amounts of data (max 10 million rows), but I run into free quota limits. Is there any way this limit could be stretched for a few days so I can finish the article without having to pay a lot for it? I'm trying to advocate the use of GAE after all :-) My app id is gae-benchmarks. Great work on the latest releases (most specifically no more 1000 row limit in 1.3.1 and always on in 1.4). Takes away a lot of the reasons why not to use GAE! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Article about tuning Java apps for AppEngine
Hi Paul, You can enable billing for this. I doubt you'll end up paying a lot for it. Storage charges are based on snapshots and averaged for the month. Good luck! -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Dec 13, 2010 at 4:54 AM, Paul Bakker paul.bakker...@gmail.comwrote: Hi Ikai, Glad you like it. The article gets quite some hits so it seems there's a lot of interest in this topic. I got some questions relating large volumes of data. I'm working on another article that goes into the details of this. To get some real numbers I need to run some benchmarks with larger amounts of data (max 10 million rows), but I run into free quota limits. Is there any way this limit could be stretched for a few days so I can finish the article without having to pay a lot for it? I'm trying to advocate the use of GAE after all :-) My app id is gae-benchmarkshttps://appengine.google.com/dashboard?app_id=gae-benchmarks . Great work on the latest releases (most specifically no more 1000 row limit in 1.3.1 and always on in 1.4). Takes away a lot of the reasons why not to use GAE! -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.comgoogle-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Article about tuning Java apps for AppEngine
Since I've seen a lot of people with performance problems in their Java applications on GAE I decided to write an article with some advices on how to design applications that deal with the differences of the GAE architecture compared to a normal dedicated server environment. I hope it's useful and takes away some of the negative feedback. http://paulonjava.blogspot.com/2010/12/tuning-google-appengine.html -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Article about tuning Java apps for AppEngine
Hi Paul, Thanks for the post! Since the launch of Always On, though, the cold start issue impacts developers far less than before. Everyone we've spoken to will gladly enable billing for the feature if it frees them to use the framework of their choice. One suggestion: break the post into 2 portions, one about startup time, and one about general Memcache/Datastore strategies. This'll allow folks to bypass the tips for startup time if they are simply going for Always On and are already set on using a framework with a higher load time. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Mon, Dec 6, 2010 at 4:56 AM, Paul Bakker paul.bakker...@gmail.com wrote: Since I've seen a lot of people with performance problems in their Java applications on GAE I decided to write an article with some advices on how to design applications that deal with the differences of the GAE architecture compared to a normal dedicated server environment. I hope it's useful and takes away some of the negative feedback. http://paulonjava.blogspot.com/2010/12/tuning-google-appengine.html -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
