Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
Why is this unique to instance hours? Surely all quota are subject to exhaustion in face of a traffic spike. Why you can set a daily budget. Although artifically capping instances would of course limit use of the other quota too :) On Sun, Sep 11, 2011 at 12:04 AM, Steve wrote: > I (and many others I think) have been frustrated with how instance hours > billing can explode in the face of traffic spikes. I've submitted Issue > 5858 to help us put a hard limit on that scaling out. Regardless, it is > quite clear that the only real hope is to implement multi-threaded request > handling. Java has that option and the new Pytohn 2.7 runtime is supposed > to bring that. Seeing how important concurrent handling is going to be to > keeping bills reasonable, it's a real shame that the Python's New-GIL > improved concurrency was rejected for Python 2.7 in part to keep encouraging > adoption of 3.X. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/qui7GhQvcCcJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
The GIL (old or new) is not a problem for concurrency unless your application spends large quantities of time CPU-bound. As soon as you make an I/O request, some other thread will run. The GIL doesn't hurt. Jeff On Sat, Sep 10, 2011 at 4:04 PM, Steve wrote: > I (and many others I think) have been frustrated with how instance hours > billing can explode in the face of traffic spikes. I've submitted Issue > 5858 to help us put a hard limit on that scaling out. Regardless, it is > quite clear that the only real hope is to implement multi-threaded request > handling. Java has that option and the new Pytohn 2.7 runtime is supposed > to bring that. Seeing how important concurrent handling is going to be to > keeping bills reasonable, it's a real shame that the Python's New-GIL > improved concurrency was rejected for Python 2.7 in part to keep encouraging > adoption of 3.X. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/qui7GhQvcCcJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
On Sunday, September 11, 2011 3:27:43 PM UTC-7, Jeff Schnitzer wrote: > > The GIL (old or new) is not a problem for concurrency unless your > application spends large quantities of time CPU-bound. As soon as you > make an I/O request, some other thread will run. The GIL doesn't > hurt. > > Jeff > On my GET requests, I/O accounts for roughly half of the time to process the request. When a user POSTs new data, my app does a fair amount of recalculations and I/O is only about 20% of the request processing time. So 50% of my GETs and 80% of my POSTs would benefit from the improved concurrency of the new GIL. Cheers, Steve -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/DHKRXZZF3v0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
On Sun, Sep 11, 2011 at 6:31 PM, Steve wrote: > > On my GET requests, I/O accounts for roughly half of the time to process the > request. When a user POSTs new data, my app does a fair amount of > recalculations and I/O is only about 20% of the request processing time. So > 50% of my GETs and 80% of my POSTs would benefit from the improved > concurrency of the new GIL. Ok, you have one of those odd cpu-bound apps. If multithreaded concurrency is hardcoded to a fixed number then yeah, the GIL hurts you. I don't know what the plans are for Python, but in Javaland, Google has said that the initial limit of 10 is temporary and concurrency will be determined by CPU usage. Assuming G follows through on scheduling by CPU usage (maybe they already have), then the GIL still doesn't matter. Each instance will take requests until it maxes out X amount of CPU, then you'll start a new instance. Basically, the GIL doesn't hurt anymore than single-threading hurts Node.js systems. As long as you don't block for I/O, you will serve to the best ability of CPU resources. The GIL is just like green threads back in the early days of the JVM. Actually, I would be surprised of green threads don't start making a comeback with modern multicore architectures. Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
The old 2.5/2.7 GIL hurts performance any time there is more than one thread that is doing python work. Maybe you'd like to read up on how the old GIL burns cpu time in excess signalling and failed lock acquisition: http://www.dabeaz.com/python/NewGIL.pdf -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/KKh6yIezaVoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
What also could reduce the "instance hours" billed is a slider that sets the instance idle time. The time an instance is kept alive idle is at the moment >10min. My loading request time is about 1.5 sec (webapp). It would be very helpful to have a slider in the "Application Settings" where I can adjust the idle time in increments of 5 or 10 seconds. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Instance Hours, Python Concurrency, and Regret
That's disappointing. We'd be better off without any kind of pre-emption, only switching on I/O. Jeff On Sun, Sep 11, 2011 at 9:44 PM, Steve wrote: > The old 2.5/2.7 GIL hurts performance any time there is more than one thread > that is doing python work. Maybe you'd like to read up on how the old GIL > burns cpu time in excess signalling and failed lock acquisition: > http://www.dabeaz.com/python/NewGIL.pdf > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To view this discussion on the web visit > https://groups.google.com/d/msg/google-appengine/-/KKh6yIezaVoJ. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
