Re: RE: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
Thanks for the tips, Brandon! A quick question: those "scratchpad" datastores are python dicts, right? We need to be aware that the dicts are not coordinated across instances so they may have stale data (which might be ok depending on the app). Just want to make sure I understand your suggestion. Thanks, Dennis -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
On Saturday, May 14, 2011 8:54:54 PM UTC+1, Robert Kluin wrote: > > If the data doesn't get updated, using the blobstore might be a good > idea. As I recall, you can't update a blob. So if the data changes > it may not be the best idea. Of course, it really all depends on the > app / usage. Ah - never really looked at it that much yet, maybe I'll save the blobstore for when I go back to pure functional code and immutable data structures again :) > I store serialized dicts / lists in text / blob properties quite > often. It works well for me. > Yep, that might be they way to go - I never need to search by the internal fields, and whereas for SQL the idea of partial updates works well, in the datastore you always pull back the whole object and re-save the whole object even if it's just a single field/property being updated. Feels somewhat unclean not to follow the usual discipline of at least partially abstracting the data model at the back end, but I only use the datastore as a blind repository anyway. Cheers -- T -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
If the data doesn't get updated, using the blobstore might be a good idea. As I recall, you can't update a blob. So if the data changes it may not be the best idea. Of course, it really all depends on the app / usage. I store serialized dicts / lists in text / blob properties quite often. It works well for me. Robert On Sat, May 14, 2011 at 04:41, Tim wrote: > I suspected that might be the case, but if I have a few minutes to spare > sometime I might try it out. > My startup and servicing costs is pretty minimal - I've got a > one-page-webapp and most of my calls are just AJAX calls to load and update > data, so it's typically just object to JSON and back again. So given the way > the API calls are counted and how my traffic works, I might look at getting > rid of using the datastore for fields and just store what was a collection > of items as a single larger JSON text blob (and look at moving to the > blobstore), and tune the caching in the client in terms of writing back > changes. > I suppose a dynamic API for the scheduler might make it a bit tricky for GAE > to plan (at the small scale) how to schedule work on machines, but if it was > more like a declaration of hints at startup about how long I'd prefer to > hang around when idle before being killed etc - even a declaration in > app.yaml - then I think it might go some way to allaying some of the > concerns being voiced. > -- > T > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
I don't know what I want instances to do.. I am finishing up a complete re-write of my code so that if I have to move to Java or GO that my code is perfect in Python and I can pay for a port, not a re-write. On the App I'm baking the code to make sure I didn't blow anything up on, I'm now cruising at 16 instances and using .025 CPU seconds a second. So why I need 16 instances if I'm only doing 1/40th of a CPU worth of work is beyond me. That'd imply I've got 640x as much horse power as I need. Also there are a lot of things you can do much better than the example code would imply. I took request to first bit from 330ms down to 120ms. And I think I can get that to 70ms. I could probably be more aggressive if I didn't have error handling. Who needs that I never make errors. I tried several things with regards to how .py was loaded and the CGIhandler, I can't measure any difference. If anyone cares zipping data makes it faster out of datastore, a tie out of memcache, big (400k) wins big if the compression is good, and you lose on anything under 20k. Assumes 25% average compression. Considered zip based on size. (may do that) Anything worth writing to MemCache is probably worth writing to a "scratchpad" datastore that you access by keyname. The writes are cheap compared to a cache miss on memcache, or you wouldn't have memcached it would you? YMMV From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Tim Sent: Saturday, May 14, 2011 1:42 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing I suspected that might be the case, but if I have a few minutes to spare sometime I might try it out. My startup and servicing costs is pretty minimal - I've got a one-page-webapp and most of my calls are just AJAX calls to load and update data, so it's typically just object to JSON and back again. So given the way the API calls are counted and how my traffic works, I might look at getting rid of using the datastore for fields and just store what was a collection of items as a single larger JSON text blob (and look at moving to the blobstore), and tune the caching in the client in terms of writing back changes. I suppose a dynamic API for the scheduler might make it a bit tricky for GAE to plan (at the small scale) how to schedule work on machines, but if it was more like a declaration of hints at startup about how long I'd prefer to hang around when idle before being killed etc - even a declaration in app.yaml - then I think it might go some way to allaying some of the concerns being voiced. -- T -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
I suspected that might be the case, but if I have a few minutes to spare sometime I might try it out. My startup and servicing costs is pretty minimal - I've got a one-page-webapp and most of my calls are just AJAX calls to load and update data, so it's typically just object to JSON and back again. So given the way the API calls are counted and how my traffic works, I might look at getting rid of using the datastore for fields and just store what was a collection of items as a single larger JSON text blob (and look at moving to the blobstore), and tune the caching in the client in terms of writing back changes. I suppose a dynamic API for the scheduler might make it a bit tricky for GAE to plan (at the small scale) how to schedule work on machines, but if it was more like a declaration of hints at startup about how long I'd prefer to hang around when idle before being killed etc - even a declaration in app.yaml - then I think it might go some way to allaying some of the concerns being voiced. -- T -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
Hey Tim,
That will (probably) just stop your module being cached, making it
respond more slowly to requests, increasing the number of instances
needed to service concurrent requests. I think you're still better
off to try to make instance startup and request servicing as efficient
as possible.
An API to request instance death is a potentially interesting idea though.
Robert
On Fri, May 13, 2011 at 09:28, Tim wrote:
>
> Currently all my python scripts use the script-or-module mechanism as
> recommended by the docs
>
> application = webapp.WSGIApplication([('/somepage',
> SomeHandler), ('/anotherpage', AnotherHandler)], debug=True)
> # GAE will look for a main() with no args and, if found, caches this script
> # so we supply one to make the requests more efficient
> def main():
> run_wsgi_app(application)
> # How a file can be both a script and an imprtable module...
> if __name__ == "__main__":
> main()
>
> I'm wondering if I rename that function to not be "main", hence making the
> page just a script and no longer an importable module, will that effectively
> kill the instance after the script runs (my load time is minimal - most of
> my pages simply do datatore-query -> JSON or the reverse) and so minimise my
> instance hours ??
> Would it be possible to add something to the GAE API's so that an instance
> can suggest how long it would like to hang around for, or an ability to
> explicitly request to kill itself ?
> --
> T
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
I'm happy to hear explanations. I didn't change the code I just did an
update withouth changing version.
-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of nickmilon
Sent: Friday, May 13, 2011 2:58 PM
To: Google App Engine
Subject: [google-appengine] Re: 1.5 improvements Make me less scared of
Pricing
@Brandon Wirtz
All this can't be explained by you upgrading to 1.5 as @Stephen writes.
@Vinuth Madinur +1 for all your points especially minimum granularity.
;-)
Nick Milon
On May 13, 4:28 pm, Tim wrote:
> Currently all my python scripts use the script-or-module mechanism as
> recommended by the docs
>
> application = webapp.WSGIApplication([('/somepage',
> SomeHandler), ('/anotherpage', AnotherHandler)], debug=True)
>
> # GAE will look for a main() with no args and, if found, caches this
> script # so we supply one to make the requests more efficient def
> main():
> run_wsgi_app(application)
>
> # How a file can be both a script and an imprtable module...
> if __name__ == "__main__":
> main()
>
> I'm wondering if I rename that function to not be "main", hence making
> the page just a script and no longer an importable module, will that
> effectively kill the instance after the script runs (my load time is
> minimal - most of my pages simply do datatore-query -> JSON or the
> reverse) and so minimise my instance hours ??
>
> Would it be possible to add something to the GAE API's so that an
> instance can suggest how long it would like to hang around for, or an
> ability to explicitly request to kill itself ?
>
> --
> T
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.5 improvements Make me less scared of Pricing
@Brandon Wirtz
All this can't be explained by you upgrading to 1.5 as @Stephen
writes.
@Vinuth Madinur +1 for all your points especially minimum
granularity.
;-)
Nick Milon
On May 13, 4:28 pm, Tim wrote:
> Currently all my python scripts use the script-or-module mechanism as
> recommended by the docs
>
> application = webapp.WSGIApplication([('/somepage',
> SomeHandler), ('/anotherpage', AnotherHandler)], debug=True)
>
> # GAE will look for a main() with no args and, if found, caches this script
> # so we supply one to make the requests more efficient
> def main():
> run_wsgi_app(application)
>
> # How a file can be both a script and an imprtable module...
> if __name__ == "__main__":
> main()
>
> I'm wondering if I rename that function to not be "main", hence making the
> page just a script and no longer an importable module, will that effectively
> kill the instance after the script runs (my load time is minimal - most of
> my pages simply do datatore-query -> JSON or the reverse) and so minimise my
> instance hours ??
>
> Would it be possible to add something to the GAE API's so that an instance
> can suggest how long it would like to hang around for, or an ability to
> explicitly request to kill itself ?
>
> --
> T
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.5 improvements Make me less scared of Pricing
Currently all my python scripts use the script-or-module mechanism as
recommended by the docs
application = webapp.WSGIApplication([('/somepage',
SomeHandler), ('/anotherpage', AnotherHandler)], debug=True)
# GAE will look for a main() with no args and, if found, caches this script
# so we supply one to make the requests more efficient
def main():
run_wsgi_app(application)
# How a file can be both a script and an imprtable module...
if __name__ == "__main__":
main()
I'm wondering if I rename that function to not be "main", hence making the
page just a script and no longer an importable module, will that effectively
kill the instance after the script runs (my load time is minimal - most of
my pages simply do datatore-query -> JSON or the reverse) and so minimise my
instance hours ??
Would it be possible to add something to the GAE API's so that an instance
can suggest how long it would like to hang around for, or an ability to
explicitly request to kill itself ?
--
T
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.5 improvements Make me less scared of Pricing
No doubt when I moved to HR my instances basically quit dying. With M/ S an idle instance would be killed off quickly, with HR they hang around for days. At the time I was very happy about this. This was a complaint in the past (how fast Google killed idle instances.) But now I think the reverse will happen and people will want them terminated as quickly as possible. On May 13, 6:40 am, Greg wrote: > I'm not seeing any significant reduction in instances either. I did > notice that I got a lot more idle instances when I moved from MS to HR > datastore, for some reason - maybe they are starting to rationalise > this and haven't got to my instances yet. > > Cheers > Greg. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.5 improvements Make me less scared of Pricing
I'm not seeing any significant reduction in instances either. I did notice that I got a lot more idle instances when I moved from MS to HR datastore, for some reason - maybe they are starting to rationalise this and haven't got to my instances yet. Cheers Greg. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
And it would be great to have different pricing for idle instance hours vs active instance hours. Because my app is letting other apps execute, while blocking a little memory. On Fri, May 13, 2011 at 1:20 PM, Vinuth Madinur wrote: > Plus: > > 1. 15 minutes minimum granularity. > 2. Tiered Instance pricing. My app consumes about 40 -80 MB, but I'll be > paying for 128 MB 0r 256 MB minimum. > 3. It's not like they can't meter these or aren't metering these. They > already have the numbers. But they wont price things based on it, which > sucks. > 4. $9 tax. (not all sites start making revenue quick and not everyone is > based out of US. $9 is quite a sum to just have the ability to cross free > quotas.) > > > > On Fri, May 13, 2011 at 8:50 AM, Raymond C. wrote: > >> But still 6 instances for 13 QPS (2.22 x 6) sounds stupid to me if the >> pricing will be instance based. A single regular class VPS can handle at >> least like 30+ QPS. >> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine" group. >> To post to this group, send email to google-appengine@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine?hl=en. >> > > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
Plus: 1. 15 minutes minimum granularity. 2. Tiered Instance pricing. My app consumes about 40 -80 MB, but I'll be paying for 128 MB 0r 256 MB minimum. 3. It's not like they can't meter these or aren't metering these. They already have the numbers. But they wont price things based on it, which sucks. 4. $9 tax. (not all sites start making revenue quick and not everyone is based out of US. $9 is quite a sum to just have the ability to cross free quotas.) On Fri, May 13, 2011 at 8:50 AM, Raymond C. wrote: > But still 6 instances for 13 QPS (2.22 x 6) sounds stupid to me if the > pricing will be instance based. A single regular class VPS can handle at > least like 30+ QPS. > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine" group. > To post to this group, send email to google-appengine@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing
Hmmm... The only possible explanation is that Greg just wanted to shut me up about instances and python, so he hacked the dashboard to lie when I upgraded my code next... (When there is no plausible explanation propose an implausible one) -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of JH Sent: Thursday, May 12, 2011 7:55 PM To: Google App Engine Subject: [google-appengine] Re: 1.5 improvements Make me less scared of Pricing Interesting, I saw no change to my # of instances with 1.5 On May 12, 9:29 pm, "Brandon Wirtz" wrote: > I am slow to upgrade because I like to know that things won't explode, > but this graph shows how 1.5 reduced the number of instances I need > compared to > 1.4 (maybe 1.3) that I was running. Usually I version thing so you > can't see the graph before. > > And yes the traffic stayed about the same. > > I have seen similar drops everywhere I redeployed after upgrading to > 1.5 > > image002.png > 124KViewDownload -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.5 improvements Make me less scared of Pricing
But still 6 instances for 13 QPS (2.22 x 6) sounds stupid to me if the pricing will be instance based. A single regular class VPS can handle at least like 30+ QPS. -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: 1.5 improvements Make me less scared of Pricing
Interesting, I saw no change to my # of instances with 1.5 On May 12, 9:29 pm, "Brandon Wirtz" wrote: > I am slow to upgrade because I like to know that things won't explode, but > this graph shows how 1.5 reduced the number of instances I need compared to > 1.4 (maybe 1.3) that I was running. Usually I version thing so you can't > see the graph before. > > And yes the traffic stayed about the same. > > I have seen similar drops everywhere I redeployed after upgrading to 1.5 > > image002.png > 124KViewDownload -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
