date:20110904

And what would be wrong if google was to constrain their bots to visit
GAE pages only once per week?

Or even better, what if GAE had a mechanism by which the apps could
announce when they have something new to be crawled?


On Sep 4, 12:07 am, "Brandon Wirtz"  wrote:
> Returning a 503 is REALLY REALLY Bad for SEO.
>
> 304 Seems to be ignored by Google Bot on GAE, but also Google Bot will try
> queries to which there are no links, and which no user has ever made.
>
> If your latency goes higher Google Bot will throttle back.  but the only way
> to slow down the page serving is to put a wait timer, which burns Instance
> time. Rob Paul to Pay Peter.
>
> From: google-appengine@googlegroups.com
> [mailto:google-appengine@googlegroups.com] On Behalf Of Rajkumar
> Radhakrishnan
> Sent: Saturday, September 03, 2011 6:41 PM
> To: google-appengine@googlegroups.com
> Subject: Re: [google-appengine] Re: Google Bot Is Your Enemy
>
> @Brandon :
>
> This is the case for one of my popular web-sites too. I believe if latency
> time increases, Google bot will automatically scale down its crawl rate and
> I also fear that such an increased latency will have a negative effect on
> the page ranking.
>
> By the way, have you started using 304 (Not Modified) responses, for pages
> which have not modified ? This can reduce the resource usage by Google Bot.
>
> Worst case option is to resort to a check for every Nth request (10 < N <
> 100) from the bot (of course, using memcache) and send a 503 status..
>
> 503 Service Unavailable
>
> The server is currently unavailable (because it is overloaded or down for
> maintenance). Generally, this is a temporary state.
>
> ..and this should also give a hint to the Google bot to scale down its crawl
> rate. This will be useful when you want to retain better latency and want to
> hint Google bot alone. Again, this can have an effect on the page rank too.
> And I am not sure which is worse bad latency or a 503.
>
> Anyone else has experience in this space ?
>
> Thanks & Regards,
>
> Raj
>
> On Sun, Sep 4, 2011 at 2:52 AM, Darien Caldwell 
> wrote:
>
> Nice, so basically Google is using their own service to tack on
> additional charges to your bill. Doesn't sound ethical.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
>  .
> For more options, visit this group 
> athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
>
> ~~
>
> Build online database applications, over Google App Engine.
>
> iFreeTools Creator -http://creator.ifreetools.com
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group 
> athttp://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] how to delete indexes? just in index.yaml?

2011-09-04 Thread saintthor

i deleted some indexes in index.yaml. uploaded. then in console
Datastore Indexes, i see the deleted index are still serving.

are they really deleted?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The whole instances pricing thing is too complicated for average brains like mine

2011-09-04 Thread Francois Masurel

Hi Tim,

You right, CPU measure is quite mysterious, but there is nothing we can do 
about it and we just have to optimize our code to reduce CPU use.

Francois

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/kWwVQsrLCRUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is Google killing GAE?

Is GAE not just an API that is layered on top of their existing
infrastructure?

Somebody already asked something to the effect of what it would cost
or whether Google itself would go bankrupt if it had to pay the GAE
prices.  Tim and somebody from Google labeled that (what seems like a
legitimate question) as trolling.

But if the infrastructure is already there then it is a fair question
to ask whether GAE users should bring in a better margin than the rest
of GOOG operations that are using that same infrastructure?

On Sep 3, 11:52 pm, Tim Hoffman  wrote:
> This is all just nonsense, they are changing their business model.
> They wouldn't be taking appengine out of preview and then legally committing
> to the platform for 3 years
> if they where going to can.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: how to delete indexes? just in index.yaml?

You have to perform a vacuum_indexes operation.

On Sep 4, 10:08 am, saintthor  wrote:
> i deleted some indexes in index.yaml. uploaded. then in console
> Datastore Indexes, i see the deleted index are still serving.
>
> are they really deleted?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The whole instances pricing thing is too complicated for average brains like mine

I agree the instance based pricing is too complicated because of the nature 
of how GAE works.  On GAE we have no control on resources, Google control it 
all. What we can control is what our code does during the life time within a 
request.  

On instance based pricing (the upcoming one) all we are ASKED to do is to 
make our request return faster.  However there are many area we cannot 
control regarding the time: datastore wait time, URL fetch (the only way to 
communicate between backends and frontend instance), etc

GAE have been charging for CPU time for all operations because thats what we 
can almost fully control (except extra datastore performance due to google's 
server performance), thats why most GAE users (at least the heavy users) 
think the old charge model make more sense, even GAE charge us more on CPU 
based pricing.

Many of us have raised the concern from the beginning that instance based is 
not how GAE works.  Still they are going for it without a valid reason, at 
least without much reasonable reasons (to me).  There are so many methods to 
charge us for using their machines or occupying the memory of their 
machines, yet they go for a way thats not how GAE works.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/F7j93cVfGhgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is Google killing GAE?

I never said trolling, just said it was nonsense, with no evidence to 
support the claim.

In addition the basic out of the box google infrastructure doesn't do 
appengine. Appengine is a whole new 
layer that has been developed,therwise you wouldn't see things like M/S or 
HR datastore's

And you have to ask yourself how and where does each google project/product 
makes money for google.

appengine is a completely different revenue model than gmail.  What is the 
upside for google running appengine.
The upside of running gmail for free is obvious, the same can't be said 
about appengine




-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/S86zXCI-E1cJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The whole instances pricing thing is too complicated for average brains like mine

I totally agree with you francois I just found the whining without 
 constructive dialog starting to be a more than a little painful.

I don't have a problem helping people get more out of appengine, but 
baseless unsupported sniping isn't helping anyone, and the signal to 
noise ratio is drowning out contructive suggestions for people trying to 
work out what they do about the coming changes.

As I said before I am not a google aplogist. They do good things and silly 
things ;-)

See ya

T

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/HEoCGOGxitMJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Keep it short: Who is forced to leave GAE?

The articles are not here to say HOW TO SCALE.  I was just saying scaling up 
is much easier and effective than scaling out.  You can scale up with just a 
few clicks on IAAS like AWS, without ever considering scaling out, to 
support a heavy site.

GAE can only does scale out, because its how its designed.  But saying GAE 
can scale while the other platforms can't do it easily is just wrong.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/nX4rGIGNJmQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Tim, I just curious why you think $30/month for hosing a site that has 
"(40-100 visitors a day on average, 100-250 pages views a day)" is 
reasonable?  I suppose you know with that money you can host over a thousand 
sites with that amount of traffic on a single machine elsewhere right?

Or just because you dont want to migrate out from GAE because you dont care 
to spend a thousand times more money?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/2MYYixQ4D1QJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Re: Google Bot Is Your Enemy

2011-09-04 Thread Brandon Wirtz

I would settle for plays by the same rules as all other websites.

Though I do kind of think that the bandwidth consumed by GOOG on GOOG
infrastructure should be free.

-Original Message-
From: google-appengine@googlegroups.com
[mailto:google-appengine@googlegroups.com] On Behalf Of zdravko
Sent: Sunday, September 04, 2011 12:50 AM
To: Google App Engine
Subject: [google-appengine] Re: Google Bot Is Your Enemy

And what would be wrong if google was to constrain their bots to visit GAE
pages only once per week?

Or even better, what if GAE had a mechanism by which the apps could announce
when they have something new to be crawled?


On Sep 4, 12:07 am, "Brandon Wirtz"  wrote:
> Returning a 503 is REALLY REALLY Bad for SEO.
>
> 304 Seems to be ignored by Google Bot on GAE, but also Google Bot will 
> try queries to which there are no links, and which no user has ever made.
>
> If your latency goes higher Google Bot will throttle back.  but the 
> only way to slow down the page serving is to put a wait timer, which 
> burns Instance time. Rob Paul to Pay Peter.
>
> From: google-appengine@googlegroups.com 
> [mailto:google-appengine@googlegroups.com] On Behalf Of Rajkumar 
> Radhakrishnan
> Sent: Saturday, September 03, 2011 6:41 PM
> To: google-appengine@googlegroups.com
> Subject: Re: [google-appengine] Re: Google Bot Is Your Enemy
>
> @Brandon :
>
> This is the case for one of my popular web-sites too. I believe if 
> latency time increases, Google bot will automatically scale down its 
> crawl rate and I also fear that such an increased latency will have a 
> negative effect on the page ranking.
>
> By the way, have you started using 304 (Not Modified) responses, for 
> pages which have not modified ? This can reduce the resource usage by
Google Bot.
>
> Worst case option is to resort to a check for every Nth request (10 < 
> N <
> 100) from the bot (of course, using memcache) and send a 503 status..
>
> 503 Service Unavailable
>
> The server is currently unavailable (because it is overloaded or down 
> for maintenance). Generally, this is a temporary state.
>
> ..and this should also give a hint to the Google bot to scale down its 
> crawl rate. This will be useful when you want to retain better latency 
> and want to hint Google bot alone. Again, this can have an effect on the
page rank too.
> And I am not sure which is worse bad latency or a 503.
>
> Anyone else has experience in this space ?
>
> Thanks & Regards,
>
> Raj
>
> On Sun, Sep 4, 2011 at 2:52 AM, Darien Caldwell 
> 
> wrote:
>
> Nice, so basically Google is using their own service to tack on 
> additional charges to your bill. Doesn't sound ethical.
>
> --
> You received this message because you are subscribed to the Google 
> Groups "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
>  .
> For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.
>
> --
>
> ~~
> 
>
> Build online database applications, over Google App Engine.
>
> iFreeTools Creator -http://creator.ifreetools.com
>
> --
> You received this message because you are subscribed to the Google 
> Groups "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group
athttp://groups.google.com/group/google-appengine?hl=en.

--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: how to delete indexes? just in index.yaml?

2011-09-04 Thread saintthor

thanks

i deleted 11 indexes in 17. i hope this can reduce my datastore ops.

On 9月4日, 下午4时13分, Philip  wrote:
> You have to perform a vacuum_indexes operation.
>
> On Sep 4, 10:08 am, saintthor  wrote:
>
>
>
>
>
>
>
> > i deleted some indexes in index.yaml. uploaded. then in console
> > Datastore Indexes, i see the deleted index are still serving.
>
> > are they really deleted?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Abysmal Query Performance in Backend

Hey,

Yesterday, I've had a service running to delete a huge (1 TB) backlog of 
data from our appengine instance. It was working well at around 100 
entities/sec (has about a million entities). Today, however, it slowed down 
to around 0-1 entities/sec, using the same code.

I tracked it down to queries taking very long and then failing at a rate of 
around 90% with a timeout exception. I've configured the deadline to None, 
but that didn't help. Plus, yesterday everything worked fine, I've deleted 
around a million entities before it started to choke.

The application ID is 'towermadness', the backend ID is 'deleter'.

Thanks!
Volker

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/bDMVMNEYxIwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Abysmal Query Performance in Backend

Oh, I ment the dataset has 100 million entities, so it's far from being 
done. There is still around 850 GiB worth of data that wants to be deleted.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/WPeRpAqa5j8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is Google killing GAE?

However, why should GAE end up contributing more to the bottom line
than GMail or YouTube or many other GOOG properties will ever be able
to contribute - for each dollar of cost ?

On Sep 4, 4:24 am, Tim Hoffman  wrote:
> I never said trolling, just said it was nonsense, with no evidence to
> support the claim.
>
> In addition the basic out of the box google infrastructure doesn't do
> appengine. Appengine is a whole new
> layer that has been developed,therwise you wouldn't see things like M/S or
> HR datastore's
>
> And you have to ask yourself how and where does each google project/product
> makes money for google.
>
> appengine is a completely different revenue model than gmail.  What is the
> upside for google running appengine.
> The upside of running gmail for free is obvious, the same can't be said
> about appengine

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Hi

I don't want to migrate because I don't want to deal with os and complete 
application stack.
I don't want to have to wake and find something has fallen over.

Also I know I can reduce my cost below $30 a month, probably to half that, 
when I move to HRD.

I have had that site on appengine for over 2 years.  With virtually no 
problems. No disk full, no security patches etc

(I have had to look after E10K's, I was IT manager for 300 seat call center 
and run mobile phone billing,) I know what it takes
to run this stuff.  I don't want to do any of this. I am in the process of 
leaving IT to do my own stuff.

Also $30 a month for piece of mind and no worries, I will take that any 
day, 

And finally we spend around $1000 dollars a month on Adwords, so $30 a month 
for no worries,
hosting,  why would I even think about moving.  My power bill for running a 
single 700 watt pump for a full year is $1400.
I won't pay for anything I see no value in.

T

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/suSMsaQ-D0kJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is Google killing GAE?

It's not like GMail potential is not already known after all of these
years of running it.  So, how much do they spend on it and how much
does it contribute to the bottom line?

The potential of GAE that nobody is mentioning is that at a
comparatively reasonable cost (when compare to say GMail) it keeps
GOOG in that business just in case that down the road they want to be
in that business.  I am certain that even they do not know yet how
profitable it could be 5 years down the road until they themselves see
what competition does and even what tweaking of their GAE automation
ends up producing.

On Sep 4, 5:29 am, zdravko  wrote:
> However, why should GAE end up contributing more to the bottom line
> than GMail or YouTube or many other GOOG properties will ever be able
> to contribute - for each dollar of cost ?
>
> On Sep 4, 4:24 am, Tim Hoffman  wrote:
>
>
>
>
>
>
>
> > I never said trolling, just said it was nonsense, with no evidence to
> > support the claim.
>
> > In addition the basic out of the box google infrastructure doesn't do
> > appengine. Appengine is a whole new
> > layer that has been developed,therwise you wouldn't see things like M/S or
> > HR datastore's
>
> > And you have to ask yourself how and where does each google project/product
> > makes money for google.
>
> > appengine is a completely different revenue model than gmail.  What is the
> > upside for google running appengine.
> > The upside of running gmail for free is obvious, the same can't be said
> > about appengine

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Keep it short: Who is forced to leave GAE?


So if it is so easy to scale up and out with these other solutions, why are 
you still here?

Why did you even start with appengine?

What value you do you see in appengine vs these other IAAS platforms?

I really would like to know.  You seem to only ever have negative comments 
about appengine at the moment and don't seem to
provide much in the way constructive suggestions for people to help them 
improve what they are doing with appengine.

T


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/iJL1bbdfBtIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Is Google killing GAE?

i never said it should contribute more to the bottom line, and there is no 
evidence to suggest it does.
All of this is just supposition on your part.  Do you any evidence to back 
up your claims.

Enough said by me on this now.

See ya

T

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/ezFzyqrM35YJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

I think you are definitely the target users of GAE, having money to spend 
without caring the price. ("its cheap compared to the revenue i earn from *
elsewhere*")

You raised some good points of GAE though, but my point is still "the price 
doesnt worth it".  You can host the same tiny site on heroku or AWS without 
costing you a buck.



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/YszmsgaZHlgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Keep it short: Who is forced to leave GAE?

I started on GAE because of the same reason of you (PAAS without all the 
managing stuff).  Why I see GAE negatively now is that the price is over 
expensive and the pricing model is wrong (or at least not like when I 
started using it).  Isn't that obvious?  And there are no conflict between 
the two.  Why you think raising negative comments under this situation is 
wrong? 

We are not someone who just came in and say "you sucks" to Google.  We are 
the customers and testers who have been using and helped GAE from the early 
days till today.


We are providing no constructive suggestions?  We have been doing it all 
over the forum in the past few months.  Or you just think "saying sth good 
about the new pricing" is constructive suggestions?

What I am doing here (and maybe many other who are raising concerns now) is 
because we want Google to change their mind by realizing the problems that 
the change is producing.  Of course we will go if Google insists to use the 
new pricing models and if it dont fit us, but at least we are still trying 
before it is too late.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/dZVDd3tH3o4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

回覆：Re: [google-appengine] Data Transfer to AWS

2011-09-04 Thread Natalie



We are in the same situation. How’s everyone’s progress?

We’re planning to migrate to AWS but we have quite a bit of data to move 
like you guys. We’re trying to minimize service disruption and keeping our 
site online while making the move. Here’s our scheme (would love to hear 
suggestions form you guys):

1) Upload a new app version that adds a Boolean value to every type of 
entity in Datastore. Call it “updated”. All call to put() will set this 
Boolean to true, *and* push the put() data to a GAE pull queue.

2) Use Remote API to batch get all entity with Boolean=false. This will get 
any unmodified data from DS. Data that are modified by DS after fetch can be 
retrieved from the pull queue later.

3) Transform the data and push them to AWS.

4) From AWS, lease the data from pull queue and fill up the database

5) Modify DNS record to point to the new AWS site while keeping the GAE app 
alive until it receives no traffic.


We are trying to take advantage of the GAE pull queue’s ability to be 
accessed outside of GAE. Do you guy foresee any problem with this scheme? 
We’re busy coding this at the moment and would love to hear your input. 
Thank you.

Also we are planning to use AWS Elastic Beanstalk to ease Tomcat admin 
effort. Anyone could share their experience with this technology?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/FWTtjG5urQ0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Concurrency and instance startup logic

2011-09-04 Thread Mike

Like many, I'm deeply disappointed that Google went ahead with the
extortionate price increases, but instead of complaining about it
(because it seems to be falling on deaf ears) I'm trying to come up
with a workable solution to decrease costs.

I have a couple of apps that see an average amount of traffic (around
1 request every 3 seconds, fairly constantly) and I'm trying to
persuade the scheduler to keep only 1 instance running.

I'm confident only 1 instance is needed. - Latency is around 150ms and
thread-safe is true. However, there always appears to be 2 or more
instances active. Its ridiculous to see these instances having served
1 request sitting idle for 15 minutes. My understanding was that
enabling concurrent requests would stop these additional instances
starting up because if one request is waiting on I/O (common) another
request can be handled.

What is the instance startup logic when thread-safe is true? I can not
see any difference. If anything, setting thread-safe to true has
increased the number of idle instances. In my eyes, the scheduler is
broken:

Min Pending Latency appears to have no effect. Even set at 15s you
still see new instances start up unexpectedly.

Max Idle Instances should drop down to zero. I do not want an
additional idle instance bringing the total instances to 2.

Can we have an option to decrease the 15 minute idle time before
stopping. I do not want to pay for 15 minutes instance time to serve
one request.

Another point: instance Startup time seems way too high in general. I
do not understand why it takes at least 4 seconds to start an
instance. Lightweight servlet containers can start in 200ms these
days. If startup time was optimised, the 15 minute idle period reduced
to say 5 minutes and the scheduler made less enthusiastic about
starting new instances I think the changes might be (almost)
acceptable. However, right now, I think a lot of people are most
annoyed about paying for instances that are sitting there (mostly)
idle - and this is understandable, we got used to paying for CPU time.

I would really like to know how the scheduler decides whether or not
to start up a new instance when thread-safe is true. Is this
documented anywhere?

Kind regards,

Mike.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] A suggestion: open-source the scheduler

Seeing how scheduler has some problems and everyone's bill depends on
how well it does its job, I think it would be fair and would do a
great deal to make new costs more transparent if you open-sourced the
scheduler. Ideally, one would be able to override scheduler logic for
their application, but I don't think it will happen just because the
scheduler is most likely written in C (and for other reasons). So at
least seeing how the decisions to spin up and shut down instances are
actually made, we could have a constructive discussion about it.

Please respond if you would like to see this happen too.

-Sergey

-- 
http://self.maluke.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: urlfetch stops working at times (DownloadError - ApplicationError 2), but keeps working from dev server

2011-09-04 Thread Egor Ryabkov

http://groups.google.com/group/google-appengine/browse_thread/thread/4ce6349f416c57cd/475206448da9c32d?lnk=gst&q=egor83#475206448da9c32d

On Tue, Aug 30, 2011 at 02:21, egor83  wrote:

> My GAE application fetches a web page, and it works fine when I run it
> on a development server on my PC.
> However, when it's deployed to GAE, urlfetch fails with DownloadError
> (ApplicationError: 2 ).
>
> Repeating fetches doesn't help, further attempts fail within 0.05..0.6
> seconds from each other, so it's not a timeout problem.
> The server from which I try to fetch a page is also available when
> there are problems.
>
> This happens in periods, so sometimes everything works fine:
> 21:30 to 22:20 UTC 29 August - not working
> 14:20 to 18:15 UTC 29 August - works OK
> 04:00 to 05:30 UTC 29 August - not working
> around 02:20 UTC 29 August - works OK
> 20:00 28 Aug to 00:15 UTC 29 August  - not working
>
>
> Any advice or explanation of this problem?
> Thanks in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Concurrency and instance startup logic

I have an app that has instances page looking like this: 
http://i.imgur.com/YROrD.png

It's a very small app with billing disabled. It will not work within free 
quota after the pricing change simply because the scheduler is no good. 

I think one way to fix this would be to open-source the scheduler -- that 
would add some transparency at least (I've sent a more detailed email about 
this to the group, but it seems to have been filtered as spam or whatever).

-Sergey

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/-taz0ciL5BoJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

I can host on AWS without costing a buck for the first year with a micro 
instance.

Second year it will cost.

Also I need to keep ubuntu patched, my ebs volume clean (clean out logs).
etc

AWS is not immune to outages (neither is appengine)

And heroku, well lets say I am a python person through and through, so 
heroku is not for me.

dotcloud is probably a better choice for me as an alternative to appengine.

Also I do care about what I pay for, however my time is worth far more.  I 
don't bother consulting for less than $100 per hour.
So like you say cost is all relative.

Rgds

Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/c31K9Q0SmjcJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The Amazing Story Of Appengine And The Two Orders Of Magnitude

I made a very similar request back in June -- there was zero reaction from 
Google.

My suggestion is a little different -- provide a separate slider for 
"maximum latency for task queue requests".

-Sergey

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/ZgT491lCdLUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Sender mail on GAE

2011-09-04 Thread Deepak Singh

Hi,

I have an application metasearchprodenv.appspot.com which uses java mail
service.

I want to create an email address with this domain name and use this address
as a sender mail for the application.

I could not find any way to create email address and link this to this
account.
Anybody can suggest...

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] How deployments influence instance hours?

So when we deploy a new version, let's assume there was one instance 
running, it gets shut down, a new one is then started, how many 
instance-hours are consumed that hour?

Another option, the version deployed has a different name. The default 
version is switched to that new version and the previous one stops getting 
traffic. How many instance-hours are consumed that hour?

-Sergey

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/M3JuuDRhwIEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Hi Raymond, 

Lets look at some real numbers rather than broad statements.

In fact in the second year an AWS micro instance is charged at 0.02 per hour

So 

>>> 356 * (24 * 0.02)   
170.88

a reserved instance is

>>> 356 * (24 * 0.007)  + 56
115.808

vs appengine with a single instance around $30 per month so lets make it a 
round $400

But those charges for the AWS micro instance doesn't include bandwidth EBS 
volume networking etc...
admittedly they will be small, but the EBS volume of 8GB is going to be 
close to $1 per month current pricing
so we are up to around $182 and $127 per year. Not even half the cost of 
appengine, assuming I don't get my costs down there,
(But as I said before I am confident I can once I move to HR)

The $150 - $250 dollars a year I save on EC2 is not worth the extra hassle 
of maintaining a complete stack myself.

Lets talk about the stack on EC2.

Ubuntu
Ngnix
pyramid or bobo, or webapp2, ...  ( I personally am not keen on django)
an ORM (sqlobject, or storm)  and mysql or no ORM
and run ZODB, 

all patched, configured etc... by me and maintaned by me.

I can actually do all this in my sleep, been doing that sort of thing for 
many years, but I don't want to.

The only real upside I see is this solution gives me, is the ability to take 
it somewhere else, oh and at best a few 
hundred dollars saved. If I need to scale at all, I now need to move to a 
small instance, and that is priced about the 
same as appengine.

Rgds

Tim


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/ojUoj5nBGnQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] The Amazing Story Of Appengine And The Two Orders Of Magnitude

Here's another approach, which seems a lot more powerful than either of the 
things we suggested:

http://code.google.com/p/googleappengine/issues/detail?id=5775

On Sep 4, 2011, at 7:05 AM, Sergey Schetinin wrote:

> I made a very similar request back in June -- there was zero reaction from 
> Google.
> 
> My suggestion is a little different -- provide a separate slider for "maximum 
> latency for task queue requests".
> 
> -Sergey
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/google-appengine/-/ZgT491lCdLUJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] The Amazing Story Of Appengine And The Two Orders Of Magnitude

Well,.. I intentionally tried to make a suggestion that requires as little 
deviation from the existing plan as possible. I have no faith in GAE team 
making anything but baby steps towards the community -- that request in the 
linked ticket is way too ambitious.

-Sergey

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/TIcQn3mpRLgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: GAE pricing is not suited for smaller apps

2011-09-04 Thread Vinuth Madinur

Hi Tim,

I'm curious to know why the prices will be half after you move to HR.


On Sun, Sep 4, 2011 at 5:11 PM, Tim Hoffman  wrote:

> Hi Raymond,
>
> Lets look at some real numbers rather than broad statements.
>
> In fact in the second year an AWS micro instance is charged at 0.02 per
> hour
>
> So
>
> >>> 356 * (24 * 0.02)
> 170.88
>
> a reserved instance is
>
> >>> 356 * (24 * 0.007)  + 56
> 115.808
>
> vs appengine with a single instance around $30 per month so lets make it a
> round $400
>
> But those charges for the AWS micro instance doesn't include bandwidth EBS
> volume networking etc...
> admittedly they will be small, but the EBS volume of 8GB is going to be
> close to $1 per month current pricing
> so we are up to around $182 and $127 per year. Not even half the cost of
> appengine, assuming I don't get my costs down there,
> (But as I said before I am confident I can once I move to HR)
>
> The $150 - $250 dollars a year I save on EC2 is not worth the extra hassle
> of maintaining a complete stack myself.
>
> Lets talk about the stack on EC2.
>
> Ubuntu
> Ngnix
> pyramid or bobo, or webapp2, ...  ( I personally am not keen on django)
> an ORM (sqlobject, or storm)  and mysql or no ORM
> and run ZODB,
>
> all patched, configured etc... by me and maintaned by me.
>
> I can actually do all this in my sleep, been doing that sort of thing for
> many years, but I don't want to.
>
> The only real upside I see is this solution gives me, is the ability to
> take it somewhere else, oh and at best a few
> hundred dollars saved. If I need to scale at all, I now need to move to a
> small instance, and that is priced about the
> same as appengine.
>
> Rgds
>
> Tim
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/ojUoj5nBGnQJ.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

2011-09-04 Thread Francois Masurel

Thanx for this interesting post, Tim.

I still think GAE is a viable solution for some low traffic web sites.

I managed a dedicated server for 3 years and it was really painful and very 
time consuming.

I'll try to stick to GAE for now even if my bill increases a bit (I was 
already paying for "Always on"  instances).

Francois 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Q7huS3o-D_UJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] The Amazing Story Of Appengine And The Two Orders Of Magnitude

Let's give the GAE team some credit, and assume that they can discern the 
short-term-fix issues from the long-term-solution issues.

I starred this new idea, because, in the long term, it would be a really 
amazing toolset to have for my enterprise apps (not my free app, where all I 
want to do is keep idle instances from running).  It also would be a level of 
load balancing control never before seen in the cloud world, which should get 
some people at google interested.

And I suspect that Brandon would find ways to use it that just blew us away.

On Sep 4, 2011, at 7:47 AM, Sergey Schetinin wrote:

> Well,.. I intentionally tried to make a suggestion that requires as little 
> deviation from the existing plan as possible. I have no faith in GAE team 
> making anything but baby steps towards the community -- that request in the 
> linked ticket is way too ambitious.
> 
> -Sergey
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/google-appengine/-/TIcQn3mpRLgJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Keep it short: Who is forced to leave GAE?

Raymond

Unfortunately I don't see a lot of postings detailing what the real 
alternatives to appengine are, what 
certain size traffic sites and data sets would actually cost to run on these 
alternatives.

May be you could provide a bit of an over view of your apps design, what its 
costing you, 
how much it would cost to move elsewhere and run it on some other platform 
and what it would really cost to run 
on that platform. What are the strengths and weaknesses of these other 
platforms are, so that others in this community 
can get some guidance.

Lots of people are throwing around statements like it is cheaper to run on 
heroku or AWS, but just not a lot of hard facts 
at the moment.  

A quick look at Heroku tells me a single dyno + 20GB shared DB will cost $15 
per month, and 2 dynos + 20GB 
shared DB is $50 per month.  Now I know nothing about Heroku but thats what 
their pricing page is telling me.
 (Correct these figures if they are wrong)

Which all suggests to me that Heroku would cost about the same to host a 
small appengine app.

Rgds

Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/4uORTgZzKBUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

2011-09-04 Thread de Witte

What do you mean with the limited database functionality. We are developing 
a large app with a lot of functions and so far no problems. Except that you 
have to reprogram your relational database mindset. (this sounds very geeky)

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Z5hGNq2xY1EJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: GAE pricing is not suited for smaller apps

Hi Gubbi

I feel I can halve them because feel a lot more comfortable if there are no 
instances hanging around and have them start on demand
on HR.  M/S startup has always proven to be problematic.

I have a test instance that I check every so often on HR and it has yet to 
fail to startup once or take 20+ seconds to start up.

Also moving to HR gives me the opportunity to use 2.7.  threaded requests 
means that I can serve more of the 30ms cached responses
from a single instance, which means it is less likely that more than a 
single instance will be needed most of the time.

My site really only gets traffic between 6:00am and 11:30pm (very locally 
focussed) so I could keep a single instance hot during that time, 
and assuming google sorts out the scheduler and lets me have no idle 
instances, I could let all the instances go from 11:00pm to 6:00am 
and have them started on demand.

So on that basis I firmly believe I can get down to less than $30 per month.

Rgds

Tim


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/tXcE4Dk_-MgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Keep it short: Who is forced to leave GAE?

I'm not leaving GAE, but in the comments on a RWW article, I saw that the 
AppScale folks ( http://appscale.cs.ucsb.edu/ ) seem to be trolling for 
customers.  (I mean trolling in the fishing sense, no the usenet sense.)

I'm sure those people must be on these mailing lists.  Perhaps one of them 
could speak up to how turn-key their solution is at this point?

I wonder if they've run any big apps like CDN-in-a-box on AWS, to see how the 
costs and performance compare.

On Sep 4, 2011, at 8:02 AM, Tim Hoffman wrote:

> Raymond
> 
> Unfortunately I don't see a lot of postings detailing what the real 
> alternatives to appengine are, what 
> certain size traffic sites and data sets would actually cost to run on these 
> alternatives.
> 
> May be you could provide a bit of an over view of your apps design, what its 
> costing you, 
> how much it would cost to move elsewhere and run it on some other platform 
> and what it would really cost to run 
> on that platform. What are the strengths and weaknesses of these other 
> platforms are, so that others in this community 
> can get some guidance.
> 
> Lots of people are throwing around statements like it is cheaper to run on 
> heroku or AWS, but just not a lot of hard facts 
> at the moment.  
> 
> A quick look at Heroku tells me a single dyno + 20GB shared DB will cost $15 
> per month, and 2 dynos + 20GB 
> shared DB is $50 per month.  Now I know nothing about Heroku but thats what 
> their pricing page is telling me.
>  (Correct these figures if they are wrong)
> 
> Which all suggests to me that Heroku would cost about the same to host a 
> small appengine app.
> 
> Rgds
> 
> Tim
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/google-appengine/-/4uORTgZzKBUJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Why do we keep seeing these comments "Yikes, maybe Google now has too few 
large customers and that they now on purpose have chosen to kill GAE"

Where is the evidence that allows you to draw that conclusion?

or are you actually trying to start or perpetuate an internet meme and hope 
it comes true ?


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/1LmQCFjdck8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Things like only one inequality match (such as less than or greater than) 
allowed in queries and no LIKE operator. The new full text search API may 
solve some of that.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/MTZk8YGEqw4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

I found this article: 
http://blog.labslice.com/2010/12/2010-cloud-computing-winner.html

It shows that GAE has a very tiny market share. Managers at Google may 
calculate that the market share for GAE will remain too low for them to 
invest much further in the product. I certainly hope my speculation is 
wrong, but it's good to plan for the worst.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/3dK4o-ZrXe8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Didn't you read the going out of preview notice from google.
They garuntee to provide the service for 3 years. This gives organisations 
piece of mind.

And you are drawing the conclusion from that report which is very vague, 
that google plan to kill gae.  What have you been smoking ?

Its comparing job trends not platform take up.  And they list S3 (which 
isn't even remotely comparable), so how did they measure it?
They count the number of jobs that lists each of the names s3, ec2 azure and 
appengine.  So in 2010 there 
where less jobs requesting experience in appengine than the other three.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/zt1yM4ENafkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Small enty level VPS and cloud based services v appengine

More than a few people have said in the groups lately that appengine is
unsuitable for entry level apps due to the new pricing schedule.
I am not so sure, but there hasn't been any real information about the
alternatives, so I thought I would start to collate some numbers

So to that end I have included a spreadsheet here with a summary of a number
of VPS or cloud providers solutions that I would consider
might be suitable to run a small entry level appengine app.

For the sake of the discussion you would want to run a stack that looks like
the following

linux
nginx/apache
a light weight stack say webapp2, pyramid, tipfy (a lightweight framework)
an ORM (sqlobject/Storm)
and mysql

This doesn't really equate to a heroku offering, but lets say in each case
we need a single instance of something running, 512MB at a minimum to run
the small stack and an RDBMS , with at least 1GB of
storage available if no OS is factored in and 5GB if the OS counts in the
storage allocation.

So here is a
spreadsheet.
https://docs.google.com/spreadsheet/ccc?key=0At1LTa6ONStgdExuRUl2QUdOVE0yWDFDbUJyX09QNlE&hl=en_US

I know none of these compare service wise directly with appengine. But
these are the sort of services people name
frequently as viable alternatives that are cheaper than appengine. So lets
look at the reallity

The equivalent appengine basic service would be a single permanently idle
instance running 24 hours a day, plus low volume of traffic and < 1GB of
data in the datastore. So more instances might spin up. Under 2.7 with
threading requests we might not see more instance start. A $ figure to
apply to such an appengine app
would probably be between $30 and $40 per month.

Please suggest refinements to these models, and additonal detail to go into
the spread sheet.

On the face of it I am not convinced many of these services are
significantly cheaper than appengine especially when you take into account
most of them
require you to manage the complete stack.

Hope this helps focus the discussion and provide some reality checks.

I personally have no plans to move off appengine. But due plan to do some
tuning.

Regards

Tim

--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/khEdCXoWqpsJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Woo...Tim, your talk is getting annoying. "What have you been smoking ?"? I 
think it's a question you should ask yourself.  

How about come back to the discussion after you have real usage on appengine? I 
mean, not hosting a site that get hundreds of pageview a DAY. Instead make a 
popular web app that functions for massive number of users.  Why do you think 
you know the value of appengine so much when you are actually not paying much 
on it?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/frocn-YTXk0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: GAE pricing is not suited for smaller apps

2011-09-04 Thread Stephen

On Sun, Sep 4, 2011 at 2:05 PM, Tim Hoffman  wrote:
>
> Didn't you read the going out of preview notice from google.
> They garuntee to provide the service for 3 years.

They have ALWAYS provided this guarantee. See section 10 of the Terms
of Service you agreed to:

  http://code.google.com/appengine/terms.html

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

There is a European provider called OVH. They currently host about
100k dedicated server and also offer virtual instances. The cheapest
starts at 0.0119 € / Hour and includes 256MB Ram, 8Ghz, 5GB storage
and unmetered Mbps connection: 8,8536€/month.

On Sep 4, 3:26 pm, Tim Hoffman  wrote:
> Hi
>
> More than a few people have said in the groups lately that appengine is
> unsuitable for entry level apps due to the new pricing schedule.
> I am not so sure, but there hasn't been any real information about the
> alternatives, so I thought I would start to collate some numbers
>
> So to that end I have included a spreadsheet here with a summary of a number
> of VPS or cloud providers solutions that I would consider
> might be suitable to run a small entry level appengine app.
>
> For the sake of the discussion you would want to run a stack that looks like
> the following
>
> linux
> nginx/apache
> a light weight stack say webapp2, pyramid, tipfy (a lightweight framework)
> an ORM (sqlobject/Storm)
> and mysql
>
> This doesn't really equate to a heroku offering, but lets say in each case
> we need a single instance of something running, 512MB at a minimum to run
> the small stack and an RDBMS , with at least 1GB of
> storage available if no OS is factored in and 5GB if the OS counts in the
> storage allocation.
>
> So here is a
> spreadsheet.https://docs.google.com/spreadsheet/ccc?key=0At1LTa6ONStgdExuRUl2QUdO...
>
> I know none of these compare service wise directly with appengine.  But
> these are the sort of services people name
> frequently as viable alternatives that are cheaper than appengine. So lets
> look at the reallity
>
> The equivalent appengine basic service would be a single permanently idle
> instance running 24 hours a day, plus low volume of traffic and < 1GB of
> data in the datastore. So more instances might spin up.  Under 2.7 with
> threading requests we might not see more instance start.  A $ figure to
> apply to such an appengine app
> would probably be between $30 and $40 per month.
>
> Please suggest refinements to these models, and additonal detail to go into
> the spread sheet.
>
> On the face of it I am not convinced many of these services are
> significantly cheaper than appengine especially when you take into account
> most of them
> require you to manage the complete stack.
>
> Hope this helps focus the discussion and provide some reality checks.
>
> I personally have no plans to move off appengine.  But due plan to do some
> tuning.
>
> Regards
>
> Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

There is a European provider called OVH. They currently host about
100k dedicated servers and also offer virtual instances. The cheapest
starts at 0.0119 € / Hour and includes 256MB Ram, 8Ghz, 5GB storage
and unmetered 100 Mbps connection: 8,8536€/month.

I think they should be in your table.

On Sep 4, 3:26 pm, Tim Hoffman  wrote:
> Hi
>
> More than a few people have said in the groups lately that appengine is
> unsuitable for entry level apps due to the new pricing schedule.
> I am not so sure, but there hasn't been any real information about the
> alternatives, so I thought I would start to collate some numbers
>
> So to that end I have included a spreadsheet here with a summary of a number
> of VPS or cloud providers solutions that I would consider
> might be suitable to run a small entry level appengine app.
>
> For the sake of the discussion you would want to run a stack that looks like
> the following
>
> linux
> nginx/apache
> a light weight stack say webapp2, pyramid, tipfy (a lightweight framework)
> an ORM (sqlobject/Storm)
> and mysql
>
> This doesn't really equate to a heroku offering, but lets say in each case
> we need a single instance of something running, 512MB at a minimum to run
> the small stack and an RDBMS , with at least 1GB of
> storage available if no OS is factored in and 5GB if the OS counts in the
> storage allocation.
>
> So here is a
> spreadsheet.https://docs.google.com/spreadsheet/ccc?key=0At1LTa6ONStgdExuRUl2QUdO...
>
> I know none of these compare service wise directly with appengine.  But
> these are the sort of services people name
> frequently as viable alternatives that are cheaper than appengine. So lets
> look at the reallity
>
> The equivalent appengine basic service would be a single permanently idle
> instance running 24 hours a day, plus low volume of traffic and < 1GB of
> data in the datastore. So more instances might spin up.  Under 2.7 with
> threading requests we might not see more instance start.  A $ figure to
> apply to such an appengine app
> would probably be between $30 and $40 per month.
>
> Please suggest refinements to these models, and additonal detail to go into
> the spread sheet.
>
> On the face of it I am not convinced many of these services are
> significantly cheaper than appengine especially when you take into account
> most of them
> require you to manage the complete stack.
>
> Hope this helps focus the discussion and provide some reality checks.
>
> I personally have no plans to move off appengine.  But due plan to do some
> tuning.
>
> Regards
>
> Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Hi Raymond

This particular discussion was focused on small apps. Not big apps.  Note 
the subject "GAE pricing is not suited for smaller apps"

I have been and am involved with large apps. Thats a whole different 
discussion.  And I have repeatedly stated I am not talking about pricing at 
that scale in this thread.  I and I have not defended googles pricing on 
larger scale apps any where in these groups. 

The what are you smoking comment was directed at the conclusion drawn about 
job ads for cloud providers
as an indication that google would drop gae.  Such a conclusion I find just 
find untenable, based on the evidence in that report.

T

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/bnzJJ8jSzasJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Google Bot Is Your Enemy

2011-09-04 Thread Stephen

On Sun, Sep 4, 2011 at 8:49 AM, zdravko  wrote:
>
> Or even better, what if GAE had a mechanism by which the apps could
> announce when they have something new to be crawled?

If only...

  http://www.sitemaps.org/

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: RE: [google-appengine] Re: Google Bot Is Your Enemy

+1 from me on that score Brandon.  

T

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/WUlDa8bUNGgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Small enty level VPS and cloud based services v appengine

I don't buy the $30 - $40 / month figure for GAE.  With just a tiny bit of 
tuning, my single-threaded M/S app is now running on just one instance, 
handling both periodic robots (that keep the instance from dying; this is 
unavoidable in my application), and a regular user workload:



You can clearly see the point at which I deployed my tuning. :)

I'd want to have the capacity for the occasional (rare) spin-up from concurrent 
user access, so I'd go for the $9/month plan.

Let's assume, for the sake of argument, that people take the time to optimize 
like I did, and also that they take advantage of edge caching.  And also, the 
site is small, so GoogleBot isn't going to go for the periodic kill (I've never 
seen GoogleBot assault my site).

So I think the right number for GAE for the scenario you laid out is $9.  Not 
$30-$40.

On Sep 4, 2011, at 9:26 AM, Tim Hoffman wrote:

> Hi
> 
> More than a few people have said in the groups lately that appengine is 
> unsuitable for entry level apps due to the new pricing schedule.
> I am not so sure, but there hasn't been any real information about the 
> alternatives, so I thought I would start to collate some numbers
> 
> So to that end I have included a spreadsheet here with a summary of a number 
> of VPS or cloud providers solutions that I would consider
> might be suitable to run a small entry level appengine app.
> 
> For the sake of the discussion you would want to run a stack that looks like 
> the following
> 
> linux
> nginx/apache
> a light weight stack say webapp2, pyramid, tipfy (a lightweight framework)
> an ORM (sqlobject/Storm)
> and mysql
> 
> This doesn't really equate to a heroku offering, but lets say in each case
> we need a single instance of something running, 512MB at a minimum to run the 
> small stack and an RDBMS , with at least 1GB of 
> storage available if no OS is factored in and 5GB if the OS counts in the 
> storage allocation.
> 
> So here is a spreadsheet. 
> https://docs.google.com/spreadsheet/ccc?key=0At1LTa6ONStgdExuRUl2QUdOVE0yWDFDbUJyX09QNlE&hl=en_US
> 
> I know none of these compare service wise directly with appengine.  But these 
> are the sort of services people name
> frequently as viable alternatives that are cheaper than appengine. So lets 
> look at the reallity
> 
> The equivalent appengine basic service would be a single permanently idle 
> instance running 24 hours a day, plus low volume of traffic and < 1GB of data 
> in the datastore. So more instances might spin up.  Under 2.7 with threading 
> requests we might not see more instance start.  A $ figure to apply to such 
> an appengine app
> would probably be between $30 and $40 per month.
> 
> Please suggest refinements to these models, and additonal detail to go into 
> the spread sheet.
> 
> On the face of it I am not convinced many of these services are significantly 
> cheaper than appengine especially when you take into account most of them
> require you to manage the complete stack.
> 
> Hope this helps focus the discussion and provide some reality checks.
> 
> I personally have no plans to move off appengine.  But due plan to do some 
> tuning.
> 
> Regards
> 
> Tim
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/google-appengine/-/khEdCXoWqpsJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

<>

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

Also: http://www.hetzner.de/en/hosting/produkte_vserver/vq7

That's 9.20 USD for those of us who don't need to pay VAT.

http://www.hetzner.de/en/hosting/produkte_vserver/vq12 - this one is 15 USD 
w/o VAT.

http://www.hetzner.de/en/hosting/produkte_vserver/vq19 - 23 USD

-Sergey

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/tN9uSf9GP9UJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Small enty level VPS and cloud based services v appengine

Hi Joshua

Thanks for your input  I feel I can get my small instances down to the 
$9-$15 per month territory too.
But I wanted to not have to argue that point too much here ;-)

Anyone running a small instance that is well optimised should have a good 
handle on what it will cost them. 

So lets restate - For purposes of comparison the approximate cost for an 
appengine small app is  going to be 
anywhere from $0 (below free threshold) up to say $30 and the sweet spot 
which should be attainable by 
most apps in the category is going to be in the $9-$15 per month territory

Hows that sound ?

Regrds

Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/JeMUJBSoW6oJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

Joshua is correct with some tuning you can improve your instance count
quite well. I have set Max Idle Instances to 2 and kept Min Pending
latency at auto and I am seeing some good results.

However, I still think there is needed some competition here in the US
market. Traffic is way cheaper in Europe than here. If you take a look
at the offer from Hetzner for 15$/month you'll get 2TB of traffic
included. With the new GAE pricing this could cost you about 300$
here. The OVH offer for 8€ even includes a 100MBits flatrate.

On Sep 4, 4:22 pm, Tim Hoffman  wrote:
> Hi Joshua
>
> Thanks for your input  I feel I can get my small instances down to the
> $9-$15 per month territory too.
> But I wanted to not have to argue that point too much here ;-)
>
> Anyone running a small instance that is well optimised should have a good
> handle on what it will cost them.
>
> So lets restate - For purposes of comparison the approximate cost for an
> appengine small app is  going to be
> anywhere from $0 (below free threshold) up to say $30 and the sweet spot
> which should be attainable by
> most apps in the category is going to be in the $9-$15 per month territory
>
> Hows that sound ?
>
> Regrds
>
> Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: urlfetch stops working at times (DownloadError - ApplicationError 2), but keeps working from dev server

2011-09-04 Thread nischalshetty

Can you post the URL that you are trying to fetch from? 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/fyjn7OJ_0HIJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Small enty level VPS and cloud based services v appengine

Amazingly enough AWS still prices their bandwidth from EU datacenter
at $0.120 per GB

On 4 September 2011 17:33, Philip  wrote:
> Joshua is correct with some tuning you can improve your instance count
> quite well. I have set Max Idle Instances to 2 and kept Min Pending
> latency at auto and I am seeing some good results.
>
> However, I still think there is needed some competition here in the US
> market. Traffic is way cheaper in Europe than here. If you take a look
> at the offer from Hetzner for 15$/month you'll get 2TB of traffic
> included. With the new GAE pricing this could cost you about 300$
> here. The OVH offer for 8€ even includes a 100MBits flatrate.
>
> On Sep 4, 4:22 pm, Tim Hoffman  wrote:
>> Hi Joshua
>>
>> Thanks for your input  I feel I can get my small instances down to the
>> $9-$15 per month territory too.
>> But I wanted to not have to argue that point too much here ;-)
>>
>> Anyone running a small instance that is well optimised should have a good
>> handle on what it will cost them.
>>
>> So lets restate - For purposes of comparison the approximate cost for an
>> appengine small app is  going to be
>> anywhere from $0 (below free threshold) up to say $30 and the sweet spot
>> which should be attainable by
>> most apps in the category is going to be in the $9-$15 per month territory
>>
>> Hows that sound ?
>>
>> Regrds
>>
>> Tim
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>



-- 
http://self.maluke.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The Amazing Story Of Appengine And The Two Orders Of Magnitude

2011-09-04 Thread Emlyn

Here's the next post, showing the results of just changing the
performance sliders. Again, hard data. graphs, all the good stuff.

http://point7.wordpress.com/2011/09/04/appengine-tuning-1/

On 3 September 2011 19:46, Emlyn  wrote:
> Hi all,
>
> I don't think I've posted here before, but I've been an appengine user
> for a while now (closing on 2 years? Is that even possible?). And like
> many, I had a rude shock with the new pricing (going from $0.50/day to
> $50/day).
>
> However, I dug into what I'm actually being charged for, and I think
> it's all actually in my control to sort out, and that in itself is
> sort of fascinating. I wrote a long blog post on this, which people
> might find interesting.
>
> The Amazing Story Of Appengine And The Two Orders Of Magnitude
> http://point7.wordpress.com/2011/09/03/the-amazing-story-of-appengine-and-the-two-orders-of-magnitude/
>
> I'd be really grateful for feedback, especially if I've gotten
> anything wildly wrong. I haven't actually made any of the changes that
> I've foreshadowed in the post, that's for the next day or two, and
> I'll write a followup article on how it goes.
>
> Thanks in advance for having a look!
>
> --
> Emlyn
>
> http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
> Buzz posts,
> comments and all.
> http://point7.wordpress.com - My blog
> Find me on Facebook and Buzz
>



-- 
Emlyn

http://my.syyn.cc - Synchonise Google+, Facebook, WordPress and Google
Buzz posts,
comments and all.
http://point7.wordpress.com - My blog
Find me on Facebook and Buzz

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

Let me know if you would like to help collate this information, and I can 
give you write permissions on this
spreadsheet.

Hopefully this is a useful excercise.

Also I need to look at what the minimum spec for a typhoonae or appscale 
deployment would be, if someone wanted to move
off appengine but try and not chnage their code in any significant way. 
 Anyone got any details there?  

Regards 

Tim


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/0S2nqXXLG0UJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Small enty level VPS and cloud based services v appengine

2011-09-04 Thread Barry Hunter

OT, but probably because if they made it much cheaper, it would be
huge lure for US customers (not worried about latency) to use the EU
data center.

But even for EU bound traffic, they probably route it to US
datacenters first then route it via internal networks to the EU.
Mainly for quality of service reasons, they have more control over
their internal network (even it it works via public internet) than
pure public internet.

So they would be paying US 'entry/exit' rates, but only charging EU rates.

On Sun, Sep 4, 2011 at 3:38 PM, Sergey Schetinin  wrote:
> Amazingly enough AWS still prices their bandwidth from EU datacenter
> at $0.120 per GB
>
> On 4 September 2011 17:33, Philip  wrote:
>> Joshua is correct with some tuning you can improve your instance count
>> quite well. I have set Max Idle Instances to 2 and kept Min Pending
>> latency at auto and I am seeing some good results.
>>
>> However, I still think there is needed some competition here in the US
>> market. Traffic is way cheaper in Europe than here. If you take a look
>> at the offer from Hetzner for 15$/month you'll get 2TB of traffic
>> included. With the new GAE pricing this could cost you about 300$
>> here. The OVH offer for 8€ even includes a 100MBits flatrate.
>>
>> On Sep 4, 4:22 pm, Tim Hoffman  wrote:
>>> Hi Joshua
>>>
>>> Thanks for your input  I feel I can get my small instances down to the
>>> $9-$15 per month territory too.
>>> But I wanted to not have to argue that point too much here ;-)
>>>
>>> Anyone running a small instance that is well optimised should have a good
>>> handle on what it will cost them.
>>>
>>> So lets restate - For purposes of comparison the approximate cost for an
>>> appengine small app is  going to be
>>> anywhere from $0 (below free threshold) up to say $30 and the sweet spot
>>> which should be attainable by
>>> most apps in the category is going to be in the $9-$15 per month territory
>>>
>>> Hows that sound ?
>>>
>>> Regrds
>>>
>>> Tim
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to 
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
>
>
> --
> http://self.maluke.com/
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Small enty level VPS and cloud based services v appengine

AWS doesn't route EU traffic via US, that would be nuts. And the EU
customers would be outraged too.

On 4 September 2011 17:53, Barry Hunter  wrote:
> OT, but probably because if they made it much cheaper, it would be
> huge lure for US customers (not worried about latency) to use the EU
> data center.
>
> But even for EU bound traffic, they probably route it to US
> datacenters first then route it via internal networks to the EU.
> Mainly for quality of service reasons, they have more control over
> their internal network (even it it works via public internet) than
> pure public internet.
>
> So they would be paying US 'entry/exit' rates, but only charging EU rates.
>
> On Sun, Sep 4, 2011 at 3:38 PM, Sergey Schetinin  wrote:
>> Amazingly enough AWS still prices their bandwidth from EU datacenter
>> at $0.120 per GB
>>
>> On 4 September 2011 17:33, Philip  wrote:
>>> Joshua is correct with some tuning you can improve your instance count
>>> quite well. I have set Max Idle Instances to 2 and kept Min Pending
>>> latency at auto and I am seeing some good results.
>>>
>>> However, I still think there is needed some competition here in the US
>>> market. Traffic is way cheaper in Europe than here. If you take a look
>>> at the offer from Hetzner for 15$/month you'll get 2TB of traffic
>>> included. With the new GAE pricing this could cost you about 300$
>>> here. The OVH offer for 8€ even includes a 100MBits flatrate.
>>>
>>> On Sep 4, 4:22 pm, Tim Hoffman  wrote:
 Hi Joshua

 Thanks for your input  I feel I can get my small instances down to the
 $9-$15 per month territory too.
 But I wanted to not have to argue that point too much here ;-)

 Anyone running a small instance that is well optimised should have a good
 handle on what it will cost them.

 So lets restate - For purposes of comparison the approximate cost for an
 appengine small app is  going to be
 anywhere from $0 (below free threshold) up to say $30 and the sweet spot
 which should be attainable by
 most apps in the category is going to be in the $9-$15 per month territory

 Hows that sound ?

 Regrds

 Tim
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appengine@googlegroups.com.
>>> To unsubscribe from this group, send email to 
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at 
>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>
>>
>>
>> --
>> http://self.maluke.com/
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to 
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>



-- 
http://self.maluke.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

@Zutesmog
Why did you left IRC? I think its better if I send you my remarks to
the list there instead in the groups ;-)

Microsoft: The price is only correct for September. In October there
will be a 20% price cut for the smallest instance. Input bandwidth is
free and outgoing bandwidth depends on the region where your instance
is deployed: North America and Europe regions: $0.15 per GB out / Asia
Pacific Region: $0.20 per GB out

On Sep 4, 5:01 pm, Sergey Schetinin  wrote:
> AWS doesn't route EU traffic via US, that would be nuts. And the EU
> customers would be outraged too.
>
> On 4 September 2011 17:53, Barry Hunter  wrote:
>
>
>
>
>
>
>
>
>
> > OT, but probably because if they made it much cheaper, it would be
> > huge lure for US customers (not worried about latency) to use the EU
> > data center.
>
> > But even for EU bound traffic, they probably route it to US
> > datacenters first then route it via internal networks to the EU.
> > Mainly for quality of service reasons, they have more control over
> > their internal network (even it it works via public internet) than
> > pure public internet.
>
> > So they would be paying US 'entry/exit' rates, but only charging EU rates.
>
> > On Sun, Sep 4, 2011 at 3:38 PM, Sergey Schetinin  wrote:
> >> Amazingly enough AWS still prices their bandwidth from EU datacenter
> >> at $0.120 per GB
>
> >> On 4 September 2011 17:33, Philip  wrote:
> >>> Joshua is correct with some tuning you can improve your instance count
> >>> quite well. I have set Max Idle Instances to 2 and kept Min Pending
> >>> latency at auto and I am seeing some good results.
>
> >>> However, I still think there is needed some competition here in the US
> >>> market. Traffic is way cheaper in Europe than here. If you take a look
> >>> at the offer from Hetzner for 15$/month you'll get 2TB of traffic
> >>> included. With the new GAE pricing this could cost you about 300$
> >>> here. The OVH offer for 8€ even includes a 100MBits flatrate.
>
> >>> On Sep 4, 4:22 pm, Tim Hoffman  wrote:
>  Hi Joshua
>
>  Thanks for your input  I feel I can get my small instances down to the
>  $9-$15 per month territory too.
>  But I wanted to not have to argue that point too much here ;-)
>
>  Anyone running a small instance that is well optimised should have a good
>  handle on what it will cost them.
>
>  So lets restate - For purposes of comparison the approximate cost for an
>  appengine small app is  going to be
>  anywhere from $0 (below free threshold) up to say $30 and the sweet spot
>  which should be attainable by
>  most apps in the category is going to be in the $9-$15 per month 
>  territory
>
>  Hows that sound ?
>
>  Regrds
>
>  Tim
>
> >>> --
> >>> You received this message because you are subscribed to the Google Groups 
> >>> "Google App Engine" group.
> >>> To post to this group, send email to google-appengine@googlegroups.com.
> >>> To unsubscribe from this group, send email to 
> >>> google-appengine+unsubscr...@googlegroups.com.
> >>> For more options, visit this group 
> >>> athttp://groups.google.com/group/google-appengine?hl=en.
>
> >> --
> >>http://self.maluke.com/
>
> >> --
> >> You received this message because you are subscribed to the Google Groups 
> >> "Google App Engine" group.
> >> To post to this group, send email to google-appengine@googlegroups.com.
> >> To unsubscribe from this group, send email to 
> >> google-appengine+unsubscr...@googlegroups.com.
> >> For more options, visit this group 
> >> athttp://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to 
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group 
> > athttp://groups.google.com/group/google-appengine?hl=en.
>
> --http://self.maluke.com/

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Another HR Refactoring Issue - When to clear a cache?

My monkeypatching solution (see my recent post in the -python group), which 
Guido says I shouldn't use, but which is just so darned pretty I can't help it, 
has gotten me through the first challenge of switching to HR, which is dealing 
with google search results containing keys into my old app's data store.

So now I'm looking at the big Kahuna problem of consistency.  Here's my first 
messy challenge there:

My app puts a list of boards on the home page for a town, along with the list 
of meetings.  Generating that list of boards was taking a lot of CPU, but they 
hardly ever change, so I put in a memcache system that built the HTML when it 
wasn't in the cache, and then cached it before serving.  I clear the cache 
whenever the list of boards changes in some way.

Well that ain't gonna work in HR.  It's quite possible that I update a board, 
clear the cache, and someone comes and hits that page before "eventually 
consistent" comes to pass.  So now I've got a cached copy of the stale data.

(Note that I cannot use entity groups to solve this because some boards are 
municipal agencies, and therefore cannot be parented to the town that is 
building its list.  I could parent all boards to some global parent, but, well, 
yuck.)

I have some different ideas about how to fix this, but I'm wondering if anyone 
else who's done the port to HR has come up with a solution they find 
particularly elegant?  I assume this is a pretty common problem, so there must 
be a design pattern out there… somewhere.

Here are my ideas:

- Clear the cache with a periodic task that re-clears it several times.  I'm 
thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16, 32, 64, 
128, 256, 512 seconds, and then pray that we have consistency)

- Checksum the modified or new board, and put that sum into memcache.  When 
generating the new board, confirm that any checksums are good.  This seems more 
deterministic, except I don't trust memache not to squelch the checksum record. 
 So perhaps I should do something in the datastore.  This feels like it's be 
about 10x as much code as the stupid geometric flush.

Any suggestions?

-Joshua

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Sender mail on GAE

2011-09-04 Thread Deepak Singh

Any one can suggest ...

On Sun, Sep 4, 2011 at 4:44 PM, Deepak Singh wrote:

> Hi,
>
> I have an application metasearchprodenv.appspot.com which uses java mail
> service.
>
> I want to create an email address with this domain name and use this
> address as a sender mail for the application.
>
> I could not find any way to create email address and link this to this
> account.
> Anybody can suggest...
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Echo the sentiments of another poster

2011-09-04 Thread John Wheeler

I wanted to put what another poster wrote in another thread because it is so 
dead-on. Google coming out with all these rationales and exclamation marks! 
behind all their "our premium services are not cheap!" means nothing. That's 
not how you advertised it initially. You said we wouldn't have to worry 
about this, and that you're just rationalizing away and arguing with the 
developers who made the platform successful. This is the grossest example of 
vendor-lock in. If your answer if for us to rejig our apps in two weeks. 
Please, be reasonable Google. Some of us have embraced the no hassles 
promise of your platform with more than 1 app. We don't have the resources 
to meet with your ridiculous time tables across multiple apps.

"What we tried is not exactly what we are going to be buying. Nobody had a 
clue what the real prices would look like and earlier speculations were shot 
down as ridiculous projections. 

What we tried was a platform where I dont have to worry about Instances and 
tuning the scheduler for costs. Tell me with a straight face that this is 
the same platform." 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/hK0_BckO2KsJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

I wrote about the 3 years in another thread. My speculation is that this is 
a slow kill, because they can't make the kill obvious since media then would 
report a lot about how Google has failed with yet another product, this time 
in the important cloud computing space. And many existing customers would 
get outraged if they killed GAE too soon. So they let GAE die slowly by 
outrageous pricing and dwindling investment in the product, and instead they 
focus on cloud computing on higher abstraction levels such as Google Apps 
and Docs.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/XrNSPe7C1r4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Echo the sentiments of another poster

*"What we tried was a platform where I dont have to worry about Instances 
and tuning the scheduler for costs."* -- very interesting quote

That's exactly what I have been writing in other threads. Google should work 
on the instances and the scheduler under the hood, and not use it to rip off 
the customers as other cloud service providers likely do. When big media 
starts to learn that cloud service companies have been ripping off their 
customers, then they will be marked as too greedy for the 21th century 
markets, resulting in a bad goodwill for those companies.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/emE9T6qzQG4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: The Amazing Story Of Appengine And The Two Orders Of Magnitude

2011-09-04 Thread GAEfan

+1

Also, the ability to make a spider bot wait, without spawning a new
task.  Although that may hurt search engine placement.

It is a conflict of interest for google to charge for instances, and
then send their bots around to run up your costs.

Google should be able to add some logic to check if the new instance
were caused by googlebot or the task queue, and make that instance
free.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] 50$ credit, is it per app or per account?

Thanks

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

2011-09-04 Thread Strom

Regarding available memory, memcache is free in GAE.

On Sep 4, 4:26 pm, Tim Hoffman  wrote:
> Hi
>
> More than a few people have said in the groups lately that appengine is
> unsuitable for entry level apps due to the new pricing schedule.
> I am not so sure, but there hasn't been any real information about the
> alternatives, so I thought I would start to collate some numbers
>
> So to that end I have included a spreadsheet here with a summary of a number
> of VPS or cloud providers solutions that I would consider
> might be suitable to run a small entry level appengine app.
>
> For the sake of the discussion you would want to run a stack that looks like
> the following
>
> linux
> nginx/apache
> a light weight stack say webapp2, pyramid, tipfy (a lightweight framework)
> an ORM (sqlobject/Storm)
> and mysql
>
> This doesn't really equate to a heroku offering, but lets say in each case
> we need a single instance of something running, 512MB at a minimum to run
> the small stack and an RDBMS , with at least 1GB of
> storage available if no OS is factored in and 5GB if the OS counts in the
> storage allocation.
>
> So here is a
> spreadsheet.https://docs.google.com/spreadsheet/ccc?key=0At1LTa6ONStgdExuRUl2QUdO...
>
> I know none of these compare service wise directly with appengine.  But
> these are the sort of services people name
> frequently as viable alternatives that are cheaper than appengine. So lets
> look at the reallity
>
> The equivalent appengine basic service would be a single permanently idle
> instance running 24 hours a day, plus low volume of traffic and < 1GB of
> data in the datastore. So more instances might spin up.  Under 2.7 with
> threading requests we might not see more instance start.  A $ figure to
> apply to such an appengine app
> would probably be between $30 and $40 per month.
>
> Please suggest refinements to these models, and additonal detail to go into
> the spread sheet.
>
> On the face of it I am not convinced many of these services are
> significantly cheaper than appengine especially when you take into account
> most of them
> require you to manage the complete stack.
>
> Hope this helps focus the discussion and provide some reality checks.
>
> I personally have no plans to move off appengine.  But due plan to do some
> tuning.
>
> Regards
>
> Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Echo the sentiments of another poster

2011-09-04 Thread Strom

2 weeks? Ridiculous time tables?

Dude, the new pricing was announced in the beginning of May. You can
only blame yourself for waiting 4 months with the optimization.

On Sep 4, 7:22 pm, John Wheeler  wrote:
> I wanted to put what another poster wrote in another thread because it is so
> dead-on. Google coming out with all these rationales and exclamation marks!
> behind all their "our premium services are not cheap!" means nothing. That's
> not how you advertised it initially. You said we wouldn't have to worry
> about this, and that you're just rationalizing away and arguing with the
> developers who made the platform successful. This is the grossest example of
> vendor-lock in. If your answer if for us to rejig our apps in two weeks.
> Please, be reasonable Google. Some of us have embraced the no hassles
> promise of your platform with more than 1 app. We don't have the resources
> to meet with your ridiculous time tables across multiple apps.
>
> "What we tried is not exactly what we are going to be buying. Nobody had a
> clue what the real prices would look like and earlier speculations were shot
> down as ridiculous projections.
>
> What we tried was a platform where I dont have to worry about Instances and
> tuning the scheduler for costs. Tell me with a straight face that this is
> the same platform."

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Abysmal Query Performance in Backend

2011-09-04 Thread Alfred Fuller

The datastore does not delete things right away. It marks them as delete and
waits for a "compaction" to actually remove the data. If a lot of data is
deleted at the start of a query, the datastore will have to skip all the
deleted rows until it finds the first real entity. This is what is causing
your timeouts.

You can avoid this problem by using cursors to continue work
in subsequent requests. Cursors will cause your query to start at the last
deleted row instead of the start of the query. Another good alternative is
to use the Map framework (which also bookmarks it's progress for
continuation between requests).

On Sun, Sep 4, 2011 at 1:52 AM, Volker Schönefeld <
volker.schoenef...@gmail.com> wrote:

> Oh, I ment the dataset has 100 million entities, so it's far from being
> done. There is still around 850 GiB worth of data that wants to be deleted.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/WPeRpAqa5j8J.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Echo the sentiments of another poster

To be fair we don't know yet what the actual cost will be for frontend 
instances, but if the cost for the frontend instances becomes higher than 
100% of the total cost of the other quotas, then I would start worrying.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/usEusi0Yk_YJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: 回覆：Re: [google-appengine] Data Transfer to AWS

2011-09-04 Thread Robert Kluin

The only problem I see is that you won't be able to get existing data with
'updated=False', the old data won't be indexed. You'll need to just get
everything and maybe skip the stuff you have already pushed.  Otherwise
sounds like the idea might work.




On Sep 4, 2011 6:08 AM, "Natalie"  wrote:
>
> We are in the same situation. How’s everyone’s progress?
>
> We’re planning to migrate to AWS but we have quite a bit of data to move
like you guys. We’re trying to minimize service disruption and keeping our
site online while making the move. Here’s our scheme (would love to hear
suggestions form you guys):
>
> 1) Upload a new app version that adds a Boolean value to every type of
entity in Datastore. Call it “updated”. All call to put() will set this
Boolean to true, and push the put() data to a GAE pull queue.
>
> 2) Use Remote API to batch get all entity with Boolean=false. This will
get any unmodified data from DS. Data that are modified by DS after fetch
can be retrieved from the pull queue later.
>
> 3) Transform the data and push them to AWS.
>
> 4) From AWS, lease the data from pull queue and fill up the database
>
> 5) Modify DNS record to point to the new AWS site while keeping the GAE
app alive until it receives no traffic.
>
>
> We are trying to take advantage of the GAE pull queue’s ability to be
accessed outside of GAE. Do you guy foresee any problem with this scheme?
We’re busy coding this at the moment and would love to hear your input.
Thank you.
>
> Also we are planning to use AWS Elastic Beanstalk to ease Tomcat admin
effort. Anyone could share their experience with this technology?
>
>
> --
> You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
> To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/FWTtjG5urQ0J.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] 50$ credit, is it per app or per account?

2011-09-04 Thread Rohan Chandiramani

Per app.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/k8R2i2KPjKsJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 50$ credit, is it per app or per account?

Thanks
If I get 50 credit and disable billing, so i can enable it few months
from now, will i still keep 50$ credit?

On Sep 4, 2:32 pm, Rohan Chandiramani  wrote:
> Per app.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

2011-09-04 Thread Supercobra Thatbytes

Personally I don't think it is a slow kill. I think cloud computing is a key 
business opportunity for Google and that their GAE product is second to 
none. The Google guys just got the pricing wrong. In fact, if they priced 
right and added few languages like PHP and Ruby and add a SQL database (SQL 
is in testing mode now) there would be not a lot of reasons to use AWS or 
Heroku.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/p48CPFftWeAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Yes, could be that Google will keep investing in GAE, because it could have 
great future potential. Cloud computing WILL become huge, no doubt about 
that. Google making a strong investment in GAE would be really good. I love 
the simplicity and scalability of GAE. Maybe the new pricing is something 
some business managers at Google have come up with, above the heads of the 
developers, lol. :D Or the pricing may even turn out to be fair even though 
it looks a bit suspicious at face value.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Kr_SZ3e0oQ0J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Abysmal Query Performance in Backend

Hey Alfred,

thanks for the reply. I've got a couple more questions regarding the 
compaction, I hope you can answer a few:

- When do those compaction events happen? 
- What is their frequency? 
- How can I tell if one happened?

Thanks!
Volker

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/gOrBlHV1s2EJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Backends CPU Usage

Since running a single B1 backend, the CPU usage of our instance has 
increased by an order of magnitude. Nothing else has changed.

So my question is:

In addition to the explicit backend cost, does the backend cpu usage count 
towards the CPU Time resource on the front page? Or is this "indirect" cost 
because the backend uses the datastore API, and that is billed as CPU time?

Considering that the cost of the CPU time is about tenfold that of the 
backend, this is a nasty hidden cost and I think it should be made more 
explicit.

Cheers,
Volker 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/2xd-P_8N008J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 50$ credit, is it per app or per account?

2011-09-04 Thread jay

If we have changed our setting should we see the $50 in the current
balance?

I tried to claim mine for my three apps a few days ago but see no
change.

neptunespride
blightoftheimmortals
jupitersfolly

I did see somebody else post about this over the weekend but I cant
find that thread (its a little busy in there these days)

On Sep 5, 4:39 am, Andrei  wrote:
> Thanks
> If I get 50 credit and disable billing, so i can enable it few months
> from now, will i still keep 50$ credit?
>
> On Sep 4, 2:32 pm, Rohan Chandiramani  wrote:
>
>
>
>
>
>
>
> > Per app.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Another HR Refactoring Issue - When to clear a cache?

2011-09-04 Thread Tom Phillips

"I clear the cache whenever the list of boards changes in some way"

How about update the cache at that point instead of clearing it?

Need be you could even generate the HTML for the cache update with a
URLFetch to  your UI handler where you include the added/changed board
key(s) as parameters, so they can be gotten with strong consistency on
that request and merge into the query result.

/Tom

On Sep 4, 11:16 am, Joshua Smith  wrote:
> My monkeypatching solution (see my recent post in the -python group), which 
> Guido says I shouldn't use, but which is just so darned pretty I can't help 
> it, has gotten me through the first challenge of switching to HR, which is 
> dealing with google search results containing keys into my old app's data 
> store.
>
> So now I'm looking at the big Kahuna problem of consistency.  Here's my first 
> messy challenge there:
>
> My app puts a list of boards on the home page for a town, along with the list 
> of meetings.  Generating that list of boards was taking a lot of CPU, but 
> they hardly ever change, so I put in a memcache system that built the HTML 
> when it wasn't in the cache, and then cached it before serving.  I clear the 
> cache whenever the list of boards changes in some way.
>
> Well that ain't gonna work in HR.  It's quite possible that I update a board, 
> clear the cache, and someone comes and hits that page before "eventually 
> consistent" comes to pass.  So now I've got a cached copy of the stale data.
>
> (Note that I cannot use entity groups to solve this because some boards are 
> municipal agencies, and therefore cannot be parented to the town that is 
> building its list.  I could parent all boards to some global parent, but, 
> well, yuck.)
>
> I have some different ideas about how to fix this, but I'm wondering if 
> anyone else who's done the port to HR has come up with a solution they find 
> particularly elegant?  I assume this is a pretty common problem, so there 
> must be a design pattern out there… somewhere.
>
> Here are my ideas:
>
> - Clear the cache with a periodic task that re-clears it several times.  I'm 
> thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16, 32, 
> 64, 128, 256, 512 seconds, and then pray that we have consistency)
>
> - Checksum the modified or new board, and put that sum into memcache.  When 
> generating the new board, confirm that any checksums are good.  This seems 
> more deterministic, except I don't trust memache not to squelch the checksum 
> record.  So perhaps I should do something in the datastore.  This feels like 
> it's be about 10x as much code as the stupid geometric flush.
>
> Any suggestions?
>
> -Joshua

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 50$ credit, is it per app or per account?

i changed billing and after few minutes could see 50 credit for each
app

On Sep 4, 5:44 pm, jay  wrote:
> If we have changed our setting should we see the $50 in the current
> balance?
>
> I tried to claim mine for my three apps a few days ago but see no
> change.
>
> neptunespride
> blightoftheimmortals
> jupitersfolly
>
> I did see somebody else post about this over the weekend but I cant
> find that thread (its a little busy in there these days)
>
> On Sep 5, 4:39 am, Andrei  wrote:
>
>
>
>
>
>
>
> > Thanks
> > If I get 50 credit and disable billing, so i can enable it few months
> > from now, will i still keep 50$ credit?
>
> > On Sep 4, 2:32 pm, Rohan Chandiramani  wrote:
>
> > > Per app.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Max request rate for a single Java instance, limited to 1000?

2011-09-04 Thread de Witte

I'm trying optimize our application to the max, so far so good, but...

It appears that the threadpool of the Java instance is limited to 18, for 
which ~10 are used to handle requests.

Our average latency is 100ms. 1 minute / 100ms = 600 requests, x10 threads, 
so a single instance should be able to handle *6000 requests a minute. *

According to the logs the limit is 1000 request a minute. A higher rate will 
trigger a new instance regardless the average latency time. regardless any 
settings.

Is this correct or can we get a higher request rate per minute?

Otherwise there is no *cost *involved reason to optimize if your latency is 
below *600ms*. (Hence, frontend is most of the time waiting (idle) for the 
backend)

Another side effect of this limitation is that the the Min Pending Latency 
setting has no effect.


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/o3uDifrLVaEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

2011-09-04 Thread Tapir

how much memory can you use is not clear. and it is not totally free.
and installing a memache in a ec2 instance needs less than 5 minutes
and will run faster.

On Sep 5, 1:34 am, Strom  wrote:
> Regarding available memory, memcache is free in GAE.
>
> On Sep 4, 4:26 pm, Tim Hoffman  wrote:
>
>
>
>
>
>
>
> > Hi
>
> > More than a few people have said in the groups lately that appengine is
> > unsuitable for entry level apps due to the new pricing schedule.
> > I am not so sure, but there hasn't been any real information about the
> > alternatives, so I thought I would start to collate some numbers
>
> > So to that end I have included a spreadsheet here with a summary of a number
> > of VPS or cloud providers solutions that I would consider
> > might be suitable to run a small entry level appengine app.
>
> > For the sake of the discussion you would want to run a stack that looks like
> > the following
>
> > linux
> > nginx/apache
> > a light weight stack say webapp2, pyramid, tipfy (a lightweight framework)
> > an ORM (sqlobject/Storm)
> > and mysql
>
> > This doesn't really equate to a heroku offering, but lets say in each case
> > we need a single instance of something running, 512MB at a minimum to run
> > the small stack and an RDBMS , with at least 1GB of
> > storage available if no OS is factored in and 5GB if the OS counts in the
> > storage allocation.
>
> > So here is a
> > spreadsheet.https://docs.google.com/spreadsheet/ccc?key=0At1LTa6ONStgdExuRUl2QUdO...
>
> > I know none of these compare service wise directly with appengine.  But
> > these are the sort of services people name
> > frequently as viable alternatives that are cheaper than appengine. So lets
> > look at the reallity
>
> > The equivalent appengine basic service would be a single permanently idle
> > instance running 24 hours a day, plus low volume of traffic and < 1GB of
> > data in the datastore. So more instances might spin up.  Under 2.7 with
> > threading requests we might not see more instance start.  A $ figure to
> > apply to such an appengine app
> > would probably be between $30 and $40 per month.
>
> > Please suggest refinements to these models, and additonal detail to go into
> > the spread sheet.
>
> > On the face of it I am not convinced many of these services are
> > significantly cheaper than appengine especially when you take into account
> > most of them
> > require you to manage the complete stack.
>
> > Hope this helps focus the discussion and provide some reality checks.
>
> > I personally have no plans to move off appengine.  But due plan to do some
> > tuning.
>
> > Regards
>
> > Tim

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

Hi Strom

Yep, though accessing it counts in rpc quotas.
I tried to keep the stack really small ;-)

Honestly you can't compare any VPS with appengine feature wise unless you 
build some really big
stack yourself and you still don't get the seemless scaling. But for really 
small apps the scaling is 
probably irrelevant, so I erred in favour of IAAS type services ;-)  These 
types of apps also don't 
need to worry about load balancers, distributed datastores like cassandra or 
mongodb etc as well.


See ya

T

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Kje0PhSUPxEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 50$ credit, is it per app or per account?

2011-09-04 Thread Tapir

I haven't get the credits yet.
I changed the balance 2 days ago.

On Sep 5, 6:08 am, Andrei  wrote:
> i changed billing and after few minutes could see 50 credit for each
> app
>
> On Sep 4, 5:44 pm, jay  wrote:
>
>
>
>
>
>
>
> > If we have changed our setting should we see the $50 in the current
> > balance?
>
> > I tried to claim mine for my three apps a few days ago but see no
> > change.
>
> > neptunespride
> > blightoftheimmortals
> > jupitersfolly
>
> > I did see somebody else post about this over the weekend but I cant
> > find that thread (its a little busy in there these days)
>
> > On Sep 5, 4:39 am, Andrei  wrote:
>
> > > Thanks
> > > If I get 50 credit and disable billing, so i can enable it few months
> > > from now, will i still keep 50$ credit?
>
> > > On Sep 4, 2:32 pm, Rohan Chandiramani  wrote:
>
> > > > Per app.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Small enty level VPS and cloud based services v appengine

2011-09-04 Thread Barry Hunter

Not all traffic, only traffic to/from US origin.

On Sun, Sep 4, 2011 at 4:01 PM, Sergey Schetinin  wrote:
> AWS doesn't route EU traffic via US, that would be nuts. And the EU
> customers would be outraged too.
>
> On 4 September 2011 17:53, Barry Hunter  wrote:
>> OT, but probably because if they made it much cheaper, it would be
>> huge lure for US customers (not worried about latency) to use the EU
>> data center.
>>
>> But even for EU bound traffic, they probably route it to US
>> datacenters first then route it via internal networks to the EU.
>> Mainly for quality of service reasons, they have more control over
>> their internal network (even it it works via public internet) than
>> pure public internet.
>>
>> So they would be paying US 'entry/exit' rates, but only charging EU rates.
>>
>> On Sun, Sep 4, 2011 at 3:38 PM, Sergey Schetinin  wrote:
>>> Amazingly enough AWS still prices their bandwidth from EU datacenter
>>> at $0.120 per GB
>>>
>>> On 4 September 2011 17:33, Philip  wrote:
 Joshua is correct with some tuning you can improve your instance count
 quite well. I have set Max Idle Instances to 2 and kept Min Pending
 latency at auto and I am seeing some good results.

 However, I still think there is needed some competition here in the US
 market. Traffic is way cheaper in Europe than here. If you take a look
 at the offer from Hetzner for 15$/month you'll get 2TB of traffic
 included. With the new GAE pricing this could cost you about 300$
 here. The OVH offer for 8€ even includes a 100MBits flatrate.

 On Sep 4, 4:22 pm, Tim Hoffman  wrote:
> Hi Joshua
>
> Thanks for your input  I feel I can get my small instances down to the
> $9-$15 per month territory too.
> But I wanted to not have to argue that point too much here ;-)
>
> Anyone running a small instance that is well optimised should have a good
> handle on what it will cost them.
>
> So lets restate - For purposes of comparison the approximate cost for an
> appengine small app is  going to be
> anywhere from $0 (below free threshold) up to say $30 and the sweet spot
> which should be attainable by
> most apps in the category is going to be in the $9-$15 per month territory
>
> Hows that sound ?
>
> Regrds
>
> Tim

 --
 You received this message because you are subscribed to the Google Groups 
 "Google App Engine" group.
 To post to this group, send email to google-appengine@googlegroups.com.
 To unsubscribe from this group, send email to 
 google-appengine+unsubscr...@googlegroups.com.
 For more options, visit this group at 
 http://groups.google.com/group/google-appengine?hl=en.


>>>
>>>
>>>
>>> --
>>> http://self.maluke.com/
>>>
>>> --
>>> You received this message because you are subscribed to the Google Groups 
>>> "Google App Engine" group.
>>> To post to this group, send email to google-appengine@googlegroups.com.
>>> To unsubscribe from this group, send email to 
>>> google-appengine+unsubscr...@googlegroups.com.
>>> For more options, visit this group at 
>>> http://groups.google.com/group/google-appengine?hl=en.
>>>
>>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to 
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
>
>
> --
> http://self.maluke.com/
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Another HR Refactoring Issue - When to clear a cache?

2011-09-04 Thread Nick Johnson

Hi Joshua,

Can you not do a consistent get for the data when you have a cache miss, to
ensure you're fetching the latest copy, and cache that?

Looking at how NDB does its caching may be instructive here.

-Nick Johnson

On Mon, Sep 5, 2011 at 1:16 AM, Joshua Smith wrote:

> My monkeypatching solution (see my recent post in the -python group), which
> Guido says I shouldn't use, but which is just so darned pretty I can't help
> it, has gotten me through the first challenge of switching to HR, which is
> dealing with google search results containing keys into my old app's data
> store.
>
> So now I'm looking at the big Kahuna problem of consistency.  Here's my
> first messy challenge there:
>
> My app puts a list of boards on the home page for a town, along with the
> list of meetings.  Generating that list of boards was taking a lot of CPU,
> but they hardly ever change, so I put in a memcache system that built the
> HTML when it wasn't in the cache, and then cached it before serving.  I
> clear the cache whenever the list of boards changes in some way.
>
> Well that ain't gonna work in HR.  It's quite possible that I update a
> board, clear the cache, and someone comes and hits that page before
> "eventually consistent" comes to pass.  So now I've got a cached copy of the
> stale data.
>
> (Note that I cannot use entity groups to solve this because some boards are
> municipal agencies, and therefore cannot be parented to the town that is
> building its list.  I could parent all boards to some global parent, but,
> well, yuck.)
>
> I have some different ideas about how to fix this, but I'm wondering if
> anyone else who's done the port to HR has come up with a solution they find
> particularly elegant?  I assume this is a pretty common problem, so there
> must be a design pattern out there… somewhere.
>
> Here are my ideas:
>
> - Clear the cache with a periodic task that re-clears it several times.
>  I'm thinking a recurring geometric retry would be prudent (1, 2, 4, 8, 16,
> 32, 64, 128, 256, 512 seconds, and then pray that we have consistency)
>
> - Checksum the modified or new board, and put that sum into memcache.  When
> generating the new board, confirm that any checksums are good.  This seems
> more deterministic, except I don't trust memache not to squelch the checksum
> record.  So perhaps I should do something in the datastore.  This feels like
> it's be about 10x as much code as the stupid geometric flush.
>
> Any suggestions?
>
> -Joshua
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Trying to get idle instances to 0

2011-09-04 Thread de Witte

Another one has been posted here, combined, it would work perfectly.

http://code.google.com/p/googleappengine/issues/detail?id=5755


-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/-mvm1OklE58J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Abysmal Query Performance in Backend

2011-09-04 Thread Alfred Fuller

On Sun, Sep 4, 2011 at 1:31 PM, Volker Schönefeld <
volker.schoenef...@gmail.com> wrote:

> Hey Alfred,
>
> thanks for the reply. I've got a couple more questions regarding the
> compaction, I hope you can answer a few:
>
> - When do those compaction events happen?
>

The process that controls the compaction balances need with many other
considerations. The fact that you have done a lot of deletes should cause it
to compact sooner than it otherwise would have.

- What is their frequency?
>

On the order of days

- How can I tell if one happened?
>

Your query will suddenly start working again.


> Thanks!
> Volker
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/gOrBlHV1s2EJ.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 50$ credit, is it per app or per account?

2011-09-04 Thread Rohan Chandiramani

This is the thread your looking for :

https://groups.google.com/forum/#!topic/google-appengine/HZQoLuAIZMs

this is the original blog post : 

http://googleappengine.blogspot.com/2011/08/50-credit-for-new-billing-signups-and.html

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/9dJ2YVyMq78J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Does datastore pricing prevent pubsub?

2011-09-04 Thread Dennis Peterson

Here's an interesting pair of blog posts covering the new datastore read
costs and some tips on minimizing them:
http://point7.wordpress.com/

http://point7.wordpress.com/2011/09/03/the-amazing-story-of-appengine-and-the-two-orders-of-magnitude/

http://point7.wordpress.com/2011/09/04/appengine-tuning-1/

This
guy's costs went up two orders of magnitude according to comparative
bulling, and datastore reads were a big part of it.


On Fri, Sep 2, 2011 at 4:36 PM, Strom  wrote:

> There's this FAQ
> http://code.google.com/appengine/kb/postpreviewpricing.html
> And then there's also the sample bills under "Billing History" on your
> app dashboard.
>
> On Sep 2, 11:01 pm, Dennis Peterson  wrote:
> > Ahh, good, much better. Thanks.
> >
> > I'm having trouble finding the official final details on the new pricing.
> >
> >
> >
> >
> >
> >
> >
> > On Fri, Sep 2, 2011 at 3:25 PM, Strom  wrote:
> > > Read operations cost $0.07 per 100k ops.
> > > 100 pageviews containing 100 posts would be ((100 * 100) / 10) *
> > > $0.07 = $0.007.
> >
> > > On Sep 2, 4:31 pm, Dennis Peterson  wrote:
> > > > Maybe I should explain...the details don't matter, the key point here
> is
> > > > that the design in the GoogleIO talk involves a separate entity for
> each
> > > > post. Google charges a buck per 10K ops, so if each post returned is
> an
> > > op,
> > > > then a hundred pageviews showing a hundred posts each is a buck.
> >
> > > > That seems pretty outlandish. Is it really correct or am I missing
> > > > something?
> >
> > > > On Thu, Sep 1, 2011 at 1:03 PM, DennisP 
> > > wrote:
> > > > > Last I saw Google was defining each entity returned as an
> operation..
> >
> > > > > If you were building a social network or twitter clone using the
> > > > > techniques Google recommends in this talk:
> >
> > > > >
> http://www.google.com/events/io/2009/sessions/BuildingScalableComplex.
> > > ..
> >
> > > > > ...you could easily pull back a hundred entities each time you pull
> > > > > someone's feed. It'd cost you a buck for every hundred pageviews or
> > > > > so.
> >
> > > > > Do I have that right?
> >
> > > > > --
> > > > > You received this message because you are subscribed to the Google
> > > Groups
> > > > > "Google App Engine" group.
> > > > > To post to this group, send email to
> google-appengine@googlegroups.com
> > > .
> > > > > To unsubscribe from this group, send email to
> > > > > google-appengine+unsubscr...@googlegroups.com.
> > > > > For more options, visit this group at
> > > > >http://groups.google.com/group/google-appengine?hl=en.
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com.
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

There are two points need to mention I think:

- GAE instance serve one (or multiple requests on concurrency) requests at a 
time only
- GAE operations (e.g. datastore) charge extra, which depends on what your 
app does.  Currently under the new pricing, more than 50% of the charge of 
my app is for Datastore write and read. (i.e. more than instance charge)

I know its hard to compare GAE with others just by numbers.  But since you 
are doing it...

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/8NqU3zBXTVgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: 50$ credit, is it per app or per account?

After getting 50 credit can i freeze it by disabling billing, so i can
use it later?

On Sep 4, 8:47 pm, Rohan Chandiramani  wrote:
> This is the thread your looking for :
>
> https://groups.google.com/forum/#!topic/google-appengine/HZQoLuAIZMs
>
> this is the original blog post :
>
> http://googleappengine.blogspot.com/2011/08/50-credit-for-new-billing...

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: GAE pricing is not suited for smaller apps

Yes I agree it definitely can has its future if doing it right.  Just the 
new pricing at this time is just forcing a number of existing heavy users 
out of the game and never look back.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Qi12WmZIg3kJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Small enty level VPS and cloud based services v appengine

2011-09-04 Thread Adam Sah

I added two linux VPS services to the spreadsheet:
 - linode.com, which has been great for me over the years-- very reliable, 
solid tech support, etc.
 - intovps.com, which is 1/2 the price of linode (!!).  I just signed up and 
it seems to work as advertised.

I'd be **very** interested to see someone setup TyphoonAE on a VPS and 
compare true cost-- it would be hilarious if TyphoonAE+VPS beat GAE native.

hope this helps,
adam

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/TXgxhZQWNVwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: 回覆：Re: [google-appengine] Data Transfer to AWS