Re: [google-appengine] Re: changes does not reflect

[Vik]

Jeff

I have mentioned that but let me say again

We have only one version deployed and that is the default version. Please
see below. The behavior is quite odd . And not sure why it does not work on
GAE when it works fine on local.

 3 
instances
 | 105.50 MBytes | java | api_version: 1.0 *Yes* 0:09:03 ago by
vik@gmail.com

Thankx and Regards

Vik
Founder
http://www.sakshum.org
http://blog.sakshum.org


On Fri, Aug 24, 2012 at 10:35 PM, Jeff Schnitzer wrote:

> You don't seem to respond to any of the comments regarding the default
> version.
>
> In the Versions page of the admin console, make sure you have the
> default version set properly.
>
> Jeff
>
> On Fri, Aug 24, 2012 at 1:59 AM, Vik  wrote:
> > Hello
> >
> > I updated the problem with more analysis at
> >
> http://stackoverflow.com/questions/12103783/issues-in-version-uploaded-on-google-app-engine
> >
> >
> > Thankx and Regards
> >
> > Vik
> > Founder
> > http://www.sakshum.org
> > http://blog.sakshum.org
> >
> >
> > On Thu, Aug 23, 2012 at 9:15 PM, Vik  wrote:
> >>
> >> We have a GAE for java app using gwt
> >>
> >> I can run the app and test it locally all works good. But on
> deployoment i
> >> do not see the changes. i tried clearing browser cache , different
> browsers,
> >> private mode but no luck.
> >> I can verify in the versions that the deployment did happen properly.
> >>
> >> Any idea on how to debug what is the issue
> >>
> >> Thankx and Regards
> >>
> >> Vik
> >> Founder
> >> http://www.sakshum.org
> >> http://blog.sakshum.org
> >
> >
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Alternate to Conversion API

[Richard Watson]

Yup, here's an example of how to create a doc on Docs and export it as a 
PDF:
http://www.streamhead.com/creating-pdf-file-google-app-engine/ 

I'm almost 100% sure that functionality is safe as it's being used by many 
more people. If you try it out, please tell us how you find it?

On Saturday, August 25, 2012 7:15:57 AM UTC+2, timh wrote:
>
> The conversion api seems to map to functionality in google docs.
>
> Could the facilities in google docs for upload/download and conversion be 
> used ? ( Possibly not depending on what your app does.)
> Or are they being deprecated as well ?
>
> T
>
> On Tuesday, August 21, 2012 7:00:57 PM UTC+8, aswath wrote:
>>
>> Hello,
>> We were deeply involved in utilizing the conversion api for the HTML to 
>> PDF conversion.  Suddenly, I got the email from Google about the plan for 
>> decommissioning from Nov 2012.
>>
>> Does anyone has suggestions for doing the HTML to PDF conversion that is 
>> compatible with Google Appengine for Java.  
>>
>>
>> Regards
>> -Aswath
>> www.AccountingGuru.in
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/sD-nT6wLYGEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Sudden slowdown of files API

[Jeff Schnitzer]

I don't have any official insight, but the Files API is pretty core to
appengine - it ties together GAE, Cloud Storage, and the blobstore.
Google keeps enhancing the Files API, so I would be utterly shocked if
they deprecated it.  It addresses a very concrete need; without it the
blobstore is almost useless.

Google tends to keep APIs in "experimental" phase as long as possible
(hell, look at the task queue api) even if they are essential tools.
I wouldn't worry about the Files API going away - although as an
experimental API, it might change in backwards-incompatible ways.  If
there is something wrong with it, the right thing to do is create an
issue at http://code.google.com/p/googleappengine/issues/list.

The APIs you have to worry about are 1) the ones that don't get much
usage, like the Conversion API and 2) the ones that Google hasn't
figured out how much to charge for yet, like the fulltext Search API.
#1 may just disappear; #2 may end up with absurd prices (ie, backends
and ssl).

Jeff

On Thu, Aug 23, 2012 at 9:25 AM, tempy  wrote:
> Hi all,
>
> Although I'm not sure if the two are connected, after updating to the
> 1.7 SDK, blobstore writes using the Files API have become very slow on
> finalization, frequently failing completely with an IOException when
> the blobs get to around 2 megabytes. These requests fail after running
> for 30 seconds. I've filed a production issue here:
> http://code.google.com/p/googleappengine/issues/detail?id=7992
>
> I wonder if anyone else has seen such a slowdown.
>
> Also - this leads me to a question about the File API in general. I
> know Google doesn't comment on feature timing issues, but just in case
> anyone has any insight... is it a bad idea to rely on the File API?
> Its been experimental for such a long time that I'm worried that it
> will eventually be eliminated, just because it hasn't been promoted
> already. Its also the weakest link in my app - it produces a lot of
> random errors, more than any other piece of GAE.
>
> If relying on the Files API is a bad idea, how would you guys
> accomplish the following: I'm using GAE as a sync-backend for a mobile
> app. The mobile app receives deltas from the GAE app, and these deltas
> are pre-generated. Generating them on demand could possibly take too
> long, thus the offline pre-generation. The deltas could be larger than
> the datastore entity-size limit, which is why I'm storing them as
> blobs - and then serving them to the app when they're needed. So, if
> not for the Files API, how would you guys store these deltas? I really
> hope to avoid the complication of using an additional system that's
> external to GAE, especially when the blobstore+Files API seem so
> perfectly suited to the task.
>
> Thanks,
> Mike
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: changes does not reflect

[Jeff Schnitzer]

You don't seem to respond to any of the comments regarding the default version.

In the Versions page of the admin console, make sure you have the
default version set properly.

Jeff

On Fri, Aug 24, 2012 at 1:59 AM, Vik  wrote:
> Hello
>
> I updated the problem with more analysis at
> http://stackoverflow.com/questions/12103783/issues-in-version-uploaded-on-google-app-engine
>
>
> Thankx and Regards
>
> Vik
> Founder
> http://www.sakshum.org
> http://blog.sakshum.org
>
>
> On Thu, Aug 23, 2012 at 9:15 PM, Vik  wrote:
>>
>> We have a GAE for java app using gwt
>>
>> I can run the app and test it locally all works good. But on deployoment i
>> do not see the changes. i tried clearing browser cache , different browsers,
>> private mode but no luck.
>> I can verify in the versions that the deployment did happen properly.
>>
>> Any idea on how to debug what is the issue
>>
>> Thankx and Regards
>>
>> Vik
>> Founder
>> http://www.sakshum.org
>> http://blog.sakshum.org
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Alternate to Conversion API

[timh]

The conversion api seems to map to functionality in google docs.

Could the facilities in google docs for upload/download and conversion be 
used ? ( Possibly not depending on what your app does.)
Or are they being deprecated as well ?

T

On Tuesday, August 21, 2012 7:00:57 PM UTC+8, aswath wrote:
>
> Hello,
> We were deeply involved in utilizing the conversion api for the HTML to 
> PDF conversion.  Suddenly, I got the email from Google about the plan for 
> decommissioning from Nov 2012.
>
> Does anyone has suggestions for doing the HTML to PDF conversion that is 
> compatible with Google Appengine for Java.  
>
>
> Regards
> -Aswath
> www.AccountingGuru.in
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/aEoZPxDAwFkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Kristopher Giesing]

>
> Like Jon explained in the post I linked, the scheduler will favor routing 
>> traffic to idle dynamic instance rather than idle reserved idle instance 
>> and it will always try to maintain the invariant of N x Min-Idle-Instances 
>> by starting new instance if the reserved instances are busy.
>>
> PS. The behavior described above is not really the problem IMHO, the 
problem is that the scheduler favors routing traffic to NONEXISTENT dynamic 
instances rather than idle reserved instances.  No one seems to understand 
why that would ever be a good idea.

- Kris

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/y6iVMtCCPKgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Kristopher Giesing]

On Friday, August 24, 2012 2:59:11 PM UTC-7, Johan Euphrosine (Google) 
wrote:
>
>
> On Aug 24, 2012 11:28 PM, "Mos" > 
> wrote:
> >
> > Thanks Johan. I read the post some days before.
> >
> > As often discussed on the mailing-list before and as Jeff said in this 
> thread.
> > It's the combination of "Requests should never be sent to cold 
> instances." 
>
> Please star this existing feature request:
> http://code.google.com/p/googleappengine/issues/detail?id=7865
>
Done.
 

> > and(!) the behavior of min idle instance which doesn't make any sense.
>
> Like Jon explained in the post I linked, the scheduler will favor routing 
> traffic to idle dynamic instance rather than idle reserved idle instance 
> and it will always try to maintain the invariant of N x Min-Idle-Instances 
> by starting new instance if the reserved instances are busy.
>
OK, the post by Jon was an interesting read because it explains why Google 
seems to think everything is working as intended.  What doesn't seem to be 
penetrating is that it doesn't matter what some definition on a piece of 
paper somewhere says the system is supposed to do, if that definition 
doesn't actually help developers build good products.

The feature starred above absolutely needs to be implemented.  I just wish 
there was an easier way of getting customers who are frustrated by the 
instancing behavior to focus on that one feature request, because the naive 
interpretation of the existing GAE tuning parameters suggests it shouldn't 
be necessary.

- Kris

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/w3m3ZmnH18cJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] delete Master/Slave application after migration without deleting data

[Chris Ramsdale]

Glad you made the move.  Mind pinging me off thread with the app ID so that
we can dig in further?

-- Chris

Product Manager, Google App Engine


On Fri, Aug 24, 2012 at 4:20 PM, c h  wrote:

> So i finally finished my migration to HIgh Replication, and i want to
> delete my old app so that i don't have to pay for it.  I disabled billing,
> but faster that you can say "i don't want this anymore" i have a payment
> due and can't delete the app.  the payment due is for storage of data that
> i just copied to HRD, and obviously don't need - i also don't want to pay
> hundreds to run the bulk delete tool.  Is there anything i can do, or
> should i just let the app languish with an ever-increasing balance due?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/QQaeRCr179IJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: The cronjobs keep coming every hour.

[Takashi Matsuo]

Can you file a production issue with your app-ids?


On Sat, Aug 25, 2012 at 11:55 AM, Mahron  wrote:

> Same here. every 60 minutes was running every minute. Depleted my
> frontend quota.
> Resolved the issue by changing to 55 minutes.
> Serious bug, should be fixed.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>


-- 
Takashi Matsuo | Developers Advocate | tmat...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Takashi Matsuo]

On Sat, Aug 25, 2012 at 5:24 AM, Mos  wrote:

>  > Setting Max Pending Latency doesn't force requests to be in the pending
> queue for the specified time. Please use Min Pending Latency instead.
>
> As you know my setting to "Min Pending Latency" was automatic. The
> expectation is that GAE takes a reasonable default latency if it is
> "automatic".
> And you say:  Every parallel request starts a new instance if it is
> "automatic"? That' would be a "Min Pending Latency" of zero and not
> "automatic".
>
> > If it doesn't work, try 2 min idle instances then
>
> Please check the responses of other user in this thread.  This feature is
> totally broken and can not be used.


>
> >> And around the 16th august?
>  > Sigh... isn't it a waist of time? What is the reason you picked that
> date?
>
> Did you see/studied my pictures from the first post of this thread?
> The statistic shows that on this date the instance creation gets crazy.  I
> double checked it with the Pingdom reports.
> Starting on this day there were even more downtimes.
>
> > So I'd say please try 2. If you still saw the user-facing loading
> requests, you need more resident instance to eliminate the user-facing
> loading requests.
>
> Again: As wrote in my post before that does not work. Check the responses
> from Kristopher and Jeff on this thread.
>
>
Yeah, it's very nice to hear concrete examples from Kristopher and Jeff,
other than just saying "I've tried that, but it didn't work".


>
> > So what is your expected behavior and actual result? Nobody in our
> team can do anything if you just keep saying "the setting that used to work
> doesn't work anymore" without trying mu suggestion.
> > I think my answer is clear at least for some points. 1) You'd better use
> 'min pending latency' instead of 'max pending latency' to prevent new
> instances to spin up as much as possible. 2) If you need longer instance
> lives, set appropriate number of min idle instances.
>
> As I wrote: I tried different settings. As many other people in this group
> as well.
> Me and other people are reporting: The settings are broken!
> It's very easy to reproduce. Please set up an application, send one
> request per minute (or second), configure 1 or 2 or 3 min idle instances
> and check what is happening. You will see that new  instances are started
> although resistant instances are available.
>

It's nice if we have a complete reproducible case. I've just started an
experiment you mentioned. This time, it's just a helloworld application,
and I set 1 min idle instances and 1 minutes cron.

Presumably it will just work fine. Then I will try with slightly different
condition. That way, I hope I can determine what kind of condition could be
the culprit or not. What do you think? Can you provide some simple projects
for that experiment?


> Please take it serious and let somebody of the engineers check this!
>

(I'm one of the engineers btw) A reproducible case is always the best thing
to get engineers' attention.

Regards,

-- Takashi


> Cheers
> Mos
>
>
> On Fri, Aug 24, 2012 at 8:43 PM, Takashi Matsuo wrote:
>
>>
>> Hi Mos,
>>
>> On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:
>>
>>> Hello Takashi,
>>>
>>>
>>> > Actually there were almost 8 requests in a second. So App Engine
>>> likely needed more than one instance at this particular moment.
>>>
>>> I thought this is why GAE has the concept of "pending-latency"  (which
>>> we discussed below).
>>> Meaning:  Incoming requests may wait up to 15 seconds before starting a
>>> new instance. Therefore when 8 requests in one second occur that
>>> should not mean that more instance needs to be started. Especially if
>>> there is no other traffic in this minute, as seen in my example.
>>> Otherwise it would be a very bad implementation:
>>> Starting a new instance means around 30s waiting time.  Serving 8
>>> parallel requests from one instance, would result in a maximum of
>>> 8 seconds for the last request (assuming that each request takes around
>>> 1 second).
>>> There is no reason for this concrete example to fire up more instances
>>> and let requests wait more then 30 seconds until a new instance is loaded.
>>>
>>
>> Do you really read my e-mail?
>>
>> Setting Max Pending Latency doesn't force requests to be in the pending
>> queue for the specified time. Please use Min Pending Latency instead.
>> Can you try this first? If it doesn't work, try 2 min idle instances then.
>>
>>
>>>
>>> > ... here is what you've seen in the past weeks.
>>> >
>>> >* You have been almost always set 'Automatic-2' idle instance setting.
>>> >* More than 3 weeks ago, number of loading requests were very few.
>>> > * Recently you have seen more loading requests than before.
>>>
>>> That, right!  To be even more concrete: At the 16. august the problems
>>> got significant worse. Please check especially the time area from 16.
>>> august until today.
>>>
>>> > First of all, it seems that you deployed 2 new versions on Aug 1 and
>>> Aug 2. Can you

[google-appengine] Re: The cronjobs keep coming every hour.

[Mahron]

Same here. every 60 minutes was running every minute. Depleted my
frontend quota.
Resolved the issue by changing to 55 minutes.
Serious bug, should be fixed.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

RE: [google-appengine] Google Support Request: Can I get more Quota and Apps

[Drake]

Maybe, but I think it takes time for them to become available after enabling
billing.

Because of memory limitation mult-tenant is not an option.

> -Original Message-
> From: google-appengine@googlegroups.com [mailto:google-
> appeng...@googlegroups.com] On Behalf Of Johan Euphrosine
> Sent: Friday, August 24, 2012 4:00 PM
> To: google-appengine@googlegroups.com
> Subject: Re: [google-appengine] Google Support Request: Can I get more
> Quota and Apps
>
> On Fri, Aug 24, 2012 at 11:45 PM, Drake  wrote:
> > Can I get more apps for appsad...@stremor.com I am almost out and will
> > need
> > 5 on Monday which will put me over my limit. And 10 more in 2 weeks.
>
> Are they paid app? AFAIK this limit only applied to free app.
>
> >
> >
> >
> > Also can I get the URL fetch limit taken off of stremor-basher which
> > is meant to be a load tester so I can DDoS myself.
> >
> >
> >
> > Thanks!
> >
> >
> >
> > -Brandon
> >
> >
> >
> > PS
> >
> > www.unpartial.com is coming Monday.
> >
> >
> >
> >
> >
> >
> >
> > --
> > You received this message because you are subscribed to the Google
> > Groups "Google App Engine" group.
> > To post to this group, send email to google-appengine@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group at
> > http://groups.google.com/group/google-appengine?hl=en.
>
>
>
> --
> Johan Euphrosine (proppy)
> Developer Programs Engineer
> Google Developer Relations
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to google-
> appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.



-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Google Support Request: Can I get more Quota and Apps

[Johan Euphrosine]

On Fri, Aug 24, 2012 at 11:45 PM, Drake  wrote:
> Can I get more apps for appsad...@stremor.com I am almost out and will need
> 5 on Monday which will put me over my limit. And 10 more in 2 weeks.

Are they paid app? AFAIK this limit only applied to free app.

>
>
>
> Also can I get the URL fetch limit taken off of stremor-basher which is
> meant to be a load tester so I can DDoS myself.
>
>
>
> Thanks!
>
>
>
> -Brandon
>
>
>
> PS
>
> www.unpartial.com is coming Monday.
>
>
>
>
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.



-- 
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Google Support Request: Can I get more Quota and Apps

[Drake]

Can I get more apps for appsad...@stremor.com I am almost out and will need
5 on Monday which will put me over my limit. And 10 more in 2 weeks. 

 

Also can I get the URL fetch limit taken off of stremor-basher which is
meant to be a load tester so I can DDoS myself.

 

Thanks!

 

-Brandon

 

PS

www.unpartial.com is coming Monday.

 

 

 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: Re: [google-appengine] Re: App Engine Chat Time!

[Johan Euphrosine]

On Wed, Jun 20, 2012 at 11:06 PM, Joshua Bronson  wrote:
> I stopped by #appengine today at 9AM Pacific and someone there said IRC
> office hours were no longer being held. Is there a regular Google Hangout
> time or something to replace them? I found office hours were the only
> reliable place to ping someone at Google on important tickets that had been
> accidentally overlooked.

We regularly handle office hours on Google+ now (every week) and it
has been a long time since we held IRC office hours.

For pinging us about important tickets that we missed, feel free to
create new posts on the groups with a list of issue urls.

You are also welcome to start a new thread about IRC office hours, why
you miss them, how do they compare to Google+ hangout, and if others
member of the community feels the same way.

>
> Thanks,
> Josh
>
>
> On Wednesday, July 6, 2011 3:02:29 PM UTC-3, schuppe wrote:
>>
>> Yes, chat time is still there:
>> """
>> Every first and third Wednesday of the month, the App Engine team hosts
>> IRC Chat Time, an opportunity for you to get answers to your App
>> Engine-related questions in real-time.
>>
>> These chat sessions take place on the #appengine channel on
>> irc.freenode.net. For a list of IRC clients, see
>> http://en.wikipedia.org/wiki/List_of_IRC_clients.
>>
>> We welcome all App Engine-related questions and we will try to answer
>> as many as we can in the hour session.
>>
>> """ -
>> https://www.google.com/calendar/event?eid=YzZscXRsNnJwMHQ4Zjl1dHNncHVrbDd2ZTRfMjAxMTA3MDdUMDIwMDAwWiBkZXZlbG9wZXItY2FsZW5kYXJAZ29vZ2xlLmNvbQ&gsessionid=OK
>>
>> Next one is today.
>>
>>
>> On Mon, Jul 4, 2011 at 6:12 AM, TWiemann  wrote:
>>>
>>> Hi All,
>>>
>>> i've got a short question: does the Chat Time still exist? I'd like to
>>> join you and i hope to find some answers for some questions using Google App
>>> Engine.
>>>
>>> Thanks a lot!
>>>
>>> Theres
>>> Germany
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/5cf9t4HTVzUJ.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.



-- 
Johan Euphrosine (proppy)
Developer Programs Engineer
Google Developer Relations

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Johan Euphrosine]

On Fri, Aug 24, 2012 at 11:12 PM, Armen Danielyan  wrote:
> Hi Mos,
>
> I have experienced very similar issues for the last week. The loading time
> of my website increased from 2-3 seconds to 10-15 seconds all of a sudden.
> When it happened I tried to change the app settings, changed it from F1 to
> F2, increased the idle instances setting but nothing helped.
>
> During the week I was playing with settings with no results. Suddenly the
> issue has been solved by itself today. I want to stress, that yesterday the
> problem was still there, and I haven't changed any settings since then.
>
> It means that the problem was on Google's side, and they solved it silently.
> It's a shame Google doesn't accept their mistakes, and keep saying that it's
> our fault because we didn't configure our applications in a right way. I
> will never deploy any new application on GAE.
>

Hi Armen,

Are you affected by
http://code.google.com/p/googleappengine/issues/detail?id=7706?

The engineering team is working on improving the high performance
variance for apps that need to load a lot of code on loading request
(typically Java apps with "big" dependency like spring, guice or
depending on a lot of jars).

If you app is hit by this problem, please star this issue and comment
with your application id.

Thanks in advance.

>
> On Friday, August 24, 2012 5:28:01 PM UTC-4, Mos wrote:
>>
>> Thanks Johan. I read the post some days before.
>>
>> As often discussed on the mailing-list before and as Jeff said in this
>> thread.
>> It's the combination of "Requests should never be sent to cold instances."
>> and(!) the behavior of min idle instance which doesn't make any sense.
>>
>> Please check the last comment of
>> http://code.google.com/p/googleappengine/issues/detail?id=8004 where wrote
>> down the problems in my point of view.
>>
>> Senior Java-developers on this list which have many months of experience
>> with GAE stated again again that there is a big issue around instance
>> handling.
>> I think you have to trust your power-user and assign a team to work on
>> this!
>>
>> On Fri, Aug 24, 2012 at 10:58 PM, Johan Euphrosine 
>> wrote:
>>>
>>> Hi all,
>>>
>>> Please review the following thread where the lead engineer working on the
>>> scheduler (Jon McAlister) took the time to explain in great detail the
>>> behavior of min idle instance.
>>> https://groups.google.com/d/msg/google-appengine/nRtzGtG9790/hLS16qux_04J
>>>
>>> Once you read this, we can discuss if what you're experiencing is really
>>> a bug, or if you want the scheduler to behave differently from its current
>>> implementation, in which case the more constructive way out of this
>>> discussion is to fill feature request, and get it starred by your peers.
>>>
>>> On Aug 24, 2012 10:24 PM, "Mos"  wrote:

 > Setting Max Pending Latency doesn't force requests to be in the
 > pending queue for the specified time. Please use Min Pending Latency
 > instead.

 As you know my setting to "Min Pending Latency" was automatic. The
 expectation is that GAE takes a reasonable default latency if it is
 "automatic".
 And you say:  Every parallel request starts a new instance if it is
 "automatic"? That' would be a "Min Pending Latency" of zero and not
 "automatic".

 > If it doesn't work, try 2 min idle instances then

 Please check the responses of other user in this thread.  This feature
 is totally broken and can not be used.

 >> And around the 16th august?
 > Sigh... isn't it a waist of time? What is the reason you picked that
 > date?

 Did you see/studied my pictures from the first post of this thread?
 The statistic shows that on this date the instance creation gets crazy.
 I double checked it with the Pingdom reports.
 Starting on this day there were even more downtimes.

 > So I'd say please try 2. If you still saw the user-facing loading
 > requests, you need more resident instance to eliminate the user-facing
 > loading requests.

 Again: As wrote in my post before that does not work. Check the
 responses from Kristopher and Jeff on this thread.

 > So what is your expected behavior and actual result? Nobody in our
 > team can do anything if you just keep saying "the setting that used to 
 > work
 > doesn't work anymore" without trying mu suggestion.
 > I think my answer is clear at least for some points. 1) You'd better
 > use 'min pending latency' instead of 'max pending latency' to prevent new
 > instances to spin up as much as possible. 2) If you need longer instance
 > lives, set appropriate number of min idle instances.

 As I wrote: I tried different settings. As many other people in this
 group as well.
 Me and other people are reporting: The settings are broken!
 It's very easy to reproduce. Please set up an application, send one
 request per minute (or second), configure 1 

[google-appengine] Re: HRD Migration & Blob keys

[Phil McDonnell]

Think I've got it now. No need to reply here.

Thanks,
Phil

On Fri, Aug 24, 2012 at 10:46 AM, Phil McDonnell  wrote:

> Is anyone familiar with this blobstore issue in the HRD migration?
>
> Thanks,
> Phil
>
>
> On Thu, Aug 23, 2012 at 4:39 PM, Phil McDonnell <
> phil.a.mcdonn...@gmail.com> wrote:
>
>> I'm looking at migrating to HRD and I use the blob store.  In my
>> datastore I have one Kind of entity that stores a blob key as one of the
>> fields in the Entity. I see that the HRD migration warns about blob keys
>> stored in "serialized form." Pardon my ignorance, but are my blob keys that
>> I mentioned stored in serialized form? Quotation from the documentation &
>> example of my blob key in the Entity (highlighted yellow) below.
>>
>> Thanks,
>> Phil
>>
>> *Warning from the HRD Migration:*
>>
>> If your application has blob data in Blobstore, you need to migrate that
>> blob data as well. Check the checkbox labeled *Migrate Blobstore Data*.
>> The blobs will be copied and given new blob keys in the new application,
>> and the Datastore will be updated with the new blob keys. (However, any
>> blob keys stored in serialized form within the Datastore will *not* be
>> migrated; see Migrating Serialized Blob 
>> Keys,
>> below, for more information.)
>>
>> *Example of my datastore from the datastore viewer:*
>>   ID/Namecomplete couponKeydate dealIDdealVariant fraudulentnumSoldppemail
>> price purchasesold uidverified address1address2 charitycity fullnamestatezip
>> id=174125
>> True
>> AMIfv95dj6WTPGjP8P38yQrSmjbPs6eqWCFpvafA0Y9KoROcostO8OiHMWOwZKFfqmNwfi2s9pXyK7Bz8cOIG-CSPnv-T9r0ac9Fn_c5carOHrpRBxFz-Pm3zcI4j93-MDHssxDpC6PdnKY5P3Dr66BrGLHWqoBNzQ
>> View 
>> blob
>> 2012-01-23 05:19:04.658000 asdf 0False 1a...@asdf.com 4.0 FalseTrue10709668
>> True 
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Armen Danielyan]

Hi Mos,

I have experienced very similar issues for the last week. The loading time 
of my website increased from 2-3 seconds to 10-15 seconds all of a sudden. 
When it happened I tried to change the app settings, changed it from F1 to 
F2, increased the idle instances setting but nothing helped.

During the week I was playing with settings with no results. Suddenly the 
issue has been solved by itself today. I want to stress, that yesterday the 
problem was still there, and I haven't changed any settings since then. 

It means that the problem was on Google's side, and they solved it 
silently. It's a shame Google doesn't accept their mistakes, and keep 
saying that it's our fault because we didn't configure our applications in 
a right way. I will never deploy any new application on GAE.


On Friday, August 24, 2012 5:28:01 PM UTC-4, Mos wrote:
>
> Thanks Johan. I read the post some days before.
>
> As often discussed on the mailing-list before and as Jeff said in this 
> thread.
> It's the combination of "Requests should never be sent to cold instances." 
> and(!) the behavior of min idle instance which doesn't make any sense.
>
> Please check the last comment of 
> http://code.google.com/p/googleappengine/issues/detail?id=8004 where 
> wrote down the problems in my point of view.
>
> Senior Java-developers on this list which have many months of experience 
> with GAE stated again again that there is a big issue around instance 
> handling.
> I think you have to trust your power-user and assign a team to work on 
> this!
>
> On Fri, Aug 24, 2012 at 10:58 PM, Johan Euphrosine 
> 
> > wrote:
>
>> Hi all,
>>
>> Please review the following thread where the lead engineer working on the 
>> scheduler (Jon McAlister) took the time to explain in great detail the 
>> behavior of min idle instance.
>> https://groups.google.com/d/msg/google-appengine/nRtzGtG9790/hLS16qux_04J
>>
>> Once you read this, we can discuss if what you're experiencing is really 
>> a bug, or if you want the scheduler to behave differently from its current 
>> implementation, in which case the more constructive way out of this 
>> discussion is to fill feature request, and get it starred by your peers.
>>  On Aug 24, 2012 10:24 PM, "Mos" > 
>> wrote:
>>
>>>  > Setting Max Pending Latency doesn't force requests to be in the 
>>> pending queue for the specified time. Please use Min Pending Latency 
>>> instead.
>>>
>>> As you know my setting to "Min Pending Latency" was automatic. The 
>>> expectation is that GAE takes a reasonable default latency if it is 
>>> "automatic".
>>> And you say:  Every parallel request starts a new instance if it is 
>>> "automatic"? That' would be a "Min Pending Latency" of zero and not 
>>> "automatic".
>>>
>>> > If it doesn't work, try 2 min idle instances then
>>>
>>> Please check the responses of other user in this thread.  This feature 
>>> is totally broken and can not be used.
>>>
>>> >> And around the 16th august?  
>>> > Sigh... isn't it a waist of time? What is the reason you picked that 
>>> date? 
>>>
>>> Did you see/studied my pictures from the first post of this thread?
>>> The statistic shows that on this date the instance creation gets crazy.  
>>> I double checked it with the Pingdom reports.
>>> Starting on this day there were even more downtimes.
>>>
>>> > So I'd say please try 2. If you still saw the user-facing loading 
>>> requests, you need more resident instance to eliminate the user-facing 
>>> loading requests.
>>>
>>> Again: As wrote in my post before that does not work. Check the 
>>> responses from Kristopher and Jeff on this thread.
>>>
>>> > So what is your expected behavior and actual result? Nobody in our 
>>> team can do anything if you just keep saying "the setting that used to work 
>>> doesn't work anymore" without trying mu suggestion.
>>> > I think my answer is clear at least for some points. 1) You'd better 
>>> use 'min pending latency' instead of 'max pending latency' to prevent new 
>>> instances to spin up as much as possible. 2) If you need longer instance 
>>> lives, set appropriate number of min idle instances.
>>>
>>> As I wrote: I tried different settings. As many other people in this 
>>> group as well.
>>> Me and other people are reporting: The settings are broken!
>>> It's very easy to reproduce. Please set up an application, send one 
>>> request per minute (or second), configure 1 or 2 or 3 min idle instances 
>>> and check what is happening. You will see that new  instances are started 
>>> although resistant instances are available.
>>>
>>> Please take it serious and let somebody of the engineers check this!
>>>
>>> Cheers
>>> Mos 
>>>
>>>
>>> On Fri, Aug 24, 2012 at 8:43 PM, Takashi Matsuo 
>>> 
>>> > wrote:
>>>

 Hi Mos,

 On Sat, Aug 25, 2012 at 1:39 AM, Mos 
 > wrote:

> Hello Takashi,
>
>
> > Actually there were almost 8 requests in a second. So App Engine 
> likely needed more than one instance at this partic

Re: [google-appengine] Weird Instance Scheduler

[Mos]

Thanks Johan. I read the post some days before.

As often discussed on the mailing-list before and as Jeff said in this
thread.
It's the combination of "Requests should never be sent to cold instances."
and(!) the behavior of min idle instance which doesn't make any sense.

Please check the last comment of
http://code.google.com/p/googleappengine/issues/detail?id=8004 where wrote
down the problems in my point of view.

Senior Java-developers on this list which have many months of experience
with GAE stated again again that there is a big issue around instance
handling.
I think you have to trust your power-user and assign a team to work on this!

On Fri, Aug 24, 2012 at 10:58 PM, Johan Euphrosine wrote:

> Hi all,
>
> Please review the following thread where the lead engineer working on the
> scheduler (Jon McAlister) took the time to explain in great detail the
> behavior of min idle instance.
> https://groups.google.com/d/msg/google-appengine/nRtzGtG9790/hLS16qux_04J
>
> Once you read this, we can discuss if what you're experiencing is really a
> bug, or if you want the scheduler to behave differently from its current
> implementation, in which case the more constructive way out of this
> discussion is to fill feature request, and get it starred by your peers.
> On Aug 24, 2012 10:24 PM, "Mos"  wrote:
>
>>  > Setting Max Pending Latency doesn't force requests to be in the
>> pending queue for the specified time. Please use Min Pending Latency
>> instead.
>>
>> As you know my setting to "Min Pending Latency" was automatic. The
>> expectation is that GAE takes a reasonable default latency if it is
>> "automatic".
>> And you say:  Every parallel request starts a new instance if it is
>> "automatic"? That' would be a "Min Pending Latency" of zero and not
>> "automatic".
>>
>> > If it doesn't work, try 2 min idle instances then
>>
>> Please check the responses of other user in this thread.  This feature is
>> totally broken and can not be used.
>>
>> >> And around the 16th august?
>> > Sigh... isn't it a waist of time? What is the reason you picked that
>> date?
>>
>> Did you see/studied my pictures from the first post of this thread?
>> The statistic shows that on this date the instance creation gets crazy.
>> I double checked it with the Pingdom reports.
>> Starting on this day there were even more downtimes.
>>
>> > So I'd say please try 2. If you still saw the user-facing loading
>> requests, you need more resident instance to eliminate the user-facing
>> loading requests.
>>
>> Again: As wrote in my post before that does not work. Check the responses
>> from Kristopher and Jeff on this thread.
>>
>> > So what is your expected behavior and actual result? Nobody in our
>> team can do anything if you just keep saying "the setting that used to work
>> doesn't work anymore" without trying mu suggestion.
>> > I think my answer is clear at least for some points. 1) You'd better
>> use 'min pending latency' instead of 'max pending latency' to prevent new
>> instances to spin up as much as possible. 2) If you need longer instance
>> lives, set appropriate number of min idle instances.
>>
>> As I wrote: I tried different settings. As many other people in this
>> group as well.
>> Me and other people are reporting: The settings are broken!
>> It's very easy to reproduce. Please set up an application, send one
>> request per minute (or second), configure 1 or 2 or 3 min idle instances
>> and check what is happening. You will see that new  instances are started
>> although resistant instances are available.
>>
>> Please take it serious and let somebody of the engineers check this!
>>
>> Cheers
>> Mos
>>
>>
>> On Fri, Aug 24, 2012 at 8:43 PM, Takashi Matsuo wrote:
>>
>>>
>>> Hi Mos,
>>>
>>> On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:
>>>
 Hello Takashi,


 > Actually there were almost 8 requests in a second. So App Engine
 likely needed more than one instance at this particular moment.

 I thought this is why GAE has the concept of "pending-latency"  (which
 we discussed below).
 Meaning:  Incoming requests may wait up to 15 seconds before starting a
 new instance. Therefore when 8 requests in one second occur that
 should not mean that more instance needs to be started. Especially if
 there is no other traffic in this minute, as seen in my example.
 Otherwise it would be a very bad implementation:
 Starting a new instance means around 30s waiting time.  Serving 8
 parallel requests from one instance, would result in a maximum of
 8 seconds for the last request (assuming that each request takes around
 1 second).
 There is no reason for this concrete example to fire up more instances
 and let requests wait more then 30 seconds until a new instance is loaded.

>>>
>>> Do you really read my e-mail?
>>>
>>> Setting Max Pending Latency doesn't force requests to be in the pending
>>> queue for the specified time. Please use Min Pending Latency 

Re: [google-appengine] after hrd migration app cannot find app data

[Alejandro D. Garin]

Hi Takashi,

so I have this declaration for my ids

  @PrimaryKey

  @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)

  @Extension(vendorName="datanucleus", key="gae.encoded-pk", value="true")

  private String id;


is that no longer supported? I wouldn't migrate if I knew this ! My
application is useless at this time.



On Fri, Aug 24, 2012 at 4:57 PM, Takashi Matsuo  wrote:

>
> I think you're using a serialized form of old keys on the old apps. You
> need to convert those keys by yourself.
>
> Here is an example:
>
> http://stackoverflow.com/questions/9760205/migration-to-hrd-how-to-convert-string-encoded-keys-to-new-application
>
> -- Takashi
>
>
> On Sat, Aug 25, 2012 at 1:44 AM, Alejandro Diego Garin 
> wrote:
>
>> Hi,
>>
>> Anyone could help with this? I just finished the migration I see this
>> error in the logs.
>>
>>
>> unexpected exception: javax.jdo.JDOFatalUserException: Illegal argument
>> NestedThrowables:
>> java.lang.IllegalArgumentException: app s~jardincitosonline-hrd cannot 
>> access app jardincitosonline's data
>>
>>
>> Thanks.
>> Alejandro.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/google-appengine/-/LCDKuH30Mz8J.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
>
>
> --
> Takashi Matsuo | Developers Advocate | tmat...@google.com
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Johan Euphrosine]

Hi all,

Please review the following thread where the lead engineer working on the
scheduler (Jon McAlister) took the time to explain in great detail the
behavior of min idle instance.
https://groups.google.com/d/msg/google-appengine/nRtzGtG9790/hLS16qux_04J

Once you read this, we can discuss if what you're experiencing is really a
bug, or if you want the scheduler to behave differently from its current
implementation, in which case the more constructive way out of this
discussion is to fill feature request, and get it starred by your peers.
On Aug 24, 2012 10:24 PM, "Mos"  wrote:

> > Setting Max Pending Latency doesn't force requests to be in the pending
> queue for the specified time. Please use Min Pending Latency instead.
>
> As you know my setting to "Min Pending Latency" was automatic. The
> expectation is that GAE takes a reasonable default latency if it is
> "automatic".
> And you say:  Every parallel request starts a new instance if it is
> "automatic"? That' would be a "Min Pending Latency" of zero and not
> "automatic".
>
> > If it doesn't work, try 2 min idle instances then
>
> Please check the responses of other user in this thread.  This feature is
> totally broken and can not be used.
>
> >> And around the 16th august?
> > Sigh... isn't it a waist of time? What is the reason you picked that
> date?
>
> Did you see/studied my pictures from the first post of this thread?
> The statistic shows that on this date the instance creation gets crazy.  I
> double checked it with the Pingdom reports.
> Starting on this day there were even more downtimes.
>
> > So I'd say please try 2. If you still saw the user-facing loading
> requests, you need more resident instance to eliminate the user-facing
> loading requests.
>
> Again: As wrote in my post before that does not work. Check the responses
> from Kristopher and Jeff on this thread.
>
> > So what is your expected behavior and actual result? Nobody in our
> team can do anything if you just keep saying "the setting that used to work
> doesn't work anymore" without trying mu suggestion.
> > I think my answer is clear at least for some points. 1) You'd better use
> 'min pending latency' instead of 'max pending latency' to prevent new
> instances to spin up as much as possible. 2) If you need longer instance
> lives, set appropriate number of min idle instances.
>
> As I wrote: I tried different settings. As many other people in this group
> as well.
> Me and other people are reporting: The settings are broken!
> It's very easy to reproduce. Please set up an application, send one
> request per minute (or second), configure 1 or 2 or 3 min idle instances
> and check what is happening. You will see that new  instances are started
> although resistant instances are available.
>
> Please take it serious and let somebody of the engineers check this!
>
> Cheers
> Mos
>
>
> On Fri, Aug 24, 2012 at 8:43 PM, Takashi Matsuo wrote:
>
>>
>> Hi Mos,
>>
>> On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:
>>
>>> Hello Takashi,
>>>
>>>
>>> > Actually there were almost 8 requests in a second. So App Engine
>>> likely needed more than one instance at this particular moment.
>>>
>>> I thought this is why GAE has the concept of "pending-latency"  (which
>>> we discussed below).
>>> Meaning:  Incoming requests may wait up to 15 seconds before starting a
>>> new instance. Therefore when 8 requests in one second occur that
>>> should not mean that more instance needs to be started. Especially if
>>> there is no other traffic in this minute, as seen in my example.
>>> Otherwise it would be a very bad implementation:
>>> Starting a new instance means around 30s waiting time.  Serving 8
>>> parallel requests from one instance, would result in a maximum of
>>> 8 seconds for the last request (assuming that each request takes around
>>> 1 second).
>>> There is no reason for this concrete example to fire up more instances
>>> and let requests wait more then 30 seconds until a new instance is loaded.
>>>
>>
>> Do you really read my e-mail?
>>
>> Setting Max Pending Latency doesn't force requests to be in the pending
>> queue for the specified time. Please use Min Pending Latency instead.
>> Can you try this first? If it doesn't work, try 2 min idle instances then.
>>
>>
>>>
>>> > ... here is what you've seen in the past weeks.
>>> >
>>> >* You have been almost always set 'Automatic-2' idle instance setting.
>>> >* More than 3 weeks ago, number of loading requests were very few.
>>> > * Recently you have seen more loading requests than before.
>>>
>>> That, right!  To be even more concrete: At the 16. august the problems
>>> got significant worse. Please check especially the time area from 16.
>>> august until today.
>>>
>>> > First of all, it seems that you deployed 2 new versions on Aug 1 and
>>> Aug 2. Can you describe what kind of changes in those versions?
>>>
>>> I checked it in our version control. As I wrote no related changes were
>>> made! Just Html/Css stuff:
>>>  * One picture

[google-appengine] Re: Connectivity to Twitter API completely gone for 1 of my apps

[Bryce Cutt]

Related to discussion on Twitter Dev board:
https://dev.twitter.com/discussions/3218


On Friday, August 24, 2012 7:26:34 AM UTC-7, JH wrote:
>
> As of yesterday my app, ticker-app, can no longer connect to Twitter's 
> API.  It has worked fine for over a year.  As of yesterday any urlfetch 
> call to twitter.com times out with DeadlineExceeded.  However, using 
> other app ID's I have they can still urlfetch to twitter.  I have to assume 
> it has something to do with which data center ticker-app is hosted in?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/H2tF0K3pFooJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Mos]

 > Setting Max Pending Latency doesn't force requests to be in the pending
queue for the specified time. Please use Min Pending Latency instead.

As you know my setting to "Min Pending Latency" was automatic. The
expectation is that GAE takes a reasonable default latency if it is
"automatic".
And you say:  Every parallel request starts a new instance if it is
"automatic"? That' would be a "Min Pending Latency" of zero and not
"automatic".

> If it doesn't work, try 2 min idle instances then

Please check the responses of other user in this thread.  This feature is
totally broken and can not be used.

>> And around the 16th august?
> Sigh... isn't it a waist of time? What is the reason you picked that
date?

Did you see/studied my pictures from the first post of this thread?
The statistic shows that on this date the instance creation gets crazy.  I
double checked it with the Pingdom reports.
Starting on this day there were even more downtimes.

> So I'd say please try 2. If you still saw the user-facing loading
requests, you need more resident instance to eliminate the user-facing
loading requests.

Again: As wrote in my post before that does not work. Check the responses
from Kristopher and Jeff on this thread.

> So what is your expected behavior and actual result? Nobody in our
team can do anything if you just keep saying "the setting that used to work
doesn't work anymore" without trying mu suggestion.
> I think my answer is clear at least for some points. 1) You'd better use
'min pending latency' instead of 'max pending latency' to prevent new
instances to spin up as much as possible. 2) If you need longer instance
lives, set appropriate number of min idle instances.

As I wrote: I tried different settings. As many other people in this group
as well.
Me and other people are reporting: The settings are broken!
It's very easy to reproduce. Please set up an application, send one request
per minute (or second), configure 1 or 2 or 3 min idle instances and check
what is happening. You will see that new  instances are started although
resistant instances are available.

Please take it serious and let somebody of the engineers check this!

Cheers
Mos


On Fri, Aug 24, 2012 at 8:43 PM, Takashi Matsuo  wrote:

>
> Hi Mos,
>
> On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:
>
>> Hello Takashi,
>>
>>
>> > Actually there were almost 8 requests in a second. So App Engine likely
>> needed more than one instance at this particular moment.
>>
>> I thought this is why GAE has the concept of "pending-latency"  (which we
>> discussed below).
>> Meaning:  Incoming requests may wait up to 15 seconds before starting a
>> new instance. Therefore when 8 requests in one second occur that
>> should not mean that more instance needs to be started. Especially if
>> there is no other traffic in this minute, as seen in my example.
>> Otherwise it would be a very bad implementation:
>> Starting a new instance means around 30s waiting time.  Serving 8
>> parallel requests from one instance, would result in a maximum of
>> 8 seconds for the last request (assuming that each request takes around 1
>> second).
>> There is no reason for this concrete example to fire up more instances
>> and let requests wait more then 30 seconds until a new instance is loaded.
>>
>
> Do you really read my e-mail?
>
> Setting Max Pending Latency doesn't force requests to be in the pending
> queue for the specified time. Please use Min Pending Latency instead.
> Can you try this first? If it doesn't work, try 2 min idle instances then.
>
>
>>
>> > ... here is what you've seen in the past weeks.
>> >
>> >* You have been almost always set 'Automatic-2' idle instance setting.
>> >* More than 3 weeks ago, number of loading requests were very few.
>> > * Recently you have seen more loading requests than before.
>>
>> That, right!  To be even more concrete: At the 16. august the problems
>> got significant worse. Please check especially the time area from 16.
>> august until today.
>>
>> > First of all, it seems that you deployed 2 new versions on Aug 1 and
>> Aug 2. Can you describe what kind of changes in those versions?
>>
>> I checked it in our version control. As I wrote no related changes were
>> made! Just Html/Css stuff:
>>  * One picture upload
>>  * One html change
>>  * One JavaScript change
>>  * One css change
>>
>>
>> > And, to be fair, we didn't think of any change in our scheduler around
>> 3 weeks ago which can cause this issue.
>>
>> And around the 16th august?
>
>
> Sigh... isn't it a waist of time? What is the reason you picked that date?
>
>
>>
>
>
>> > More than 3 weeks before, those 2 idle instances might have had longer
>> lives than now, but it was not a concrete behavior. Please think this way:
>> you were just kind of lucky.
>>
>> That shouldn't be luck! If GAE is not able to start Java instances in
>> 5sec to 10 second, there needs be a guarantee that instances have longer
>> lives.  Otherwise Java applications on GAE are unusable because

[google-appengine] The cronjobs keep coming every hour.

[David Hardwick]

We have had similar support requests yesterday...every hour even though 
scheduled once per week.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/UJcdsdjTlMMJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Mos]

Thanks Kris for describing your case. That's what I saw in my experiments
also. The "min instance setting" is not the solution because it doesn't
work as expected.
I hope someone from GAE's team takes it serious and elaborate on this.

On Fri, Aug 24, 2012 at 9:49 PM, Kristopher Giesing
wrote:

> Hi Takashi,
>
> I ran some experiments with an instance that had requests pending only
> from my own scripts (no user facing traffic at all).
>
> What I found was that sending requests at about 1req/sec, regularly
> spaced, caused GAE to spin up new instances randomly.  If I set the min
> instances setting to anything but "automatic", the very first request would
> cause a new instance to spin up (this was true even if the min instances
> was some high number, like 8, and I waited for all 8 instances to finish
> launching before sending a request - so in this case the # of instances
> started at 9 for the very first request).
>
> The only solution I found for this behavior was to package the entire app
> as a backend.
>
> - Kris
>
>
> On Friday, August 24, 2012 11:43:23 AM UTC-7, Takashi Matsuo (Google)
> wrote:
>
>>
>> Hi Mos,
>>
>> On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:
>>
>>> Hello Takashi,
>>>
>>>
>>> > Actually there were almost 8 requests in a second. So App Engine
>>> likely needed more than one instance at this particular moment.
>>>
>>> I thought this is why GAE has the concept of "pending-latency"  (which
>>> we discussed below).
>>> Meaning:  Incoming requests may wait up to 15 seconds before starting a
>>> new instance. Therefore when 8 requests in one second occur that
>>> should not mean that more instance needs to be started. Especially if
>>> there is no other traffic in this minute, as seen in my example.
>>> Otherwise it would be a very bad implementation:
>>> Starting a new instance means around 30s waiting time.  Serving 8
>>> parallel requests from one instance, would result in a maximum of
>>> 8 seconds for the last request (assuming that each request takes around
>>> 1 second).
>>> There is no reason for this concrete example to fire up more instances
>>> and let requests wait more then 30 seconds until a new instance is loaded.
>>>
>>
>> Do you really read my e-mail?
>>
>> Setting Max Pending Latency doesn't force requests to be in the pending
>> queue for the specified time. Please use Min Pending Latency instead.
>> Can you try this first? If it doesn't work, try 2 min idle instances then.
>>
>>
>>>
>>> > ... here is what you've seen in the past weeks.
>>> >
>>> >* You have been almost always set 'Automatic-2' idle instance setting.
>>> >* More than 3 weeks ago, number of loading requests were very few.
>>> > * Recently you have seen more loading requests than before.
>>>
>>> That, right!  To be even more concrete: At the 16. august the problems
>>> got significant worse. Please check especially the time area from 16.
>>> august until today.
>>>
>>> > First of all, it seems that you deployed 2 new versions on Aug 1 and
>>> Aug 2. Can you describe what kind of changes in those versions?
>>>
>>> I checked it in our version control. As I wrote no related changes were
>>> made! Just Html/Css stuff:
>>>  * One picture upload
>>>  * One html change
>>>  * One JavaScript change
>>>  * One css change
>>>
>>>
>>> > And, to be fair, we didn't think of any change in our scheduler around
>>> 3 weeks ago which can cause this issue.
>>>
>>> And around the 16th august?
>>
>>
>> Sigh... isn't it a waist of time? What is the reason you picked that
>> date?
>>
>>
>>>
>>
>>
>>> > More than 3 weeks before, those 2 idle instances might have had longer
>>> lives than now, but it was not a concrete behavior. Please think this way:
>>> you were just kind of lucky.
>>>
>>> That shouldn't be luck! If GAE is not able to start Java instances in
>>> 5sec to 10 second, there needs be a guarantee that instances have longer
>>> lives.  Otherwise Java applications on GAE are unusable because user would
>>> have a lot of 30seconds wait time  (--> "failed requests"). (See also next
>>> comment regarding resistant instances)
>>>
>>>
>>> > If you want some instances always active, please set min idle
>>> instances.
>>>
>>> I tried this some days ago. I had one resistant instance. But that
>>> changed nothing.  Instances get started and stopped as before. I assumed
>>> that requests would go to the resistant instance first. But that was no the
>>> case. Resistant instance was idle, but a dynamic instance got started and
>>> the request waits 30sec.
>>
>> Please check other discussion on this list and issues that reported
>>> similar observations.
>>>
>>
>> So I'd say please try 2. If you still saw the user-facing loading
>> requests, you need more resident instance to eliminate the user-facing
>> loading requests.
>>
>>
>>>
>>> > As you can see, I'm still not convinced to believe that the scheduler
>>> is misbehaving. I understand that you're having experiences which are bit
>>> worse than 3 weeks ago, and understand yo

Re: [google-appengine] Weird Instance Scheduler

[Jeff Schnitzer]

I had a similar experience the last time I experimented with the
min-idle slider.  GAE would route new requests to cold starts rather
than use the idle resident instance.  Min-idle seemed like it should
be renamed "always idle" - it made my UX horrible, my bill higher, and
I haven't touched it since.

There may be rational logic to the scheduler behavior, but it feels
broken to just about everyone who experiments with it.  I can only
guess at how many people have given up on GAE because of this issue.

Every time this comes up (and it comes up a *lot*), I'm going to repeat:

---> Requests should never be sent to cold instances.

Addressing that one issue seems like it would fix all the other
issues, or at least make them transparent enough that we can figure
out how to tune the scheduler on our own.

Jeff

On Fri, Aug 24, 2012 at 3:49 PM, Kristopher Giesing
 wrote:
> Hi Takashi,
>
> I ran some experiments with an instance that had requests pending only from
> my own scripts (no user facing traffic at all).
>
> What I found was that sending requests at about 1req/sec, regularly spaced,
> caused GAE to spin up new instances randomly.  If I set the min instances
> setting to anything but "automatic", the very first request would cause a
> new instance to spin up (this was true even if the min instances was some
> high number, like 8, and I waited for all 8 instances to finish launching
> before sending a request - so in this case the # of instances started at 9
> for the very first request).
>
> The only solution I found for this behavior was to package the entire app as
> a backend.
>
> - Kris
>
>
> On Friday, August 24, 2012 11:43:23 AM UTC-7, Takashi Matsuo (Google) wrote:
>>
>>
>> Hi Mos,
>>
>> On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:
>>>
>>> Hello Takashi,
>>>
>>>
>>> > Actually there were almost 8 requests in a second. So App Engine likely
>>> > needed more than one instance at this particular moment.
>>>
>>> I thought this is why GAE has the concept of "pending-latency"  (which we
>>> discussed below).
>>> Meaning:  Incoming requests may wait up to 15 seconds before starting a
>>> new instance. Therefore when 8 requests in one second occur that
>>> should not mean that more instance needs to be started. Especially if
>>> there is no other traffic in this minute, as seen in my example.
>>> Otherwise it would be a very bad implementation:
>>> Starting a new instance means around 30s waiting time.  Serving 8
>>> parallel requests from one instance, would result in a maximum of
>>> 8 seconds for the last request (assuming that each request takes around 1
>>> second).
>>> There is no reason for this concrete example to fire up more instances
>>> and let requests wait more then 30 seconds until a new instance is loaded.
>>
>>
>> Do you really read my e-mail?
>>
>> Setting Max Pending Latency doesn't force requests to be in the pending
>> queue for the specified time. Please use Min Pending Latency instead.
>> Can you try this first? If it doesn't work, try 2 min idle instances then.
>>
>>>
>>>
>>> > ... here is what you've seen in the past weeks.
>>> >
>>> >* You have been almost always set 'Automatic-2' idle instance setting.
>>> >* More than 3 weeks ago, number of loading requests were very few.
>>> > * Recently you have seen more loading requests than before.
>>>
>>> That, right!  To be even more concrete: At the 16. august the problems
>>> got significant worse. Please check especially the time area from 16. august
>>> until today.
>>>
>>> > First of all, it seems that you deployed 2 new versions on Aug 1 and
>>> > Aug 2. Can you describe what kind of changes in those versions?
>>>
>>> I checked it in our version control. As I wrote no related changes were
>>> made! Just Html/Css stuff:
>>>  * One picture upload
>>>  * One html change
>>>  * One JavaScript change
>>>  * One css change
>>>
>>>
>>> > And, to be fair, we didn't think of any change in our scheduler around
>>> > 3 weeks ago which can cause this issue.
>>>
>>> And around the 16th august?
>>
>>
>> Sigh... isn't it a waist of time? What is the reason you picked that date?
>>
>>>
>>>
>>>
>>>
>>> > More than 3 weeks before, those 2 idle instances might have had longer
>>> > lives than now, but it was not a concrete behavior. Please think this way:
>>> > you were just kind of lucky.
>>>
>>> That shouldn't be luck! If GAE is not able to start Java instances in
>>> 5sec to 10 second, there needs be a guarantee that instances have longer
>>> lives.  Otherwise Java applications on GAE are unusable because user would
>>> have a lot of 30seconds wait time  (--> "failed requests"). (See also next
>>> comment regarding resistant instances)
>>>
>>>
>>> > If you want some instances always active, please set min idle
>>> > instances.
>>>
>>> I tried this some days ago. I had one resistant instance. But that
>>> changed nothing.  Instances get started and stopped as before. I assumed
>>> that requests would go to the resistant instance first. But that was 

Re: [google-appengine] Need advice on what to do POST HRD migration

[Takashi Matsuo]

On Sat, Aug 25, 2012 at 1:35 AM, n3phele  wrote:

> I have a question about what do to *after* migration to the HRD data store.
>
> After migration I end up with
>
> foo.appspot.com (old master/slave)
>
> and
>
> foo-hrd.appspot.com (new HRD app)
>
> I have chosen the appengine setup so that requests to foo.appspot.com are
> redirected to foo-hrd.appspot.com
>
> My question is:
>
> Can I safely delete the foo.appspot.com application using the appengine
> dashboard and be sure that the redirect foo.appspot.com will remain in
> place?
>

Yes, you can safely delete the old app.


>
> Thanks in advance
> -N
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/nrFOVU87HSgJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>



-- 
Takashi Matsuo | Developers Advocate | tmat...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Almost all emails not delivered.

[Takashi Matsuo]

Hi Prashant,

Great to hear that!


On Sat, Aug 25, 2012 at 4:51 AM, Prashant Hegde wrote:

> Hello Takashi,
>
> Thanks so much for the pointer. I must have missed reading this
> discussion. Sorry about that. I enabled DKIM for the domain, and it seems
> like app has started delivering emails.
>
> Best Regards.
>
>
>
> On 22-Aug-2012, at 7:12 AM, Takashi Matsuo wrote:
>
>
> Hi Prashant,
>
> Have you seen this e-mail?
> https://groups.google.com/d/topic/google-appengine/K6_mISYU5_E/discussion
>
> Are you using DKIM? If no, please consider using it. You can also fill the
> form in that e-mail.
>
> -- Takashi
>
>
> On Wed, Aug 22, 2012 at 10:31 AM, prashant wrote:
>
>> Our email delivery is impacted. Can some one look into this issue please
>> !
>>
>> Thanks.
>>
>>
>> On Saturday, August 18, 2012 3:09:15 PM UTC+5:30, prashant wrote:
>>>
>>> Hello!
>>>
>>> http://code.google.com/p/**googleappengine/issues/detail?**id=7974
>>>
>>> We have reported a production issue (above) on email delivery failures
>>> for our app. Kindly look into it. Emails have suddenly stopped (almost all
>>> with only sporadic emails delivered ) for the last 2 weeks. It was fine
>>> earlier.
>>>
>>> Best Regards
>>> Prashant
>>>
>>>
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msg/google-appengine/-/yheO8AG0czMJ.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
>
>
> --
> Takashi Matsuo | Developers Advocate | tmat...@google.com
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>



-- 
Takashi Matsuo | Developers Advocate | tmat...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] after hrd migration app cannot find app data

[Takashi Matsuo]

I think you're using a serialized form of old keys on the old apps. You
need to convert those keys by yourself.

Here is an example:
http://stackoverflow.com/questions/9760205/migration-to-hrd-how-to-convert-string-encoded-keys-to-new-application

-- Takashi


On Sat, Aug 25, 2012 at 1:44 AM, Alejandro Diego Garin wrote:

> Hi,
>
> Anyone could help with this? I just finished the migration I see this
> error in the logs.
>
>
> unexpected exception: javax.jdo.JDOFatalUserException: Illegal argument
> NestedThrowables:
> java.lang.IllegalArgumentException: app s~jardincitosonline-hrd cannot access 
> app jardincitosonline's data
>
>
> Thanks.
> Alejandro.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/LCDKuH30Mz8J.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>



-- 
Takashi Matsuo | Developers Advocate | tmat...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Almost all emails not delivered.

[Prashant Hegde]

Hello Takashi,

Thanks so much for the pointer. I must have missed reading this discussion. 
Sorry about that. I enabled DKIM for the domain, and it seems like app has 
started delivering emails.

Best Regards.



On 22-Aug-2012, at 7:12 AM, Takashi Matsuo wrote:

> 
> Hi Prashant,
> 
> Have you seen this e-mail?
> https://groups.google.com/d/topic/google-appengine/K6_mISYU5_E/discussion
> 
> Are you using DKIM? If no, please consider using it. You can also fill the 
> form in that e-mail.
> 
> -- Takashi
> 
> 
> On Wed, Aug 22, 2012 at 10:31 AM, prashant  wrote:
> Our email delivery is impacted. Can some one look into this issue please ! 
> 
> Thanks.
> 
> 
> On Saturday, August 18, 2012 3:09:15 PM UTC+5:30, prashant wrote:
> Hello! 
> 
> http://code.google.com/p/googleappengine/issues/detail?id=7974 
> 
> We have reported a production issue (above) on email delivery failures for 
> our app. Kindly look into it. Emails have suddenly stopped (almost all with 
> only sporadic emails delivered ) for the last 2 weeks. It was fine earlier. 
> 
> Best Regards 
> Prashant 
> 
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To view this discussion on the web visit 
> https://groups.google.com/d/msg/google-appengine/-/yheO8AG0czMJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
> 
> 
> 
> -- 
> Takashi Matsuo | Developers Advocate | tmat...@google.com
> 
> 
> -- 
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Kristopher Giesing]

Hi Takashi,

I ran some experiments with an instance that had requests pending only from 
my own scripts (no user facing traffic at all).

What I found was that sending requests at about 1req/sec, regularly spaced, 
caused GAE to spin up new instances randomly.  If I set the min instances 
setting to anything but "automatic", the very first request would cause a 
new instance to spin up (this was true even if the min instances was some 
high number, like 8, and I waited for all 8 instances to finish launching 
before sending a request - so in this case the # of instances started at 9 
for the very first request).

The only solution I found for this behavior was to package the entire app 
as a backend.

- Kris

On Friday, August 24, 2012 11:43:23 AM UTC-7, Takashi Matsuo (Google) wrote:
>
>
> Hi Mos,
>
> On Sat, Aug 25, 2012 at 1:39 AM, Mos  >wrote:
>
>> Hello Takashi,
>>
>>
>> > Actually there were almost 8 requests in a second. So App Engine likely 
>> needed more than one instance at this particular moment.
>>
>> I thought this is why GAE has the concept of "pending-latency"  (which we 
>> discussed below).
>> Meaning:  Incoming requests may wait up to 15 seconds before starting a 
>> new instance. Therefore when 8 requests in one second occur that
>> should not mean that more instance needs to be started. Especially if 
>> there is no other traffic in this minute, as seen in my example.
>> Otherwise it would be a very bad implementation:
>> Starting a new instance means around 30s waiting time.  Serving 8 
>> parallel requests from one instance, would result in a maximum of
>> 8 seconds for the last request (assuming that each request takes around 1 
>> second).
>> There is no reason for this concrete example to fire up more instances 
>> and let requests wait more then 30 seconds until a new instance is loaded.
>>
>
> Do you really read my e-mail?
>
> Setting Max Pending Latency doesn't force requests to be in the pending 
> queue for the specified time. Please use Min Pending Latency instead.
> Can you try this first? If it doesn't work, try 2 min idle instances then.
>  
>
>>
>> > ... here is what you've seen in the past weeks.
>> >
>> >* You have been almost always set 'Automatic-2' idle instance setting.
>> >* More than 3 weeks ago, number of loading requests were very few.
>> > * Recently you have seen more loading requests than before.
>>
>> That, right!  To be even more concrete: At the 16. august the problems 
>> got significant worse. Please check especially the time area from 16. 
>> august until today. 
>>
>> > First of all, it seems that you deployed 2 new versions on Aug 1 and 
>> Aug 2. Can you describe what kind of changes in those versions?
>>
>> I checked it in our version control. As I wrote no related changes were 
>> made! Just Html/Css stuff:
>>  * One picture upload
>>  * One html change
>>  * One JavaScript change
>>  * One css change
>>
>>
>> > And, to be fair, we didn't think of any change in our scheduler around 
>> 3 weeks ago which can cause this issue.
>>
>> And around the 16th august?  
>
>
> Sigh... isn't it a waist of time? What is the reason you picked that date? 
>  
>
>>  
>
>
>> > More than 3 weeks before, those 2 idle instances might have had longer 
>> lives than now, but it was not a concrete behavior. Please think this way: 
>> you were just kind of lucky. 
>>
>> That shouldn't be luck! If GAE is not able to start Java instances in 
>> 5sec to 10 second, there needs be a guarantee that instances have longer 
>> lives.  Otherwise Java applications on GAE are unusable because user would 
>> have a lot of 30seconds wait time  (--> "failed requests"). (See also next 
>> comment regarding resistant instances)
>>
>>
>> > If you want some instances always active, please set min idle instances.
>>
>> I tried this some days ago. I had one resistant instance. But that 
>> changed nothing.  Instances get started and stopped as before. I assumed 
>> that requests would go to the resistant instance first. But that was no the 
>> case. Resistant instance was idle, but a dynamic instance got started and 
>> the request waits 30sec.   
>
> Please check other discussion on this list and issues that reported 
>> similar observations. 
>>
>
> So I'd say please try 2. If you still saw the user-facing loading 
> requests, you need more resident instance to eliminate the user-facing 
> loading requests.
>  
>
>>  
>> > As you can see, I'm still not convinced to believe that the scheduler 
>> is misbehaving. I understand that you're having experiences which are bit 
>> worse than 3 weeks ago, and understand your feeling that you want to tell 
>> us 'fix it', but I'd say it's > >still something in the line of 'expected 
>> behavior' at least for now.
>> > If you feel differently, please let me know.
>>
>> Yes I do feel differently (please see answers above). 
>>
>> Please accept 
>> http://code.google.com/p/googleappengine/issues/detail?id=8004
>>
>
> So what is your expected behavior a

[google-appengine] Re: Why is memcache.Get so much slower than datastore_v3.Get

[Bryce Cutt]

Other than the one outlier at the top of your image your numbers look 
pretty typical to me.

Memcache can have a very variable response time. I have seen them go as 
high as 1 second on a bad day but this has been rare. On a usual day I see 
average times around 5ms to 20ms with occasional spikes to 200ms. The 
numbers are always very application specific as well; I can have two 
applications doing the exact same memcache requests and one will be twice 
as fast as the other all the time (due to noisy neighbors or different data 
centers or whatever).

I get the impression from other posts that those numbers are not abnormal. 
I believe Brandon Wirtz posted a ton of numbers at some point based on his 
extensive experience optimizing for speed on GAE.

Here are the latency averages GAE is reporting today:
http://code.google.com/status/appengine/detail/memcache/2012/08/24

- Bryce


On Friday, August 24, 2012 8:52:22 AM UTC-7, Mobicage wrote:
>
> Hi
>
> I experience that getting things (successfully) from memcache is much 
> slower than getting them from the datastore:
> see the following appstat report:
> http://oi49.tinypic.com/2yvrnlz.jpg
>
> My memcache keys are rather long e.g.
>
> 'v1.Z2V0X3Byb2ZpbGUuY29tLm1vYmljYWdlLmRhbC5wcm9maWxlMTExEQAAAGNhcmxAbW9iaWNhZ2UuY29t'
>
> What could be the reason for such behaviour?
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/_QV59N1lZ7YJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Need help as we are stuck due to issues on gae

[Vik]

Hello

can someone please advise us on
http://stackoverflow.com/questions/12103783/issues-in-version-uploaded-on-google-app-engine

Thankx and Regards

Vik
Founder
http://www.sakshum.org
http://blog.sakshum.org

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Need advice on what to do POST HRD migration

[n3phele]

I have a question about what do to *after* migration to the HRD data store.

After migration I end up with

foo.appspot.com (old master/slave)

and

foo-hrd.appspot.com (new HRD app)

I have chosen the appengine setup so that requests to foo.appspot.com are 
redirected to foo-hrd.appspot.com

My question is:

Can I safely delete the foo.appspot.com application using the appengine 
dashboard and be sure that the redirect foo.appspot.com will remain in 
place?

Thanks in advance
-N
 

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/nrFOVU87HSgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] after hrd migration app cannot find app data

[Alejandro Diego Garin]

Hi,

Anyone could help with this? I just finished the migration I see this error 
in the logs.


unexpected exception: javax.jdo.JDOFatalUserException: Illegal argument
NestedThrowables:
java.lang.IllegalArgumentException: app s~jardincitosonline-hrd cannot access 
app jardincitosonline's data


Thanks.
Alejandro.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/LCDKuH30Mz8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Takashi Matsuo]

Hi Mos,

On Sat, Aug 25, 2012 at 1:39 AM, Mos  wrote:

> Hello Takashi,
>
>
> > Actually there were almost 8 requests in a second. So App Engine likely
> needed more than one instance at this particular moment.
>
> I thought this is why GAE has the concept of "pending-latency"  (which we
> discussed below).
> Meaning:  Incoming requests may wait up to 15 seconds before starting a
> new instance. Therefore when 8 requests in one second occur that
> should not mean that more instance needs to be started. Especially if
> there is no other traffic in this minute, as seen in my example.
> Otherwise it would be a very bad implementation:
> Starting a new instance means around 30s waiting time.  Serving 8 parallel
> requests from one instance, would result in a maximum of
> 8 seconds for the last request (assuming that each request takes around 1
> second).
> There is no reason for this concrete example to fire up more instances and
> let requests wait more then 30 seconds until a new instance is loaded.
>

Do you really read my e-mail?

Setting Max Pending Latency doesn't force requests to be in the pending
queue for the specified time. Please use Min Pending Latency instead.
Can you try this first? If it doesn't work, try 2 min idle instances then.


>
> > ... here is what you've seen in the past weeks.
> >
> >* You have been almost always set 'Automatic-2' idle instance setting.
> >* More than 3 weeks ago, number of loading requests were very few.
> > * Recently you have seen more loading requests than before.
>
> That, right!  To be even more concrete: At the 16. august the problems got
> significant worse. Please check especially the time area from 16. august
> until today.
>
> > First of all, it seems that you deployed 2 new versions on Aug 1 and Aug
> 2. Can you describe what kind of changes in those versions?
>
> I checked it in our version control. As I wrote no related changes were
> made! Just Html/Css stuff:
>  * One picture upload
>  * One html change
>  * One JavaScript change
>  * One css change
>
>
> > And, to be fair, we didn't think of any change in our scheduler around 3
> weeks ago which can cause this issue.
>
> And around the 16th august?


Sigh... isn't it a waist of time? What is the reason you picked that date?


>


> > More than 3 weeks before, those 2 idle instances might have had longer
> lives than now, but it was not a concrete behavior. Please think this way:
> you were just kind of lucky.
>
> That shouldn't be luck! If GAE is not able to start Java instances in 5sec
> to 10 second, there needs be a guarantee that instances have longer lives.
> Otherwise Java applications on GAE are unusable because user would have a
> lot of 30seconds wait time  (--> "failed requests"). (See also next comment
> regarding resistant instances)
>
>
> > If you want some instances always active, please set min idle instances.
>
> I tried this some days ago. I had one resistant instance. But that changed
> nothing.  Instances get started and stopped as before. I assumed that
> requests would go to the resistant instance first. But that was no the
> case. Resistant instance was idle, but a dynamic instance got started and
> the request waits 30sec.

Please check other discussion on this list and issues that reported similar
> observations.
>

So I'd say please try 2. If you still saw the user-facing loading requests,
you need more resident instance to eliminate the user-facing loading
requests.


>
> > As you can see, I'm still not convinced to believe that the scheduler is
> misbehaving. I understand that you're having experiences which are bit
> worse than 3 weeks ago, and understand your feeling that you want to tell
> us 'fix it', but I'd say it's > >still something in the line of 'expected
> behavior' at least for now.
> > If you feel differently, please let me know.
>
> Yes I do feel differently (please see answers above).
>
> Please accept
> http://code.google.com/p/googleappengine/issues/detail?id=8004
>

So what is your expected behavior and actual result? Nobody in our team can
do anything if you just keep saying "the setting that used to work doesn't
work anymore" without trying mu suggestion.

I think my answer is clear at least for some points. 1) You'd better use
'min pending latency' instead of 'max pending latency' to prevent new
instances to spin up as much as possible. 2) If you need longer instance
lives, set appropriate number of min idle instances.

-- Takashi


>
>
> Thanks
> Mos
> http://www.mosbase.com
>
>
> On Fri, Aug 24, 2012 at 4:22 PM, Takashi Matsuo wrote:
>
>>
>> Hi Mos,
>>
>> On Fri, Aug 24, 2012 at 6:05 PM, Mos  wrote:
>>
>>> > A possible explanation could be that the traffic pattern had changed.
>>>
>>> No. It's the same. Check for example the Request/Seconds statistics of
>>> my application for the last 30 days!
>>
>>
>>> >> It's very obvious that one instance should be enough for my
>>> application. And that was almost the case the last months!
>>> > Actually it's not 

[google-appengine] Re: HRD Migration & Blob keys

[Phil McDonnell]

Is anyone familiar with this blobstore issue in the HRD migration?

Thanks,
Phil

On Thu, Aug 23, 2012 at 4:39 PM, Phil McDonnell
wrote:

> I'm looking at migrating to HRD and I use the blob store.  In my datastore
> I have one Kind of entity that stores a blob key as one of the fields in
> the Entity. I see that the HRD migration warns about blob keys stored in
> "serialized form." Pardon my ignorance, but are my blob keys that I
> mentioned stored in serialized form? Quotation from the documentation &
> example of my blob key in the Entity (highlighted yellow) below.
>
> Thanks,
> Phil
>
> *Warning from the HRD Migration:*
>
> If your application has blob data in Blobstore, you need to migrate that
> blob data as well. Check the checkbox labeled *Migrate Blobstore Data*.
> The blobs will be copied and given new blob keys in the new application,
> and the Datastore will be updated with the new blob keys. (However, any
> blob keys stored in serialized form within the Datastore will *not* be
> migrated; see Migrating Serialized Blob 
> Keys,
> below, for more information.)
>
> *Example of my datastore from the datastore viewer:*
>   ID/Namecomplete couponKeydate dealIDdealVariant fraudulentnumSoldppemail
> price purchasesold uidverified address1address2 charitycity fullnamestatezip
> id=174125True
> AMIfv95dj6WTPGjP8P38yQrSmjbPs6eqWCFpvafA0Y9KoROcostO8OiHMWOwZKFfqmNwfi2s9pXyK7Bz8cOIG-CSPnv-T9r0ac9Fn_c5carOHrpRBxFz-Pm3zcI4j93-MDHssxDpC6PdnKY5P3Dr66BrGLHWqoBNzQ
> View 
> blob
> 2012-01-23 05:19:04.658000 asdf 0False 1a...@asdf.com 4.0 FalseTrue10709668
> True 
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Queries hitting deadline

[Phil McDonnell]

I can definitely do the work in a task, but the deadline I'm hitting is the
read deadline from the datastore. I believe that will be the same for
tasks, right?

Thanks,
Phil


On Fri, Aug 24, 2012 at 12:18 AM, timh  wrote:

> If your count keeps increasing you will always run into some sort of time
> limit.  Why not consider doing this processing in a task (they can run for
> 10mins, or multiple tasks. )  I am assuming your trying to summarise etc
>
> T
>
>
> On Friday, August 24, 2012 3:26:00 AM UTC+8, Phil wrote:
>>
>> In some initialization work my app needs to run through the all of the
>> datastore entities of a given kind.  I have a lot of these entities (80k
>> currently) and it's increasing rapidly. I'm currently trying to read these
>> in using a single datastore query, but running up against the default
>> datastore timeout of 30 seconds.
>>
>> Is there a good practice for sharding this or otherwise breaking this up
>> so that I won't hit these deadlines? I was thinking I would do a keyOnly
>> query and then break up the keys into a number of reasonably sized
>> sub-queries, but perhaps there is a better approach out there?
>>
>> Thanks,
>> Phil
>>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/5x0WQc9ODVwJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Connectivity to Twitter API completely gone for 1 of my apps

[Jeff Schnitzer]

On Fri, Aug 24, 2012 at 1:00 PM, JH  wrote:
> I don't want to get excited prematurely but it appears to be working again.
> Also, note this is not a rate-limiting issue.  I am authenticating with
> oauth.  When Twitter rate limits your IP, which they do for all app engine
> IP's I've ever used, they will let you know you are rate limited with an
> error response.

According to https://dev.twitter.com/docs/rate-limiting :


Blacklisting

We ask that you honor the rate limit. If you or your application
abuses the rate limits we will blacklist it. If you are blacklisted
you will be unable to get a response from the Twitter API.


Sounds like your problem.

Jeff

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Connectivity to Twitter API completely gone for 1 of my apps

[Jeff Schnitzer]

On Fri, Aug 24, 2012 at 12:19 PM, KC  wrote:
> We've been having exactly the same problem problem in one of our apps in GAE
> since yesterday.
> We are using Twitter4J to connect to Twitter, and receiving errors like:
> "Could not fetch URL: https://api.twitter.com/oauth/request_token";
> It worked perfectly for more than 1 year. Any clue how to tackle this?

You need to proxy any calls to services which have rate limits (or
might have rate limits in the future).

Jeff

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Connectivity to Twitter API completely gone for 1 of my apps

[JH]

I don't want to get excited prematurely but it appears to be working again. 
 Also, note this is not a rate-limiting issue.  I am authenticating with 
oauth.  When Twitter rate limits your IP, which they do for all app engine 
IP's I've ever used, they will let you know you are rate limited with an 
error response.

On Friday, August 24, 2012 9:26:34 AM UTC-5, JH wrote:
>
> As of yesterday my app, ticker-app, can no longer connect to Twitter's 
> API.  It has worked fine for over a year.  As of yesterday any urlfetch 
> call to twitter.com times out with DeadlineExceeded.  However, using 
> other app ID's I have they can still urlfetch to twitter.  I have to assume 
> it has something to do with which data center ticker-app is hosted in?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/nfoB0eMI4XYJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Mos]

Hello Takashi,

> Actually there were almost 8 requests in a second. So App Engine likely
needed more than one instance at this particular moment.

I thought this is why GAE has the concept of "pending-latency"  (which we
discussed below).
Meaning:  Incoming requests may wait up to 15 seconds before starting a new
instance. Therefore when 8 requests in one second occur that
should not mean that more instance needs to be started. Especially if there
is no other traffic in this minute, as seen in my example.
Otherwise it would be a very bad implementation:
Starting a new instance means around 30s waiting time.  Serving 8 parallel
requests from one instance, would result in a maximum of
8 seconds for the last request (assuming that each request takes around 1
second).
There is no reason for this concrete example to fire up more instances and
let requests wait more then 30 seconds until a new instance is loaded.

> ... here is what you've seen in the past weeks.
>
>* You have been almost always set 'Automatic-2' idle instance setting.
>* More than 3 weeks ago, number of loading requests were very few.
> * Recently you have seen more loading requests than before.

That, right!  To be even more concrete: At the 16. august the problems got
significant worse. Please check especially the time area from 16. august
until today.

> First of all, it seems that you deployed 2 new versions on Aug 1 and Aug
2. Can you describe what kind of changes in those versions?

I checked it in our version control. As I wrote no related changes were
made! Just Html/Css stuff:
 * One picture upload
 * One html change
 * One JavaScript change
 * One css change

> And, to be fair, we didn't think of any change in our scheduler around 3
weeks ago which can cause this issue.

And around the 16th august?

> More than 3 weeks before, those 2 idle instances might have had longer
lives than now, but it was not a concrete behavior. Please think this way:
you were just kind of lucky.

That shouldn't be luck! If GAE is not able to start Java instances in 5sec
to 10 second, there needs be a guarantee that instances have longer lives.
Otherwise Java applications on GAE are unusable because user would have a
lot of 30seconds wait time  (--> "failed requests"). (See also next comment
regarding resistant instances)

> If you want some instances always active, please set min idle instances.

I tried this some days ago. I had one resistant instance. But that changed
nothing.  Instances get started and stopped as before. I assumed that
requests would go to the resistant instance first. But that was no the
case. Resistant instance was idle, but a dynamic instance got started and
the request waits 30sec.  Please check other discussion on this list and
issues that reported similar observations.

> As you can see, I'm still not convinced to believe that the scheduler is
misbehaving. I understand that you're having experiences which are bit
worse than 3 weeks ago, and understand your feeling that you want to tell
us 'fix it', but I'd say it's > >still something in the line of 'expected
behavior' at least for now.
> If you feel differently, please let me know.

Yes I do feel differently (please see answers above).

Please accept http://code.google.com/p/googleappengine/issues/detail?id=8004

Thanks
Mos
http://www.mosbase.com


On Fri, Aug 24, 2012 at 4:22 PM, Takashi Matsuo  wrote:

>
> Hi Mos,
>
> On Fri, Aug 24, 2012 at 6:05 PM, Mos  wrote:
>
>> > A possible explanation could be that the traffic pattern had changed.
>>
>> No. It's the same. Check for example the Request/Seconds statistics of my
>> application for the last 30 days!
>
>
>> >> It's very obvious that one instance should be enough for my
>> application. And that was almost the case the last months!
>> > Actually it's not true. In particular, check this log:
>>
>> That's one expection where one client did 8 request in a minute  (+ one
>> pingdom). Nothing else this minute.
>> In those exceptional cases it could be ok if a second instance starts.
>> (Nevertheless can't one instance not
>> handle 8 requests a  minute?)
>>
>
> The issue here is not 8 requests in a minute. Actually there were almost 8
> requests in a second. So App Engine likely needed more than one instance at
> this particular moment. Anyway, as you say, probably it's just a reason for
> one of the loading requests you're seeing, and this is not very important
> thing in this topic.
>
> It's kind of digressing, but at a first glance, the Requests/Seconds stat
> seems an appropriate data source to discuss how many instances are actually
> needed, but in fact, it's not. The real traffic is not spreading equally.
>
>
>>
>> As I described:  Instances are started and stopped without reason, even
>> if less traffic per minute is available!
>
>
> Okay. As far as I understand, here is what you've seen in the past weeks.
>
> * You have been almost always set 'Automatic-2' idle instance setting.
> * More than 3 weeks ago, number of loading r

[google-appengine] Re: Connectivity to Twitter API completely gone for 1 of my apps

[KC]

We've been having exactly the same problem problem in one of our apps in 
GAE since yesterday.
We are using Twitter4J to connect to Twitter, and receiving errors like: "Could 
not fetch URL: https://api.twitter.com/oauth/request_token";
It worked perfectly for more than 1 year. Any clue how to tackle this?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/-5C1Tco04iQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Creating a game on app engine - calculating costs, thoughts?

[Jeff Schnitzer]

When you say "higher in frequency" what do you mean?  How many queries
per second?  How many games total will be active in your system at
once?

Keep in mind that your query will be eventually consistent.  When
someone joins a game, that fact will not become visible to the queries
right away - often for seconds.  But your query is by nature
"eventual" since someone could have joined the game immediately after
the query was run, so presumably you already handle this case.

One immediate improvement is to convert to your regular query to a
keys-only query followed by a batch fetch.  If you have caching, you
only pay for the keys-only results (1/7th the cost).  I see you're
using Java; if you use Objectify4 this is automatically done for you
with the @Cache annotation and "hybrid" queries (the default with
@Cache entities).

The next step is to cache the keys-only query itself in memcache for a
limited amount of time.  It would be nice to expire it explicitly when
you know it changes - which is only when a game is created or a game
fills up (not for simple add or leave game).  The problem is, there's
no guarantee that the subsequent query (which will populate the cache)
does not contain stale eventual data.  I can't think of a way around
this other than just making sure the expiry period is short so it gets
refreshed; you're just trying to reduce the query count, not eliminate
it.

You can't query by playerNames.size() but you can store an indexed
field numPlayers in the entity.

There's no selecting of specific fields out for an entity (with the
exception of Projection queries, which are very limited and probably
not what you are after) so you can't get everything-but-gameState.
This probably won't affect price (a read op is a read op no matter the
size of the entity) but it could affect latency (more RPCs to get the
same bit of data).  The usual solution is simply to put the gameState
blob into separate entity with the same id.

Jeff

On Thu, Aug 23, 2012 at 6:37 PM, Mark  wrote:
> Hi,
>
> I'm trying to see if it's feasible to write a game using app engine as the
> backend.
>
> The task I'm most worried about is the discoverability of games. A user can
> create a new game (which creates a Game object in the datastore). Then other
> users can hit an endpoint to get a listing of open games, sorted by creation
> date.
>
> Creating a new game will be a very low frequency operation, so no worries
> there. Users polling for new open games will be much higher in frequency
> though. Each time a user requests a listing of games, I need to go fetch a
> page of Game objects from the datastore. Let's say a page size = 20. And the
> Game object looks like this:
>
> class Game {
> String name;
> long timestampCreated;
> long timestampStarted;
> String mapName;
> String hostUsername;
> int numPlayersMax;
> List playerNames; // usernames of other players currently
> joined
> Text gameState; // game state as serialized json string
> }
>
> the query would be something like:
>
> select Games where timestampStarted = 0
> and playerNames.size() < numPlayersMax
> order by timestampCreated DESC limit 20;
>
> First, I'm hoping it's possible to avoid loading the Text field in this
> listings query since it's not needed for rendering yet. Still there would be
> a fair number of fields to fetch and deserialize, and I'm afraid my costs
> would quickly make it impractical.
>
> I could try keeping a list of open games in memcache, but not sure if I
> could achieve consistency between the memcache state and the datastore state
> of games. Will be looking into that.
>
> I've got a very similar game to this already published, using a single jetty
> instance + mysql on one machine. It's starting to get too much traffic  now,
> and scaling would be a pain. App engine's scalability looks great for
> something like this, but not sure if the # of writes and inefficient reads I
> need to perform are a good idea.
>
> Any general thoughts on this would be greatly appreciated. I'm starting to
> dig into this now and any thoughts on how you'd implement the above would be
> great. In my already-published game (mentioned above), I ran a dark test
> where I'd call through to a (quickly implemented) mirror of the game running
> on app engine. It quickly ate through the daily quota!
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/cvpOxGrMRaoJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App

[google-appengine] Re: Connectivity to Twitter API completely gone for 1 of my apps

[Василий Уточкин]

I'm having same troubles with mine twitter-api-based app. And, actually, 
without twitter api this app is shit now. This is sad.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/SSK7iYP-DCAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Why is memcache.Get so much slower than datastore_v3.Get

[Mobicage]

Hi

I experience that getting things (successfully) from memcache is much slower 
than getting them from the datastore:
see the following appstat report:
http://oi49.tinypic.com/2yvrnlz.jpg

My memcache keys are rather long e.g.
'v1.Z2V0X3Byb2ZpbGUuY29tLm1vYmljYWdlLmRhbC5wcm9maWxlMTExEQAAAGNhcmxAbW9iaWNhZ2UuY29t'

What could be the reason for such behaviour?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/kgRhcuENbNUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Connectivity to Twitter API completely gone for 1 of my apps

[Jeff Schnitzer]

You are most likely right.  Given the shared pool of IPs, they could
get blacklisted at any time:

https://dev.twitter.com/docs/rate-limiting

This is going to become a progressively more serious problem as time
goes on (GAE becomes more successful, useful public services add rate
limiting).  Star this issue:
http://code.google.com/p/googleappengine/issues/detail?id=6644

For now it's probably best to make calls to external services through
a proxy with a dedicated IP address you control.  I'd love to hear
suggestions for "cheapest possible solution" - $11/mo for a 256MB
rackspacecloud instance is as good as I can come up with.

Jeff

On Fri, Aug 24, 2012 at 10:26 AM, JH  wrote:
> As of yesterday my app, ticker-app, can no longer connect to Twitter's API.
> It has worked fine for over a year.  As of yesterday any urlfetch call to
> twitter.com times out with DeadlineExceeded.  However, using other app ID's
> I have they can still urlfetch to twitter.  I have to assume it has
> something to do with which data center ticker-app is hosted in?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/eMHHC7P3PtUJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Creating a game on app engine - calculating costs, thoughts?

[de Witte]

You will be fine.

The client will send a request for getting the 20 most recent pending games.

The server will return this by a response object, we call it a "game list 
data object".

1- You can easily cache this response object using MemCache. 

2- You can also query only for keys which is a lot cheaper.

3- You can also cache Game Entities.

Combine these three techniques to get the most out of GAE.

Note that the game list doesn't have to be perfect synchronized and up teo 
date because you can always popup "Sorry this game is full, please select 
another one".









Op vrijdag 24 augustus 2012 13:27:57 UTC+2 schreef Richard Watson het 
volgende:
>
> How large is numPlayersMax, typically?  And how big is the gameState field?
>
> On Friday, August 24, 2012 12:37:37 AM UTC+2, Mark wrote:
>>
>> Hi,
>>
>> I'm trying to see if it's feasible to write a game using app engine as 
>> the backend.
>>
>> The task I'm most worried about is the discoverability of games. A user 
>> can create a new game (which creates a Game object in the datastore). Then 
>> other users can hit an endpoint to get a listing of open games, sorted by 
>> creation date.
>>
>> Creating a new game will be a very low frequency operation, so no worries 
>> there. Users polling for new open games will be much higher in frequency 
>> though. Each time a user requests a listing of games, I need to go fetch a 
>> page of Game objects from the datastore. Let's say a page size = 20. And 
>> the Game object looks like this:
>>
>> class Game {
>> String name;
>> long timestampCreated;
>> long timestampStarted;
>> String mapName;
>> String hostUsername;
>> int numPlayersMax;
>> List playerNames; // usernames of other players currently 
>> joined
>> Text gameState; // game state as serialized json string
>> }
>>
>> the query would be something like:
>>
>> select Games where timestampStarted = 0 
>> and playerNames.size() < numPlayersMax 
>> order by timestampCreated DESC limit 20;
>> 
>> First, I'm hoping it's possible to avoid loading the Text field in this 
>> listings query since it's not needed for rendering yet. Still there would 
>> be a fair number of fields to fetch and deserialize, and I'm afraid my 
>> costs would quickly make it impractical.
>>
>> I could try keeping a list of open games in memcache, but not sure if I 
>> could achieve consistency between the memcache state and the datastore 
>> state of games. Will be looking into that.
>>
>> I've got a very similar game to this already published, using a single 
>> jetty instance + mysql on one machine. It's starting to get too much 
>> traffic  now, and scaling would be a pain. App engine's scalability looks 
>> great for something like this, but not sure if the # of writes and 
>> inefficient reads I need to perform are a good idea.
>>
>> Any general thoughts on this would be greatly appreciated. I'm starting 
>> to dig into this now and any thoughts on how you'd implement the above 
>> would be great. In my already-published game (mentioned above), I ran a 
>> dark test where I'd call through to a (quickly implemented) mirror of the 
>> game running on app engine. It quickly ate through the daily quota!
>>
>> Thanks
>>
>>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/Pp3U1srN43EJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Google App Engine Master/Slave Datastore Deprecation

[Barry Hunter]

Its probably not decided yet.

The ideal for Google would be to get down to nobody using M/S - ie 0 active
applications. Then it can just be turned off and nobody will even notice.
This avoids the issue completely.

Thats unrealistic, so the more people they can beg to move - the easier the
tough decision to kill it will be. But it almost certainly will come, so
the sooner you move, the less chance your application will suddenly
disappear.

If they just turned it off, then yes data would probably be "gone". Might
still be on a google server somewhere, on a storage device. But will be
inaccessible, and useless.

If there are suffienct people still using it when the decisision is made,
they may decide its worth the effort to archive the data somewhere. Make a
backup in Google Storage or something, and let the developer access that
'last backup'. The application will already be dead by then.

It all comes down to risk - how long can your nerve hold out :)


On Fri, Aug 24, 2012 at 3:38 PM, Sargis Dallakyan wrote:

> Thanks for the discussion. Now I understand this from Google's
> perspective, but it's no clear what happens if I do not migrate. Will all
> my data be lost? Thanks.
>
>
> On Friday, August 24, 2012 7:00:58 AM UTC-7, barryhunter wrote:
>
>> While it appears M/S should be around until April 20, 2015 - guessing
>> Google are just trying to push people over.
>>
>> The less time they can spend applying band-aids to the M/S serving
>> infestructure (and also many new features still have to be be tested
>> against M/S - the code common to the two platforms) - the more time they
>> can work on new stuff.
>>
>> It's not a 'threat', its a begging letter. They want to kill M/S sooner*.
>>
>>
>>
>> *(and I imagine it will happen. Keeping M/S will become
>> a serious material technical burden before 2015. )
>>
>>
>>
>>  On Fri, Aug 24, 2012 at 2:39 PM, Joshua Smith wrote:
>>
>>> Yeah, yeah, I know that many of my old M/S applications are running on
>>> the officially deprecated database. And I know that they might suffer a
>>> little more downtime as a result. But I also know that they are working
>>> just fine, and I'm not particularly interested in going through the pain of
>>> migrating (particularly working around all those annoying consistency
>>> things -- I've done that twice now, and each time it took quite a lot of
>>> programming to hide this stuff from the users).
>>>
>>> So the question: Is this notice just a helpful reminder, or is there an
>>> implicit "or else" between the lines that I'm not seeing?
>>>
>>> -Joshua
>>>
>>> On Aug 24, 2012, at 9:10 AM, Google App Engine 
>>> wrote:
>>>
>>> Dear App Engine Developer,
>>>
>>> We’ve noticed that you are running at least one application configured
>>> to use the Master/Slave (M/S) datastore. This application configuration was 
>>> officially
>>> deprecatedon
>>>  April 4, 2012, in accordance with our deprecation
>>> policy , in
>>> favor of the more reliable High Replication Datastore (HRD). HRD uses the 
>>> Paxos
>>> algorithmto
>>>  serve your application out of multiple datacenters, meaning better
>>> redundancy in the face of datacenter issues, more consistent datastore
>>> performance, and no planned downtime.
>>>
>>> When we deprecated the M/S datastore, we introduced a migration 
>>> toolthat
>>>  allows you to easily migrate all your datastore and blobstore data to
>>> a new HRD application. Migrating your application will not require you to
>>> change your application’s URL, whether it serves from appspot.com or a
>>> custom domin. Please note that even if your application does not store any
>>> data in the datastore, it will still benefit from the automated datacenter
>>> failover that is only available to HRD applications.
>>>
>>> Before migrating your application, you should read about the differences
>>> between M/S and HRD, and understand how the consistency policy for 
>>> HRDmight
>>>  affect the queries in your application.
>>>
>>> All HRD applications that have billing enabled are covered by App
>>> Engine’s 99.95% uptime SLA .
>>> Along with the substantial reliability improvements, many new App Engine
>>> features are only being made available to HRD applications, including the
>>> Python 2.7 language option, Full Text Search (FTS), and Page Speed
>>> integration.
>>>
>>> We strongly encourage you to migrate your applications as soon as
>>> possible. If you have technical questions about HRD or the migration
>>> process, you can post them to Stack 
>>> Over

[google-appengine] Re: Creating a game on app engine - calculating costs, thoughts?

[Mark]

numPlayersMax will be capped at 8. I'm pretty sure I have to store the 
participating usernames as a List so I can find a user's games in a single 
query:

select from Games where playerNames = "my-username";

This has me thinking - app engine also offers mysql - this might be better 
suited to a relational database. But if I outgrow a single mysql instance, 
there's no "automatic" scaling, like with the datastore?

Thanks

On Thursday, 23 August 2012 18:37:37 UTC-4, Mark wrote:
>
> Hi,
>
> I'm trying to see if it's feasible to write a game using app engine as the 
> backend.
>
> The task I'm most worried about is the discoverability of games. A user 
> can create a new game (which creates a Game object in the datastore). Then 
> other users can hit an endpoint to get a listing of open games, sorted by 
> creation date.
>
> Creating a new game will be a very low frequency operation, so no worries 
> there. Users polling for new open games will be much higher in frequency 
> though. Each time a user requests a listing of games, I need to go fetch a 
> page of Game objects from the datastore. Let's say a page size = 20. And 
> the Game object looks like this:
>
> class Game {
> String name;
> long timestampCreated;
> long timestampStarted;
> String mapName;
> String hostUsername;
> int numPlayersMax;
> List playerNames; // usernames of other players currently 
> joined
> Text gameState; // game state as serialized json string
> }
>
> the query would be something like:
>
> select Games where timestampStarted = 0 
> and playerNames.size() < numPlayersMax 
> order by timestampCreated DESC limit 20;
> 
> First, I'm hoping it's possible to avoid loading the Text field in this 
> listings query since it's not needed for rendering yet. Still there would 
> be a fair number of fields to fetch and deserialize, and I'm afraid my 
> costs would quickly make it impractical.
>
> I could try keeping a list of open games in memcache, but not sure if I 
> could achieve consistency between the memcache state and the datastore 
> state of games. Will be looking into that.
>
> I've got a very similar game to this already published, using a single 
> jetty instance + mysql on one machine. It's starting to get too much 
> traffic  now, and scaling would be a pain. App engine's scalability looks 
> great for something like this, but not sure if the # of writes and 
> inefficient reads I need to perform are a good idea.
>
> Any general thoughts on this would be greatly appreciated. I'm starting to 
> dig into this now and any thoughts on how you'd implement the above would 
> be great. In my already-published game (mentioned above), I ran a dark test 
> where I'd call through to a (quickly implemented) mirror of the game 
> running on app engine. It quickly ate through the daily quota!
>
> Thanks
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/vFuSLS8u3BMJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Disk image issue while installing

[JChlipala]

I am using a Mac running Mac OS X. When I try to install App Engine, I get the 
following error: "The Google App Engine Runtime could not be extracted (perhaps 
you are running the Launcher from the dmg? If so, drag copy the Launcher to 
your local disk first.) GoogleAppEngineLauncher.app may not work correctly." I 
have Bootcamp installed to use Windows for certain things.  I am trying to 
install for Python 2.7.

Anybody know how to fix this? Thanks for any help!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/92QgX-I5pUcJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Google App Engine Master/Slave Datastore Deprecation

[Sargis Dallakyan]

Thanks for the discussion. Now I understand this from Google's perspective, 
but it's no clear what happens if I do not migrate. Will all my data be 
lost? Thanks.

On Friday, August 24, 2012 7:00:58 AM UTC-7, barryhunter wrote:
>
> While it appears M/S should be around until April 20, 2015 - guessing 
> Google are just trying to push people over. 
>
> The less time they can spend applying band-aids to the M/S serving 
> infestructure (and also many new features still have to be be tested 
> against M/S - the code common to the two platforms) - the more time they 
> can work on new stuff. 
>
> It's not a 'threat', its a begging letter. They want to kill M/S sooner*.
>
>
>
> *(and I imagine it will happen. Keeping M/S will become a serious material 
> technical burden before 2015. )
>
>
>
> On Fri, Aug 24, 2012 at 2:39 PM, Joshua Smith 
> 
> > wrote:
>
>> Yeah, yeah, I know that many of my old M/S applications are running on 
>> the officially deprecated database. And I know that they might suffer a 
>> little more downtime as a result. But I also know that they are working 
>> just fine, and I'm not particularly interested in going through the pain of 
>> migrating (particularly working around all those annoying consistency 
>> things -- I've done that twice now, and each time it took quite a lot of 
>> programming to hide this stuff from the users).
>>
>> So the question: Is this notice just a helpful reminder, or is there an 
>> implicit "or else" between the lines that I'm not seeing?
>>
>> -Joshua
>>
>> On Aug 24, 2012, at 9:10 AM, Google App Engine 
>> > 
>> wrote:
>>
>> Dear App Engine Developer, 
>>
>> We’ve noticed that you are running at least one application configured to 
>> use the Master/Slave (M/S) datastore. This application configuration was 
>> officially 
>> deprecatedon
>>  April 4, 2012, in accordance with our deprecation 
>> policy , in 
>> favor of the more reliable High Replication Datastore (HRD). HRD uses the 
>> Paxos 
>> algorithmto
>>  serve your application out of multiple datacenters, meaning better 
>> redundancy in the face of datacenter issues, more consistent datastore 
>> performance, and no planned downtime. 
>>
>> When we deprecated the M/S datastore, we introduced a migration 
>> toolthat
>>  allows you to easily migrate all your datastore and blobstore data to 
>> a new HRD application. Migrating your application will not require you to 
>> change your application’s URL, whether it serves from appspot.com or a 
>> custom domin. Please note that even if your application does not store any 
>> data in the datastore, it will still benefit from the automated datacenter 
>> failover that is only available to HRD applications. 
>>
>> Before migrating your application, you should read about the differences 
>> between M/S and HRD, and understand how the consistency policy for 
>> HRDmight
>>  affect the queries in your application. 
>>
>> All HRD applications that have billing enabled are covered by App 
>> Engine’s 99.95% uptime SLA . 
>> Along with the substantial reliability improvements, many new App Engine 
>> features are only being made available to HRD applications, including the 
>> Python 2.7 language option, Full Text Search (FTS), and Page Speed 
>> integration. 
>>
>> We strongly encourage you to migrate your applications as soon as 
>> possible. If you have technical questions about HRD or the migration 
>> process, you can post them to Stack 
>> Overflow. 
>> Any general migration discussions can be posted to our Google 
>> Group.
>>  
>>
>>
>>
>> Thank You,
>>
>> The App Engine Team 
>>
>> © 2012 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043 
>> You have received this mandatory email service announcement to update you 
>> about important changes to your Google App Engine account. 
>>
>>
>>  -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to 
>> google-a...@googlegroups.com
>> .
>> To unsubscribe from this group, send email to 
>> google-appengi...@googlegroups.com .
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com

Re: [google-appengine] Duplicate app when you have too many apps

[Takashi Matsuo]

Hi Dmitry,

Which account you're using for app engine? I'm going to look into it.

-- Takashi


On Fri, Aug 24, 2012 at 9:43 PM, Dmitry Kichinsky <
dmitry.kichin...@gmail.com> wrote:

> I'm having the same problem migrating to HRD:  not allowed to create
> anymore apps;reached app creation 
> limit?
>
>
>
> On Saturday, January 7, 2012 12:11:49 AM UTC+4, Ikai Lan (Google) wrote:
>
>> Migrating to HRD using the new migration tool will create an alias, so
>> requests to the old app ID should go to the new one. You do still have to
>> create a new application ID, however.
>>
>> I've allocated a few more app IDs for your account. The current policy is
>> that you can have up to 10 free applications and an unlimited amount of
>> paid applications, but I think it's reasonable make exemptions on a
>> case-by-case basis for people who have been using GAE for a long time who
>> are migrating to HRD.
>>
>> --
>> Ikai Lan
>> Developer Programs Engineer, Google App Engine
>> plus.ikailan.com | twitter.**com/ikai 
>>
>>
>>
>> On Fri, Jan 6, 2012 at 11:27 AM, Ricardo Bánffy  wrote:
>>
>>> Hi folks.
>>>
>>> Is there a way around the "not allowed to create anymore apps;reached
>>> app creation limit?" message when you are trying to migrate an app to
>>> HRD? BTW, is there a way to keep the app id?
>>>
>>> --
>>> Ricardo Bánffy
>>> http://www.dieblinkenlights.**com 
>>> http://twitter.com/rbanffy
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "Google App Engine" group.
>>> To post to this group, send email to google-a...@googlegroups.**com.
>>> To unsubscribe from this group, send email to google-appengi...@**
>>> googlegroups.com.
>>>
>>> For more options, visit this group at http://groups.google.com/**
>>> group/google-appengine?hl=en
>>> .
>>>
>>>
>>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/p1fMaoHMUxkJ.
>
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>



-- 
Takashi Matsuo | Developers Advocate | tmat...@google.com

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Connectivity to Twitter API completely gone for 1 of my apps

[JH]

As of yesterday my app, ticker-app, can no longer connect to Twitter's API. 
 It has worked fine for over a year.  As of yesterday any urlfetch call to 
twitter.com times out with DeadlineExceeded.  However, using other app ID's 
I have they can still urlfetch to twitter.  I have to assume it has 
something to do with which data center ticker-app is hosted in?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/eMHHC7P3PtUJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Takashi Matsuo]

Hi Mos,

On Fri, Aug 24, 2012 at 6:05 PM, Mos  wrote:

> > A possible explanation could be that the traffic pattern had changed.
>
> No. It's the same. Check for example the Request/Seconds statistics of my
> application for the last 30 days!


> >> It's very obvious that one instance should be enough for my
> application. And that was almost the case the last months!
> > Actually it's not true. In particular, check this log:
>
> That's one expection where one client did 8 request in a minute  (+ one
> pingdom). Nothing else this minute.
> In those exceptional cases it could be ok if a second instance starts.
> (Nevertheless can't one instance not
> handle 8 requests a  minute?)
>

The issue here is not 8 requests in a minute. Actually there were almost 8
requests in a second. So App Engine likely needed more than one instance at
this particular moment. Anyway, as you say, probably it's just a reason for
one of the loading requests you're seeing, and this is not very important
thing in this topic.

It's kind of digressing, but at a first glance, the Requests/Seconds stat
seems an appropriate data source to discuss how many instances are actually
needed, but in fact, it's not. The real traffic is not spreading equally.


>
> As I described:  Instances are started and stopped without reason, even if
> less traffic per minute is available!


Okay. As far as I understand, here is what you've seen in the past weeks.

* You have been almost always set 'Automatic-2' idle instance setting.
* More than 3 weeks ago, number of loading requests were very few.
* Recently you have seen more loading requests than before.

First of all, it seems that you deployed 2 new versions on Aug 1 and Aug 2.
Can you describe what kind of changes in those versions?
 I'd like to make sure that there is no changes that can cause the
scheduler/app server behaving differently.

Especially, if you want me to escalate this issue to our engineering team,
you should provide the exact information. You say 'My application is
unchanged', but in fact you deployed the new version on that day when you
described the issue started. I need to make sure that there is no big
change which can cause something bad.

And, to be fair, we didn't think of any change in our scheduler around 3
weeks ago which can cause this issue.

Secondly, you're setting max idle instances = 2. It does not guarantee that
you have always 2 instances. It just guarantees that we will never charge
you for more than 2 idle instances at any time.

More than 3 weeks before, those 2 idle instances might have had longer
lives than now, but it was not a concrete behavior. Please think this way:
you were just kind of lucky. Now, presumably one or two of those instances
are occasionally killed for some reasons(there should be certain legitimate
reasons, but those are something you don't need to care).

If you want some instances always active, please set min idle instances.
Certainly it will cost you a bit more, and you will loose the pending
queue, but considering the access pattern of your app(no bursty traffic
except for few access from the iPhone browser), I would recommend trying
this setting in order to achieve what you want here. I'd recommend 2 idle
instances in this case, but you should decide the number.


> > * What is the purpose of max-pending-latency = 14.9 setting?
>
> " is high App Engine will allow requests to wait rather than start new
> Instances to process them"
> --> One attempt to stop GAE to create unnecessary instances.
>

I think you should set min pending latency instead of max pending latency
if you want to prevent new instance to spin up. However, if you're going to
set min idle instances, this setting will almost loose effect. If you don't
want to set any min idle instances for whatever reason, please consider
setting min pending latency instead of max pending latency.


>
> > * Can you try automatic-automatic for idle instances setting?
>
> I played around with this the last days and nothing changed. As I wrote:
> I had those configuration for months and it worked fine 3-4 weeks ago!
>

> > * What is the purpose of those pingdom check? What happens if you stop
> that?
>
> To be alerted if GAE is down a again. "What happens if you stop that?" -->
> I wouldn't be angry anymore because I wouldn't recognize downtime's of my
> GAE application. ;)
>

> Please forward
> http://code.google.com/p/googleappengine/issues/detail?id=8004  to the
> relevant GAE deparment.
>

As you can see, I'm still not convinced to believe that the scheduler is
misbehaving. I understand that you're having experiences which are bit
worse than 3 weeks ago, and understand your feeling that you want to tell
us 'fix it', but I'd say it's still something in the line of 'expected
behavior' at least for now.

If you feel differently, please let me know.

Regards,

-- Takashi


>
> Thanks!
>
>
> On Fri, Aug 24, 2012 at 1:39 AM, Takashi Matsuo wrote:
>
>>
>> Hi Mos,
>>
>> On Thu, Aug 23, 2012 at 

Re: [google-appengine] Re: Google App Engine Master/Slave Datastore Deprecation

[Barry Hunter]

While it appears M/S should be around until April 20, 2015 - guessing
Google are just trying to push people over.

The less time they can spend applying band-aids to the M/S serving
infestructure (and also many new features still have to be be tested
against M/S - the code common to the two platforms) - the more time they
can work on new stuff.

It's not a 'threat', its a begging letter. They want to kill M/S sooner*.



*(and I imagine it will happen. Keeping M/S will become a serious material
technical burden before 2015. )



On Fri, Aug 24, 2012 at 2:39 PM, Joshua Smith wrote:

> Yeah, yeah, I know that many of my old M/S applications are running on the
> officially deprecated database. And I know that they might suffer a little
> more downtime as a result. But I also know that they are working just fine,
> and I'm not particularly interested in going through the pain of migrating
> (particularly working around all those annoying consistency things -- I've
> done that twice now, and each time it took quite a lot of programming to
> hide this stuff from the users).
>
> So the question: Is this notice just a helpful reminder, or is there an
> implicit "or else" between the lines that I'm not seeing?
>
> -Joshua
>
> On Aug 24, 2012, at 9:10 AM, Google App Engine <
> appengine.nore...@google.com> wrote:
>
> Dear App Engine Developer,
>
> We’ve noticed that you are running at least one application configured to
> use the Master/Slave (M/S) datastore. This application configuration was 
> officially
> deprecatedon
>  April 4, 2012, in accordance with our deprecation
> policy , in
> favor of the more reliable High Replication Datastore (HRD). HRD uses the 
> Paxos
> algorithmto
>  serve your application out of multiple datacenters, meaning better
> redundancy in the face of datacenter issues, more consistent datastore
> performance, and no planned downtime.
>
> When we deprecated the M/S datastore, we introduced a migration 
> toolthat 
> allows you to easily migrate all your datastore and blobstore data to
> a new HRD application. Migrating your application will not require you to
> change your application’s URL, whether it serves from appspot.com or a
> custom domin. Please note that even if your application does not store any
> data in the datastore, it will still benefit from the automated datacenter
> failover that is only available to HRD applications.
>
> Before migrating your application, you should read about the differences
> between M/S and HRD, and understand how the consistency policy for 
> HRDmight
>  affect the queries in your application.
>
> All HRD applications that have billing enabled are covered by App Engine’s 
> 99.95%
> uptime SLA . Along with the
> substantial reliability improvements, many new App Engine features are only
> being made available to HRD applications, including the Python 2.7 language
> option, Full Text Search (FTS), and Page Speed integration.
>
> We strongly encourage you to migrate your applications as soon as
> possible. If you have technical questions about HRD or the migration
> process, you can post them to Stack 
> Overflow.
> Any general migration discussions can be posted to our Google 
> Group.
>
>
>
> Thank You,
>
> The App Engine Team
>
> © 2012 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043
> You have received this mandatory email service announcement to update you
> about important changes to your Google App Engine account.
>
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Error - 'module' object has no attribute '_has_repeated' - google datastore

[Ujjal Bhattacharjee]

Hi,

I am pretty new to google app engine and using the google datastore.

I have few files like 
Destination, Trip, Answer, Question (see attached). Whenever the app engine 
tries ti initialize the Trip object, I am getting the error below. Please 
help.

File "C:\ujjal\my project\cravel\code2\Trip.py", line 7, in 

class Trip(ndb.Model):

  File "C:\ujjal\my project\cravel\code2\Trip.py", line 10, in Trip

destinations = ndb.StructuredProperty(Destination1, repeated = True)

  File "C:\Program 
Files\Google\google_appengine\google\appengine\ext\ndb\utils.

py", line 136, in positional_wrapper

return wrapped(*args, **kwds)

  File "C:\Program 
Files\Google\google_appengine\google\appengine\ext\ndb\model.

py", line 1993, in __init__

if modelclass._has_repeated:

AttributeError: 'module' object has no attribute '_has_repeated'

INFO 2012-08-24 13:27:59,703 dev_appserver.py:2952] "GET /_edit/what 
HTTP/1.

1" 500 -






  

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/w6pF4kYYNRAJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

from google.appengine.ext import ndb
from google.appengine.api import memcache
import logging

import Destination1
import Trip


class Answer(ndb.Model):
	destinations = ndb.StructuredProperty(Destination1, repeated = True)
	trips = ndb.StructuredProperty(Trip)

	user_name = ndb.StringProperty()
	user_id = ndb.StringProperty()
	created = ndb.DateTimeProperty(auto_now_add = True)
	lastModified = ndb.DateTimeProperty(auto_now = True)
	
	
	def render(self):
	logging.error('in Answer render')
	   # self._render_text = self.content.replace('\n', '')
	return render_str("cravel-content-answer.html", a = self)
from google.appengine.ext import ndb
from google.appengine.api import memcache
import logging


class Destination1(ndb.Model):
	name = ndb.StringProperty()
	location = ndb.StringProperty()
	user_name = ndb.StringProperty()
	user_id = ndb.StringProperty()
	created = ndb.DateTimeProperty(auto_now_add = True)
	lastModified = ndb.DateTimeProperty(auto_now = True)
	
	def render(self):
	logging.error('in Destination render')
	   # self._render_text = self.content.replace('\n', '')
	return render_str("cravel-content.html", d = self)

	
	@classmethod
	def getDestinationByName(self, name):
		dest = Destination1.query().filter(Destination1.name == name)
		return dest
		
	@classmethod
	def getDestinationByLocation():
		dest = ndb.GqlQuery("");
		return dest
		
	@classmethod
	def getDestinationByTypeNearLocation():
		dest = ndb.GqlQuery("");
		return dest
		
	@classmethod
	def getAllDestinations(update=True):
		#key = 'top'
		#dests = memcache.get(key)
			
		#if dests is None or update:
		logging.error('DB Query')
		dests = Destination1.query() #Destination1.all().order('-created')
		#	memcache.set(key, dests)
		#else:
		#	logging.error('No DB Query')
		
		return dests
from google.appengine.ext import ndb
from google.appengine.api import memcache
import logging
import Answer

class Question(ndb.Model):
	question = ndb.StringProperty()
	answers = ndb.StructuredProperty(Answer)

	user_name = ndb.StringProperty()
	user_id = ndb.StringProperty()
	created = ndb.DateTimeProperty(auto_now_add = True)
	lastModified = ndb.DateTimeProperty(auto_now = True)
		
	def render(self):
	logging.error('in Question render')
	   # self._render_text = self.content.replace('\n', '')
	return render_str("cravel-content-question.html", q = self)

	
	@classmethod
	def getQuestionByName(self, question):
		questions = Question.query().filter(Question.question == question)
		return questions
		
	@classmethod
	def getAllQuestions(update=True):
		#key = 'top'
		#dests = memcache.get(key)
			
		#if dests is None or update:
		logging.error('DB Query')
		questions = Question.query()
		#	memcache.set(key, dests)
		#else:
		#	logging.error('No DB Query')
		
		return questions
from google.appengine.ext import ndb
from google.appengine.api import memcache
import logging
import Destination1


class Trip(ndb.Model):
	name = ndb.StringProperty()
	links = ndb.StringProperty()
	destinations = ndb.StructuredProperty(Destination1, repeated = True)

	user_name = ndb.StringProperty()
	user_id = ndb.StringProperty()
	created = ndb.DateTimeProperty(auto_now_add = True)
	lastModified = ndb.DateTimeProperty(auto_now = True)
		
	def render(self):
	logging.error('in Trip render')
	   # self._render_text = self.content.replace('\n', '')
	return render_str("cravel-content-trip.html", t = self)

	
	@classmethod
	def getTripByName(self, name):
		trips = Trip.query().fi

[google-appengine] MapperPipeline converting tupes to lists?

[jread]

I have been working on implementing a pipeline which will push data from 
AppEngine into BigQuery. I am defining my mapper pipeline class in the same 
manner as in this article 
https://developers.google.com/bigquery/articles/datastoretobigquery#definepipeline
 except 
that I am passing a "filters" param to the input_reader. Specifically, I am 
passing this:

params={
   "input_reader":{
   "entity_kind": entity_type,
   "filters": [('pid', '=', pid), ('tags', '=', '2000')]
   },
   "output_writer":{
   "filesystem": "gs",
   "gs_bucket_name": settings.GOOGLE_CLOUD_STORAGE_BUCKET_NAME,
   "output_sharding": "input"
   }
}

When I do that, however, lines 532 and 533 of mapreduce.input_readers.py 
are raising an exception stating that "Filter should be a tuple" and 
showing me the first filter but represented as a list instead of a tuple. 
Commenting out these two lines in input_readers.py allows the pipeline to 
proceed as expected and everything works properly. Am I doing something 
wrong here or is there a bug that is changing my tuples into lists?

Thanks,
Jeff.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/A_DW3_ahN6AJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Google App Engine Master/Slave Datastore Deprecation

[Joshua Smith]

Yeah, yeah, I know that many of my old M/S applications are running on the 
officially deprecated database. And I know that they might suffer a little more 
downtime as a result. But I also know that they are working just fine, and I'm 
not particularly interested in going through the pain of migrating 
(particularly working around all those annoying consistency things -- I've done 
that twice now, and each time it took quite a lot of programming to hide this 
stuff from the users).

So the question: Is this notice just a helpful reminder, or is there an 
implicit "or else" between the lines that I'm not seeing?

-Joshua

On Aug 24, 2012, at 9:10 AM, Google App Engine  
wrote:

> Dear App Engine Developer, 
> 
> We’ve noticed that you are running at least one application configured to use 
> the Master/Slave (M/S) datastore. This application configuration was 
> officially deprecated on April 4, 2012, in accordance with our deprecation 
> policy, in favor of the more reliable High Replication Datastore (HRD). HRD 
> uses the Paxos algorithm to serve your application out of multiple 
> datacenters, meaning better redundancy in the face of datacenter issues, more 
> consistent datastore performance, and no planned downtime. 
> 
> When we deprecated the M/S datastore, we introduced a migration tool that 
> allows you to easily migrate all your datastore and blobstore data to a new 
> HRD application. Migrating your application will not require you to change 
> your application’s URL, whether it serves from appspot.com or a custom domin. 
> Please note that even if your application does not store any data in the 
> datastore, it will still benefit from the automated datacenter failover that 
> is only available to HRD applications. 
> 
> Before migrating your application, you should read about the differences 
> between M/S and HRD, and understand how the consistency policy for HRD might 
> affect the queries in your application. 
> 
> All HRD applications that have billing enabled are covered by App Engine’s 
> 99.95% uptime SLA. Along with the substantial reliability improvements, many 
> new App Engine features are only being made available to HRD applications, 
> including the Python 2.7 language option, Full Text Search (FTS), and Page 
> Speed integration. 
> 
> We strongly encourage you to migrate your applications as soon as possible. 
> If you have technical questions about HRD or the migration process, you can 
> post them to Stack Overflow. Any general migration discussions can be posted 
> to our Google Group. 
> 
> 
> Thank You,
> 
> The App Engine Team 
> 
> © 2012 Google Inc. 1600 Amphitheatre Parkway, Mountain View, CA 94043 
> You have received this mandatory email service announcement to update you 
> about important changes to your Google App Engine account.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Duplicate app when you have too many apps

[Dmitry Kichinsky]

I'm having the same problem migrating to HRD:  not allowed to create 
anymore apps;reached app creation 
limit?
 

On Saturday, January 7, 2012 12:11:49 AM UTC+4, Ikai Lan (Google) wrote:
>
> Migrating to HRD using the new migration tool will create an alias, so 
> requests to the old app ID should go to the new one. You do still have to 
> create a new application ID, however.
>
> I've allocated a few more app IDs for your account. The current policy is 
> that you can have up to 10 free applications and an unlimited amount of 
> paid applications, but I think it's reasonable make exemptions on a 
> case-by-case basis for people who have been using GAE for a long time who 
> are migrating to HRD.
>
> --
> Ikai Lan 
> Developer Programs Engineer, Google App Engine
> plus.ikailan.com | twitter.com/ikai
>
>
>
> On Fri, Jan 6, 2012 at 11:27 AM, Ricardo Bánffy 
> > wrote:
>
>> Hi folks.
>>
>> Is there a way around the "not allowed to create anymore apps;reached
>> app creation limit?" message when you are trying to migrate an app to
>> HRD? BTW, is there a way to keep the app id?
>>
>> --
>> Ricardo Bánffy
>> http://www.dieblinkenlights.com
>> http://twitter.com/rbanffy
>>
>> --
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To post to this group, send email to 
>> google-a...@googlegroups.com
>> .
>> To unsubscribe from this group, send email to 
>> google-appengi...@googlegroups.com .
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/p1fMaoHMUxkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] The cronjobs keep coming every hour.

[aegerup]

Whats wrong when the cronjobs goes out every hour even thou it´s programmed to 
go out every week?
Has this happened to anyone else?
I even erased the email-adress and the jobs still keeps going out...
Please! Help me!

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/8qiTkgxfwBgJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] High Replication Datastore migration

[Swapnil Talekar]

Hi,
I am using Google CloudSQL for one of my projects and not . The appengine 
dashboard shows that I am using High Replication for storage of the app. 
But I have received an email from Google that I am using a Master/Slave 
datastore and should migrate to High Replication. 
Does this mean that I cannot use CloudSQL and have to use AppEngine (NoSQL) 
Datastore?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/fMmDfuSH01YJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: Re: [google-appengine] Re: App Engine Chat Time!

[John V Denley]

Why are we not using google+ hangouts for these sessions now?

On Wednesday, 6 July 2011 19:02:29 UTC+1, schuppe wrote:
>
> Yes, chat time is still there:
> """
> Every first and third Wednesday of the month, the App Engine team hosts 
> IRC Chat Time, an opportunity for you to get answers to your App 
> Engine-related questions in real-time.
>
> These chat sessions take place on the #appengine channel on
> irc.freenode.net.
>  
> For a list of IRC clients, see 
> http://en.wikipedia.org/wiki/List_of_IRC_clients
> .
>
> We welcome all App Engine-related questions and we will try to answer
> as many as we can in the hour session.
> """ - 
> https://www.google.com/calendar/event?eid=YzZscXRsNnJwMHQ4Zjl1dHNncHVrbDd2ZTRfMjAxMTA3MDdUMDIwMDAwWiBkZXZlbG9wZXItY2FsZW5kYXJAZ29vZ2xlLmNvbQ&gsessionid=OK
>
> Next one is today.
>
>
> On Mon, Jul 4, 2011 at 6:12 AM, TWiemann  >wrote:
>
>> Hi All,
>>
>> i've got a short question: does the Chat Time still exist? I'd like to 
>> join you and i hope to find some answers for some questions using Google 
>> App Engine.
>>
>> Thanks a lot!
>>
>> Theres
>> Germany
>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "Google App Engine" group.
>> To view this discussion on the web visit 
>> https://groups.google.com/d/msg/google-appengine/-/fnfte77NTo0J.
>>
>> To post to this group, send email to 
>> google-a...@googlegroups.com
>> .
>> To unsubscribe from this group, send email to 
>> google-appengi...@googlegroups.com .
>> For more options, visit this group at 
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/3jgqy06ZWU4J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Creating a game on app engine - calculating costs, thoughts?

[Richard Watson]

How large is numPlayersMax, typically?  And how big is the gameState field?

On Friday, August 24, 2012 12:37:37 AM UTC+2, Mark wrote:
>
> Hi,
>
> I'm trying to see if it's feasible to write a game using app engine as the 
> backend.
>
> The task I'm most worried about is the discoverability of games. A user 
> can create a new game (which creates a Game object in the datastore). Then 
> other users can hit an endpoint to get a listing of open games, sorted by 
> creation date.
>
> Creating a new game will be a very low frequency operation, so no worries 
> there. Users polling for new open games will be much higher in frequency 
> though. Each time a user requests a listing of games, I need to go fetch a 
> page of Game objects from the datastore. Let's say a page size = 20. And 
> the Game object looks like this:
>
> class Game {
> String name;
> long timestampCreated;
> long timestampStarted;
> String mapName;
> String hostUsername;
> int numPlayersMax;
> List playerNames; // usernames of other players currently 
> joined
> Text gameState; // game state as serialized json string
> }
>
> the query would be something like:
>
> select Games where timestampStarted = 0 
> and playerNames.size() < numPlayersMax 
> order by timestampCreated DESC limit 20;
> 
> First, I'm hoping it's possible to avoid loading the Text field in this 
> listings query since it's not needed for rendering yet. Still there would 
> be a fair number of fields to fetch and deserialize, and I'm afraid my 
> costs would quickly make it impractical.
>
> I could try keeping a list of open games in memcache, but not sure if I 
> could achieve consistency between the memcache state and the datastore 
> state of games. Will be looking into that.
>
> I've got a very similar game to this already published, using a single 
> jetty instance + mysql on one machine. It's starting to get too much 
> traffic  now, and scaling would be a pain. App engine's scalability looks 
> great for something like this, but not sure if the # of writes and 
> inefficient reads I need to perform are a good idea.
>
> Any general thoughts on this would be greatly appreciated. I'm starting to 
> dig into this now and any thoughts on how you'd implement the above would 
> be great. In my already-published game (mentioned above), I ran a dark test 
> where I'd call through to a (quickly implemented) mirror of the game 
> running on app engine. It quickly ate through the daily quota!
>
> Thanks
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/9xDhEod_2f8J.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Creating a game on app engine - calculating costs, thoughts?

[Rerngvit Yanggratoke]

I'm also looking into a similar problem. You can use Channel API to avoid
polling and thus unnecessary datastore operations. However, Channel API
also has limited free quota.


On Fri, Aug 24, 2012 at 5:37 AM, Mark  wrote:

> Hi,
>
> I'm trying to see if it's feasible to write a game using app engine as the
> backend.
>
> The task I'm most worried about is the discoverability of games. A user
> can create a new game (which creates a Game object in the datastore). Then
> other users can hit an endpoint to get a listing of open games, sorted by
> creation date.
>
> Creating a new game will be a very low frequency operation, so no worries
> there. Users polling for new open games will be much higher in frequency
> though. Each time a user requests a listing of games, I need to go fetch a
> page of Game objects from the datastore. Let's say a page size = 20. And
> the Game object looks like this:
>
> class Game {
> String name;
> long timestampCreated;
> long timestampStarted;
> String mapName;
> String hostUsername;
> int numPlayersMax;
> List playerNames; // usernames of other players currently
> joined
> Text gameState; // game state as serialized json string
> }
>
> the query would be something like:
>
> select Games where timestampStarted = 0
> and playerNames.size() < numPlayersMax
> order by timestampCreated DESC limit 20;
>
> First, I'm hoping it's possible to avoid loading the Text field in this
> listings query since it's not needed for rendering yet. Still there would
> be a fair number of fields to fetch and deserialize, and I'm afraid my
> costs would quickly make it impractical.
>
> I could try keeping a list of open games in memcache, but not sure if I
> could achieve consistency between the memcache state and the datastore
> state of games. Will be looking into that.
>
> I've got a very similar game to this already published, using a single
> jetty instance + mysql on one machine. It's starting to get too much
> traffic  now, and scaling would be a pain. App engine's scalability looks
> great for something like this, but not sure if the # of writes and
> inefficient reads I need to perform are a good idea.
>
> Any general thoughts on this would be greatly appreciated. I'm starting to
> dig into this now and any thoughts on how you'd implement the above would
> be great. In my already-published game (mentioned above), I ran a dark test
> where I'd call through to a (quickly implemented) mirror of the game
> running on app engine. It quickly ate through the daily quota!
>
> Thanks
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To view this discussion on the web visit
> https://groups.google.com/d/msg/google-appengine/-/cvpOxGrMRaoJ.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>



-- 
Best Regards,
Rerngvit Yanggratoke

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Weird Instance Scheduler

[Mos]

> A possible explanation could be that the traffic pattern had changed.

No. It's the same. Check for example the Request/Seconds statistics of my
application for the last 30 days!

>> It's very obvious that one instance should be enough for my application.
And that was almost the case the last months!
> Actually it's not true. In particular, check this log:

That's one expection where one client did 8 request in a minute  (+ one
pingdom). Nothing else this minute.
In those exceptional cases it could be ok if a second instance starts.
(Nevertheless can't one instance not
handle 8 requests a  minute?)

As I described:  Instances are started and stopped without reason, even if
less traffic per minute is available!

> * What is the purpose of max-pending-latency = 14.9 setting?

" is high App Engine will allow requests to wait rather than start new
Instances to process them"
--> One attempt to stop GAE to create unnecessary instances.

> * Can you try automatic-automatic for idle instances setting?

I played around with this the last days and nothing changed. As I wrote:  I
had those configuration for months and it worked fine 3-4 weeks ago!

> * What is the purpose of those pingdom check? What happens if you stop
that?

To be alerted if GAE is down a again. "What happens if you stop that?" -->
I wouldn't be angry anymore because I wouldn't recognize downtime's of my
GAE application. ;)

Please forward
http://code.google.com/p/googleappengine/issues/detail?id=8004  to the
relevant GAE deparment.

Thanks!


On Fri, Aug 24, 2012 at 1:39 AM, Takashi Matsuo  wrote:

>
> Hi Mos,
>
> On Thu, Aug 23, 2012 at 4:58 AM, Mos  wrote:
>
>> Does anybody else experience abnormal behavior of the instance-scheduler
>> the last three weeks (the last 7 days it got even worse)?  (Java / HRD)
>> Or does anybody has profound knowledge about it?
>>
>> Background:  My application is unchanged for weeks, configuration not
>> changed and application's traffic is constant.
>> Traffic: One request per minute from Pingdom and around 200 additional
>> pageviews the day (== around 1500 pageviews the day). The peek is not more
>> then 3-4 request per minute.
>>
>
> A possible explanation could be that the traffic pattern had changed.
>
>
>>
>> It's very obvious that one instance should be enough for my application.
>> And that was almost the case the last months!
>>
>
> Actually it's not true. In particular, check this log:
>
> https://appengine.google.com/logs?app_id=s~krisen-talk&version_id=1-0.360912144269287698&severity_level_override=1&severity_level=3&tz=Europe%2FBerlin&filter=&filter_type=regex&date_type=datetime&date=2012-08-23&time=23%3A57%3A00&limit=20&view=Search
>
> You can see the iPhone client repeatedly requests your dynamic resources
> in a very short amount of time. Presumably it's due to some kind of
> 'prefetch' feature of that device. Are you aware of those accesses, and
> that this access pattern can cause a new instance starting?
>
> I don't think this is the only reason, but this can explain that some
> portion of your loading requests are expected behavior.
>
> Now I'd like to ask you some questions.
>
>
> * What is the purpose of max-pending-latency = 14.9 setting?
> * Can you try automatic-automatic for idle instances setting?
> * What is the purpose of those pingdom check? What happens if you stop
> that?
>
>
>>
>> But now GAE creates most of the time 3 instances, whereby on has a long
>> life-time for days and the other ones are restarted around
>> 10 to 30 times the day.
>> Because load request takes between 30s to 40s  and requests are waiting
>> for loading instances, there are many request that
>> fail  (Users and Pingdom agree: *A request that takes more then a couple
>> of seconds is a failed request!*)
>>
>> Please check the attached screenshots that show the behavior!
>>
>> Note:
>> - Killing instances manually did not help
>> - Idle Instances were ( Automatic – 2 ) .  Changing it to whatever
>> didn't not change anything; e.g. like ( Automatic – 4 )
>>
>> Thanks and Cheers
>>
>> Mos
>>
>>
>>
>>
>>
>>
>>  --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appengine@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>
>
>
> --
> Takashi Matsuo | Developers Advocate | tmat...@google.com
>
>  --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appengine@googlegroups.com.
> To unsubscribe fr

[google-appengine] Backend and appstats does not function together vers 1.7.1 (Java)

[Oliver Billing]

When configuring a backend.xml I get the following error and the local 
instance wont start

 com.google.appengine.tools.appstats.Recorder cannot be cast to 
com.google.appengine.tools.development.ApiProxyLocal

Any known workarounds?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/vd4mo3iWiHYJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Queries hitting deadline

[timh]

If your count keeps increasing you will always run into some sort of time 
limit.  Why not consider doing this processing in a task (they can run for 
10mins, or multiple tasks. )  I am assuming your trying to summarise etc

T

On Friday, August 24, 2012 3:26:00 AM UTC+8, Phil wrote:
>
> In some initialization work my app needs to run through the all of the 
> datastore entities of a given kind.  I have a lot of these entities (80k 
> currently) and it's increasing rapidly. I'm currently trying to read these 
> in using a single datastore query, but running up against the default 
> datastore timeout of 30 seconds.
>
> Is there a good practice for sharding this or otherwise breaking this up 
> so that I won't hit these deadlines? I was thinking I would do a keyOnly 
> query and then break up the keys into a number of reasonably sized 
> sub-queries, but perhaps there is a better approach out there?
>
> Thanks,
> Phil
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To view this discussion on the web visit 
https://groups.google.com/d/msg/google-appengine/-/5x0WQc9ODVwJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.