[google-appengine] Re: Mail Service & Quarantine

[Grégoire Galinand]

No, i receive a mail from : [mydomain.com] 
with object : Quarantine Summary 1/19/2010

In the body, i've got :
These messages were quarantined as potential junk or virus-infected
messages, and have not been sent to your inbox. To deliver a message
to your inbox, click the Deliver link. To view the message in your
Message Center, click the Subject link.

and next, a table who contains a line by mail sended with my code and
for all, a link named Deliver. When i click on, the mail is deliver in
my mailbox.


I think this quantine mail is sended once a day or periodically. That
is why i've got a lot of mail test on the table.


Gregoire.

On 19 jan, 19:00, Robert Kluin  wrote:
> Do you mean the message is being marked as a spam?
>
> Robert
>
> 2010/1/19 Grégoire Galinand :
>
>
>
> > Hi,
>
> > I've got a problem with Java Low Level API. This is my code who is
> > launched in a doPost method :
>
> > MailService mailServ = MailServiceFactory.getMailService();
> > MailService.Message mess = new MailService.Message();
>
> > mess.setSender("gregoire.galin...@mydomain.com");
> > mess.setReplyTo("gregoire.galin...@mydomain.com");
> > mess.setTo("user.te...@mydomain.com");
> > mess.setSubject("Test Mail Service");
> > mess.setHtmlBody("HTML Test");
>
> > mailServ.send(mess);
>
> > When i execute this code, i'm logged with the account
> > gregoire.galin...@mydomain.com
> > This account is the only registered account in the developer's list on
> > the appengine console.
> > So my application is deployed on server.
>
> > The problem : the mail arrived on quanrantine summary mail in the
> > user.test1 mailbox.
>
> > Can you have a response to my problem ?
>
> > I'm french so excuse my english :)
>
> > --
> > You received this message because you are subscribed to the Google Groups 
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to 
> > google-appengine+unsubscr...@googlegroups.com.
> > For more options, visit this group 
> > athttp://groups.google.com/group/google-appengine?hl=en.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Eric Ka Ka Ng]

I have also installed this lib and am having fun with it. personally i
'feel' the overhead is really insignificant

refer to your diagram,

i'm not sure whether the blue bars are actually showing time without
the profiler, i think it may mean something else (i also wanna know
what the blue bar and red rectangle refer to, and will dig into the
doc to find out what it actually means. anyone know?). the overhead
should be the time labeled with 'overhead' (in your case, which is
0ms)

my personal experience is, for most calls, the 'overhead' is always
<1% of the 'real'

- eric

2010/1/20 Danny Tuppeny :
> I haven't used this in production yet (I only read that you can), though I'd
> avoid leaving it on for long periods if you don't need it.
> When running in production, it will show two bars - the one showing how much
> time was taken without the profiler, and one showing how long was actually
> taking (including the profiler overhead, since that's what you'll be charged
> with). There's a screenshot here:
> http://2.bp.blogspot.com/_DkVkyz39PB0/SyLYRJPlsgI/Deg/5FiuJi3SpjQ/s1600-h/スクリーンショット（2009-12-12+8.39.08）.png
> The overhead isn't insignificant, so you probably don't want to leave the
> profiler turned on, but the blue bars show the time without the profiler, so
> the results are still accurate :-)
> Danny
>
> 2010/1/20 Eric Ka Ka Ng 
>>
>> wow!! this is THE TOOL that I'm looking for long time!! previously i
>> have to write codes to do similar profiling and analysis with much
>> limited scales and functions. Now, THIS provides almost everything I
>> want already without any efforts from developers! I could say it is
>> one of the best tool on GAE. Great Work!!
>>
>> how about the overhead? from the doc I know we already can read from
>> the INFO log about its overhead for recording the RPC events and
>> storing it. for normal applications, what's your experiences or
>> expectations on the added overhead? (like <1% CPU time increase?) if
>> the overhead is relatively small, I think we can include it even in
>> the production system without noticeable performance impact.
>>
>>
>> many thanks,
>> eric
>>
>> 2010/1/19 mani doraisamy :
>> > I guess the reason why this is so damn good is because it gives an
>> > integrated picture across datastore, memcache and web request, without
>> > much overhead! Given that the variations are fixed (GAE's limitation
>> > of not allowing all libraries/api/rpc calls), this gives the full
>> > picture of the application.
>> >
>> > On Jan 19, 4:18 pm, mani doraisamy  wrote:
>> >> Thanks for the link, Danny. This is the best profiling tool i have
>> >> used. Well done, Guido!
>> >>
>> >> Will get back with the performance data for off-peak Vs peak hours.
>> >>
>> >> thanks,
>> >> mani
>> >>
>> >> On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>> >>
>> >>
>> >>
>> >> > Have you identified what part of your page is taking additional time?
>> >>
>> >> > Guido van Rossum wrote a library called Appstats to help profile time
>> >> > spent on API calls. I'd recommend setting it up (it works in
>> >> > production) to see if you can identify where this time is being
>> >> > spent.
>> >>
>> >> > I wrote a short article about Appstats on my blog with some
>> >> > screenshots showing the sort of data you can get out of it:
>> >>
>> >>
>> >> > >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>> >>
>> >> > Hope this helps.
>> >>
>> >> > Danny
>> >>
>> >> > On Jan 16, 8:24 pm, mani doraisamy 
>> >> > wrote:
>> >>
>> >> > > Recently, we have been running into frequent performance problems,
>> >> > > especially between 7-10pm IST. Requests that used to take 600ms are
>> >> > > taking almost 4 secs.
>> >>
>> >> > > - Has anyone faced similar problems recently?
>> >> > > - What is the limit for memcache per account? How do we find the
>> >> > > cache
>> >> > > expiry pattern? (actual expiry Vs specified expiry)
>> >>
>> >> > > thanks,
>> >> > > mani
>> >
>> > --
>> > You received this message because you are subscribed to the Google
>> > Groups "Google App Engine" group.
>> > To post to this group, send email to google-appeng...@googlegroups.com.
>> > To unsubscribe from this group, send email to
>> > google-appengine+unsubscr...@googlegroups.com.
>> > For more options, visit this group at
>> > http://groups.google.com/group/google-appengine?hl=en.
>> >
>> >
>> >
>> >
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this gr

[google-appengine] Mail inbound limitation.

[Chen Jie]

Is there any limitation of the total inbound mail messages? I didn't
found it through the Quotas document, there are only limits on mail
sent.
Or one inbound message is a Mail API call? can somebody explain it?
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Mail inbound limitation.

[Nick Johnson (Google)]

Hi Chen,

There are no direct limitations on the number of inbound emails you can
receive. Practical limitations are the number of requests you can handle and
your other resources.

-Nick Johnson

On Wed, Jan 20, 2010 at 9:53 AM, Chen Jie  wrote:

> Is there any limitation of the total inbound mail messages? I didn't
> found it through the Quotas document, there are only limits on mail
> sent.
> Or one inbound message is a Mail API call? can somebody explain it?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Mail Service & Quarantine

[Nick Johnson (Google)]

Hi Gregoire,

2010/1/20 Grégoire Galinand 

> No, i receive a mail from : [mydomain.com] 
> with object : Quarantine Summary 1/19/2010
>
> In the body, i've got :
> These messages were quarantined as potential junk or virus-infected
> messages, and have not been sent to your inbox. To deliver a message
> to your inbox, click the Deliver link. To view the message in your
> Message Center, click the Subject link.
>
> and next, a table who contains a line by mail sended with my code and
> for all, a link named Deliver. When i click on, the mail is deliver in
> my mailbox.
>

This 'quarantine' is not being performed by any Google service. You need to
contact the administrator of your mail system and determine why this is
happening and how you can prevent it.

-Nick Johnson


>
> I think this quantine mail is sended once a day or periodically. That
> is why i've got a lot of mail test on the table.
>
>
> Gregoire.
>
> On 19 jan, 19:00, Robert Kluin  wrote:
> > Do you mean the message is being marked as a spam?
> >
> > Robert
> >
> > 2010/1/19 Grégoire Galinand :
> >
> >
> >
> > > Hi,
> >
> > > I've got a problem with Java Low Level API. This is my code who is
> > > launched in a doPost method :
> >
> > > MailService mailServ = MailServiceFactory.getMailService();
> > > MailService.Message mess = new MailService.Message();
> >
> > > mess.setSender("gregoire.galin...@mydomain.com");
> > > mess.setReplyTo("gregoire.galin...@mydomain.com");
> > > mess.setTo("user.te...@mydomain.com");
> > > mess.setSubject("Test Mail Service");
> > > mess.setHtmlBody("HTML Test");
> >
> > > mailServ.send(mess);
> >
> > > When i execute this code, i'm logged with the account
> > > gregoire.galin...@mydomain.com
> > > This account is the only registered account in the developer's list on
> > > the appengine console.
> > > So my application is deployed on server.
> >
> > > The problem : the mail arrived on quanrantine summary mail in the
> > > user.test1 mailbox.
> >
> > > Can you have a response to my problem ?
> >
> > > I'm french so excuse my english :)
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> > > For more options, visit this group athttp://
> groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Blobstore API Feedback

[Nick Johnson (Google)]

Hi Piotr,

On Wed, Jan 20, 2010 at 7:33 AM, Piotr Sikora wrote:

> Hi Nick,
> thanks for reply, but I must disagree ;)
>
>
>  500 is the correct response to return in this case, as the URL the user
>> requested (your handler script) was found, while the blobkey returned by
>> your script was not valid.
>>
>
> You assume that there is some URL-to-blob_key mapping done, which is not
> always the case.
>

I don't assume any such thing. Your code asking the infrastructure to send
the user a non-existent blob is an issue with your code, not with the user's
request, and as such, it should return a 500 status code, not a 404.

-Nick Johnson


>
> For example, I made my Blobstore data available at "/blob/". At
> the moment I verify that  really exists by getting its BlobInfo
> and I return 404 when it doesn't, but in my opinion, this should be
> required.
>
> Also, since this is distributed system, race condidion is always possible,
> because concurrent request can delete  from the Blobstore at any
> time, which means that in such case:
>
> REQ1: client: GET /blob/valid_blob_key
> REQ1: handler: b = BlobInfo.get(valid_blob_key)
> REQ1: handler: b exists
>
> REQ2: client: GET /delete/valid_blob_key
> REQ2: handler: BlobInfo.delete(valid_blob_key)
> REQ2: handler: return 200 OK
> REQ2: client: 200 OK received
>
> REQ1: handler: set_header(BLOB_KEY_HEADER, b.key())
> REQ1: handler: return 200 OK
> REQ1: client: 500 Internal Server Error received
>
> client will receive 500 Internal Server Error with Google's default error
> page (not even ours).
>
>
>  Error codes should make sense from the client's
>> point of view, not the server's.
>>
>
> Yes, and that's exactly what I mean. From client's perspective data at
> "/url" was not found.
>
> Also, please note, that from all the things I wrote in my original e-mail,
> this is the smallest issue ;)
>
>
> Best regards,
> Piotr Sikora < piotr.sik...@frickle.com >
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Configuration issues - Google Apps account, GAE Application

[Alex]

Hi,
Three (related) questions!

(1)
Why am I unable to access the App Dashboard from the GAE Launcher
Dashboard link? Is this normal? Clicking 'Dashboard' directs me to a
google account sign in page, and so does not accept my not  google
apps sign in. However when I 'Deploy' I am, as I would expect,
prompted for my Google Apps sign in (that works fine).

(2)
Why does the dashboard in my Google Apps account not have my Google
Appengine application listed under Sites? Why do I have to navigate -
using the browser address bar -  to 
https://appengine.google.com/dashboard?&app_id=
to see it? It is as though these things are not tied together.

(3)
How, from https://appengine.google.com/dashboard?&app_id= can
I see the uploaded files in my application?

Having written my app - which I thought was the hard bit - and having
read and reread docs - not knowing the answers to these questions is
worrying. Where have I gone wrong?

Thanks
Alex
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Selling App Engine Apps

[Nick Johnson (Google)]

Hi Andrew,

On Wed, Jan 20, 2010 at 2:29 AM, Andrew Chilton wrote:

> Hi Brian,
>
> 2010/1/20 bFlood :
> > openid - I really wish Google would just build support for
> > FriendConnect directly into GAE. I don't have a good answer for you
> > right now, maybe someone else has done this already. that being said,
> > Nick built the very useful AEoid plugin: http://bit.ly/75CEy5
>
> Yep, I've seen Nick's OpenID and I do have a question for him
> (probably in a different channel somewhere). Essentially it's OpenID
> but of course the Google Apps Hosted OpenID has extended discovery
> which I don't think (or can't get) the Python OpenID library to do.
>

That's a good point. I'm not sure if the janrain openid library supports
XRDS. Feel free to file a bug against aeoid, especially if you discover more
details.

-Nick Johnson


>
> Anyone had any success doing this? Especially with the endpoint
> https://www.google.com/accounts/o8/site-xrds?hd=example.org
>
> > remote disable - [snip]
>
> Thanks for your descriptions Brian, they were pretty helpful
> (especially from someone actually doing it). I still really want to
> just have one app with proper login from their own Google Apps but
> still haven't come across a solution to do it!
>
> Cheers,
> Andy
>
> --
> contact: Andrew Chilton
> website: http://www.chilts.org/
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Configuration issues - Google Apps account, GAE Application

[Nick Johnson (Google)]

Hi,

On Wed, Jan 20, 2010 at 10:22 AM, Alex  wrote:

> Hi,
> Three (related) questions!
>
> (1)
> Why am I unable to access the App Dashboard from the GAE Launcher
> Dashboard link? Is this normal? Clicking 'Dashboard' directs me to a
> google account sign in page, and so does not accept my not  google
> apps sign in. However when I 'Deploy' I am, as I would expect,
> prompted for my Google Apps sign in (that works fine).
>

This is a known bug.


>
> (2)
> Why does the dashboard in my Google Apps account not have my Google
> Appengine application listed under Sites? Why do I have to navigate -
> using the browser address bar -  to
> https://appengine.google.com/dashboard?&app_id=
> to see it? It is as though these things are not tied together.
>

Google Apps and Google App Engine are separate products. Google Apps will
only list an App Engine app if it's been mapped to a subdomain of the apps
account you're viewing.


>
> (3)
> How, from https://appengine.google.com/dashboard?&app_id= can
> I see the uploaded files in my application?
>

Source download is not available. You need to keep a copy of your source
code yourself.

-Nick Johnson


> Having written my app - which I thought was the hard bit - and having
> read and reread docs - not knowing the answers to these questions is
> worrying. Where have I gone wrong?
>
> Thanks
> Alex
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Trouble viewing apps

[Nick Johnson (Google)]

Hi Ben,

You need to log in at http://appengine.google.com/a/freckleinteractive.net

-Nick Johnson

On Wed, Jan 20, 2010 at 1:39 AM, Ben  wrote:

> I know I've made google app engine applications before with my
> b...@freckleinteractive.net login. However, tonight when I try to login
> I get a request to verify my account, which I know I've already
> done...
>
> Any help would be greatly appreciated,
> - Ben
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: uploading 1MB file over slow connection - GAE request timeout

[Nick Johnson (Google)]

Hi Locke,

The same applies to downloads: The response is not sent to the user until
your script finishes executing, and thus the time the user takes to download
the response is not factored into your overall execution time.

-Nick Johnson

On Tue, Jan 19, 2010 at 9:37 PM, Locke  wrote:

> Mr. Milon,
>
> Mr. Johnson explained that UPLOADS do not start your 30s counter until
> they complete. My question is about DOWNLOADS in which your app begins
> execution and THEN sends data to the user, executing all the while.
>
> Please review the RFC 2616 if you are confused.
>
> On Jan 19, 3:54 pm, nickmilon  wrote:
> > Locke,
> > Wrong !  it will not as my experience tells and Nick above is
> > explaining
> >
> > Nick
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Blobstore API Feedback

[Piotr Sikora]

Hi Nick,


I don't assume any such thing. Your code asking the infrastructure to send
the user a non-existent blob is an issue with your code, not with the 
user's

request, and as such, it should return a 500 status code, not a 404.


did you read the race condition case? It can barely be issue with 
application code.


Best regards,
Piotr Sikora < piotr.sik...@frickle.com >

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Scalability problem in GAE

[mani doraisamy]

Here are the stats for the same request at 05:37 Vs 10:49 on
2010-01-19:
http://247-test.appspot.com/stats/details?time=1263908256446
http://247-test.appspot.com/stats/details?time=1263926946573

It just returns 2 entities. But the datastore query degrades from
256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
but not as much as datastore. Any suggestions?

thanks,
mani


On Jan 19, 4:18 pm, mani doraisamy  wrote:
> Thanks for the link, Danny. This is the best profiling tool i have
> used. Well done, Guido!
>
> Will get back with the performance data for off-peak Vs peak hours.
>
> thanks,
> mani
>
> On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>
>
>
> > Have you identified what part of your page is taking additional time?
>
> > Guido van Rossum wrote a library called Appstats to help profile time
> > spent on API calls. I'd recommend setting it up (it works in
> > production) to see if you can identify where this time is being spent.
>
> > I wrote a short article about Appstats on my blog with some
> > screenshots showing the sort of data you can get out of it:
>
> >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>
> > Hope this helps.
>
> > Danny
>
> > On Jan 16, 8:24 pm, mani doraisamy  wrote:
>
> > > Recently, we have been running into frequent performance problems,
> > > especially between 7-10pm IST. Requests that used to take 600ms are
> > > taking almost 4 secs.
>
> > > - Has anyone faced similar problems recently?
> > > - What is the limit for memcache per account? How do we find the cache
> > > expiry pattern? (actual expiry Vs specified expiry)
>
> > > thanks,
> > > mani
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Blobstore API Feedback

[Nick Johnson (Google)]

Hi Piotr,

On Wed, Jan 20, 2010 at 10:44 AM, Piotr Sikora wrote:

> Hi Nick,
>
>
>  I don't assume any such thing. Your code asking the infrastructure to send
>> the user a non-existent blob is an issue with your code, not with the
>> user's
>> request, and as such, it should return a 500 status code, not a 404.
>>
>
> did you read the race condition case? It can barely be issue with
> application code.


I did, and it's unfortunate - feel free to file a bug - but it's still far
from clear that this should result in returning a 404 to the user.

-Nick Johnson


>
>
> Best regards,
> Piotr Sikora < piotr.sik...@frickle.com >
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Nick Johnson, Developer Programs Engineer, App Engine
Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
368047
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Scalability problem in GAE

[mani doraisamy]

Sorry. I got the degradation wrong. Datastore degrades from 40ms to
1283ms.

thanks,
mani

On Jan 20, 3:48 pm, mani doraisamy  wrote:
> Here are the stats for the same request at 05:37 Vs 10:49 on
> 2010-01-19:http://247-test.appspot.com/stats/details?time=1263908256446http://247-test.appspot.com/stats/details?time=1263926946573
>
> It just returns 2 entities. But the datastore query degrades from
> 256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
> but not as much as datastore. Any suggestions?
>
> thanks,
> mani
>
> On Jan 19, 4:18 pm, mani doraisamy  wrote:
>
>
>
> > Thanks for the link, Danny. This is the best profiling tool i have
> > used. Well done, Guido!
>
> > Will get back with the performance data for off-peak Vs peak hours.
>
> > thanks,
> > mani
>
> > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>
> > > Have you identified what part of your page is taking additional time?
>
> > > Guido van Rossum wrote a library called Appstats to help profile time
> > > spent on API calls. I'd recommend setting it up (it works in
> > > production) to see if you can identify where this time is being spent.
>
> > > I wrote a short article about Appstats on my blog with some
> > > screenshots showing the sort of data you can get out of it:
>
> > >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>
> > > Hope this helps.
>
> > > Danny
>
> > > On Jan 16, 8:24 pm, mani doraisamy  wrote:
>
> > > > Recently, we have been running into frequent performance problems,
> > > > especially between 7-10pm IST. Requests that used to take 600ms are
> > > > taking almost 4 secs.
>
> > > > - Has anyone faced similar problems recently?
> > > > - What is the limit for memcache per account? How do we find the cache
> > > > expiry pattern? (actual expiry Vs specified expiry)
>
> > > > thanks,
> > > > mani
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Scalability problem in GAE

[mani doraisamy]

Same request, but another query has degraded from 45ms to 846ms:
http://247-test.appspot.com/stats/details?time=1263968821144
http://247-test.appspot.com/stats/details?time=1263908256446

thanks,
mani


On Jan 20, 4:00 pm, mani doraisamy  wrote:
> Sorry. I got the degradation wrong. Datastore degrades from 40ms to
> 1283ms.
>
> thanks,
> mani
>
> On Jan 20, 3:48 pm, mani doraisamy  wrote:
>
>
>
> > Here are the stats for the same request at 05:37 Vs 10:49 on
> > 2010-01-19:http://247-test.appspot.com/stats/details?time=1263908256446http://24...
>
> > It just returns 2 entities. But the datastore query degrades from
> > 256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
> > but not as much as datastore. Any suggestions?
>
> > thanks,
> > mani
>
> > On Jan 19, 4:18 pm, mani doraisamy  wrote:
>
> > > Thanks for the link, Danny. This is the best profiling tool i have
> > > used. Well done, Guido!
>
> > > Will get back with the performance data for off-peak Vs peak hours.
>
> > > thanks,
> > > mani
>
> > > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>
> > > > Have you identified what part of your page is taking additional time?
>
> > > > Guido van Rossum wrote a library called Appstats to help profile time
> > > > spent on API calls. I'd recommend setting it up (it works in
> > > > production) to see if you can identify where this time is being spent.
>
> > > > I wrote a short article about Appstats on my blog with some
> > > > screenshots showing the sort of data you can get out of it:
>
> > > >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>
> > > > Hope this helps.
>
> > > > Danny
>
> > > > On Jan 16, 8:24 pm, mani doraisamy  wrote:
>
> > > > > Recently, we have been running into frequent performance problems,
> > > > > especially between 7-10pm IST. Requests that used to take 600ms are
> > > > > taking almost 4 secs.
>
> > > > > - Has anyone faced similar problems recently?
> > > > > - What is the limit for memcache per account? How do we find the cache
> > > > > expiry pattern? (actual expiry Vs specified expiry)
>
> > > > > thanks,
> > > > > mani
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Nickolas Daskalou]

Hi mani,

I have a question. How did you get appstats to not require an admin login in
production?

Nick


2010/1/20 mani doraisamy 

> Same request, but another query has degraded from 45ms to 846ms:
> http://247-test.appspot.com/stats/details?time=1263968821144
> http://247-test.appspot.com/stats/details?time=1263908256446
>
> thanks,
> mani
>
>
> On Jan 20, 4:00 pm, mani doraisamy  wrote:
> > Sorry. I got the degradation wrong. Datastore degrades from 40ms to
> > 1283ms.
> >
> > thanks,
> > mani
> >
> > On Jan 20, 3:48 pm, mani doraisamy  wrote:
> >
> >
> >
> > > Here are the stats for the same request at 05:37 Vs 10:49 on
> > > 2010-01-19:
> http://247-test.appspot.com/stats/details?time=1263908256446http://24...
> >
> > > It just returns 2 entities. But the datastore query degrades from
> > > 256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
> > > but not as much as datastore. Any suggestions?
> >
> > > thanks,
> > > mani
> >
> > > On Jan 19, 4:18 pm, mani doraisamy  wrote:
> >
> > > > Thanks for the link, Danny. This is the best profiling tool i have
> > > > used. Well done, Guido!
> >
> > > > Will get back with the performance data for off-peak Vs peak hours.
> >
> > > > thanks,
> > > > mani
> >
> > > > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
> >
> > > > > Have you identified what part of your page is taking additional
> time?
> >
> > > > > Guido van Rossum wrote a library called Appstats to help profile
> time
> > > > > spent on API calls. I'd recommend setting it up (it works in
> > > > > production) to see if you can identify where this time is being
> spent.
> >
> > > > > I wrote a short article about Appstats on my blog with some
> > > > > screenshots showing the sort of data you can get out of it:
> >
> > > > >
> http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
> >
> > > > > Hope this helps.
> >
> > > > > Danny
> >
> > > > > On Jan 16, 8:24 pm, mani doraisamy 
> wrote:
> >
> > > > > > Recently, we have been running into frequent performance
> problems,
> > > > > > especially between 7-10pm IST. Requests that used to take 600ms
> are
> > > > > > taking almost 4 secs.
> >
> > > > > > - Has anyone faced similar problems recently?
> > > > > > - What is the limit for memcache per account? How do we find the
> cache
> > > > > > expiry pattern? (actual expiry Vs specified expiry)
> >
> > > > > > thanks,
> > > > > > mani
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Java, Query class, SELECT with || (logical OR) clause not working

[kattus]

Thank you for reply (I saw it only now..). I don't remember now which
version I was using,
I have already overcome the issue by changing the database schema.


On Dec 11 2009, 2:13 am, "Ikai L (Google)"  wrote:
> Are you running 1.2.6 SDK or 1.2.8? The issue has been resolved in 1.2.8.
>
>
>
>
>
> On Tue, Dec 8, 2009 at 11:38 AM, kattus  wrote:
> > Hi,
>
> > I am using the Java version of the GAE API, trying to make a query
> > such as:
>
> > SELECT FROM com.xxx.yyy.server.someclass WHERE (name == 'aaa' || name
> > == 'bbb') && accountId == 'test1'
>
> > I get UnsupportedDatastoreOperationException.
>
> > The problem seems to be in the || (logical OR) clause. Looking into
> > the documentation this should be supported (the OR operator is applied
> > to the same field, in this case 'name'):
>
> >http://code.google.com/appengine/docs/java/datastore/queriesandindexe...
>
> > Look at the "Query Filters" section.
>
> > Maybe I am missing something trivial.. the exception doesn't give any
> > additional information about what may be the problem, it is my guess
> > based on trial and error that the problem is in the OR operator.
>
> > Thank you.
>
> > --
>
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Ikai Lan
> Developer Programs Engineer, Google App Engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Blobstore API Feedback

[Piotr Sikora]

Hi Nick,


I did, and it's unfortunate - feel free to file a bug - but it's still far
from clear that this should result in returning a 404 to the user.


I don't think this is a bug, but I believe that 404 response is simply 
better and more obvious than 500 in this case and since Blobstore API is 
still "experimental" and "you were eager to get my feedback so that you 
could improve the API", I think it's good time to talk about this now ;)


Just take a look at competing solutions that work this way:
- S3 returns 404 for NoSuchBucket and NoSuchKey errors [1],
- Azure Blob Service returns 404 for ContainerNotFound and BlobNotFound 
errors [2],

- nginx returns 404 when file pointed by "X-Accel-Redirect" doesn't exist.

[1] 
http://docs.amazonwebservices.com/AmazonS3/latest/API/index.html?ErrorCodeList.html

[2] http://msdn.microsoft.com/en-us/library/dd179439.aspx

Best regards,
Piotr Sikora < piotr.sik...@frickle.com >

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Mail Service & Quarantine

[Grégoire Galinand]

Thanks ! Indeed, it came from postini service who generate this
quarantine !


thank you very much for your quick responses.

On 20 jan, 11:10, "Nick Johnson (Google)" 
wrote:
> Hi Gregoire,
>
> 2010/1/20 Grégoire Galinand 
>
> > No, i receive a mail from : [mydomain.com] 
> > with object : Quarantine Summary 1/19/2010
>
> > In the body, i've got :
> > These messages were quarantined as potential junk or virus-infected
> > messages, and have not been sent to your inbox. To deliver a message
> > to your inbox, click the Deliver link. To view the message in your
> > Message Center, click the Subject link.
>
> > and next, a table who contains a line by mail sended with my code and
> > for all, a link named Deliver. When i click on, the mail is deliver in
> > my mailbox.
>
> This 'quarantine' is not being performed by any Google service. You need to
> contact the administrator of your mail system and determine why this is
> happening and how you can prevent it.
>
> -Nick Johnson
>
>
>
>
>
>
>
> > I think this quantine mail is sended once a day or periodically. That
> > is why i've got a lot of mail test on the table.
>
> > Gregoire.
>
> > On 19 jan, 19:00, Robert Kluin  wrote:
> > > Do you mean the message is being marked as a spam?
>
> > > Robert
>
> > > 2010/1/19 Grégoire Galinand :
>
> > > > Hi,
>
> > > > I've got a problem with Java Low Level API. This is my code who is
> > > > launched in a doPost method :
>
> > > > MailService mailServ = MailServiceFactory.getMailService();
> > > > MailService.Message mess = new MailService.Message();
>
> > > > mess.setSender("gregoire.galin...@mydomain.com");
> > > > mess.setReplyTo("gregoire.galin...@mydomain.com");
> > > > mess.setTo("user.te...@mydomain.com");
> > > > mess.setSubject("Test Mail Service");
> > > > mess.setHtmlBody("HTML Test");
>
> > > > mailServ.send(mess);
>
> > > > When i execute this code, i'm logged with the account
> > > > gregoire.galin...@mydomain.com
> > > > This account is the only registered account in the developer's list on
> > > > the appengine console.
> > > > So my application is deployed on server.
>
> > > > The problem : the mail arrived on quanrantine summary mail in the
> > > > user.test1 mailbox.
>
> > > > Can you have a response to my problem ?
>
> > > > I'm french so excuse my english :)
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups "Google App Engine" group.
> > > > To post to this group, send email to google-appengine@googlegroups.com
> > .
> > > > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > > > For more options, visit this group athttp://
> > groups.google.com/group/google-appengine?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> Nick Johnson, Developer Programs Engineer, App Engine
> Google Ireland Ltd. :: Registered in Dublin, Ireland, Registration Number:
> 368047
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: "Request was aborted after waiting too long" followed by random DeadlineExceededError on import.

[Jason C]

I was under the impression that something happened internally at
Google to adjust the way that apps were balanced around machines and/
or other internal tuning.

Additionally, we run a ping every 10 seconds to keep an instance hot.
While I understand how this doesn't have much effect in a distributed
environment (though practically speaking in this case it does seem to
have a positive effect), and while I also understand how this is
"abuses" a shared resource, I'm currently afraid to turn it off.

j

On Jan 19, 8:10 pm, "Wesley Chun (Google)" 
wrote:
> dave, jason,
>
> just wanted to do a follow-up to see where things stand with your apps
> now. i'm coming across a similar user issue and was wondering whether
> it's the same problem or not. can you post your complete error stack
> traces if you're still running into this issue? here's the issue filed
> by the other user FYI, who's app seems to have few requests but each
> one has high latency:
>
> http://code.google.com/p/googleappengine/issues/detail?id=2621
>
> if your respective apps don't suffer from this problem any more, what
> did you do to resolve it or did it magically go away?
>
> thanks,
> -- wesley
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> "Core Python Programming", Prentice Hall, (c)2007,2001
> "Python Fundamentals", Prentice Hall, (c)2009
>    http://corepython.com
>
> wesley.j.chun :: wesc+...@google.com
> developer relations :: google app engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Blobstore upload gives 500 error when there is no file.

[Kenneth]

Hi,

If the user submits the form to the blobstore without a file I'm
getting a google 500 error.  This error is not reported in my logs.
This was working fine before Christmas so it is a regression.  If they
select a file then everything works.  Note that in our application the
file is optional, the user doesn't have to send a file.

Is anyone actually using the blobstore api? Please don't suggest
working around it by changing the action in the javascript.  Also,
this works fine in the dev server.

Same issue reported here:
http://groups.google.com/group/google-appengine/browse_thread/thread/128c5df63a819c2d/

Thanks!
Kenneth
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Scalability problem in GAE

[mani doraisamy]

Is that a technical question or process question?

On Jan 20, 4:35 pm, Nickolas Daskalou  wrote:
> Hi mani,
>
> I have a question. How did you get appstats to not require an admin login in
> production?
>
> Nick
>
> 2010/1/20 mani doraisamy 
>
>
>
> > Same request, but another query has degraded from 45ms to 846ms:
> >http://247-test.appspot.com/stats/details?time=1263968821144
> >http://247-test.appspot.com/stats/details?time=1263908256446
>
> > thanks,
> > mani
>
> > On Jan 20, 4:00 pm, mani doraisamy  wrote:
> > > Sorry. I got the degradation wrong. Datastore degrades from 40ms to
> > > 1283ms.
>
> > > thanks,
> > > mani
>
> > > On Jan 20, 3:48 pm, mani doraisamy  wrote:
>
> > > > Here are the stats for the same request at 05:37 Vs 10:49 on
> > > > 2010-01-19:
> >http://247-test.appspot.com/stats/details?time=1263908256446http://24...
>
> > > > It just returns 2 entities. But the datastore query degrades from
> > > > 256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
> > > > but not as much as datastore. Any suggestions?
>
> > > > thanks,
> > > > mani
>
> > > > On Jan 19, 4:18 pm, mani doraisamy  wrote:
>
> > > > > Thanks for the link, Danny. This is the best profiling tool i have
> > > > > used. Well done, Guido!
>
> > > > > Will get back with the performance data for off-peak Vs peak hours.
>
> > > > > thanks,
> > > > > mani
>
> > > > > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>
> > > > > > Have you identified what part of your page is taking additional
> > time?
>
> > > > > > Guido van Rossum wrote a library called Appstats to help profile
> > time
> > > > > > spent on API calls. I'd recommend setting it up (it works in
> > > > > > production) to see if you can identify where this time is being
> > spent.
>
> > > > > > I wrote a short article about Appstats on my blog with some
> > > > > > screenshots showing the sort of data you can get out of it:
>
> >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>
> > > > > > Hope this helps.
>
> > > > > > Danny
>
> > > > > > On Jan 16, 8:24 pm, mani doraisamy 
> > wrote:
>
> > > > > > > Recently, we have been running into frequent performance
> > problems,
> > > > > > > especially between 7-10pm IST. Requests that used to take 600ms
> > are
> > > > > > > taking almost 4 secs.
>
> > > > > > > - Has anyone faced similar problems recently?
> > > > > > > - What is the limit for memcache per account? How do we find the
> > cache
> > > > > > > expiry pattern? (actual expiry Vs specified expiry)
>
> > > > > > > thanks,
> > > > > > > mani
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Nickolas Daskalou]

Technical. I tried editing the appstats code to turn off this admin user
requirement, but I kept getting a 500 error.


2010/1/21 mani doraisamy 

> Is that a technical question or process question?
>
> On Jan 20, 4:35 pm, Nickolas Daskalou  wrote:
> > Hi mani,
> >
> > I have a question. How did you get appstats to not require an admin login
> in
> > production?
> >
> > Nick
> >
> > 2010/1/20 mani doraisamy 
> >
> >
> >
> > > Same request, but another query has degraded from 45ms to 846ms:
> > >http://247-test.appspot.com/stats/details?time=1263968821144
> > >http://247-test.appspot.com/stats/details?time=1263908256446
> >
> > > thanks,
> > > mani
> >
> > > On Jan 20, 4:00 pm, mani doraisamy  wrote:
> > > > Sorry. I got the degradation wrong. Datastore degrades from 40ms to
> > > > 1283ms.
> >
> > > > thanks,
> > > > mani
> >
> > > > On Jan 20, 3:48 pm, mani doraisamy 
> wrote:
> >
> > > > > Here are the stats for the same request at 05:37 Vs 10:49 on
> > > > > 2010-01-19:
> > >http://247-test.appspot.com/stats/details?time=1263908256446http://24.
> ..
> >
> > > > > It just returns 2 entities. But the datastore query degrades from
> > > > > 256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
> > > > > but not as much as datastore. Any suggestions?
> >
> > > > > thanks,
> > > > > mani
> >
> > > > > On Jan 19, 4:18 pm, mani doraisamy 
> wrote:
> >
> > > > > > Thanks for the link, Danny. This is the best profiling tool i
> have
> > > > > > used. Well done, Guido!
> >
> > > > > > Will get back with the performance data for off-peak Vs peak
> hours.
> >
> > > > > > thanks,
> > > > > > mani
> >
> > > > > > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
> >
> > > > > > > Have you identified what part of your page is taking additional
> > > time?
> >
> > > > > > > Guido van Rossum wrote a library called Appstats to help
> profile
> > > time
> > > > > > > spent on API calls. I'd recommend setting it up (it works in
> > > > > > > production) to see if you can identify where this time is being
> > > spent.
> >
> > > > > > > I wrote a short article about Appstats on my blog with some
> > > > > > > screenshots showing the sort of data you can get out of it:
> >
> > >
> http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
> >
> > > > > > > Hope this helps.
> >
> > > > > > > Danny
> >
> > > > > > > On Jan 16, 8:24 pm, mani doraisamy  >
> > > wrote:
> >
> > > > > > > > Recently, we have been running into frequent performance
> > > problems,
> > > > > > > > especially between 7-10pm IST. Requests that used to take
> 600ms
> > > are
> > > > > > > > taking almost 4 secs.
> >
> > > > > > > > - Has anyone faced similar problems recently?
> > > > > > > > - What is the limit for memcache per account? How do we find
> the
> > > cache
> > > > > > > > expiry pattern? (actual expiry Vs specified expiry)
> >
> > > > > > > > thanks,
> > > > > > > > mani
> >
> > > --
> > > You received this message because you are subscribed to the Google
> Groups
> > > "Google App Engine" group.
> > > To post to this group, send email to google-appengine@googlegroups.com
> .
> > > To unsubscribe from this group, send email to
> > > google-appengine+unsubscr...@googlegroups.com e...@googlegroups.com>
> > > .
> > > For more options, visit this group at
> > >http://groups.google.com/group/google-appengine?hl=en.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Engine and Gzip

[bFlood]

issue created here with Test Url:
http://code.google.com/p/googleappengine/issues/detail?id=2657


On Jan 19, 10:35 pm, "Wesley Chun (Google)" 
wrote:
> niklas does have a workaround, but some users are indeed running into
> this problem where there shouldn't be an issue, especially since it's
> documented that App Engine supports it, as detailed 
> here:http://code.google.com/appengine/kb/general.html#compression
>
> however, it's not a universal issue as i'm also using FF3 (Mac) but do
> NOT have this problem:
>
> REQUEST
>
> Host    XXX.appspot.com
> User-Agent      Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:
> 1.9.0.8) Gecko/2009032608 Firefox/3.0.8 GTB6
> Accept  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> Accept-Language en-us,en;q=0.5
> Accept-Encoding gzip,deflate
> Accept-Charset  ISO-8859-1,utf-8;q=0.7,*;q=0.7
> Keep-Alive      300
> Connection      keep-alive
> Cache-Control   max-age=0
>
> RESPONSE
>
> Content-Type    text/plain
> Cache-Control   public, max-age=0
> Content-Encoding        gzip
> Date    Tue, 19 Jan 2010 23:37:23 GMT
> Server  Google Frontend
> Content-Length  1571
>
> this means it's possible there's a problem generating the response.
> brian and philip, can you guys create an issue for this in the issue
> tracker? please reference this thread as well as brian's earlier
> thread in that issue, and i can pass it off to the appropriate team
> members to investigate. don't forget to star the thing once it's been
> created. (if you do a search, you'll find other gzip issues but
> they're all different than yours'.)
>
> thanks!
> -- wesley
> - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> "Core Python Programming", Prentice Hall, (c)2007,2001
> "Python Fundamentals", Prentice Hall, (c)2009
>    http://corepython.com
>
> wesley.j.chun :: wesc+...@google.com
> developer relations :: google app engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Robert Kluin]

I think I just edited the main function in appstats/ui.py.  I can not
recall how the original looked, but mine is now:

def main():
  """Main program.  Auth check, then create and run the WSGIApplication."""
  if not os.getenv('SERVER_SOFTWARE', '').startswith('Dev'):
if users.get_current_user() is None:
  print 'Status: 403'
  print
  print 'Forbidden'
  return
  app = webapp.WSGIApplication(URLMAP, debug=DEBUG)
  util.run_bare_wsgi_app(app)

I wanted to require login, but not admin to view the stats.

Robert







On Wed, Jan 20, 2010 at 10:46 AM, Nickolas Daskalou  wrote:
> Technical. I tried editing the appstats code to turn off this admin user
> requirement, but I kept getting a 500 error.
>
>
> 2010/1/21 mani doraisamy 
>>
>> Is that a technical question or process question?
>>
>> On Jan 20, 4:35 pm, Nickolas Daskalou  wrote:
>> > Hi mani,
>> >
>> > I have a question. How did you get appstats to not require an admin
>> > login in
>> > production?
>> >
>> > Nick
>> >
>> > 2010/1/20 mani doraisamy 
>> >
>> >
>> >
>> > > Same request, but another query has degraded from 45ms to 846ms:
>> > >http://247-test.appspot.com/stats/details?time=1263968821144
>> > >http://247-test.appspot.com/stats/details?time=1263908256446
>> >
>> > > thanks,
>> > > mani
>> >
>> > > On Jan 20, 4:00 pm, mani doraisamy  wrote:
>> > > > Sorry. I got the degradation wrong. Datastore degrades from 40ms to
>> > > > 1283ms.
>> >
>> > > > thanks,
>> > > > mani
>> >
>> > > > On Jan 20, 3:48 pm, mani doraisamy 
>> > > > wrote:
>> >
>> > > > > Here are the stats for the same request at 05:37 Vs 10:49 on
>> > > > > 2010-01-19:
>> >
>> > > >http://247-test.appspot.com/stats/details?time=1263908256446http://24...
>> >
>> > > > > It just returns 2 entities. But the datastore query degrades from
>> > > > > 256ms to 1760ms (6.8x degradation). Memcache also seems to
>> > > > > degrade,
>> > > > > but not as much as datastore. Any suggestions?
>> >
>> > > > > thanks,
>> > > > > mani
>> >
>> > > > > On Jan 19, 4:18 pm, mani doraisamy 
>> > > > > wrote:
>> >
>> > > > > > Thanks for the link, Danny. This is the best profiling tool i
>> > > > > > have
>> > > > > > used. Well done, Guido!
>> >
>> > > > > > Will get back with the performance data for off-peak Vs peak
>> > > > > > hours.
>> >
>> > > > > > thanks,
>> > > > > > mani
>> >
>> > > > > > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>> >
>> > > > > > > Have you identified what part of your page is taking
>> > > > > > > additional
>> > > time?
>> >
>> > > > > > > Guido van Rossum wrote a library called Appstats to help
>> > > > > > > profile
>> > > time
>> > > > > > > spent on API calls. I'd recommend setting it up (it works in
>> > > > > > > production) to see if you can identify where this time is
>> > > > > > > being
>> > > spent.
>> >
>> > > > > > > I wrote a short article about Appstats on my blog with some
>> > > > > > > screenshots showing the sort of data you can get out of it:
>> >
>> >
>> > > >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>> >
>> > > > > > > Hope this helps.
>> >
>> > > > > > > Danny
>> >
>> > > > > > > On Jan 16, 8:24 pm, mani doraisamy
>> > > > > > > 
>> > > wrote:
>> >
>> > > > > > > > Recently, we have been running into frequent performance
>> > > problems,
>> > > > > > > > especially between 7-10pm IST. Requests that used to take
>> > > > > > > > 600ms
>> > > are
>> > > > > > > > taking almost 4 secs.
>> >
>> > > > > > > > - Has anyone faced similar problems recently?
>> > > > > > > > - What is the limit for memcache per account? How do we find
>> > > > > > > > the
>> > > cache
>> > > > > > > > expiry pattern? (actual expiry Vs specified expiry)
>> >
>> > > > > > > > thanks,
>> > > > > > > > mani
>> >
>> > > --
>> > > You received this message because you are subscribed to the Google
>> > > Groups
>> > > "Google App Engine" group.
>> > > To post to this group, send email to
>> > > google-appeng...@googlegroups.com.
>> > > To unsubscribe from this group, send email to
>> > >
>> > > google-appengine+unsubscr...@googlegroups.com> > > e...@googlegroups.com>
>> > > .
>> > > For more options, visit this group at
>> > >http://groups.google.com/group/google-appengine?hl=en.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appen

[google-appengine] Re: Put data store operations use increasingly higher CPU as the number of entities in the entity group increases

[Lucian Baciu]

Could someone from Google please help!

Thanks,
Lucian

On Jan 20, 12:30 am, Lucian Baciu  wrote:
> Yes, in some cases I might fetch about three entities by their keys
> from the datastore. But I doubt this is the problem because even for a
> small entity like Label in my model I see the same high CPU on
> inserts. And, all get requests in my app are fast and have no
> problems.
>
> Here is the definition of this simpler entity:
>
> @PersistenceCapable(identityType = IdentityType.APPLICATION)
> public class Label {
>         @PrimaryKey
>     @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
>     private Key key;
>
>         @Persistent
>         private String label;
>
>         @Persistent
>         private Account account;
>
>     @Persistent
>     @Extension(vendorName="datanucleus", key="gae.parent-pk",
> value="true")
>     private Key accountKey;
>
>     @Persistent
>     private int contactsCounter;
>
> }
>
> And it has one custom index:
> label ▲
> Includes ancestors
>
> I know this custom index it's on one property and the query should
> have used the build in index but, the index definition was auto
> generated by app engine, maybe it's because it includes an ancestor.
>
> And still, even for this Label entity, if I insert a Label entity in a
> large group I see the same 40 second CPU usage...
>
> On Jan 20, 12:20 am, Robert Kluin  wrote:
>
>
>
> > Lucian,
> >   The only other thing I am wondering about is how much time it might
> > be taking to fetch and set those reference (key) properties.  Are you
> > fetching several entities to populate those fields?  I know that
> > should not be related to the size of your entity groups, but it might
> > be something else to look into.
>
> > Robert
>
> > On Tue, Jan 19, 2010 at 4:57 PM, Lucian Baciu  wrote:
> > > First of all, thank you both for helping me.
>
> > > I don't think I have an exploding index problem. Account entity has no
> > > custom indexes defined, and as far as I understand from the docs an
> > > exploding index means two or more list properties in the same custom
> > > index. No other entity in my model has two or more list properties in
> > > a custom index. If a custom index is not defined for those list
> > > properties in the same entity, the build in indexes do not have a
> > > exploding index problem either.
>
> > > My put operations look like, for example:
> > > Task dataTask = new Task();
> > > dataTask = pm.makePersistent(dataTask);
>
> > > The keys for all my entities are generated by the App Engine.
> > > Properties like attachedContact or responsibleContact are set with
> > > existing key values on the new Task entity being created.
>
> > > On Jan 19, 11:21 pm, Eli Jones  wrote:
> > >> Also, as Robert mentioned.. if you have any indexes defined on those List
> > >> properties for the Account Model, that could lead to an exploding index..
>
> > >> I'd say you have an exploding index somewhere.. I'm just not clear on 
> > >> where
> > >> since I don't know your program setup... and Java on App Engine isn't my
> > >> main squeeze.
>
> > >> On Tue, Jan 19, 2010 at 3:18 PM, Lucian Baciu  
> > >> wrote:
> > >> > I have a root entity Account:
>
> > >> > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > >> > public class Account implements Serializable{
> > >> >   �...@primarykey
> > >> >   �...@persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> > >> >    private Long id;
>
> > >> >       �...@persistent
> > >> >        private String title;
>
> > >> >       �...@persistent
> > >> >        private Date createdDate;
>
> > >> >       �...@persistent(mappedBy = "account")
> > >> >       �...@order(extensions = @Extension(vendorName="datanucleus", 
> > >> > key="list-
> > >> > ordering", value="key asc"))
> > >> >    private List contacts;
>
> > >> >       �...@persistent(mappedBy = "account")
> > >> >       �...@order(extensions = @Extension(vendorName="datanucleus", 
> > >> > key="list-
> > >> > ordering", value="key asc"))
> > >> >    private List labels;
>
> > >> >       �...@persistent(mappedBy = "account")
> > >> >       �...@order(extensions = @Extension(vendorName="datanucleus", 
> > >> > key="list-
> > >> > ordering", value="key asc"))
> > >> >    private List tasks;
>
> > >> >       �...@persistent(mappedBy = "account")
> > >> >       �...@order(extensions = @Extension(vendorName="datanucleus", 
> > >> > key="list-
> > >> > ordering", value="key asc"))
> > >> >    private List notes;
>
> > >> >    //other owned child collections
> > >> > }
>
> > >> > Task entity:
>
> > >> > @PersistenceCapable(identityType = IdentityType.APPLICATION)
> > >> > public class Task {
> > >> >       �...@primarykey
> > >> >   �...@persistent(valueStrategy = IdGeneratorStrategy.IDENTITY)
> > >> >    private Key key;
>
> > >> >       �...@persistent
> > >> >        private Text task;
>
> > >> >       �...@persistent
> > >> >        private Date dueDate;
>
> > >> >       �...@persistent
> > >> >        private Key cr

[google-appengine] Re: Scalability problem in GAE

[mani doraisamy]

It uses webapp (with routes) instead of Django and with custom
authentication. That's the reason, i guess.

thanks,
mani

On Jan 20, 8:46 pm, Nickolas Daskalou  wrote:
> Technical. I tried editing the appstats code to turn off this admin user
> requirement, but I kept getting a 500 error.
>
> 2010/1/21 mani doraisamy 
>
>
>
> > Is that a technical question or process question?
>
> > On Jan 20, 4:35 pm, Nickolas Daskalou  wrote:
> > > Hi mani,
>
> > > I have a question. How did you get appstats to not require an admin login
> > in
> > > production?
>
> > > Nick
>
> > > 2010/1/20 mani doraisamy 
>
> > > > Same request, but another query has degraded from 45ms to 846ms:
> > > >http://247-test.appspot.com/stats/details?time=1263968821144
> > > >http://247-test.appspot.com/stats/details?time=1263908256446
>
> > > > thanks,
> > > > mani
>
> > > > On Jan 20, 4:00 pm, mani doraisamy  wrote:
> > > > > Sorry. I got the degradation wrong. Datastore degrades from 40ms to
> > > > > 1283ms.
>
> > > > > thanks,
> > > > > mani
>
> > > > > On Jan 20, 3:48 pm, mani doraisamy 
> > wrote:
>
> > > > > > Here are the stats for the same request at 05:37 Vs 10:49 on
> > > > > > 2010-01-19:
> > > >http://247-test.appspot.com/stats/details?time=1263908256446http://24.
> > ..
>
> > > > > > It just returns 2 entities. But the datastore query degrades from
> > > > > > 256ms to 1760ms (6.8x degradation). Memcache also seems to degrade,
> > > > > > but not as much as datastore. Any suggestions?
>
> > > > > > thanks,
> > > > > > mani
>
> > > > > > On Jan 19, 4:18 pm, mani doraisamy 
> > wrote:
>
> > > > > > > Thanks for the link, Danny. This is the best profiling tool i
> > have
> > > > > > > used. Well done, Guido!
>
> > > > > > > Will get back with the performance data for off-peak Vs peak
> > hours.
>
> > > > > > > thanks,
> > > > > > > mani
>
> > > > > > > On Jan 17, 9:37 pm, Danny Tuppeny  wrote:
>
> > > > > > > > Have you identified what part of your page is taking additional
> > > > time?
>
> > > > > > > > Guido van Rossum wrote a library called Appstats to help
> > profile
> > > > time
> > > > > > > > spent on API calls. I'd recommend setting it up (it works in
> > > > > > > > production) to see if you can identify where this time is being
> > > > spent.
>
> > > > > > > > I wrote a short article about Appstats on my blog with some
> > > > > > > > screenshots showing the sort of data you can get out of it:
>
> >http://blog.dantup.com/2010/01/profiling-google-app-engine-with-appstats
>
> > > > > > > > Hope this helps.
>
> > > > > > > > Danny
>
> > > > > > > > On Jan 16, 8:24 pm, mani doraisamy 
> > > > wrote:
>
> > > > > > > > > Recently, we have been running into frequent performance
> > > > problems,
> > > > > > > > > especially between 7-10pm IST. Requests that used to take
> > 600ms
> > > > are
> > > > > > > > > taking almost 4 secs.
>
> > > > > > > > > - Has anyone faced similar problems recently?
> > > > > > > > > - What is the limit for memcache per account? How do we find
> > the
> > > > cache
> > > > > > > > > expiry pattern? (actual expiry Vs specified expiry)
>
> > > > > > > > > thanks,
> > > > > > > > > mani
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > Groups
> > > > "Google App Engine" group.
> > > > To post to this group, send email to google-appengine@googlegroups.com
> > .
> > > > To unsubscribe from this group, send email to
> > > > google-appengine+unsubscr...@googlegroups.com > > >  e...@googlegroups.com> > e...@googlegroups.com>
> > > > .
> > > > For more options, visit this group at
> > > >http://groups.google.com/group/google-appengine?hl=en.
>
> > --
> > You received this message because you are subscribed to the Google Groups
> > "Google App Engine" group.
> > To post to this group, send email to google-appeng...@googlegroups.com.
> > To unsubscribe from this group, send email to
> > google-appengine+unsubscr...@googlegroups.com > e...@googlegroups.com>
> > .
> > For more options, visit this group at
> >http://groups.google.com/group/google-appengine?hl=en.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Engine and Gzip

[Flips]

I have discovered that I get compressed content when I am at home. If
I am at work I do not get compressed content via app engine. We use a
dedicated hardware firewall, but I do not think that this is the
problem because I recieve the app engine dashboard compressed and
according to its response headers it is using the google frontend too.

On 20 Jan., 17:07, bFlood  wrote:
> issue created here with Test 
> Url:http://code.google.com/p/googleappengine/issues/detail?id=2657
>
> On Jan 19, 10:35 pm, "Wesley Chun (Google)" 
> wrote:
>
> > niklas does have a workaround, but some users are indeed running into
> > this problem where there shouldn't be an issue, especially since it's
> > documented that App Engine supports it, as detailed 
> > here:http://code.google.com/appengine/kb/general.html#compression
>
> > however, it's not a universal issue as i'm also using FF3 (Mac) but do
> > NOT have this problem:
>
> > REQUEST
>
> > Host    XXX.appspot.com
> > User-Agent      Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:
> > 1.9.0.8) Gecko/2009032608 Firefox/3.0.8 GTB6
> > Accept  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> > Accept-Language en-us,en;q=0.5
> > Accept-Encoding gzip,deflate
> > Accept-Charset  ISO-8859-1,utf-8;q=0.7,*;q=0.7
> > Keep-Alive      300
> > Connection      keep-alive
> > Cache-Control   max-age=0
>
> > RESPONSE
>
> > Content-Type    text/plain
> > Cache-Control   public, max-age=0
> > Content-Encoding        gzip
> > Date    Tue, 19 Jan 2010 23:37:23 GMT
> > Server  Google Frontend
> > Content-Length  1571
>
> > this means it's possible there's a problem generating the response.
> > brian and philip, can you guys create an issue for this in the issue
> > tracker? please reference this thread as well as brian's earlier
> > thread in that issue, and i can pass it off to the appropriate team
> > members to investigate. don't forget to star the thing once it's been
> > created. (if you do a search, you'll find other gzip issues but
> > they're all different than yours'.)
>
> > thanks!
> > -- wesley
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > "Core Python Programming", Prentice Hall, (c)2007,2001
> > "Python Fundamentals", Prentice Hall, (c)2009
> >    http://corepython.com
>
> > wesley.j.chun :: wesc+...@google.com
> > developer relations :: google app engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Engine and Gzip

[bFlood]

weird. after your post, I just tried my test url from an Amazon EC2
instance and it was gziped for both the sample browser User-Agent and
one set to "gzip"

Ran it from Fiddler so the only thing that could be different between
the EC2 instance and my work/home computers is the IP addresses.
(right?)

weird, seems flakey
brian

On Jan 20, 12:13 pm, Flips  wrote:
> I have discovered that I get compressed content when I am at home. If
> I am at work I do not get compressed content via app engine. We use a
> dedicated hardware firewall, but I do not think that this is the
> problem because I recieve the app engine dashboard compressed and
> according to its response headers it is using the google frontend too.
>
> On 20 Jan., 17:07, bFlood  wrote:
>
>
>
> > issue created here with Test 
> > Url:http://code.google.com/p/googleappengine/issues/detail?id=2657
>
> > On Jan 19, 10:35 pm, "Wesley Chun (Google)" 
> > wrote:
>
> > > niklas does have a workaround, but some users are indeed running into
> > > this problem where there shouldn't be an issue, especially since it's
> > > documented that App Engine supports it, as detailed 
> > > here:http://code.google.com/appengine/kb/general.html#compression
>
> > > however, it's not a universal issue as i'm also using FF3 (Mac) but do
> > > NOT have this problem:
>
> > > REQUEST
>
> > > Host    XXX.appspot.com
> > > User-Agent      Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US; rv:
> > > 1.9.0.8) Gecko/2009032608 Firefox/3.0.8 GTB6
> > > Accept  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> > > Accept-Language en-us,en;q=0.5
> > > Accept-Encoding gzip,deflate
> > > Accept-Charset  ISO-8859-1,utf-8;q=0.7,*;q=0.7
> > > Keep-Alive      300
> > > Connection      keep-alive
> > > Cache-Control   max-age=0
>
> > > RESPONSE
>
> > > Content-Type    text/plain
> > > Cache-Control   public, max-age=0
> > > Content-Encoding        gzip
> > > Date    Tue, 19 Jan 2010 23:37:23 GMT
> > > Server  Google Frontend
> > > Content-Length  1571
>
> > > this means it's possible there's a problem generating the response.
> > > brian and philip, can you guys create an issue for this in the issue
> > > tracker? please reference this thread as well as brian's earlier
> > > thread in that issue, and i can pass it off to the appropriate team
> > > members to investigate. don't forget to star the thing once it's been
> > > created. (if you do a search, you'll find other gzip issues but
> > > they're all different than yours'.)
>
> > > thanks!
> > > -- wesley
> > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > > "Core Python Programming", Prentice Hall, (c)2007,2001
> > > "Python Fundamentals", Prentice Hall, (c)2009
> > >    http://corepython.com
>
> > > wesley.j.chun :: wesc+...@google.com
> > > developer relations :: google app engine
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: permissions error adding app engine to google apps account

[Snaggle]

I encountered a similar situation, and scratched my head for a few
days.  Forums and searching turned up this thread as the most useful
resource I could find, but I still wasn't able to solve the issue.

After fiddling around for awhile, I discovered that when I created my
app-id, I had specified that authentication be restricted to members
of a particular domain.  The domain I specified was a Google Apps
powered domain I used for dev and testing of GAE apps that weren't
ready for prime time... but the domain I wanted to deploy on was a
different Google Apps powered domain.

Specifically, what tipped me off was a notice on the Application
Settings page in the online GAE management console: under the App
Title was the Authentication Options section with a notice saying
"When this application was created it was configured to allow anyone
with a valid [not-the-domain-i-wanted-to-deploy-on].com Google Apps
domain to sign in if the Google Accounts API is used for
authentication. Learn more".

Perhaps this message should actually say "This app was created with
Authentication restricted to users registered with Google Apps on
[domain].com.  This cannot be changed after the ApplicationID is
created.  If you require the ability to authenticate outside users,
you must use a different authentication method that the Google
Accounts API."

Basically, I'm suggesting anyone who comes here with this exact
problem - double check that the domain to which you restricted your
authentication is also where you wish to deploy. (Unfortunately, this
cannot be changed after initial app configuration...  I'm not sure if
that is a security feature or a manageability bug.)
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] I can't visite *.appspot.com from China, anybody knows why?

[Yiou]

I can't visite *.appspot.com from China, anybody knows why?

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Danny Tuppeny]

2010/1/20 Eric Ka Ka Ng 

> i'm not sure whether the blue bars are actually showing time without
> the profiler, i think it may mean something else (i also wanna know
> what the blue bar and red rectangle refer to, and will dig into the
> doc to find out what it actually means. anyone know?). the overhead
> should be the time labeled with 'overhead' (in your case, which is
> 0ms)
>

The screenshot I saw said blue was time taken and red was "time charged",
which I assumed meant the *actual* time taken with the overhead. It's
possible it meant that the timings were rounded for the costing, I don't
know. I fixed what I was doing, so I didn't investigate any further.
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Danny Tuppeny]

2010/1/20 Danny Tuppeny 

> 2010/1/20 Eric Ka Ka Ng 
>
>> i'm not sure whether the blue bars are actually showing time without
>>
>> the profiler, i think it may mean something else (i also wanna know
>> what the blue bar and red rectangle refer to, and will dig into the
>> doc to find out what it actually means. anyone know?). the overhead
>> should be the time labeled with 'overhead' (in your case, which is
>> 0ms)
>>
>
> The screenshot I saw said blue was time taken and red was "time charged",
> which I assumed meant the *actual* time taken with the overhead. It's
> possible it meant that the timings were rounded for the costing, I don't
> know. I fixed what I was doing, so I didn't investigate any further.
>

I just saw the screenshots posted above, which makes me think I'm wrong. It
looks more like the red bar is the time taken (and charged), but the blue
bar is the time the app spent blocked waiting for the return. Eg. your
script waited for 5 seconds for the response, but for some reason
(transmission time? blocking?) it was only 1 second of CPU in the datastore.

Hopefully someone knowledgeable might know for sure :-)
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Danny Tuppeny]

2010/1/20 mani doraisamy 

> Same request, but another query has degraded from 45ms to 846ms:
> http://247-test.appspot.com/stats/details?time=1263968821144
> http://247-test.appspot.com/stats/details?time=1263908256446
>

Which calls are you looking at? They look like different queries (with
different call stacks) from what I can see. You could try increasing
MAX_REPR to log more text on each line (such as query/response) to check
they're the same.

Also - Found some more info here:
http://docs.google.com/Doc?docid=0AQwBadvrXbuVZGZkM3p0czRfMTlkNjI1M2NjOQ&pli=1
Explains why you might be charged differently to what it actually took (eg.
if your query ran on a 486, it would take longer to run, but your charge
would still be based on the standard processor).
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Robert Kluin]

The blue line is wall-clock time.  The red line is CPU or API time.
As Danny mentioned, the blue line indicates "blocking" time of each
call.  The red line indicates what you are charged for each.

Robert








On Wed, Jan 20, 2010 at 12:43 PM, Danny Tuppeny  wrote:
> 2010/1/20 Danny Tuppeny 
>>
>> 2010/1/20 Eric Ka Ka Ng 
>>>
>>> i'm not sure whether the blue bars are actually showing time without
>>> the profiler, i think it may mean something else (i also wanna know
>>> what the blue bar and red rectangle refer to, and will dig into the
>>> doc to find out what it actually means. anyone know?). the overhead
>>> should be the time labeled with 'overhead' (in your case, which is
>>> 0ms)
>>
>> The screenshot I saw said blue was time taken and red was "time charged",
>> which I assumed meant the *actual* time taken with the overhead. It's
>> possible it meant that the timings were rounded for the costing, I don't
>> know. I fixed what I was doing, so I didn't investigate any further.
>
> I just saw the screenshots posted above, which makes me think I'm wrong. It
> looks more like the red bar is the time taken (and charged), but the blue
> bar is the time the app spent blocked waiting for the return. Eg. your
> script waited for 5 seconds for the response, but for some reason
> (transmission time? blocking?) it was only 1 second of CPU in the datastore.
> Hopefully someone knowledgeable might know for sure :-)
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: App Engine and Gzip

[Eli Jones]

I would say that you this issue here...

Your work firewall trusts *.google.com (which is where dashboard comes from)
but does not trust *.appspot.com (which is where the apps are running from).

Thus.. gzip headers get mangled for *.appspot.com traffic.. so the firewall
can scan the packets.

I think this is the issue that most people are facing who are complaining
that appspot is not gzipping traffic.

On Wed, Jan 20, 2010 at 12:13 PM, Flips  wrote:

> I have discovered that I get compressed content when I am at home. If
> I am at work I do not get compressed content via app engine. We use a
> dedicated hardware firewall, but I do not think that this is the
> problem because I recieve the app engine dashboard compressed and
> according to its response headers it is using the google frontend too.
>
> On 20 Jan., 17:07, bFlood  wrote:
> > issue created here with Test Url:
> http://code.google.com/p/googleappengine/issues/detail?id=2657
> >
> > On Jan 19, 10:35 pm, "Wesley Chun (Google)" 
> > 
> >
> > wrote:
> >
> > > niklas does have a workaround, but some users are indeed running into
> > > this problem where there shouldn't be an issue, especially since it's
> > > documented that App Engine supports it, as detailed here:
> http://code.google.com/appengine/kb/general.html#compression
> >
> > > however, it's not a universal issue as i'm also using FF3 (Mac) but do
> > > NOT have this problem:
> >
> > > REQUEST
> >
> > > HostXXX.appspot.com
> > > User-Agent  Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US;
> rv:
> > > 1.9.0.8) Gecko/2009032608 Firefox/3.0.8 GTB6
> > > Accept  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> > > Accept-Language en-us,en;q=0.5
> > > Accept-Encoding gzip,deflate
> > > Accept-Charset  ISO-8859-1,utf-8;q=0.7,*;q=0.7
> > > Keep-Alive  300
> > > Connection  keep-alive
> > > Cache-Control   max-age=0
> >
> > > RESPONSE
> >
> > > Content-Typetext/plain
> > > Cache-Control   public, max-age=0
> > > Content-Encodinggzip
> > > DateTue, 19 Jan 2010 23:37:23 GMT
> > > Server  Google Frontend
> > > Content-Length  1571
> >
> > > this means it's possible there's a problem generating the response.
> > > brian and philip, can you guys create an issue for this in the issue
> > > tracker? please reference this thread as well as brian's earlier
> > > thread in that issue, and i can pass it off to the appropriate team
> > > members to investigate. don't forget to star the thing once it's been
> > > created. (if you do a search, you'll find other gzip issues but
> > > they're all different than yours'.)
> >
> > > thanks!
> > > -- wesley
> > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > > "Core Python Programming", Prentice Hall, (c)2007,2001
> > > "Python Fundamentals", Prentice Hall, (c)2009
> > >http://corepython.com
> >
> > > wesley.j.chun :: wesc+...@google.com 
> > > developer relations :: google app engine
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: App Engine and Gzip

[Eli Jones]

Oh, and I mentioned this in my reply on the Issue page... but I'll put it
here so others can see the correct way to test.

Mainly, you should create a test page that just prints out what Browser
Headers it receives whenever someone visits it.

That way, you can verify if it's really receiving the gzip header.

On Wed, Jan 20, 2010 at 12:57 PM, Eli Jones  wrote:

> I would say that you this issue here...
>
> Your work firewall trusts *.google.com (which is where dashboard comes
> from) but does not trust *.appspot.com (which is where the apps are
> running from).
>
> Thus.. gzip headers get mangled for *.appspot.com traffic.. so the
> firewall can scan the packets.
>
> I think this is the issue that most people are facing who are complaining
> that appspot is not gzipping traffic.
>
> On Wed, Jan 20, 2010 at 12:13 PM, Flips  wrote:
>
>> I have discovered that I get compressed content when I am at home. If
>> I am at work I do not get compressed content via app engine. We use a
>> dedicated hardware firewall, but I do not think that this is the
>> problem because I recieve the app engine dashboard compressed and
>> according to its response headers it is using the google frontend too.
>>
>> On 20 Jan., 17:07, bFlood  wrote:
>> > issue created here with Test Url:
>> http://code.google.com/p/googleappengine/issues/detail?id=2657
>> >
>> > On Jan 19, 10:35 pm, "Wesley Chun (Google)" 
>> > 
>> >
>> > wrote:
>> >
>> > > niklas does have a workaround, but some users are indeed running into
>> > > this problem where there shouldn't be an issue, especially since it's
>> > > documented that App Engine supports it, as detailed here:
>> http://code.google.com/appengine/kb/general.html#compression
>> >
>> > > however, it's not a universal issue as i'm also using FF3 (Mac) but do
>> > > NOT have this problem:
>> >
>> > > REQUEST
>> >
>> > > HostXXX.appspot.com
>> > > User-Agent  Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US;
>> rv:
>> > > 1.9.0.8) Gecko/2009032608 Firefox/3.0.8 GTB6
>> > > Accept
>>  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
>> > > Accept-Language en-us,en;q=0.5
>> > > Accept-Encoding gzip,deflate
>> > > Accept-Charset  ISO-8859-1,utf-8;q=0.7,*;q=0.7
>> > > Keep-Alive  300
>> > > Connection  keep-alive
>> > > Cache-Control   max-age=0
>> >
>> > > RESPONSE
>> >
>> > > Content-Typetext/plain
>> > > Cache-Control   public, max-age=0
>> > > Content-Encodinggzip
>> > > DateTue, 19 Jan 2010 23:37:23 GMT
>> > > Server  Google Frontend
>> > > Content-Length  1571
>> >
>> > > this means it's possible there's a problem generating the response.
>> > > brian and philip, can you guys create an issue for this in the issue
>> > > tracker? please reference this thread as well as brian's earlier
>> > > thread in that issue, and i can pass it off to the appropriate team
>> > > members to investigate. don't forget to star the thing once it's been
>> > > created. (if you do a search, you'll find other gzip issues but
>> > > they're all different than yours'.)
>> >
>> > > thanks!
>> > > -- wesley
>> > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
>> > > "Core Python Programming", Prentice Hall, (c)2007,2001
>> > > "Python Fundamentals", Prentice Hall, (c)2009
>> > >http://corepython.com
>> >
>> > > wesley.j.chun :: wesc+...@google.com 
>> > > developer relations :: google app engine
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com
>> .
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>>
>>
>
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Scalability problem in GAE

[mani doraisamy]

Oops! The url seems to redirect to newer requests now. It no longer
refers to the same request that i had referred to.
Enhancement request for Appstats: Generating url with surrogate key
for each request (instead of time=1264004278534) would be of great
help for bookmarking/reference.

Anyway, i had checked the call stacks. The problem seems to be with
datastore requests. I am not really concerned about being billed right
now. The requests degrading significantly is the real problem. Any
suggestions?

pls let me know, if need another set of requests reproducing the same
problem.

thanks,
mani

On Jan 20, 10:51 pm, Danny Tuppeny  wrote:
> 2010/1/20 mani doraisamy 
>
> > Same request, but another query has degraded from 45ms to 846ms:
> >http://247-test.appspot.com/stats/details?time=1263968821144
> >http://247-test.appspot.com/stats/details?time=1263908256446
>
> Which calls are you looking at? They look like different queries (with
> different call stacks) from what I can see. You could try increasing
> MAX_REPR to log more text on each line (such as query/response) to check
> they're the same.
>
> Also - Found some more info 
> here:http://docs.google.com/Doc?docid=0AQwBadvrXbuVZGZkM3p0czRfMTlkNjI1M2N...
> Explains why you might be charged differently to what it actually took (eg.
> if your query ran on a 486, it would take longer to run, but your charge
> would still be based on the standard processor).
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: I can't visite *.appspot.com from China, anybody knows why?

[Wooble]

On Jan 20, 9:45 am, Yiou  wrote:
> I can't visite *.appspot.com from China, anybody knows why?

The Chinese government periodically blocks appspot.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] "Disable Billing" and "Change Budget" buttons missing on the Billing Settings page

[Johann C. Rocholl]

I want to increase the daily budget for my scoretool app, but the
buttons on the Billing Settings page are missing. They disappeared
several days ago. I am the only developer, and the billing
administrator. I have successfully adjusted the Billing Settings
before. My current daily budget is $5, but I want to double it to
prevent "503 Over Quota" errors. I have already posted the issue to
this group 2 days ago, but not received a response.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Robert Kluin]

Mani,
  Just a note. Guido's AppStats uses memcache to store the stats info
so it will sometimes disappear.  I think it also only stores the most
recent 100 (or something like that) requests.  You might want to take
screen-shots or something to be sure people will see what you want to
show them.

Robert







On Wed, Jan 20, 2010 at 1:25 PM, mani doraisamy
 wrote:
> Oops! The url seems to redirect to newer requests now. It no longer
> refers to the same request that i had referred to.
> Enhancement request for Appstats: Generating url with surrogate key
> for each request (instead of time=1264004278534) would be of great
> help for bookmarking/reference.
>
> Anyway, i had checked the call stacks. The problem seems to be with
> datastore requests. I am not really concerned about being billed right
> now. The requests degrading significantly is the real problem. Any
> suggestions?
>
> pls let me know, if need another set of requests reproducing the same
> problem.
>
> thanks,
> mani
>
> On Jan 20, 10:51 pm, Danny Tuppeny  wrote:
>> 2010/1/20 mani doraisamy 
>>
>> > Same request, but another query has degraded from 45ms to 846ms:
>> >http://247-test.appspot.com/stats/details?time=1263968821144
>> >http://247-test.appspot.com/stats/details?time=1263908256446
>>
>> Which calls are you looking at? They look like different queries (with
>> different call stacks) from what I can see. You could try increasing
>> MAX_REPR to log more text on each line (such as query/response) to check
>> they're the same.
>>
>> Also - Found some more info 
>> here:http://docs.google.com/Doc?docid=0AQwBadvrXbuVZGZkM3p0czRfMTlkNjI1M2N...
>> Explains why you might be charged differently to what it actually took (eg.
>> if your query ran on a 486, it would take longer to run, but your charge
>> would still be based on the standard processor).
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: App Engine and Gzip

[bFlood]

I believe eli is correct. the gzip header seems to stripped as it
leaves our network. we do have a hardware firewall, still need to find
out how its configured

thanks to everyone, sorry for all the fuss

On Jan 20, 1:13 pm, Eli Jones  wrote:
> Oh, and I mentioned this in my reply on the Issue page... but I'll put it
> here so others can see the correct way to test.
>
> Mainly, you should create a test page that just prints out what Browser
> Headers it receives whenever someone visits it.
>
> That way, you can verify if it's really receiving the gzip header.
>
>
>
> On Wed, Jan 20, 2010 at 12:57 PM, Eli Jones  wrote:
> > I would say that you this issue here...
>
> > Your work firewall trusts *.google.com (which is where dashboard comes
> > from) but does not trust *.appspot.com (which is where the apps are
> > running from).
>
> > Thus.. gzip headers get mangled for *.appspot.com traffic.. so the
> > firewall can scan the packets.
>
> > I think this is the issue that most people are facing who are complaining
> > that appspot is not gzipping traffic.
>
> > On Wed, Jan 20, 2010 at 12:13 PM, Flips  wrote:
>
> >> I have discovered that I get compressed content when I am at home. If
> >> I am at work I do not get compressed content via app engine. We use a
> >> dedicated hardware firewall, but I do not think that this is the
> >> problem because I recieve the app engine dashboard compressed and
> >> according to its response headers it is using the google frontend too.
>
> >> On 20 Jan., 17:07, bFlood  wrote:
> >> > issue created here with Test Url:
> >>http://code.google.com/p/googleappengine/issues/detail?id=2657
>
> >> > On Jan 19, 10:35 pm, "Wesley Chun (Google)" 
> >> > 
>
> >> > wrote:
>
> >> > > niklas does have a workaround, but some users are indeed running into
> >> > > this problem where there shouldn't be an issue, especially since it's
> >> > > documented that App Engine supports it, as detailed here:
> >>http://code.google.com/appengine/kb/general.html#compression
>
> >> > > however, it's not a universal issue as i'm also using FF3 (Mac) but do
> >> > > NOT have this problem:
>
> >> > > REQUEST
>
> >> > > Host    XXX.appspot.com
> >> > > User-Agent      Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10.5; en-US;
> >> rv:
> >> > > 1.9.0.8) Gecko/2009032608 Firefox/3.0.8 GTB6
> >> > > Accept
> >>  text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8
> >> > > Accept-Language en-us,en;q=0.5
> >> > > Accept-Encoding gzip,deflate
> >> > > Accept-Charset  ISO-8859-1,utf-8;q=0.7,*;q=0.7
> >> > > Keep-Alive      300
> >> > > Connection      keep-alive
> >> > > Cache-Control   max-age=0
>
> >> > > RESPONSE
>
> >> > > Content-Type    text/plain
> >> > > Cache-Control   public, max-age=0
> >> > > Content-Encoding        gzip
> >> > > Date    Tue, 19 Jan 2010 23:37:23 GMT
> >> > > Server  Google Frontend
> >> > > Content-Length  1571
>
> >> > > this means it's possible there's a problem generating the response.
> >> > > brian and philip, can you guys create an issue for this in the issue
> >> > > tracker? please reference this thread as well as brian's earlier
> >> > > thread in that issue, and i can pass it off to the appropriate team
> >> > > members to investigate. don't forget to star the thing once it's been
> >> > > created. (if you do a search, you'll find other gzip issues but
> >> > > they're all different than yours'.)
>
> >> > > thanks!
> >> > > -- wesley
> >> > > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> >> > > "Core Python Programming", Prentice Hall, (c)2007,2001
> >> > > "Python Fundamentals", Prentice Hall, (c)2009
> >> > >    http://corepython.com
>
> >> > > wesley.j.chun :: wesc+...@google.com 
> >> > > developer relations :: google app engine
>
> >> --
> >> You received this message because you are subscribed to the Google Groups
> >> "Google App Engine" group.
> >> To post to this group, send email to google-appeng...@googlegroups.com.
> >> To unsubscribe from this group, send email to
> >> google-appengine+unsubscr...@googlegroups.com >>  e...@googlegroups.com>
> >> .
> >> For more options, visit this group at
> >>http://groups.google.com/group/google-appengine?hl=en.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] full-text search of 10 million strings

[Mats]

Hi,

I'm making an app that will search through a database of 10 million
records (all strings less than 500 bytes). What I need is to return
the 100 strings that closest matches a search query. Would this be
feasible to do on App Engine with Compass or something like appengine-
search (http://www.billkatz.com/2009/6/Simple-Full-Text-Search-for-App-
Engine) ? I don't really need any writes to the search index while
it's on app engine so I could precompute it if necessary.

Mats
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] "Disable Billing" and "Change Budget" buttons missing on the Billing Settings page

[Ikai L (Google)]

Hi Johann,

Sorry for this delay. We'd really like Billing Issues to go through our
issue tracker and not the groups:
http://code.google.com/support/bin/request.py?contact_type=AppEngineBillingSupport

Can
you post the issue there?

On Wed, Jan 20, 2010 at 10:33 AM, Johann C. Rocholl wrote:

> I want to increase the daily budget for my scoretool app, but the
> buttons on the Billing Settings page are missing. They disappeared
> several days ago. I am the only developer, and the billing
> administrator. I have successfully adjusted the Billing Settings
> before. My current daily budget is $5, but I want to double it to
> prevent "503 Over Quota" errors. I have already posted the issue to
> this group 2 days ago, but not received a response.
>
> --
> You received this message because you are subscribed to the Google Groups
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to
> google-appengine+unsubscr...@googlegroups.com
> .
> For more options, visit this group at
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>


-- 
Ikai Lan
Developer Programs Engineer, Google App Engine
-- 

You received this message because you are subscribed to the Google Groups "Google App Engine" group.

To post to this group, send email to google-appeng...@googlegroups.com.

To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com.

For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Blobstore upload with empty file

[Hazzadous]

Sorry, missed these updates.  Yes, I am currently selection form
action based on if any file fields are non empty.  JavaScript, as you
allude to.

On Jan 5, 10:37 pm, Andy Lo-A-Foe  wrote:
> You can use JavaScript to check if the file form field is filled in
> before submitting.
>
> Regards,
> Andy
>
> On Jan 3, 11:02 pm, Hazzadous  wrote:
>
>
>
> > Having a little trouble using blobstore when file form fields are
> > empty.  I'm getting "Server Error" and nothing useful by the way of
> > debugging from this error or in the logs.  Handler as 
> > perhttp://codepad.org/TKnI0WuQandlive version 
> > athttp://rafikirecords-sandbox.appspot.com/upload_form
>
> > Can anyone point to a possible resolution to this problem?
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] full-text search of 10 million strings

[Barry Hunter]

Simple-Text-Search (or Searchable) would probably work unless you need
complex queries and/or proper ranking (but you say 'closest' so
possibly not)

>From what I understand of Compass etc, they pretty much need to load
the whole index into memory to use it, so are not going to work on
large corpus's.


2010/1/20 Mats :
> Hi,
>
> I'm making an app that will search through a database of 10 million
> records (all strings less than 500 bytes). What I need is to return
> the 100 strings that closest matches a search query. Would this be
> feasible to do on App Engine with Compass or something like appengine-
> search (http://www.billkatz.com/2009/6/Simple-Full-Text-Search-for-App-
> Engine) ? I don't really need any writes to the search index while
> it's on app engine so I could precompute it if necessary.
>
> Mats
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Billing enabled and then flips back to Free

[jread]

Hi. I'm trying to enable billing on a new application (id: steprep-
staging). After enabling it, the status in the dashboard changes to
'Activating Billing' but then after a little while if flips back to
'Free'. Does a certain amount of time need to pass before the enabling
takes effect or is there something wrong?

Thanks,
Jeff.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Cron jobs fail with error "Request was aborted after waiting too long to attempt to service your request."

[Xlorep DarkHelm]

Just as an FYI, I also have been seeing this bug happen over and over,
I have a cron job (that quite simply just drops off a task to be
processed into the default task queue), and it fails probably 60% of
the time, from casual glance at my logs. I originally just wanted this
cron to run once a day, but because of the high failure rate, I am
forced to make it run once an hour (and have code to gracefully end if
it already successfully ran within a day). Pretty frustrating to deal
with, and I'd appreciate if this problem was resolved.

I'm fairly confident in how I'm using the API as far as it is matching
the details in the documentation (the cron job is extraordinarily
simple).

My tasks also have this crop up, but they fortunately wait and try
again, which makes them work a lot better than the cron currently,
which just simply ends and refuses to try again without me using this
current work-around.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] View this page "Google App Engine Open Source Projects"

[Jason (Google)]

Added Objectify-Appengine

Click on 
http://groups.google.com/group/google-appengine/web/google-app-engine-open-source-projects
- or copy & paste it into your browser's address bar if that doesn't
work.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Re: Scalability problem in GAE

[Eric Ka Ka Ng]

yes exactly. I think this is the correct interpretation of the blue
line and red line.

- eric

2010/1/21 Robert Kluin :
> The blue line is wall-clock time.  The red line is CPU or API time.
> As Danny mentioned, the blue line indicates "blocking" time of each
> call.  The red line indicates what you are charged for each.
>
> Robert
>
>
>
>
>
>
>
>
> On Wed, Jan 20, 2010 at 12:43 PM, Danny Tuppeny  wrote:
>> 2010/1/20 Danny Tuppeny 
>>>
>>> 2010/1/20 Eric Ka Ka Ng 

 i'm not sure whether the blue bars are actually showing time without
 the profiler, i think it may mean something else (i also wanna know
 what the blue bar and red rectangle refer to, and will dig into the
 doc to find out what it actually means. anyone know?). the overhead
 should be the time labeled with 'overhead' (in your case, which is
 0ms)
>>>
>>> The screenshot I saw said blue was time taken and red was "time charged",
>>> which I assumed meant the *actual* time taken with the overhead. It's
>>> possible it meant that the timings were rounded for the costing, I don't
>>> know. I fixed what I was doing, so I didn't investigate any further.
>>
>> I just saw the screenshots posted above, which makes me think I'm wrong. It
>> looks more like the red bar is the time taken (and charged), but the blue
>> bar is the time the app spent blocked waiting for the return. Eg. your
>> script waited for 5 seconds for the response, but for some reason
>> (transmission time? blocking?) it was only 1 second of CPU in the datastore.
>> Hopefully someone knowledgeable might know for sure :-)
>> --
>> You received this message because you are subscribed to the Google Groups
>> "Google App Engine" group.
>> To post to this group, send email to google-appeng...@googlegroups.com.
>> To unsubscribe from this group, send email to
>> google-appengine+unsubscr...@googlegroups.com.
>> For more options, visit this group at
>> http://groups.google.com/group/google-appengine?hl=en.
>>
>>
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

Re: [google-appengine] Deploying more than 10 apps

[Nir Yariv]

Unfortunately these are apps I'm developing so I can't be added by
others - and I don't want to delete existing apps in order to add new
ones..

- Nir

2010/1/19 Piotr Jaroszyński :
>> Is there a way to get more than 10 apps working on the Google App
>> Engine, with billing or otherwise? I have 6 apps running and at least
>> 3 I'd deploy immediately if that wouldn't mean I max out my app
>> limit...
>
> Last time i checked apps that you have been added to by others didn't
> count to the limit. You can also delete apps, so I wouldn't hesitate
> at all.
>
> --
> Best Regards
> Piotr Jaroszyński
>
> --
> You received this message because you are subscribed to the Google Groups 
> "Google App Engine" group.
> To post to this group, send email to google-appeng...@googlegroups.com.
> To unsubscribe from this group, send email to 
> google-appengine+unsubscr...@googlegroups.com.
> For more options, visit this group at 
> http://groups.google.com/group/google-appengine?hl=en.
>
>
>
>
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] incoming XMPP down for app engine?

[Sam]

Looks like it has been down for a couple hours ..
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: incoming XMPP down for app engine?

[KAVIN]

Me too.My robot based on GAE went offline for a few hours.

On Jan 21, 12:14 pm, Sam  wrote:
> Looks like it has been down for a couple hours ..
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Help!! Infinite loop when trying to sign up for app engine

[Erem]

Hey guys,

I have entered an unescapable loop on the "Welcome to Google App
Engine" page when completing sign up for app engine. This is what
happens:

(1) I log in and arrive at the "Welcome to Google App Engine" page.
(2) As it requests, I click "Create an application.", fill out the
form, and click "Save", creating the app.
(3) It returns me to "Welcome to Google App Engine", and prompts me to
create an application.

Points of interest:
  * This is a new Google account associated with my company email
address. I've been using app engine personally for several months, and
it still works fine on that personal account.
  * The company's email is managed by Google Apps.
* the email address under which I'm signing up is ehboto at the
domain of cloudmedhealth (dot ) com
  * Once I have created an app, I can no longer create an app of the
same name, telling me that somewhere in the guts the app was actually
created but is simply not being presented in the dashboard.
  * This is reproduced in 3 separate browsers: FF, Chrome, Konqueror
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: Help!! Infinite loop when trying to sign up for app engine

[Erem]

Why do I feel sheepish?

I just saw the front page of the discussion group. Which described my
exact problem.

The only emergency was a failure to read directions.

*facepalm*

On Jan 20, 9:18 pm, Erem  wrote:
> Hey guys,
>
> I have entered an unescapable loop on the "Welcome to Google App
> Engine" page when completing sign up for app engine. This is what
> happens:
>
> (1) I log in and arrive at the "Welcome to Google App Engine" page.
> (2) As it requests, I click "Create an application.", fill out the
> form, and click "Save", creating the app.
> (3) It returns me to "Welcome to Google App Engine", and prompts me to
> create an application.
>
> Points of interest:
>   * This is a new Google account associated with my company email
> address. I've been using app engine personally for several months, and
> it still works fine on that personal account.
>   * The company's email is managed by Google Apps.
>     * the email address under which I'm signing up is ehboto at the
> domain of cloudmedhealth (dot ) com
>   * Once I have created an app, I can no longer create an app of the
> same name, telling me that somewhere in the guts the app was actually
> created but is simply not being presented in the dashboard.
>   * This is reproduced in 3 separate browsers: FF, Chrome, Konqueror
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: incoming XMPP down for app engine?

[Hani Naguib]

Me too
Posted about it 3-4 hours ago (on appengine java forum) , have not
heard anything from anyone.

-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] Re: auto run tasks using dev sdk

[Andy Freeman]

> Any other suggestions on how to easily test with background tasks
> using the SDK so they run automatically with hopefully more than 1
> running concurrently?

The development server does not support concurrent execution.

On Jan 13, 3:25 pm, Philip  wrote:
> There really needs to be an easier way to do local testing with queues
> and tasks doing what they do when an app is deployed.
>
> Any other suggestions on how to easily test with background tasks
> using the SDK so they run automatically with hopefully more than 1
> running concurrently?
>
> On Jan 13, 1:13 pm, "Wesley Chun (Google)" 
> wrote:
>
>
>
> > greetings!
>
> > you are correct. the Task queues in the development server are
> > controlled by a POST that you send from the Task Queues page of the
> > admin console. this is (mostly) desired by developers because they
> > have more control over when tasks get executed.
>
> > in order to run them automatically, you'll need to do some wizardry to
> > simulate those requests. in the Python world, you could probably do
> > one of these two things:
>
> > - a record-n-playback macro either via a tool like Selenium or
> > Windmill
> > - if you want a pure command-line script, check out the Mechanize
> > package which simulates a browser
>
> > let us know how you end up implementing it. does anyone else out there
> > have a different way of auto-executing tasks with the dev server?
>
> > cheers,
> > -- wesley
> > - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -
> > "Core Python Programming", Prentice Hall, (c)2007,2001
> > "Python Fundamentals", Prentice Hall, (c)2009
> >    http://corepython.com
>
> > wesley.j.chun :: wesc+...@google.com
> > developer relations :: google app engine
>
> > On Jan 11, 9:36 pm, Philip  wrote:
>
> > > I've read some articles on auto running tasks using the developer sdk
> > > and python. The articles I've found don't provide a good workaround. I
> > > am POSTing tasks using the Task class with parameters. I would like to
> > > run concurrent background tasks on my dev machine. If I can only run
> > > them single threaded, I guess that's OK for now.
>
> > > What type of shell script or equivalent should I write that will
> > > automatically run tasks by first discovering them in the queues and
> > > then submitting them as POSTs to the appropriate queue URLs with the
> > > original parameters?
>
> > > Note: The GAE SDK is running on OS X.- Hide quoted text -
>
> - Show quoted text -
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] What is wrong with my URL mapping? (app.yaml)

[hyn]

What I'm trying to do is very simple... point /users/registration/.*
to users.py.
In users.py, I have:

webapp.WSGIApplication([('/', RegistrationHandler)], debug = True)

And my app.yaml:

handlers:
- url: /static/stylesheets
  static_dir: static/stylesheets

- url: /users/registration/.*
  script: users.py

- url: /
  script: main.py

- url: .*
  static_files: static/pages/not_found.html
  upload: static/pages/not_found.html

Accessing /users/registration/ results in 404.
It doesn't actually show not_found.html, it just shows blank, with a
404 in the GAE launcher log.
If I change '/' to '/.*' in the parameter to webapp.WSGIApplication(),
the page loads. But I only want '/' to map to RegistrationHandler.
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.

[google-appengine] When is isAdmin() in the UserService returning true?

[dflorey]

Is there a rule when this method returns true?
On some Google Apps domains it works, on others not. Do I have to add
the admin as developer to the app?
-- 
You received this message because you are subscribed to the Google Groups 
"Google App Engine" group.
To post to this group, send email to google-appeng...@googlegroups.com.
To unsubscribe from this group, send email to 
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at 
http://groups.google.com/group/google-appengine?hl=en.