[google-appengine] Re: managing bulk emails from app engine app
Split the work into manageable chunks and create one TaskQueue task per chunk. TaskQueue jobs have a request time limit of ten minutes [0], which may well be enough to send all your emails. [0]: https://developers.google.com/appengine/docs/java/backends/overview Joakim On Saturday, July 7, 2012 7:19:12 PM UTC+2, Vivek Kumar wrote: Hello We have around 1 registered users with us who are blood donors. We want to send them periodic emails like every month to remind them or their account status etc. Obviously, a single request cannot process so many emails. So, what is the suggested way to handle this use case without hitting request time limits etc on app engine? our app is in GAE for java Vik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Oah95qLypI8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: managing bulk emails from app engine app
I find it most convenient to make the 'manageable chunk' one single email. It's a map/reduce-type problem. You want to map the enqueue email task across every user. The simplest solution is to iterate across the users and enqueue a task for each. You can do it in batches of 100 (actually QueueConstants.maxTasksPerAdd()); I find the Guava Lists.partition() and Lists.transform() methods really handy here. With a keys-only query you can probably iterate across 100k records in the ~60s before a query times out. If not, you can enqueue the mapping process. You could even use the google-provided MapReduce toolkit. Ultimately you end up with a zillion email this user tasks which you can run as parallel as you wish. Jeff On Sun, Jul 8, 2012 at 12:54 AM, Joakim joakim.edenh...@gmail.com wrote: Split the work into manageable chunks and create one TaskQueue task per chunk. TaskQueue jobs have a request time limit of ten minutes [0], which may well be enough to send all your emails. [0]: https://developers.google.com/appengine/docs/java/backends/overview Joakim On Saturday, July 7, 2012 7:19:12 PM UTC+2, Vivek Kumar wrote: Hello We have around 1 registered users with us who are blood donors. We want to send them periodic emails like every month to remind them or their account status etc. Obviously, a single request cannot process so many emails. So, what is the suggested way to handle this use case without hitting request time limits etc on app engine? our app is in GAE for java Vik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Oah95qLypI8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: No App Engine Connected Android Project option in Eclipse???
Hi Weston/Sriram, are you able to see the menu option 'App engine connected android project'? I'm having problems as described here: http://stackoverflow.com/questions/11276518/app-engine-connected-android-project-not-available/11382135#11382135 Thanks. On Sunday, April 15, 2012 9:31:43 PM UTC-7, Sriram Saroop (Google) wrote: FYI - for GPE (Google Plugin for eclipse) issues, you can discuss them in the GPE forumhttps://groups.google.com/forum/#!forum/google-plugin-eclipse (google-plugin-ecli...@googlegroups.com) - Sriram On Mon, Apr 16, 2012 at 9:33 AM, Sriram (Google) sar...@google.comwrote: Hi folks, 1) Can you check if this option exists : New-Other-Google - App Engine connected Android project ? 2) If not, under About Eclipse - Installation Details - Installed Software, check if the Google App Engine Tools for Android plugin has been installed. Also, make sure that you install ADT *before* you install GPE, otherwise the Google App Engine Tools for Android plugin won't be installed. The installation instructions are here: https://developers.google.com/eclipse/docs/appeng_android_install_setup#install Regards, Sriram On Sunday, 15 April 2012 06:49:17 UTC+5:30, David Scott wrote: Did you end up solving this? I'm having the same problem On Wednesday, 21 September 2011 21:50:47 UTC+10, Blue wrote: Hi! I'm trying to create an App Engine Connected Android Project in eclipse, but that option is missing in new project menu. (like here: http://bradabrams.com/2011/05/**google-io-session-overview-** android-app-engine-a-**developers-dream-combination/http://bradabrams.com/2011/05/google-io-session-overview-android-app-engine-a-developers-dream-combination/) I have updated all eclipse plugins, but still nothing. (GPE v2.4, ADT v12.0) Please help. Thanks, Gregor -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/wtsWCnphmigJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: managing bulk emails from app engine app
Hi Vik, technical details aside, keep in mind that the spam filter settings on App Engine can be pretty strict. We have been sending mails for 18 months now, increasing the number gradually, and we also used DKIM signing. Still, suddenly some time last week our mails stopped getting sent. From all I can tell some Google mechanism decided that we're a spammer. We have inquired with Google instantly, but of course the GAE customer service is poor as always and never got back to us. So we switched to an external email provider now (mailgun, but there are dozens out there), and I'm quite happy with that, since tracking of emails is a lot more convenient too. It's not free though. Long story short, if you all of a sudden send 10k mails, be prepared to get filtered by Google because they think you're a spammer. Make sure to spread the load, and track what you have sent already, so you know what you'd need to resend if push comes to shove. Cheers, Per On Saturday, July 7, 2012 7:19:12 PM UTC+2, Vivek Kumar wrote: Hello We have around 1 registered users with us who are blood donors. We want to send them periodic emails like every month to remind them or their account status etc. Obviously, a single request cannot process so many emails. So, what is the suggested way to handle this use case without hitting request time limits etc on app engine? our app is in GAE for java Vik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/CZHqGIOGAAUJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: managing bulk emails from app engine app
Offload both transactional and bulk email to a 3rd-party email specialist. It's their core competence, and you're likely to be able to send bulk email with one api call once your list is set up. If you have issues you'll have better insight into what those are. You can get delivery and read reports, and automatic remove-me-from-your-list handling. On Sunday, July 8, 2012 5:27:04 PM UTC+2, Per wrote: Hi Vik, technical details aside, keep in mind that the spam filter settings on App Engine can be pretty strict. We have been sending mails for 18 months now, increasing the number gradually, and we also used DKIM signing. Still, suddenly some time last week our mails stopped getting sent. From all I can tell some Google mechanism decided that we're a spammer. We have inquired with Google instantly, but of course the GAE customer service is poor as always and never got back to us. So we switched to an external email provider now (mailgun, but there are dozens out there), and I'm quite happy with that, since tracking of emails is a lot more convenient too. It's not free though. Long story short, if you all of a sudden send 10k mails, be prepared to get filtered by Google because they think you're a spammer. Make sure to spread the load, and track what you have sent already, so you know what you'd need to resend if push comes to shove. Cheers, Per On Saturday, July 7, 2012 7:19:12 PM UTC+2, Vivek Kumar wrote: Hello We have around 1 registered users with us who are blood donors. We want to send them periodic emails like every month to remind them or their account status etc. Obviously, a single request cannot process so many emails. So, what is the suggested way to handle this use case without hitting request time limits etc on app engine? our app is in GAE for java Vik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/m4NlskwCrF8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: managing bulk emails from app engine app
I should add that we also use a third-party email system. Ultimately it's no harder to make a REST call to an external service than to make a call to MailService, and many services offer a much more robust solution. GAE's mail system has silly limitations like the inability to send emails with inline (Content-ID) image attachments. Just make sure that your 'unit of work' is the same as a single urlfetch. You wouldn't want to put 100 REST calls in a single task - imagine it failed at #50, then the whole task will retry and re-spam #1-#49. If your mail service does send this message to these 100 people, great; otherwise stick to one-email-per-task. We do one-email-per-task. Jeff On Sun, Jul 8, 2012 at 8:57 AM, Richard Watson richard.wat...@gmail.com wrote: Offload both transactional and bulk email to a 3rd-party email specialist. It's their core competence, and you're likely to be able to send bulk email with one api call once your list is set up. If you have issues you'll have better insight into what those are. You can get delivery and read reports, and automatic remove-me-from-your-list handling. On Sunday, July 8, 2012 5:27:04 PM UTC+2, Per wrote: Hi Vik, technical details aside, keep in mind that the spam filter settings on App Engine can be pretty strict. We have been sending mails for 18 months now, increasing the number gradually, and we also used DKIM signing. Still, suddenly some time last week our mails stopped getting sent. From all I can tell some Google mechanism decided that we're a spammer. We have inquired with Google instantly, but of course the GAE customer service is poor as always and never got back to us. So we switched to an external email provider now (mailgun, but there are dozens out there), and I'm quite happy with that, since tracking of emails is a lot more convenient too. It's not free though. Long story short, if you all of a sudden send 10k mails, be prepared to get filtered by Google because they think you're a spammer. Make sure to spread the load, and track what you have sent already, so you know what you'd need to resend if push comes to shove. Cheers, Per On Saturday, July 7, 2012 7:19:12 PM UTC+2, Vivek Kumar wrote: Hello We have around 1 registered users with us who are blood donors. We want to send them periodic emails like every month to remind them or their account status etc. Obviously, a single request cannot process so many emails. So, what is the suggested way to handle this use case without hitting request time limits etc on app engine? our app is in GAE for java Vik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/m4NlskwCrF8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: managing bulk emails from app engine app
Hie Actually that was my worry as well like sending through app engine may mark emails spam and thus wasting our efforts. We ocassionaly send newsletter to all our subscribers and for that we are already using VerticalResponse.com which gives us 1 emails credit every month for free being a non profit. Right now, we want to handle use cases like: Sending email once a month to all the blood donors to verify/update their profile So, if we use any 3rd party system then any recommendations? If non -profit friendly then that would be awesome! Vik On Sunday, 8 July 2012 09:13:15 UTC-7, Jeff Schnitzer wrote: I should add that we also use a third-party email system. Ultimately it's no harder to make a REST call to an external service than to make a call to MailService, and many services offer a much more robust solution. GAE's mail system has silly limitations like the inability to send emails with inline (Content-ID) image attachments. Just make sure that your 'unit of work' is the same as a single urlfetch. You wouldn't want to put 100 REST calls in a single task - imagine it failed at #50, then the whole task will retry and re-spam #1-#49. If your mail service does send this message to these 100 people, great; otherwise stick to one-email-per-task. We do one-email-per-task. Jeff On Sun, Jul 8, 2012 at 8:57 AM, Richard Watson richard.wat...@gmail.com wrote: Offload both transactional and bulk email to a 3rd-party email specialist. It's their core competence, and you're likely to be able to send bulk email with one api call once your list is set up. If you have issues you'll have better insight into what those are. You can get delivery and read reports, and automatic remove-me-from-your-list handling. On Sunday, July 8, 2012 5:27:04 PM UTC+2, Per wrote: Hi Vik, technical details aside, keep in mind that the spam filter settings on App Engine can be pretty strict. We have been sending mails for 18 months now, increasing the number gradually, and we also used DKIM signing. Still, suddenly some time last week our mails stopped getting sent. From all I can tell some Google mechanism decided that we're a spammer. We have inquired with Google instantly, but of course the GAE customer service is poor as always and never got back to us. So we switched to an external email provider now (mailgun, but there are dozens out there), and I'm quite happy with that, since tracking of emails is a lot more convenient too. It's not free though. Long story short, if you all of a sudden send 10k mails, be prepared to get filtered by Google because they think you're a spammer. Make sure to spread the load, and track what you have sent already, so you know what you'd need to resend if push comes to shove. Cheers, Per On Saturday, July 7, 2012 7:19:12 PM UTC+2, Vivek Kumar wrote: Hello We have around 1 registered users with us who are blood donors. We want to send them periodic emails like every month to remind them or their account status etc. Obviously, a single request cannot process so many emails. So, what is the suggested way to handle this use case without hitting request time limits etc on app engine? our app is in GAE for java Vik -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/m4NlskwCrF8J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/Ae97GJ8XlaYJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: managing bulk emails from app engine app
suddenly some time last week our mails stopped getting sent. From all I can tell some Google mechanism decided that we're a spammer. We have inquired http://code.google.com/p/googleappengine/issues/detail?id=6181 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Login permission for some user
We need a new function in our app which only some users can login the panel. Can this possible with app yaml -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] unexplained request on appengine cuota detail
I have a page, I use it as an application on facebook. it have no users no visits, but quota details shows me 22% of frontend at the end of day. in logs there is no request. why? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/W_IpzzGOL7MJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Need help or consulting: Request was aborted errors
I have an app that produces: request was aborted errors. My users are
getting pissed. So am I. I thought App Engine was meant to scale ?
The app is running up to 40 instances to serve 500 users. All 500 users
will query the app within a 5 second window (like a mini-DDOS). It is
python 2.7 and marked as thread-safe. Why one instance can only serve
under 20 users, I don't know ! Average memory usage is 44MB on F1
instances.
Latency is set at max 500msec. I have tried setting idle instances to
20-Automatic. Billing has been around $30 for the last 12 hours. However
this is CRAZY. It cost me $30 to push out 1.25GB of data to under 500
users (500 is the peak!) ?!?
The stupid part is the code looks like this:
global GAME_DATA
global GAME_DATA_TIMESTAMP
delta = datetime.datetime.now() - GAME_DATA_TIMESTAMP
if delta.total_seconds() 5.:
# return cached data
game = GAME_DATA
else:
# get new
cg_ref = CurrentGame.get_by_key_name(current_game_key_name)
game = cg_ref.k
GAME_DATA = game
GAME_DATA_TIMESTAMP = datetime.datetime.now()
data = {'s': 'OK', 'g': game.board }
self.response.out.write(json.dumps(data))
Yes, that's right... we don't actually do 500 db queries...
At this point, I think there is some internal throttle with GAE that only
allows a limited number of http connections at once and just kills off the
rest or something.
Can only explain what is going on here, or point me to a competent
consultant who I can pay to tell me how to fix this ?
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/Bgk6iG2U7pkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Need help or consulting: Request was aborted errors
Since you have a multi-threaded app, the very first thing I would try
is switch to F4 instances. You may not need the memory, but
concurrency is theoretically limited by CPU utilization. You may find
that one F4 gives you better results than four F1s.
Jeff
On Sun, Jul 8, 2012 at 1:14 PM, Richard steven...@gmail.com wrote:
I have an app that produces: request was aborted errors. My users are
getting pissed. So am I. I thought App Engine was meant to scale ?
The app is running up to 40 instances to serve 500 users. All 500 users
will query the app within a 5 second window (like a mini-DDOS). It is
python 2.7 and marked as thread-safe. Why one instance can only serve under
20 users, I don't know ! Average memory usage is 44MB on F1 instances.
Latency is set at max 500msec. I have tried setting idle instances to
20-Automatic. Billing has been around $30 for the last 12 hours. However
this is CRAZY. It cost me $30 to push out 1.25GB of data to under 500 users
(500 is the peak!) ?!?
The stupid part is the code looks like this:
global GAME_DATA
global GAME_DATA_TIMESTAMP
delta = datetime.datetime.now() - GAME_DATA_TIMESTAMP
if delta.total_seconds() 5.:
# return cached data
game = GAME_DATA
else:
# get new
cg_ref = CurrentGame.get_by_key_name(current_game_key_name)
game = cg_ref.k
GAME_DATA = game
GAME_DATA_TIMESTAMP = datetime.datetime.now()
data = {'s': 'OK', 'g': game.board }
self.response.out.write(json.dumps(data))
Yes, that's right... we don't actually do 500 db queries...
At this point, I think there is some internal throttle with GAE that only
allows a limited number of http connections at once and just kills off the
rest or something.
Can only explain what is going on here, or point me to a competent
consultant who I can pay to tell me how to fix this ?
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/Bgk6iG2U7pkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Where can I download the GAE documentation for offline use?
it's in the SVN repo: http://code.google.com/p/googleappengine/source/browse/#svn%2Ftrunk%2Fdocs On Saturday, 7 July 2012 18:32:47 UTC+12, Edward Samson wrote: The old download site used to provide this link: http://googleappengine.googlecode.com/files/google-appengine-docs-20120228.zip But since the new site layout, there hasn't been any such download. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tCtpkhvNYaIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Need help or consulting: Request was aborted errors
Jeff: Thanks for the reponse/suggestion.
Here are one of my most common requests:
ms=30 cpu_ms=0 api_cpu_ms=0 cpm_usd=0.000263
Seems to be no problems with anything...
And here is another (different request):
ms=15376 cpu_ms=37 api_cpu_ms=17 cpm_usd=0.003428 pending_ms=8265
Seriously ? 15 seconds response ? Of which, 8 seconds sitting in pending ?
And what happened to the other 7 seconds ?
This does not seem to be a CPU issue (cpu_ms is super low!).
My customers requests are just sitting about doing nothing... and I don't know
enough to know where the latency is coming from! This really is very
frustrating.
I have also tried to limit the number of idle instances and watched the error
rate. There is NO correlation between the two. :(
-R
On Sunday, July 8, 2012 4:58:10 PM UTC-4, Jeff Schnitzer wrote:
Since you have a multi-threaded app, the very first thing I would try
is switch to F4 instances. You may not need the memory, but
concurrency is theoretically limited by CPU utilization. You may find
that one F4 gives you better results than four F1s.
Jeff
On Sun, Jul 8, 2012 at 1:14 PM, Richard steven...@gmail.com wrote:
I have an app that produces: request was aborted errors. My users are
getting pissed. So am I. I thought App Engine was meant to scale ?
The app is running up to 40 instances to serve 500 users. All 500 users
will query the app within a 5 second window (like a mini-DDOS). It is
python 2.7 and marked as thread-safe. Why one instance can only serve
under
20 users, I don't know ! Average memory usage is 44MB on F1 instances.
Latency is set at max 500msec. I have tried setting idle instances to
20-Automatic. Billing has been around $30 for the last 12 hours.
However
this is CRAZY. It cost me $30 to push out 1.25GB of data to under 500
users
(500 is the peak!) ?!?
The stupid part is the code looks like this:
global GAME_DATA
global GAME_DATA_TIMESTAMP
delta = datetime.datetime.now() - GAME_DATA_TIMESTAMP
if delta.total_seconds() 5.:
# return cached data
game = GAME_DATA
else:
# get new
cg_ref =
CurrentGame.get_by_key_name(current_game_key_name)
game = cg_ref.k
GAME_DATA = game
GAME_DATA_TIMESTAMP = datetime.datetime.now()
data = {'s': 'OK', 'g': game.board }
self.response.out.write(json.dumps(data))
Yes, that's right... we don't actually do 500 db queries...
At this point, I think there is some internal throttle with GAE that
only
allows a limited number of http connections at once and just kills off
the
rest or something.
Can only explain what is going on here, or point me to a competent
consultant who I can pay to tell me how to fix this ?
--
You received this message because you are subscribed to the Google
Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/Bgk6iG2U7pkJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/e8hIk5LfoUQJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Need help or consulting: Request was aborted errors
Richard, Another option would be to move the Game State request to a Gohttps://developers.google.com/appengine/docs/go/overview instance, either as a backendhttps://developers.google.com/appengine/docs/go/config/backendsor as a separate version. I believe a single Go instance should be able the handle 500 request / second. You could then share the Game State between the Python version and the Go version through Memcache, cacheing to instance memory every 5 sec. - Kyle -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tfYMKilbUNoJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Ownership verification not working with SNI certificates
Hi Ben, Many types of Google fetches, such as those done by Webmaster Tools, are not currently sending SNI details in their SSL handshakes and will therefore fail to connect. This is a known issue and we hope to have this resolved in the coming week. Thanks, Cayden Meyer Product Manager, Google App Engine On 4 July 2012 21:08, Ben.SaaSt supp...@saast.com wrote: SNI certificates are great and work mostly fine for me (thanks Google!), except for a couple of issues: 1. Google Webmaster Tools cannot find (and therefore verify) my domain ( https://app.saast.com) 2. Google (https://www.google.com/gadgets/directory/verify) cannot verify the ownership of a gadget on my site (https://app.saast.com/gadget.xml) All of this works correctly if a use the appspot url, so I think the problem might come from the SNI implementation in some Google servers. Am I doing something wrong? Thank you for your help. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/gdJ7uR6wmeMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Need help or consulting: Request was aborted errors
Sounds odd. I'd also suggest trying F4 (just for the fun of it, even if the
chance is low that you'll find something, it won't hurt to try it for a few
hours).
Did you check out appstats yet?
Also, in my experience the whole fiddling-with-sliders was never
productive, I'd suggest resetting to automatic/automatic values, it just
seemed more reliable to me.
You didn't mention if you had checked the quota details screen, there are a
couple of per-minute quotas that seem to bite people out of the blue. I
once had a backup-job that was stuck and cost me lots and lots of money, it
was running in the background on the built-in version, maybe something like
this can also contribute to throttling the core application?
Well, admittedly we're all poking in the dark here, maybe you can detail
what else you have tried and where you've looked already?
All the best
Per
On Sunday, July 8, 2012 10:14:05 PM UTC+2, Richard wrote:
I have an app that produces: request was aborted errors. My users are
getting pissed. So am I. I thought App Engine was meant to scale ?
The app is running up to 40 instances to serve 500 users. All 500 users
will query the app within a 5 second window (like a mini-DDOS). It is
python 2.7 and marked as thread-safe. Why one instance can only serve
under 20 users, I don't know ! Average memory usage is 44MB on F1
instances.
Latency is set at max 500msec. I have tried setting idle instances to
20-Automatic. Billing has been around $30 for the last 12 hours. However
this is CRAZY. It cost me $30 to push out 1.25GB of data to under 500
users (500 is the peak!) ?!?
The stupid part is the code looks like this:
global GAME_DATA
global GAME_DATA_TIMESTAMP
delta = datetime.datetime.now() - GAME_DATA_TIMESTAMP
if delta.total_seconds() 5.:
# return cached data
game = GAME_DATA
else:
# get new
cg_ref = CurrentGame.get_by_key_name(current_game_key_name)
game = cg_ref.k
GAME_DATA = game
GAME_DATA_TIMESTAMP = datetime.datetime.now()
data = {'s': 'OK', 'g': game.board }
self.response.out.write(json.dumps(data))
Yes, that's right... we don't actually do 500 db queries...
At this point, I think there is some internal throttle with GAE that only
allows a limited number of http connections at once and just kills off the
rest or something.
Can only explain what is going on here, or point me to a competent
consultant who I can pay to tell me how to fix this ?
--
You received this message because you are subscribed to the Google Groups
Google App Engine group.
To view this discussion on the web visit
https://groups.google.com/d/msg/google-appengine/-/nsTNa0Ub8PEJ.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Java compiler level vs. Installed project facet
I made a Java Google AppEngine application called Guestbook in Eclipse 3.7. However, I am encountering an error that lacks a quick fix: * * Description ResourcePathLocationType Java compiler level does not match the version of the installed Javaproject facet . Guestbook Unknown Faceted Project Problem (Java Version Mismatch) All the solutions I have found on Google hint at examining Project Properties | Facets. However, I lack a Project Facets tab under my project's properties. Why do I lack a project facet tab under properties? What is an installed Java project facet? How do I resolve this error? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/mU6LUEPQnY0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Need help or consulting: Request was aborted errors
I could if memcache actually worked. But it does not. I originally tried to use it and found that I could not push the game state to memcache and then have the other instances pull it. They would get versions of it that were up to 5 minutes old. My timings are 5 second windows. 5 secs to submit all scores 5 secs to reap scores and calc leaderboards 5 secs to fan out results to clients Experience shows that memcache is just broken for that sort of timing. As for using Go instead of Python. I am not sure I follow why Go should be better. The lag is not coming from CPU or queries. Right now I am running 50 instances to serve 500 game clients. $48 for the last 18 hours. 11% of my requests result in Request was aborted. Yeah, that is 12 THOUSAND fails. Back in the year 1995, ftp.cdrom.com could serve 2000 clients simultaneously on a Pentium Pro 200MHz and I cannot serve 20 clients on a 500Mhz virtual box ? I still contend there is some internal throttling going on somewhere. -R On Sunday, July 8, 2012 6:23:02 PM UTC-4, Kyle Finley wrote: Richard, Another option would be to move the Game State request to a Gohttps://developers.google.com/appengine/docs/go/overview instance, either as a backendhttps://developers.google.com/appengine/docs/go/config/backendsor as a separate version. I believe a single Go instance should be able the handle 500 request / second. You could then share the Game State between the Python version and the Go version through Memcache, cacheing to instance memory every 5 sec. - Kyle -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/SZqwMIIOZREJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Need help or consulting: Request was aborted errors
Not sure where I saw it, but I believe an application must respond within a second at most *on average*, or GAE reserves the right to throttle it. So, if you have some kind of slowdown inside your app, then requests start piling up. I'm not a Python developer, but if you had the equivalent of a synchronised map in there, with lots of concurrent access, then you might end up in a situation like this. Not sure about what's really to blame of course, but I'd strongly recommend setting up a controlled loadtesting environment (just copy the app, and start firing requests at it). Add plenty of logging to your app, enable appstats, and then slowly increase the load. Maybe even strip the application down, starting only with read access. I'm sure you will find something, and I would love to hear what it was! I wish you luck! On Monday, July 9, 2012 3:19:04 AM UTC+2, Richard wrote: I could if memcache actually worked. But it does not. I originally tried to use it and found that I could not push the game state to memcache and then have the other instances pull it. They would get versions of it that were up to 5 minutes old. My timings are 5 second windows. 5 secs to submit all scores 5 secs to reap scores and calc leaderboards 5 secs to fan out results to clients Experience shows that memcache is just broken for that sort of timing. As for using Go instead of Python. I am not sure I follow why Go should be better. The lag is not coming from CPU or queries. Right now I am running 50 instances to serve 500 game clients. $48 for the last 18 hours. 11% of my requests result in Request was aborted. Yeah, that is 12 THOUSAND fails. Back in the year 1995, ftp.cdrom.com could serve 2000 clients simultaneously on a Pentium Pro 200MHz and I cannot serve 20 clients on a 500Mhz virtual box ? I still contend there is some internal throttling going on somewhere. -R On Sunday, July 8, 2012 6:23:02 PM UTC-4, Kyle Finley wrote: Richard, Another option would be to move the Game State request to a Gohttps://developers.google.com/appengine/docs/go/overview instance, either as a backendhttps://developers.google.com/appengine/docs/go/config/backendsor as a separate version. I believe a single Go instance should be able the handle 500 request / second. You could then share the Game State between the Python version and the Go version through Memcache, cacheing to instance memory every 5 sec. - Kyle -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/iAnWI4OPNRsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Need help or consulting: Request was aborted errors
Yep, I know about that. The internal limit is 1000msec, but you should aim for 750msec just to 'be safe'. Here are my timings: ms=15376 cpu_ms=37 api_cpu_ms=17 cpm_usd=0.003428 pending_ms=8265 37 msec of CPU after waiting 8.2 seconds to execute. Some math: - 500 incoming requests - 40 servers = ~ 15 requests/server ... but each request takes ~55 msec of CPU + API That's 825 msec for THE WHOLE LOT ! So why on earth is there 8 second of waiting before that request was serviced ? -R On Sunday, July 8, 2012 10:13:28 PM UTC-4, Per wrote: Not sure where I saw it, but I believe an application must respond within a second at most *on average*, or GAE reserves the right to throttle it. So, if you have some kind of slowdown inside your app, then requests start piling up. I'm not a Python developer, but if you had the equivalent of a synchronised map in there, with lots of concurrent access, then you might end up in a situation like this. Not sure about what's really to blame of course, but I'd strongly recommend setting up a controlled loadtesting environment (just copy the app, and start firing requests at it). Add plenty of logging to your app, enable appstats, and then slowly increase the load. Maybe even strip the application down, starting only with read access. I'm sure you will find something, and I would love to hear what it was! I wish you luck! On Monday, July 9, 2012 3:19:04 AM UTC+2, Richard wrote: I could if memcache actually worked. But it does not. I originally tried to use it and found that I could not push the game state to memcache and then have the other instances pull it. They would get versions of it that were up to 5 minutes old. My timings are 5 second windows. 5 secs to submit all scores 5 secs to reap scores and calc leaderboards 5 secs to fan out results to clients Experience shows that memcache is just broken for that sort of timing. As for using Go instead of Python. I am not sure I follow why Go should be better. The lag is not coming from CPU or queries. Right now I am running 50 instances to serve 500 game clients. $48 for the last 18 hours. 11% of my requests result in Request was aborted. Yeah, that is 12 THOUSAND fails. Back in the year 1995, ftp.cdrom.com could serve 2000 clients simultaneously on a Pentium Pro 200MHz and I cannot serve 20 clients on a 500Mhz virtual box ? I still contend there is some internal throttling going on somewhere. -R On Sunday, July 8, 2012 6:23:02 PM UTC-4, Kyle Finley wrote: Richard, Another option would be to move the Game State request to a Gohttps://developers.google.com/appengine/docs/go/overview instance, either as a backendhttps://developers.google.com/appengine/docs/go/config/backendsor as a separate version. I believe a single Go instance should be able the handle 500 request / second. You could then share the Game State between the Python version and the Go version through Memcache, cacheing to instance memory every 5 sec. - Kyle -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/J0pwVLK7RoEJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Where can I download the GAE documentation for offline use?
Unfortunately, the version in the SVN repo is not current. Since we migrated our documentation to the new site, we temporarily stopped providing the current offline doc. We're working on it, and I hope we'll be able to provide the offline doc again soon. -- Takashi On Mon, Jul 9, 2012 at 6:30 AM, Emanuele Ziglioli theb...@emanueleziglioli.it wrote: it's in the SVN repo: http://code.google.com/p/googleappengine/source/browse/#svn%2Ftrunk%2Fdocs On Saturday, 7 July 2012 18:32:47 UTC+12, Edward Samson wrote: The old download site used to provide this link: http://googleappengine.**googlecode.com/files/google-** appengine-docs-20120228.ziphttp://googleappengine.googlecode.com/files/google-appengine-docs-20120228.zip But since the new site layout, there hasn't been any such download. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/tCtpkhvNYaIJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- Takashi Matsuo -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Log Filtering Broken
Has anyone noticed that Filtering in AdminConsole/Logs has been broken for a while? Basically after you enter a filter expression the log does NOT re-paginate results. Thus forcing you to page through bunch of empty pages. Star this issue: http://code.google.com/p/googleappengine/issues/detail?id=7669 -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/uk1P6INQitMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
