Re: [google-appengine] Re: Timeouts to Paypal
Oy, okay. I'll see if I can get to the bottom of this. As far as I know we're not doing anything to people accessing Paypal, so it might be something on their end. Have you guys asked in their developer forums? If so, can you link me to the thread so I can reach out to some folks at Paypal that I know? -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Tue, Dec 20, 2011 at 8:55 PM, Vivek Puri v...@vivekpuri.com wrote: I was still getting timeouts and finally put a proxy in place to avoid further issues. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/oJXkjJAhJA0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Timeouts to Paypal
Here is the link - https://www.x.com/developers/paypal/forums/general-developer-support/requests-google-appengine-timing-out -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/wR0HPmyuJrsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Timeouts to Paypal
Thanks Vivek. I'll see if there's a better long term solution for these problems. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Wed, Dec 21, 2011 at 11:16 AM, Vivek Puri v...@vivekpuri.com wrote: Here is the link - https://www.x.com/developers/paypal/forums/general-developer-support/requests-google-appengine-timing-out -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/wR0HPmyuJrsJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
RE: [google-appengine] Re: Timeouts to Paypal
They throttle by IP, and the Fetches IP pool isn't large enough for the number of PayPal Users. I don't think they like you much because they didn't seem interested in fixing the issue. I filed a support ticket. I suspect if you called and said Hi, this is Google you'd get farther than I did. From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Ikai Lan (Google) Sent: Wednesday, December 21, 2011 11:14 AM To: google-appengine@googlegroups.com Subject: Re: [google-appengine] Re: Timeouts to Paypal Oy, okay. I'll see if I can get to the bottom of this. As far as I know we're not doing anything to people accessing Paypal, so it might be something on their end. Have you guys asked in their developer forums? If so, can you link me to the thread so I can reach out to some folks at Paypal that I know? -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com http://plus.ikailan.com/ | twitter.com/ikai On Tue, Dec 20, 2011 at 8:55 PM, Vivek Puri v...@vivekpuri.com wrote: I was still getting timeouts and finally put a proxy in place to avoid further issues. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/oJXkjJAhJA0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com mailto:google-appengine%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
Is there any update to this? Apparently you can not call PayPal from the development server either. When I try to call api-3t.paypal.com I get SocketTimeoutExceptions 100% of the time from the development server. If I run the same URLFetch code (which I've implemented as java.net.URL HttpUrlConnection, etc) from a public static void main, it works just fine. How can this be Google? -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/cMjhdk6m3zAJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
Scratch the last comment. For anyone else having the problem, you'll need to set your connection/read timeout to 10+ seconds on URL fetch. I had it set to 5 seconds, and that is not enough. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/H6oYWhQR5AwJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
I was still getting timeouts and finally put a proxy in place to avoid further issues. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/oJXkjJAhJA0J. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
Hey, we had to set up a reverse NGINX proxy through AWS EC2 to communicate with the PayPal production server. We could never get it to work reliably directly from GAE. I don't know why. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine/-/obH2LepD1KMJ. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts to Paypal
Ikai This is the production Paypal. Timeouts are happening in 1 out of 10 urlfetch calls. Thank you On Dec 2, 1:03 pm, Ikai Lan (Google) ika...@google.com wrote: Vivek, is this to the Paypal sandbox or the real Paypal API? The sandbox has been observed as being pretty slow, but the real API is supposed to be much better. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Wed, Nov 30, 2011 at 1:49 PM, Brandon Wirtz drak...@digerat.com wrote: Pay pal throttles by IP, you should use their Call back method where you submit the request, and they hit a url when the transaction is processed. Also consider using a proxy to do the submission. -Original Message- From: google-appengine@googlegroups.com [mailto:google-appengine@googlegroups.com] On Behalf Of Vivek Puri Sent: Wednesday, November 30, 2011 1:28 PM To: Google App Engine Subject: [google-appengine] Timeouts to Paypal For the past couple weeks, we have been getting timeouts on urlfetch requests made to Paypal. We have set deadline of 55 seconds, and even then the request times out. If anyone else is also facing similar issue, please respond. Also, if anyone from GAE team has any ideas on this, please let me know. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
[google-appengine] Re: Timeouts have increased since maintenance on August 18th ?
For the past couple of days I've been seeing lots of datastore timeouts. The confusing thing is it seems to be application specific, as it's only one app that I'm seeing the problem, all the others appear to be running fine. On Aug 28, 6:58 am, Sylvain sylvain.viv...@gmail.com wrote: Hi, Today, I've checked the log for one of my app and I've noticed than one of my handler produces a lot of timeouts since the maintenance on August 18th. Just after the message Datastore writes are temporarily unavailable. is gone, a lot of timeouts are raised an now the number seems to be very high. Did you notice such behavior ? Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts have increased since maintenance on August 18th ?
I've seen a lot of timeouts today On Aug 28, 11:10 pm, Joe Bowman bowman.jos...@gmail.com wrote: For the past couple of days I've been seeing lots of datastore timeouts. The confusing thing is it seems to be application specific, as it's only one app that I'm seeing the problem, all the others appear to be running fine. On Aug 28, 6:58 am, Sylvain sylvain.viv...@gmail.com wrote: Hi, Today, I've checked the log for one of my app and I've noticed than one of my handler produces a lot of timeouts since the maintenance on August 18th. Just after the message Datastore writes are temporarily unavailable. is gone, a lot of timeouts are raised an now the number seems to be very high. Did you notice such behavior ? Regards --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: timeouts even after chunking datastore calls
Hi notcourage, The following lines attempt to iterate through all of the entities in a kind and deletes them one at a time. all = Thing.all() for x in all: ... Since you check the time to do a redirect, it looks like this should avoid the 30 second time limit. Do you know what kind of timeout error you are seeing (datastore vs request deadline)? One thing that might help is fetching a small number of entities per request and deleting just a few at a time. It might look something like: db.delete(Thing.all().fetch(10)) Note, you don't need to iterate over the results of the all query as delete will take a list of entities. Happy coding, Jeff On May 20, 9:54 am, notcourage klr...@gmail.com wrote: def clearDatastore(request): ti = time.clock() all = Thing.all() for x in all: x.delete() if time.clock() - ti = 8: return HttpResponseRedirect(/clearDatastore) Any ideas on why the above times out? Thx. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
I just got three timeouts about an hour ago within 10 minutes to my app (appid: brps), all generated same traceback. They happened at: 01-05 11:50AM 18.670 01-05 11:50AM 13.719 01-05 11:41AM 50.052 It's not a big deal to me, just want to provide the info. But I do have a question, can we catch `Timeout` exception? == One of the error log: 01-05 11:50AM 13.719 /get? blog=4589801858088630122post=430278962333729325callback=jsonp1231109286447_=1231109287404 500 11474ms 8190ms-cpu 1kb See details 65.210.123.XXX - - [05/Jan/2009:11:50:25 -0800] GET /get? blog=4589801858088630122post=430278962333729325callback=jsonp1231109286447_=1231109287404 HTTP/1.1 500 1115 - - 01-05 11:50AM 25.180 Traceback (most recent call last): File /base/python_lib/versions/1/google/appengine/ext/webapp/ __init__.py, line 498, in __call__ handler.get(*groups) File /base/data/home/apps/brps/1.330487559911947040/index.py, line 100, in get p = post.get(blog_id, post_id) File /base/data/home/apps/brps/1.330487559911947040/brps/post.py, line 75, in get p = Post.get_by_key_name(key_name) File /base/python_lib/versions/1/google/appengine/ext/db/ __init__.py, line 837, in get_by_key_name return get(*keys) File /base/python_lib/versions/1/google/appengine/ext/db/ __init__.py, line 1032, in get entities = datastore.Get(keys) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 217, in Get raise _ToDatastoreError(err) File /base/python_lib/versions/1/google/appengine/api/ datastore.py, line 1637, in _ToDatastoreError raise errors[err.application_error](err.error_detail) Timeout --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
Hi Aaron, I don't believe that saving an entity with a ReferenceProperty would be noticeably more expensive than using a unique string instead, the ReferenceProperty just saves a key of the referenced entity behind the scenes. It is possible, since you are doing a significant number of writes, that the issues you are seeing are related to datastore contention. Are many of these writes saving to the same entity group? If this is the case, keep in mind that writes to entity groups happen in serial, not parallel. Writing too often to the same entity group will definitely increase your time outs. If it is an issue with contention it would be probable that you would only notice it as traffic to your application increases. More traffic potentially means more possibility you are increasing your write frequency to the same entity group. -Marzia On Sat, Jan 3, 2009 at 9:46 AM, Aaron OBrien aobrien...@gmail.com wrote: My app serves a fair amount of requests about 4 a second and ruffly half of them save to the datastore. It is very fault tolerant so our users shouldn't be noticing it - and if they are it's still fairly minor. In the last 9 hours we have had 10% of these requests fail (about 13,700 requests). Prior to 2 days ago it was not uncommon to see the failure rate around 1% less than 100 failures for over 20 hours. The code that saves to the data store saves a model that has a reference to another object. This referenced object is pretty simple - StringProperty objects and a couple DateProperty objects. Is saving this reference more expensive than creating a uniqueId of the (simple)object and manually saving a the custom defined uniqueId to it? If this continues throughout the day I guess I'll turn profiling back on and see what's going on... Any thoughts on re-working the reference? Is it a more expensive operation to save and retrieve references to objects rather than a custom unqique key to the object? If I reworked the relationship it would force me to turn back around and query to get the referenced object when I need it though. Thanks for helping! -Aaron On Sat, Jan 3, 2009 at 2:56 AM, Paul Kinlan paul.kin...@gmail.com wrote: Hi, I also regularly get datastore timeouts for reads and writes. Looking at the status monitor there never seem to be any problems. Kind Regards, Paul Kinlan 2009/1/3 Xavier Mathews xavieruni...@gmail.com Place it in the issue tracker. On 01/02/2009, Alexander Kojevnikov alexan...@kojevnikov.com wrote: My application is getting a very high number of data store timeouts. Is there an unreported issue going on right now? The status monitor doesn't show anything unusual: http://code.google.com/status/appengine/detail/datastore/2009/01/02 -- Xavier A. Mathews Student/Browser Specialist/Developer/Web-Master Client Based Tech Support Specialist Hazel Crest Illinois xavieruni...@gmail.com¥xavierunited@hotmail.com¥truestar...@yahoo.com Fear of a name, only increases fear of the thing itself. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
Hi, I also regularly get datastore timeouts for reads and writes. Looking at the status monitor there never seem to be any problems. Kind Regards, Paul Kinlan 2009/1/3 Xavier Mathews xavieruni...@gmail.com Place it in the issue tracker. On 01/02/2009, Alexander Kojevnikov alexan...@kojevnikov.com wrote: My application is getting a very high number of data store timeouts. Is there an unreported issue going on right now? The status monitor doesn't show anything unusual: http://code.google.com/status/appengine/detail/datastore/2009/01/02 -- Xavier A. Mathews Student/Browser Specialist/Developer/Web-Master Client Based Tech Support Specialist Hazel Crest Illinois xavieruni...@gmail.com¥xavierunited@hotmail.com¥truestar...@yahoo.com Fear of a name, only increases fear of the thing itself. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
My app serves a fair amount of requests about 4 a second and ruffly half of them save to the datastore. It is very fault tolerant so our users shouldn't be noticing it - and if they are it's still fairly minor. In the last 9 hours we have had 10% of these requests fail (about 13,700 requests). Prior to 2 days ago it was not uncommon to see the failure rate around 1% less than 100 failures for over 20 hours. The code that saves to the data store saves a model that has a reference to another object. This referenced object is pretty simple - StringProperty objects and a couple DateProperty objects. Is saving this reference more expensive than creating a uniqueId of the (simple)object and manually saving a the custom defined uniqueId to it? If this continues throughout the day I guess I'll turn profiling back on and see what's going on... Any thoughts on re-working the reference? Is it a more expensive operation to save and retrieve references to objects rather than a custom unqique key to the object? If I reworked the relationship it would force me to turn back around and query to get the referenced object when I need it though. Thanks for helping! -Aaron On Sat, Jan 3, 2009 at 2:56 AM, Paul Kinlan paul.kin...@gmail.com wrote: Hi, I also regularly get datastore timeouts for reads and writes. Looking at the status monitor there never seem to be any problems. Kind Regards, Paul Kinlan 2009/1/3 Xavier Mathews xavieruni...@gmail.com Place it in the issue tracker. On 01/02/2009, Alexander Kojevnikov alexan...@kojevnikov.com wrote: My application is getting a very high number of data store timeouts. Is there an unreported issue going on right now? The status monitor doesn't show anything unusual: http://code.google.com/status/appengine/detail/datastore/2009/01/02 -- Xavier A. Mathews Student/Browser Specialist/Developer/Web-Master Client Based Tech Support Specialist Hazel Crest Illinois xavieruni...@gmail.com¥xavierunited@hotmail.com¥truestar...@yahoo.com Fear of a name, only increases fear of the thing itself. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
Nope no that we know of.be sure that it is not a server time out and that there is no bug in the app preventing it to load correctly! It may just be loading time or the save time could be to long..space issue! ETC Xavier A. Mathews Student/Browser Specialist/Developer/Web-Master Client Based Tech Support Specialist Hazel Crest Illinois xavieruni...@gmail.com¥xavierunited@hotmail.com¥truestar...@yahoo.com Fear of a name, only increases fear of the thing itself. On Fri, Jan 2, 2009 at 11:44 AM, Aaron aobrien...@gmail.com wrote: My application is getting a very high number of data store timeouts. Is there an unreported issue going on right now? -Aaron O'Brien --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
My application is getting a very high number of data store timeouts. Is there an unreported issue going on right now? The status monitor doesn't show anything unusual: http://code.google.com/status/appengine/detail/datastore/2009/01/02 --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Timeouts
Place it in the issue tracker. On 01/02/2009, Alexander Kojevnikov alexan...@kojevnikov.com wrote: My application is getting a very high number of data store timeouts. Is there an unreported issue going on right now? The status monitor doesn't show anything unusual: http://code.google.com/status/appengine/detail/datastore/2009/01/02 -- Xavier A. Mathews Student/Browser Specialist/Developer/Web-Master Client Based Tech Support Specialist Hazel Crest Illinois xavieruni...@gmail.com¥xavierunited@hotmail.com¥truestar...@yahoo.com Fear of a name, only increases fear of the thing itself. --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
