[google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles
Hi Ikai Not exactly I am using this to connect to the remote_api def connect(appid,auth_func,host): remote_api_stub.ConfigureRemoteDatastore(appid, '/remote_api', auth_func, host) Its seems you never get a handle back on which you make the call, it just hooks up the remote data store under the hood to all the db api methods. Each time I was starting a new set of pushes connect was being called. I now track if I have called ConfigureRemoteDatastore and don't call it again. That way the current connection is re-used. Regards Tim On Dec 4, 2:06 am, "Ikai L (Google)" wrote: > Tim, > > Ah, great find. If I'm to understand you correctly, you were using a > different process off App Engine to log into App Engine, hitting the login > page N times in a span of M minutes. This resulted in you not being able to > authenticate anymore, but after you cached the auth cookie, it started > working fine. > > On Thu, Dec 3, 2009 at 3:01 AM, Tim Hoffman wrote: > > Hi Ikai -- You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appeng...@googlegroups.com. To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en.
Re: [google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles
Tim, Ah, great find. If I'm to understand you correctly, you were using a different process off App Engine to log into App Engine, hitting the login page N times in a span of M minutes. This resulted in you not being able to authenticate anymore, but after you cached the auth cookie, it started working fine. On Thu, Dec 3, 2009 at 3:01 AM, Tim Hoffman wrote: > Hi Ikai > > > I am using remote_api called from within plone. > > Because I could potentially get multiple concurrent calls from plone > to push content I was creating a new connection > (requiring new authentication ) on each request to push content from > plone to app engine. > > Unfortunately when I started pushing large amounts of content each > content entity being pushed was creating a new > connection (invoking authentication). When I hit 200 of these in the > space of say 10mins I would get hit by google infrastructure > which would basically stop transactions. CheckSuccess didn't like > getting a redirect to the authentication stuff. > And I couldn't re-authenticate for about 10 mins anyway. Looks like I > was running afoul of systems designed to prevent > spamming etc Any other connection attempts from the same host > using that user id would also fail. > And none of this would be seen in the app engine logs, because the > remote_api auth/connection was being redirected > even before we got to appengine. > > I found that if I cached the connection and didn't continually re- > authenticate I had no problems. > > Hope that makes sense. > > Rgds > > Tim > On Dec 3, 3:35 am, "Ikai L (Google)" wrote: > > Tim, great find. What do you mean re-use connections? If you let me know, > > I'll update the docs and open an issue to provide a better error message. > > > > On Tue, Dec 1, 2009 at 8:56 PM, Tim Hoffman wrote: > > > Ok > > > > > Got the the bottom of it all. > > > > > You need to re-use connections. If you make too many authenticated > > > connections then googles higher up infrastructure > > > shuts you down for about 10 min. > > > > > It does highlight some problems in the remote_api code as the error > > > messages passed back are particularly informative unless you really > > > dig. but at least I understand where the problem is. > > > > > T > > > > > On Dec 2, 11:31 am, Tim Hoffman wrote: > > > > OK > > > > > > Started debugging inside the apiproxy_rpc.py(113)CheckSuccess() > > > > and what I believe is happening is correct. > > > > > > the rpc call MaekSyncCall is getting a redirect to a request to login > > > > > > apiproxy_rpc.py(113)CheckSuccess() > > > > > > The exception is a HTTPErrror which is a redirect (302) > > > > to > > > > > > 'https://www.google.com/a/polytechnic.wa.edu.au/ServiceLogin? > > > > service=ah&passive=true&continue= > http://psc-prod1.appspot.com/_ah/login > > > > %3Fcon stuff deleted', > > > > > > Obviously requesting me to log in again. > > > > > > So somewhere I am running afoul of higher layers in google that watch > > > > what is going on. > > > > I am going to need some help from google engineers to get to the > > > > bottom of this one. > > > > > > Rgds > > > > > > Tim Hoffman > > > > > > On Dec 2, 11:04 am, Tim Hoffman wrote: > > > > > > > Something I have observerd and can now prove repeatably > > > > > > > Once I get the error 302 inside the remote_api call. It takes > > > > > somewhere between 2 and 10 mins before I can resubmit the > transaction. > > > > > During that that time, if I try the same call from another process > (on > > > > > the same box with the same auth details) the call still gets the > error > > > > > until such time as something in google lets it go. > > > > > > > It really is looking like I am being clobbered by some google watch > > > > > dog. > > > > > > > T > > > > > > > On Dec 2, 9:58 am, Tim Hoffman wrote: > > > > > > > > I am still getting this problem. > > > > > > > > One of the things I have noticed is the errors (302) are being > > > > > > propogated by google infrastructure and not recorded any way > > > > > > in the apps logs. > > > > > > > > My gut feel is that some other google infrastructure thinks I am > > > > > > doing something I shouldn't requires me to re-authenticate but > > > > > > the remote_api proxies can't deal with a 302 inside a Namerror in > > > > > > CheckSuccess see stack below. > > > > > > > > Obviously what I am doing is different to the bulkloader client. > > > > > > > > Anyone got any suggestions on how I might go about getting to the > > > > > > bottom of this. > > > > > > > > T > > > > > > > > 2009-12-02T01:55:49 INFO root Exception sending Rollback: > > > > > > Traceback (most recent call last): > > > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > > > google_appengine/google/appengine/api/datastore.py", line 1989, > in > > > > > > RunInTransactionCustomRetries > > > > > > tx.handle, resp) > > > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > > > google_appengine/
[google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles
Hi Ikai I am using remote_api called from within plone. Because I could potentially get multiple concurrent calls from plone to push content I was creating a new connection (requiring new authentication ) on each request to push content from plone to app engine. Unfortunately when I started pushing large amounts of content each content entity being pushed was creating a new connection (invoking authentication). When I hit 200 of these in the space of say 10mins I would get hit by google infrastructure which would basically stop transactions. CheckSuccess didn't like getting a redirect to the authentication stuff. And I couldn't re-authenticate for about 10 mins anyway. Looks like I was running afoul of systems designed to prevent spamming etc Any other connection attempts from the same host using that user id would also fail. And none of this would be seen in the app engine logs, because the remote_api auth/connection was being redirected even before we got to appengine. I found that if I cached the connection and didn't continually re- authenticate I had no problems. Hope that makes sense. Rgds Tim On Dec 3, 3:35 am, "Ikai L (Google)" wrote: > Tim, great find. What do you mean re-use connections? If you let me know, > I'll update the docs and open an issue to provide a better error message. > > On Tue, Dec 1, 2009 at 8:56 PM, Tim Hoffman wrote: > > Ok > > > Got the the bottom of it all. > > > You need to re-use connections. If you make too many authenticated > > connections then googles higher up infrastructure > > shuts you down for about 10 min. > > > It does highlight some problems in the remote_api code as the error > > messages passed back are particularly informative unless you really > > dig. but at least I understand where the problem is. > > > T > > > On Dec 2, 11:31 am, Tim Hoffman wrote: > > > OK > > > > Started debugging inside the apiproxy_rpc.py(113)CheckSuccess() > > > and what I believe is happening is correct. > > > > the rpc call MaekSyncCall is getting a redirect to a request to login > > > > apiproxy_rpc.py(113)CheckSuccess() > > > > The exception is a HTTPErrror which is a redirect (302) > > > to > > > > 'https://www.google.com/a/polytechnic.wa.edu.au/ServiceLogin? > > > service=ah&passive=true&continue=http://psc-prod1.appspot.com/_ah/login > > > %3Fcon stuff deleted', > > > > Obviously requesting me to log in again. > > > > So somewhere I am running afoul of higher layers in google that watch > > > what is going on. > > > I am going to need some help from google engineers to get to the > > > bottom of this one. > > > > Rgds > > > > Tim Hoffman > > > > On Dec 2, 11:04 am, Tim Hoffman wrote: > > > > > Something I have observerd and can now prove repeatably > > > > > Once I get the error 302 inside the remote_api call. It takes > > > > somewhere between 2 and 10 mins before I can resubmit the transaction. > > > > During that that time, if I try the same call from another process (on > > > > the same box with the same auth details) the call still gets the error > > > > until such time as something in google lets it go. > > > > > It really is looking like I am being clobbered by some google watch > > > > dog. > > > > > T > > > > > On Dec 2, 9:58 am, Tim Hoffman wrote: > > > > > > I am still getting this problem. > > > > > > One of the things I have noticed is the errors (302) are being > > > > > propogated by google infrastructure and not recorded any way > > > > > in the apps logs. > > > > > > My gut feel is that some other google infrastructure thinks I am > > > > > doing something I shouldn't requires me to re-authenticate but > > > > > the remote_api proxies can't deal with a 302 inside a Namerror in > > > > > CheckSuccess see stack below. > > > > > > Obviously what I am doing is different to the bulkloader client. > > > > > > Anyone got any suggestions on how I might go about getting to the > > > > > bottom of this. > > > > > > T > > > > > > 2009-12-02T01:55:49 INFO root Exception sending Rollback: > > > > > Traceback (most recent call last): > > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > > google_appengine/google/appengine/api/datastore.py", line 1989, in > > > > > RunInTransactionCustomRetries > > > > > tx.handle, resp) > > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > > google_appengine/google/appengine/api/apiproxy_stub_map.py", line 72, > > > > > in MakeSyncCall > > > > > apiproxy.MakeSyncCall(service, call, request, response) > > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > > google_appengine/google/appengine/api/apiproxy_stub_map.py", line > > 266, > > > > > in MakeSyncCall > > > > > rpc.CheckSuccess() > > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > > google_appengine/google/appengine/api/apiproxy_rpc.py", line 111, in > > > > > CheckSuccess > > > > > raise self.exception > > > > > NameError: global name 'txdata
Re: [google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles
Tim, great find. What do you mean re-use connections? If you let me know, I'll update the docs and open an issue to provide a better error message. On Tue, Dec 1, 2009 at 8:56 PM, Tim Hoffman wrote: > Ok > > Got the the bottom of it all. > > You need to re-use connections. If you make too many authenticated > connections then googles higher up infrastructure > shuts you down for about 10 min. > > It does highlight some problems in the remote_api code as the error > messages passed back are particularly informative unless you really > dig. but at least I understand where the problem is. > > T > > > > On Dec 2, 11:31 am, Tim Hoffman wrote: > > OK > > > > Started debugging inside the apiproxy_rpc.py(113)CheckSuccess() > > and what I believe is happening is correct. > > > > the rpc call MaekSyncCall is getting a redirect to a request to login > > > > apiproxy_rpc.py(113)CheckSuccess() > > > > The exception is a HTTPErrror which is a redirect (302) > > to > > > > 'https://www.google.com/a/polytechnic.wa.edu.au/ServiceLogin? > > service=ah&passive=true&continue=http://psc-prod1.appspot.com/_ah/login > > %3Fcon stuff deleted', > > > > Obviously requesting me to log in again. > > > > So somewhere I am running afoul of higher layers in google that watch > > what is going on. > > I am going to need some help from google engineers to get to the > > bottom of this one. > > > > Rgds > > > > Tim Hoffman > > > > On Dec 2, 11:04 am, Tim Hoffman wrote: > > > > > Something I have observerd and can now prove repeatably > > > > > Once I get the error 302 inside the remote_api call. It takes > > > somewhere between 2 and 10 mins before I can resubmit the transaction. > > > During that that time, if I try the same call from another process (on > > > the same box with the same auth details) the call still gets the error > > > until such time as something in google lets it go. > > > > > It really is looking like I am being clobbered by some google watch > > > dog. > > > > > T > > > > > On Dec 2, 9:58 am, Tim Hoffman wrote: > > > > > > I am still getting this problem. > > > > > > One of the things I have noticed is the errors (302) are being > > > > propogated by google infrastructure and not recorded any way > > > > in the apps logs. > > > > > > My gut feel is that some other google infrastructure thinks I am > > > > doing something I shouldn't requires me to re-authenticate but > > > > the remote_api proxies can't deal with a 302 inside a Namerror in > > > > CheckSuccess see stack below. > > > > > > Obviously what I am doing is different to the bulkloader client. > > > > > > Anyone got any suggestions on how I might go about getting to the > > > > bottom of this. > > > > > > T > > > > > > 2009-12-02T01:55:49 INFO root Exception sending Rollback: > > > > Traceback (most recent call last): > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > google_appengine/google/appengine/api/datastore.py", line 1989, in > > > > RunInTransactionCustomRetries > > > > tx.handle, resp) > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > google_appengine/google/appengine/api/apiproxy_stub_map.py", line 72, > > > > in MakeSyncCall > > > > apiproxy.MakeSyncCall(service, call, request, response) > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > google_appengine/google/appengine/api/apiproxy_stub_map.py", line > 266, > > > > in MakeSyncCall > > > > rpc.CheckSuccess() > > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > > google_appengine/google/appengine/api/apiproxy_rpc.py", line 111, in > > > > CheckSuccess > > > > raise self.exception > > > > NameError: global name 'txdata' is not defined > > > > > > -- > > > > 2009-12-02T01:55:49 ERROR Zope.SiteErrorLoghttp:// > 10.8.0.134:49081/swan/portal_skins/custom/send_uos > > > > Traceback (innermost last): > > > > Module ZPublisher.Publish, line 119, in publish > > > > Module ZPublisher.mapply, line 88, in mapply > > > > Module ZPublisher.Publish, line 42, in call_object > > > > Module Shared.DC.Scripts.Bindings, line 313, in __call__ > > > > Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec > > > > Module Products.PythonScripts.PythonScript, line 328, in _exec > > > > Module None, line 25, in send_uos > > > >- > > > >- Line 25 > > > > Module None, line 11, in push > > > >- > > > >- Line 11 > > > > Module Products.PlonePSC.tools.psc_manager, line 639, in > > > > push_to_appengine > > > > Module google.appengine.ext.db, line 1064, in get_or_insert > > > > Module google.appengine.api.datastore, line 1884, in > > > > RunInTransaction > > > > Module google.appengine.api.datastore, line 1982, in > > > > RunInTransactionCustomRetries > > > > Module google.appengine.ext.db, line 1059, in txn > > > > Module google.appengine.ext.db, line 981, in get_by_key_name > > > > Module google.appengine.ext.db, line 1180, in get > > > >
[google-appengine] Re: Continuing HTTPErrr 302 when using remote_api after a 200 transactions - need to re-use connection handles
Ok Got the the bottom of it all. You need to re-use connections. If you make too many authenticated connections then googles higher up infrastructure shuts you down for about 10 min. It does highlight some problems in the remote_api code as the error messages passed back are particularly informative unless you really dig. but at least I understand where the problem is. T On Dec 2, 11:31 am, Tim Hoffman wrote: > OK > > Started debugging inside the apiproxy_rpc.py(113)CheckSuccess() > and what I believe is happening is correct. > > the rpc call MaekSyncCall is getting a redirect to a request to login > > apiproxy_rpc.py(113)CheckSuccess() > > The exception is a HTTPErrror which is a redirect (302) > to > > 'https://www.google.com/a/polytechnic.wa.edu.au/ServiceLogin? > service=ah&passive=true&continue=http://psc-prod1.appspot.com/_ah/login > %3Fcon stuff deleted', > > Obviously requesting me to log in again. > > So somewhere I am running afoul of higher layers in google that watch > what is going on. > I am going to need some help from google engineers to get to the > bottom of this one. > > Rgds > > Tim Hoffman > > On Dec 2, 11:04 am, Tim Hoffman wrote: > > > Something I have observerd and can now prove repeatably > > > Once I get the error 302 inside the remote_api call. It takes > > somewhere between 2 and 10 mins before I can resubmit the transaction. > > During that that time, if I try the same call from another process (on > > the same box with the same auth details) the call still gets the error > > until such time as something in google lets it go. > > > It really is looking like I am being clobbered by some google watch > > dog. > > > T > > > On Dec 2, 9:58 am, Tim Hoffman wrote: > > > > I am still getting this problem. > > > > One of the things I have noticed is the errors (302) are being > > > propogated by google infrastructure and not recorded any way > > > in the apps logs. > > > > My gut feel is that some other google infrastructure thinks I am > > > doing something I shouldn't requires me to re-authenticate but > > > the remote_api proxies can't deal with a 302 inside a Namerror in > > > CheckSuccess see stack below. > > > > Obviously what I am doing is different to the bulkloader client. > > > > Anyone got any suggestions on how I might go about getting to the > > > bottom of this. > > > > T > > > > 2009-12-02T01:55:49 INFO root Exception sending Rollback: > > > Traceback (most recent call last): > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > google_appengine/google/appengine/api/datastore.py", line 1989, in > > > RunInTransactionCustomRetries > > > tx.handle, resp) > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > google_appengine/google/appengine/api/apiproxy_stub_map.py", line 72, > > > in MakeSyncCall > > > apiproxy.MakeSyncCall(service, call, request, response) > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > google_appengine/google/appengine/api/apiproxy_stub_map.py", line 266, > > > in MakeSyncCall > > > rpc.CheckSuccess() > > > File "/opt/ktstudio/zope/instances/prod/swantafe.buildout/ > > > google_appengine/google/appengine/api/apiproxy_rpc.py", line 111, in > > > CheckSuccess > > > raise self.exception > > > NameError: global name 'txdata' is not defined > > > > -- > > > 2009-12-02T01:55:49 ERROR > > > Zope.SiteErrorLoghttp://10.8.0.134:49081/swan/portal_skins/custom/send_uos > > > Traceback (innermost last): > > > Module ZPublisher.Publish, line 119, in publish > > > Module ZPublisher.mapply, line 88, in mapply > > > Module ZPublisher.Publish, line 42, in call_object > > > Module Shared.DC.Scripts.Bindings, line 313, in __call__ > > > Module Shared.DC.Scripts.Bindings, line 350, in _bindAndExec > > > Module Products.PythonScripts.PythonScript, line 328, in _exec > > > Module None, line 25, in send_uos > > > - > > > - Line 25 > > > Module None, line 11, in push > > > - > > > - Line 11 > > > Module Products.PlonePSC.tools.psc_manager, line 639, in > > > push_to_appengine > > > Module google.appengine.ext.db, line 1064, in get_or_insert > > > Module google.appengine.api.datastore, line 1884, in > > > RunInTransaction > > > Module google.appengine.api.datastore, line 1982, in > > > RunInTransactionCustomRetries > > > Module google.appengine.ext.db, line 1059, in txn > > > Module google.appengine.ext.db, line 981, in get_by_key_name > > > Module google.appengine.ext.db, line 1180, in get > > > Module google.appengine.api.datastore, line 234, in Get > > > Module google.appengine.api.apiproxy_stub_map, line 72, in > > > MakeSyncCall > > > Module google.appengine.api.apiproxy_stub_map, line 266, in > > > MakeSyncCall > > > Module google.appengine.api.apiproxy_rpc, line 111, in CheckSuccess > > > HTTPError: HTTP Error 302: Found > > > > On Nov 18, 7:14 am, Tim Hoffman wrote: > > > > > Hi Mathew > > > > > Appid sent in mail.
