[appengine-java] Diagnosing problems with backends that are freezing
Are are using backends to parallelize large batches of work for our users. This involves adding 100's of tasks to a dedicated queue which spreads the tasks across 5 dynamic backends. Many of these tasks do a url fetch to external web services. When we run this for one user it seems to work fine but as soon as we run it for many users (e.g. in a nightly batch run) we see most of the backends eventually stopping processing new tasks. We've tried rate limiting it but that has not helped either. I can manually stop the backends that are frozen and new ones will fire up and start processing until they too freeze. Eventually it all completes after a few iterations of stopping frozen servers. Obviously this is severely limiting our ability to scale so I'm wondering how to diagnose this problem. We cannot reproduce this on localhost since its not really multi-threaded and doesn't truly replicate the deployed data patterns. We could possibly handle this by using pull queues but that's more code to write and I'd rather let the queue/backend scheduler do that work for us. What I'd like to do is attach a profiler to a backend - that would immediately tell me where the freezes are coming from. Is that even possible? If not, does anybody have any other tricks that we could use to diagnose frozen backends? fyi : the "frozen" instances have varied levels of memory from 150mb to 250mb, consume zero cpu when frozen and process no new tasks. Thanks for your attention and help, Steve -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Server Error : Deployment Failed
and me On Aug 20, 3:13 pm, BoulderGae wrote: > Me Too! > > On Aug 19, 10:34 pm, culov wrote: > > > > > > > > > havent been able to deploy in the last hour... > > > On Aug 19, 8:24 pm, Shawn Brown wrote: > > > > Hello, > > > > I consistently get: > > > > Unable to update: > > > java.io.IOException: Error posting to > > > URL:https://appengine.google.com/api/appversion/clonefiles?app_id=fluency... > > > 500 Internal Server Error > > > > On a related note, are uploads to appengine (meaning deploying) very > > > slow. I was averaging around 32kbps which is well below what my > > > connection tests at 256kbps. Well that is before deploying failed > > > completely. Now all's I see is: > > > > at > > > com.google.appengine.tools.admin.AbstractServerConnection.send(AbstractServ > > > erConnection.java:249) > > > at > > > com.google.appengine.tools.admin.AbstractServerConnection.post(AbstractServ > > > erConnection.java:207) > > > at > > > com.google.appengine.tools.admin.AppVersionUpload.send(AppVersionUpload.jav > > > a:639) > > > at > > > com.google.appengine.tools.admin.AppVersionUpload.cloneFiles(AppVersionUplo > > > ad.java:502) > > > at > > > com.google.appengine.tools.admin.AppVersionUpload.beginTransaction(AppVersi > > > onUpload.java:462) > > > at > > > com.google.appengine.tools.admin.AppVersionUpload.doUpload(AppVersionUpload > > > .java:137) > > > at > > > com.google.appengine.tools.admin.AppAdminImpl.doUpdate(AppAdminImpl.java:30 > > > 0) > > > at > > > com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:52) > > > at > > > com.google.appengine.tools.admin.AppCfg$UpdateAction.execute(AppCfg.java:59 > > > 8) > > > at com.google.appengine.tools.admin.AppCfg.(AppCfg.java:157) > > > at com.google.appengine.tools.admin.AppCfg.(AppCfg.java:65) > > > at com.google.appengine.tools.admin.AppCfg.main(AppCfg.java:61) > > > > and > > > > 37% Cloning 1027 application files. > > > 37% on backend null. > > > > So what do I need to do to reply? > > > > Shawn -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Facebook Login integration with java appengine
We used Janrain which has the added advantage of supporting N networks instead of just one. One caveat is that it makes the login slower because you go through Janrain redirects during login so test this to ensure it's fast enough for you if you try it. Steve On May 15, 12:35 am, Ravi Sharma wrote: > Hi, > I am usinghttp://code.google.com/p/facebook-java-api/to login using > facebook. But unable to do so as documenttaion is not quite completed there > and posted my problem on this site but long time no response. > I am sure in this group many people must have done with their java app to > login using facebook. Could you please share your code or give me some > guidance. > > Requirement: > I will be dispalying google/yahoo/facebook logo on my site and when user > click on Facebook logo it will take it to facebook page to authenticate.Once > authenticate it will call my callback url and i should be able to query > logged in user's name etc using facebook api(java). > > I am sorry posting a facebook login question here, but the project it self > seems inactive and no one is responding. > > Thanks in advance > Ravi. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] New Content-Length must be specified error
I've got an application that just started producing the following error from servlets. javax.servlet.ServletException: Content-Length must be specified at com.google.gwt.user.server.rpc.RPCServletUtils.readContentAsUtf8(RPCServletUtils.java: 168) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.readContent(AbstractRemoteServiceServlet.java: 179) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processPost(RemoteServiceServlet.java: 215) at com.google.gwt.user.server.rpc.AbstractRemoteServiceServlet.doPost(AbstractRemoteServiceServlet.java: 62) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) While this is obviously a problem with my client RPC code, what worries me is that this just started happening on the server without any changes from me. Has anybody else seen this? If there are platform changes that are deployed to GAE that can destabilize our code without us knowing about it then that's a real concern. We have a dev & QA version of our app to try to avoid this situation but this occurred in already tested code. Does anyone have any insight into this change in behaviour? p.s. If I discover that it was actually my own deployed code that broke this then I will update this thread. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Security Exception using Guice Servlet
I have the answer. With help from the folks at Google I was told that the underlying problem was a ClassNotFoundException for isSerializable from the GWT toolkit. This explains why my other app worked ok, because it contained the GWT jar. This also means that the true problem was being masked by the Appengine and this was the cause of my woe. Apparently very soon they will be allowing the root cause for these kinds of problems to be seen so that this masking of errors won't occur in the future. FTW! Thanks Google team, I'm happy to be moving forward again. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Security Exception using Guice Servlet
The plot thickens. I just uploaded another version of my app with similar code, including the offending MappingUtils class. This version has no troubles with it. This means that the MappingUtils class is not really the problem - there is something specific to the version that is manifesting a SecurityException but the true problem is not described in that exception. If you are a kind-hearted Google engineer, my problematic version is "integrate-1-0-0" I'm using the fact that versions in Appengine can run different code but share the same database. This is great for separating code out for different purposes (and reducing the classes loaded for threads since they don't need everything) but it seems that there's a classloader price to be paid here. Help! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Security Exception using Guice Servlet
OK, so I'm still digging and don't have answer yet but I thought I'd give an update in case it sparks an idea and it'll be useful as a future reference. Firstly getting to the real error is a challenge because Guice on Appengine does a lot of recursion when creating dependency trees. In this case the real SecurityException is 4 levels down the the appengine only shows three levels. So I put a try catch in my servlet config class (IntegrationServletConfig) and then unwrapped the nested exceptions so see the culprit. Bear in mind that I cannot reproduce locally so I have to upload a new version for each test - yuck! This did yield a result. The SecurityException comes from Class.getDeclaredMethods java.lang.SecurityException: Unable to get members for class com.pbw.core.database.MappingUtils at com.google.appengine.runtime.Request.process-131516640218e6d2(Request.java) at java.lang.Class.getDeclaredMethods(Class.java:252) at com.google.inject.spi.InjectionPoint$Factory $2.getMembers(InjectionPoint.java:395) at com.google.inject.spi.InjectionPoint$Factory $2.getMembers(InjectionPoint.java:393) at com.google.inject.spi.InjectionPoint.addInjectorsForMembers(InjectionPoint.java: 359) at com.google.inject.spi.InjectionPoint.addInjectionPoints(InjectionPoint.java: 353) at com.google.inject.spi.InjectionPoint.forInstanceMethodsAndFields(InjectionPoint.java: 296) at com.google.inject.MembersInjectorStore.createWithListeners(MembersInjectorStore.java: 78) at com.google.inject.MembersInjectorStore.access $000(MembersInjectorStore.java:35) at com.google.inject.MembersInjectorStore $1.create(MembersInjectorStore.java:43) at com.google.inject.MembersInjectorStore $1.create(MembersInjectorStore.java:40) at com.google.inject.internal.FailableCache $1.apply(FailableCache.java:35) at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:549) So now I know that something about the methods in my MappingUtils class are not ok for the appengine. I guess that the real answer is in this line... at com.google.appengine.runtime.Request.process-131516640218e6d2(Request.java) but I'll need someone at Google to look into that as it only occurs on the appengine. This got me thinking - what has changed? I've been using the class with no problems for 6 months. Here are the candidates.. - a new method : I added a new method that returns Class : when I removed it the error still occurs - a newer version of the Appengine : I upgraded to 1.3.1 : I downgraded to 1.3.0 and had the same problem. Now I'm lost for ideas. If you have a suggestion for a next move I'd love to hear it. Thanks -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Security Exception using Guice Servlet
I forgot to add : I suspected a Guice bug so I build the Guice trunk and tried that but there was no change in behaviour so that's out. On Mar 18, 4:17 pm, steveb wrote: > Hi, I've started to see a new exception for my application and I'm > hoping that a kind Googler can look this up for me. My appid is > catapult-lb and here's the stack trace. > > I'd be grateful for any help as this has been working for more than 6 > months and now (probably due to a code change of mine) I'm now hitting > this mysterious road-block. > > Thanks. > > com.google.inject.internal.ComputationException: > com.google.inject.internal.ComputationException: > com.google.inject.internal.ComputationException: > com.google.inject.internal.ComputationException: > java.lang.SecurityException: Unable to get members for class > com.pbw.core.database.MappingUtils > at com.google.inject.internal.MapMaker > $StrategyImpl.compute(MapMaker.java:553) > at com.google.inject.internal.MapMaker > $StrategyImpl.compute(MapMaker.java:419) > at com.google.inject.internal.CustomConcurrentHashMap > $ComputingImpl.get(CustomConcurrentHashMap.java:2041) > at com.google.inject.internal.FailableCache.get(FailableCache.java: > 46) > at > com.google.inject.internal.ConstructorInjectorStore.get(ConstructorInjector > Store.java: > 48) > at > com.google.inject.internal.ConstructorBindingImpl.initialize(ConstructorBin > dingImpl.java: > 120) > at > com.google.inject.internal.InjectorImpl.initializeJitBinding(InjectorImpl.j > ava: > 443) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImp > l.java: > 698) > at > com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(In > jectorImpl.java: > 630) > at > com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.j > ava: > 223) > at > com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java : > 174) > at > com.google.inject.internal.InjectorImpl.getInternalFactory(InjectorImpl.jav a: > 704) > at com.google.inject.internal.FactoryProxy.notify(FactoryProxy.java: > 45) > at > com.google.inject.internal.BindingProcessor.runCreationListeners(BindingPro > cessor.java: > 238) > at > com.google.inject.internal.InternalInjectorCreator.initializeStatically(Int > ernalInjectorCreator.java: > 141) > at > com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCr > eator.java: > 114) > at com.google.inject.InjectorBuilder.build(InjectorBuilder.java:90) > at com.google.inject.Guice.createInjector(Guice.java:105) > at com.google.inject.Guice.createInjector(Guice.java:76) > at > com.pbw.integrate.config.IntegrationServletConfig.getInjector(Unknown > Source) > at > com.google.inject.servlet.GuiceServletContextListener.contextInitialized(Gu > iceServletContextListener.java: > 43) > at > org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java: > 530) -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Security Exception using Guice Servlet
Hi, I've started to see a new exception for my application and I'm hoping that a kind Googler can look this up for me. My appid is catapult-lb and here's the stack trace. I'd be grateful for any help as this has been working for more than 6 months and now (probably due to a code change of mine) I'm now hitting this mysterious road-block. Thanks. com.google.inject.internal.ComputationException: com.google.inject.internal.ComputationException: com.google.inject.internal.ComputationException: com.google.inject.internal.ComputationException: java.lang.SecurityException: Unable to get members for class com.pbw.core.database.MappingUtils at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:553) at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:419) at com.google.inject.internal.CustomConcurrentHashMap $ComputingImpl.get(CustomConcurrentHashMap.java:2041) at com.google.inject.internal.FailableCache.get(FailableCache.java: 46) at com.google.inject.internal.ConstructorInjectorStore.get(ConstructorInjectorStore.java: 48) at com.google.inject.internal.ConstructorBindingImpl.initialize(ConstructorBindingImpl.java: 120) at com.google.inject.internal.InjectorImpl.initializeJitBinding(InjectorImpl.java: 443) at com.google.inject.internal.InjectorImpl.createJustInTimeBinding(InjectorImpl.java: 698) at com.google.inject.internal.InjectorImpl.createJustInTimeBindingRecursive(InjectorImpl.java: 630) at com.google.inject.internal.InjectorImpl.getJustInTimeBinding(InjectorImpl.java: 223) at com.google.inject.internal.InjectorImpl.getBindingOrThrow(InjectorImpl.java: 174) at com.google.inject.internal.InjectorImpl.getInternalFactory(InjectorImpl.java: 704) at com.google.inject.internal.FactoryProxy.notify(FactoryProxy.java: 45) at com.google.inject.internal.BindingProcessor.runCreationListeners(BindingProcessor.java: 238) at com.google.inject.internal.InternalInjectorCreator.initializeStatically(InternalInjectorCreator.java: 141) at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java: 114) at com.google.inject.InjectorBuilder.build(InjectorBuilder.java:90) at com.google.inject.Guice.createInjector(Guice.java:105) at com.google.inject.Guice.createInjector(Guice.java:76) at com.pbw.integrate.config.IntegrationServletConfig.getInjector(Unknown Source) at com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java: 43) at org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java: 530) -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Ant and providing email / password
Here's how I did it... -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Text Search Support for Java
I'm also very interested in this feature. I'm looking at plugging in Lucene ( http://www.kimchy.org/searchable-google-appengine-with-compass ) to achieve search but I can imagine that there will be lots of issues with storage, indexing, security, CPU use etc with this solution. If we can get an indication of rough roadmap then we can decide whether to delay and wait or bite off this hairy bit of integration work. p.s. would it be possible to bridge between java and python SearchableModel? I think this would be less total work than plugging in Lucene. I'd be happy to hear any war stories in this area. Thanks, Steve On Dec 3, 7:53 am, lent wrote: > Hi, > > This question is targeted at Google guys. Is there any plans to have > limited support for full text search for Java, i.e. something like the > equivalent of SearchableModel in Python, and if so when can this be > expected? When is FULL support for full text search be expected for > appengine (Python and Java)? > > Regards, > Len -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
