[appengine-java] GUICE injection and access denied (java.lang.RuntimePermission modifyThreadGroup)
Hi, I'm trying to do this; public class PostalServiceImpl extends RemoteServiceServlet implements PostalService { private static final Logger log = Logger.getLogger(PostalServiceImpl.class.getName()); @Inject private PostalManager postalManager; public PostalServiceImpl() { Injector injector = Guice.createInjector(new NetGazelleModule()); injector.injectMembers(this); } public String getCity(String postal) { log.info(PostalServiceImpl.getCity( + postal + )); if (postalManager == null) { log.severe(PostalServiceImpl.getCity: postalManager is null !); } return postalManager.getCity(postal); } ... But I get the following exception; INFO: The server is running at http://localhost:/ Jun 11, 2010 6:38:57 AM com.google.inject.internal.FinalizableReferenceQueue init INFO: Failed to start reference finalizer thread. Reference cleanup will only occur when new references are created. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 25) at java.lang.reflect.Method.invoke(Method.java:597) at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java: 100) at com.google.inject.internal.FinalizableReferenceQueue.init(FinalizableReferenceQueue.java: 124) at com.google.inject.internal.MapMaker $QueueHolder.clinit(MapMaker.java:787) at com.google.inject.internal.MapMaker$WeakEntry.init(MapMaker.java: 946) at com.google.inject.internal.MapMaker$Strength $1.newEntry(MapMaker.java:312) at com.google.inject.internal.MapMaker $StrategyImpl.newEntry(MapMaker.java:498) at com.google.inject.internal.MapMaker $StrategyImpl.newEntry(MapMaker.java:419) at com.google.inject.internal.CustomConcurrentHashMap $ComputingImpl.get(CustomConcurrentHashMap.java:2029) at com.google.inject.internal.BytecodeGen.getClassLoader(BytecodeGen.java: 149) at com.google.inject.internal.BytecodeGen.getClassLoader(BytecodeGen.java: 132) at com.google.inject.internal.BytecodeGen.newFastClass(BytecodeGen.java: 162) at com.google.inject.DefaultConstructionProxyFactory $1.init(DefaultConstructionProxyFactory.java:54) at com.google.inject.DefaultConstructionProxyFactory.create(DefaultConstructionProxyFactory.java: 52) at com.google.inject.ProxyFactory.create(ProxyFactory.java:152) at com.google.inject.ConstructorInjectorStore.createConstructor(ConstructorInjectorStore.java: 84) at com.google.inject.ConstructorInjectorStore.access $000(ConstructorInjectorStore.java:31) at com.google.inject.ConstructorInjectorStore $1.create(ConstructorInjectorStore.java:39) at com.google.inject.ConstructorInjectorStore $1.create(ConstructorInjectorStore.java:35) at com.google.inject.internal.FailableCache $1.apply(FailableCache.java:35) at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:549) at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:419) at com.google.inject.internal.CustomConcurrentHashMap $ComputingImpl.get(CustomConcurrentHashMap.java:2041) at com.google.inject.internal.FailableCache.get(FailableCache.java: 46) at com.google.inject.ConstructorInjectorStore.get(ConstructorInjectorStore.java: 52) at com.google.inject.ConstructorBindingImpl.initialize(ConstructorBindingImpl.java: 57) at com.google.inject.InjectorImpl.initializeBinding(InjectorImpl.java: 377) at com.google.inject.BindingProcessor$1$1.run(BindingProcessor.java: 169) at com.google.inject.BindingProcessor.initializeBindings(BindingProcessor.java: 224) at com.google.inject.InjectorBuilder.initializeStatically(InjectorBuilder.java: 120) at com.google.inject.InjectorBuilder.build(InjectorBuilder.java:105) at com.google.inject.Guice.createInjector(Guice.java:92) at com.google.inject.Guice.createInjector(Guice.java:69) at com.google.inject.Guice.createInjector(Guice.java:59) at com.netgazelle.server.service.PostalServiceImpl.init(PostalServiceImpl.java: 20) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java: 39) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java: 27) at java.lang.reflect.Constructor.newInstance(Constructor.java:513) at java.lang.Class.newInstance0(Class.java:355) at java.lang.Class.newInstance(Class.java:308)
[appengine-java] Session object or memcache?
Hello all what is the best option from performance point of view, storing frequently used objects in session or loading from datastore for every request(we can add memcache support before trying to fetch from datastore).Can anybody tell me good link to develop high performance and fast application development best practices link.I am using java with JDO. thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: What is the best option for RPC (web services) on App engine?
http://code.google.com/p/gwt-syncproxy/ may be suitable for your needs. On Jun 10, 11:13 pm, dilbert dilbert.elbo...@gmail.com wrote: First I'd like to explain what I mean by RPC. I'd like to be able to write interfaces like this (simple Java interface): public interface EchoService { String echo(String message); } The framework would allow the creation of client classes that would handle the serialization from/to the RPC service. Of course the framework should support the serialization of ArrayLists, HashMaps and other collections and should also support the serialization of objects marked with the java.io.Serializable interface (or some other interface). We would create the RPC Client this way: EchoService echoService = RpcClientFactory.createInstance(EchoService.class,http://bla.com/ smartApp/echo); And of course use it this way: String echoMessage = echoService.echo(The message !!!); The server side servlet would implement the previously mentioned interface. public class Service extends WhateverServlet implements EchoService { @Override String echo(String message) { return server sends: + message; } } A few additional nice features to have would be: -support for asynchronous calls (where the developer would provide a callback for handling the result, similar to GWT RPC) -the developers should be able to access the RPC client somehow to be able to handle Cookies or other http headers. -the client side should be usable from Android. After a long search I have found that the framework that most closely matches these requirements is Hessian (http://hessian.caucho.com/). It uses a binary protocol so it should be very fast and it works on Android. AFAIK it does not support async calls. However, not all is well. The current Hessian implementation does not handle exceptions well. It throws a SecurityException like this: java.lang.SecurityException: java.lang.IllegalAccessException: Reflection is not allowed on private java.lang.Throwable java.lang.Throwable.cause I considered GWT RPC for a while but it does not have a proper Java client or I could not find one. So I wanted to ask is there any other library that could be used in this case? Could the App engine team write such a library and add it to the SDK. Any suggestions? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Already has the maximum number of versions, but I have only one.
Still encountering the same problem in the last 4-5 days. I don't know whether this is connected with the last datastore outage, but it's really annoying that I must wait 15 minutes to upload a new version. 40% Uploading 0 files. 52% Initializing precompilation... 90% Deploying new version. 95% Will check again in 1 seconds 98% Will check again in 2 seconds 99% Will check again in 4 seconds 99% Will check again in 8 seconds 99% Will check again in 16 seconds 99% Will check again in 32 seconds 99% Will check again in 64 seconds 99% Will check again in 128 seconds 99% Rolling back the update. Regards, Miroslav On 06/02/2010 10:53 PM, Miroslav Genov wrote: Today all day I get the same error when trying to upload an updated version of my application. I try to remove one existing version (currently I have 5 version of my app) but the administrative panel is failing with the following error: A server error has occurred.. Any idea what is causing it ? Also I wanna mention that the upload of my application is really slow and I have to wait several minutes to upload only 1 file change. Regards, Miroslav On 05/06/2010 03:20 PM, Tim Yates wrote: Hiya! For the past 3 hours, been trying to update my app engine java instance, but have been getting the error: Error posting to URL:https://appengine.google.com/api/appversion/deploy?app_id=check-emailversion=1; 403 Forbidden Too Many Versions (403) The application already has the maximum number of versions. I see on the system status that things are under investigation http://code.google.com/status/appengine/ Is this one of the things being investigated? Tim PS: I only have a single version deployed, and this version is set to be the default -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Manually create entity table and add rows using the dashboard.
Try Extended GQL parser and simple 'gqlext.jsp' tool - http://code.google.com/p/audao/wiki/ExtendedGQLParser You can execute INSERT, SELECT, UPDATE and DELETE commands. The full documentation is at http://audao.spoledge.com/doc-gae-features.html#gqlext Vaclav On Jun 11, 6:02 am, mar_novice mariocape1...@gmail.com wrote: Is there a way to use the dashboard dataviewer to manually create an entity table and add entities in it? So far, what I know is I can add an entity in a given kind if there are already existing entities of that kind. What if I am about to add the very first entity of that kind? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: org.datanucleus.jdo.exceptions.ClassNotPersistenceCapableException:
Hi Paul, Your second set of log entries shows the problem and your way out of it. The first set of log entries complains that your compiled classes may not have been enhanced. The second set of log entries confirms this since your enhancement process did not complete successfully due to the unexpected RuntimeException. I have never used the Eclipse plug-in (a shame, since I use NetBeans now and there is no plug-in officially-supported by Google), but I see that The URL file:/D:/MasteringWave/Eclipse/WaveAssistant/war/WEB-INF/lib/ datanucleus-appengine-1.0.6.final.jar is already registered, and you are trying to register an identical plugin located at URL file:/C:/ eclipse/plugins/com.google.appengine.eclipse.sdkbundle. 1.3.3_1.3.3.v20100423/appengine-java-sdk-1.3.3.1/lib/user/orm/ datanucleus-appengine-1.0.6.final.jar. I'll let you chose which .jar file to drop, but you might find it easier not to include the one that currently finds its way to your D:/ MasteringWave/Eclipse/WaveAssistant/war/WEB-INF/lib folder. Regards, Ian On Jun 10, 10:18 am, paul pjorti...@gmail.com wrote: Hi, This is the first time that I have worked with Datastore, I followed the tutorial herehttp://code.google.com/appengine/docs/java/datastore/usingjdo.html, but whenever I run the project I got this error: org.datanucleus.jdo.exceptions.ClassNotPersistenceCapableException: The class The class my class is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data/annotations for the class are not found. is not persistable. This means that it either hasnt been enhanced, or that the enhanced version of the file is not in the CLASSPATH (or is hidden by an unenhanced version), or the Meta-Data for the class is not found. Error on MailHandler at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java: 241) at org.datanucleus.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java: 674) at org.datanucleus.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java: 694) at com.waveassistant.app.MailHandler.doPost(MailHandler.java:82) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) ... Also, I tried to clean/build the project, just maybe there is a problem on the logs. So I got this error also: java.lang.RuntimeException: Unexpected exception at com.google.appengine.tools.enhancer.Enhancer.execute(Enhancer.java: 59) at com.google.appengine.tools.enhancer.Enhance.init(Enhance.java: 60) at com.google.appengine.tools.enhancer.Enhance.main(Enhance.java:41) Caused by: java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Unknown Source) at com.google.appengine.tools.enhancer.Enhancer.execute(Enhancer.java: 57) ... 2 more Caused by: org.datanucleus.exceptions.NucleusException: Plugin (Bundle) org.datanucleus.store.appengine is already registered. Ensure you dont have multiple JAR versions of the same plugin in the classpath. The URL file:/D:/MasteringWave/Eclipse/WaveAssistant/war/ WEB-INF/lib/datanucleus-appengine-1.0.6.final.jar is already registered, and you are trying to register an identical plugin located at URL file:/C:/eclipse/plugins/ com.google.appengine.eclipse.sdkbundle.1.3.3_1.3.3.v20100423/ appengine-java-sdk-1.3.3.1/lib/user/orm/datanucleus- appengine-1.0.6.final.jar. at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java: 434) at org.datanucleus.plugin.NonManagedPluginRegistry.registerBundle(NonManagedPluginRegistry.java: 340) at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensions(NonManagedPluginRegistry.java: 222) at org.datanucleus.plugin.NonManagedPluginRegistry.registerExtensionPoints(NonManagedPluginRegistry.java: 153) at org.datanucleus.plugin.PluginManager.registerExtensionPoints(PluginManager.java: 82) at org.datanucleus.OMFContext.init(OMFContext.java:160) at org.datanucleus.enhancer.DataNucleusEnhancer.init(DataNucleusEnhancer.java: 172) at org.datanucleus.enhancer.DataNucleusEnhancer.init(DataNucleusEnhancer.java: 150) at org.datanucleus.enhancer.DataNucleusEnhancer.main(DataNucleusEnhancer.java: 1157) ... 7 more Please Advise. Thanks in advance, Paul -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to
Re: [appengine-java] GUICE injection and access denied (java.lang.RuntimePermission modifyThreadGroup)
It seems to do no difference if I add the @Singleton to PostalManager or not. Also, I've tried both; public class MyModule extends AbstractModule { @Override protected void configure() { } } and public class MyModule extends AbstractModule { @Override protected void configure() { bind(PostalManager.class).asEagerSingleton(); } } Any other ideas as to why I get the exception ? Thanks. On Fri, Jun 11, 2010 at 10:10 AM, RAVINDER MAAN rsmaan...@gmail.com wrote: have u declared PostalManager as @Singleton. On Fri, Jun 11, 2010 at 12:42 PM, KasperDK kbhdk1...@gmail.com wrote: Hi, I'm trying to do this; public class PostalServiceImpl extends RemoteServiceServlet implements PostalService { private static final Logger log = Logger.getLogger(PostalServiceImpl.class.getName()); �...@inject private PostalManager postalManager; public PostalServiceImpl() { Injector injector = Guice.createInjector(new NetGazelleModule()); injector.injectMembers(this); } public String getCity(String postal) { log.info(PostalServiceImpl.getCity( + postal + )); if (postalManager == null) { log.severe(PostalServiceImpl.getCity: postalManager is null !); } return postalManager.getCity(postal); } ... But I get the following exception; INFO: The server is running at http://localhost:/ Jun 11, 2010 6:38:57 AM com.google.inject.internal.FinalizableReferenceQueue init INFO: Failed to start reference finalizer thread. Reference cleanup will only occur when new references are created. java.lang.reflect.InvocationTargetException at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java: 39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java: 25) at java.lang.reflect.Method.invoke(Method.java:597) at com.google.appengine.tools.development.agent.runtime.Runtime.invoke(Runtime.java: 100) at com.google.inject.internal.FinalizableReferenceQueue.init(FinalizableReferenceQueue.java: 124) at com.google.inject.internal.MapMaker $QueueHolder.clinit(MapMaker.java:787) at com.google.inject.internal.MapMaker$WeakEntry.init(MapMaker.java: 946) at com.google.inject.internal.MapMaker$Strength $1.newEntry(MapMaker.java:312) at com.google.inject.internal.MapMaker $StrategyImpl.newEntry(MapMaker.java:498) at com.google.inject.internal.MapMaker $StrategyImpl.newEntry(MapMaker.java:419) at com.google.inject.internal.CustomConcurrentHashMap $ComputingImpl.get(CustomConcurrentHashMap.java:2029) at com.google.inject.internal.BytecodeGen.getClassLoader(BytecodeGen.java: 149) at com.google.inject.internal.BytecodeGen.getClassLoader(BytecodeGen.java: 132) at com.google.inject.internal.BytecodeGen.newFastClass(BytecodeGen.java: 162) at com.google.inject.DefaultConstructionProxyFactory $1.init(DefaultConstructionProxyFactory.java:54) at com.google.inject.DefaultConstructionProxyFactory.create(DefaultConstructionProxyFactory.java: 52) at com.google.inject.ProxyFactory.create(ProxyFactory.java:152) at com.google.inject.ConstructorInjectorStore.createConstructor(ConstructorInjectorStore.java: 84) at com.google.inject.ConstructorInjectorStore.access $000(ConstructorInjectorStore.java:31) at com.google.inject.ConstructorInjectorStore $1.create(ConstructorInjectorStore.java:39) at com.google.inject.ConstructorInjectorStore $1.create(ConstructorInjectorStore.java:35) at com.google.inject.internal.FailableCache $1.apply(FailableCache.java:35) at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:549) at com.google.inject.internal.MapMaker $StrategyImpl.compute(MapMaker.java:419) at com.google.inject.internal.CustomConcurrentHashMap $ComputingImpl.get(CustomConcurrentHashMap.java:2041) at com.google.inject.internal.FailableCache.get(FailableCache.java: 46) at com.google.inject.ConstructorInjectorStore.get(ConstructorInjectorStore.java: 52) at com.google.inject.ConstructorBindingImpl.initialize(ConstructorBindingImpl.java: 57) at com.google.inject.InjectorImpl.initializeBinding(InjectorImpl.java: 377) at com.google.inject.BindingProcessor$1$1.run(BindingProcessor.java: 169) at com.google.inject.BindingProcessor.initializeBindings(BindingProcessor.java: 224) at com.google.inject.InjectorBuilder.initializeStatically(InjectorBuilder.java: 120) at com.google.inject.InjectorBuilder.build(InjectorBuilder.java:105) at
[appengine-java] Records Count
How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: JDO - Deleted data reappears
Hi Ian, Thanks for your reply. I expect I only had the server running for a minute or so at the time so that could well explain it. I have not seen it happen again today so I will assume this is the case and keep pushing ahead with what i'm doing. Regards, Andrew On 11/06/2010, at 6:30 PM, Ian Marshall ianmarshall...@gmail.com wrote: Hi Andrew, Without looking at your code, how long elapses between deleting your items using your web application and stopping your development server? I have found that writes to my local GAE/J datastore can take up to 1 minute to be written to the relevant file on my local file system (which the dev app server uses as its persistent datastore). I always wait around two minutes after the last update to this file before stopping my dev app server. Regards, Ian On Jun 10, 7:38 am, Andrew am2...@gmail.com wrote: Hi, I'm in the very early stages of building an app with GAE and am trying to do my first CRUD process. Unfortunately I've run into a problem where when I delete items they seem to disappear during the current session in development, but when I stop and restart the development server they reappear. I've pasted my DAO code here if anyone is able to give me some advice on what I may have done wrong: http://java.pastebin.com/Jr23cMTc Best regards, Andrew. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Disable URLFetchService cache
Ooopppss.. my mistake, im looking at an old version of my app. I think setting the headers with no-cache is enough and no need for the timestamp. On Jun 11, 6:27 pm, mar_novice mariocape1...@gmail.com wrote: Unfortunately, even using URL urlObj = new URL(url); HttpURLConnection connection = (HttpURLConnection) urlObj.openConnection(); connection.addRequestProperty(Cache-Control, no-cache,max-age=0); connection.addRequestProperty(Pragma, no-cache); sometimes give me cached results. I'll try combining the two suggestions. On Jun 11, 9:48 am, mar_novice mariocape1...@gmail.com wrote: How quickly? Not that fast. I'm still testing my app and upon refreshing the page, sometimes what I get are the cached results. I even get sometimes the result that was supposed to be yesterday's result, meaning, a day old result. Can you somehow explain how the urlfetchservice cache works so that we can somehow avoid getting unexpected results from it. And I thought urlfetch does not do caching but it does. It seems that it is not in the documentation or I just missed it. Adding a timestamp seems a good workaround. On Jun 11, 3:01 am, Ikai L (Google) ika...@google.com wrote: How quickly are you making requests? One thing to try - I can't guarantee it'll work - is to append a timestamp parameters to the URL like this:http://www.somedomain.com?v=sometimestamp. You'll have to balance this against your latency requirements, as the cache is several orders of magnitude faster. On Wed, Jun 9, 2010 at 5:29 AM, mar_novice mariocape1...@gmail.com wrote: Is there a way to disable urlfetchService cache? When I made a urlfetch, sometimes, what I get is the cached result. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine Blog:http://googleappengine.blogspot.com Twitter:http://twitter.com/app_engine Reddit:http://www.reddit.com/r/appengine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Records Count
Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Performance issue for GAE auto-restart
Hi, just joining in, trying to sum up: In the best case (if we only use the low level datastore + no frameworks on top) we can only hope that a fresh restart of our app will take 4.5 seconds? However if at least one user is hitting the site every 60 seconds from somewhere in the world, then our app should be kept alive and no need for restarts? Not that I'm planning on doing it, but what prevents developers from simply pinging the site every 60 seconds to force the app to stay in memory? I ask what prevents it because if I play by the rules, I'm going to get penalized if other developers are 'cheating' and doing the artificial pinging, Thanks On Jun 9, 5:49 am, Jake jbrooko...@cast.org wrote: Hey Tin, Several people were using GAE's built in timing mechanism to ping the server to accomplish the same thing. Google came out and officially discouraged this as it tends to throw off any attempts they are currently making to fix the problem. I can't tell people what to do, but I opted to stop using this hack in the hopes that they will solve the problem. Besides, you'll notice that this hack doesn't really work over the long run. It will essentially restart your server every minute and won't really prevent the loading request from falling on your users instead of on your ping. Jake On Jun 8, 1:54 pm, Tin tin.c...@gmail.com wrote: Hi all: We have found a temporary solution for this issue: Try with an AJAX timer kicking the server (doing nothing), maybe one request per minute (or less) and GAE will keep our site / users in the same node. In our testing it could avoid the GAE web instance reloaded, but if the request interval is long, it would cause another Datastore performance issue:http://goo.gl/98zkthatwill be fixed in near future. More info please refer to here:http://goo.gl/mzQR -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Performance issue for GAE auto-restart
Hi Mark: The 4.5 seconds is for struts 1.2 in my case. The startup time of 1.3~3.9 seconds is reported by using a more lightweight framework (slim3). Please refer to http://groups.google.com/group/slim3-user/browse_thread/thread/e9d2f7a52f3408 On 6月11日, 下午7時32分, Mark mar...@gmail.com wrote: Hi, just joining in, trying to sum up: In the best case (if we only use the low level datastore + no frameworks on top) we can only hope that a fresh restart of our app will take 4.5 seconds? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Records Count
Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: App Engine Scalability
This is relevant too http://code.google.com/p/googleappengine/issues/detail?id=2456 On Jun 10, 11:31 pm, Thomas mylee...@gmail.com wrote: Hi: Please seehttp://groups.google.com/group/google-appengine-java/browse_thread/th... for more detail. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: MD5 Hash
Hey, Nothing additional on the build path. I am using Enterprise Edition of Java. I'm pretty sure the security package falls under Standard Edition, but meh. Who knows? Jake On Jun 10, 4:59 pm, Michael mrher...@gmail.com wrote: I am using eclipse, do I need to add it to my build path? I don't think I need to since it is not an external library? I'll play around with it some more. Hopfully I can find something. Michael On Jun 10, 3:35 pm, Jake jbrooko...@cast.org wrote: Hey, The method below works fine for me. No extra dependencies. Just worked. So, not at all helpful other than perhaps something wonky with your java install? Jake import java.security.MessageDigest; private static String md5Hex (String email) { try { MessageDigest md = MessageDigest.getInstance(MD5); // more stuff } catch (Exception ex) { /* No Action */ } } On Jun 10, 2:42 pm, Michael mrher...@gmail.com wrote: Any ideas on what is wrong then? On Jun 10, 1:03 pm, Nacho Coloma icol...@gmail.com wrote: That should not be your problem:http://code.google.com/appengine/docs/java/jrewhitelist.html I am using MD5 in AppEngine without issues. On Jun 10, 3:17 pm, Michael mrher...@gmail.com wrote: Hi, I am pretty new to app engine (and Java) and I am seem to have run in to a wall. I want to do an MD5 hash and I was trying to use the java.security package, but I get: [ERROR] Line 3: The import java.security cannot be resolved I am thinking that it is because app engine can't use java.security? Here is the MD5 code I have import java.security.MessageDigest; import java.security.NoSuchAlgorithmException; ... try { MessageDigest digest = java.security.MessageDigest.getInstance(MD5); digest.update(test.getBytes()); PW = digest.digest().toString(); } catch (NoSuchAlgorithmException e) { } Am I doing something wrong, or is there a different way to get a MD5 hash? I also need to do the same thing client side using GWT if their is any advice for that too. Thanks Michael -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Records Count
A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Records Count
number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.comwrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Performance issue for GAE auto-restart
It would be great if JPA/JDO could be disabled in the Google eclipse plugin. I don't need it, but I've not yet figured out how to get rid of the jars... On 7 Jun., 17:59, Jake jbrooko...@cast.org wrote: Hi all, @Thomas: I will look into that, thank you. The way we generate our pages in this framework, however, involves some interesting dynamic generation of image URLs that are mounted within the application itself. It ends up being a bit more complicated than just displaying one string or another for the src attribute. Whether this is a good practice on GAE is certainly up for debate, but I presume this wouldn't help the frequent restart issue, correct? It just means that fewer files would be able to trigger a restart? @Tin: I'm not sure I understand what I will learn from this. From what I understand, contextDestroyed() is never called so all I would see is an additional log statement near GAE's loading request logging line? I've never used a ServletContextListener, so perhaps it has other features that I'm not aware of? Thanks! Jake On Jun 7, 11:41 am, Tin tin.c...@gmail.com wrote: Hi Jack: I suggest that maybe you can write a ServletContextListener to monitor the web instance reloading issue, for our testing we just put a info logging in the contextInitialized(..) method: public void contextInitialized(ServletContextEvent evt) { log.info(Web context inited.); } And modify the DEFAULT logging level to INFO in the logging.properties file. If the web instance was reloaded, we could trace this in the logging messages. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Performance issue for GAE auto-restart
Hi Mark, Yes, you can do better than 4.5 seconds without a framework. Also, there are layers that can be placed over the low level datastore (e.g Objectify) that add ease-of-use without noticeable additions to the startup time. The 60 second pinging thing is, indeed, done by some users and is officially discouraged by Google. At one point, I set up a similar pinging feature and found that it really didn't work. Theoretically it should, but the pings just caused loading requests every minute or so, which simply reduced the likelihood that a user would see the loading request; it didn't solve the problem. Jake On Jun 11, 7:32 am, Mark mar...@gmail.com wrote: Hi, just joining in, trying to sum up: In the best case (if we only use the low level datastore + no frameworks on top) we can only hope that a fresh restart of our app will take 4.5 seconds? However if at least one user is hitting the site every 60 seconds from somewhere in the world, then our app should be kept alive and no need for restarts? Not that I'm planning on doing it, but what prevents developers from simply pinging the site every 60 seconds to force the app to stay in memory? I ask what prevents it because if I play by the rules, I'm going to get penalized if other developers are 'cheating' and doing the artificial pinging, Thanks On Jun 9, 5:49 am, Jake jbrooko...@cast.org wrote: Hey Tin, Several people were using GAE's built in timing mechanism to ping the server to accomplish the same thing. Google came out and officially discouraged this as it tends to throw off any attempts they are currently making to fix the problem. I can't tell people what to do, but I opted to stop using this hack in the hopes that they will solve the problem. Besides, you'll notice that this hack doesn't really work over the long run. It will essentially restart your server every minute and won't really prevent the loading request from falling on your users instead of on your ping. Jake On Jun 8, 1:54 pm, Tin tin.c...@gmail.com wrote: Hi all: We have found a temporary solution for this issue: Try with an AJAX timer kicking the server (doing nothing), maybe one request per minute (or less) and GAE will keep our site / users in the same node. In our testing it could avoid the GAE web instance reloaded, but if the request interval is long, it would cause another Datastore performance issue:http://goo.gl/98zkthatwillbe fixed in near future. More info please refer to here:http://goo.gl/mzQR -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Can't delete blobs via Google's admin UI
Does anybody else has the same problem? An answer to this question would help me a lot. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Records Count
Have u solved the problem of like SQL? If yes then please share with me as i have not been able to solve it. On Jun 11, 2:53 pm, RAVINDER MAAN rsmaan...@gmail.com wrote: number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.comwrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit
Re: [appengine-java] Re: Records Count
No Ravi I did not find any solution yet.Neither for like nor for count.Its such a basic thing it think there must be a way.I dont see any reply from google people too. On Fri, Jun 11, 2010 at 8:12 PM, Ravi ping2r...@gmail.com wrote: Have u solved the problem of like SQL? If yes then please share with me as i have not been able to solve it. On Jun 11, 2:53 pm, RAVINDER MAAN rsmaan...@gmail.com wrote: number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.com wrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to
[appengine-java] Re: Performance issue for GAE auto-restart
Jake, Is there any official comments on this from google which says that it discourage the pinging every 60 seconds or so. Also if we have some framework which we will always like to have if doing some production application then what is the way out. Thanks, Rahul On Jun 11, 10:16 am, Jake jbrooko...@cast.org wrote: Hi Mark, Yes, you can do better than 4.5 seconds without a framework. Also, there are layers that can be placed over the low level datastore (e.g Objectify) that add ease-of-use without noticeable additions to the startup time. The 60 second pinging thing is, indeed, done by some users and is officially discouraged by Google. At one point, I set up a similar pinging feature and found that it really didn't work. Theoretically it should, but the pings just caused loading requests every minute or so, which simply reduced the likelihood that a user would see the loading request; it didn't solve the problem. Jake On Jun 11, 7:32 am, Mark mar...@gmail.com wrote: Hi, just joining in, trying to sum up: In the best case (if we only use the low level datastore + no frameworks on top) we can only hope that a fresh restart of our app will take 4.5 seconds? However if at least one user is hitting the site every 60 seconds from somewhere in the world, then our app should be kept alive and no need for restarts? Not that I'm planning on doing it, but what prevents developers from simply pinging the site every 60 seconds to force the app to stay in memory? I ask what prevents it because if I play by the rules, I'm going to get penalized if other developers are 'cheating' and doing the artificial pinging, Thanks On Jun 9, 5:49 am, Jake jbrooko...@cast.org wrote: Hey Tin, Several people were using GAE's built in timing mechanism to ping the server to accomplish the same thing. Google came out and officially discouraged this as it tends to throw off any attempts they are currently making to fix the problem. I can't tell people what to do, but I opted to stop using this hack in the hopes that they will solve the problem. Besides, you'll notice that this hack doesn't really work over the long run. It will essentially restart your server every minute and won't really prevent the loading request from falling on your users instead of on your ping. Jake On Jun 8, 1:54 pm, Tin tin.c...@gmail.com wrote: Hi all: We have found a temporary solution for this issue: Try with an AJAX timer kicking the server (doing nothing), maybe one request per minute (or less) and GAE will keep our site / users in the same node. In our testing it could avoid the GAE web instance reloaded, but if the request interval is long, it would cause another Datastore performance issue:http://goo.gl/98zkthatwillbefixed in near future. More info please refer to here:http://goo.gl/mzQR -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Session object or memcache?
No doubt if your application is low traffic you will definitely need memcache as session will be lost every minute when the jvm restarts and you need to reload everything and also session is highly discouraged due to various reasons. Thanks, Rahul On Jun 11, 4:16 am, RAVINDER MAAN rsmaan...@gmail.com wrote: Hello all what is the best option from performance point of view, storing frequently used objects in session or loading from datastore for every request(we can add memcache support before trying to fetch from datastore).Can anybody tell me good link to develop high performance and fast application development best practices link.I am using java with JDO. thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Session object or memcache?
I was not aware about loss of session objects.strange!! On Fri, Jun 11, 2010 at 8:39 PM, Rahul rahul.jun...@gmail.com wrote: No doubt if your application is low traffic you will definitely need memcache as session will be lost every minute when the jvm restarts and you need to reload everything and also session is highly discouraged due to various reasons. Thanks, Rahul On Jun 11, 4:16 am, RAVINDER MAAN rsmaan...@gmail.com wrote: Hello all what is the best option from performance point of view, storing frequently used objects in session or loading from datastore for every request(we can add memcache support before trying to fetch from datastore).Can anybody tell me good link to develop high performance and fast application development best practices link.I am using java with JDO. thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Records Count
The best you could probably do is load into memory a structure that keeps the employee type for every employee. When the user types some letters of a name to search for you can perform a keys-only query and lookup the employee type in this structure (rather than load the entity) You would need 2 bits per employee (for 4 types) so even a million employees would only require 250KB of memory (an array of 30K longs) which shouldn't take too long to read from memcache when an instance starts. You would then need to iterate over every result and count which type the employee is. To optimise this you might cache the counts in memcache of 1 and 2 letter searches. So then you would only need to count on the fly three letter searches i.e. all employees with names starting with smi On 11 Jun 2010, at 20:53, RAVINDER MAAN wrote: number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.com wrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to
[appengine-java] Re: update_indexes error
Can someone on the app engine team help me out? Thanks, Keyur On Jun 10, 5:45 pm, keyurva keyu...@gmail.com wrote: I have a Java app deployed on app engine and I use appcfg.py of the Python SDK to vacuum and update my indexes. Today I first ran vacuum_indexes and that completed successfully - i.e. it en-queued tasks to delete my existing indexes. The next step was probably a mistake on my part - I then ran update_indexes even though my previous indexes weren't yet deleted. Needless to say that my update_indexes call errored out. So much so that now when I look at my app engine console, it shows the status of all my indexes as Error. Can someone help my out of my fix?! Thanks, Keyur -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Discussion on will-it-play-in-app-engine
I was able to run drools (http://www.jboss.org/drools) on GAE. Required libraries, that works fine: * drools-api-5.0.1.jar * drools-compiler-5.0.1.jar * antlr.jar * antlr-runtime-3.1.1.jar Problemtic ones: * mvel2-2.0.18.jar - At least this version is required, older ones have compatibility issues with GAE * core-3.4.2.v_883_R34x-unsigned.jar - All official distributions are signed. Unfortunatelly, eclipse certificate causes some problems on GAE. I had to unsign this jar manually in order to use them on GAE. * drools-core-5.0.1.jar - There are two uncompatible classes in this library: org.drools.util.ChainedProperties: it contains 'ClassLoader.getSystemClassLoader()', which is illegal code on GAE. It is enough to comment all those invocation - they are unneeded. org.drools.util.asm.ClassFieldInspector: calling method 'private String getResourcePath(final Class ? clazz)' ends up with hard to track StackOverflowError. To fix it you have to change body of this method to 'return / + clazz.getCanonicalName().replace(., /) + .class;'. Replacing all '.' with '/' in class names solves everything! I modified those classes (if you need source code, just send me an e- mail), put them into WEB-INF/classes (to be sure that my version will be loaded before classes from drools-core-5.0.1.jar) And that's all. Let me know if you have some problems with this instruction -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Unable to locate the JVM error on Eclipse Galileo on Snow Leopard
Hello, I'm getting Unable to locate the JVM when i try to create a demo project. My configuration details - Mac OS X 10.6.3 - Eclipse Galileo - java version 1.6.0_20 - javac 1.6.0_20 Thanks in advance for the help. Cheers /Ven -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Task Queue
Hi i'm new to task queue API concept and its difficult for me to understand the concept theoretically so plz send me one simple complete Task queue program it helps me to understand it. Plz Help me. Regards Sathish -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Transaction exception when not using a transaction ?
I have the same exception with my app. Here is my code: PersistenceManager pm = PMF.get().getPersistenceManager(); Query query = this.pm.newQuery(select key from + Subscription.class.getName() + where nextMail = :dateParam status == :statusParam); ListKey keys = (ListKey) query.execute(new Date(), 1); for (Key key: keys) { Queue queue = QueueFactory.getDefaultQueue(); TaskOptions task = url('handler'); task.param('key', KeyFactory.keyToString(key)); queue.add(task); } pm.close(); This throws the exception during pm.close() : NestedThrowablesStackTrace: java.lang.IllegalArgumentException: can't operate on multiple entity groups in a single transaction. at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(DatastoreApiHelper.java: 34) at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(DatastoreApiHelper.java: 67) at com.google.appengine.api.datastore.DatastoreServiceImpl $1.run(DatastoreServiceImpl.java:128) at com.google.appengine.api.datastore.TransactionRunner.runInTransaction(TransactionRunner.java: 30) at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java: 111) at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java: 84) at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java: 72) at org.datanucleus.store.appengine.RuntimeExceptionWrappingDatastoreService.get(RuntimeExceptionWrappingDatastoreService.java: 63) at org.datanucleus.store.appengine.DatastorePersistenceHandler.get(DatastorePersistenceHandler.java: 96) at org.datanucleus.store.appengine.DatastorePersistenceHandler.get(DatastorePersistenceHandler.java: 106) at org.datanucleus.store.appengine.DatastorePersistenceHandler.fetchObject(DatastorePersistenceHandler.java: 478) at org.datanucleus.state.JDOStateManagerImpl.loadUnloadedFieldsInFetchPlan(JDOStateManagerImpl.java: 1627) at org.datanucleus.state.JDOStateManagerImpl.detach(JDOStateManagerImpl.java: 3560) at org.datanucleus.ObjectManagerImpl.performDetachOnClose(ObjectManagerImpl.java: 3270) at org.datanucleus.ObjectManagerImpl.close(ObjectManagerImpl.java: 807) at org.datanucleus.jdo.JDOPersistenceManager.close(JDOPersistenceManager.java: 271) The only thing I can think of is that PMF.get() is being recycled from a previous request, and somehow it is in a bad state. But the doc clearly says that we should recycle this object. What is going on ? On Jun 8, 9:05 am, JD liva...@gmail.com wrote: I am not using atransaction(see original POST), which is the problem. On Jun 4, 11:03 am, Chau Huynh cmhu...@gmail.com wrote: Did you have different entity groups inside atransaction? Google have a constraint of what can be done in atransactionherehttp://code.google.com/appengine/docs/python/datastore/transactions.h... On Fri, Jun 4, 2010 at 5:45 PM, JD liva...@gmail.com wrote: After debugging, I was able to get rid of the exception by removing a pm.newQuery() which was happening on a different entity group. To me this is a bug in App Engine, as there is no reason the query on a particular entity group should fail outside atransaction. On Jun 3, 12:25 pm, Millisecond millisec...@gmail.com wrote: Hmmm. How is your object model structured? Do entities have other entities as direct references? If you have class A { B b } and setting b to an instance of B that already existed might throw the error you're seeing. I changed my structure to be class A { String bKey } for a variety of other reasons, but guess it also may have gotten around this problem. Alternatively, post as much code as you can and I'll see if anything else rings a bell, early on I spent a frustrated day or two working around this exception in a variety of places so know it can be frustrating. -C On Jun 3, 6:33 am, JD liva...@gmail.com wrote: I added a call to flush() after every change on the PM-managed object, but that did not help. Also doing a pm.flush() does not throw this exception, but doing a pm.close() does - which seems to be contrary to your reasoning (I would expect flush to throw the same exception if multiple entity groups were at cause). On Jun 3, 1:33 am, Millisecond millisec...@gmail.com wrote: Even though you're not using transactions, I think it's trying to make the .close() call atomic (maybe with an internal implicit transaction), failing or succeeding as a whole. And as I understand it, entities not in the same group can be stored on separate machines so can't be operated on atomically. I've worked around this by calling a flush() after most any change on a PM-managed object until close(). If this has a huge number of
[appengine-java] How can I acquire blobkeys by uploaded input(type:file multiple)
How can I acquire blobkeys by uploaded input(type:file multiple) Hi. I develop project by GAE/J(Eclipse3.5 + SDK1.3.3.1). When describing it by the following sources, two or more files cannot be acquired well. JSP body % BlobstoreService blobstoreService = BlobstoreServiceFactory.getBlobstoreService(); % form action=%= blobstoreService.createUploadUrl(upload) % method=post enctype=multipart/form-data ... input type=file name=myFile id=myFile multiple / input type=submit value=upload /form --- JAVA BlobstoreService service; MapString, BlobKey blobs; BlobKey blobkey; BlobInfoFactory blobfactory; BlobInfo blobinfo; request = RequestLocator.get(); service = BlobstoreServiceFactory.getBlobstoreService(); blobs = service.getUploadedBlobs(request); blobkey = blobs.get(myFile); //get blob info blobfactory = new BlobInfoFactory(); blobinfo = blobfactory.loadBlobInfo(blobkey); When two or more files are specified input(id=myFile) area, only last file blobkey can be acquired. How can I acquire all blobkeys by 'service.getUploadedBlobs(request)'? I do not know how to do it. Could anyone help me? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Performance issue for GAE auto-restart
Hmm ok I see, so I'm just using JDO without any frameworks on top of that, but as mentioned in the slim3 tests, JDO does add overhead (sometimes a significant amount). I don't plan to be doing any large queries (the slim3 example fetches 10,000 objects which I don't plan on doing at any point), but what is important to me is that startup time. One to two seconds is fine, four seconds starts pushing it, anything above that is pretty bad. If we start using slim3 though, we lose the flexibility in moving to another host (not that I know of any that can compare to app engine) since it'll be interacting directly with big table instead of using jdo or jpa is an intermediate? Thanks On Fri, Jun 11, 2010 at 8:07 AM, Rahul rahul.jun...@gmail.com wrote: Jake, Is there any official comments on this from google which says that it discourage the pinging every 60 seconds or so. Also if we have some framework which we will always like to have if doing some production application then what is the way out. Thanks, Rahul On Jun 11, 10:16 am, Jake jbrooko...@cast.org wrote: Hi Mark, Yes, you can do better than 4.5 seconds without a framework. Also, there are layers that can be placed over the low level datastore (e.g Objectify) that add ease-of-use without noticeable additions to the startup time. The 60 second pinging thing is, indeed, done by some users and is officially discouraged by Google. At one point, I set up a similar pinging feature and found that it really didn't work. Theoretically it should, but the pings just caused loading requests every minute or so, which simply reduced the likelihood that a user would see the loading request; it didn't solve the problem. Jake On Jun 11, 7:32 am, Mark mar...@gmail.com wrote: Hi, just joining in, trying to sum up: In the best case (if we only use the low level datastore + no frameworks on top) we can only hope that a fresh restart of our app will take 4.5 seconds? However if at least one user is hitting the site every 60 seconds from somewhere in the world, then our app should be kept alive and no need for restarts? Not that I'm planning on doing it, but what prevents developers from simply pinging the site every 60 seconds to force the app to stay in memory? I ask what prevents it because if I play by the rules, I'm going to get penalized if other developers are 'cheating' and doing the artificial pinging, Thanks On Jun 9, 5:49 am, Jake jbrooko...@cast.org wrote: Hey Tin, Several people were using GAE's built in timing mechanism to ping the server to accomplish the same thing. Google came out and officially discouraged this as it tends to throw off any attempts they are currently making to fix the problem. I can't tell people what to do, but I opted to stop using this hack in the hopes that they will solve the problem. Besides, you'll notice that this hack doesn't really work over the long run. It will essentially restart your server every minute and won't really prevent the loading request from falling on your users instead of on your ping. Jake On Jun 8, 1:54 pm, Tin tin.c...@gmail.com wrote: Hi all: We have found a temporary solution for this issue: Try with an AJAX timer kicking the server (doing nothing), maybe one request per minute (or less) and GAE will keep our site / users in the same node. In our testing it could avoid the GAE web instance reloaded, but if the request interval is long, it would cause another Datastore performance issue:http://goo.gl/98zkthatwillbefixed in near future. More info please refer to here:http://goo.gl/mzQR -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Records Count
thanks john any thought for like operator of sql.how can we implement like operator of sql in GAE On Fri, Jun 11, 2010 at 8:49 PM, John Patterson jdpatter...@gmail.comwrote: The best you could probably do is load into memory a structure that keeps the employee type for every employee. When the user types some letters of a name to search for you can perform a keys-only query and lookup the employee type in this structure (rather than load the entity) You would need 2 bits per employee (for 4 types) so even a million employees would only require 250KB of memory (an array of 30K longs) which shouldn't take too long to read from memcache when an instance starts. You would then need to iterate over every result and count which type the employee is. To optimise this you might cache the counts in memcache of 1 and 2 letter searches. So then you would only need to count on the fly three letter searches i.e. all employees with names starting with smi On 11 Jun 2010, at 20:53, RAVINDER MAAN wrote: number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.comwrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received
Re: [appengine-java] Re: Records Count
To do a suffix search, instead of storing a field value smith you would need to store a multi valued property with smith, mith, ith... You can do a prefix search without the need to do this. On 12 Jun 2010, at 00:12, RAVINDER MAAN wrote: thanks john any thought for like operator of sql.how can we implement like operator of sql in GAE On Fri, Jun 11, 2010 at 8:49 PM, John Patterson jdpatter...@gmail.com wrote: The best you could probably do is load into memory a structure that keeps the employee type for every employee. When the user types some letters of a name to search for you can perform a keys-only query and lookup the employee type in this structure (rather than load the entity) You would need 2 bits per employee (for 4 types) so even a million employees would only require 250KB of memory (an array of 30K longs) which shouldn't take too long to read from memcache when an instance starts. You would then need to iterate over every result and count which type the employee is. To optimise this you might cache the counts in memcache of 1 and 2 letter searches. So then you would only need to count on the fly three letter searches i.e. all employees with names starting with smi On 11 Jun 2010, at 20:53, RAVINDER MAAN wrote: number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.com wrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- Regards, Ravinder Singh Maan -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe
[appengine-java] Re: update_indexes error
FWIW, I followed what was suggested in the error logs and that worked for me: 1. Empty the index.yaml file (create a backup first) 2. Run vacuum_indexes again 3. Look at your app's admin console and don't go to the next step till all your indexes are deleted. 4. Specify the indexes you want to be created in index.yaml 5. Run update_indexes 6. Look at your app's admin console and it should show that your indexes are now building. 7. Enjoy the fruits of your labor :) Cheers, Keyur On Jun 11, 9:45 am, keyurva keyu...@gmail.com wrote: Can someone on the app engine team help me out? Thanks, Keyur On Jun 10, 5:45 pm, keyurva keyu...@gmail.com wrote: I have a Java app deployed on app engine and I use appcfg.py of the Python SDK to vacuum and update my indexes. Today I first ran vacuum_indexes and that completed successfully - i.e. it en-queued tasks to delete my existing indexes. The next step was probably a mistake on my part - I then ran update_indexes even though my previous indexes weren't yet deleted. Needless to say that my update_indexes call errored out. So much so that now when I look at my app engine console, it shows the status of all my indexes as Error. Can someone help my out of my fix?! Thanks, Keyur -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Can't delete blobs via Google's admin UI
Who has NOT this problem? Is there help out there?! Hello?! I can't delete blobs I've uploaded and I don't think I'm in a singularity that could explain this miss-behavior. Comments from Googlers like We're working on this issue! or WTF are you talking about?! would really help. On Jun 11, 4:32 pm, Alexander Orlov alexander.or...@loxal.net wrote: Does anybody else has the same problem? An answer to this question would help me a lot. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Application that holds images dynamically(Open Image Server)
Hi, everybody. I wrote a web app that lets you upload images and store them in app engine. it is very easy to use (and very easy to integrate in other projects). if you need such functionality, please consider using this one. You can get source code from homepage http://sourceforge.net/projects/openimageserver/ . To see how it works just go to showcase ( http://oisshowcase.appspot.com ) , create some albums upload some images and let me know what you think. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Uploading to blobstore gives OutOfMemoryError
I have the same issue. On Jun 4, 11:51 am, eighty eightyste...@gmail.com wrote: +1 On Jun 2, 4:14 pm, Jean Hsu jeanhs...@gmail.com wrote: Hi all, I am trying to set up a basic file upload to blobstore, but I get this OutOfMemoryError: WARNING: Error for /_ah/upload/ aghvbWdkcmVzc3IcCxIVX19CbG9iVXBsb2FkU2Vzc2lvbl9fGMACDA java.lang.OutOfMemoryError: Java heap space at java.util.Arrays.copyOf(Arrays.java:2786) at java.io.ByteArrayOutputStream.write(ByteArrayOutputStream.java:71) at javax.mail.internet.MimeMultipart.readTillFirstBoundary(MimeMultipart.java: 316) at javax.mail.internet.MimeMultipart.parse(MimeMultipart.java:186) at javax.mail.internet.MimeMultipart.getCount(MimeMultipart.java:109) at com.google.appengine.api.blobstore.dev.UploadBlobServlet.handleUpload(Uploa dBlobServlet.java: 135) at com.google.appengine.api.blobstore.dev.UploadBlobServlet.access $000(UploadBlobServlet.java:72) at com.google.appengine.api.blobstore.dev.UploadBlobServlet $1.run(UploadBlobServlet.java:100) at java.security.AccessController.doPrivileged(Native Method) at com.google.appengine.api.blobstore.dev.UploadBlobServlet.doPost(UploadBlobS ervlet.java: 98) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 511); I used the Memory Analyzer on Eclipse and it said that the memory leak suspect is QueuedThreadPool. I found this information about a memory leak bug: http://jira.codehaus.org/browse/JETTY-1188 How can I figure out what release of jetty is running locally? It looks like they fixed this in version 6.1.23: http://jira.codehaus.org/secure/ReleaseNote.jspa?projectId=11160vers... Has anyone else had this issue? Thanks, Jean -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Performance issue for GAE auto-restart
Hi: You can define the dao interfaces and write the implementation of the interfaces with datastore low level api or some fast startup framework other than JDO or JPA. If you need to migrate to other platform which supports standard RDBMS, you implement the dao interface with JDO, JPA or JDBC. The entity relationship model design in appengine is very different from other RDBMS. You can not use more than two inequality filters or the 'OR' operator in your query. So the design which works great with appengine is probably not good with RDBMS and vice versa. I think even the model redesign is necessary when you leave appengine. Unless you don't choose to use appengine, use JDO(JPA) save you nothing. On 6月12日, 上午1時07分, Mark Wyszomierski mar...@gmail.com wrote: Hmm ok I see, so I'm just using JDO without any frameworks on top of that, but as mentioned in the slim3 tests, JDO does add overhead (sometimes a significant amount). I don't plan to be doing any large queries (the slim3 example fetches 10,000 objects which I don't plan on doing at any point), but what is important to me is that startup time. One to two seconds is fine, four seconds starts pushing it, anything above that is pretty bad. If we start using slim3 though, we lose the flexibility in moving to another host (not that I know of any that can compare to app engine) since it'll be interacting directly with big table instead of using jdo or jpa is an intermediate? Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Records Count
For count, you need to implement sharded counters - http://code.google.com/appengine/articles/sharding_counters.html The code required to do the same is provided. Just understand what sharded counters are and make use of the provided code. -Nischal On Jun 11, 10:37 pm, Nacho Coloma icol...@gmail.com wrote: GAE already includes a left-like operator (property LIKE 'foo%'). I don't know how to use it with JDO though, but it should be in the Reference Guide. For a full LIKE operator (LIKE '%foo%') there is a recent entry in the AppEngine blog:http://googleappengine.blogspot.com/2010/04/making-your-app-searchabl... Hope this helps. On Jun 11, 7:12 pm, RAVINDER MAAN rsmaan...@gmail.com wrote: thanks john any thought for like operator of sql.how can we implement like operator of sql in GAE On Fri, Jun 11, 2010 at 8:49 PM, John Patterson jdpatter...@gmail.comwrote: The best you could probably do is load into memory a structure that keeps the employee type for every employee. When the user types some letters of a name to search for you can perform a keys-only query and lookup the employee type in this structure (rather than load the entity) You would need 2 bits per employee (for 4 types) so even a million employees would only require 250KB of memory (an array of 30K longs) which shouldn't take too long to read from memcache when an instance starts. You would then need to iterate over every result and count which type the employee is. To optimise this you might cache the counts in memcache of 1 and 2 letter searches. So then you would only need to count on the fly three letter searches i.e. all employees with names starting with smi On 11 Jun 2010, at 20:53, RAVINDER MAAN wrote: number of employee entities is going to be high .So i guess that possibility of getting more then 1000 employees for any search is quite high. On Fri, Jun 11, 2010 at 6:56 PM, John Patterson jdpatter...@gmail.comwrote: A lot depends on how many Employee entities you have. You could do a sub search for each of the 4 employee types and use COUNT to get the results. This will only work up to 1000 employees per employee type. Twig can execute queries in parallel so this would take no more time than running your original query. On 11 Jun 2010, at 19:39, RAVINDER MAAN wrote: Thanks for your reply Ravi let me explain the entire scenario.I have an employee entity .User can search employees by giving part of name .There are four type of employees .I have to show number of employees matching user search criteria for every type of employee under different tabs.Actual results are to be displayed only if user open that tab. In SQL term we could write following query Select count(*) from employee where name like '%searchname%' and type ='A' ; My first problem is that i dint find any equivalent of like operator.Is there any way to do it? Secondly we can not keep record count for each search combination.what will you suggest for this case? Thanks once again. On Fri, Jun 11, 2010 at 4:23 PM, Ravi ping2r...@gmail.com wrote: Google app engine designed to work for any numbers of records with in minimum time, so SQL features like counting the records whenever needed is not supported. So you need to take care of such counter by urself at the time of adding or deleting the records in a table/entity. If you just want total number of records and dont care if counting happened in last 24 hours then look into low level api for datastore statistics, GAE refreshes total count every 24 hours and you can read the total count from there. And if you want count of records with some where clause, then you need to maintain it. like if u want like this select count(0) from tb where tb.propA='someValue' then you may need to create a new entity say counterForTb, and have few fields entityName and count and where clause columns and whenevr you add a new record in tb then increase the count of that record in counter table and on delete decrease it. Something for google guys Just realized that all columns are indexed unless we specify to not to index, and there must be some index meta data/statistics saved somewhere in google data stores about index(like total records matching to this index). And if that stats can be accessible through some APIs then we may be able to get total count just by reading data from indexes stats. Is this something feasible? On Jun 11, 11:39 am, RAVINDER MAAN rsmaan...@gmail.com wrote: How can we get records count for a query in JDO. In sql we have select count(*) from table_name i want to get number of records returned by a query. what is the efficient way to do that.One option is to use size() function .I think it is not best way. -- You received this
[appengine-java] Dump Datastore Queries
I want to dump all queries to the datastore for performance analysis reason. Is there any way to do it in local development environment -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Need help on java.security.AccessControlException: access denied
Hello Ikai, While, I moved file to new location WEB-INF/ssma_std.txt, It worked. Thank you so much. Nick On Jun 7, 2:38 pm, Ikai L (Google) ika...@google.com wrote: Can you try placing the file into WEB-INF and setting filename to WEB-INF/ssma_std.txt? On Sat, Jun 5, 2010 at 4:42 PM, Nick lgfr...@gmail.com wrote: Hi, I'm new to GoogleAppEngine, please help me on resolving following erro... 1. I would like to read a ssma_stud.txt file as an input file. This file resides in a path web/ssma_stud.txt 2. As well, updated appengine-web.xml with following... system-properties property name=com.google.gdata.DisableCookieHandler value=true/ /system-properties 3. My objective is to read a text file and generate a HashMap which I can use into my servlet Code: String fileName = /ssma_stud.txt; ArrayList arrayListLineReadData = new ArrayList(); try { BufferedReader inBuffReader = new BufferedReader(new FileReader(fileName)); if (!inBuffReader.ready()) throw new IOException(); 4. on this line BufferedReader inBuffReader = new BufferedReader(new FileReader(fileName)); I get following error..: java.security.AccessControlException: access denied (java.io.FilePermission \ssma_stud.txt read) at java.security.AccessControlContext.checkPermission(AccessControlContext.java: 323) at java.security.AccessController.checkPermission(AccessController.java: 546) at java.lang.SecurityManager.checkPermission(SecurityManager.java: 532) at com.google.appengine.tools.development.DevAppServerFactory $CustomSecurityManager.checkPermission(DevAppServerFactory.java:166) at java.lang.SecurityManager.checkRead(SecurityManager.java:871) at java.io.FileInputStream.init(FileInputStream.java:100) at java.io.FileInputStream.init(FileInputStream.java:66) at java.io.FileReader.init(FileReader.java:41) at com.lgframe.design.C_Stud_Comb_Axial_Bend_Servlet.getArrayListLineReadData(C_Stud_Comb_Axial_Bend_Servlet.java: 411) at com.lgframe.design.C_Stud_Comb_Axial_Bend_Servlet.doPost(C_Stud_Comb_Axial_Bend_Servlet.java: 114) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 511) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1166) at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java: 51) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java: 43) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java: 122) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 418) at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java: 70) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 152) at com.google.appengine.tools.development.JettyContainerService $ApiProxyHandler.handle(JettyContainerService.java:349) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 542) at org.mortbay.jetty.HttpConnection $RequestHandler.content(HttpConnection.java:938) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:755) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java: 409) at org.mortbay.thread.QueuedThreadPool
Re: [appengine-java] Dump Datastore Queries
Have you checked out Appstats? On 12 Jun 2010, at 09:38, Phuong Nguyen wrote: I want to dump all queries to the datastore for performance analysis reason. Is there any way to do it in local development environment -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com . To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en . -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.