[google-appengine] Re: About a heavy CPU demanding app
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Rodrigo wrote: [...] I read all documentation I could, including the FAQ, but I still do not know: is it possible to have cron jobs whose execution times exceed 30 minutes? You can't have any request, cron job or otherwise, whose execution time exceeds 30 *seconds*. And that's wall clock time, not CPU time. - -- David Given d...@cowlark.com -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (Cygwin) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iEYEARECAAYFAkqpPC8ACgkQf9E0noFvlzjZhQCgs4PitHLIVeoCzzYHRQxrUE/0 rRUAoNFEerbnObpmW++jFvnHfjnIlkEF =3uD6 -END PGP SIGNATURE- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[appengine-java] Startup takes forever
If I make a request to my app, and (I assume) AppEngine decides it needs to start up a new servlet instance for me, then warming up the server can take about 5000ms. Is this normal? For example, the first two items in my log for a request will be: 09-08 12:57AM 30.917 [standard notification of incoming request] 09-08 12:57AM 35.344 com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue$SystemLoader loadFinalizer: Not allowed to access system class loader. Note the five second difference between the two timestamps. This all happens before my app has even started running any code. As this only happens the first time, it's not really a problem, but does cause nasty red warning banners in my logs. Possibly related, the first time I do a database request in one of these new servers, I'm getting the following scary backtrace. Everything *works* --- it's just a warning. Can I safely ignore it or is this a symptom of something wrong with my code? 09-08 12:57AM 35.361 com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue init: Failed to start reference finalizer thread. Reference cleanup will only occur when new references are created. java.lang.reflect.InvocationTargetException at com.google.appengine.runtime.Request.process-5d48060cebaffe54(Request.java) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Method.java:40) at com.google.appengine.repackaged.com.google.common.base.FinalizableReferenceQueue.init(FinalizableReferenceQueue.java:124) at com.google.appengine.repackaged.com.google.common.labs.misc.InterningPools$WeakInterningPool.clinit(InterningPools.java:104) at com.google.appengine.repackaged.com.google.common.labs.misc.InterningPools.newWeakInterningPool(InterningPools.java:48) at com.google.appengine.repackaged.com.google.io.protocol.ProtocolSupport.clinit(ProtocolSupport.java:55) at com.google.apphosting.api.DatastorePb$GetRequest.freeze(DatastorePb.java:7233) at com.google.apphosting.api.DatastorePb$GetRequest$1.init(DatastorePb.java:7177) at com.google.apphosting.api.DatastorePb$GetRequest.clinit(DatastorePb.java:7174) at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java:90) at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java:69) at com.google.appengine.api.datastore.DatastoreServiceImpl.get(DatastoreServiceImpl.java:57) at com.cowlark.stellation2.server.db.GAEPersistenceInterface.atomicallyModify(GAEPersistenceInterface.java:130) at com.cowlark.stellation2.server.db.GAEPersistenceInterface.lock(GAEPersistenceInterface.java:160) at com.cowlark.stellation2.server.db.ServerDB.lock(ServerDB.java:110) at com.cowlark.stellation2.server.RPCServiceImpl.ping(RPCServiceImpl.java:81) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source) at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source) at java.lang.reflect.Method.invoke(Method.java:40) at com.google.gwt.user.server.rpc.RPC.invokeAndEncodeResponse(RPC.java:527) at com.google.gwt.user.server.rpc.RemoteServiceServlet.processCall(RemoteServiceServlet.java:166) at com.google.gwt.user.server.rpc.RemoteServiceServlet.doPost(RemoteServiceServlet.java:86) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java:35) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java:237) at
[google-appengine] Re: Text based RPG
Brandon N. Wirtz wrote: You could build one of these... You'd go broke running it. A MUD is always running.. It handles multiple threads and not only accepts requests but sends them to the user.. I actually *am* building something like one of these (using Java). The biggest problem I've got is that GAE has no synchronisation mechanisms. None. There are transactions, but as they only work on single entities they're useless. This means that it's very hard to run game logic that needs to synchronously touch the global database. To work round this I've implemented my own semaphore backed by the datastore; it's extremely nasty but does actually work. I then read the entire game database out of a blob in a single entity into RAM, run the game logic, write the database back again, and release the semaphore. That is also extremely nasty but it does all actually work. (My game's actually intended to be run on a standalone server, not GAE, but the persistence layers are all abstracted out to let me host on GAE for development purposes.) It'll congest rather badly but by the time congestion becomes an issue I should have a more suitable server. Performance isn't bad; I'm usually getting request times of about 500ms when the server's hot. That breaks down to: startup: anything from 10 to 5000ms db load: 50ms game logic: 50ms build client update: 50ms (varies according to the size of the update) db save: 100ms Startup time is very strange. Sometimes it's practically nil, usually it's 100ms or so, and if the server's not hot it something takes many seconds (together with weird warnings about failing to start the finalisation thread). It's all out of my control --- it happens before the app starts. Note that the cheapest part of the whole process is actually running the game logic! Most of the time most requests take 0ms, as nothing will have actually happened since the last request... GAE no process can run for more than a few seconds, so you'd have to use some polling tricks on the client to ask what happened and have a chron job run the main thread every few seconds... I'm running the game logic asynchronously on-demand. After all, you only need to change things when someone's looking! This works beautifully, but if it's been a couple of days since the last request it can take a while; I'm planning to use a cron job to update the server every half hour or so to avoid this. If you're writing a text-based game things will be both easier and harder. Easier, in that you don't have to do all the complex database synchronisation between the client and the server that I'm doing, and harder, because the game logic itself will be more complex and the client will need to poll much more frequently. (I can get away with a poll every ten minutes or so. A MUD can't.) -- ┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─ http://www.cowlark.com ─ │ │ They laughed at Newton. They laughed at Einstein. Of course, they │ also laughed at Bozo the Clown. --- Carl Sagan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: Text based RPG
Nick Johnson (Google) wrote: [...] Not true - Transactions can encompass entire entity groups. With the correct entity group configuration, you could perform updates on local areas of the map transactionally. Is it possible to directly access entities in a group other than the root one? I was under the impression not (but I could well be wrong). And I'm afraid it still wouldn't help --- I'd still need synchronisation between different groups when dealing with multiple areas on the map; and AppEngine's transactions are optimistic rather than blocking, which are no use to me. In addition the overhead to performing a single database access appears to be so high that it's *much* faster to read and write the entire database in two accesses than it is to read parts of it using multiple queries! Although that may change as the database size grows. -- ┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─ http://www.cowlark.com ─ │ │ They laughed at Newton. They laughed at Einstein. Of course, they │ also laughed at Bozo the Clown. --- Carl Sagan --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] Re: 30 Max simultaneous requests (maxThreads)?
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Adligo wrote: [...] I mentioned is stored as a Session Attribute. I assume that the app engine must be keeping the Session in a location that can be accessed by all threads. I'm afraid not! The *only* mechanism for communicating between requests is, AFAIK (and if I'm wrong someone please correct me), memcache or the datastore. Any request may be hosted in any servlet on any JVM on any machine in any datacentre in any continent, and you can't rely on any two requests running in the same place. (Or on the moon in the Google moonbase, but given the latency that would be unlikely.) AppEngine is perfectly entitled to run your code anywhere it feels like, although it's probably going to run it close to the data. Otherwise how could anyone maintain any sort or security on the server side (most Jaas HttpFilters I have seen store the Subject[User] as a Session Attribute, including 2 that I wrote by hand for custom requirements and Spring security) ? AppEngine does implement servlet sessions, but it's just a wrapper around the datastore --- it will automatically save your session objects and reload them for you. I don't believe it's defined when this happens, though, so you probably can't change properties in one session and rely on being able to see them change in another in real time. Personally, I find it easiest if I just forget everything I ever knew about servlets and start again from scratch when dealing with AppEngine. It's sufficiently strange not to be able to take anything for granted. (People here are most likely already tired of my rants about not being able to synchronise between threads...) - -- ┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─ http://www.cowlark.com ─ │ │ People who think they know everything really annoy those of us who │ know we don't. --- Bjarne Stroustrup -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKprdKf9E0noFvlzgRAu3bAJ9/hEJSHlsaluusBljVRZyz/QF4HQCguPKw 2kt1/NVlqrU/T84idi6h11I= =3mR2 -END PGP SIGNATURE- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[appengine-java] Re: Can't make logging work
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 rick wrote: Did you try changing level to FINEST in your logging.properties file ? Yes, that did it. Thanks very much! (And it would be dead handy if the App Engine documentation could mention this --- it is the only form of debugging available...) - -- ┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─ http://www.cowlark.com ─ │ │ People who think they know everything really annoy those of us who │ know we don't. --- Bjarne Stroustrup -BEGIN PGP SIGNATURE- Version: GnuPG v1.4.9 (GNU/Linux) Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ iD8DBQFKkYP/f9E0noFvlzgRArgKAJ94eRBZ9BhaovPJjK46aC96GulDQQCgss+2 64Pi73XX/yK4z/vdstoBazo= =coWh -END PGP SIGNATURE- --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en -~--~~~~--~~--~--~---
[appengine-java] Can't make logging work
-BEGIN PGP SIGNED MESSAGE-
Hash: SHA1
Yes, another dumb question.
I am totally unable to make logging work on deployed apps. I've followed
the instructions rigorously, but nothing ever shows up in the 'Logging'
section of the Administration console.
I'm using the following helper class:
public class Log
{
private static final Logger _log =
Logger.getLogger(RPCServiceImpl.class.getName());
public static void log(String s, Object... o)
{
long now = new Date().getTime() % 100L;
s = now + : + String.format(s, o);
_log.info(s);
System.out.println(s);
}
}
...and then calling it with 'Log.log(Hello, world!)' or equivalent.
Note that I'm both calling Logger.info() *and* writing to stdout ---
nothing ever shows up.
I have the standard logging.properties, my appengine-web.xml file is
pointing at it, and as far as I know everything's set up correctly. I
just never see anything. What am I missing here?
- --
┌─── ｄｇ＠ｃｏｗｌａｒｋ．ｃｏｍ ─ http://www.cowlark.com ─
│
│ People who think they know everything really annoy those of us who
│ know we don't. --- Bjarne Stroustrup
-BEGIN PGP SIGNATURE-
Version: GnuPG v1.4.9 (GNU/Linux)
Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/
iD8DBQFKkIYpf9E0noFvlzgRAnnyAJ9u2piGART2oHxBoQGHGTUCqhPhkACgudQW
UbJAib2z2GU5/igd4cnuY3A=
=xksW
-END PGP SIGNATURE-
--~--~-~--~~~---~--~~
You received this message because you are subscribed to the Google Groups
Google App Engine for Java group.
To post to this group, send email to google-appengine-java@googlegroups.com
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en
-~--~~~~--~~--~--~---
[google-appengine] Re: Serialised events
Adam wrote: David, I would suggest using a Request Repeater pattern rather than Task Queues. What you want to achieve might be difficult or impossible with Task Queues. That's very cunning --- ta. I can use that. However, I'm not sure it quite meets all my requirements. This is good for ensuring that the event queue gets flushed periodically, which I'm going to want to do, but I also want to flush the queue on-demand (so that I can quickly process any pending events while the user's actually logged in). Traditionally I'd do this with a big lock around the event processing routine, but appengine can't do this. I could use transactions, but I'd have to process each event --- which will involve touching and updating multiple entities all over my database --- inside a transaction, and that's going to be hideously expensive should contention happen (I'll end up processing the same event multiple times!) due to the optimistic transactions. There has to be a better way to do this! -- David Given d...@cowlark.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
[google-appengine] [appengine-java] Serialised events
Hi, I'm trying to write a game using AppEngine. As a result, I need some mechanism for running the game AI. I've considered various architectures and the approach that seems the most workable is to have, in the datastore, a priority queue of events based on timestamp. I post events to the queue, and then process them in order whenever it's necessary to update the state of the game world. Most of this isn't hard. The problem is that I can't find any way of achieving the 'in order' part of the problem. It's vitally important that the event processing happens serially, or else I'll end up processing events out of order, with hilarious consequences --- I would like my unit to pick up cargo *before* it departs the current location, thanks very much! Normally I'd achieve this by having a single thread do all the AI processing, so the AI becomes intrinsically serialised. Naturally, this isn't possible in AppEngine. The only other approach I can think of is to put some kind of big lock around the AI processing routine, to ensure that only one thread at a time handles events; but AppEngine doesn't have that kind of lock. Transactions aren't suitable because they're not blocking. This must be a situation people have encountered before --- any suggestions as to the best way to achieve this sort of thing? -- David Given d...@cowlark.com --~--~-~--~~~---~--~~ You received this message because you are subscribed to the Google Groups Google App Engine group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to google-appengine+unsubscr...@googlegroups.com For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~--~~~~--~~--~--~---
