[appengine-java] task queues and retries
One of my tasks encountered a failure and wasn't retried. I think it's because the servlet returned a 200 response, but I don't know why it did since the exception bubbled all the way up to the base servlet. Do we need to explicitly return a 5xx error code in exception cases for retries to work? 07-18 02:46PM 06.907 /cleanerq 200 29515ms 9156cpu_ms 5404api_cpu_ms 0kb AppEngine-Google; (+http://code.google.com/appengine) 0.1.0.2 - - [18/Jul/2010:14:46:36 -0700] POST /cleanerq HTTP/1.1 200 0 http://11.latest.wordwisegame.appspot.com/starttasks; AppEngine- Google; (+http://code.google.com/appengine) 11.latest.wordwisegame.appspot.com ms=29516 cpu_ms=9157 api_cpu_ms=5404 cpm_usd=0.254392 queue_name=cleaner-queue task_name=5681724147391472665 exit_code=104 ... com.honkentuber.Servlet doIt: Transaction failed to commit javax.jdo.JDODataStoreException: Transaction failed to commit at org.datanucleus.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java: 304) at org.datanucleus.jdo.JDOTransaction.rollback(JDOTransaction.java: 153) at org.datanucleus.store.appengine.jdo.DatastoreJDOTransaction.rollback(DatastoreJDOTransaction.java: 71) at com.honkentuber.Database.rollbackIfNecessary(Database.java:393) at com.honkentuber.wordwise.db.WordwiseDatabase.fixUserCreateDateAndGameCountsTxn(WordwiseDatabase.java: 808) at com.honkentuber.wordwise.db.WordwiseDatabase.fixNextUsers(WordwiseDatabase.java: 654) at com.honkentuber.wordwise.servlet.CleanerQueueServlet.newHtmlResponse(CleanerQueueServlet.java: 168) at com.honkentuber.wordwise.servlet.WordwiseBaseServlet.newHtmlResponse(WordwiseBaseServlet.java: 63) at com.honkentuber.Servlet.dispatchHtml(Servlet.java:105) at com.honkentuber.Servlet.doIt(Servlet.java:49) at com.honkentuber.Servlet.doPost(Servlet.java:33) at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 511) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1166) at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java: 92) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java: 97) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java: 35) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java: 43) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java: 238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 542) at org.mortbay.jetty.HttpConnection $RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java: 76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java: 135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java: 250) at com.google.apphosting.base.RuntimePb$EvaluationRuntime $6.handleBlockingRequest(RuntimePb.java:7115) at com.google.apphosting.base.RuntimePb$EvaluationRuntime $6.handleBlockingRequest(RuntimePb.java:7113) at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApplicationHandler.java: 24) at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java: 398) at com.google.net.rpc.impl.Server$2.run(Server.java:852) at com.google.tracing.LocalTraceSpanRunnable.run(LocalTraceSpanRunnable.java: 56) at com.google.tracing.LocalTraceSpanBuilder.internalContinueSpan(LocalTraceSpanBuilder.java: 576) at
[appengine-java] __unapplied_write__?
I'm seeing some entities in my data store named __unapplied_write__entity. I couldn't find any documentation on this. What is it? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] indexes and Eclipse plugin
I'm getting some very confusing behavior regarding indexes in my app. Maybe someone can help me figure out what's going on. I have a persistent entity, GameDataV1. I'd pushed several versions of my app to appspot with no issues until last week. Then, after a push, all indexes for GameDataV1 were gone. I could still query it based on the index in a dependent object, but all queries on fields in GameDataV1 failed. I tried pushing a new version to no avail. This evening I tried inspecting the datastore-indexes-auto.xml file running locally out of eclipse. I deleted the DB and indexes to start with a clean slate. In general, the first time I execute a query requiring an index, I see that index show up in the XML file. But periodically I'll run a query and the file will collapse to an empty datastore-indexes element. Queries continue to execute without error, but return 0 entries. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] dashboard history
is it possible to see history beyond 18 hours in the dashboard? or at least to export data periodically so I can analyze longer trends? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: JSP code debug: Compiled JSP location in local environment
FYI, on a Mac I found it in /var/folders/zz/zzzivhrRnAmviuee++-ZXU+ +NMs/-Tmp-/Jetty_127_0_0_1__war.g0qk00/jsp/org/apache/jsp -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] error message
a more descriptive error message would help here. I think the real issue is the class/type of the parameter. org.datanucleus.exceptions.NucleusUserException: Query requires 1 parameters, yet 1 values have been provided. at org.datanucleus.store.query.Query.getParameterMapForValues(Query.java: 1983) at org.datanucleus.store.query.Query.executeWithArray(Query.java: 1371) at org.datanucleus.jdo.JDOQuery.executeWithArray(JDOQuery.java:312) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] send mail failing
I've been seeing a handful of these each day on my app. These are very small messages going only to 1 recipient. I could move mail sending out of the normal request flow (I think cron jobs are the only option right now for background processing, yes?) - is that the recommended remedy? com.honkentuber.wordwise.Emailer sendTurnSummaryMessage: failed to send message for turn VALID: [VANE] com.google.apphosting.api.ApiProxy$ApiDeadlineExceededException: The API call mail.Send() took too long to respond and was cancelled. at com.google.apphosting.runtime.ApiProxyImpl.doSyncCall(ApiProxyImpl.java: 146) at com.google.apphosting.runtime.ApiProxyImpl.access $000(ApiProxyImpl.java:43) at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java: 104) at com.google.apphosting.runtime.ApiProxyImpl$1.run(ApiProxyImpl.java: 102) at java.security.AccessController.doPrivileged(Native Method) at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java: 102) at com.google.apphosting.runtime.ApiProxyImpl.makeSyncCall(ApiProxyImpl.java: 43) at com.google.appengine.tools.appstats.Recorder.makeSyncCall(Recorder.java: 225) at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:98) at com.google.apphosting.api.ApiProxy.makeSyncCall(ApiProxy.java:48) at com.google.appengine.api.mail.MailServiceImpl.doSend(MailServiceImpl.java: 95) at com.google.appengine.api.mail.MailServiceImpl.send(MailServiceImpl.java: 32) at com.google.appengine.api.mail.stdimpl.GMTransport.sendMessage(GMTransport.java: 247) at javax.mail.Transport.send(Transport.java:95) at javax.mail.Transport.send(Transport.java:48) at com.honkentuber.wordwise.Emailer.sendTurnSummaryMessage(Emailer.java: 272) at com.honkentuber.wordwise.servlet.GameServlet.processRequest(GameServlet.java: 84) at com.honkentuber.wordwise.servlet.GameServlet.newJsonResponse(GameServlet.java: 190) at com.honkentuber.wordwise.servlet.WordwiseBaseServlet.newJsonResponse(WordwiseBaseServlet.java: 68) at com.honkentuber.Servlet.dispatchJson(Servlet.java:81) at com.honkentuber.Servlet.doIt(Servlet.java:51) at com.honkentuber.Servlet.doGet(Servlet.java:28) at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 511) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1166) at com.google.appengine.tools.appstats.AppstatsFilter.doFilter(AppstatsFilter.java: 92) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlobUploadFilter.java: 97) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFilter.java: 35) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java: 43) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 418) at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHandlerMap.java: 238) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 542) at org.mortbay.jetty.HttpConnection $RequestHandler.headerComplete(HttpConnection.java:923) at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcRequestParser.java: 76) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest(JettyServletEngineAdapter.java: 135) at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java: 250) at com.google.apphosting.base.RuntimePb$EvaluationRuntime $6.handleBlockingRequest(RuntimePb.java:5838) at com.google.apphosting.base.RuntimePb$EvaluationRuntime $6.handleBlockingRequest(RuntimePb.java:5836) at
[appengine-java] Re: slow responses for query
One more followup on this, when you say you just pay your cost at write time, possibly asynchronously are you referring to cron jobs? As far as I can tell there's no other support for concurrent processing. On May 4, 3:28 am, Ikai L (Google) ika...@google.com wrote: Query performance is generally a function of how many objects have to be returned in the simple case, which seems to be what you are doing here. How many PlayerGameStates exist per User? Queries work like this: - iterate through the index to find all PlayerGameStates with the property given to find the keys of all PlayerGameStates matching this property - run the query for all objects Are PlayerGameStates child entities of a User? I have to double check, but I believe this could reduce the App Engine datastore's ability to parallelize the queries-by-key, though the effect of this should be negligible if you are only fetching 5 objects. The reason I mentioned denormalizing is because the concept of normalization doesn't make as much sense with App Engine's datastore. We don't provide referential integrity, for instance, and transactions exist around entity groups, not globally. You don't pay a penalty for storing data that is sparse - some User entities can be huge, some can be tiny. You should optimize for reads. Doing multiple writes and the risk of data getting out of sync can suck, but as long as it is recoverable you should be okay. Contrast the performance of the query above with storing PlayerGameStates inside User. You'd do this: - Get user by key That's it! This is really fast, so you don't have to go through doing an unnecessary query - you just pay your cost at write time, possibly asynchronously. On Tue, May 4, 2010 at 12:14 PM, Philip Tucker ptuc...@gmail.com wrote: I don't think duplicating the data won't work because then I'll be doing 2 writes for every turn of the game, and I need that to be fast. I don't mind if fetching a user's game list is a little bit slow, but 5-20 seconds is too much. I can denormalize all the data such that Game 2 fields for each entry in PlayerGameState if that helps, but I'd still like to know why it's so slow now. I do have appstat installed. For one particular misbehaving request, the query fetches 5 PlayerGameState records. It iterates through the first 4 taking ~20 ms each, then I see a RunQuery taking 8 seconds. I don't see an easy way to export appstat data, but I can include more details if necessary. Here's the query code in question. SetKey myUserCache = Sets.newHashSet(); Query query = pm.newQuery(PlayerGameState.class); query.declareParameters( com.google.appengine.api.datastore.Key userKeyParam); query.setFilter(userKey == userKeyParam); ListPlayerGameState playerStates = (ListPlayerGameState) query.execute(playerKey); ListGame games = Lists.newArrayList(); for (PlayerGameState playerState : playerStates) { Game game = playerState.getGame(); myUserCache.add(game.getPlayerState().getUserKey()); myUserCache.add(game.getOpponentState().getUserKey()); games.add(game); } On May 3, 1:16 am, Ikai L (Google) ika...@google.com wrote: Indexes don't work the same way they do in relational databases. If you can fetch a value with a query, that likely means an index exists on that property. In a relational database, when an index is not present, the database will do a full table scan. A fetch by Key is always fast. Have you considered storing multiple games inside a single Game instance? Or simply storing the data you need both in the Game instances? If you want to cut down on request latency, do as many by key gets as possible. Don't try to approach the modeling from a normalization perspective. How many entities are you retrieving? And have you tried looking at what's happening via AppStats yet ( http://code.google.com/appengine/docs/java/tools/appstats.html)? On Fri, Apr 30, 2010 at 1:54 AM, Philip Tucker ptuc...@gmail.com wrote: I've got a Game object that includes two PlayerGameState objects. Both are persistence-capable. My query fetches all PlayerGameState objects for a particular user, then I get the associated game and other player. The initial fetch is always fast, and the entire operation is generally lass than a second. But periodically, maybe 5 of the time, it takes much longer, from 5-20 seconds. It's not fetching the PlayerGameState objects that's slow, it's accessing PlagerGameState.game and Game.playerStates. I shouldn't need to add an index on the keys, should I? I'd like to fetch the Game objects directly but as far as I can tell we can't query on a joined table; ie, I can't do something like select from Game, PlayerGameState join on PlayerGameState.game where PlayerGameState.userKey == userKeyParam. @PersistenceCapable(identityType
[appengine-java] Re: indexes
Thanks. By table I mean annotated Java class. One reason I was confused about the indexes is I don't see many of the ones I would expect when I navigate to https://appengine.google.com/datastore/indexes, but I guess they must exist or my queries would be failing. On Apr 30, 2:16 pm, Tristan tristan.slomin...@gmail.com wrote: uhm... read the next section? http://code.google.com/appengine/docs/java/datastore/queriesandindexe... as far as indexes getting applied to existing tables... (not sure by what you mean by table), but yes, that's why when you upload new index it is building, it's being applied to existing stuff. On Apr 30, 12:54 am, Philip Tucker ptuc...@gmail.com wrote: According to this, The App Engine datastore maintains an index for every query an application intends to make. http://code.google.com/appengine/docs/java/datastore/queriesandindexe... Does GAE just scan the app Java code for Query.setFilter, or do we need to do annotate queries for these indexes to be generated. Also, it appears new indexes do not get applied to existing tables when we deploy application code with new queries or @INDEX annotations - is this correct? Is there a way to do that, or are we required to create a new table and copy the data over? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: slow responses for query
On May 4, 3:28 am, Ikai L (Google) ika...@google.com wrote: Query performance is generally a function of how many objects have to be returned in the simple case, which seems to be what you are doing here. How many PlayerGameStates exist per User? No more than 10 so far. Like I said, it was only 5 in this case. Are PlayerGameStates child entities of a User? No. They contain a userKey, which is the field on which I'm querying them. The reason I mentioned denormalizing is because the concept of normalization doesn't make as much sense with App Engine's datastore. We don't provide referential integrity, for instance, and transactions exist around entity groups, not globally. You don't pay a penalty for storing data that is sparse - some User entities can be huge, some can be tiny. You should optimize for reads. Doing multiple writes and the risk of data getting out of sync can suck, but as long as it is recoverable you should be okay. Contrast the performance of the query above with storing PlayerGameStates inside User. You'd do this: But then I'd have to run a separate query to fetch the game. Unless I duplicate the game state and move all those fields into PlayerGameState, which perhaps is what you're suggesting. But then I'd have to fetch both users every time, right (to update the duplicate game state), which means every player needs to contain the userKey of everyone they're playing. Most of my reads are by gameKey. There's only one case where I'm fetching all the games for a user. I don't really need this case to be optimized: 2-3 seconds would be acceptable. 80-90% of the time it's ~1 second. But these occasional 10-20 second fetches are crazy. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] slow responses for query
I've got a Game object that includes two PlayerGameState objects. Both are persistence-capable. My query fetches all PlayerGameState objects for a particular user, then I get the associated game and other player. The initial fetch is always fast, and the entire operation is generally lass than a second. But periodically, maybe 5 of the time, it takes much longer, from 5-20 seconds. It's not fetching the PlayerGameState objects that's slow, it's accessing PlagerGameState.game and Game.playerStates. I shouldn't need to add an index on the keys, should I? I'd like to fetch the Game objects directly but as far as I can tell we can't query on a joined table; ie, I can't do something like select from Game, PlayerGameState join on PlayerGameState.game where PlayerGameState.userKey == userKeyParam. @PersistenceCapable(identityType = IdentityType.APPLICATION) public class Game { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key key; @Persistent(mappedBy = game) @Element(dependent = true) private ListPlayerGameState playerStates; ... } @PersistenceCapable(identityType = IdentityType.APPLICATION) public class PlayerGameState { @SuppressWarnings(unused) @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key key; @Persistent private Key userKey; @Persistent private Game game; ... } -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] indexes
According to this, The App Engine datastore maintains an index for every query an application intends to make. http://code.google.com/appengine/docs/java/datastore/queriesandindexes.html#Introducing_Indexes Does GAE just scan the app Java code for Query.setFilter, or do we need to do annotate queries for these indexes to be generated. Also, it appears new indexes do not get applied to existing tables when we deploy application code with new queries or @INDEX annotations - is this correct? Is there a way to do that, or are we required to create a new table and copy the data over? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] cc sender
Is it possible to send an email via Transport.send without it sending a CC: to the FROM: address? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: including data file in JAR
That worked, thanks. On Apr 18, 11:55 pm, Vaclav Bartacek vaclav.barta...@spolecne.cz wrote: Better way is to use getClass().getResourceAsStream for accessing files packed in jars or for acccessing files located directly in the war you can use the following (in a servlet): ServletContext ctx = getServletContext(); InputStream is = ctx.getResourceAsStream( resourceName ); Do not forget to declare the data file as the resource file in appengine-web.xml (if it is located outside of the WEB-INF directory): resource-files include path=/resources/data-file.csv / /resource-files Vaclav On Apr 18, 4:00 am, Philip Tucker ptuc...@gmail.com wrote: I have a data file I need to access on the server. If I include it in my source path on the client I can load it via ClassLoader.getSystemResourceAsStream. But this breaks on the server (I'm not sure if the eclipse plugin is even deploying it). What's the best way to do this? I don't want to incur a DB hit if I can avoid it, and it's too much data to embed in a class (~4MB). -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] including data file in JAR
I have a data file I need to access on the server. If I include it in my source path on the client I can load it via ClassLoader.getSystemResourceAsStream. But this breaks on the server (I'm not sure if the eclipse plugin is even deploying it). What's the best way to do this? I don't want to incur a DB hit if I can avoid it, and it's too much data to embed in a class (~4MB). -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] User.nickName
When I access the nickName field of the userService.getCurrentUser() object, I don't get the nickName from Google profiles. For example, if I go to http://www.google.com/profiles/me/editprofile for my account (ptuc...@gmail.com) my nickName is Philip, but in AppEngine I get ptucker. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: error removing from a list
That seems to have fixed it, thanks. For the AppEngine developers - surely this is a bug. We shouldn't have to define an index on the primary key, should we? On Mar 15, 4:21 am, thierry LE CONNIAT thlec...@free.fr wrote: HI, I think it's just a problem of index missing on object Presence. TRy to add a index's description. bye On 15 mar, 07:59, Philip Tucker ptuc...@gmail.com wrote: I have a Room class containing a list of Presence classes. Here are the annotations. @PersistenceCapable(identityType = IdentityType.APPLICATION) public class Room { private final static Logger LOG = Logger.getLogger(Room.class.getName()); @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key key; @Persistent private Key creatorKey; @Persistent private String name; @Persistent(mappedBy = room) @Element(dependent = true) private ListPresence presences; ... } @PersistenceCapable(identityType = IdentityType.APPLICATION) public class Presence { @SuppressWarnings(unused) @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key key; @Persistent private Key userKey; @SuppressWarnings(unused) @Persistent private Room room; @Persistent private long lastAccessMs = 0; ... } I'm getting an error when I remove multiple entries from presences via removeAll. Here's the stack trace. I see this error only on the live site, not on a local build. com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found.. datastore-index kind=Presence ancestor=true source=manual property name=__key__ direction=desc/ /datastore-index at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datast oreApiHelper.java: 39) at com.google.appengine.api.datastore.DatastoreApiHelper.makeSyncCall(Datastor eApiHelper.java: 60) at com.google.appengine.api.datastore.PreparedQueryImpl.runQuery(PreparedQuery Impl.java: 115) at com.google.appengine.api.datastore.PreparedQueryImpl.asIterator(PreparedQue ryImpl.java: 81) at com.google.appengine.api.datastore.BasePreparedQuery $1.iterator(BasePreparedQuery.java:25) at org.datanucleus.store.appengine.DatastoreAbstractListStoreSpecialization.ge tIndicesOf(DatastoreAbstractListStoreSpecialization.java: 124) at org.datanucleus.store.mapped.scostore.AbstractListStore.getIndicesOf(Abstra ctListStore.java: 393) at org.datanucleus.store.mapped.scostore.AbstractListStore.removeAll(AbstractL istStore.java: 232) at org.datanucleus.sco.backed.List.removeAll(List.java:957) at com.honkentuber.chat.Room.updatePresence(Room.java:97) at com.honkentuber.chat.ChatDatabase.storeAndFetchMessages(ChatDatabase.java: 212) at com.honkentuber.chat.ChatServlet.newJspResponse(ChatServlet.java: 36) -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] requestDispatcher vs resp.getWriter
I have a servlet that returns a JSON string. For some reason it works fine when I dispatch to a JSP, but sometimes fails when I write directly to the response. this works: req.setAttribute(json, json.toJSONString()); req.getRequestDispatcher(json.jsp).forward(req, resp); this (sometimes) doesn't work: json.writeJSONString(resp.getWriter()); Unfortunately the error is sporadic and I'm not reproducing right now, but it was an Illegal argument exception in NucleusJDOHelper. The request is reading and writing to persistence. I'm calling PersistenceManager.close() immediately after this. Any ideas? I can provide more details if necessary. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: can not remove with @Order annotation
: 36) at com.honkentuber.Servlet.doIt(Servlet.java:38) at com.honkentuber.Servlet.doGet(Servlet.java:24) at javax.servlet.http.HttpServlet.service(HttpServlet.java:693) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 487) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1093) at com.google.appengine.api.blobstore.dev.ServeBlobFilter.doFilter(ServeBlobFilter.java: 51) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1084) at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java: 43) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1084) at com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java: 121) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1084) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 360) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 181) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 712) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 405) at com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java: 70) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 139) at com.google.appengine.tools.development.JettyContainerService $ApiProxyHandler.handle(JettyContainerService.java:352) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java: 139) at org.mortbay.jetty.Server.handle(Server.java:313) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java: 506) at org.mortbay.jetty.HttpConnection $RequestHandler.headerComplete(HttpConnection.java:830) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381) at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java: 396) at org.mortbay.thread.BoundedThreadPool $PoolThread.run(BoundedThreadPool.java:442) On Mar 14, 10:02 pm, Max Ross (Google) maxr+appeng...@google.com wrote: Please post your model object definitions and the stack trace. Thanks! Max On Sun, Mar 14, 2010 at 8:51 PM, Philip Tucker ptuc...@gmail.com wrote: If I annotate a dependent 1:many relationship with @Order(mappedBy = foo), I get a NPE when I try to remove something from the list. When I remove the annotation everything works fine. Is this a known limitation? If not I can post a stack trace and more details. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] can not remove with @Order annotation
If I annotate a dependent 1:many relationship with @Order(mappedBy = foo), I get a NPE when I try to remove something from the list. When I remove the annotation everything works fine. Is this a known limitation? If not I can post a stack trace and more details. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Google Plugin for Eclipse 1.3 plans
This sounds like a good plan and I look forward to testing the new plug-in. - Philip On Feb 4, 2:52 pm, Keith Platfoot kplatf...@google.com wrote: Hey all, Many of you have reported incompatibilities between the Google Plugin for Eclipse and other build systems/project structures, most notably Maven and J2EE/Dynamic Web Projects. For example, issue 1515 (Eclipse plugin requires fixed location for war directory)http://code.google.com/p/googleappengine/issues/detail?id=1515 has collected well over 100 stars, and a similar one in the GWT issue tracker has attracted a lot of attention as well. Based on this feedback, we've decided to try and address as many of the incompatibilities as possible with our upcoming 1.3 release of the plugin. To deliver on that goal, we want to share with your our current plans, and also let you know about a preview build that we'll make available so you can test the changes for yourself and give us feedback before final release. Our plans for the 1.3 release include 4 changes designed to make integration with Maven and J2EE projects easier: 1. The WAR directory can now be configured to be *any* project-relative path (e.g. src/main/webapp if you're using Maven). You'll also be able to specify whether that directory is source-only (typical Maven/J2EE scenario), or whether it should also function as the WAR output directory from which to run/debug or deploy to App Engine. If your WAR directory is input *and* output (which will remain the default for new Web App projects), the plugin will manage synchronizing the contents of WEB-INF/lib WEB-INF/classes with your project's build path and compiled output. Otherwise, we'll leave your WAR source directory alone and you'll need to specify your WAR output location when launching, deploying, etc (the plugin will remember the location once you set it the first time). 2. The Web App launch configuration UI is being redesigned to allow you to see, and if necessary change, *any* of the launch arguments. Previously, we were waiting until launch time to set many of these arguments based on heuristics that were invisible and inaccessible to you. Now you'll be in full control of how your projects get launched. Also, we're adding the capability to automatically migrate your launch configurations when necessary, for example, updating the -javaagent flag when changing App Engine SDKs. 3. GWT/App Engine projects will no longer require our SDK library on the classpath. This means Maven users will be able to pull in JAR files from their M2 repository as they're accustomed to and the plugin won't mind a bit. 4. The severity of any problem marker generated by the plugin will be fully customizable via an Errors/Warnings preference page (similar to the Java Errors/Warnings page), letting you specify either Error, Warning, or Ignore. We'll also be including a few smaller features and bug fixes as well. What does everyone think about the 4 changes outlined above? We've been testing the plugin against various Maven and J2EE configurations to try to ensure that we've eliminated the most critical roadblocks. However, we're very interested in also having you folks take it for a spin before the official release date (slated for next month). We're not quite ready yet, but stay tuned for a 1.3 preview build to be made available hopefully in a few weeks. We'll distribute it as a zip file for dropin installationhttp://code.google.com/eclipse/docs/install-from-zip.html so it will come with the standard warnings and caveats (use with a clean Eclipse install and workspace, use at your risk, etc.). However, it will hopefully give you a chance to give us any last-minute feedback about our changes before the final release. Thanks, Keith -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Any examples for low level datastore? or suggest another way?
Hi All, I want to use the low level datastore, I looked at the documentation and it looks ... complex. http://code.google.com/intl/zh-HK/appengine/docs/java/javadoc/com/google/appengine/api/datastore/package-summary.html Basically, I have a client program which needs to create tables and their entities dynamically, so the columns would change over time. This does not fit well with JDO as it works for fixed column entitys that are annotated. So I thought to use the low level API but it looks difficult. Is there any example code around? Alternatively, can someone suggest another way? Phil -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] IdGeneratorStrategy.SEQUENCE
I'm using IdGeneratorStrategy.SEQUENCE for the primary key of a table, but it appears AppEngine bounces between 2 different sequence generators as I create new entries. IDs are unique, but not monotonically increasing. It's possible I've coded something wrong, but I'm oretty sure this is a bug. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: IdGeneratorStrategy.SEQUENCE
Thanks! Unless I'm misreading the documentation for SEQUENCE, that breaks the contract of the JDP annotation API, doesn't it? Is there a way to annotate a field as a true sequence, or do I need to create my own sequence and assign the value myself? On Dec 21, 9:29 am, Max Ross (Google) maxr+appeng...@google.com wrote: Hi Philip, IdGeneratorStrategy.SEQUENCE support is implemented on top of DatastoreService.allocateIds(), which is itself the mechanism that the datastore uses internally to assign ids. So, all properties of datastore id allocation apply to SEQUENCE. There is a lot of good information about these properties in this thread:http://groups.google.com/group/google-appengine/browse_thread/thread/... But in short, sequences are guaranteed to be unique but not monotically increasing. Hope this helps, Max On Mon, Dec 21, 2009 at 12:04 AM, Philip Tucker ptuc...@gmail.com wrote: I'm using IdGeneratorStrategy.SEQUENCE for the primary key of a table, but it appears AppEngine bounces between 2 different sequence generators as I create new entries. IDs are unique, but not monotonically increasing. It's possible I've coded something wrong, but I'm oretty sure this is a bug. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: IdGeneratorStrategy.SEQUENCE
Hmm, I thought I'd read that in the DataNucleus documentation, but I can't find any clear definition of it. I must have just assumed sequences were sequential. Borrowing from your example here (http://code.google.com/p/datanucleus- appengine/source/browse/trunk/tests/org/datanucleus/test/ SequenceExamplesJDO.java), it seems if I specify a named sequence and set SequenceStrategy.CONTIGUOUS (NONCONTIGUOUS might work too, I'm guessing !TRANSACTIONAL is the key) that I get a monotonically increasing sequence. Does that seem right? Thanks for the input, Philip On Dec 21, 2:46 pm, Max Ross (Google) maxr+appeng...@google.com wrote: Where are you reading the definition of the contract? On Mon, Dec 21, 2009 at 2:41 PM, Philip Tucker ptuc...@gmail.com wrote: Thanks! Unless I'm misreading the documentation for SEQUENCE, that breaks the contract of the JDP annotation API, doesn't it? Is there a way to annotate a field as a true sequence, or do I need to create my own sequence and assign the value myself? On Dec 21, 9:29 am, Max Ross (Google) maxr+appeng...@google.commaxr%2bappeng...@google.com wrote: Hi Philip, IdGeneratorStrategy.SEQUENCE support is implemented on top of DatastoreService.allocateIds(), which is itself the mechanism that the datastore uses internally to assign ids. So, all properties of datastore id allocation apply to SEQUENCE. There is a lot of good information about these properties in this thread: http://groups.google.com/group/google-appengine/browse_thread/thread/... But in short, sequences are guaranteed to be unique but not monotically increasing. Hope this helps, Max On Mon, Dec 21, 2009 at 12:04 AM, Philip Tucker ptuc...@gmail.com wrote: I'm using IdGeneratorStrategy.SEQUENCE for the primary key of a table, but it appears AppEngine bounces between 2 different sequence generators as I create new entries. IDs are unique, but not monotonically increasing. It's possible I've coded something wrong, but I'm oretty sure this is a bug. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: What does precompilation-enabledtrue/precompilation-enabled do?
Whenever I attempt to use pre-compilation I receive a 503 Service Unavailable error and a roll back is made. There is no further explanation in the logs. I made several tries at different times. Unable to update: java.io.IOException: Error posting to URL: http://appengine.google.com/api/appversion/precompile?app_id={removed}version=2; 503 Service Unavailable Try Again (503) An unexpected failure has occurred. Please try again. at com.google.appengine.tools.admin.ServerConnection.send (ServerConnection.java:143) at com.google.appengine.tools.admin.ServerConnection.post (ServerConnection.java:81) at com.google.appengine.tools.admin.AppVersionUpload.send (AppVersionUpload.java:427) at com.google.appengine.tools.admin.AppVersionUpload.precompile (AppVersionUpload.java:135) at com.google.appengine.tools.admin.AppVersionUpload.doUpload (AppVersionUpload.java:112) at com.google.appengine.tools.admin.AppAdminImpl.update (AppAdminImpl.java:56) at com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy (AppEngineBridgeImpl.java:271) at com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace (DeployProjectJob.java:148) at org.eclipse.core.internal.resources.InternalWorkspaceJob.run (InternalWorkspaceJob.java:38) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:55) Am I missing something? - Philip On Dec 7, 6:18 pm, Ikai L (Google) ika...@google.com wrote: Yes, pre-compilation reduces the time to load an application. This will benefit you on your first request after a deploy, after you've been cycled out or if more application instances are created to scale up with your load. You will see up to 30% improved loading time on your first request. Pre-compilation works by doing a bit of class loading work ahead of time in the App Engine environment before the request comes in. It's still opt-in, so you'll need to enable it if you want it. On Thu, Dec 3, 2009 at 4:29 PM, Gary B gary.b...@gmail.com wrote: The 1.2.8 release notes state that pre-compilation reduces the length of loading requests. Does pre-compilation reduce the *time* to load the application? How does pre-compilation work? I noticed that the appcfg update command logs information about precompilation. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- Ikai Lan Developer Programs Engineer, Google App Engine -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.