thanks for the info. Getting feedback like this from someone as close to the low level implementation as yourself is priceless!
I think I'll be fine using either a SortedSet or a List with the @Order and the datanucleus extension on it for the Activity.ratePlans collection. This set could just be fine to order by a name, or even the Id. I know there wont be too many of these children in our clients use cases. Even in the cases where we want to allow user-specified ordering, the list shouldn't be that long. We're modeling reservations for adventure outfitters, there are only so many people you can take down a river :) I'm glad to hear you say not to optimize too eagerly, I've kinda taken the approach that I'd just make things work, and then see how I can tweak performance as needed. anyway, thanks again for all the great advice and info! -bryce On Mon, Dec 14, 2009 at 11:21 PM, Max Ross (Google) <maxr+appeng...@google.com> wrote: > A Set will not suffer from this same problem. The datastore has no way of > knowing what the sort function of your SortedSet might be so this will be > treated as an unordered Collection as well, but the sort will happen > in-memory as the collection is populated. If the collection has an explicit > ordering then the JDO default is probably what you want since it does all > the shifting that you'd need to implement yourself, but there is this bug > you've run into, and I suspect there are others. If you don't want the > overhead of a datastore index on displayOrder and you know you're not going > to have too many child objects, doing the sort in-memory may be fine. Don't > optimize too early though. I'd see what the performance of writes look like > with the datastore index and then go from there. > > Max > > On Mon, Dec 14, 2009 at 10:10 PM, bryce cottam <bcot...@gmail.com> wrote: >> >> do SortedSets suffer from the same problem? In particular, I could >> define a displayOrder property that was rather sparse (every 10th int >> or something: 10, 20, 30.... etc.) and if I needed to insert something >> between the first and second (which shouldn't happen very often) I >> could just set it's displayOrder to 15 and throw it in the set. >> >> The only reason I bring it up, is because our app allows users to >> order certain types of data (really just one type), but frankly, they >> don't move them around much at all, it's pretty much defined at >> creation time, but *sometimes* they alter that ordering. I don't >> anticipate this being a huge problem, heck I could sort the set after >> it gets out of the datastore if I had to using a displayOrder field or >> something. I don't want to bog down my performance doing this through >> some implicit index. >> >> thanks, >> -bryce >> >> >> On Mon, Dec 14, 2009 at 11:04 PM, Max Ross (Google) >> <maxr+appeng...@google.com> wrote: >> > If you order by anything other than your primary key you'll take a >> > performance and cpu hit on writes but there won't be a penalty on your >> > reads. The reason the JDO default is such a problem is that it adds an >> > implicit property to each child entity containing the position of that >> > entity in its parent's list. If you ever remove or add an element in >> > this >> > list at a location that isn't the end, everything between that element >> > and >> > the end of the list needs to get shifted down. In an RDBMS you'd just >> > issue >> > an update statement to increment everything by 1 relative to its current >> > value, but the datastore doesn't support this type of relative update, >> > so >> > ends up doing one write per entity whose index needs shifting. So in >> > short, >> > don't do it this way. :-) >> > >> > Max >> > >> > On Mon, Dec 14, 2009 at 9:42 PM, bryce cottam <bcot...@gmail.com> wrote: >> >> >> >> hahaha, no that doesn't sound draconian. I remember reading that the >> >> ordering of lists can be a performance hit, I haven't bothered to look >> >> too deep into it yet (since I'm only doing a proof of concept right >> >> now). I have some questions about it though. For instance, what if I >> >> were to order by some kind of displayOrder field that I maintain? >> >> Would that also suffer a performance lag? I mainly want to move to >> >> the GAE for performance and scalability, so I think I'd appreciate a >> >> runtime/compile time warning that told me about things I was doing >> >> that would hurt my performance. >> >> >> >> Thanks again for helping me out so much. >> >> -bryce >> >> >> >> >> >> >> >> On Mon, Dec 14, 2009 at 9:06 PM, Max Ross (Google) >> >> <maxr+appeng...@google.com> wrote: >> >> > Ok I've got good news for you. I need to do more testing but I think >> >> > different parent types for child objects should work fine as long as >> >> > you >> >> > use >> >> > list-ordering for your one-to-many relationships, which you really >> >> > ought >> >> > to >> >> > be doing anyway. Position properties are already incredibly >> >> > inefficient, >> >> > and it turns out the bug you're running into only applies when there >> >> > are >> >> > position properties in play. You can read more about it in this >> >> > section: >> >> > >> >> > >> >> > http://code.google.com/appengine/docs/java/datastore/relationships.html#Owned_One_to_Many_Relationships >> >> > (How Ordered Collections Maintain Their Order). >> >> > >> >> > So, if you change your one-to-many relationship on RatePlan to: >> >> > @Persistent >> >> > @Order(extensions = @Extension(vendorName="datanucleus", >> >> > key="list-ordering", value="id")) >> >> > private List<RatePlan> ratePlans = new ArrayList<RatePlan>(); >> >> > >> >> > you should be in business. >> >> > >> >> > Position properties are so bad I'm considering turning them into a >> >> > meta-data >> >> > error that you have to explicitly silence with a jvm property. What >> >> > do >> >> > you >> >> > think, too draconian? >> >> > >> >> > Max >> >> > >> >> > On Mon, Dec 14, 2009 at 11:09 AM, Max Ross (Google) >> >> > <maxr+appeng...@google.com> wrote: >> >> >> >> >> >> Sure, sorry it was such a headache for you. When I've got a fix >> >> >> ready >> >> >> I'll let you know. >> >> >> >> >> >> On Mon, Dec 14, 2009 at 10:57 AM, bryce cottam <bcot...@gmail.com> >> >> >> wrote: >> >> >>> >> >> >>> good point, I kinda got lost in the exceptions I was getting :) >> >> >>> I wound up sorting it out by making an embedded class instead of >> >> >>> having multiple parent classes for a given child class. Although, >> >> >>> it >> >> >>> would be nice if a child class could have different parent types. >> >> >>> >> >> >>> Thanks for all your help Max. >> >> >>> -bryce >> >> >>> >> >> >>> >> >> >>> On Mon, Dec 14, 2009 at 11:30 AM, Max Ross (Google) >> >> >>> <maxr+appeng...@google.com> wrote: >> >> >>> > I believe the exception you're getting is the result of having >> >> >>> > multiple >> >> >>> > parents for the same type, and unfortunately that bug isn't fixed >> >> >>> > in >> >> >>> > this >> >> >>> > release. The bugs fixed in this release are: >> >> >>> > >> >> >>> > - Incorrect exception for multiple fields of same type >> >> >>> > >> >> >>> > http://code.google.com/p/datanucleus-appengine/issues/detail?id=172 >> >> >>> > >> >> >>> > - Non-persistent base classes do not work >> >> >>> > >> >> >>> > http://code.google.com/p/datanucleus-appengine/issues/detail?id=169 >> >> >>> > >> >> >>> > - Relationships in abstract base classes don't work >> >> >>> > >> >> >>> > http://code.google.com/p/datanucleus-appengine/issues/detail?id=171 >> >> >>> > >> >> >>> > >> >> >>> > On Sat, Dec 12, 2009 at 1:09 AM, bryce cottam <bcot...@gmail.com> >> >> >>> > wrote: >> >> >>> >> >> >> >>> >> hmmm.... I think I may have installed the patch wrong perhaps. >> >> >>> >> I >> >> >>> >> still get the same error. >> >> >>> >> I'm using the same code I sent you for the test case Max. Do >> >> >>> >> you >> >> >>> >> see >> >> >>> >> any tell-tale signs in here that indicate I've mis-configured >> >> >>> >> something? I have datanucleus-appengine-1.0.4.1-RC2.jar on my >> >> >>> >> classpath and I removed the previous one (pretty much the same >> >> >>> >> jar >> >> >>> >> but >> >> >>> >> with .final.jar as it's name. I followed the steps outlined by >> >> >>> >> Jonathan. >> >> >>> >> >> >> >>> >> here is the stack trace: >> >> >>> >> java.lang.ClassCastException: oid is not instanceof >> >> >>> >> javax.jdo.identity.ObjectIdentity >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> com.resmark.model.RatePlan.jdoCopyKeyFieldsFromObjectId(RatePlan.java) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.setObjectAsValue(PersistenceCapableMapping.java:657) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.setObject(PersistenceCapableMapping.java:364) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastoreRelationFieldManager$1.setObjectViaMapping(DatastoreRelationFieldManager.java:128) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastoreRelationFieldManager$1.apply(DatastoreRelationFieldManager.java:104) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastoreRelationFieldManager.storeRelations(DatastoreRelationFieldManager.java:78) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastoreFieldManager.storeRelations(DatastoreFieldManager.java:812) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPostProcess(DatastorePersistenceHandler.java:288) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects(DatastorePersistenceHandler.java:241) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObject(DatastorePersistenceHandler.java:225) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3185) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3161) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1298) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1175) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:669) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.datanucleus.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:694) >> >> >>> >> at >> >> >>> >> com.resmark.JdoTestServlet.create(JdoTestServlet.java:183) >> >> >>> >> at >> >> >>> >> com.resmark.JdoTestServlet.doTest(JdoTestServlet.java:90) >> >> >>> >> at >> >> >>> >> com.resmark.JdoTestServlet.doGet(JdoTestServlet.java:33) >> >> >>> >> at >> >> >>> >> javax.servlet.http.HttpServlet.service(HttpServlet.java:693) >> >> >>> >> at >> >> >>> >> javax.servlet.http.HttpServlet.service(HttpServlet.java:806) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:121) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:352) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) >> >> >>> >> at org.mortbay.jetty.Server.handle(Server.java:313) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830) >> >> >>> >> at >> >> >>> >> org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514) >> >> >>> >> at >> >> >>> >> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) >> >> >>> >> at >> >> >>> >> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396) >> >> >>> >> at >> >> >>> >> >> >> >>> >> >> >> >>> >> >> >> >>> >> org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) >> >> >>> >> >> >> >>> >> I've got to be missing something. :( >> >> >>> >> >> >> >>> >> thanks, >> >> >>> >> -bryce >> >> >>> >> >> >> >>> >> >> >> >>> >> On Fri, Dec 11, 2009 at 8:54 AM, bryce cottam >> >> >>> >> <bcot...@gmail.com> >> >> >>> >> wrote: >> >> >>> >> > Thanks a million Max, I've been on the road for a few days but >> >> >>> >> > I'm >> >> >>> >> > looking >> >> >>> >> > forward to trying this out when I get back. Thanks for the >> >> >>> >> > link >> >> >>> >> > Jonathan. >> >> >>> >> > >> >> >>> >> > -bryce >> >> >>> >> > >> >> >>> >> > On Dec 11, 2009 6:50 AM, "Jonathan 'J5' Cook" >> >> >>> >> > <jonathan.j5.c...@gmail.com> wrote: >> >> >>> >> > >> >> >>> >> > Thanks for the fix, Max and perfect timing for me :) I just >> >> >>> >> > upgraded >> >> >>> >> > to 1.2.8 and started having this issue last night. >> >> >>> >> > >> >> >>> >> > For those who don't/didn't already know (like myself), here is >> >> >>> >> > a >> >> >>> >> > link >> >> >>> >> > to instructions Max wrote on how to install a new plugin >> >> >>> >> > version >> >> >>> >> > into >> >> >>> >> > the SDK: >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > >> >> >>> >> > http://code.google.com/p/datanucleus-appengine/wiki/HowToUpdateTheSDKWithANewPluginVersion >> >> >>> >> > >> >> >>> >> > Regards, >> >> >>> >> > J5 >> >> >>> >> > >> >> >>> >> > On Dec 10, 11:59 am, "Max Ross (Google)" >> >> >>> >> > <maxr+appeng...@google.com> >> >> >>> >> > wrote: >> >> >>> >> >> I've posted a bug fix release candidate containing this fix >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> >> >> >>> >> >> here:http://datanucleus-appengine.googlecode.com/files/appengine-orm-1.0.4... >> >> >>> >> > >> >> >>> >> >> > On Tue, Dec 8, 2009 at 5:26 PM, bryce cottam >> >> >>> >> >> > <bcot...@gmail.com> >> >> >>> >> >> > wrote: >> >> >>> >> >> > > > That's great news Ma... >> >> >>> >> > >> >> >>> >> >> > On Dec 8, 2009 5:20 PM, "Max Ross (Google)" >> >> >>> >> >> > <maxr+appeng...@google.com<maxr%2bappeng...@google.com>> >> >> >>> >> > >> >> >>> >> >> > wrote: > > > I've filed bug > >> >> >>> >> >> > >> >> >>> >> >> > > >> >> >>> >> >> > > > >> >> >>> >> >> > > > > >http://code.google.com/p/datanucleus-appengine/issues/detail?id=1... >> >> >>> >> > >> >> >>> >> >> > On Tue, Dec 8, 2009 at 11:29 AM, bryce cottam >> >> >>> >> >> > <bcot...@gmail.com> >> >> >>> >> >> > wrote: >> >> >>> >> >> > > > >> > > It's all goo... >> >> >>> >> > >> >> >>> >> >> > >> >> >>> >> >> > >> >> >>> >> >> > >> >> >>> >> >> > >> >> >>> >> >> > >> >> >>> >> >> > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> >> >> >>> >> > >> >> >>> >> >> > . > > For more options, visit this group at > >> >> >>> >> >> > >http://groups.google.com/group/google-appengine-j... >> >> >>> >> >> >> >>> >> -- >> >> >>> >> >> >> >>> >> You received this message because you are subscribed to the >> >> >>> >> Google >> >> >>> >> Groups >> >> >>> >> "Google App Engine for Java" group. >> >> >>> >> To post to this group, send email to >> >> >>> >> google-appengine-j...@googlegroups.com. >> >> >>> >> To unsubscribe from this group, send email to >> >> >>> >> google-appengine-java+unsubscr...@googlegroups.com. >> >> >>> >> For more options, visit this group at >> >> >>> >> http://groups.google.com/group/google-appengine-java?hl=en. >> >> >>> >> >> >> >>> >> >> >> >>> > >> >> >>> > -- >> >> >>> > >> >> >>> > You received this message because you are subscribed to the >> >> >>> > Google >> >> >>> > Groups >> >> >>> > "Google App Engine for Java" group. >> >> >>> > To post to this group, send email to >> >> >>> > google-appengine-j...@googlegroups.com. >> >> >>> > To unsubscribe from this group, send email to >> >> >>> > google-appengine-java+unsubscr...@googlegroups.com. >> >> >>> > For more options, visit this group at >> >> >>> > http://groups.google.com/group/google-appengine-java?hl=en. >> >> >>> > >> >> >>> >> >> >>> -- >> >> >>> >> >> >>> You received this message because you are subscribed to the Google >> >> >>> Groups >> >> >>> "Google App Engine for Java" group. >> >> >>> To post to this group, send email to >> >> >>> google-appengine-j...@googlegroups.com. >> >> >>> To unsubscribe from this group, send email to >> >> >>> google-appengine-java+unsubscr...@googlegroups.com. >> >> >>> For more options, visit this group at >> >> >>> http://groups.google.com/group/google-appengine-java?hl=en. >> >> >>> >> >> >>> >> >> >> >> >> > >> >> > -- >> >> > >> >> > You received this message because you are subscribed to the Google >> >> > Groups >> >> > "Google App Engine for Java" group. >> >> > To post to this group, send email to >> >> > google-appengine-j...@googlegroups.com. >> >> > To unsubscribe from this group, send email to >> >> > google-appengine-java+unsubscr...@googlegroups.com. >> >> > For more options, visit this group at >> >> > http://groups.google.com/group/google-appengine-java?hl=en. >> >> > >> >> >> >> -- >> >> >> >> You received this message because you are subscribed to the Google >> >> Groups >> >> "Google App Engine for Java" group. >> >> To post to this group, send email to >> >> google-appengine-j...@googlegroups.com. >> >> To unsubscribe from this group, send email to >> >> google-appengine-java+unsubscr...@googlegroups.com. >> >> For more options, visit this group at >> >> http://groups.google.com/group/google-appengine-java?hl=en. >> >> >> >> >> > >> > -- >> > >> > You received this message because you are subscribed to the Google >> > Groups >> > "Google App Engine for Java" group. >> > To post to this group, send email to >> > google-appengine-j...@googlegroups.com. >> > To unsubscribe from this group, send email to >> > google-appengine-java+unsubscr...@googlegroups.com. >> > For more options, visit this group at >> > http://groups.google.com/group/google-appengine-java?hl=en. >> > >> >> -- >> >> You received this message because you are subscribed to the Google Groups >> "Google App Engine for Java" group. >> To post to this group, send email to >> google-appengine-j...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine-java+unsubscr...@googlegroups.com. >> For more options, visit this group at >> http://groups.google.com/group/google-appengine-java?hl=en. >> >> > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
