A Set will not suffer from this same problem. The datastore has no way of knowing what the sort function of your SortedSet might be so this will be treated as an unordered Collection as well, but the sort will happen in-memory as the collection is populated. If the collection has an explicit ordering then the JDO default is probably what you want since it does all the shifting that you'd need to implement yourself, but there is this bug you've run into, and I suspect there are others. If you don't want the overhead of a datastore index on displayOrder and you know you're not going to have too many child objects, doing the sort in-memory may be fine. Don't optimize too early though. I'd see what the performance of writes look like with the datastore index and then go from there.
Max On Mon, Dec 14, 2009 at 10:10 PM, bryce cottam <bcot...@gmail.com> wrote: > do SortedSets suffer from the same problem? In particular, I could > define a displayOrder property that was rather sparse (every 10th int > or something: 10, 20, 30.... etc.) and if I needed to insert something > between the first and second (which shouldn't happen very often) I > could just set it's displayOrder to 15 and throw it in the set. > > The only reason I bring it up, is because our app allows users to > order certain types of data (really just one type), but frankly, they > don't move them around much at all, it's pretty much defined at > creation time, but *sometimes* they alter that ordering. I don't > anticipate this being a huge problem, heck I could sort the set after > it gets out of the datastore if I had to using a displayOrder field or > something. I don't want to bog down my performance doing this through > some implicit index. > > thanks, > -bryce > > > On Mon, Dec 14, 2009 at 11:04 PM, Max Ross (Google) > <maxr+appeng...@google.com <maxr%2bappeng...@google.com>> wrote: > > If you order by anything other than your primary key you'll take a > > performance and cpu hit on writes but there won't be a penalty on your > > reads. The reason the JDO default is such a problem is that it adds an > > implicit property to each child entity containing the position of that > > entity in its parent's list. If you ever remove or add an element in > this > > list at a location that isn't the end, everything between that element > and > > the end of the list needs to get shifted down. In an RDBMS you'd just > issue > > an update statement to increment everything by 1 relative to its current > > value, but the datastore doesn't support this type of relative update, so > > ends up doing one write per entity whose index needs shifting. So in > short, > > don't do it this way. :-) > > > > Max > > > > On Mon, Dec 14, 2009 at 9:42 PM, bryce cottam <bcot...@gmail.com> wrote: > >> > >> hahaha, no that doesn't sound draconian. I remember reading that the > >> ordering of lists can be a performance hit, I haven't bothered to look > >> too deep into it yet (since I'm only doing a proof of concept right > >> now). I have some questions about it though. For instance, what if I > >> were to order by some kind of displayOrder field that I maintain? > >> Would that also suffer a performance lag? I mainly want to move to > >> the GAE for performance and scalability, so I think I'd appreciate a > >> runtime/compile time warning that told me about things I was doing > >> that would hurt my performance. > >> > >> Thanks again for helping me out so much. > >> -bryce > >> > >> > >> > >> On Mon, Dec 14, 2009 at 9:06 PM, Max Ross (Google) > >> <maxr+appeng...@google.com <maxr%2bappeng...@google.com>> wrote: > >> > Ok I've got good news for you. I need to do more testing but I think > >> > different parent types for child objects should work fine as long as > you > >> > use > >> > list-ordering for your one-to-many relationships, which you really > ought > >> > to > >> > be doing anyway. Position properties are already incredibly > >> > inefficient, > >> > and it turns out the bug you're running into only applies when there > are > >> > position properties in play. You can read more about it in this > >> > section: > >> > > >> > > http://code.google.com/appengine/docs/java/datastore/relationships.html#Owned_One_to_Many_Relationships > >> > (How Ordered Collections Maintain Their Order). > >> > > >> > So, if you change your one-to-many relationship on RatePlan to: > >> > @Persistent > >> > @Order(extensions = @Extension(vendorName="datanucleus", > >> > key="list-ordering", value="id")) > >> > private List<RatePlan> ratePlans = new ArrayList<RatePlan>(); > >> > > >> > you should be in business. > >> > > >> > Position properties are so bad I'm considering turning them into a > >> > meta-data > >> > error that you have to explicitly silence with a jvm property. What > do > >> > you > >> > think, too draconian? > >> > > >> > Max > >> > > >> > On Mon, Dec 14, 2009 at 11:09 AM, Max Ross (Google) > >> > <maxr+appeng...@google.com <maxr%2bappeng...@google.com>> wrote: > >> >> > >> >> Sure, sorry it was such a headache for you. When I've got a fix > ready > >> >> I'll let you know. > >> >> > >> >> On Mon, Dec 14, 2009 at 10:57 AM, bryce cottam <bcot...@gmail.com> > >> >> wrote: > >> >>> > >> >>> good point, I kinda got lost in the exceptions I was getting :) > >> >>> I wound up sorting it out by making an embedded class instead of > >> >>> having multiple parent classes for a given child class. Although, > it > >> >>> would be nice if a child class could have different parent types. > >> >>> > >> >>> Thanks for all your help Max. > >> >>> -bryce > >> >>> > >> >>> > >> >>> On Mon, Dec 14, 2009 at 11:30 AM, Max Ross (Google) > >> >>> <maxr+appeng...@google.com <maxr%2bappeng...@google.com>> wrote: > >> >>> > I believe the exception you're getting is the result of having > >> >>> > multiple > >> >>> > parents for the same type, and unfortunately that bug isn't fixed > in > >> >>> > this > >> >>> > release. The bugs fixed in this release are: > >> >>> > > >> >>> > - Incorrect exception for multiple fields of same type > >> >>> > > http://code.google.com/p/datanucleus-appengine/issues/detail?id=172 > >> >>> > > >> >>> > - Non-persistent base classes do not work > >> >>> > > http://code.google.com/p/datanucleus-appengine/issues/detail?id=169 > >> >>> > > >> >>> > - Relationships in abstract base classes don't work > >> >>> > > http://code.google.com/p/datanucleus-appengine/issues/detail?id=171 > >> >>> > > >> >>> > > >> >>> > On Sat, Dec 12, 2009 at 1:09 AM, bryce cottam <bcot...@gmail.com> > >> >>> > wrote: > >> >>> >> > >> >>> >> hmmm.... I think I may have installed the patch wrong perhaps. I > >> >>> >> still get the same error. > >> >>> >> I'm using the same code I sent you for the test case Max. Do you > >> >>> >> see > >> >>> >> any tell-tale signs in here that indicate I've mis-configured > >> >>> >> something? I have datanucleus-appengine-1.0.4.1-RC2.jar on my > >> >>> >> classpath and I removed the previous one (pretty much the same > jar > >> >>> >> but > >> >>> >> with .final.jar as it's name. I followed the steps outlined by > >> >>> >> Jonathan. > >> >>> >> > >> >>> >> here is the stack trace: > >> >>> >> java.lang.ClassCastException: oid is not instanceof > >> >>> >> javax.jdo.identity.ObjectIdentity > >> >>> >> at > >> >>> >> > >> >>> >> > com.resmark.model.RatePlan.jdoCopyKeyFieldsFromObjectId(RatePlan.java) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.setObjectAsValue(PersistenceCapableMapping.java:657) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.mapped.mapping.PersistenceCapableMapping.setObject(PersistenceCapableMapping.java:364) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastoreRelationFieldManager$1.setObjectViaMapping(DatastoreRelationFieldManager.java:128) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastoreRelationFieldManager$1.apply(DatastoreRelationFieldManager.java:104) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastoreRelationFieldManager.storeRelations(DatastoreRelationFieldManager.java:78) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastoreFieldManager.storeRelations(DatastoreFieldManager.java:812) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPostProcess(DatastorePersistenceHandler.java:288) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects(DatastorePersistenceHandler.java:241) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObject(DatastorePersistenceHandler.java:225) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManagerImpl.java:3185) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.java:3161) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java:1298) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.ObjectManagerImpl.persistObject(ObjectManagerImpl.java:1175) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.jdo.JDOPersistenceManager.jdoMakePersistent(JDOPersistenceManager.java:669) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.datanucleus.jdo.JDOPersistenceManager.makePersistent(JDOPersistenceManager.java:694) > >> >>> >> at > >> >>> >> com.resmark.JdoTestServlet.create(JdoTestServlet.java:183) > >> >>> >> at > com.resmark.JdoTestServlet.doTest(JdoTestServlet.java:90) > >> >>> >> at > com.resmark.JdoTestServlet.doGet(JdoTestServlet.java:33) > >> >>> >> at > >> >>> >> javax.servlet.http.HttpServlet.service(HttpServlet.java:693) > >> >>> >> at > >> >>> >> javax.servlet.http.HttpServlet.service(HttpServlet.java:806) > >> >>> >> at > >> >>> >> > >> >>> >> > org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:487) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1093) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(TransactionCleanupFilter.java:43) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > com.google.appengine.tools.development.StaticFileFilter.doFilter(StaticFileFilter.java:121) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1084) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:360) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:712) > >> >>> >> at > >> >>> >> > >> >>> >> > org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:405) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > com.google.apphosting.utils.jetty.DevAppEngineWebAppContext.handle(DevAppEngineWebAppContext.java:70) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > com.google.appengine.tools.development.JettyContainerService$ApiProxyHandler.handle(JettyContainerService.java:352) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:139) > >> >>> >> at org.mortbay.jetty.Server.handle(Server.java:313) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:506) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:830) > >> >>> >> at > >> >>> >> org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:514) > >> >>> >> at > >> >>> >> org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:211) > >> >>> >> at > >> >>> >> org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:381) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:396) > >> >>> >> at > >> >>> >> > >> >>> >> > >> >>> >> > org.mortbay.thread.BoundedThreadPool$PoolThread.run(BoundedThreadPool.java:442) > >> >>> >> > >> >>> >> I've got to be missing something. :( > >> >>> >> > >> >>> >> thanks, > >> >>> >> -bryce > >> >>> >> > >> >>> >> > >> >>> >> On Fri, Dec 11, 2009 at 8:54 AM, bryce cottam <bcot...@gmail.com > > > >> >>> >> wrote: > >> >>> >> > Thanks a million Max, I've been on the road for a few days but > >> >>> >> > I'm > >> >>> >> > looking > >> >>> >> > forward to trying this out when I get back. Thanks for the link > >> >>> >> > Jonathan. > >> >>> >> > > >> >>> >> > -bryce > >> >>> >> > > >> >>> >> > On Dec 11, 2009 6:50 AM, "Jonathan 'J5' Cook" > >> >>> >> > <jonathan.j5.c...@gmail.com> wrote: > >> >>> >> > > >> >>> >> > Thanks for the fix, Max and perfect timing for me :) I just > >> >>> >> > upgraded > >> >>> >> > to 1.2.8 and started having this issue last night. > >> >>> >> > > >> >>> >> > For those who don't/didn't already know (like myself), here is > a > >> >>> >> > link > >> >>> >> > to instructions Max wrote on how to install a new plugin > version > >> >>> >> > into > >> >>> >> > the SDK: > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > >> >>> >> > > http://code.google.com/p/datanucleus-appengine/wiki/HowToUpdateTheSDKWithANewPluginVersion > >> >>> >> > > >> >>> >> > Regards, > >> >>> >> > J5 > >> >>> >> > > >> >>> >> > On Dec 10, 11:59 am, "Max Ross (Google)" > >> >>> >> > <maxr+appeng...@google.com <maxr%2bappeng...@google.com>> > >> >>> >> > wrote: > >> >>> >> >> I've posted a bug fix release candidate containing this fix > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> > >> >>> >> >> here: > http://datanucleus-appengine.googlecode.com/files/appengine-orm-1.0.4... > >> >>> >> > > >> >>> >> >> > On Tue, Dec 8, 2009 at 5:26 PM, bryce cottam > >> >>> >> >> > <bcot...@gmail.com> > >> >>> >> >> > wrote: > >> >>> >> >> > > > That's great news Ma... > >> >>> >> > > >> >>> >> >> > On Dec 8, 2009 5:20 PM, "Max Ross (Google)" > >> >>> >> >> > <maxr+appeng...@google.com <maxr%2bappeng...@google.com>< > maxr%2bappeng...@google.com <maxr%252bappeng...@google.com>>> > >> >>> >> > > >> >>> >> >> > wrote: > > > I've filed bug > > >> >>> >> >> > > >> >>> >> >> > > > >> >>> >> >> > > > > > http://code.google.com/p/datanucleus-appengine/issues/detail?id=1... > >> >>> >> > > >> >>> >> >> > On Tue, Dec 8, 2009 at 11:29 AM, bryce cottam > >> >>> >> >> > <bcot...@gmail.com> > >> >>> >> >> > wrote: > >> >>> >> >> > > > >> > > It's all goo... > >> >>> >> > > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > > >> >>> >> >> > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > <google-appengine-java%2bunsubscr...@googlegroups.com<google-appengine-java%252bunsubscr...@googlegroups.com> > > > >> >>> >> > > >> >>> >> >> > . > > For more options, visit this group at > > >> >>> >> >> > >http://groups.google.com/group/google-appengine-j... > >> >>> >> > >> >>> >> -- > >> >>> >> > >> >>> >> You received this message because you are subscribed to the > Google > >> >>> >> Groups > >> >>> >> "Google App Engine for Java" group. > >> >>> >> To post to this group, send email to > >> >>> >> google-appengine-j...@googlegroups.com. > >> >>> >> To unsubscribe from this group, send email to > >> >>> >> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > >> >>> >> For more options, visit this group at > >> >>> >> http://groups.google.com/group/google-appengine-java?hl=en. > >> >>> >> > >> >>> >> > >> >>> > > >> >>> > -- > >> >>> > > >> >>> > You received this message because you are subscribed to the Google > >> >>> > Groups > >> >>> > "Google App Engine for Java" group. > >> >>> > To post to this group, send email to > >> >>> > google-appengine-j...@googlegroups.com. > >> >>> > To unsubscribe from this group, send email to > >> >>> > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > >> >>> > For more options, visit this group at > >> >>> > http://groups.google.com/group/google-appengine-java?hl=en. > >> >>> > > >> >>> > >> >>> -- > >> >>> > >> >>> You received this message because you are subscribed to the Google > >> >>> Groups > >> >>> "Google App Engine for Java" group. > >> >>> To post to this group, send email to > >> >>> google-appengine-j...@googlegroups.com. > >> >>> To unsubscribe from this group, send email to > >> >>> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > >> >>> For more options, visit this group at > >> >>> http://groups.google.com/group/google-appengine-java?hl=en. > >> >>> > >> >>> > >> >> > >> > > >> > -- > >> > > >> > You received this message because you are subscribed to the Google > >> > Groups > >> > "Google App Engine for Java" group. > >> > To post to this group, send email to > >> > google-appengine-j...@googlegroups.com. > >> > To unsubscribe from this group, send email to > >> > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > >> > For more options, visit this group at > >> > http://groups.google.com/group/google-appengine-java?hl=en. > >> > > >> > >> -- > >> > >> You received this message because you are subscribed to the Google > Groups > >> "Google App Engine for Java" group. > >> To post to this group, send email to > >> google-appengine-j...@googlegroups.com. > >> To unsubscribe from this group, send email to > >> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > >> For more options, visit this group at > >> http://groups.google.com/group/google-appengine-java?hl=en. > >> > >> > > > > -- > > > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > > For more options, visit this group at > > http://groups.google.com/group/google-appengine-java?hl=en. > > > > -- > > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > > > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
