[appengine-java] How to remove JAR files from lib folder in eclipse?
Hello all, I'm trying to get rid of unnecessary JARs in my project, as I don't use any ORMs a lot of stuff can go. However, when I do that I get errors from the eclipse plugin saying "The App Engine SDK JAR datanucleus-appengine-1.0.9.final.jar is missing in the WEB-INF/lib directory " and so on. Everything works fine both in dev and production, but seeing those red marks and constantly saying "deploy despite errors" is going to drive me batty, anyone know how to get rid of them? Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: does Google+ use GAE ?
Could it be that the load created by g+ is responsible for the recent chaos and lack of reliability? On Jul 19, 7:47 am, Didier Durand wrote: > Hi, > > J. Smarr, tech lead for G+, says: "Our backends are built mostly on > top of BigTable and Colossus/GFS" in this short description of Google+ > insfrastructure:http://www.quora.com/Can-someone-provide-some-information-about-Googl > > regards > > didier > > On Jul 19, 6:57 am, "J.Ganesan" wrote: > > > > > > > > > I am curious to know the persistence engine used by Google +. Is it > > not Google App Engine ? > > > J.Ganesan -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Experimental blobstore api returning null keys on blob creation
I'd just like to update that after the recent instability and errors, this problem has gotten MUCH WORSE. I really hope that something can be done, soon. On Jul 17, 12:21 pm, tempy wrote: > That's exactly my issue, starred, thanks Tom. > > This has been a high priority issue for 3 months now, and still open! > > On Jul 17, 4:41 am, Tom Fishman wrote: > > > > > > > > > Assuming you're talking about new blobs, please check this > > issue:http://code.google.com/p/googleappengine/issues/detail?id=4872. > > > I think there is no "good" workaround yet. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Experimental blobstore api returning null keys on blob creation
That's exactly my issue, starred, thanks Tom. This has been a high priority issue for 3 months now, and still open! On Jul 17, 4:41 am, Tom Fishman wrote: > Assuming you're talking about new blobs, please check this > issue:http://code.google.com/p/googleappengine/issues/detail?id=4872. > > I think there is no "good" workaround yet. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Experimental blobstore api returning null keys on blob creation
Hello all, I have code that uses the new blobstore fileservice api to write data submitted by POST to a blob. This code hasn't changed in around 2 months and has been working reliably, but suddenly its failing. No exceptions or anything, but the AppEngineFile.getBlobKey() call returns null, even though I can see that the blob has been created in the blobstore without problems. It doesn't fail reliably, but frequently. This is critical code for me, and its really a problem. In addition to this error, I've been getting lots of other random errors as others have been reporting. I use GAE as a backend for a mobile app, and just recently one of these bursts of errors hapenned while one version of my app was going through appstore approval. Naturally, the app recovers gracefully, but the user still sees a nice "sync failed" error message. So, this instability is affecting me in multiple ways. I really hope that with the new pricing we get some better accountability, and also some more transparency to see if GAE really is living up to that SLA we're supposed to get. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Instructions for migrating a GAE-Java app to the HR datastore
Ok, roughly it went like this: - Download and install python sdk - Create a python app with the same ID as my java app but a new version, enable the datastore admin in the configuration for that version, and upload. - Create a new java app with the HR datastore. - Create a python app with the same ID but a new version for the new app, enable remote_api and datastore admin. Also, in that app's appengine_config.py file, make sure to set the remoteapi_CUSTOM_ENVIRONMENT_AUTHENTICATION setting to the id of your source app - For the love of all that is good, write some code to clean up orphaned datastore entities in your source app's datastore, or the transfer will fail in a fascinating and CPU-devouring manner. - Once you've done that, select all the entities from your source app's datastore admin section (you have to change the dashboard to the python version of your source app to see the datastore admin) and choose to transfer them to your target newly created HR app. You'll have a chance to provide the target app's ID after you choose to transfer entities. - Watch the task queue, when its empty the transfer is done. It took a few minutes for all my 20 MB worth of data (split up over tens of thousands of tiny entities). - Now switch to your new HR app, make sure everything looks ok. If you had any blobs in the blobstore, this is when you figure out some manual way to move them. If you had blobkeys in your datastore, they are now all broken and this is where you figure out how to fix them. - In the appengine-web.xml file for your old app, change the ID to the new one, set an appropriate version, and upload. This is all you have to do to get all your code/config/jsps/static content into the new version. - If you're using google apps for your domain, log into google apps and remove the old version of your app from your domain. - In the dashboard for your new app, add your domain. For me it took about 30 minutes to switch over. - Don't forget to re-enable billing on your target up. - Apologize to users. My app is small enough for now that I didn't bother making any datastore readonly, I just found a quiet time and did all this as fast as possible. It wasn't the worst thing in the world, but it wasn't exactly "seamless" either. If I had a lot of blobs and/or traffic, I imagine this would be a nightmare. This post comes with no warranties, implied or otherwise, best of luck! On Jun 15, 1:12 pm, tempy wrote: > I've finally decided to upgrade, while my data still isn't huge. > Maybe I'm missing something, but I'm yet to find an article or blog > post or group entry that has simple "recommended" steps, instead I'm > piecing the procedure together. As I understand it, it goes something > like this: > > - Install python SDK, create python version of source Java app. > > - This should open the datastore admin section in the dashboard for my > app. > > - Create new java app with HR datastore, this will be the target. > Create python version of that app, and also follow the directions > here:http://code.google.com/appengine/docs/adminconsole/datastoreadmin.htm... > > - Copy all data from one app to the other. > > - Go > here:http://code.google.com/support/bin/request.py?contact_type=AppEngineB... > And beg them to make an alias from the old ID to the new one so that > my custom domain remains functional. > > - Enable billing for new app, hope it goes smoothly. > > I'm fuzzy on pretty much all of this and I'm sure I'm missing > something, and while I normally don't mind muddling through, this > procedure means downtime so I would rather not run into any > surprises. Has anyone done this? Is there some guidance out there > that I'm missing? Does anyone know how this effects existing oauth > tokens generated for the app, or new ones? > > Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Instructions for migrating a GAE-Java app to the HR datastore
I've finally decided to upgrade, while my data still isn't huge. Maybe I'm missing something, but I'm yet to find an article or blog post or group entry that has simple "recommended" steps, instead I'm piecing the procedure together. As I understand it, it goes something like this: - Install python SDK, create python version of source Java app. - This should open the datastore admin section in the dashboard for my app. - Create new java app with HR datastore, this will be the target. Create python version of that app, and also follow the directions here: http://code.google.com/appengine/docs/adminconsole/datastoreadmin.html#Copying_Entities_to_Another_Application - Copy all data from one app to the other. - Go here: http://code.google.com/support/bin/request.py?contact_type=AppEngineBillingSupport And beg them to make an alias from the old ID to the new one so that my custom domain remains functional. - Enable billing for new app, hope it goes smoothly. I'm fuzzy on pretty much all of this and I'm sure I'm missing something, and while I normally don't mind muddling through, this procedure means downtime so I would rather not run into any surprises. Has anyone done this? Is there some guidance out there that I'm missing? Does anyone know how this effects existing oauth tokens generated for the app, or new ones? Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to design an in-memory cache?
Prashant, thats exactly what we were talking about, that code looks great, thanks! On Jun 15, 12:46 pm, Prashant wrote: > Hi, > > I am using such implementation, with 15 mins refresh time, for my > application. you can see the implementation here ... > > https://code.google.com/p/claymus/source/browse/trunk/src/com/claymus... > > > > > > > > On Wed, Jun 15, 2011 at 3:37 PM, tempy wrote: > > Didier and J. Ganesan, I'm not talking about memcache. I want > > something as simple as a static map with synchronized access, within > > which I would cache data. > > > Jeff, how would one accomplish that 30 minute timeout, without > > threads, timers, etc. I suppose I could keep the time of entry, and > > if more than 30 minutes has elapsed then I fall back onto memcache or > > the datastore, not pretty but it'll do. And yes what you mention > > would be very nice to have. > > > On Jun 14, 5:33 pm, "J.Ganesan" wrote: > > > Why not call memcacheService.delete( keyObject ) ? > > > > J.Ganesan > > > > On Jun 14, 8:25 pm, tempy wrote: > > > > > Hello all, > > > > I'm wondering if its possible to design a cache using data in static > > > > variables. I have certain pieces of small and unchanging data, such > > > > as a banner for the top of the site whose contents might only change > > > > every couple of days. This is perfect for storing in a static > > > > variable and falling back onto the memcache as necessary. The one > > > > thing I don't know how to do is to clear that cache. If I want to > > > > update that data, I can clear the memcache, but there's no way that I > > > > can think of to clear static data cache, other than to wait until a > > > > particular instance goes down, and who knows when that will happen. > > > > Any ideas? Thanks! > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-java@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com. > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to design an in-memory cache?
Didier and J. Ganesan, I'm not talking about memcache. I want something as simple as a static map with synchronized access, within which I would cache data. Jeff, how would one accomplish that 30 minute timeout, without threads, timers, etc. I suppose I could keep the time of entry, and if more than 30 minutes has elapsed then I fall back onto memcache or the datastore, not pretty but it'll do. And yes what you mention would be very nice to have. On Jun 14, 5:33 pm, "J.Ganesan" wrote: > Why not call memcacheService.delete( keyObject ) ? > > J.Ganesan > > On Jun 14, 8:25 pm, tempy wrote: > > > > > > > > > Hello all, > > I'm wondering if its possible to design a cache using data in static > > variables. I have certain pieces of small and unchanging data, such > > as a banner for the top of the site whose contents might only change > > every couple of days. This is perfect for storing in a static > > variable and falling back onto the memcache as necessary. The one > > thing I don't know how to do is to clear that cache. If I want to > > update that data, I can clear the memcache, but there's no way that I > > can think of to clear static data cache, other than to wait until a > > particular instance goes down, and who knows when that will happen. > > Any ideas? Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to design an in-memory cache?
Hello all, I'm wondering if its possible to design a cache using data in static variables. I have certain pieces of small and unchanging data, such as a banner for the top of the site whose contents might only change every couple of days. This is perfect for storing in a static variable and falling back onto the memcache as necessary. The one thing I don't know how to do is to clear that cache. If I want to update that data, I can clear the memcache, but there's no way that I can think of to clear static data cache, other than to wait until a particular instance goes down, and who knows when that will happen. Any ideas? Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How does JDO convert persistent class field names to entity property names?
I am slowly (painfully) migrating from JDO to the low-level API. One thing that's giving me problems is this: I have many persistent fields that start with an underscore: private int _field1; private int _field2;, etc. I am, for the time being, accessing this data with JDO in one part of the app, and with the low-level API in another part. I've noticed that JDO strips out the underscore for some fields for the entity property names, but not for all such fields. I tried digging through the datanucleus-GAE code to figure out what the rule for jdo-object- >GAE-entity field name conversion is, but quickly got lost. Any help would be much appreciated. Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Performance of blobstore upload API vs the new files api
Hello all, I'm using GAE as a backend for a desktop app, and the app sends potentially large compressed data to GAE, (max file size around 5mb - average around 500k). I see two ways to accomplish this upload - using the regular blobstore upload API, or the new file API. The downside of the regular blobstore api is that the desktop app needs to first request an upload URL, and then do multipart-form encoding of the data that its sending. Also, the desktop app authenticates via oauth, and receives some data back from GAE as http headers. Using the regular blobstore upload api makes all of this more complicated. So, I would like to have the app do a regular POST and then have GAE write the data to the blobstore via the new files api, but I'm not sure about how that performs next to the standard API. Performance is very important so I need the request to return as quickly as possible, and I suspect that the new API won't write to the blobstore as fast as the standard API. Thanks! -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] What am I to make of these isolated exceptions?
I woke up to a bunch of these exceptions, though no code has changed: Class "X" has collection field "collection of Ys" and this has no mapping in the table for the element class "Y" owner field "ParentField" Because they don't look like the usual datastore-is-on-the-fritz errors, I go off on a wild goose chase trying to diagnose. Eventually they stop happening. What am I to make of these? -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: 500 Errors on blobstore upload, perhaps related to oauth?
Problem solved, for anyone going through something similar, see my stackoverflow post: http://stackoverflow.com/questions/4096990/getting-500-on-blobstore-upload-in-production-dev-server-works-fine/4098470#4098470 On Nov 3, 10:51 pm, tempy wrote: > After investigating further, things get more confusing. I guess the > oauth stuff in the upload URL is as it should be, since disabling > oauth in my app did not change the presence of the oauth stuff in the > upload URL. > > After watching the exchange between my client and GAE app with > fiddler, I realized that the 500 that was being returned was coming > from my GAE code and not the blobservice, and it contained an > exception that my GAE app produces when the blobstore calls the "on- > success" URL that is supplied to the > blobstoreService.createUploadUrl(). The exception is caused by the > blobstoreService.getUploadedBlobs() call returning an empty map. > What's confusing me is that evidently the blobstore is failing for > some reason, and yet it is still calling the "on-success" URL within > my app. Is this the expected behavior when a blobstore upload fails? > It would be great to get some sort of meaningful error from the > blobstore, instead of just having the failure passed on to my app in > the form of an empty map. > > On Nov 2, 9:59 pm, tempy wrote: > > > > > > > > > Hello, > > I have yet to get the blobstore functioning correctly in my app. My > > code works in development, but not in production. > > > I use the built-in oauth support and have an installed app connecting > > to the GAE app. The installed app requests an upload URL from an > > endpoint that is secured by oauth, and I was surprised to see that > > BlobStoreService.CreateUploadURL() produces a URL that has all the > > oauth stuff in it. Here is the URL it produced: > > >http://MYAPPo.appspot.com/_ah/upload/?oauth_token=1%2FTRaFrr7-6m5... > > > When my installed app then tried to POST to this URL, it gets a 500. > > The Blobstore then proceeds to call my GAE app with the URL I supplied > > to CreateUploadURL(), and passes along all the parameters that the > > client app sent to the Blobstore, but (as expected) without a > > Blobstore key. I don't see any blobs in the blob viewer. > > > I wonder if the oauth code is somehow at fault here, but other than > > that I don't really know where to start diagnosing this. > > > Thanks, > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: 500 Errors on blobstore upload, perhaps related to oauth?
After investigating further, things get more confusing. I guess the oauth stuff in the upload URL is as it should be, since disabling oauth in my app did not change the presence of the oauth stuff in the upload URL. After watching the exchange between my client and GAE app with fiddler, I realized that the 500 that was being returned was coming from my GAE code and not the blobservice, and it contained an exception that my GAE app produces when the blobstore calls the "on- success" URL that is supplied to the blobstoreService.createUploadUrl(). The exception is caused by the blobstoreService.getUploadedBlobs() call returning an empty map. What's confusing me is that evidently the blobstore is failing for some reason, and yet it is still calling the "on-success" URL within my app. Is this the expected behavior when a blobstore upload fails? It would be great to get some sort of meaningful error from the blobstore, instead of just having the failure passed on to my app in the form of an empty map. On Nov 2, 9:59 pm, tempy wrote: > Hello, > I have yet to get the blobstore functioning correctly in my app. My > code works in development, but not in production. > > I use the built-in oauth support and have an installed app connecting > to the GAE app. The installed app requests an upload URL from an > endpoint that is secured by oauth, and I was surprised to see that > BlobStoreService.CreateUploadURL() produces a URL that has all the > oauth stuff in it. Here is the URL it produced: > > http://MYAPPo.appspot.com/_ah/upload/?oauth_token=1%2FTRaFrr7-6m5... > > When my installed app then tried to POST to this URL, it gets a 500. > The Blobstore then proceeds to call my GAE app with the URL I supplied > to CreateUploadURL(), and passes along all the parameters that the > client app sent to the Blobstore, but (as expected) without a > Blobstore key. I don't see any blobs in the blob viewer. > > I wonder if the oauth code is somehow at fault here, but other than > that I don't really know where to start diagnosing this. > > Thanks, > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] 500 Errors on blobstore upload, perhaps related to oauth?
Hello, I have yet to get the blobstore functioning correctly in my app. My code works in development, but not in production. I use the built-in oauth support and have an installed app connecting to the GAE app. The installed app requests an upload URL from an endpoint that is secured by oauth, and I was surprised to see that BlobStoreService.CreateUploadURL() produces a URL that has all the oauth stuff in it. Here is the URL it produced: http://MYAPPo.appspot.com/_ah/upload/?oauth_token=1%2FTRaFrr7-6m5xhHfTNxEWro&oauth_consumer_key=anonymous&oauth_nonce=jp3RjU8Q&oauth_signature_method=HMAC-SHA1&oauth_signature=%2BArexxxQfQrcg%3D&oauth_version=1.0&oauth_timestamp=1288724065/AMmfu6ZXzzxvVnfoJ_K5kHbgI-uk87Jp_0AV140OVlmUURx8x8qJCeP_GqXSrl5cm3-WtXObp8hQ8DxLBNUaYATNBgt3DyA_0PLeSByXaA-g5FvxLzpCMj/' When my installed app then tried to POST to this URL, it gets a 500. The Blobstore then proceeds to call my GAE app with the URL I supplied to CreateUploadURL(), and passes along all the parameters that the client app sent to the Blobstore, but (as expected) without a Blobstore key. I don't see any blobs in the blob viewer. I wonder if the oauth code is somehow at fault here, but other than that I don't really know where to start diagnosing this. Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Need advice on a design for an easily queryable key-value user preferences datamodel
One more thing - your example is actually a perfect illustration of what I have in mind - users will sign up to be notified by email of all sorts of application events. I haven't read much about it yet, but I take it that sending out such mass-notifications is exactly the sort of task that the Mapper API is envisioned for? On Oct 19, 8:19 pm, "Ikai Lan (Google)" wrote: > This is a problem that can probably be easily solved with the low-level API. > Entities are schemaless and you can treate properties like a Map. You could > probably do something like this: > > Entity settings = new Entity("Settings", user.getKey()); > settings.setUnindexedProperty("dailyEmail", false); > settings.setUnindexedProperty("backgroundColor", "red"); > // etc > > I'm using unindexed properties because it seems unlikely you would want to > query on user settings (would you ever want to know every user's timezone? > You might - but this would be something you might want to do in reporting, > and the Mapper API is key here since indexes take space). > > Read more about the low-level API here: > > http://code.google.com/appengine/docs/java/javadoc/com/google/appengi... > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > > > On Mon, Oct 18, 2010 at 5:04 AM, tempy wrote: > > Hello all, > > In my app, the root element is a user, with which I want to associate > > a simple key-value structure for storing user preferences. It should > > be such a structure because I really can't foresee what preferences > > will be added in the future. At first I thought of using a simple > > @serialized set in the user object, but then I > > realized how difficult this would be to query. I need a simple way to > > both "Get all users where key==facebooksync && value ==true", and to > > also get all of a particular user's settings. > > > My current idea would be to have each unique key-value pair have its > > own (huge) object, with the following data > > > Setting{ > > String key; > > String value; > > List User; > > } > > > The user would also have keys pointing at the relevant Setting > > objects. > > > As I see it, this would be easily queryable, but updating settings > > could get sticky with a large number of users. > > > I would appreciate any feedback on this! > > > Thanks, > > Mike > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > unsubscr...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Need advice on a design for an easily queryable key-value user preferences datamodel
And I thought that I had gotten to the finish line without ever needing the low-level API, guess I was wrong =). Thanks for the hint Ikai! On Oct 21, 7:39 pm, "Ikai Lan (Google)" wrote: > Yes, but you will have to dig a bit: in the source: > > http://www.google.com/codesearch/p?hl=en#Qx8E-7HUBTk/trunk/python/goo... > > Schemaless entities are how datastore models are saved underneath the hood. > > -- > Ikai Lan > Developer Programs Engineer, Google App Engine > Blogger:http://googleappengine.blogspot.com > Reddit:http://www.reddit.com/r/appengine > Twitter:http://twitter.com/app_engine > > On Thu, Oct 21, 2010 at 5:47 AM, speedplane wrote: > > > > > Very cool... I didn't know that existed. Is there a similar low level > > api for python? You have a doc link? > > > On Oct 19, 2:19 pm, "Ikai Lan (Google)" > > > > > wrote: > > > This is a problem that can probably be easily solved with the low-level > > API. > > > Entities are schemaless and you can treate properties like a Map. You > > could > > > probably do something like this: > > > > Entity settings = new Entity("Settings", user.getKey()); > > > settings.setUnindexedProperty("dailyEmail", false); > > > settings.setUnindexedProperty("backgroundColor", "red"); > > > // etc > > > > I'm using unindexed properties because it seems unlikely you would want > > to > > > query on user settings (would you ever want to know every user's > > timezone? > > > You might - but this would be something you might want to do in > > reporting, > > > and the Mapper API is key here since indexes take space). > > > > Read more about the low-level API here: > > > >http://code.google.com/appengine/docs/java/javadoc/com/google/appengi... > > > > -- > > > Ikai Lan > > > Developer Programs Engineer, Google App Engine > > > Blogger:http://googleappengine.blogspot.com > > > Reddit:http://www.reddit.com/r/appengine > > > Twitter:http://twitter.com/app_engine > > > > On Mon, Oct 18, 2010 at 5:04 AM, tempy wrote: > > > > Hello all, > > > > In my app, the root element is a user, with which I want to associate > > > > a simple key-value structure for storing user preferences. It should > > > > be such a structure because I really can't foresee what preferences > > > > will be added in the future. At first I thought of using a simple > > > > @serialized set in the user object, but then I > > > > realized how difficult this would be to query. I need a simple way to > > > > both "Get all users where key==facebooksync && value ==true", and to > > > > also get all of a particular user's settings. > > > > > My current idea would be to have each unique key-value pair have its > > > > own (huge) object, with the following data > > > > > Setting{ > > > > String key; > > > > String value; > > > > List User; > > > > } > > > > > The user would also have keys pointing at the relevant Setting > > > > objects. > > > > > As I see it, this would be easily queryable, but updating settings > > > > could get sticky with a large number of users. > > > > > I would appreciate any feedback on this! > > > > > Thanks, > > > > Mike > > > > > -- > > > > You received this message because you are subscribed to the Google > > Groups > > > > "Google App Engine for Java" group. > > > > To post to this group, send email to > > > > google-appengine-j...@googlegroups.com. > > > > To unsubscribe from this group, send email to > > > > google-appengine-java+unsubscr...@googlegroups.com > > > unsubscr...@googlegroups.com> > > > %252bunsubscr...@googlegroups.com> > > > > > . > > > > For more options, visit this group at > > > >http://groups.google.com/group/google-appengine-java?hl=en. > > > -- > > You received this message because you are subscribed to the Google Groups > > "Google App Engine for Java" group. > > To post to this group, send email to > > google-appengine-j...@googlegroups.com. > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > unsubscr...@googlegroups.com> > > . > > For more options, visit this group at > >http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Need advice on a design for an easily queryable key-value user preferences datamodel
Hello all, In my app, the root element is a user, with which I want to associate a simple key-value structure for storing user preferences. It should be such a structure because I really can't foresee what preferences will be added in the future. At first I thought of using a simple @serialized set in the user object, but then I realized how difficult this would be to query. I need a simple way to both "Get all users where key==facebooksync && value ==true", and to also get all of a particular user's settings. My current idea would be to have each unique key-value pair have its own (huge) object, with the following data Setting{ String key; String value; List User; } The user would also have keys pointing at the relevant Setting objects. As I see it, this would be easily queryable, but updating settings could get sticky with a large number of users. I would appreciate any feedback on this! Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Need help choosing a frontend framework for a GAEj app, GWT seems like overkill
Since my last post I've started using jQuery paired with GSon. Though I barely know javascript, the ease with which I was able to take data from my GAE backend and stick it into already-designed HTML in an AJAXy manner was breathtaking. I think I'll be able to live without GWT's RPC for this one project. I certainly see the appeal of GWT, I'm used to building sophisticated desktop UIs. But for this app whose UI will resemble a basic tumblr site, I think jquery+gson is the right fit for the job. Now if only I could figure out history management/simple URLs/bookmark support, I'd be golden! On Sep 10, 8:08 pm, "Ikai L (Google)" wrote: > It's very hard to use GWT's RPC mechanism without using GWT. GWT's RPC > mechanism is nice because it provides a server side implementation, but on > the client side, you're still creating an AsyncCallback and programming in > an asynchronous manner. This is an area where Javascript really shines > because of first class functions you can pass around (not available in Java) > and closures (not available in Java unless you count anonymous inner > classes). > > Rather than look at a single feature, look at the bigger picture. With GWT, > you get a ton of useful widgets and a way to programmatically (or via XML > via UiBinder) define your UI, you get an out-of-the-box event bus and > history manager. This is actually a pretty neat way to develop a web app > because you can compose widgets of widgets. I'd probably pick jQuery for > more "web page" like interaction or applications with a less desktop like > experience, because the interface for working with the DOM is one of the > best I've ever used (to select and hide all parents of list elements with a > class "item", you'd do this: $("li.item").parents().hide());. jQuery's XHR > interface is simple and straightforward, and combined with a library like > Google GSON (http://code.google.com/p/google-gson/), you can build XHR rich > applications fairly easily. > > I personally prefer GWT for anything that has to deal with detecting where a > mouse click is or key handler events, or something where I have to > programmatically assemble the UI which can be represented as a hierarchical > bag of widgets. The jQuery way would be to output some HTML and jam it into > an element using the .html() function, which is really only a notch above > using the DOM's innerHTML= attribute. > > > > > > On Fri, Sep 10, 2010 at 4:47 AM, tempy wrote: > > Thanks for the advice John. I've been reading about GQuery and it > > does seem good, though I have some concerns about whether it might be > > abandoned. I think my plan is to learn JQuery, as I feel like that > > should also teach me a lot about how the browser operates, which is a > > gap in my knowledge. I think I should learn something "close to the > > metal" (metal being browser in this case) before moving on to > > frameworks that abstract the browser away to a greater extent. I > > guess if I use JQuery, switching to GQuery at any time should not be > > problematic, and I get the best of both worlds. And I would sure love > > to hang on to GWT's RPC mechanism. > > > Mike > > > On Sep 10, 12:56 am, John Patterson wrote: > > > BTW, you can use GWT in a manor very similar to JQuery. Take a look a > > > GQuery by Ray Cromwell. He has an incredible benchmark page which > > > dynamically shows how GQuery out performs JQuery in almost every measure. > > > > On 10 September 2010 05:53, John Patterson > > wrote: > > > > > You might want to look into Sitebricks which is a Google developed web > > app > > > > framework built on top of Guice. It is a simple request response > > processor > > > > with no fancy "component" abstractions like Wicket or Tapestry. This > > model > > > > fits in very well with GWT (or other client frameworks) which request > > data > > > > via RPC or small snippets of rendered html to assemble on the client. > > > > > It is still very much in active development so you have to be prepared > > to > > > > dig into the code and figure out the plumbing yourself. The author, > > Dhanji, > > > > has built a fantastic API here - simple, powerful and clever. > > > > >http://code.google.com/p/google-sitebricks/ > > > > > John > > > > > On 10 September 2010 05:35, tempy wrote: > > > > >> Thanks for the explanation Chris! > > > > >> Guess I'm learning jquery, golly, I'll be a real web
[appengine-java] Re: Need help choosing a frontend framework for a GAEj app, GWT seems like overkill
Thanks for the advice John. I've been reading about GQuery and it does seem good, though I have some concerns about whether it might be abandoned. I think my plan is to learn JQuery, as I feel like that should also teach me a lot about how the browser operates, which is a gap in my knowledge. I think I should learn something "close to the metal" (metal being browser in this case) before moving on to frameworks that abstract the browser away to a greater extent. I guess if I use JQuery, switching to GQuery at any time should not be problematic, and I get the best of both worlds. And I would sure love to hang on to GWT's RPC mechanism. Mike On Sep 10, 12:56 am, John Patterson wrote: > BTW, you can use GWT in a manor very similar to JQuery. Take a look a > GQuery by Ray Cromwell. He has an incredible benchmark page which > dynamically shows how GQuery out performs JQuery in almost every measure. > > On 10 September 2010 05:53, John Patterson wrote: > > > > > You might want to look into Sitebricks which is a Google developed web app > > framework built on top of Guice. It is a simple request response processor > > with no fancy "component" abstractions like Wicket or Tapestry. This model > > fits in very well with GWT (or other client frameworks) which request data > > via RPC or small snippets of rendered html to assemble on the client. > > > It is still very much in active development so you have to be prepared to > > dig into the code and figure out the plumbing yourself. The author, Dhanji, > > has built a fantastic API here - simple, powerful and clever. > > >http://code.google.com/p/google-sitebricks/ > > > John > > > On 10 September 2010 05:35, tempy wrote: > > >> Thanks for the explanation Chris! > > >> Guess I'm learning jquery, golly, I'll be a real web programmer yet. > >> =) > > >> On Sep 9, 7:58 pm, "Chris (Google Employee)" > >> wrote: > >> > Hi Mike, > > >> > I think the reason we don't explicitly publish a list of "frontend > >> > frameworks that play well with App Engine" is because there really is > >> > no technical barrier between front end frameworks that work in the > >> > browser such as JQuery, etc and the backend (App Engine). Many App > >> > Engine users make full use of a myriad of front end technologies/ > >> > frameworks for UI creation ranging from HTML5/JS/CSS to flash etc. > > >> > JQuery is especially popular for a variety of Web app usages, so if > >> > that works for you, I'd definitely continue trying it out. > > >> > Hope this helps, > >> > -Chris > > >> > On Sep 9, 10:43 am, tempy wrote: > > >> > > Hello all, > > >> > > A little background... I'm mostly new to web frontend development, I > >> > > come from a mostly backend and desktop background. I have a GAEj app > >> > > that provides the backend for a rather sophisticated desktop/mobile > >> > > app. This backend will also drive a website in addition to the > >> > > desktop/mobile app. > > >> > > The website will show a small subset of the data in the system. I > >> > > started with GWT, but it doesn't seem to be quite right. The website > >> > > is for presentation of a small subset of the system's data only, and > >> > > users do not interact with the data in any way other than searching > >> > > for it. As the data is mostly unstructured text and some images, I > >> > > feel most comfortable formatting it with plain old html/css. In fact, > >> > > the layout of the site will be minimal and look a lot like a blog. As > >> > > such, the website should certainly not feel like a single-page "web > >> > > application" a la gmail. With GWT, I have little use for widgets and > >> > > find myself constantly fighting with it to get a non-application look- > >> > > and-feel. > > >> > > The only functionality that I thus really need from GWT is the RPC > >> > > component, the page should be AJAX, I don't want to do round trips to > >> > > the server for every navigation event. > > >> > > So, I'm not quite sure which framework to use with the GAEj backend. > >> > > At first GWT seemed like a natural fit, but now seems like overkill. > >> > > I'm doing some research on jquery, but I haven't seen a list of > >> > > &q
[appengine-java] Re: Need help choosing a frontend framework for a GAEj app, GWT seems like overkill
Thanks for the explanation Chris! Guess I'm learning jquery, golly, I'll be a real web programmer yet. =) On Sep 9, 7:58 pm, "Chris (Google Employee)" wrote: > Hi Mike, > > I think the reason we don't explicitly publish a list of "frontend > frameworks that play well with App Engine" is because there really is > no technical barrier between front end frameworks that work in the > browser such as JQuery, etc and the backend (App Engine). Many App > Engine users make full use of a myriad of front end technologies/ > frameworks for UI creation ranging from HTML5/JS/CSS to flash etc. > > JQuery is especially popular for a variety of Web app usages, so if > that works for you, I'd definitely continue trying it out. > > Hope this helps, > -Chris > > On Sep 9, 10:43 am, tempy wrote: > > > > > Hello all, > > > A little background... I'm mostly new to web frontend development, I > > come from a mostly backend and desktop background. I have a GAEj app > > that provides the backend for a rather sophisticated desktop/mobile > > app. This backend will also drive a website in addition to the > > desktop/mobile app. > > > The website will show a small subset of the data in the system. I > > started with GWT, but it doesn't seem to be quite right. The website > > is for presentation of a small subset of the system's data only, and > > users do not interact with the data in any way other than searching > > for it. As the data is mostly unstructured text and some images, I > > feel most comfortable formatting it with plain old html/css. In fact, > > the layout of the site will be minimal and look a lot like a blog. As > > such, the website should certainly not feel like a single-page "web > > application" a la gmail. With GWT, I have little use for widgets and > > find myself constantly fighting with it to get a non-application look- > > and-feel. > > > The only functionality that I thus really need from GWT is the RPC > > component, the page should be AJAX, I don't want to do round trips to > > the server for every navigation event. > > > So, I'm not quite sure which framework to use with the GAEj backend. > > At first GWT seemed like a natural fit, but now seems like overkill. > > I'm doing some research on jquery, but I haven't seen a list of > > "frontend frameworks that play well on GAEj" anywhere. > > > Thanks for any advice, > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Need help choosing a frontend framework for a GAEj app, GWT seems like overkill
Hello all, A little background... I'm mostly new to web frontend development, I come from a mostly backend and desktop background. I have a GAEj app that provides the backend for a rather sophisticated desktop/mobile app. This backend will also drive a website in addition to the desktop/mobile app. The website will show a small subset of the data in the system. I started with GWT, but it doesn't seem to be quite right. The website is for presentation of a small subset of the system's data only, and users do not interact with the data in any way other than searching for it. As the data is mostly unstructured text and some images, I feel most comfortable formatting it with plain old html/css. In fact, the layout of the site will be minimal and look a lot like a blog. As such, the website should certainly not feel like a single-page "web application" a la gmail. With GWT, I have little use for widgets and find myself constantly fighting with it to get a non-application look- and-feel. The only functionality that I thus really need from GWT is the RPC component, the page should be AJAX, I don't want to do round trips to the server for every navigation event. So, I'm not quite sure which framework to use with the GAEj backend. At first GWT seemed like a natural fit, but now seems like overkill. I'm doing some research on jquery, but I haven't seen a list of "frontend frameworks that play well on GAEj" anywhere. Thanks for any advice, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to debug blobstore uploads with 405
Answered my own question... I was redirecting to a bad "success url", which didn't support POST. Not the most informative error, but still understandable. On Aug 27, 12:58 pm, tempy wrote: > Hello all, > > I am trying to upload to the blobstore from an installed c# client, > not a browser. So far, I've only tried the development server... I > can see in the datastore viewer that the blobs are being correctly > created because there are corresponding blobinfo objects that > correctly reflect the uploaded data, but my client gets a 405 response > from the blobstore and my "success url" never gets invoked. How does > one go about debugging this? I don't see any kind of output from the > blobstore on the eclipse console, so I'm not sure where to start. > > Thanks, > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to debug blobstore uploads with 405
Hello all, I am trying to upload to the blobstore from an installed c# client, not a browser. So far, I've only tried the development server... I can see in the datastore viewer that the blobs are being correctly created because there are corresponding blobinfo objects that correctly reflect the uploaded data, but my client gets a 405 response from the blobstore and my "success url" never gets invoked. How does one go about debugging this? I don't see any kind of output from the blobstore on the eclipse console, so I'm not sure where to start. Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Problem getting list of JDO objects with List of of keys
I've had trouble with the .getById methods, so I do it like this, which seems to work reliably: query = pm.newQuery(MyClass.class); query.setFilter("_ID == :keyList"); //_ID is the PK List results = (List)query.execute(Keys); // Keys is List On May 24, 2:19 am, Ravi wrote: > Hi, > I have a list of Key and i want to get Entity for all these key using > JDO. > Is it possible in JDO? Having a hard time to search through internet > and not finding it. > Tried using pm.getObjectsById(Collection), but it says Object not > found with keys present in the list. I wrote following code for this > > PersistenceManager pm = > DbSessionManager.getInstance().startDbSession(); > List keyList = new ArrayList(); > Key oneKey = KeyFactory.createKey("User", 9); > keyList.add(oneKey); > List users = (List)pm.getObjectsById(keyList); (Note: > s after object) > > abovce code throws Exception that User(9) not found but it does exists > in datastore and if i do > > User users = (User)pm.getObjectById(oneKey);(Note: no s after object) > > it returns the User(9) > > I also tried using Low level API like this and it works, but only > problem is that it doesnt return its owned properties(see User class > at the end, openIds are not returned and its always null, even though > it does exists). > > public Map getObjectsById(List ids) throws > AppException { > DatastoreService ds = > DatastoreServiceFactory.getDatastoreService(); > Map data = ds.get(ids); > return data; > > } > > class User{ > Key id; > String firstName; > String lastName; > List openIds; > > } > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com. > For more options, visit this group > athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to separate duration of cold-start time from user-code-time via Appstats?
So I've set up appstats, and I tend to see the following picture: The entire request took around 5 seconds, there are some RPC call bars that add up to around 2 seconds, and the rest of the request time is simply empty space in the timeline. Can I safely assume that all the empty space before the first RPC call in the timeline is spin-up time? The code that runs before the first RPC call is more or less trivial, so I can't imagine what else would be taking up all this time, but it would be nice if spin-up time was clearly delineated. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to separate duration of cold-start time from user-code-time via Appstats?
So I've set up appstats, and I tend to see the following picture: The entire request took around 5 seconds, there are some RPC call bars that add up to around 2 seconds, and the rest of the request time is simply empty space in the timeline. Can I safely assume that all the empty space before the first RPC call in the timeline is spin-up time? The code that runs before the first RPC call is more or less trivial, so I can't imagine what else would be taking up all this time, but it would be nice if spin-up time was clearly delineated. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Is it possible to have collections of embedded objects?
Well, what I need is something like this: @PersistenceCapable(detachable = "true") public class Parent { @PersistenceCapable @EmbeddedOnly public class EmbeddedChild {...} @Embedded private List Children; ... } This doesn't work, so... Do I understand you correctly, that to accomplish this I have to serialize "Children" into a Text value or something like that? On Apr 7, 7:40 pm, Tristan wrote: > what do you mean by "store a list of embedded objects"? > > if you're storing in the datastore, you have to have a way to > serialize whatever you got into one of the supported storable types. > > On Apr 7, 11:04 am, tempy wrote: > > > > > I have been dealing with lots of varying exceptions trying to > > implement/store a list of embedded objects. I found one post on this > > group claiming that this isn't supported (but the claim didn't come > > from someone at Google), and I saw no mention of lists of embedded > > objects in the google docs... So, does anyone know for sure, if this > > is actually supported, or not? > > > Thanks, > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Is it possible to have collections of embedded objects?
I have been dealing with lots of varying exceptions trying to implement/store a list of embedded objects. I found one post on this group claiming that this isn't supported (but the claim didn't come from someone at Google), and I saw no mention of lists of embedded objects in the google docs... So, does anyone know for sure, if this is actually supported, or not? Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Possible bug in pm.makePersistentAll? Random-looking exceptions, can't make heads or tails of it...
Anybody? On Mar 23, 7:57 pm, tempy wrote: > MODEL OBJECTS: > > @PersistenceCapable(detachable = "true") > @FetchGroup(name = "children", members = { @Persistent(name = > "_Children") }) > public class Parent { > @PrimaryKey > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > private Key _ID; > > @Persistent > private String _Name; > > @Persistent(mappedBy = "_Parent") > @Element(dependent = "true") > private Set _Children = new HashSet(); > > public Parent(Key ID, String Name) > { > _ID = ID; > _Name = Name; > } > > //Getters removed > > } > > @PersistenceCapable(detachable = "true") > public class Child { > @PrimaryKey > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > private Key _ID; > > @Persistent > private Parent _Parent; > > @Persistent > private Key _ParentKey; > > @Persistent > private String _Value; > > public Child(Key ID, Parent parent, String value) > { > _ID = ID; > _ParentKey = parent.GetID(); > _Parent = parent; > } > > } > > TEST CODE: > private static PersistenceManagerFactory _pmf = null; > > public String greetServer(String input) throws > IllegalArgumentException { > if(_pmf == null) > _pmf = > JDOHelper.getPersistenceManagerFactory("transactions- > optional"); > > Key parentKey = > KeyFactory.createKey(Parent.class.getSimpleName(), > "abc"); //We only ever create one parent object and then retrieve it > ArrayList keyList = new ArrayList(); > keyList.add(parentKey); > > List p = FetchAndDetachParent(keyList); > if(p == null) > { > Parent pt = new Parent(parentKey, "hello"); > ArrayList list = new ArrayList(); > list.add(pt); > AddOrAttachParent(list); > } > else //If the parent has already been created, add children > to it > { > String childKeyString = UUID.randomUUID().toString(); > Parent pt = p.get(0); > Key childKey = KeyFactory.createKey(pt.GetID(), > Child.class.getSimpleName(), childKeyString); > Child ch = new Child(childKey, pt, "hello"); > pt.GetChildren().add(ch); > > ArrayList list = new ArrayList(); > list.add(pt); > AddOrAttachParent(list); > } > > return "all ok"; > } > > public List FetchAndDetachParent(List Keys) > { > PersistenceManager pm = _pmf.getPersistenceManager(); > pm.setDetachAllOnCommit(true); > pm.getFetchPlan().addGroup("children"); > > Transaction tx = pm.currentTransaction(); > > Query query = null; > try { > tx.begin(); > > query = pm.newQuery(Parent.class); > query.setFilter("_ID == :keyList"); > > List results = (List)query.execute(Keys); > > if(results.size() == 0) > { > tx.commit(); > return null; > } > else > { > tx.commit(); > return results; > } > > } finally { > query.closeAll(); > if (tx.isActive()) > { > tx.rollback(); > } > pm.close(); > } > } > > public void AddOrAttachParent(List parents) > { > PersistenceManager pm = _pmf.getPersistenceManager(); > > Transaction tx = pm.currentTransaction(); > try > { > tx.begin(); > //pm.makePersistent(parents.get(0)); > pm.makePersistentAll(parents); //If I comment out > this line and &g
[appengine-java] Re: Possible bug in pm.makePersistentAll? Random-looking exceptions, can't make heads or tails of it...
MODEL OBJECTS: @PersistenceCapable(detachable = "true") @FetchGroup(name = "children", members = { @Persistent(name = "_Children") }) public class Parent { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key _ID; @Persistent private String _Name; @Persistent(mappedBy = "_Parent") @Element(dependent = "true") private Set _Children = new HashSet(); public Parent(Key ID, String Name) { _ID = ID; _Name = Name; } //Getters removed } @PersistenceCapable(detachable = "true") public class Child { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key _ID; @Persistent private Parent _Parent; @Persistent private Key _ParentKey; @Persistent private String _Value; public Child(Key ID, Parent parent, String value) { _ID = ID; _ParentKey = parent.GetID(); _Parent = parent; } } TEST CODE: private static PersistenceManagerFactory _pmf = null; public String greetServer(String input) throws IllegalArgumentException { if(_pmf == null) _pmf = JDOHelper.getPersistenceManagerFactory("transactions- optional"); Key parentKey = KeyFactory.createKey(Parent.class.getSimpleName(), "abc"); //We only ever create one parent object and then retrieve it ArrayList keyList = new ArrayList(); keyList.add(parentKey); List p = FetchAndDetachParent(keyList); if(p == null) { Parent pt = new Parent(parentKey, "hello"); ArrayList list = new ArrayList(); list.add(pt); AddOrAttachParent(list); } else //If the parent has already been created, add children to it { String childKeyString = UUID.randomUUID().toString(); Parent pt = p.get(0); Key childKey = KeyFactory.createKey(pt.GetID(), Child.class.getSimpleName(), childKeyString); Child ch = new Child(childKey, pt, "hello"); pt.GetChildren().add(ch); ArrayList list = new ArrayList(); list.add(pt); AddOrAttachParent(list); } return "all ok"; } public List FetchAndDetachParent(List Keys) { PersistenceManager pm = _pmf.getPersistenceManager(); pm.setDetachAllOnCommit(true); pm.getFetchPlan().addGroup("children"); Transaction tx = pm.currentTransaction(); Query query = null; try { tx.begin(); query = pm.newQuery(Parent.class); query.setFilter("_ID == :keyList"); List results = (List)query.execute(Keys); if(results.size() == 0) { tx.commit(); return null; } else { tx.commit(); return results; } } finally { query.closeAll(); if (tx.isActive()) { tx.rollback(); } pm.close(); } } public void AddOrAttachParent(List parents) { PersistenceManager pm = _pmf.getPersistenceManager(); Transaction tx = pm.currentTransaction(); try { tx.begin(); //pm.makePersistent(parents.get(0)); pm.makePersistentAll(parents); //If I comment out this line and uncomment the above one, everything is fine. If not, then sometimes this works, and sometimes it throws. tx.commit(); } finally { if (tx.isActive()) { tx.rollback(); } pm.close(); } } The exception is as follows: SEVERE: [1269368147119000] javax.servlet.ServletContext log: Exception while dispatching incoming RPC call com.google.gwt.user.server.rpc.UnexpectedException: Service method 'public abstract java.lang.String com.mike.test.client.GreetingService.greetServer(java.lang.String) throws java.lang.IllegalArgumentException' threw an unexpected exception: javax.jdo.JDOUserException: One or more instances could not be made persistent NestedThrowables: java.lang.NullPointerException at com
[appengine-java] Possible bug in pm.makePersistentAll? Random-looking exceptions, can't make heads or tails of it...
I've been scratching my head over this one for a while... I have a parent object that participates in an owned one-to-many relationship with a child object. After the parent object is created, it is retrieved with its collection of children in the selected fetch group, and then detached. Then, I add a child to the parent's collection of children, and attach it. If I persist using pm.makePersistent(parent), everything works fine. If I persist using pm.makePersistentAll(listOfParent) then sometimes it works, and sometimes it throws: Caused by: java.lang.NullPointerException at org.datanucleus.sco.SCOUtils.attachAddNewElements(SCOUtils.java: 743) I am attaching the code in the next post. -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Is it possible to add a member to a detached object's collection that wasn't itself detached?
In case my subject line was confusing, I'll elaborate: Let's say I have a parent object that has a collection of children. I detach that parent object and intentionally omit the collection of children from the fetch group, in order to avoid fetching the potentially numerous children. Then, I want to add a new child to that parent's collection. I thought I would be able to add that child to the parent's empty un- detached collection, and thus have a child-collection size of 1, and then have that child be added to the parent's collection of children when the parent is attached. This code gave me an "illegal access of field that wasn't detached, if you want to access this field please detach it" exception. The reason that I thought this was possible was the following lines from the JDO docs: "If you don't detach a field of an object, you cannot access the value for that field while the object is detached. If you don't detach a field of an object, you can update the value for that field while detached, and thereafter you can access the value for that field." (http://www.jpox.org/docs/1_2/jdo/attach_detach.html) The second line pertains to what I'm doing (updatingthe value for the field while detached), but I guess in so doing I am also trying to "access the value" for the field. In short, is what I'm trying to do possible? Cheers, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to query for parent key property of an entity's key?
Ugorji, This sounds good but I haven't found a way to accomplish it in JDO. The closest is, I think, is described here: http://code.google.com/appengine/docs/java/datastore/transactions.html#Using_Transactions "To create an object with a system-generated numeric ID and an entity group parent, you must use an entity group parent key field (such as customerKey, above). Assign the key of the parent to the parent key field, then leave the object's key field set to null. When the object is saved, the datastore populates the key field with the complete key, including the entity group parent." Which doesn't work for me because I have to explicitly create the key of the child. Tristan, I think your approach is the correct one. The single-entity-group write limit is something new to me, new and worrisome. =/ I need the entity groups that I have for transactionality and faster read speeds, but the write-limit is something I will have to deal with. Thanks for the advice! On Mar 18, 4:28 pm, Ugorji wrote: > Use an ancestor-is query. This will find all entities which have a > given ancestor as parent. It even works in a kind-less query, > returning different kinds of entities under that parent. It's also > quite efficient (especially from a pricing/CPU cost perspective) since > theoretically, all this can be done on a single database node. > > I'm not sure how to do ancestor-is query with JDO (I use my own type- > safe wrapper over low-level API), but you could try something like: > > select from ... where ANCESTOR=... (like __key__, ANCESTOR seems to be > a reserved name in app engine) > > On Mar 18, 7:56 am, tempy wrote: > > > > > That's a good point, but in this case the resulting entity group > > arrangement is intentional. Lock-in isn't a worry, I can confidently > > say that these entity-group arrangements will last as long as the > > application does. > > > Though, as you point out, I can make a "parentKey" field on the child, > > but since this parent key will also have to appear within the child's > > key, having such a field is redundant and I would rather avoid it. > > > On Mar 18, 3:51 pm, Tristan wrote: > > > > Something you may want to consider is that you are placing yourself > > > within the limitations of entity groups by sticking children under > > > parent keys. You may want to consider a model where the parent key is > > > simply a field in the child and then run a simple query testing that > > > the "parentKey" field is equal to the one you're looking for. This > > > makes queries easier and prevents entity group lock-in. > > > > Cheers! > > > > On Mar 18, 8:55 am, tempy wrote: > > > > > I want to retrieve all entities that are children of one other > > > > particular entity, by checking if the parent-key property of the > > > > child's key matches the parent key. I have a reference to the parent > > > > entity but I want to avoid loading all of its children (as there may > > > > be many children, but I only need a few). Thus I want a query that > > > > looks something like this: > > > > > query = pm.newQuery("select from " + ChildClass.class.getName() > > > > + > > > > " where :parentID.contains(ChildIDProperty.ParentID) && > > > > SomeOtherProperty > " + filterString); > > > > > But I'm not sure how to exactly address the parent key property of a > > > > key in a query. > > > > > Thanks, > > > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How to query for parent key property of an entity's key?
That's a good point, but in this case the resulting entity group arrangement is intentional. Lock-in isn't a worry, I can confidently say that these entity-group arrangements will last as long as the application does. Though, as you point out, I can make a "parentKey" field on the child, but since this parent key will also have to appear within the child's key, having such a field is redundant and I would rather avoid it. On Mar 18, 3:51 pm, Tristan wrote: > Something you may want to consider is that you are placing yourself > within the limitations of entity groups by sticking children under > parent keys. You may want to consider a model where the parent key is > simply a field in the child and then run a simple query testing that > the "parentKey" field is equal to the one you're looking for. This > makes queries easier and prevents entity group lock-in. > > Cheers! > > On Mar 18, 8:55 am, tempy wrote: > > > > > I want to retrieve all entities that are children of one other > > particular entity, by checking if the parent-key property of the > > child's key matches the parent key. I have a reference to the parent > > entity but I want to avoid loading all of its children (as there may > > be many children, but I only need a few). Thus I want a query that > > looks something like this: > > > query = pm.newQuery("select from " + ChildClass.class.getName() + > > " where :parentID.contains(ChildIDProperty.ParentID) && > > SomeOtherProperty > " + filterString); > > > But I'm not sure how to exactly address the parent key property of a > > key in a query. > > > Thanks, > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] How to query for parent key property of an entity's key?
I want to retrieve all entities that are children of one other particular entity, by checking if the parent-key property of the child's key matches the parent key. I have a reference to the parent entity but I want to avoid loading all of its children (as there may be many children, but I only need a few). Thus I want a query that looks something like this: query = pm.newQuery("select from " + ChildClass.class.getName() + " where :parentID.contains(ChildIDProperty.ParentID) && SomeOtherProperty > " + filterString); But I'm not sure how to exactly address the parent key property of a key in a query. Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Do I need unowned relationships to accomplish this...?
Exactly, CardList is potentially very large and I want to avoid having to load it just to add the cardreference. On Mar 10, 10:43 pm, WillSpecht wrote: > So if you have a reference to a new card and do > > cardList.add(cardRefference) > > all you are loading into memory is the card list and the new card. > > Is this what you are trying to avoid? > > On Mar 10, 4:06 pm, tempy wrote: > > > > > Actually cards can only be owned by one deck... so that's not a > > problem. Deck<--1...0toN-->card. > > > The thing that I am looking for is a way to add new cards without > > loading a deck's entire card collection, and to add decks without > > loading a User's entire deck collection. > > > On Mar 10, 9:15 pm, WillSpecht wrote: > > > > The way I understand it, if an object can be owned by more than one > > > object it must be unowned. I would assume that cards can be in > > > multiple decks so they must be unowned. I would assume each deck > > > would belong to one user so decks could be owned. I don't know a good > > > way to store cards that can be queried in one query unless you have > > > each card store what decks they are in. This could be even more > > > difficult if cards appear more than once in a deck. If that is true I > > > would suggest a join table. > > > > On Mar 10, 2:20 pm, tempy wrote: > > > > > I have the following datastructure: > > > > > "Users" are the root entities, and each "user" can have one or more > > > > "decks", and each deck can have one or more "cards." > > > > > When a user wants to add a deck, I would like to be able to add the > > > > deck to the user's collection of decks without first fetching all of > > > > the user's decks (potentially a large amount of data), then adding the > > > > new deck to that collection, and then persisting the user. Rather, I > > > > would like to simply instantiate the deck and append it to the user's > > > > collection of decks, without ever retrieving the entire collection. > > > > > Similarly, if a user wants to add a new card to an existing deck, I > > > > would like to add the card to the deck without first retrieving the > > > > entire deck (that is, the deck with all of its cards). > > > > > I would like to preserve the option of fetching a user with a > > > > populated collection of all their decks and to retrieve a deck with a > > > > populated collection of all its cards, which is possible with owned > > > > relationships. But to accomplish what I have mentioned above, would I > > > > be forced to use unowned relationships? (Collections of keys instead > > > > of collections of objects.) > > > > > Thanks, > > > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Do I need unowned relationships to accomplish this...?
Actually cards can only be owned by one deck... so that's not a problem. Deck<--1...0toN-->card. The thing that I am looking for is a way to add new cards without loading a deck's entire card collection, and to add decks without loading a User's entire deck collection. On Mar 10, 9:15 pm, WillSpecht wrote: > The way I understand it, if an object can be owned by more than one > object it must be unowned. I would assume that cards can be in > multiple decks so they must be unowned. I would assume each deck > would belong to one user so decks could be owned. I don't know a good > way to store cards that can be queried in one query unless you have > each card store what decks they are in. This could be even more > difficult if cards appear more than once in a deck. If that is true I > would suggest a join table. > > On Mar 10, 2:20 pm, tempy wrote: > > > > > I have the following datastructure: > > > "Users" are the root entities, and each "user" can have one or more > > "decks", and each deck can have one or more "cards." > > > When a user wants to add a deck, I would like to be able to add the > > deck to the user's collection of decks without first fetching all of > > the user's decks (potentially a large amount of data), then adding the > > new deck to that collection, and then persisting the user. Rather, I > > would like to simply instantiate the deck and append it to the user's > > collection of decks, without ever retrieving the entire collection. > > > Similarly, if a user wants to add a new card to an existing deck, I > > would like to add the card to the deck without first retrieving the > > entire deck (that is, the deck with all of its cards). > > > I would like to preserve the option of fetching a user with a > > populated collection of all their decks and to retrieve a deck with a > > populated collection of all its cards, which is possible with owned > > relationships. But to accomplish what I have mentioned above, would I > > be forced to use unowned relationships? (Collections of keys instead > > of collections of objects.) > > > Thanks, > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Do I need unowned relationships to accomplish this...?
I have the following datastructure: "Users" are the root entities, and each "user" can have one or more "decks", and each deck can have one or more "cards." When a user wants to add a deck, I would like to be able to add the deck to the user's collection of decks without first fetching all of the user's decks (potentially a large amount of data), then adding the new deck to that collection, and then persisting the user. Rather, I would like to simply instantiate the deck and append it to the user's collection of decks, without ever retrieving the entire collection. Similarly, if a user wants to add a new card to an existing deck, I would like to add the card to the deck without first retrieving the entire deck (that is, the deck with all of its cards). I would like to preserve the option of fetching a user with a populated collection of all their decks and to retrieve a deck with a populated collection of all its cards, which is possible with owned relationships. But to accomplish what I have mentioned above, would I be forced to use unowned relationships? (Collections of keys instead of collections of objects.) Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Any plans for an apps marketplace for consumers, not business?
My question says it all. I would love to leverage the centralized structure of the marketplace and the enhanced user trust that it brings with my app, but what I am working on is targeted at general consumers, not the enterprise sector... Why not make something like the marketplace for the general consumer? -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Failing to properly detach children in owned one-to-many relationship
Finally fixed after a tip from DataNucleus over on stackoverflow... pm.getFetchPlan().setGroup("decks"); pm.getFetchPlan().setGroup("cards"); should actually be pm.getFetchPlan().addGroup("decks"); pm.getFetchPlan().addGroup("cards"); The setgroup methods overwrite whatever groups are already there, and the addgroup methods just add them. Lesson learned the hard way. On Mar 9, 12:07 am, tempy wrote: > All sorts of permutations have still failed to do this for me. From > searching this group and stackoverflow, I'm starting to wonder if many > people actually use JDO attach/detach in any significant way. > Anyone?? > > On Mar 8, 12:29 am, tempy wrote: > > > > > I am trying to load the full object graph for User, which contains a > > collection of decks, which then contains a collection of cards, as > > such: > > > User: > > @PersistenceCapable(detachable = "true") > > @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) > > @FetchGroup(name = "decks", members = { @Persistent(name = > > "_Decks") }) > > public abstract class User { > > @PrimaryKey > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > protected Key _ID; > > @Persistent > > protected String _UniqueIdentifier; > > @Persistent(mappedBy = "_Owner") > > @Element(dependent = "true") > > protected Set _Decks; > > protected User() > > { > > } > > > } > > > Each Deck has a collection of Cards, as such: > > @PersistenceCapable(detachable = "true") > > @FetchGroup(name = "cards", members = { @Persistent(name = > > "_Cards") }) > > public class Deck { > > @PrimaryKey > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > private Key _ID; > > @Persistent > > String _Name; > > @Persistent(mappedBy = "_Parent") > > @Element(dependent = "true") > > private Set _Cards = new HashSet(); > > @Persistent > > private Set _Tags = new HashSet(); > > @Persistent > > private User _Owner; > > > } > > > And finally, each card: > > @PersistenceCapable > > public class Card { > > @PrimaryKey > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > private Key _ID; > > �...@persistent > > private Text _Question; > > @Persistent > > private Text _Answer; > > @Persistent > > private Deck _Parent; > > > } > > > I am trying to retrieve and then detach the entire object graph. I > > can see in the debugger that it loads fine, but then when I get to > > detaching, I can't make anything beyond the User object load. (No > > Decks, no Cards). At first I tried without a transaction to simply > > "touch" all the fields on the attached object before detaching, but > > that didn't help. Then I tried adding everything to the default fetch > > group, but that just generated warnings about GAE not supporting > > joins. I tried setting the fetch plan's max fetch depth to -1, but > > that didn't do it. Finally, I tried using FetchGroups as you can see > > above, and then retrieving with the following code: > > > PersistenceManager pm = _pmf.getPersistenceManager(); > > pm.setDetachAllOnCommit(true); > > > pm.getFetchPlan().setGroup("decks"); > > pm.getFetchPlan().setGroup("cards"); > > > Transaction tx = pm.currentTransaction(); > > > Query query = null; > > try { > > tx.begin(); > > > query = pm.newQuery(GoogleAccountsUser.class); > > //Subclass of User > > query.setFilter("_UniqueIdentifier == TheUser"); > > query.declareParameters("String TheUser"); > > > List results = (List)query.execute(ID); > > //ID = Supplied > > parameter > > > //TODO: Test for more than one result and throw > > if(results.size() == 0) > > { > > tx.commit(); > > return null; > > } > > els
[appengine-java] Re: Failing to properly detach children in owned one-to-many relationship
All sorts of permutations have still failed to do this for me. From searching this group and stackoverflow, I'm starting to wonder if many people actually use JDO attach/detach in any significant way. Anyone?? On Mar 8, 12:29 am, tempy wrote: > I am trying to load the full object graph for User, which contains a > collection of decks, which then contains a collection of cards, as > such: > > User: > @PersistenceCapable(detachable = "true") > @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) > @FetchGroup(name = "decks", members = { @Persistent(name = > "_Decks") }) > public abstract class User { > @PrimaryKey > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > protected Key _ID; > @Persistent > protected String _UniqueIdentifier; > @Persistent(mappedBy = "_Owner") > @Element(dependent = "true") > protected Set _Decks; > protected User() > { > } > > } > > Each Deck has a collection of Cards, as such: > @PersistenceCapable(detachable = "true") > @FetchGroup(name = "cards", members = { @Persistent(name = > "_Cards") }) > public class Deck { > @PrimaryKey > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > private Key _ID; > @Persistent > String _Name; > @Persistent(mappedBy = "_Parent") > @Element(dependent = "true") > private Set _Cards = new HashSet(); > @Persistent > private Set _Tags = new HashSet(); > @Persistent > private User _Owner; > > } > > And finally, each card: > @PersistenceCapable > public class Card { > @PrimaryKey > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > private Key _ID; > �...@persistent > private Text _Question; > @Persistent > private Text _Answer; > @Persistent > private Deck _Parent; > > } > > I am trying to retrieve and then detach the entire object graph. I > can see in the debugger that it loads fine, but then when I get to > detaching, I can't make anything beyond the User object load. (No > Decks, no Cards). At first I tried without a transaction to simply > "touch" all the fields on the attached object before detaching, but > that didn't help. Then I tried adding everything to the default fetch > group, but that just generated warnings about GAE not supporting > joins. I tried setting the fetch plan's max fetch depth to -1, but > that didn't do it. Finally, I tried using FetchGroups as you can see > above, and then retrieving with the following code: > > PersistenceManager pm = _pmf.getPersistenceManager(); > pm.setDetachAllOnCommit(true); > > pm.getFetchPlan().setGroup("decks"); > pm.getFetchPlan().setGroup("cards"); > > Transaction tx = pm.currentTransaction(); > > Query query = null; > try { > tx.begin(); > > query = pm.newQuery(GoogleAccountsUser.class); > //Subclass of User > query.setFilter("_UniqueIdentifier == TheUser"); > query.declareParameters("String TheUser"); > > List results = (List)query.execute(ID); > //ID = Supplied > parameter > > //TODO: Test for more than one result and throw > if(results.size() == 0) > { > tx.commit(); > return null; > } > else > { > User usr = (User)results.get(0); > > //usr = pm.detachCopy(usr); > tx.commit(); > return usr; > } > > } finally { > query.closeAll(); > if (tx.isActive()) > { > tx.rollback(); > } > pm.close(); > } > > This also doesn't work, and I'm running out of ideas. I'm new to GAE/ > JDO, so go easy please. =) > > Thanks, > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Failing to properly detach children in owned one-to-many relationship
I am trying to load the full object graph for User, which contains a collection of decks, which then contains a collection of cards, as such: User: @PersistenceCapable(detachable = "true") @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) @FetchGroup(name = "decks", members = { @Persistent(name = "_Decks") }) public abstract class User { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) protected Key _ID; @Persistent protected String _UniqueIdentifier; @Persistent(mappedBy = "_Owner") @Element(dependent = "true") protected Set _Decks; protected User() { } } Each Deck has a collection of Cards, as such: @PersistenceCapable(detachable = "true") @FetchGroup(name = "cards", members = { @Persistent(name = "_Cards") }) public class Deck { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key _ID; @Persistent String _Name; @Persistent(mappedBy = "_Parent") @Element(dependent = "true") private Set _Cards = new HashSet(); @Persistent private Set _Tags = new HashSet(); @Persistent private User _Owner; } And finally, each card: @PersistenceCapable public class Card { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key _ID; @Persistent private Text _Question; @Persistent private Text _Answer; @Persistent private Deck _Parent; } I am trying to retrieve and then detach the entire object graph. I can see in the debugger that it loads fine, but then when I get to detaching, I can't make anything beyond the User object load. (No Decks, no Cards). At first I tried without a transaction to simply "touch" all the fields on the attached object before detaching, but that didn't help. Then I tried adding everything to the default fetch group, but that just generated warnings about GAE not supporting joins. I tried setting the fetch plan's max fetch depth to -1, but that didn't do it. Finally, I tried using FetchGroups as you can see above, and then retrieving with the following code: PersistenceManager pm = _pmf.getPersistenceManager(); pm.setDetachAllOnCommit(true); pm.getFetchPlan().setGroup("decks"); pm.getFetchPlan().setGroup("cards"); Transaction tx = pm.currentTransaction(); Query query = null; try { tx.begin(); query = pm.newQuery(GoogleAccountsUser.class); //Subclass of User query.setFilter("_UniqueIdentifier == TheUser"); query.declareParameters("String TheUser"); List results = (List)query.execute(ID); //ID = Supplied parameter //TODO: Test for more than one result and throw if(results.size() == 0) { tx.commit(); return null; } else { User usr = (User)results.get(0); //usr = pm.detachCopy(usr); tx.commit(); return usr; } } finally { query.closeAll(); if (tx.isActive()) { tx.rollback(); } pm.close(); } This also doesn't work, and I'm running out of ideas. I'm new to GAE/ JDO, so go easy please. =) Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: What are the rules for object persistence between requests
Well I didn't think that GAE would start up a new JVM per request, but I assumed the GC would clean up any data created between requests, as there seems to be nothing left holding references to said data. One of these days I will read up on the details of java garbage collection... On Mar 7, 9:01 pm, Jeff Schnitzer wrote: > On Sun, Mar 7, 2010 at 11:36 AM, tempy wrote: > > Interesting... now I need to go and make sure all my logic still works > > if singletons stick around between requests. GAE is full of > > surprises! Thanks for the tip =) > > It may be true that GAE is full of surprises, but this shouldn't have > been one of them :-) > > Your GAE instance is a (modified) copy of Jetty that starts up, serves > requests for a while, and maybe gets shut down if you don't have any > more traffic. You can't possibly think that GAE would start up a JVM > and Jetty for every single request! > > Jeff -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: What are the rules for object persistence between requests
Interesting... now I need to go and make sure all my logic still works if singletons stick around between requests. GAE is full of surprises! Thanks for the tip =) On Mar 7, 7:22 pm, John Patterson wrote: > Hi Mike, static variables are a useful way to cache short term data. > They will last as long as the instance lasts which is hard to > predict. I read some time ago that there is an undocumented limit of > about 100MB of memory per instance. > > On 8 Mar 2010, at 01:02, tempy wrote: > > > > > On the dev server, I have a singleton DAO that encapsulates all logic > > relating to the datastore and memcache. I was surprised to find that > > the singleton persists between requests, and will generally go on > > existing until I restart the dev environment. Does this reflect > > production app engine behavior? If so, is there any documentation for > > when/how objects in app engine persist and/or are destroyed between > > requests? > > > Thanks, > > Mike > > > -- > > You received this message because you are subscribed to the Google > > Groups "Google App Engine for Java" group. > > To post to this group, send email to google-appengine-java@googlegroups.com > > . > > To unsubscribe from this group, send email to > > google-appengine-java+unsubscr...@googlegroups.com > > . > > For more options, visit this group > > athttp://groups.google.com/group/google-appengine-java?hl=en > > . -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] What are the rules for object persistence between requests
On the dev server, I have a singleton DAO that encapsulates all logic relating to the datastore and memcache. I was surprised to find that the singleton persists between requests, and will generally go on existing until I restart the dev environment. Does this reflect production app engine behavior? If so, is there any documentation for when/how objects in app engine persist and/or are destroyed between requests? Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Children in an owned one-to-many relationship not being retrieved, what am I doing wrong?
Solved the issue finally... Turned out that I was forgetting to set the deck's key's parent property to "user" properly, and thus the deck/ card combination was forming one entity group and the user was another group. I still don't get why this had the odd effect of loading user object with an in-tact deck collection but then making the deck's cards unreachable, even upon being "touched", but either way I've fixed my problem. Thanks for the help! On Mar 7, 3:45 pm, tempy wrote: > Same thing, but... > > I tried wrapping both the initial makepersistent() call and then the > query in transactions, without changing anything else: > > I get this, which I hope is related to my original issue: > > javax.jdo.JDOFatalUserException: Illegal argument > NestedThrowables: > java.lang.IllegalArgumentException: can't operate on multiple entity > groups in a single transaction. found both Element { > type: "GoogleAccountsUser " //This is the specific subclass of User > id: 11} > > and Element { > type: "Deck" > name: "_8ad07638-b233-4fef-9a7f-1cfbad4eaab3" > > } > > Now why would it think that decks and users are in different entity > groups... > > On Mar 7, 3:05 pm, Toby wrote: > > > > > Okay, I see. In my case it is a List not a Set ... but I think that > > really makes no difference. > > What I think is strange is that you initialize the fields: > > @Persistent(defaultFetchGroup = "true", mappedBy = "_Parent") > > @Element(dependent = "true") > > private Set _Cards = new HashSet(); > > > maybe that causes the _Cards to be reinitialized when the Entity is > > deserialized. Can you try to remove that? > > > Cheers, > > Toby > > > On Mar 7, 2:19 pm, tempy wrote: > > > > Thanks for the suggestion... > > > > I did try that exact code and sad to say it didn't help, just returned > > > a 0-size collection. I'm still trying to figure this out... > > > > On Mar 7, 10:58 am, Toby wrote: > > > > > Hello Mike, > > > > > have you tried touching them before you close the query? > > > > I had similar problem and only solved it by calling the getter for the > > > > owned collection before detaching the results. > > > > > if (results.iterator().hasNext()) { > > > > for (Product fp : results) { > > > > //this is to force the collections to be > > > > detached > > > > fp.getItems(); > > > > } > > > > return pm.detachCopyAll(results); > > > > ... > > > > > I found no better way to do that. I actually started managing > > > > relationships by hand now because you have more control and more > > > > performance especially if you do not always need the child > > > > collections. > > > > > But maybe there is also a way to trigger that through an annotation. > > > > > Cheers, > > > > Toby > > > > > On Mar 7, 1:46 am, tempy wrote: > > > > > > I have the following 3-part owned relationship... > > > > > > Users, the root entity, have a collection of Decks, as such (I am not > > > > > including the specific subclass of User as it doesn't seem to be > > > > > relevant): > > > > > > @PersistenceCapable(identityType = IdentityType.APPLICATION) > > > > > @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) > > > > > public abstract class User { > > > > > @PrimaryKey > > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > > > > protected Key _ID; > > > > > > @Persistent > > > > > protected String _UniqueIdentifier; > > > > > > @Persistent(defaultFetchGroup = "true", mappedBy = "_Owner") > > > > > @Element(dependent = "true") > > > > > protected Set _Decks; > > > > > > protected KleioUser() > > > > > { > > > > > } > > > > > > } > > > > > > Each Deck has a collection of Cards, as such: > > > > > @PersistenceCapable > > > > > public class Deck { > > > > > @PrimaryKey > > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > > >
[appengine-java] Re: Children in an owned one-to-many relationship not being retrieved, what am I doing wrong?
Same thing, but... I tried wrapping both the initial makepersistent() call and then the query in transactions, without changing anything else: I get this, which I hope is related to my original issue: javax.jdo.JDOFatalUserException: Illegal argument NestedThrowables: java.lang.IllegalArgumentException: can't operate on multiple entity groups in a single transaction. found both Element { type: "GoogleAccountsUser " //This is the specific subclass of User id: 11 } and Element { type: "Deck" name: "_8ad07638-b233-4fef-9a7f-1cfbad4eaab3" } Now why would it think that decks and users are in different entity groups... On Mar 7, 3:05 pm, Toby wrote: > Okay, I see. In my case it is a List not a Set ... but I think that > really makes no difference. > What I think is strange is that you initialize the fields: > @Persistent(defaultFetchGroup = "true", mappedBy = "_Parent") > @Element(dependent = "true") > private Set _Cards = new HashSet(); > > maybe that causes the _Cards to be reinitialized when the Entity is > deserialized. Can you try to remove that? > > Cheers, > Toby > > On Mar 7, 2:19 pm, tempy wrote: > > > > > Thanks for the suggestion... > > > I did try that exact code and sad to say it didn't help, just returned > > a 0-size collection. I'm still trying to figure this out... > > > On Mar 7, 10:58 am, Toby wrote: > > > > Hello Mike, > > > > have you tried touching them before you close the query? > > > I had similar problem and only solved it by calling the getter for the > > > owned collection before detaching the results. > > > > if (results.iterator().hasNext()) { > > > for (Product fp : results) { > > > //this is to force the collections to be detached > > > fp.getItems(); > > > } > > > return pm.detachCopyAll(results); > > > ... > > > > I found no better way to do that. I actually started managing > > > relationships by hand now because you have more control and more > > > performance especially if you do not always need the child > > > collections. > > > > But maybe there is also a way to trigger that through an annotation. > > > > Cheers, > > > Toby > > > > On Mar 7, 1:46 am, tempy wrote: > > > > > I have the following 3-part owned relationship... > > > > > Users, the root entity, have a collection of Decks, as such (I am not > > > > including the specific subclass of User as it doesn't seem to be > > > > relevant): > > > > > @PersistenceCapable(identityType = IdentityType.APPLICATION) > > > > @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) > > > > public abstract class User { > > > > @PrimaryKey > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > > > protected Key _ID; > > > > > @Persistent > > > > protected String _UniqueIdentifier; > > > > > @Persistent(defaultFetchGroup = "true", mappedBy = "_Owner") > > > > @Element(dependent = "true") > > > > protected Set _Decks; > > > > > protected KleioUser() > > > > { > > > > } > > > > > } > > > > > Each Deck has a collection of Cards, as such: > > > > @PersistenceCapable > > > > public class Deck { > > > > @PrimaryKey > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > > > private Key _ID; > > > > > @Persistent > > > > String _Name; > > > > > @Persistent(defaultFetchGroup = "true", mappedBy = "_Parent") > > > > @Element(dependent = "true") > > > > private Set _Cards = new HashSet(); > > > > > @Persistent > > > > private Set _Tags = new HashSet(); > > > > > @Persistent > > > > private KleioUser _Owner; > > > > > } > > > > > And finally, each card: > > > > > @PersistenceCapable > > > > public class Card { > > > > @PrimaryKey > > > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > > > private Key _ID; > > > > > �...@persistent > > > > private Text _Questio
[appengine-java] Re: Children in an owned one-to-many relationship not being retrieved, what am I doing wrong?
Thanks for the suggestion... I did try that exact code and sad to say it didn't help, just returned a 0-size collection. I'm still trying to figure this out... On Mar 7, 10:58 am, Toby wrote: > Hello Mike, > > have you tried touching them before you close the query? > I had similar problem and only solved it by calling the getter for the > owned collection before detaching the results. > > if (results.iterator().hasNext()) { > for (Product fp : results) { > //this is to force the collections to be detached > fp.getItems(); > } > return pm.detachCopyAll(results); > ... > > I found no better way to do that. I actually started managing > relationships by hand now because you have more control and more > performance especially if you do not always need the child > collections. > > But maybe there is also a way to trigger that through an annotation. > > Cheers, > Toby > > On Mar 7, 1:46 am, tempy wrote: > > > > > I have the following 3-part owned relationship... > > > Users, the root entity, have a collection of Decks, as such (I am not > > including the specific subclass of User as it doesn't seem to be > > relevant): > > > @PersistenceCapable(identityType = IdentityType.APPLICATION) > > @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) > > public abstract class User { > > @PrimaryKey > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > protected Key _ID; > > > @Persistent > > protected String _UniqueIdentifier; > > > @Persistent(defaultFetchGroup = "true", mappedBy = "_Owner") > > @Element(dependent = "true") > > protected Set _Decks; > > > protected KleioUser() > > { > > } > > > } > > > Each Deck has a collection of Cards, as such: > > @PersistenceCapable > > public class Deck { > > @PrimaryKey > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > private Key _ID; > > > @Persistent > > String _Name; > > > @Persistent(defaultFetchGroup = "true", mappedBy = "_Parent") > > @Element(dependent = "true") > > private Set _Cards = new HashSet(); > > > @Persistent > > private Set _Tags = new HashSet(); > > > @Persistent > > private KleioUser _Owner; > > > } > > > And finally, each card: > > > @PersistenceCapable > > public class Card { > > @PrimaryKey > > @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) > > private Key _ID; > > > �...@persistent > > private Text _Question; > > @Persistent > > private Text _Answer; > > @Persistent > > private Deck _Parent; > > > } > > > I've only run this on the dev server. When I create a new user and > > populate the corresponding decks and corresponding cards and then call > > pm.makepersistent(user), everything looks fine and I can see all the > > user, deck, and card entities in the development datastore. However, > > when I try to retrieve a user with the following query: > > > Query query = _pm.newQuery(SpecificUser.class); > > query.setFilter("_UniqueIdentifier == TheUser"); > > query.declareParameters("String TheUser"); > > > try { > > List results = > > (List)query.execute(ID); > > > if(results.size() == 0) > > return null; > > else > > return results.get(0); > > > } finally { > > query.closeAll(); > > } > > > I get the user just fine, and all the user's corresponding decks. But > > the decks have no cards. "Touching" the deck's cards collection with > > a .size() method doesn't load the cards, just returns 0, nor does > > setting the cards to the default fetch group. I also tried to set the > > pm's fetchplan to -1, but that didn't have the desired effect either. > > > The weird thing is that if I persist the Deck without a parent user, > > then I will get the deck back, along with all its cards, with no > > problems. > > > I've tried a lot of other things too and I'm starting to run low on > > ideas (and high on frustration), so any help would be appreciated. > > > Thanks, > > Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Children in an owned one-to-many relationship not being retrieved, what am I doing wrong?
I have the following 3-part owned relationship... Users, the root entity, have a collection of Decks, as such (I am not including the specific subclass of User as it doesn't seem to be relevant): @PersistenceCapable(identityType = IdentityType.APPLICATION) @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class User { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) protected Key _ID; @Persistent protected String _UniqueIdentifier; @Persistent(defaultFetchGroup = "true", mappedBy = "_Owner") @Element(dependent = "true") protected Set _Decks; protected KleioUser() { } } Each Deck has a collection of Cards, as such: @PersistenceCapable public class Deck { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key _ID; @Persistent String _Name; @Persistent(defaultFetchGroup = "true", mappedBy = "_Parent") @Element(dependent = "true") private Set _Cards = new HashSet(); @Persistent private Set _Tags = new HashSet(); @Persistent private KleioUser _Owner; } And finally, each card: @PersistenceCapable public class Card { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key _ID; @Persistent private Text _Question; @Persistent private Text _Answer; @Persistent private Deck _Parent; } I've only run this on the dev server. When I create a new user and populate the corresponding decks and corresponding cards and then call pm.makepersistent(user), everything looks fine and I can see all the user, deck, and card entities in the development datastore. However, when I try to retrieve a user with the following query: Query query = _pm.newQuery(SpecificUser.class); query.setFilter("_UniqueIdentifier == TheUser"); query.declareParameters("String TheUser"); try { List results = (List)query.execute(ID); if(results.size() == 0) return null; else return results.get(0); } finally { query.closeAll(); } I get the user just fine, and all the user's corresponding decks. But the decks have no cards. "Touching" the deck's cards collection with a .size() method doesn't load the cards, just returns 0, nor does setting the cards to the default fetch group. I also tried to set the pm's fetchplan to -1, but that didn't have the desired effect either. The weird thing is that if I persist the Deck without a parent user, then I will get the deck back, along with all its cards, with no problems. I've tried a lot of other things too and I'm starting to run low on ideas (and high on frustration), so any help would be appreciated. Thanks, Mike -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.