[appengine-java] Re: Is it safe for one ancestor to have millions of children in one List object?
Forget about this way : even if this having millions of children entities for one ancestor is theoretically feasible, you'll hit a write performance issue when you generate those entities : only 1 write / second in one given entity group (ancestor). On 17 fév, 03:16, hendrix.jason hendrix.ja...@gmail.com wrote: If I have a @PersistenceCapable class that contains a List of another @PersistenceCapable class, and that List is millions of items in size, is that OK? Does that exceed a maximum for Java's List size? What is the best way for a parent to have millions of children and still have it easy to add new ones without loading tons of data in to memory? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: 1.6.1 Pre-release SDKs are live
No full text search in this release?? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Transactions on entities in different entity groups ?
SERIOUSLY??? (why don't you give any visibility on this key feature, like you did with FTS?) On Sep 28, 10:52 pm, Ikai Lan (Google) ika...@google.com wrote: You currently can't, but we plan on rolling out a feature that will do this in the next few releases. It's currently in trusted tester at the moment. -- Ikai Lan Developer Programs Engineer, Google App Engine plus.ikailan.com | twitter.com/ikai On Mon, Sep 26, 2011 at 10:59 PM, Santosh kumar kopp@gmail.com wrote: Hi, I want to know how to create/update the entities which are in different entity groups ? for ex: I want to update the *apple* and *carrot* entities in a transaction but those parents are not same. (Here i am in such a situation) Thanks in advance :) *S*antosh *K*umar *K* http://www.accountingguru.in/** -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Help: Memcache - not hitting at all
Speaking about memcache, could someone give a clear confirmation whether I should use import net.sf.jsr107cache.Cache or javax.cache.Cache. Documentation is still misguiding about this point. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Delete Lots Of Data
you MUST have a look at http://ikaisays.com/2010/07/09/using-the-java-mapper-framework-for-app-engine/ There's a special section at the end of the post delete all entities of a given kind. You can delete 35M entities in a few minutes using mapreduce. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: how to set text to the com.google.appengine.api.datastore.Text in app engine
Do not use the method Text.toString(), use Text.getValue() instead. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Blobstore and upload question
Check out http://ikaisays.com/2010/08/11/using-the-app-engine-mapper-for-bulk-data-import/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Development server deletes automatically-generated datastore indices
Yes, I have noticed the same behaviour. My workaround: once a index has been automatically generated, I copy its xml configuration from datastore-indexes-auto.xml to datastore- indexes.xml. Thus I consider datastore-indexes-auto.xml as automatic propositions. On Jun 30, 12:14 pm, Ian Marshall ianmarshall...@gmail.com wrote: I have my datastore-indexes.xml configuration file set as datastore-indexes autoGenerate=true /datastore-indexes I make successive runs of the development server. Each time before stopping it, I wait a couple of minutes for updates to be flushed to local_db.bin and datastore-indexes-auto.xml. I have found that successive runs of the development server can cause previously-existing automatically-generated datastore indices to be erased (even though the datastore data in local_db.bin is preserved). Has anyone else found this? Is this expected behaviour or a glitch? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Best practice for importing a lot of data in the datastore
Check ikai lan's blog about the bulk load mapper. Using a mapreduce job is the solution you are looking for. On Jun 22, 4:58 am, finder-auto_admin gontran.mag...@gmail.com wrote: Hello all, I just tried to import data in my datastore from a big csv file and like others, I got the Deadline exception because my request took more than 30 seconds. I know nobody would wait for 30s to get a page loaded but this load of data is triggered when I click on a load data on an admin part of my application (not the official app engine admin part, just a page I did). A solution for this is to divide this task into several tasks. However I don't really know how to do it. In my example it's called in a servlet right before including a jsp page. Should I end my servlet and call other instances with part of the job to do for each? How can this be done? Another solution I thought of is using cron job or task queues. Are task queues limited by the 30s limit too? Same question about the cron jobs. What is the best way to do this? Many thanks, -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: How do I migrate old data (change schema) in a deployed Java app?
Data schema change is quite easy if you use a MapReduce job. See http://ikaisays.com/2010/07/09/using-the-java-mapper-framework-for-app-engine/ You can still use your JDO instructions in a mapper, even if the mapper performance is not at its best if not using the low-level API. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/aWk_6i5dxm0J. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] What happens to a MapReduce job when switching version while it's running?
I have a mapper job, that takes several hours to complete. I switched from the latest version to an older one while it was running. It seems all new tasks after I switched have been completed using the old version mapper. For my knowledge: is that what I should expect ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To view this discussion on the web visit https://groups.google.com/d/msg/google-appengine-java/-/Mq49-4VgGykJ. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: appengine-mapreduce: Returning NOBODY because of SkipAdminCheck... ??
All mappers return this log message. Please note it's an INFO message, so you shouldn't worry about it. On Apr 20, 7:49 am, Santosh kumar kopp@gmail.com wrote: Hi, I have a mapper which is running fine without errors, but finally it displaying Returning NOBODY because of SkipAdminCheck... continuously. What could be the reason ?? Thanks in advance...** -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] [mapreduce] Global limitation on concurrent mapper jobs performance
We are testing the mass creation of a very large number of entities in the datastore (several billions). We use csv files (approx. 100 Mb each), uploaded into the blobstore, and run mapper jobs on them. Our goal : minimize the overall execution time (whatever the cost). There seems to be an overall performance limitation we can not overcome, even when playing with different parameters : . Set a high value to mapreduce.mapper.inputprocessingrate (for instance 1,000,000) . Set a high value to mapreduce.mapper.shardcount (for instance 20, or 50) . Launch concurrent mapper jobs in parallel (for instance 20 jobs, 1 job per file) The overall performance sticks around 500 entities/second. Is there a specific limitation related to the blobstore reads we should be aware of? Or has someone any tips about improving this perf? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: java.lang.NoSuchMethodError: javax.persistence.EntityManager.createNamedQuery
I guess you are using [eclipse + sts + google plugins]. This shit happens to me all the time. You JUST need to edit your persisting class, add a space somewhere, and have datanucleus rebuild your class persistence stuff. I have 20+ persisting classes, and I sometimes need to edit those 20+ files JUST to add a dummy space somewhere so that datanucleus rebuilds them. Pretty annoying... Here is my workaround .If someone has a definitive answer to this, please share... -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Multi value entity SetString
Add a @Basic annotation to your SetString property. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: for the problem Applications are limited to 150000000 bytes of resource files
If your app is gwt-based, you can reduce the size of your resources by reducing the number of permutations. Gwt compiles one specific final resource package for each permutation (e.g: 5 browsers x 3 locales 15 permutations). This might lead to very heavy resources. See either http://code.google.com/p/google-web-toolkit/wiki/SoftPermutations or http://anublog.colombounplug.com/2011/02/gwt-and-multiple-permutations-of-compilation/ for the howtos. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Re: Unable to update app (Error 400)
Facing the same problem (error 400). Maybe I did the same super stupid error as you did. What did you do ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Error 400 request is invalid for unspecified reasons during deployment.
Anyone facing deployment issues today ? After several attempts, I only got errors 400 request is invalid for unspecified reasons. Any clue ? -- error log: - com.google.appengine.tools.admin.AdminException: Unable to update app: Error posting to URL: https://appengine.google.com/api/appversion/deploy?app_id=hiddenBecauseIamTo oShyversion=1 400 Bad Request Client Error (400) The request is invalid for an unspecified reason. at com.google.appengine.tools.admin.AppAdminImpl.update(AppAdminImpl.java:62) at com.google.appengine.eclipse.core.proxy.AppEngineBridgeImpl.deploy(AppEngine BridgeImpl.java:271) at com.google.appengine.eclipse.core.deploy.DeployProjectJob.runInWorkspace(Dep loyProjectJob.java:146) at org.eclipse.core.internal.resources.InternalWorkspaceJob.run(InternalWorkspa ceJob.java:38) at org.eclipse.core.internal.jobs.Worker.run(Worker.java:54) Caused by: java.io.IOException: Error posting to URL: https://appengine.google.com/api/appversion/deploy?app_id=hiddenBecauseIamTo oShyversion=1 400 Bad Request -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Query with several filters on same list property potential exploding index problem...
I handle a class MyClass that holds a list property myListProperty. I use the low-level API to query over this class, with a dynamic number of filters over this very single field. Those filters mainly use different kind of operators (inequality filters and in filters). Sample code: Query q = new Query(MyClass); q.addFilter(myListProperty, FilterOperator.IN, oneCollection); q.addFilter(myListProperty, FilterOperator.IN, anotherCollection); q.addFilter(myListProperty, FilterOperator.GREATER_THAN, oneString); q.addFilter(myListProperty, FilterOperator.LESS_THAN, anotherString); When executing this query in dev mode, it generates the following index in datastore-indexes-auto.xml: !-- Used 15 times in query history -- datastore-index kind=MyClass ancestor=false source=auto property name=myListProperty direction=asc/ property name=myListProperty direction=asc/ property name=myListProperty direction=asc/ /datastore-index You got it : this would generate a mega-exploding index problem when in productionŠ Can someone confirm that this kind of index is NOT REQUIRED when executing that query on the actual datastore ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Lexicographic order of keys
Say you manage an entity kind with a com.google.appengine.api.datastore.Key key, and you manage those keys with explicit values like key = KeyFactory.createKey(MyClass.class.getSimpleName(), AAA). Then, does the string-encoded form of the key have the same lexicographic order than the explicit key name ? In other words: if I want to perform a range query based on the explicit name (i.e. get all entities with key name from AAA to ZZZ), can I use an double inequality filter on the entity keys? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Relationships between entities of of same kind
Some time ago, relationships between entities of same kind were not supported by the datastore. This was annoying in particular when trying to movel tree structures (@OneToMany relationship between MyClass and MyClass). The datastore documentation was pretty clear about this. I cannot find this limitation in the datastore documentation anymore. Have I missed a good news about a recent release allowing relationships inside entities of the same kind ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Spring Roo 1.1.1 with GAE : how to deal with parent/child relationships
Have some of you succeeded in bootstrapping a project with Spring Roo with use of ACTUAL parent/child relationships ? I mean with real Parent(X)/Child(Y) keys, NOT with the use of Long as keys in entitiesŠ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-java@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Scaffolding enhancing the List property feature
Guys, As you know, one of the most powerful features regarding data modelling in GAE is the ListProperty. (i.e. the ability to manage a ListString property and to manage indexes and queries over those GAE-specific fields) Spring Roo and MyEclipse propose interesting stuff to scaffold CRUD applications (based on JPA entities, beans, Š), but none of them support this GAE-specific feature : Collection fields are used only to support entity relationships, and thus (a) they support user defined entities, and (b) they forbid primary types. Let's simplify : with Roo or MyEclipse, you can scaffold a ListMyUserClass field, but not a ListString field. IMO, this ruins all efforts from Spring Roo or MyEclipse dev teams to attract GAE developpers. Have some of you any clue/tips/workaround to manage Collections of primary types during the scaffolding process with one of those tools ? Thanks in advance, cyrille -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Upload to blobstore without form
Has any of you succeeded in coding a servlet that uploads a file to the blobstore WITHOUT passing through a post jsp form ? (apologies, this might have been discussed a zillion time already in this group) Would be VERY helpful to me. Thanks in advance, cyrille -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Massive delete (mapreduce based) Datastore viewer Datastore statistics unavailable
My problem: 1. I execute a massive delete over a large number of entities (30 million), using a mapreduce-based job similar to Ikai's one (*) 2. Then both the datastore viewer and the datastore statistics in the dashboard become unavailable: I get this famous the server encountered an error (blahblah) page. Yet, servlet-based queries against the datastore are ok, and the datastore indexes viewer do not show any error. Any clue ? (*) http://ikaisays.com/2010/07/09/using-the-java-mapper-framework-for-app-engin e/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Mapper Blobstore bytes read limit
Nope, I don't think I have that kind of fields. In my parent entity, 2 properties are indexed : the primary key (programatically defined based on a String), plus another key (unowned relationship). In my child entity, 2 properties as well : the primary key (programatically defined as well), plus a ListProperty (strict application of Brett's recommandation) I just got lots of com.google.apphosting.api.DeadlineExceededException exceptions (approx. 30%), which I considered caused from the high number of shards causing high concurrency somewhere/somehow. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 24 Nov 2010 10:29:17 -0800 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit BTW - did you have any data that would have created a sequentially increasing index? For instance, a Created At timestamp? This sort of thing in a high write environment can cause issues: though tablets are essentially sharded, it means sequential index writes at a high speed, causing tablets to become hot and slow your overall throughput. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Wed, Nov 24, 2010 at 10:26 AM, Ikai Lan (Google) ikai.l+gro...@google.com mailto:ikai.l%2bgro...@google.com wrote: Mind blowing. Awesome! -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Wed, Nov 24, 2010 at 5:54 AM, Cyrille Vincey crll...@gmail.com wrote: It works, and the performance is breathtaking : 8.6 million entities (4.3 lines x 2 entities per line) created in 1.5h, using 100 shardsŠ Compared to my previous non-blob-based mapper job, CPU cost remains a little high (190 CPU hours), but I can't complain. Thank you guys. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 17 Nov 2010 16:06:07 -0800 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit The bug has been fixed. Check out the latest code from the appengine-mapreduce project. Note that the ratio between blobstore bytes read and blob size is not 1:1. In my tests they were closer to 10:1. This is expected behavior for the time being. We're working on more options so users can better tune the behavior. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Wed, Nov 17, 2010 at 2:19 AM, Cyrille Vincey crll...@gmail.com wrote: VERY good news. Can't wait. Thanks. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Tue, 16 Nov 2010 12:07:59 -0800 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit We discovered a bug. We're not reading in the entire blob, but we are reading in far too much data. Fred has a fix waiting in the rafters. I'll post again when it's been pushed. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Thu, Nov 4, 2010 at 2:36 AM, Cyrille Vincey crll...@gmail.com wrote: Not a lot of interesting stuff to say : 1. My code is quite as simple as your sample code: the only real difference is that I create 2 parent/child entities in a row for one given csv line entry. 2. My csv file contains 4.3 million lines. 2. I launched the mapper job with 10 shards. 3. worker-attempt-XXX tasks had 20 retries each in average. 4. The blobstore bytes read quota (100 Go) got reached within the first 3 hours. 5. Est. 10% of the entities where actually created after 24h (with my previous non-blob-based mapper job, those 4.3 million entities where created within 1 day) 6. Log does not reveal anything interesting. I am currently running a new test with a 500,000 lines csv file (20 Mb file). Performance looks better. To me, blob file size may have an influence on the mapper performance. If you need more details, let me know. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 3 Nov 2010 12:22:10 -0700 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit This behavior doesn't seem right. No, the entire blob should
Re: [appengine-java] Re: Auto generating datastore-indexes.xml during unit tests
Check your war WEB-INF datastore-indexes.xml (NOT the war WEB-INF appengine-generated datastore-indexes-auto.xml) You might read: datastore-indexes autoGenerate=false Try to replace with : datastore-indexes autoGenerate=true On 19/11/10 01:39, Jerry jerry.morri...@gmail.com wrote: Great tip but it's not working. Running the test updates the mod date of local_db.bin but not datastore-indexes-auto.xml. I'm also configuring a LocalUserServiceTestConfig for a logged in user, but that doesn't seem to affect it. Is another setup step needed? I'd like a unit test to trigger all auto-index because my app has many possible combinations of query filters. Thanks! On Nov 11, 11:24 am, Starman r...@arrova.ca wrote: Something like the following will setup the datastore helper and create the datastore-indexes-auto.xml: LocalDatastoreServiceTestConfig dsConfig = new LocalDatastoreServiceTestConfig(); File location = new File(war/WEB-INF/appengine-generated/ local_db.bin); dsConfig.setBackingStoreLocation(location.getAbsolutePath()); // And it seems that for the local ds service to be able to load the local_db.bin file, the local // environment must match the app id and version as stored in the file. String xml; try { xml = FileUtils.readFileToString(new File(war/WEB-INF/ appengine-web.xml)); } catch (IOException e) { throw new RuntimeException(e); } String application = StringUtils.substringBetween(xml, application, /application); String version = StringUtils.substringBetween(xml, version, /version); // Default behavior is to keep everything in RAM. Tell the local services to use the file system dsConfig.setNoStorage(false); // Create the local services. gaeDatastoreHelper = new LocalServiceTestHelper(dsConfig); gaeDatastoreHelper.setEnvAppId(application); gaeDatastoreHelper.setEnvVersionId(version); // Calling setUp() will start the local ds service and load the local_db.bin file. gaeDatastoreHelper.setUp(); This way, you can even write tests against a previously generated local_db.bin if you want. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Mapper Blobstore bytes read limit
VERY good news. Can't wait. Thanks. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Tue, 16 Nov 2010 12:07:59 -0800 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit We discovered a bug. We're not reading in the entire blob, but we are reading in far too much data. Fred has a fix waiting in the rafters. I'll post again when it's been pushed. -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Thu, Nov 4, 2010 at 2:36 AM, Cyrille Vincey crll...@gmail.com wrote: Not a lot of interesting stuff to say : 1. My code is quite as simple as your sample code: the only real difference is that I create 2 parent/child entities in a row for one given csv line entry. 2. My csv file contains 4.3 million lines. 2. I launched the mapper job with 10 shards. 3. worker-attempt-XXX tasks had 20 retries each in average. 4. The blobstore bytes read quota (100 Go) got reached within the first 3 hours. 5. Est. 10% of the entities where actually created after 24h (with my previous non-blob-based mapper job, those 4.3 million entities where created within 1 day) 6. Log does not reveal anything interesting. I am currently running a new test with a 500,000 lines csv file (20 Mb file). Performance looks better. To me, blob file size may have an influence on the mapper performance. If you need more details, let me know. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 3 Nov 2010 12:22:10 -0700 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit This behavior doesn't seem right. No, the entire blob should not be getting read. We'll look into this. Do you have any more details? Could tasks be getting retried? -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Tue, Nov 2, 2010 at 9:42 AM, Cyrille Vincey crll...@gmail.com wrote: I've been testing Ikai's bulkload mapper (see url below) with a pretty big csv file (200 Mb). It works great, and I encourage most of you to consider implementing this for entity uploads. Yet, I do face one last issue with an unexpected quota : blobstore bytes read. This quota cannot be tuned via the billing settings, and it's not clear whether it limits the speed of my process or not when it's reached. See ? Yep, it's a lot of bytes readŠ Could someone confirm that the blob csv file is *NOT* fully fetched each time the mapper iterates on a new line ? (ikai's post) http://ikaisays.com/2010/08/11/using-the-app-engine-mapper-for-bulk-data-impo rt/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google
Re: [appengine-java] Re: SAXParseException: Premature end of file on dev server
. remove it then okay. On Oct 21, 5:56 pm, Cyrille Vincey crll...@gmail.com wrote: Same issue for the past 3 days. Only occuring on the dev server. I have tried for hours to solve it. No clue... Fortunately, no impact on the production environment. On 21/10/10 11:50, Ian Marshall ianmarshall...@gmail.com wrote: Hello All, I am running my web application on the GAE/J 1.3.8 development server on my local machine. I have just got the following log entry whilst testing: 21-Oct-2010 09:13:06 com.google.apphosting.utils.jetty.JettyLogger warn WARNING: fa...@null line:1 col:1 : org.xml.sax.SAXParseException: Premature end of file. I did not notice any effect on my web app: I was reading data from my dev server's datastore but not writing any data to it (apart from perhaps some data of my session). Has anyone else got this recently? Am I doing anything wrong, or could this be just an occasional glitch of the dev server which I can just ignore? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Problem setting a property to appengine entity using setProperty() method
You should have a look on the supported types in the datastore : http://code.google.com/intl/fr-FR/appengine/docs/java/datastore/dataclasses .html#Core_Value_Types In your case, try the following: entityOfTypePeople.setProperty(name, people.getName()); entityOfTypePeople.setProperty(country, people.getCountry()); Or try a simplier solution when getting started with appengine: have a look on the JDO features rather than the low-level API. Good luck. On 09/11/10 07:19, Mayumi Liyanage mayumi.liyan...@coldwin.com wrote: Hi, I have an entity which has property Person person(user defined object) so I wanted to setProperty(person, new Person()) however got an IllegalArgumentException Person is not a supported property type. It is a simple class with field String name and String country which implements Serializable. class People{ Person person; } I am trying to: entityOfTypePeople.setProperty(person, new Person()); this is where I get IllegalArgumentException Person is not a supported property type. Thanks -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Datastore viewer unavailable
Has some of you already experienced permanent unavailability of the admin UI datastore viewer ? It's been 2 days I cannot access my data through the UI now... I have no quota limitation, nor activity load, and offline jobs execution (mapreduce mainly) is ok. Any clue ? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Mapper Blobstore bytes read limit
Not a lot of interesting stuff to say : 1. My code is quite as simple as your sample code: the only real difference is that I create 2 parent/child entities in a row for one given csv line entry. 2. My csv file contains 4.3 million lines. 2. I launched the mapper job with 10 shards. 3. worker-attempt-XXX tasks had 20 retries each in average. 4. The blobstore bytes read quota (100 Go) got reached within the first 3 hours. 5. Est. 10% of the entities where actually created after 24h (with my previous non-blob-based mapper job, those 4.3 million entities where created within 1 day) 6. Log does not reveal anything interesting. I am currently running a new test with a 500,000 lines csv file (20 Mb file). Performance looks better. To me, blob file size may have an influence on the mapper performance. If you need more details, let me know. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 3 Nov 2010 12:22:10 -0700 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapper Blobstore bytes read limit This behavior doesn't seem right. No, the entire blob should not be getting read. We'll look into this. Do you have any more details? Could tasks be getting retried? -- Ikai Lan Developer Programs Engineer, Google App Engine Blogger: http://googleappengine.blogspot.com http://googleappengine.blogspot.com/ Reddit: http://www.reddit.com/r/appengine Twitter: http://twitter.com/app_engine On Tue, Nov 2, 2010 at 9:42 AM, Cyrille Vincey crll...@gmail.com wrote: I've been testing Ikai's bulkload mapper (see url below) with a pretty big csv file (200 Mb). It works great, and I encourage most of you to consider implementing this for entity uploads. Yet, I do face one last issue with an unexpected quota : blobstore bytes read. This quota cannot be tuned via the billing settings, and it's not clear whether it limits the speed of my process or not when it's reached. See ? Yep, it's a lot of bytes readŠ Could someone confirm that the blob csv file is *NOT* fully fetched each time the mapper iterates on a new line ? (ikai's post) http://ikaisays.com/2010/08/11/using-the-app-engine-mapper-for-bulk-data-impor t/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. inline: Capture d¹écran 2010-11-02 à 17.17.25.png
Re: [appengine-java] Re: Query for a Long id using getObjectById JDO
Both following syntaxes are ok for JDO : Key key1 = KeyFactory.createKey(YourClass.class.getSimpleName(), 12345L); //using Long Key key2 = KeyFactory.createKey(YourClass.class.getSimpleName(), yourtextid); //using String It's up to you to define how you want your entities to be recorded : 1. Either you let the datastore generate a key for you when recording a new entity = its key will be based on a Long 2. Or you 'force' the key with your own stringID or your own longID From: giannisdag pascoua...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 3 Nov 2010 11:24:10 -0700 (PDT) To: google-appengine-java@googlegroups.com Subject: [appengine-java] Re: Query for a Long id using getObjectById JDO After doing some reading, I realized that in GAE datastore, an entity is stored using the key value and a key name or ID. The key name has to be a string and the ID is long. The key name can be created by the application and can be created like this as an example: Key key = KeyFactory.createKey(Employee.class.getSimpleName(), alfred.sm...@example.com); The ID can only be created by the datastore, so we have to leave it with a null value. Now, when we want to retrieve-fetch an object from datastore, we have two different situations. First, if we had created the entityid with key name, we can retrieve the object by recreating the key with the above syntax and then search for the object using Employee e = pm.getObjectById(Employee.class, k); But if we have used the numeric ID, we can' t look for it using the KeyFactory, as usually humans don' t search an object using numbers, but easily remembered strings. So we have to use a query. In my question, I have to use a query. If I am somewhere wrong, please inform me. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Mapper Blobstore bytes read limit
I've been testing Ikai's bulkload mapper (see url below) with a pretty big csv file (200 Mb). It works great, and I encourage most of you to consider implementing this for entity uploads. Yet, I do face one last issue with an unexpected quota : blobstore bytes read. This quota cannot be tuned via the billing settings, and it's not clear whether it limits the speed of my process or not when it's reached. See ? Yep, it's a lot of bytes readŠ Could someone confirm that the blob csv file is *NOT* fully fetched each time the mapper iterates on a new line ? (ikai's post) http://ikaisays.com/2010/08/11/using-the-app-engine-mapper-for-bulk-data-imp ort/ -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. inline: Capture d¹écran 2010-11-02 à 17.17.25.png
Re: [appengine-java] Occasional NoClassDefFoundError
I have seen the same error from times to times. If you're on eclipse, this error may disappear if you clean your project prior to deploying it. Eclipse Project Clean... On 01/11/10 14:15, gholler georgehol...@gmail.com wrote: In the last couple of days we've been seeing java.lang.NoClassDefFoundError exceptions in our log for our GAE Java app. The class in question is one of ours that works and has worked for months. What could this be? Is this caused by problems with a static server behind the scenes? Has anyone else seen this? Thanks, George -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] Mapreduce, mutationPool, and child entities
I am trying to figure out how to put 2 parent/child entities into the mutationPool, inside a MapReduce job. Here is my issue : in my understanding, as the mutationPool is asynchronous , I cannot expect to get the parent entity key back from the mutationPool.put() method. Or can I ? According to you, would the following code work or not ? [Š] //Creation of the parent entity Entity parent = new Entity(ParentClass); parent.setProperty(fieldA, valueA); mutationPool.put(parent); //Creation of the child entity Entity child = new Entity(ChildClass, parent.getKey()); //Q: Can I expect that the key is returned directly after the mutationPool.put() method ? child.setProperty(fieldX, valueX); mutationPool.put(child); [Š] From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Mon, 13 Sep 2010 14:15:30 -0700 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] query for null missing properties You can't. Empty properties are the equivalent of unindexed properties. You'll need to iterate over all the Entities in your datastore and update them. The Mapper API is a very good tool for this: http://code.google.com/p/appengine-mapreduce/ On Mon, Sep 13, 2010 at 12:20 PM, Benjamin bsaut...@gmail.com wrote: Brain freeze guys - I added a property to my object and persisted in the datastore. The older object have now have a column that when viewed in the control panel have a value of missing. I need to query those objects so i can update them - propname == null , propname == - I can't seem to find the right syntax for this. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] org.datanucleus.store.appengine.query.RuntimeExceptionWrappingIterator.hasNext(RuntimeExceptionWrappingIterator.java:44)
You need to edit (or to create) a datastore-indexes.xml file in the WEB-INF directory. This file stores the index definition for your application. Personal advice : do not rely on the automatic index definition that you can see in the datastore-indexes-auto.xml (dir WEB-INF/appengine-generated/) In the datastore-indexes.xml file, add the definition for your index on your 3 properties exactly as specified in your error message : datastore-index kind=Bid ancestor=false source=manual property name=courseCode direction=asc/ property name=sectionCode direction=asc/ property name=amount direction=desc/ /datastore-index Then deploy again. On 31/10/10 16:21, mark chen markche...@gmail.com wrote: Hi, I am using JDO on GAE, and have been encountering this error. I googled the error, and most of the solutions were related to setting some properties in datastore-indexes.xml. However, I do not have this config file in my app as GAE has enables the Automatic index configuration. The piece of code which is giving me the error has been indicated, with the corresponding error message pasted below. Would appreciate any help. Thanks BidDAO.java //return a list of bids for a particular section public static List read(String courseCode, String sectionCode) { PersistenceManager pm = PMF.get().getPersistenceManager(); String query = select from + Bid.class.getName() + where courseCode == courseCodeParam sectionCode == sectionCodeParam + parameters String courseCodeParam, String sectionCodeParam order by amount desc; ListBid bids = (ListBid) pm.newQuery(query).execute(courseCode, sectionCode); ERROR--- ListBid detachedBids = (ListBid) pm.detachCopyAll(bids); pm.close(); if (detachedBids.size() == 0) { return null; } return detachedBids; } Uncaught exception from servlet com.google.appengine.api.datastore.DatastoreNeedIndexException: no matching index found.. datastore-index kind=Bid ancestor=false source=manual property name=courseCode direction=asc/ property name=sectionCode direction=asc/ property name=amount direction=desc/ /datastore-index at com.google.appengine.runtime.Request.process- f520ab98e5ed5cc2(Request.java) at com.google.appengine.api.datastore.DatastoreApiHelper.translateError(Datas toreApiHelper.java: 42) at com.google.appengine.api.datastore.DatastoreApiHelper $1.convertException(DatastoreApiHelper.java:98) at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java: 44) at com.google.appengine.api.utils.FutureWrapper.get(FutureWrapper.java: 42) at com.google.appengine.api.datastore.FutureHelper.getInternal(FutureHelper.j ava: 71) at com.google.appengine.api.datastore.FutureHelper.quietGet(FutureHelper.java : 32) at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities (QueryResultsSourceImpl.java: 69) at com.google.appengine.api.datastore.QueryResultsSourceImpl.loadMoreEntities (QueryResultsSourceImpl.java: 56) at com.google.appengine.api.datastore.QueryResultIteratorImpl.ensureLoaded(Qu eryResultIteratorImpl.java: 156) at com.google.appengine.api.datastore.QueryResultIteratorImpl.hasNext(QueryRe sultIteratorImpl.java: 65) at org.datanucleus.store.appengine.query.RuntimeExceptionWrappingIterator.has Next(RuntimeExceptionWrappingIterator.java: 44) at org.datanucleus.store.appengine.query.LazyResult.listIterator(LazyResult.j ava: 102) at org.datanucleus.store.appengine.query.StreamingQueryResult.iterator(Stream ingQueryResult.java: 114) at org.datanucleus.jdo.JDOPersistenceManager.detachCopyAll(JDOPersistenceMana ger.java: 1174) ---at com.bios.DAO.BidDAO.read(BidDAO.java:46) at com.bios.controller.AddBidServlet.validateBidStatus(AddBidServlet.java: 687) at org.apache.jsp.mainMenu_jsp._jspService(mainMenu_jsp.java:119) at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:94) at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java: 511) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1166) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler $CachedChain.doFilter(ServletHandler.java:1157) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java: 388) at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java: 216) at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java: 182) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java: 765) at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java: 418) at
Re: [appengine-java] Mapreduce, mutationPool, and child entities
Option 2 looks nice. Than you Ikai. From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Mon, 1 Nov 2010 15:28:54 -0700 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] Mapreduce, mutationPool, and child entities You've got a few options here: 1. Use named keys. You can then use KeyFactory and KeyFactory.Builder to create an entire Key hierarchy. 2. Use KeyRange to generate a range of IDs you can use, and use this as the key of the parent entity http://code.google.com/appengine/docs/java/javadoc/com/google/appengine/api/ datastore/KeyRange.html -- Ikai On Mon, Nov 1, 2010 at 8:11 AM, Cyrille Vincey crll...@gmail.com wrote: I am trying to figure out how to put 2 parent/child entities into the mutationPool, inside a MapReduce job. Here is my issue : in my understanding, as the mutationPool is asynchronous , I cannot expect to get the parent entity key back from the mutationPool.put() method. Or can I ? According to you, would the following code work or not ? [Š] //Creation of the parent entity Entity parent = new Entity(ParentClass); parent.setProperty(fieldA, valueA); mutationPool.put(parent); //Creation of the child entity Entity child = new Entity(ChildClass, parent.getKey()); //Q: Can I expect that the key is returned directly after the mutationPool.put() method ? child.setProperty(fieldX, valueX); mutationPool.put(child); [Š] From: Ikai Lan (Google) ikai.l+gro...@google.com Reply-To: google-appengine-java@googlegroups.com Date: Mon, 13 Sep 2010 14:15:30 -0700 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] query for null missing properties You can't. Empty properties are the equivalent of unindexed properties. You'll need to iterate over all the Entities in your datastore and update them. The Mapper API is a very good tool for this: http://code.google.com/p/appengine-mapreduce/ On Mon, Sep 13, 2010 at 12:20 PM, Benjamin bsaut...@gmail.com wrote: Brain freeze guys - I added a property to my object and persisted in the datastore. The older object have now have a column that when viewed in the control panel have a value of missing. I need to query those objects so i can update them - propname == null , propname == - I can't seem to find the right syntax for this. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Hierarchical Search
Can you please send the link to the video you refer to ? From: Ben Woodhead benwoodhead1...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Sat, 30 Oct 2010 22:56:12 -0700 (PDT) To: google-appengine-java@googlegroups.com Subject: [appengine-java] Hierarchical Search Hello Everybody, I am trying to build an efficient search like I saw in one of the google IO videos. So the goal is to use the hierarchy of the datastore to make the search more effecient. The datastore would look something like this: Database words Database words.articlesmap Database articles Each article is broken down into a set of words and the article map just holds the key to the article. The basic path Find the article i want and get its key. Use the key to search for all words.articlemap Get the parent of each articleamp. At this point I am completely lost. Heres what I have so far but I have no idea if I am on the right path and second how do I actually do the search Word.java [code] @PersistenceCapable public class Word { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key mKey; @Persistent String mWord; @Persistent ListArticleMap mArticleList; public Word(String word) { mWord = word; } public Key GetKey() { return mKey; } public void SetKey(Key key) { mKey = key; } public String GetWord() { return mWord; } public void AddArticleMap( ArticleMap article ) { if ( !mArticleList.contains(article) ) { mArticleList.add(article); } } public void RemoveArticleMap( ArticleMap article ) { if ( !mArticleList.contains(article) ) { mArticleList.remove(article); } } } [/code] ArticleMap.java [code] @PersistenceCapable public class ArticleMap { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key mKey; @Persistent private Key mArticleKey; public Key GetKey() { return mKey; } public void SetKey(Key key) { mKey = key; } public Key GetArticleKey() { return mArticleKey; } public void SetArticleKey(Key key) { mArticleKey = key; } } [/code] Article.java [code] @PersistenceCapable public class Article { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key mKey; @Persistent private Text mText; public Article( Text text ) { SetText(text); } public Key GetKey() { return mKey; } public void SetKey(Key key) { mKey = key; } public void SetText(Text text) { mText = text; } public Text GetText() { return mText; } } [/code] How do I tie the article map to the word itself, how do I tie the article to the article map. How do I search for based on article map and map that to article. Thank you for your time Ben -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Text enclosed with Text: ~~~
You should use the method Text.getValue() From: Michael Boswell mbo...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 27 Oct 2010 17:38:48 -0700 (PDT) To: google-appengine-java@googlegroups.com Subject: [appengine-java] Text enclosed with Text: ~~~ I have a com.google.appengine.api.datastore.Text object stored in the datastore, when examined through the datastore viewer, it reads as it should e.g. 'this is a test' I retrieve it with a public Text getData() { return data; } then write it out with getWriter, however I'm getting Text: this is a test instead of just 'this is a test' Sure I'm doing something daft! I'm rather new to java. Thanks in advance. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] mapreduce job is not aborted
Abort button doesn't work indeed. When I want to abort a mapreduce job, I directly mapreduce-related entities in the datastore viewer : MapReduceStat and ShardDontKnowWhat. From: aswath satrasala aswath.satras...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Thu, 28 Oct 2010 11:03:52 +0530 To: google-appengine-java@googlegroups.com Subject: [appengine-java] mapreduce job is not aborted Hello, On my app-id vs-accounting.appspot.com http://vs-accounting.appspot.com , I started a mapreduce job. It did not run, because the Entity did not had decreasing index. The status is now showing with the link abort http://abort After clicking on the abort, the job is not aborted. I look at the logs, and it is continuously performing something every minute. Can you help me in aborting the job. Regards Aswath http://vs-accounting.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] mapreduce job is not aborted
Š I directly DELETE Š (makes more sense with all the words) From: Cyrille Vincey crll...@gmail.com Date: Thu, 28 Oct 2010 09:56:51 +0200 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] mapreduce job is not aborted Abort button doesn't work indeed. When I want to abort a mapreduce job, I directly mapreduce-related entities in the datastore viewer : MapReduceStat and ShardDontKnowWhat. From: aswath satrasala aswath.satras...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Thu, 28 Oct 2010 11:03:52 +0530 To: google-appengine-java@googlegroups.com Subject: [appengine-java] mapreduce job is not aborted Hello, On my app-id vs-accounting.appspot.com http://vs-accounting.appspot.com , I started a mapreduce job. It did not run, because the Entity did not had decreasing index. The status is now showing with the link abort http://abort After clicking on the abort, the job is not aborted. I look at the logs, and it is continuously performing something every minute. Can you help me in aborting the job. Regards Aswath http://vs-accounting.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] mapreduce job is not aborted
Have you looked into your taskqueue if some tasks related to your mapreduce job are still queued ? From: aswath satrasala aswath.satras...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Thu, 28 Oct 2010 17:59:34 +0530 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] mapreduce job is not aborted Hello Cyrille, I am looking at the logs, and I am seeing some requests in the logs every 1 hour. mapreduce/mapperCallback Uncaught exception from servlet java.lang.RuntimeException: Couldn't find MR with job ID: job_1288164365012_0001 at com.google.appengine.tools.mapreduce.AppEngineJobContext.getConfigurationFro mRequest(AppEngineJobContext.java:157) at com.google.appengine.tools.mapreduce.AppEngineJobContext.init(AppEngineJob Context.java:110) Aswath On Thu, Oct 28, 2010 at 3:07 PM, aswath satrasala aswath.satras...@gmail.com wrote: thanks Cyrille, no jobs are now running. -Aswath On Thu, Oct 28, 2010 at 1:28 PM, Cyrille Vincey crll...@gmail.com wrote: Š I directly DELETE Š (makes more sense with all the words) From: Cyrille Vincey crll...@gmail.com Date: Thu, 28 Oct 2010 09:56:51 +0200 To: google-appengine-java@googlegroups.com Subject: Re: [appengine-java] mapreduce job is not aborted Abort button doesn't work indeed. When I want to abort a mapreduce job, I directly mapreduce-related entities in the datastore viewer : MapReduceStat and ShardDontKnowWhat. From: aswath satrasala aswath.satras...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Thu, 28 Oct 2010 11:03:52 +0530 To: google-appengine-java@googlegroups.com Subject: [appengine-java] mapreduce job is not aborted Hello, On my app-id vs-accounting.appspot.com http://vs-accounting.appspot.com , I started a mapreduce job. It did not run, because the Entity did not had decreasing index. The status is now showing with the link abort http://abort After clicking on the abort, the job is not aborted. I look at the logs, and it is continuously performing something every minute. Can you help me in aborting the job. Regards Aswath http://vs-accounting.appspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com mailto:google-appengine-java%2bunsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Mapreduce gives error when I execute job on appengine cloud?
Have you created a descending index on the key of the entity group you try to map ? (required for mapreduce jobs) From: suersh babu sureshgbab...@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Fri, 22 Oct 2010 14:52:17 +0530 To: google-appengine-java@googlegroups.com, google-appeng...@googlegroups.com Subject: [appengine-java] Mapreduce gives error when I execute job on appengine cloud? Hi, I am using mapreduce to run the job but when I try to run the job in mapreduce by clicking run button I am getting error like this : Error -- java.lang.RuntimeException: Full stack trace is available in the server logs. Message: Got an IOException while trying to make splits In my local machine mapreduce works fine, It executed the job without any error. I am using app engine sdk 1.3.8 Following is the server log generated by my application for the mapreduce error. 1. 10-22 02:02AM 49.783 2. com.google.appengine.tools.mapreduce.DatastoreInputFormat getSplits: Getting input splits for: Payment 3. 4. E 10-22 02:02AM 49.810 5. com.google.appengine.tools.mapreduce.MapReduceServlet handleCommand: Got exception while running command 6. java.lang.RuntimeException: Got an IOException while trying to make splits 7. at com.google.appengine.tools.mapreduce.MapReduceServlet.handleStart(MapReduceS ervlet.java:747) 8. at com.google.appengine.tools.mapreduce.MapReduceServlet.handleStartJob(MapRedu ceServlet.java:364) 9. at com.google.appengine.tools.mapreduce.MapReduceServlet.handleCommand(MapReduc eServlet.java:247) 10. at com.google.appengine.tools.mapreduce.MapReduceServlet.doPost(MapReduceServle t.java:184) 11. at javax.servlet.http.HttpServlet.service(HttpServlet.java:713) 12. at javax.servlet.http.HttpServlet.service(HttpServlet.java:806) 13. at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) 14. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .java:1166) 15. at com.veersoft.filter.ResetUserMessagesAndErrorsFilter.doFilter(ResetUserMessa gesAndErrorsFilter.java:28) 16. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .java:1157) 17. at com.google.apphosting.utils.servlet.ParseBlobUploadFilter.doFilter(ParseBlob UploadFilter.java:97) 18. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .java:1157) 19. at com.google.apphosting.runtime.jetty.SaveSessionFilter.doFilter(SaveSessionFi lter.java:35) 20. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .java:1157) 21. at com.google.apphosting.utils.servlet.TransactionCleanupFilter.doFilter(Transa ctionCleanupFilter.java:43) 22. at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler .java:1157) 23. at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:388) 24. at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216) 25. at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182) 26. at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) 27. at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:418) 28. at com.google.apphosting.runtime.jetty.AppVersionHandlerMap.handle(AppVersionHa ndlerMap.java:238) 29. at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) 30. at org.mortbay.jetty.Server.handle(Server.java:326) 31. at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542) 32. at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnectio n.java:923) 33. at com.google.apphosting.runtime.jetty.RpcRequestParser.parseAvailable(RpcReque stParser.java:76) 34. at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404) 35. at com.google.apphosting.runtime.jetty.JettyServletEngineAdapter.serviceRequest (JettyServletEngineAdapter.java:135) 36. at com.google.apphosting.runtime.JavaRuntime.handleRequest(JavaRuntime.java:261 ) 37. at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingReque st(RuntimePb.java:8486) 38. at com.google.apphosting.base.RuntimePb$EvaluationRuntime$6.handleBlockingReque st(RuntimePb.java:8484) 39. at com.google.net.rpc.impl.BlockingApplicationHandler.handleRequest(BlockingApp licationHandler.java:24) 40. at com.google.net.rpc.impl.RpcUtil.runRpcInApplication(RpcUtil.java:418) 41. at com.google.net.rpc.impl.Server$RpcTask.runInContext(Server.java:572) 42. at com.google.tracing.TraceContext$TraceContextRunnable$1.run(TraceContext.java :448) 43. at com.google.tracing.TraceContext.runInContext(TraceContext.java:688) 44. at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedC ontextNoUnref(TraceContext.java:326) 45. at com.google.tracing.TraceContext$AbstractTraceContextCallback.runInInheritedC ontext(TraceContext.java:318) 46. at com.google.tracing.TraceContext$TraceContextRunnable.run(TraceContext.java:4 46)
Re: [appengine-java] Re: Recommended maximum number of entities in an entity group
From my experience : I do NOT expect a better write performance than 1 transaction/second when creating entities inside one given entity group (with 1 entity created in each transaction). In your case : if dataset creation is an offline process, you can rely on entity groups and parent/child data modelling, no matter how many child entities you want to store. But you will have to expect high datastore contention level. My suggestion : in your data model design, only use parent/child design when transactional features are REALLY required. If transactional requirement are not so high, prefer to break your data model into smaller entity groups. On 21/10/10 17:00, nicanor.babula nicanor.bab...@gmail.com wrote: Thank you all for your responses. @alesj No, I am not confusing entity group with the actual entities stored in the datastore. @Ian Marshall Actually no, because I already did that kind of analysis. I have to use transactions in order to maintain data consistency and therefore I have to define the right entity groups. I already thought of a solution, just that I have read in the docs that is not recommended to put too many entities in an entity group. So: What is the number of child entities a parent entity can have, past which the datastore becomes slow? Hundreds? Thousands? Millions? I also understand that all entities in an entity group are stored on the same node of the datastore's distributed system and therefore I understand that if the number of entities an entity group has is too big, the queries will become slow, because will be processed by a single node. Right? Again: Which is that number? Thanks and sorry if I bored you with my long email. ;) On 21 Ott, 14:59, Ian Marshall ianmarshall...@gmail.com wrote: How about my comments below? http://www.google.com/url?url=http://groups.google.com/g/f907f736/t/f... Do they help you? On Oct 20, 6:39 pm, nicanor.babula nicanor.bab...@gmail.com wrote: Hi everbody, I have a question regarding the datastore best-practices. The appengine's official documentation says that is not a good practice to put too many entities in the same entity group. What does too many mean in this case? Hundreds? Thousands? Milions? Thanks in advance, Cristian Babula. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] SAXParseException: Premature end of file on dev server
Same issue for the past 3 days. Only occuring on the dev server. I have tried for hours to solve it. No clue... Fortunately, no impact on the production environment. On 21/10/10 11:50, Ian Marshall ianmarshall...@gmail.com wrote: Hello All, I am running my web application on the GAE/J 1.3.8 development server on my local machine. I have just got the following log entry whilst testing: 21-Oct-2010 09:13:06 com.google.apphosting.utils.jetty.JettyLogger warn WARNING: fa...@null line:1 col:1 : org.xml.sax.SAXParseException: Premature end of file. I did not notice any effect on my web app: I was reading data from my dev server's datastore but not writing any data to it (apart from perhaps some data of my session). Has anyone else got this recently? Am I doing anything wrong, or could this be just an occasional glitch of the dev server which I can just ignore? -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Unable to deploy 1.3.8 project
You need to change the GAE SDK version manually. On my mac : Eclipse Preferences Google App Engine On 19/10/10 08:03, Peter Liu tinyee...@gmail.com wrote: Ok. I installed a new Eclipse (Helios), install the Google plugin with only 1.3.8, created a new project, deploy to production, and still, the version is still Google App Engine/1.3.7. You can hit the url that print the version here: http://9.latest.vikispottest.appspot.com/vikispot9 I don't think it's my setting issue. Can anyone verify? On Oct 18, 10:41 pm, Peter Liu tinyee...@gmail.com wrote: I created a brand new project with 1.3.8 sdk, deploy to live server, and the SystemProperty.version is still: Google App Engine/1.3.7. Can anyone help? How do I verify what version of jars I deployed to production? @SuppressWarnings(serial) public class VikiSpot8Servlet extends HttpServlet { public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException { resp.setContentType(text/plain); String ver = SystemProperty.version.get(); resp.getWriter().println(ver); //resp.getWriter().println(Hello, world); } } -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] SDK 1.3.8 released!
Am I the only one who cannot download the eclipse 1.3.8 update ? On 15/10/10 14:51, Guillaume Laforge glafo...@gmail.com wrote: Excellent, thank you! The instances view is really nice and interesting! I just wanted to also mention that while upgrading one of my apps, one controller wasn't compiling against the new SDK, as there had been a little rename refactoring of a method: FetchOptions.Builder#withDefault is now: FetchOptions.Builder#withDefaults Notice the final 's'! Perhaps you could publish a list of API differences between releases? I know some OSS projects publish such documents for helping people upgrade to newer versions, or for showing users what the new APIs are. Anyhow, well done for this release! Guillaume On Fri, Oct 15, 2010 at 05:44, Takashi Matsuo tmat...@google.com wrote: Hello App Engine Developers! We're very happy to announce that SDK 1.3.8 is released today. There are many new cool features, so please download the new SDK and enjoy it! You can download the new SDK from: http://code.google.com/appengine/downloads.html Our blog post includes some screenshots of new features in admin console: http://googleappengine.blogspot.com/2010/10/new-app-engine-sdk-138-includ es-new.html Here are release notes: Java --- Version 1.3.8 = - You can run task queue tasks immediately from the admin console. - Added an OutputSettings class to the Images API to specify the JPEG encoding quality when running in production. - Support for login of multiple Google accounts within an app, and longer login sessions. For more information see: http://www.google.com/support/accounts/bin/answer.py?answer=181599 - In queue.xml, the maximum allowed bucket size is now 100. - Removed limits on zigzag merge-join queries. Therefore the error The built-in indices are not efficient enough for this query and your data. Please add a composite index for this query. will no longer be thrown in most cases, enabling more types of queries without indexes. - The whitelist has been updated to include java.net.InetAddress and some interfaces and abstract classes in javax.xml.soap, including javax.xml.soap.SOAPMessage. - Fixed an issue reserving App Ids by owners of emails containing periods, multiple cases, and googlemail.com address. http://code.google.com/p/googleappengine/issues/detail?id=1196 - Fixed an issue where TaskOptions had no public getters, making testing impossible. http://code.google.com/p/googleappengine/issues/detail?id=3243 - Fixed an issue on the development server where PNGs were being returned as JPEGs. http://code.google.com/p/googleappengine/issues/detail?id=3661 Python - Version 1.3.8 == - Builtin app.yaml handlers are available for common application functions, such as appstats. http://code.google.com/appengine/docs/python/config/appconfig.html#Builti n_Handlers - The Admin Console now provides an experimental tool to delete all entities in the datastore or all entities of a given type. This is available only if enabled using the datastore_admin builtin. Deleting entities will count against application quota. http://code.google.com/appengine/docs/python/datastore/creatinggettingand deletingdata.html#Deleting_Entities_in_Bulk - You can run task queue tasks immediately from the Admin Console. - You can now specify the quality of JPEG images via the Image API's execute_transforms function. Available in production only. - Support for login of multiple Google accounts within an app, and longer login sessions. For more information see: http://www.google.com/support/accounts/bin/answer.py?answer=181599 - In queue.yaml, the maximum allowed bucket size is now 100. - Precompilation is now enabled by default. To disable, use the --no_precompilation flag when updating your app. - BlobInfo now has an open() method that returns a BlobReader. - BlobReader now accepts a BlobInfo. - Removed limits on zigzag merge-join queries. Therefore the error The built-in indices are not efficient enough for this query and your data. Please add a composite index for this query. will no longer be thrown in most cases, enabling more types of queries without indexes. - Fixed an issue with task queue tasks not running on the dev_appserver when using Python 2.6. - Fixed an issue on the dev_appserver where auto task running wasn't working for BulkAdd. - Fixed an issue reserving App Ids by owners of similarly-named mails accounts containing periods, multiple cases, and googlemail.com address. http://code.google.com/p/googleappengine/issues/detail?id=1196 - Fixed an issue on the development server where PNGs were being returned as JPEGs. http://code.google.com/p/googleappengine/issues/detail?id=3661 -- Takashi Matsuo Developer Relations Developer Advocate for Google App Engine/iGoogle Google Japan, Inc. -- You received this
Re: [appengine-java] Fwd: entity update question
Yes you do. From: Vik vik@gmail.com Reply-To: google-appengine-java@googlegroups.com Date: Wed, 13 Oct 2010 18:47:33 +0530 To: Google App Engine for Java google-appengine-java@googlegroups.com Subject: [appengine-java] Fwd: entity update question Hie I do for create like try{ Employee e = new Employee(); pm.makePersistent(e); }finally(){ pm.close(); } For update case I am getting the entity by id and updating one of the attribute like Employee e = pm.getObectById(empPK, Employee.class); e.setSalary(1000); the question is do i need to call the below statement in this case as well? pm.makePersistent(e) Thankx and Regards Vik Founder www.sakshum.com http://www.sakshum.com www.sakshum.blogspot.com http://www.sakshum.blogspot.com -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Implementing inequalities with multiple properties.
My workaround (I'm doing the testing so far) : - create a extra property that merges the content of the N properties you wish to filter on - query on that single extrat property with inequality filters That is what geohash does when merging latitude AND longitude into one single property : http://hitching.net/2009/11/10/scalable-fast-accurate-geo-apps-using-google -app-engine-geohash-faultline-correction/ On 12/10/10 17:51, roberto_sc roberto.cal...@gmail.com wrote: I have a requirement to make searches with inequalities with multiple properties. This is not possible, as described in http://code.google.com/appengine/docs/java/datastore/queriesandindexes.htm l#Restrictions_on_Queries But I have to do it anyways. So, is there any hint on how to deal with this? I think many people has faced this problem. Thank you. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] makeTransientAll not working
Remove the line pm.makeTransientAll((Collection)results); and it should work fine. And no need for the line query.closeAll();) as well. On 11/10/10 12:14, Puneet puneet.nah...@gmail.com wrote: In the below method, the makeTransientAll method is not working. I am not able to show the contents of returned List in jsp. @Override public ListParty getParty (String name) { ListParty results = null; PersistenceManager pm = PMF.get().getPersistenceManager(); Query query = pm.newQuery(Party.class); query.setFilter(companyName.startsWith(nameParam)); query.declareParameters(String nameParam); try { results = (ListParty) query.execute(name); pm.makeTransientAll((Collection)results); } finally { query.closeAll(); pm.close(); } return results; } If I copy the contents of results to a new ArrayList, then the app is working fine. This means that the pm.makeTransientAll method is not working. Please help as to where i may be doing wrong. Thanks. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Problem with Persistent class
Same exception happened to me once. I was by error managing 2 pm instances, and trying to get an object with pm1 and to update it with pm2. Envoyé de mon iPad Le 3 oct. 2010 à 16:43, lisandrodc lisandr...@gmail.com a écrit : Hi ! I have a problem when persist a class. The exception is: Object with id is handled by another ObjectManager org.datanucleus.exceptions.NucleusUserException: Objeto con id es manejado por otro ObjectManager at org.datanucleus.ObjectManagerImpl.findStateManager(ObjectManagerImpl.java: 756) at org.datanucleus.store.mapped.scostore.ElementContainerStore.validateElementForWriting(ElementContainerStore.java: 372) at org.datanucleus.store.mapped.scostore.FKListStore.validateElementForWriting(FKListStore.java: 609) I'm not understand because (empty id...). Thepersitent class is @PersistenceCapable(identityType = IdentityType.APPLICATION,detachable=true) public class RegFechaUsuario extends Fecha { @Persistent private int puntos; @Persistent private Long idUsuarioFecha; @Persistent private Long idFechaOriginal; And @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class Fecha { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private String nombre; @Persistent private Date fechaIni; @Persistent private Date fechaFin; @Persistent(defaultFetchGroup = true) ListPartido partidos; private Long id2; Regards! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: ClassCastException [Lcom.google.appengine.api.datastore.Key
As far as I know, the execute() method does not work with arrays arguments. Modify the declaration of your 'group_keys' from 'Key[]' to 'Key'. By the way, all query errors happen to be assigned to the 'execute' line in debug mode. Le 2 oct. 2010 à 19:41, hendrix.jason hendrix.ja...@gmail.com a écrit : Cryille, Using debug mode, I can see that the group_keys array is being populated correctly. The error is appearing on the last line: ListGroup groups = (ListGroup) q.execute(group_keys); Thanks, Jason On Sep 30, 3:41 am, Cyrille Vincey crll...@gmail.com wrote: Try : Key k = KeyFactory.createKey(User.class.getSimpleName(), Key.getId()); Instead of : Key k = KeyFactory.createKey(User.class.getSimpleName(), Long.parseLong(Key)); On 30/09/10 01:16, hendrix.jason hendrix.ja...@gmail.com wrote: What could this error mean: java.lang.ClassCastException: [Lcom.google.appengine.api.datastore.Key; cannot be cast to com.google.appengine.api.datastore.Key It's happening on the last line of this code: PersistenceManager pm2 = PMF.get().getPersistenceManager(); Key k = KeyFactory.createKey(User.class.getSimpleName(),Long.parseLong(Key)); User u = pm2.getObjectById(User.class, k); Long[] group_array = u.getGroups(); Key[] group_keys = new Key[group_array.length]; int c = 0; while(c group_array.length){ group_keys[c] = KeyFactory.createKey(Group.class.getSimpleName(), group_array[c]); c++; } javax.jdo.Query q = pm2.newQuery(Group.class,key.contains(:key)); ListGroup groups = (ListGroup) q.execute(group_keys); -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] no matching index found..
The GAE Eclipse plugin is supposed to create all required indexes in the config files when running the app in development mode. Those indexes are configured in the datastore-indexes-auto.xml file (dir : WEB-INF appengine-generated) Sometimes it happens some indexes are missing. Then create or edit the datastore-indexes.xml (dir : WEB-INF) and create them manually. See http://code.google.com/intl/fr-FR/appengine/docs/java/config/indexconfig.ht ml On 27/09/10 09:11, Prateek bittooagar...@gmail.com wrote: Hi, Yesterday i just tried deploying my app on GAE. Everything worked fine in the local deployment. But as soon as i tried access the same in GAE prod, I received a Error saying that, no matching index found.. datastore-index kind=MyBlogSpotComment ancestor=true source=manual property name=myBlogSpotComments_INTEGER_IDX direction=asc/ /datastore-index I read in a forum saying that the Indexes in must yet be getting created but i m not able to understand it My datastores Indexes dashboard shows as: MyBlogSpotURL myBlogSpotURLs_INTEGER_IDX ▲ Includes ancestors Serving MyMobileNumbers myMobileNumbers_INTEGER_IDX ▲ Includes ancestors Serving Regards, Prateek Agarwal -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Problem with persist a child class
Sorry, I did not see that your child class was inherited from another class. I tried using inheritance with JDO some times ago. Unsuccessfully : this is not supported by JDO yet. See http://code.google.com/intl/fr-FR/appengine/docs/java/datastore/dataclasses. html#Inheritance On 24/09/10 01:30, lisandrodc lisandr...@gmail.com wrote: Hi!Cyrille I add a primary key in my child class: @PersistenceCapable(identityType = IdentityType.APPLICATION,detachable=true) public class RegFechaUsuario extends Fecha { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private int puntos; @Persistent private Long idUsuarioFecha; @Persistent private Long idFechaOriginal; And the exception is: Error in meta-data for model.RegFechaUsuario: More than one primary key field. Caused by: Error in meta-data for model.RegFechaUsuario: More than one primary key field. org.datanucleus.store.appengine.MetaDataValidator $DatastoreMetaDataException: Error in meta-data for model.RegFechaUsuario: More than one primary key field. at org.datanucleus.store.appengine.MetaDataValidator.validatePrimaryKey(MetaDataV alidator.java: 416) at org.datanucleus.store.appengine.MetaDataValidator.validate(MetaDataValidator.j ava: 120) at org.datanucleus.store.appengine.DatastoreManager.validateMetaDataForClass(Data storeManager.java: 766) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPreProcess(D atastorePersistenceHandler.java: 328) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects(Data storePersistenceHandler.java: 251) at org.datanucleus.store.appengine.BatchPutManager.processBatchState(BatchPutMana ger.java: 35) at org.datanucleus.store.appengine.BatchManager.finish(BatchManager.java: 54) at org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager $BatchManagerWrapper.call(DatastoreJDOPersistenceManager.java:127) at org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager $BatchManagerWrapper.access$200(DatastoreJDO On 23 sep, 05:25, Cyrille Vincey crll...@gmail.com wrote: There is no primary key in your child class. Add one. On 23/09/10 04:40, lisandrodc lisandr...@gmail.com wrote: Hi ! I have a problem when persist a chid class. The parent class: @PersistenceCapable @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class Fecha { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private String nombre; @Persistent private Date fechaIni; @Persistent private Date fechaFin; @Persistent(defaultFetchGroup = true) ListPartido partidos; private Long id2; private Long kind; The child class: @PersistenceCapable(identityType = IdentityType.APPLICATION,detachable=true) public class RegFechaUsuario extends Fecha { @Persistent private int puntos; @Persistent private Long idUsuarioFecha; @Persistent private Long idFechaOriginal; The method at persist: public void crearRegFechaUsuario(RegFechaUsuario regFechaUsuario) { //regFechaUsuario is an object initialized Transaction tx = pm.currentTransaction(); try { tx.begin(); pm.makePersistentAll(regFechaUsuario); //here is the exception tx.commit(); } finally { // pm.close(); if (tx.isActive()) { tx.rollback(); } } } And the exception is (internal error of the library of google apps or datanucleus), the cast: Problem accessing /Prode/JugarFecha.action. Reason: java.lang.Long cannot be cast to java.lang.String Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.String at org.datanucleus.store.appengine.DatastoreRelationFieldManager.checkForParent Sw itch(DatastoreRelationFieldManager.java: 202) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.setObjectViaMapping(DatastoreRelationFieldManager.java:133) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.apply(DatastoreRelationFieldManager.java:112) at org.datanucleus.store.appengine.DatastoreRelationFieldManager.storeRelations (D atastoreRelationFieldManager.java: 81) at org.datanucleus.store.appengine.DatastoreFieldManager.storeRelations(Datasto re FieldManager.java: 955) at org.datanucleus.store.appengine.DatastorePersistenceHandler.storeRelations(D at astorePersistenceHandler.java: 546) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPostProces s( DatastorePersistenceHandler.java: 304) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects(Da ta storePersistenceHandler.java: 256) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObject(Dat as torePersistenceHandler.java: 240) at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateMan ag erImpl.java: 3185
Re: [appengine-java] Creating indexes on List of String elements
You're facing the exploding indexes problem. You should never put more than 1 list properties in one composite index. This problem is described in the GAE doc. http://code.google.com/intl/fr-FR/appengine/docs/python/datastore/queriesan dindexes.html#Big_Entities_and_Exploding_Indexes On 24/09/10 15:54, Ice13ill andrei.fifi...@gmail.com wrote: I'm trying to implement a search based on a list of keywords (string). The list is created based on the content of the objects' fields and i use it when matching against a query text. The problem is that i can't create indexes on App engine (i get Error after about 30 mins, and i have to delete it). There are about 90,000 entities of that kind and the index should looke like this: keyWordsList ▲ , keyWordsList ▲ , keyWordsList ▲ , key ▼ and i have about four of them. My question: do i want to much ? :) I tried uploading about 4 indexes and i got Error. I tried uploading 1 and worked (but needed about 2-3 h). Is there a problem if i try to create 4 indexes of this magnitude (and why)? Can i be sure that uploading one by one will work? Or maybe this problem is related to AppEngine datastore problems ? Please advice ! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Creating indexes on List of String elements
I am afraid you will have to change your data model. By the way, I remember that someone shared so times ago his code for a full text search implementation in GAE. Maybe you should check this out. On 24/09/10 17:01, Ice13ill andrei.fifi...@gmail.com wrote: Hmm... i understand the problem. But, if i want to query using a text with multiple words and NO sorting order, i use the build-in index supplied default right ? So, what do i do for 90.000 entities ?? because it seams that the build-in index is not powerful enough ! The built-in indices are not efficient enough for this query and your data. Please add a composite index for this query.. An index is missing but we are unable to tell you which one due to a bug in the App Engine SDK. If your query only contains equality filters you most likely need a composite index on all the properties referenced in those filters. So the solution is creating a index like this: keyWordsList ▲ , keyWordsList ▲ , keyWordsList ▲ Isn't this really the build-in index ? On Sep 24, 5:16 pm, Cyrille Vincey crll...@gmail.com wrote: You're facing the exploding indexes problem. You should never put more than 1 list properties in one composite index. This problem is described in the GAE doc.http://code.google.com/intl/fr-FR/appengine/docs/python/datastore/que ... dindexes.html#Big_Entities_and_Exploding_Indexes On 24/09/10 15:54, Ice13ill andrei.fifi...@gmail.com wrote: I'm trying to implement a search based on a list of keywords (string). The list is created based on the content of the objects' fields and i use it when matching against a query text. The problem is that i can't create indexes on App engine (i get Error after about 30 mins, and i have to delete it). There are about 90,000 entities of that kind and the index should looke like this: keyWordsList ▲ , keyWordsList ▲ , keyWordsList ▲ , key ▼ and i have about four of them. My question: do i want to much ? :) I tried uploading about 4 indexes and i got Error. I tried uploading 1 and worked (but needed about 2-3 h). Is there a problem if i try to create 4 indexes of this magnitude (and why)? Can i be sure that uploading one by one will work? Or maybe this problem is related to AppEngine datastore problems ? Please advice ! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Problem with persist a child class
There is no primary key in your child class. Add one. On 23/09/10 04:40, lisandrodc lisandr...@gmail.com wrote: Hi ! I have a problem when persist a chid class. The parent class: @PersistenceCapable @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class Fecha { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private String nombre; @Persistent private Date fechaIni; @Persistent private Date fechaFin; @Persistent(defaultFetchGroup = true) ListPartido partidos; private Long id2; private Long kind; The child class: @PersistenceCapable(identityType = IdentityType.APPLICATION,detachable=true) public class RegFechaUsuario extends Fecha { @Persistent private int puntos; @Persistent private Long idUsuarioFecha; @Persistent private Long idFechaOriginal; The method at persist: public void crearRegFechaUsuario(RegFechaUsuario regFechaUsuario) { //regFechaUsuario is an object initialized Transaction tx = pm.currentTransaction(); try { tx.begin(); pm.makePersistentAll(regFechaUsuario); //here is the exception tx.commit(); } finally { // pm.close(); if (tx.isActive()) { tx.rollback(); } } } And the exception is (internal error of the library of google apps or datanucleus), the cast: Problem accessing /Prode/JugarFecha.action. Reason: java.lang.Long cannot be cast to java.lang.String Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.String at org.datanucleus.store.appengine.DatastoreRelationFieldManager.checkForParentSw itch(DatastoreRelationFieldManager.java: 202) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.setObjectViaMapping(DatastoreRelationFieldManager.java:133) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.apply(DatastoreRelationFieldManager.java:112) at org.datanucleus.store.appengine.DatastoreRelationFieldManager.storeRelations(D atastoreRelationFieldManager.java: 81) at org.datanucleus.store.appengine.DatastoreFieldManager.storeRelations(Datastore FieldManager.java: 955) at org.datanucleus.store.appengine.DatastorePersistenceHandler.storeRelations(Dat astorePersistenceHandler.java: 546) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPostProcess( DatastorePersistenceHandler.java: 304) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects(Data storePersistenceHandler.java: 256) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObject(Datas torePersistenceHandler.java: 240) at org.datanucleus.state.JDOStateManagerImpl.internalMakePersistent(JDOStateManag erImpl.java: 3185) at org.datanucleus.state.JDOStateManagerImpl.makePersistent(JDOStateManagerImpl.j ava: 3161) at org.datanucleus.ObjectManagerImpl.persistObjectInternal(ObjectManagerImpl.java : 1298) at org.datanucleus.sco.SCOUtils.validateObjectForWriting(SCOUtils.java: 1476) at org.datanucleus.store.mapped.scostore.ElementContainerStore.validateElementFor Writing(ElementContainerStore.java: 380) at org.datanucleus.store.mapped.scostore.FKListStore.validateElementForWriting(FK ListStore.java: 609) at org.datanucleus.store.mapped.scostore.FKListStore.internalAdd(FKListStore.java : 344) at org.datanucleus.store.appengine.DatastoreFKListStore.internalAdd(DatastoreFKLi stStore.java: 146) at org.datanucleus.store.mapped.scostore.AbstractListStore.addAll(AbstractListSto re.java: 128) at org.datanucleus.store.mapped.mapping.CollectionMapping.postInsert(CollectionMa pping.java: 157) at org.datanucleus.store.appengine.DatastoreRelationFieldManager.runPostInsertMap pingCallbacks(DatastoreRelationFieldManager.java: 217) at org.datanucleus.store.appengine.DatastoreRelationFieldManager.access $200(DatastoreRelationFieldManager.java:48) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.apply(DatastoreRelationFieldManager.java:116) at org.datanucleus.store.appengine.DatastoreRelationFieldManager.storeRelations(D atastoreRelationFieldManager.java: 81) at org.datanucleus.store.appengine.DatastoreFieldManager.storeRelations(Datastore FieldManager.java: 955) at org.datanucleus.store.appengine.DatastorePersistenceHandler.storeRelations(Dat astorePersistenceHandler.java: 546) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertPostProcess( DatastorePersistenceHandler.java: 304) at org.datanucleus.store.appengine.DatastorePersistenceHandler.insertObjects(Data storePersistenceHandler.java: 256) at org.datanucleus.store.appengine.BatchPutManager.processBatchState(BatchPutMana ger.java: 35) at org.datanucleus.store.appengine.BatchManager.finish(BatchManager.java: 54) at org.datanucleus.store.appengine.jdo.DatastoreJDOPersistenceManager $BatchManagerWrapper.call(DatastoreJDOPersistenceManager.java:127)
[appengine-java] MapReduce - Active shards
What drives the number of active shards in a mapreduce job ? In my case, mapreduce jobs execution usually work fine, but only ONE shard actually processes entities, the 3 others don't. Any clue ? On 14/09/10 00:06, Benjamin bsaut...@gmail.com wrote: OK thanks Ikai, I was trying to avoid that but see no problem with it. Ben On Sep 13, 5:15 pm, Ikai Lan (Google) ikai.l+gro...@google.com wrote: You can't. Empty properties are the equivalent of unindexed properties. You'll need to iterate over all the Entities in your datastore and update them. The Mapper API is a very good tool for this: http://code.google.com/p/appengine-mapreduce/ On Mon, Sep 13, 2010 at 12:20 PM, Benjamin bsaut...@gmail.com wrote: Brain freeze guys - I added a property to my object and persisted in the datastore. The older object have now have a column that when viewed in the control panel have a value of missing. I need to query those objects so i can update them - propname == null , propname == - I can't seem to find the right syntax for this. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.comgoogle-appengine-java%2B unsubscr...@googlegroups.com . For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Problem with persist a child class
Be aware that inheritance is not supported by GAE yet. The problem might come from this issue. See : http://code.google.com/intl/fr-FR/appengine/docs/java/datastore/dataclasses. html cyrille On 19/09/10 18:55, lisandrodc lisandr...@gmail.com wrote: Thanks dushyant , but the exception is of an error of JDO,datanucleus (seemingly). That it has to see with the keys it seems to me, is not the problem that I from my code can use Long.parseLong(String) to parse. This is something internal that that should do datanucleus. Regards. Lisandro On 19 sep, 12:19, dushyant bing dush...@gmail.com wrote: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.String use Long.parseLong(String) to parse Sent from my iPad On Sep 19, 2010, at 7:52 PM, lisandrodc lisandr...@gmail.com wrote: Hi! I have a problem with persist a child class. The classes are: The parent class: @PersistenceCapable @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class Fecha { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private String nombre; @Persistent private Date @Persistent private Date fechaFin; @Persistent(defaultFetchGroup = true) ListPartido partidos; The child class: @PersistenceCapable public class RegFechaUsuario extends Fecha { @Persistent private int puntos; @Persistent private Long idUsuarioFecha; @Persistent private Long idFechaOriginal; And the code of run the exception is: Transaction tx = pm.currentTransaction(); try { tx.begin(); pm.makePersistentAll(regFechaUsuario); //here is the exception tx.commit(); And the regFechaUsuario is created with the build: public RegFechaUsuario(String nombre, Date fechaIni, Date fechaFin, Long idUsuarioFecha, Long idFechaOriginal, ListPartidopartidos) { //super(nombre,fechaIni,fechaFin,partidos); this.puntos = 0; this.idUsuarioFecha = idUsuarioFecha; this.idFechaOriginal = idFechaOriginal; this.partidos=partidos; } The exception is: Problem accessing /Prode/JugarFecha.action. Reason: java.lang.Long cannot be cast to java.lang.String Caused by: java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.String at org.datanucleus.store.appengine.DatastoreRelationFieldManager.checkForParentS witch(DatastoreRelationFieldManager.java: 202) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.setObjectViaMapping(DatastoreRelationFieldManager.java:133) at org.datanucleus.store.appengine.DatastoreRelationFieldManager $1.apply(DatastoreRelationFieldManager.java:112) at org.datanucleus.store.appengine.DatastoreRelationFieldManager.storeRelations( DatastoreRelationFieldManager.java: 81) at org.datanucleus.store.appengine.DatastoreFieldManager.storeRelations(Datastor eFieldManager.java: 955) at org.datanucleus.store.appengine.DatastorePersistenceHandler.storeRelations(Da tastorePersistenceHandler.java: 546) at org.datanucleus.store.appengine.DatastorePersistenceHandler. Regards Lisandro -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group athttp://groups.google.com/group/google-appengine-java?hl=en. -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
[appengine-java] KeyFactory.createKey() method creates unrecognized keys
Dear All, I am facing a strange issue using the KeyFactory.createKey() method : keys generated by this method are stored as unowned relationships, but are actually not recognized by the datastore viewer (the key-URL does not work). Hopefully you could help me on this. A few words about my source below. Entity A is a root entity. Entity B is a child of entity A. Entity C is another root entity. I use JDO for persistence. All my entity primary keys are declared this way : ... @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key key; ... In my entity C, I need to store an unowned relationship to an entity B. My parameters when creating C : the key IDs (type 'Long') of A and B. My code to create entity C is quite simple, as follow : ... Key aKey = KeyFactory.createKey(ClassA.class.getSimpleName(), akeyId); Key bKey = aKey.getChild(ClassB.class.getSimpleName(), bKeyId); entityC.setEntityBKey(bKey); pm.makePersistent(entityC); ... So far it's ok, but the bKey stored in entity C is not recognized by other programs. Any clue ? Thanks for your help. Cyrille -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: removing element from list not persisted in datastore
Stepmas, please keep us informed with your tests on that particular point, as we are currently changing our data model to match Brett's recommandation about list properties in child object. On 09/09/10 11:58, Frederik Pfisterer pfiste...@gmail.com wrote: I experienced this with lists of complex objects in owned relationships, not with lists of strings, see http://code.google.com/p/datanucleus-appengine/issues/detail?id=218 . Lists of strings should not be a problem since they're stored in the same table, basically the listproperties Bret mentions in his talk. Are you using GWT? Fred On 8 Sep., 11:50, androidDeveloper stepmas...@googlemail.com wrote: Hi all, I read the post http://groups.google.com/group/google-appengine-java/browse_thread/th... which describes a problem with the local datastore and empty lists. This problem seems to be fixed. But maybe I have found another one. I have a list ListString favoritesUserId in a index class to which I can add or remove keys (encoded Strings). So there is the possibility, that the list gets empty, after all items are removed. The class is a child of another class. The structure is kind of the index-class pattern introduced by Bret Slatkin (Building Scalable, complex Apps on App engine) So here is the problem. When I try to save an item in the list, it works fine. Removing items works fine too, as long the list after removing the item is not empty. When I remove the last item and save the object, the last item stays in the list. I have not tried it in production yet, cause I am not ready for testing prod. Maybe this is a local problem with the SDK? Does anyone know what the solution to this problem is? Thanks! -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Uploading Downloading data
Massive upload / download : have a look on the appengine bulkloader. If no need for massive download, just... develop a java servlet. On 07/09/10 19:11, dciesli...@yahoo.com dciesli...@yahoo.com wrote: Does anyone has or know where to find detail example how to use uploading downloading data using Google App Engine? I have developed a simple Web application and would like now to send and receive data from my local computer. On Google App Engine site I found one page that discusses these issues (app.yaml, appcfg.py, remote_api etc.). I tried to implement this but without success. So the best would be some more detailed example how it is really done (sample code?). (My environment is: Windows, eclipse, java). Any help greatly appreciated. Damian -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Re: Problem with inheritance and JDO
Go there... http://code.google.com/intl/fr-FR/appengine/docs/java/datastore/relationship s.html#Polymorphic_Relationships ... And you'll read this : Even though the JDO specification includes support for polymorphic relationships, polymorhpic relationships are not yet supported in the App Engine JDO implementation. This is a limitation we hope to remove in future releases of the product. If you need to refer to multiple types of objects via a common base class we recommend following the same strategy we recommend for implementing unowned relationships: store a Key reference. Bye. On 08/09/10 02:14, lisandrodc lisandr...@gmail.com wrote: Thanks Cyrille, but.. Might someone confirm to me authentically that really inheritance is not supported? If it is like that, a serious enough failing On the part of the documentation of google it does not confirm it but he does not deny it either... The error is error. org.datanucleus.exceptions.NoPersistenceInformationException: The class com.onwebconsulting.inventory.model.Part is required to be persistable yet no Meta-Data/Annotations can be found for this class. Please check that the Meta-Data/annotations is defined in a valid file Regards Lisandro On 7 sep, 09:48, Cyrille Vincey crll...@gmail.com wrote: I have read somewhere in the appengine doc that class inheritance is not really supported by JDO yet... On 07/09/10 00:39, lisandrodc lisandr...@gmail.com wrote: Hi all, I've used hibernate for a while, and I'm having a bit of trouble switching to JDO. I have several entities that all entities will share. Rather than a parent table, I'd prefer these columns be embedded directly in each entity. As such, I've defined my abstract entity and child entity in the classes below. However, whenever I try to run my unit tests or the jetty environment, I always receive the following error. org.datanucleus.exceptions.NoPersistenceInformationException: The class com.onwebconsulting.inventory.model.Part is required to be persistable yet no Meta-Data/Annotations can be found for this class. Please check that the Meta-Data/annotations is defined in a valid file location.]] According to my Eclipse console I have 1 class that has been Enhanced. Any ideas what I'm doing wrong? DataNucleus Enhancer completed with success for 1 classes. Timings : input=268 ms, enhance=51 ms, total=319 ms. Consult the log for full details @Inheritance(strategy = InheritanceStrategy.NEW_TABLE) public class Part extends Auditable{ @Persistent private String name; @Persistent private String description; @Persistent private String partNumber; @Persistent private long quantity; @Persistent private String manuacturersPartNumber; @Persistent private String manufacturerName; @Persistent private String suppliersPartNumber; @Persistent private String supplierName; @Persistent private String supplierWebsite; @Persistent private float salePrice; @Persistent private float supplierPrice; ... Getters and Setters } @PersistenceCapable(identityType = IdentityType.APPLICATION) @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class Auditable { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private Date createDate; @Persistent private Date updatedDate; @Persistent private long version; public Key getId() { return id; } public Date getCreateDate() { return createDate; } public Date getUpdatedDate() { return updatedDate; } public long getVersion(){ return version; } -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.
Re: [appengine-java] Problem with inheritance and JDO
I have read somewhere in the appengine doc that class inheritance is not really supported by JDO yet... On 07/09/10 00:39, lisandrodc lisandr...@gmail.com wrote: Hi all, I've used hibernate for a while, and I'm having a bit of trouble switching to JDO. I have several entities that all entities will share. Rather than a parent table, I'd prefer these columns be embedded directly in each entity. As such, I've defined my abstract entity and child entity in the classes below. However, whenever I try to run my unit tests or the jetty environment, I always receive the following error. org.datanucleus.exceptions.NoPersistenceInformationException: The class com.onwebconsulting.inventory.model.Part is required to be persistable yet no Meta-Data/Annotations can be found for this class. Please check that the Meta-Data/annotations is defined in a valid file location.]] According to my Eclipse console I have 1 class that has been Enhanced. Any ideas what I'm doing wrong? DataNucleus Enhancer completed with success for 1 classes. Timings : input=268 ms, enhance=51 ms, total=319 ms. Consult the log for full details @Inheritance(strategy = InheritanceStrategy.NEW_TABLE) public class Part extends Auditable{ @Persistent private String name; @Persistent private String description; @Persistent private String partNumber; @Persistent private long quantity; @Persistent private String manuacturersPartNumber; @Persistent private String manufacturerName; @Persistent private String suppliersPartNumber; @Persistent private String supplierName; @Persistent private String supplierWebsite; @Persistent private float salePrice; @Persistent private float supplierPrice; ... Getters and Setters } @PersistenceCapable(identityType = IdentityType.APPLICATION) @Inheritance(strategy = InheritanceStrategy.SUBCLASS_TABLE) public abstract class Auditable { @PrimaryKey @Persistent(valueStrategy = IdGeneratorStrategy.IDENTITY) private Key id; @Persistent private Date createDate; @Persistent private Date updatedDate; @Persistent private long version; public Key getId() { return id; } public Date getCreateDate() { return createDate; } public Date getUpdatedDate() { return updatedDate; } public long getVersion(){ return version; } -- You received this message because you are subscribed to the Google Groups Google App Engine for Java group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.