I was considering calling list, but what's nice about the iterative
method is that if I ever run out of time on a task (i.e. catch the
exception that is thrown when the deadline is approaching) I can grab
a cursor from the iterator and kick off a new task starting where I
left off on the query... I'll play around with a few things, although
it's looking like the times I'm getting are totally valid.
Thanks for responding, guys! I'm going to sit down with appstats
sometime today and possibly rethink a couple of my methods.
On Feb 8, 5:43 am, Strom <st...@masendav.org> wrote:
> I'm not too sure how that iteration works under the covers, but if you
> want to iterate over the result set multiple times then this is surely
> faster:
>
> List<Key<FakeEntity>> arr =
> ofy.query(FakeEntity.class).filter("lookAtMe", "woo").listKeys(); //
> Or use .list() to get the actual entities
>
> On Feb 7, 8:30 pm, HalcyonDays <jbpap...@gmail.com> wrote:
>
>
>
> > Thanks, I'll definitely run it through appstats as soon as I get a
> > chance...
>
> > ...here's the code, though, if you'd like to take a look. Nothing too
> > crazy. I'm using Objectify.
>
> > Objectify ofy = new DAO().ofy();
> > ArrayList<Key<FakeEntity>> arr = new ArrayList<Key<FakeEntity>>();
>
> > Query<FakeEntity> query =
> > ofy.query(FakeEntity.class).filter("lookAtMe", "woo");
> > QueryResultIterator<FakeEntity> iterator = query.iterator();
>
> > while(iterator.hasNext()){
> > arr.add(iterator.next().getKey());
>
> > }
>
> > ofy.delete(arr);
>
> > Essentially just gets an objectify object, does a query over my test
> > object "FakeEntity" looking for all entries whose "lookAtMe" field is
> > set to "woo," iterates over them, adding to a list, and then passes
> > the list to the delete call.
>
> > I know iterating over each one is not necessary with Objectify and I
> > can simply pass the iterator to the delete call, but my test is
> > simulating what the real code will need to do (i.e. look at each
> > object before determining if it can be deleted or not.)
>
> > Nothing too crazy, but there may be some fundamental misunderstanding
> > of how bulk deletes should be done.
>
> > On Feb 7, 12:34 pm, Robert Kluin <robert.kl...@gmail.com> wrote:
>
> > > Sometimes it can be more expensive than expected, but there is also a
> > > fair chance your code is missing some 'basic' (ie common app engine)
> > > optimizations. Use Appstats to profile your code, look for lots of
> > > RPC calls done in serial. Make sure you read about keys only queries
> > > and batch operations.
>
> > > Also, post your test code. If there is low-hanging-fruit, people will
> > > (probably) let you know about it pretty quickly.
>
> > >http://code.google.com/appengine/docs/java/tools/appstats.htmlhttp://...
>
> > > Robert
>
> > > On Mon, Feb 7, 2011 at 11:54, HalcyonDays <jbpap...@gmail.com> wrote:
> > > > Hello all -
>
> > > > I've been doing a few load tests with the task queue, and I ran across
> > > > some interestingly large numbers... I constructed a very simple test
> > > > to put batches of a thousand or so fake items into the datastore at a
> > > > time, and then created a task to see how many I could query, count,
> > > > and remove in a reasonable amount of time. Wall (real) time was
> > > > fantastic for both tests, roughly 18653 ms to delete 21,000 of my test
> > > > items... the number that had me a teensy bit worried was the CPU
> > > > time... 1494101 cpu_ms. So... essentially 20-some minutes. If this
> > > > is normal, then it's just something I'll have to be aware of for any
> > > > expensive tasks... but is this standard/to be expected? It seems odd,
> > > > even on something as massively parallel as the datastore that I would
> > > > be able to rack up that much CPU time in a little under 20 seconds.
>
> > > > Anyway, just curious.
>
> > > > Thanks!
>
> > > > --
> > > > You received this message because you are subscribed to the Google
> > > > Groups "Google App Engine" group.
> > > > To post to this group, send email to google-appengine@googlegroups.com.
> > > > To unsubscribe from this group, send email to
> > > > google-appengine+unsubscr...@googlegroups.com.
> > > > For more options, visit this group
> > > > athttp://groups.google.com/group/google-appengine?hl=en.
--
You received this message because you are subscribed to the Google Groups
"Google App Engine" group.
To post to this group, send email to google-appengine@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine?hl=en.
