We are really excited with this release, which is the first feature-
complete release of SimpleDS. Lots of things have been included in so
little time.
Getting up-to-date with AppEngine
============================
It's hard to keep up the pace with these guys. This release includes:
* Unindexed attributes.
* Cursors support.
* IN and != clauses.
Cache
=====
We have included a great Level 1 and Level 2 cache [1]. If you come
from JDO/JPA, you may already know how it works:
* Level 1 cache: This is basically a Map bound to the current thread.
Until the end of the current request, any get() invocation will check
this cache first. If a match is found, no invocation will be
propagated to GAE.
* Level 2 cache: Datastore entities are also stored in memcache, which
is a second chance to get a positive match.
Cacheable entities must be marked with @Cacheable, with an optional
expiration time. This feature will work with single and batch get(),
and the cache entries are updated with put() and delete()
invocations.
// Invoke memcache or the datastore
List<MyData> data = entityManager.get(key1, key2, key3);
// this does not invoke anything (resolved by the Level 1 cache)
MyData d2 = entityManager.get(key1);
Functions
========
We are going extremely functional these days [2]. This release
includes a package with functions to transform collections and
PagedList instances, which can be combined with batch get() to get
even better performance results. This is a simple example, equivalent
to a common situation with relationships:
// n + 1 requests to the datastore
List<MyData> data = entityManager.find(query);
Collection<Parent> parents =
Lists.newArrayListwithCapacity(data.size());
for (MyData d : data) {
parents.add(entityManager.get(d.getKey().getParent());
}
// Transformations: 2 requests
List<MyData> data = entityManager.find(query);
Collection<Key> parentKeys = Collections2.transform(data, new
EntityToParentKeyFunction(MyData.class));
Collection<Parent> parents = entityManager.get(parentKeys);
That's all it takes to get all entities and their parents, and store
them in the Level 1 cache so any request by PK will not hit memcache
or the datastore. This release includes functions to retrieve parent
and foreign keys, and also works with PagedList. We have taken two
real-world snapshots with just this optimization (transformation +
cacheable), applied to a single loop:
Before: http://www.flickr.com/photos/koliseocom/4575062969/sizes/l/
After: http://www.flickr.com/photos/koliseocom/4575696456/sizes/o
Notice that requests "before" are targeted to the datastore, while
most of the "after" are memcache requests.
Background tasks
==============
I know that background tasks [3] are in the roadmap for AppEngine, but
we needed these today. This started as an exercise to upgrade the
datastore schema (add properties, delete entities, etc) and ended up
as a full reusable implementation of tasks that I expect to deprecate
once that AppEngine includes its own, probably better, implementation.
Background tasks require adding a servlet to web.xml (and optionally
appengine-web.xml as an admin-console entry) and configuring the tasks
on application start using plain Java. Tasks can be invoked directly
by cron triggers, queues or by POST requests.
public class WriteBehindCacheTask extends IterableTask<MyClass> {
protected WriteBehindCacheTask() {
super("my-task-id");
}
@Override
protected SimpleQuery createQuery(TaskRequest request) {
return entityManager.createQuery(MyClass.class);
}
@Override
protected void process(MyClass entity, TaskRequest request) {
// ...modify entity...
entityManager.put(entity);
}
}
Query cursors and execution deferral will be done transparently (no
need to limit or handle cursors). There are some implementation
superclasses depending on what you need to do, and some of them just
use the raw AppEngine datastore and don't even require SimpleDS to
work.
This is the first feature-complete release of SimpleDS. Next on the
roadmap: cached queries and we may be revisiting relations.
As always, all kind of feedback is welcome.
Nacho.
[1] http://code.google.com/p/simpleds/wiki/Cache
[2] http://code.google.com/p/simpleds/wiki/Functions
[3] http://code.google.com/p/simpleds/wiki/BackgroundTasks
--
You received this message because you are subscribed to the Google Groups
"Google App Engine for Java" group.
To post to this group, send email to google-appengine-j...@googlegroups.com.
To unsubscribe from this group, send email to
google-appengine-java+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/google-appengine-java?hl=en.
