Currently hasNext() will block on the first batch of results. There is currently no way to do what you are asking in Java.
On Tue, Dec 7, 2010 at 2:40 AM, Maxim Veksler <ma...@vekslers.org> wrote: > Hi Max, > > Could you please provide input on the following design: > > We design for DataSource redundancy. For this we've built a flow which > obtains data from 2 sources: First from GAE DataSource using PreparedQuery > and if that fails (0.5sec timeout) from our backup service feed which we > access using URLFetch (auto scaled EC2 nodes). > > We strive for maximum performance, So we plan to issue async call to each > of the data sources and later when the data becomes required poll the 2 > sources, using the first one available. > > As for the URLFetch service, we will use this patten: > Future<HTTPResponse> f = fetchService.fetchAsync(request); > ... > if(f.isDone()) { ... } > > But for PreparedQuery.asIterator(), I'm not sure how this is implement on > your side. I would love an answer confirming that this code will not block > on hasNext() (While datasource is asynchronously getting results)? > > Iterator<Entity> iterator = > getAsyncDatastoreService().prepare(findGeoIP).asIterator(); > .... > if(iterator.hasNext()) { ... } > > > > Thanks for the insights. > > Maxim. > > On Mon, Nov 29, 2010 at 9:08 PM, Max Ross (Google) < > maxr+appeng...@google.com <maxr%2bappeng...@google.com>> wrote: > >> Hi Luke, >> >> First the awesome news: >> As of 1.4.0, many queries are implicitly asynchronous. When you call >> PreparedQuery.asIterable() or PreparedQuery.asIterator(), we initiate the >> query in the background and then immediately return. This lets you do work >> while the first batch of results is being fetched. And, when the first >> batch has been consumed we immediately request the next batch. If you're >> performing a significant amount of work with each Entity as you iterate you >> will probably see a latency win as a result of this. >> >> Now the less awesome news: >> We didn't get around to making the List returned by PreparedQuery.asList() >> work this same magic, but you can expect this in a future release. >> >> Some deeper thoughts: >> The underlying RPCs between your app and the datastore fetch results in >> batches. We fetch an initial batch of results, and once that batch has been >> consumed we fetch the next batch. But, there's nothing in the API that maps >> to these batches - it's either a List containing the entire result set or an >> Iterable/Iterator that returns Entities one at a time. An API that provides >> async access to the individual results returned by an Iterable/Iterator >> (Iterator<Future<Entity>>) doesn't really make sense since you don't know >> which call to hasNext() is going to require a new batch to be fetched, and >> without that knowledge, the knowledge of what is going to trigger something >> "expensive", you can't really make appropriate use of an asynchronous API. >> >> Going forward, we're definitely interested in exposing these batches >> directly, and an explicitly async API for these batches makes a lot of sense >> since fetching these batches would map directly to something "expensive" on >> the server side. >> >> Hope this helps, >> Max >> >> On Fri, Nov 26, 2010 at 4:41 PM, Luke <lvale...@gmail.com> wrote: >> >>> i was taking a look at the 1.4.0 javadoc for AsyncDatastoreService. i >>> see the get, put and delete operations return a Future, but the >>> prepare methods return a naked PreparedQuery object, and it doesn't >>> look like PreparedQuery has any async get methods. >>> >>> does the AsyncDatastoreService not support asynchronous queries, or is >>> there something i'm missing? >>> >>> glad to see at lets the get and put methods are async, hoping to get >>> async queries too (as well as async interfaces to more services). >>> >>> -- >>> You received this message because you are subscribed to the Google Groups >>> "Google App Engine for Java" group. >>> To post to this group, send email to >>> google-appengine-j...@googlegroups.com. >>> To unsubscribe from this group, send email to >>> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> >>> . >>> For more options, visit this group at >>> http://groups.google.com/group/google-appengine-java?hl=en. >>> >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "Google App Engine for Java" group. >> To post to this group, send email to >> google-appengine-j...@googlegroups.com. >> To unsubscribe from this group, send email to >> google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> >> . >> For more options, visit this group at >> http://groups.google.com/group/google-appengine-java?hl=en. >> > > -- > You received this message because you are subscribed to the Google Groups > "Google App Engine for Java" group. > To post to this group, send email to > google-appengine-j...@googlegroups.com. > To unsubscribe from this group, send email to > google-appengine-java+unsubscr...@googlegroups.com<google-appengine-java%2bunsubscr...@googlegroups.com> > . > For more options, visit this group at > http://groups.google.com/group/google-appengine-java?hl=en. > -- You received this message because you are subscribed to the Google Groups "Google App Engine for Java" group. To post to this group, send email to google-appengine-j...@googlegroups.com. To unsubscribe from this group, send email to google-appengine-java+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/google-appengine-java?hl=en.