On Nov 16, 10:05 pm, Anders <[EMAIL PROTECTED]> wrote: > And another thing: In a distributed system, must there really be any > functional difference between RAM and disk memory? Of course, disk > access is magnitudes slower than accessing RAM, but as I see it, at > least theoretically, in a distributed system RAM could be made to > function as a permanent storage. > > Think of a single huge distributed virtual memory where data is stored > in a duplicated and fail-safe way. Then the disk space would only be > used for swapping memory pages to and from faster memory such as RAM > (and in the future maybe even other forms of fast memory). Only at the > very lowest level (Distributed OS 'kernel' level) would developers > need to know if some chunk of memory is physically stored on disk or > on some other form of memory.
The limitations and design concerns you're talking about here are not unique to App Engine. The tradeoffs between disk and memory, and the tradeoffs between different kinds of cache (e.g. disk buffer cache, L1/ L2, memcache), are core computer science. What's the latency? The throughput? The cache-hit rate? Does the kernel or does the app have a better understanding of the data access patterns? If the operating system was not able to solve these issues and remove them from the mind-set of the developer, then it's unlikely that App Engine will :-P In the case of automatic caching of the datastore, it's the case that the application does have a better opportunity to cache than we could from behind the scenes. The app knows better what is the tradeoff between data staleness and end-user latency. The app knows how to keep the cached data consistent with the true data. If we tried to do these things within the datastore, since it is an API that is perfectly consistent and guarantees that new writes immediately appear in query results, we wouldn't be able to achieve a good cache-hit rate as every datastore write would force us to invalidate every single query-cache for an app. The app can do much better than we can for datastore caching. Jon --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Google App Engine" group. To post to this group, send email to google-appengine@googlegroups.com To unsubscribe from this group, send email to [EMAIL PROTECTED] For more options, visit this group at http://groups.google.com/group/google-appengine?hl=en -~----------~----~----~----~------~----~------~--~---
