On Wed, Oct 20, 2010 at 11:22, Thomas Müller <thomas.muel...@day.com> wrote: > Let's discuss partitioning / sharding in another thread. Asynchronous > change merging is not about how to manage huge repositories (for that > you need partitioning / sharding),
Having different consistency expectations for certain parts in order to make certain writes faster is not necessarily about huge repositories. > But I don't want that "the network is the new disk" (in terms of > bottleneck, in terms of performance problem). Networking delay may be > the bottleneck even if cluster nodes are in the same room, specially > when you keep the whole repository in memory, or use SSDs. Network delay in a proper setup is faster than the delay of a disk drive nowadays (leaving out the other end of the network that has to store the data it receives). For some numbers, see http://www.tele-task.de/archive/video/flash/11097/ slide starting at about 3:21 (until ~7:00). This underlies the importance of leveraging in-memory storage, btw, as Jukka also pointed out. > Asynchronous operation is bad for reservation systems, banking > applications, or if you can't guarantee sticky sessions. Here you need > synchronous operations or at least locking. If you want to support > both cases in the same repository, you could use virtual repositories > (which are also good for partitioning / sharding). We could use virtual repositories for that, but that should be more easily configurable then and make sure it is performant. Still, it would need to be configurable. > I think the persistence API should be synchronous > as it is now. What is then the reason for asynchronous change merging, if not for performance? AFAIU, the synchronous operation of the (cloud) persistence layer makes it quite slow compared to a pure local storage (especially if that one is in-memory). Regards, Alex -- Alexander Klimetschek alexander.klimetsc...@day.com
