On Tue, 28 Aug 2012 16:31:20 +0200, Martijn Pieters <m...@zopatista.com> wrote : > Anything else different? Did you make any performance comparisons > between RelStorage and NEO?
I believe the main difference compared to all other ZODB Storage implementation is the finer-grained locking scheme: in all storage implementations I know, there is a database-level lock during the entire second phase of 2PC, whereas in NEO transactions are serialised only when they alter a common set of objects. This removes an argument in favour of splitting databases (ie, using mountpoints): evading the tpc_vote..tpc_finish database-level locking. Also, NEO distributes objects over several servers (aka, some or all servers might not contain the whole database), for load balancing/ parallelism purposes. This is not possible if one relies on relational database replication alone. I forgot in the original mail to mention that NEO does all conflict resolutions on client side rather than server side. The same happens in relStorage, but this is different from ZEO. Packing on client side makes it easier to get the setup right: with ZEO you will get more conflicts than normal if it cannot load some class which implements conflict resolution, and this might go unnoticed until someone worries about a performance drop or so. With client-side resolution, if you don't see Broken Objects, conflict resolution for those classes works. Some comments on some points you mentioned: > * NEO supports MySQL and sqlite, RelStorage MySQL, PostgreSQL and > Oracle. It should be rather easy to adapt to more back-ends. We (Nexedi) are not interested in proprietary software, so we will probably not implement Oracle support ourselves. For PostgreSQL, it's just that we do not have a setup at hand and the experience to implement a client properly. I expect that it would not take more than a week to get PostgreSQL implemented by someone used to it and knowing python, but new to NEO. Just to demonstrate that NEO really does not rely on fancy features of SQL servers, you may dig in older revisions in NEO's git repository. You can find a "btree.py"[1] test storage, which is based on ZODB.BTree class. It was just a toy, without persistence support (I initially intended to provide it, but never finished it) and hence limited by the available amount of RAM. But it was otherwise a fully functional NEO storage backend. I think it took me a week-end to put it together, while discovering ZODB.Btree API and adapting NEO's storage backend API along the way (this was the first non-MySQL backend ever implemented, so API was a bit too ad-hoc at that time). sqlite was chosen as a way to get rid of the need to setup a stand-alone SQL server in addition to NEO storage process. We are not sure yet of how well our database schema holds when there are several (10+) GB of data in each storage node. > * RelStorage can act as a BlobStorage, NEO can not. I would like to stress that this has nothing to do with design, rather it's just not implemented. We do not wish to rely on filesystem-level sharing, so we consider something along the lines of providing a FUSE-based to share blob storage, which then can abstract the blobs being distributed over several servers. This is just the general idea, we don't have much experience with blob handling ourselves (which is why we preferred to leave it asides rather than providing an unrealistic - and hence unusable - implementation). [1]http://git.erp5.org/gitweb/neoppod.git/blob/75d83690bd4a34cfe5ed83c949e4a32c7dec7c82:/neo/storage/database/btree.py Regards, -- Vincent Pelletier ERP5 - open source ERP/CRM for flexible enterprises _______________________________________________ For more information about ZODB, see http://zodb.org/ ZODB-Dev mailing list - ZODB-Dev@zope.org https://mail.zope.org/mailman/listinfo/zodb-dev