What about the "partition" blobs into evenly sized shards and distribute them over more nodes. Then you can adapt this to the block size your neo4j-store is optimized for and the cache penalties should be that bad.
Just link the nodes that contain the block data, but also make them all accessible from a "blob"-root e.g. via numbered relationships. And then retrieve the whole binary data via a traverser. Cheers Michael Am 06.02.2011 um 22:22 schrieb Rick Bullotta: > Axel/Michael: > > The real issue to consider is that Neo4J is not well suited for storing > large strings/blobs of data. It is very easy to do, but not really an > optimal use of Neo4J. The transaction logs and in-memory caches will > quickly get clogged with the large property values. It is probably better > to create an abstraction between the "raw" document storage and the "access" > veneer, the latter of which Neo is a fine choice. You could use any K-V > store, document store, or even the file system itself, and get the "best of > both worlds". What I do think, however, is that Neo is not the best choice > for the document storage itself (at present). > > One other approach that might work is to extend Neo to be able to tell the > Neo engine not to cache specific properties and to not place certain actions > in the transaction log. This way, Neo itself might be used for storage, > without the "transaction clog" and "cache poisoning". ;-) > > Just a thought... > > Rick > > -----Original Message----- > From: user-boun...@lists.neo4j.org [mailto:user-boun...@lists.neo4j.org] On > Behalf Of Michael Hunger > Sent: Sunday, February 06, 2011 4:15 PM > To: Neo4j user discussions > Subject: Re: [Neo4j] Some brainstorming ideas - 1. Use Neo4j as Document > Store > > Axel, > > Actually you could do both. > > I collapsed the "file-system-structure" to a simple notion of collection + > key as in mongodb but of course you could use the "filesystem metaphor" to > model the b-tree that resolves a key that represents a "path" to the > document node. > > Michael > > > Am 06.02.2011 um 22:09 schrieb Axel Morgner: > >> Hi Michael, >> >> just for clarification: Do you refer to how to store the internal >> structure of BSON documents, or how to store documents in a >> filesystem-like tree structure? >> >> Axel >> >> On 06.02.2011 21:50, Michael Hunger wrote: >>> Hi there, >>> >>> this weekend I was away from a computer so I spent some time > brainstorming things. I would be interested in any feedback on those > thoughts and would like to broaden the audience beyond the devteam. >>> >>> My question: Has anyone ever tried to use Neo4j as a document store ala > MongoDB. >>> >>> The idea. From the root node have relationships to a number of buckets or > "collections". >>> >>> From each collection root have a b-tree to all key-nodes of the system. >>> >>> Each key-node is just connected by some specfic relationships (like > "contains" or "link" or "subdocument") to its data. (i.e. nodes, > relationships and their properties) not necessarily to other parts of the > graph. >>> >>> So for each key node one uses just a default traverser (depth-first, end > of graph, limited to outgoing relationships of the mentioned types) to > collect the whole document (or custom traversers to retrieve only parts of > the documents). >>> >>> That could be an interesting experiment to run. >>> >>> WDYT? >>> >>> Michael >>> _______________________________________________ >>> Neo4j mailing list >>> User@lists.neo4j.org >>> https://lists.neo4j.org/mailman/listinfo/user >> >> _______________________________________________ >> Neo4j mailing list >> User@lists.neo4j.org >> https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
