2010/6/8 <rick.bullo...@burningskysoftware.com>: > Hi, Johan. > > > > In the scenario you describe below, if one attempts to open an > existing Neo DB that has been created with a "non-standard" block size, > will it be able to get that information from the DB itself or must you > provide that information when you attempt via config parameters to open > it? >
It will read the config parameter the first time (when the neo4j database is created) and store those block size config values in the store itself. This means that if you open the database the next time those config parameters will be ignored and the values used at creation time will be used. > > > Thanks, > > > > Rick > > > > > > > > -------- Original Message -------- > Subject: Re: [Neo4j] Node creation limit > From: Johan Svensson <[1]jo...@neotechnology.com> > Date: Tue, June 08, 2010 5:15 am > To: Neo4j user discussions <[2]u...@lists.neo4j.org> > I just added code in trunk so block size for string and array store > can be configured when the store is created. This will be available in > the 1.1 release but if you want to try it out now use 1.1-SNAPSHOT and > create a new store like this: > Map<String,String> config = new HashMap<String, String>(); > config.put( "string_block_size", "60" ); > config.put( "array_block_size", "300" ); > // create a new store with string block size 60 and array block size > 300 > new EmbeddedGraphDatabase( "path-to-db-that-do-not-exist", config > ).shutdown(); > The default value (120 bytes) was picked to fit common/avg size > string/array properties in one block since it will be slower to load a > property that is spread out on many blocks. Since datasets vary a lot > in string size / array size and the like I think it is better to have > it configurable at creation time. When tweaking these values remember > that strings will consume twice the string length in bytes so a string > block size of 60 will be able to fit a string of length 30 in a single > block. > Regarding scaling 1.0 and 1.1 releases have a limit of 4 billion > records / store file so if you need to store 4 billion strings you > have to make sure every string fits in a single block. This limit will > be increased to 32 billion or more in the 1.2 release. > -Johan > On Mon, Jun 7, 2010 at 4:27 PM, Biren Gandhi > <[3]biren.gan...@gmail.com> wrote: > > Similar issue on my side as well. Test data is ok, but production > data > > (100 million+ objects, 200 relationships per object and 10 properties > > per object, with multi-million queries per day about search and > > traversal) would need clear disk sizing calculations due to iops and > > other hardware limits in a monolithic storage model. > > > > Has anyone been able to use neo4j succeessfully in scaling needs > > similar to mentioned avove? > > > > -b > > > > On Jun 7, 2010, at 4:45 AM, Craig Taverner <[4]cr...@amanzi.com> > wrote: > > > >>> Is there a specific constrain on disk space? Normally disk space > >>> isn't > >>> a problem... it's cheap and there's usually loads of it. > >>> > >> > >> Actually for most of my use cases the disk space has been fine. > >> Except for > >> one data source, that surprised me by expanding from less than a gig > >> of > >> original binary data, to over 20GB database. While this too can be > >> managed, > >> it was just a sample, and so I have yet to see what the customers > >> 'real > >> data' will do to the database (several hundred times larger, I'm > >> expecting). > >> When we get to that point we will need to decide how to deal with > it. > >> Currently we 'solve' the issue by allowing the user to filter out > >> data on > >> import, so we don't store everything. This will not satisfy all > users, > >> however. > _______________________________________________ > Neo4j mailing list > [5]u...@lists.neo4j.org > [6]https://lists.neo4j.org/mailman/listinfo/user > > References > > 1. mailto://jo...@neotechnology.com/ > 2. mailto://user@lists.neo4j.org/ > 3. mailto://biren.gan...@gmail.com/ > 4. mailto://cr...@amanzi.com/ > 5. mailto://User@lists.neo4j.org/ > 6. https://lists.neo4j.org/mailman/listinfo/user > _______________________________________________ > Neo4j mailing list > User@lists.neo4j.org > https://lists.neo4j.org/mailman/listinfo/user > -- Mattias Persson, [matt...@neotechnology.com] Hacker, Neo Technology www.neotechnology.com _______________________________________________ Neo4j mailing list User@lists.neo4j.org https://lists.neo4j.org/mailman/listinfo/user
