Thanks Andrey! The only reason I remove all the records, just make sure that tests are run at the same environment. Ok, I think I have to just re-create DB before each test.
I provided DB size changes just FYI. Assuming it can give you a clue what's going on. On Wednesday, January 29, 2014 1:33:18 PM UTC-5, Andrey Lomakin wrote: > > Hi Andrey, > > About db size if you just insert records in database and then close it you > will have the same 142 Mb as if you would delete all the records, > additional 3Gb is space consumed by write ahead log if you kill process > caches will not be flushed and db will restore itself from operation log. > > About your cache size (actually you can read about all mechanics here > https://github.com/orientechnologies/orientdb/wiki/plocal-storage-components) > but operation log records are queued into cache and then are flushed > asynchronously with 1 second interval so if your cache is not filled up for > 1 second you never work with disc, only with memory, so you do not suffer > from disk I/O overhead. > > About your test try do not remove all records (OrientDВ does not gives > space back) but truncate cluster after test, you removed a lot of records > and database attempts to reuse given space which usually requires more > operations, so if you still wish to remove all records and insert them back > just increase WAL cache size (set system property > -Dstorage.wal.cacheSize=6000) meanwhile I will look on your test tomorrow > to verify that it is expected behavior. > > > > > On Wed, Jan 29, 2014 at 8:02 PM, Andrey Yesyev > <[email protected]<javascript:> > > wrote: > >> Some more data. >> >> After I added 1 million records - DB size grew to 133M >> I deleted all those records (delete from V) - DB size grew to 479M >> I added 1 million records more - DB size grew to 4.4Gb !!!! >> I deleted them all - DB size dropped to 4.2Gb >> I restarted DB - DB size after restart 142M (There is no data in DB at >> this point) >> I added 1 million records - DB size grew to 4.3Gb again !!!! >> >> Every time I was adding data I got those messages >> >> *2014-01-29 12:41:56:433 INFO Max cache limit is reached (3000 vs. 3329), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:02:690 INFO Max cache limit is reached (3000 vs. 3325), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:11:614 INFO Max cache limit is reached (3000 vs. 3327), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:16:327 INFO Max cache limit is reached (3000 vs. 3327), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:29:467 INFO Max cache limit is reached (3000 vs. 3327), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:45:358 INFO Max cache limit is reached (3000 vs. 3327), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:49:333 INFO Max cache limit is reached (3000 vs. 3035), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> *2014-01-29 12:42:57:850 INFO Max cache limit is reached (3000 vs. 3285), >> sync flush is performed. [OWriteAheadLog$LogSegment]* >> >> and insertion rate stayed the same low ~1600 records/sec. >> >> Just in case, here is how I inserting data >> >> private void addIdentities__(List<Identity> identities){ >> for(Identity identity:identities){ >> String clusterName = >> identity.getCollection().getCase().getName().toLowerCase(); >> if(!db.getClusterNames().contains(clusterName)) >> db.addCluster(clusterName, >> OStorage.CLUSTER_TYPE.PHYSICAL); >> Vertex v = graph.addVertex(classes.get(identity.getType())); >> v.setProperty(Fields.NAME, identity.getNormValue()); >> v.setProperty(Fields.COUNT, identity.getCount()); >> } >> } >> >> public void addIdentities(List<Identity> identities){ >> if(graph instanceof OrientGraph){ >> int size = identities.size(); >> int step = 100000; >> int from = 0; >> int to = step; >> if(size<step) >> to = size; >> while(true){ >> addIdentities__(identities.subList(from, to)); >> graph.commit(); >> if(to==size) >> break; >> from = to; >> to += step; >> if(to>size) >> to = size; >> } >> } else if(graph instanceof OrientGraphNoTx){ >> addIdentities__(identities); >> } >> } >> >> >> At this test I'm using OrientGraph. >> >> Any ideas what's going on there? >> >> Thanks! >> >> -Andrey >> >> >> -- >> >> --- >> You received this message because you are subscribed to the Google Groups >> "OrientDB" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected] <javascript:>. >> For more options, visit https://groups.google.com/groups/opt_out. >> > > > > -- > Best regards, > Andrey Lomakin. > > Orient Technologies > the Company behind OrientDB > > -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
