Hi Andrey, About db size if you just insert records in database and then close it you will have the same 142 Mb as if you would delete all the records, additional 3Gb is space consumed by write ahead log if you kill process caches will not be flushed and db will restore itself from operation log.
About your cache size (actually you can read about all mechanics here https://github.com/orientechnologies/orientdb/wiki/plocal-storage-components) but operation log records are queued into cache and then are flushed asynchronously with 1 second interval so if your cache is not filled up for 1 second you never work with disc, only with memory, so you do not suffer from disk I/O overhead. About your test try do not remove all records (OrientDВ does not gives space back) but truncate cluster after test, you removed a lot of records and database attempts to reuse given space which usually requires more operations, so if you still wish to remove all records and insert them back just increase WAL cache size (set system property -Dstorage.wal.cacheSize=6000) meanwhile I will look on your test tomorrow to verify that it is expected behavior. On Wed, Jan 29, 2014 at 8:02 PM, Andrey Yesyev <[email protected]>wrote: > Some more data. > > After I added 1 million records - DB size grew to 133M > I deleted all those records (delete from V) - DB size grew to 479M > I added 1 million records more - DB size grew to 4.4Gb !!!! > I deleted them all - DB size dropped to 4.2Gb > I restarted DB - DB size after restart 142M (There is no data in DB at > this point) > I added 1 million records - DB size grew to 4.3Gb again !!!! > > Every time I was adding data I got those messages > > *2014-01-29 12:41:56:433 INFO Max cache limit is reached (3000 vs. 3329), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:02:690 INFO Max cache limit is reached (3000 vs. 3325), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:11:614 INFO Max cache limit is reached (3000 vs. 3327), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:16:327 INFO Max cache limit is reached (3000 vs. 3327), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:29:467 INFO Max cache limit is reached (3000 vs. 3327), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:45:358 INFO Max cache limit is reached (3000 vs. 3327), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:49:333 INFO Max cache limit is reached (3000 vs. 3035), > sync flush is performed. [OWriteAheadLog$LogSegment]* > *2014-01-29 12:42:57:850 INFO Max cache limit is reached (3000 vs. 3285), > sync flush is performed. [OWriteAheadLog$LogSegment]* > > and insertion rate stayed the same low ~1600 records/sec. > > Just in case, here is how I inserting data > > private void addIdentities__(List<Identity> identities){ > for(Identity identity:identities){ > String clusterName = > identity.getCollection().getCase().getName().toLowerCase(); > if(!db.getClusterNames().contains(clusterName)) > db.addCluster(clusterName, OStorage.CLUSTER_TYPE.PHYSICAL); > Vertex v = graph.addVertex(classes.get(identity.getType())); > v.setProperty(Fields.NAME, identity.getNormValue()); > v.setProperty(Fields.COUNT, identity.getCount()); > } > } > > public void addIdentities(List<Identity> identities){ > if(graph instanceof OrientGraph){ > int size = identities.size(); > int step = 100000; > int from = 0; > int to = step; > if(size<step) > to = size; > while(true){ > addIdentities__(identities.subList(from, to)); > graph.commit(); > if(to==size) > break; > from = to; > to += step; > if(to>size) > to = size; > } > } else if(graph instanceof OrientGraphNoTx){ > addIdentities__(identities); > } > } > > > At this test I'm using OrientGraph. > > Any ideas what's going on there? > > Thanks! > > -Andrey > > > -- > > --- > You received this message because you are subscribed to the Google Groups > "OrientDB" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > For more options, visit https://groups.google.com/groups/opt_out. > -- Best regards, Andrey Lomakin. Orient Technologies the Company behind OrientDB -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. For more options, visit https://groups.google.com/groups/opt_out.
