Hi Everyone, Recently we discovered and inefficiency in the 2.2.X server that was slowing down our performance by 90%. We created a way to avoid the scenario where this happened and I would like to share that with the community and see if the team is willing to address ir (If they have not already done so in 3.0).
This happens when working with fairly large transactions of 1000 documents or more. The root cause is a loop in OAbstractPaginatedStorage.commit(OTransaction, Runnable) over a collection named newRecords (all new records created by the transaction?). Inside this loop is a call to OTransactionRealAbstract.updateIdentityAfterCommit(ORID, ORID). That method in turn has a loop over a collection named indexEntries (all indexed records modified by the transaction?). So, when inserting into an indexed class, it appears that these methods have O(n^2) performance where n is the number of records created by the commit. When the number of records created by a commit is a few thousand or more, this nested loop starts to dominate the sampler profile in Visual VM, and throughput (in records per second) gets worse as the number of records per commit grows. To counter this we count the number of documents in a transaction (For us this can vary greatly) and hard-commit when we reach the 1k document limit as we have seen performance drop severely after that. Hop this helps. - Stefán Baxter -- --- You received this message because you are subscribed to the Google Groups "OrientDB" group. To unsubscribe from this group and stop receiving emails from it, send an email to orient-database+unsubscr...@googlegroups.com. For more options, visit https://groups.google.com/d/optout.
