---------- Forwarded message ---------- From: Alexej Sailer <alexejsai...@gmail.com> Date: 2016-08-02 10:13 GMT+02:00 Subject: Speed up initial Bundle load > 10k from database To: us...@jackrabbit.apache.org
I'm using jackrabbit and from time to time it happens, that I have more than 10k Childnodes. So I noticed that in this case jackrabbit slows down loading them initially into the bundle cache. After debugging I found the method protected NodePropBundle loadBundle(NodeId id) in OraclePersistenceManager and there, every node bundle is loaded one by one, which is extremely slow after 2k bundles. So I wrote a simple workaround: I start a bunch off threads at exactly this method and load all bundles from the database at once for 60 000 it takes me 5 sec I think, and I cache them exactly in this place, so that If I start loading > 10k child nodes one by one, they are loaded from this cache. And that speeds up the load 10x. This is simply a bundle cache but, which is filled up extremely fast. Now, the cache invalidation is done by using protected and public methods from PersistenceManager. It works fine. But I am not sure whether this is consistent in Apache JackRabbit implementation. Who can answer this questions? I will provide the code as well, if there are interests to that. BR, Alexej Sailer protected synchronized void storeBundle(NodePropBundle bundle) throws ItemStateException { AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(bundle.getId().toString()); super.storeBundle(bundle); } @Override protected synchronized void destroyBundle(NodePropBundle bundle) throws ItemStateException { AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(bundle.getId().toString()); super.destroyBundle(bundle); } @Override protected void evictBundle(NodeId id) { AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(id.toString()); super.evictBundle(id); } @Override public synchronized void onExternalUpdate(ChangeLog changes) { for (ItemState state : changes.modifiedStates()) { AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(getBundleId(state).toString()); } for (ItemState state : changes.deletedStates()) { AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(getBundleId(state).toString()); } for (ItemState state : changes.addedStates()) { AsyncBundleReader.getNodePropBundleCache().getCache().invalidate(getBundleId(state).toString()); } super.onExternalUpdate(changes); }