Hi Team, In Oak many a times we perform operations which traverse the tree as part of some processing. For e.g. commit hooks, side grade, indexing etc. For small tree this works fine and in case of failure the processing can be done again from start.
However for large operations like reindexing whole repository for some index this posses a problem. For example consider a Mongo setup having 100M+ nodes and we need to provision a new index. This would trigger an IndexUpdate which would go through all the nodes in the repository (in some depth first manner) and then build up the index. This process can take long time say 1-2 days for a Mongo based setup. As with any remote setup such a flow may get interrupted due to some network issue or outage on Mongo/RDB side. In such a case the whole traversal is started again from start. Same would be the case for any sidegrade operation where we convert a big repository from one form to another. To improve the resiliency of such operations (OAK-2063) we need a way to "resume" traversal in a tree from some last known point. For operations performed on a sorted list such a "resume" is easy but doing that over a tree traversal looks tricky. Thoughts on what approach can be taken for enabling this? May be if we can expect a stable order in traversal at a given revision then we can keep track of paths t certain depth and then on retry skip processing of subtrees untill untill we get that path Chetan Mehrotra
