Thanks for your replies, Evgeny. Replying to the "status walk" part.
(TL;DR: Could we optimise by doing the db scan before the stat?) I think we want to ensure, as far as possible, - no significant performance degradation if user does not opt in to the new feature; - overhead when enabled should not be disproportionate to the operation. We can achieve the former simply by skipping the scanning/syncing steps entirely when the user has not chosen pristines-on-demand mode. That is fine. About the overhead, you suggest the overhead does not seem too high, at least for now, and maybe you are right. But it can be subjective: some people have considered the status walk too costly in the past and have dedicated a lot of effort to reducing it. I have some ideas about reducing this overhead anyway. Some ways we could potentially optimise when it's enabled are: - Fetch pristines in a backgroud thread; - Further limit the scan (by depth etc.); - Scan for missing pristines (quick db op) before statting files for mods. We could fetch pristines in a backgroud thread, making the "foreground" operation thread wait, just in time, for each pristine before accessing it. Positives: the end result is efficient. Negatives: we don't have precendent for threaded operations, and they can be tricky, so unknown and potentially large effort to complete it. We could further limit the scan (by depth etc.). Positives: easy to implement some steps in this direction. Negatives: only ever gets us closer, but never all the way, towards fetching only what is really needed; and risk of introducing buggy cases where it fetches too little. We could swap the scanning logic around to do the (quick) check for missing pristines before deciding whether a (slower) file "stat" is necessary. Positives: eliminates the redundant "stat" overhead which may be significant in working trees containing many files. Negatives: some re-work needed in the current implementation. Of these, the last one currently looks viable and useful. Does that one look promising to you? - Julian
