On 02/02/15 23:23, Warren Young wrote: >> The annoying thing is that when it fails, it wipes away whatever >> progress it has made. > Yes, well, that’s the nature of transactional DB updates: all or nothing.
Implied: There's only one way to use transactions when performing initial synchronizations. >> How difficult would it be to allow fossil to pick up where it left >> off in such a case? > Are you seriously asking for Fossil to allow a local clone to be in an > inconsistent state after an error? I'm working on a distributed storage system which borrows a few of the basic ideas from fossil (immutable artifacts, etc). We perform a DB commit after an entire *application layer* transactional unit has been transferred. When an initial synchronization breaks in our system, we use a simple method to skip artifacts when it resumes; we divide the timeline into week-long chunks, and hash all the artifact hashes in order of time in each chunk; once a week is encountered where the hashes don't match, we perform a fine-grained sync of the artifacts for that week. We deal with up to GB-size artifacts (indexed in DB, but not stored there), and the entire data storage can be in the region of terabytes. The system runs over the regular Internet. It must be tolerant to faults like "pull network cable", or ISP hiccups, because restarting from scratch after 90% has been synchronized simply isn't an option. In terms of the type of data, our data and fossil's data is very different, but in terms of the time it takes to synchronize large data stores/repositories, we're in the exact same situation. We don't expect synchronizations to fail; they rarely will, but it will happen sooner or later, so we were forced to find a way to skip work that has already been done. We don't remove all the work which has been done after a failed initial synchronization. Nor do we leave it in an "inconsistent state" -- we leave it in an "incomplete/resumable" state, and it's clearly tagged as such. I'm not even asking if fossil could be made fully resumable after an exit either, a "An error was detected; retry [y/n]?" would be another option. If it's impossible for fossil to do anything like that without major rewiring of the internals, then *that* would be relevant to my question. So to answer your question: No. Not that I ever implied anything of the sort. -- Kind Regards, Jan _______________________________________________ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
