On 21/11/15 22:15, Karl Berry wrote: > ag> of its new atomic file-swap ability: > http://lists.gnu.org/archive/html/coreutils/2015-11/msg00068.html > > Nice, but no "new" facility can be assumed for general use. That's why > I hoped there would be a gnulib module that make use of such fancy new > facilities where possible, and fall back to whatever > best-possible-attempt otherwise. To save Gavin the time, trouble, and > exposure of trying to implement it. This is not a new problem. One > would sure think it's been solved somewhere already in GNU and can be > reused. > > mthl> There is a big difference between the behavior expected at > compile time and install time. > > Oh, whatever. Using gcc was just an example. Same thing if I type > "make install" and hit CTRL-C (or CTRL-\ or kill -9 from another > terminal or unplugging the computer or whatever) in the middle of > installing some huge binary: I don't expect it to remain uncorrupted. > And, in practice, I believe that it is likely to get corrupted. > > You will want every installation operation to be > atomic to garantee the integrity of the system. > > Sure, I agree it is desirable, but I don't believe it is, in fact, > implemented by much of anything, except (I suppose) at the > package-manager level. > > What's different about the dir file is that it amalgamates stuff from > many packages, and isn't easy to reconstruct (e.g., can't just type > "make install" again). Thus I can see that in principle it is worth > extra trouble to try to preserve. But I'm not volunteering to implement > it, so it's just an opinion ... -k
Replacing a file robustly is a tricky problem, and rarely done correctly, with most people resorting to a database for such functionality. I looked at the problem recently for http://github.com/pixelb/crudini and the implementation there uses existing file system APIs which should work on local and most contemporary network file systems. It's been extensively tested, small races found and fixed and is now seen to be robust for this task. The lineage of changes are: https://github.com/pixelb/crudini/commit/231aaafc https://github.com/pixelb/crudini/commit/8d549be8 https://github.com/pixelb/crudini/commit/df33e66a https://github.com/pixelb/crudini/commit/7108dd7e https://github.com/pixelb/crudini/commit/4e1af9ff https://github.com/pixelb/crudini/commit/10875e6a https://github.com/pixelb/crudini/commit/3f0d2051 Now while the above works, there are caveats noted for each of the provided replacement methods. Using newer apis like renameat() or exchangedata() if available would avoid most of the caveats I think. Given how tricky all this is, it really should be done once and reused, and I plan to incorporate the above mentioned functionality into a rewrite utility which can be called by whatever to replace files with ACID guarantees. I've most of the bits figured out now, so I just need some time to pull it together. cheers, Pádraig.
