On 08/09/2016 11:12, Arnt Gulbrandsen wrote: > Thomas Tempelmann writes: >> Is that just a question of manpower or is there an algorithm problem >> that's hard to solve? Or is it just the fact that the bandwidth to the >> server is limited? > > Restoring is much too slow to be explained by bandwidth. My worst > restore took several days and used less than 1% of available bandwidth. > > Single-file restores are fast (at least fast enough not to irritate me). > I think the problem is that that requests aren't pipelined. The client > asks for a file, the server retrieves the file from the storage backend, > the file is transferred, the client writes it to the file system, the > client thinks about what file to ask for next, GOTO 1. If you're > restoring half a million small files the network connection will be idle > a lot of the time. > > Life is much better if you restore a single big file (and even better if > you don't have to restore).
A question for Colin: would it be possible for tarsnap to have a faster restore mode where it simply created an unencrypted tar file on the client equivalent to a given tarsnap archive, to be restored from as necessary, or does the architecture preclude it? That would at least speed up full restores after a crash that lost everything. -- Schrödinger's cat had 18 half lives.