On Tue, May 22, 2018 at 8:30 PM, Davor Josipovic <dav...@live.com> wrote: >> Now that is interesting. 40k doesn't seem to be such a large amount of >> data for modern computers. Very slow and fragmented hard drive? Or perhaps >> there's something else going on that is manifesting this way? > > The HDD is indeed on the slowside, and together with low memory... > > But I think this also show how I/O intensive SVN is. On the client side, for > each committed file, one copy is placed in .svn folder, and an other copy in > a temporary folder (which is deleted after file transfer in v1.9). So for > each file committed, a double copy is made client-side. This temporary copy > is really necessary? > > Server-side, I see similar disk bashing. For each committed file, max 2 (?) > copies are made in the transaction directory. > > So any way to reduce the I/O?
A couple of improvements have been made in 1.10. It would be interesting to test this same scenario with a 1.10 client (to verify the client-side improvements) and if possible also with a 1.10 server, with a dumped+loaded FSFS back-end (to get an idea of any server-side improvements). Client-side there were some changes to make the (http) client stream svndiff deltas without creating temporary files (during 'commit' and 'import'): https://svn.apache.org/r1803143 https://svn.apache.org/r1803144 https://svn.apache.org/r1804819 Server-side, LZ4 compression was added as the default compression algorithm in the FSFS back-end, and on the wire (not sure if that helps in your case ... it won't impact the number of files that are created temporarily I suppose): https://subversion.apache.org/docs/release-notes/1.10.html#lz4 Also, maybe a commit such as this one helps too (though it seems to only talk about reads): https://svn.apache.org/r1759399 As always, probably further improvements can be made I guess, but someone will need to dig into it. If you'd be able to test with 1.10 both client and server-side, that would give some more data points about where we are with the latest state of things ... Finally, one last tip: this (rather old) performance tuning page suggests putting the "transactions" folder on a separate (faster) filesystem to speed up commits: https://www.orcaware.com/svn/wiki/Server_performance_tuning_for_Linux_and_Unix#Reducing_the_number_of_writes I don't know how realistic that is in your setup (plus you need to make sure that transactions filesystem is large enough for your commits). -- Johan