On 6/11/2013 8:52 AM, C. Michael Pilato wrote:
One advantage of being in a room full of Subversion developers, specifically
the guy that implemented all this stuff, is that I can ask him directly
about how to respond to this mail. :-) Hopefully I will accurately
represent the answers Stefan Fuhrmann just gave me to your questions.
On 06/10/2013 03:05 PM, Thomas Harold wrote:
a) Why are directory/property deltifications turned off by default?
Stefan's jovial first answer was, "Because I'm a chicken." :-)
Fortunately, he didn't stop there. He explained that there is no known
correctness risk -- you're not going to damage your repositories by enabling
the feature. But he wanted to phase in the feature to allow time to collect
real-world data about the amount of space savings folks are actually
observing in their repositories. The feature is on by default in his
proposed next version of the FSFS format.
b) Is there a global setting file that can be used to enable
directory/property deltifications? Or will we have to update the fsfs.conf
file for each newly created repository in order to get this feature?
In 1.8, you'll need to toggle this for each new repository manually.
So in order to get full use of the "Directory and property storage
reduction" in a retroactive manner as noted in:
http://subversion.apache.org/docs/release-notes/1.8.html#fsfs-enhancements
We will need to:
1. svnadmin dump oldreponame
2. svnadmin create newreponame
3. Modify db/fsfs.conf
[rep-sharing]
enable-rep-sharing = true # defaults to true in 1.8
[deltification]
enable-dir-deltification = true
enable-props-deltification = true
This can be done in automated fashion with sed (or awk). For example,
to change compress-packed-revprops:
sed 's/^[#[:space:]]*compress-packed-revprops =
false[#[:space:]]*$/compress-packed-revprops = true/g' --in-place=bkp
newreponame/db/fsfs.conf
Or to enable deltification (as well as enable-rep-sharing) all at once:
sed 's/^[#[:space:]]*enable-rep-sharing =
false[#[:space:]]*$/enable-rep-sharing =
true/g;s/^[#[:space:]]*enable-dir-deltification =
false[#[:space:]]*$/enable-dir-deltification =
true/g;s/^[#[:space:]]*enable-props-deltification =
false[#[:space:]]*$/enable-props-deltification = true/g' --in-place=bkp
newreponame/db/fsfs.conf
Naturally, you should have a backup of your db/fsfs.conf file. While
sed creates one with --in-place=bkp, you should not rely solely on that
method.
4. svnadmin load newreponame
5. svnadmin pack newreponame
- Not all of our repos were packed
Even without turning on deltification, I'm seeing a size difference
between our 1.7 repositories and the ones that we loaded in from dump
files in 1.8.1. Possibly from enable-rep-sharing now being set to
'true' by default in 1.8.
Test repo #1 went from 2.0G to 1.7G (85% of original).
Test repo #2 went from 2.4G down to 2.1G (88% of original). File count
in repo #2's fsfs folder only went from 3808 to about 3684 (1820
revisions). Even though we did "svnadmin load" using 1.8.1, I still had
to "svnadmin pack" to cut the file count down to 1692.
Test repo #3 is 10202MB with 47199 files and 45990 revisions (in 1.6 /
1.7). The revs had been packed, but not the revprops under the old
system. The gzip'd dump file is 35.8GB. I enabled the two
deltification options in db/fsfs.conf before doing the "svnadmin load"
step. Finished size was 7330MB (72% of original) with 92108 files.
File count drops to 2380 after packing and repository size drops to
7072MB (69% of original).
