Re: [fossil-users] Question on repo size after repeated binary file commits?
On Sun, Dec 22, 2013 at 9:57 PM, wrote: > Really, I am only implying some minimal file statistic like 'DeltaSize(%)' > or somesuch to show the user it is in fact compared internally. The current > message contradicts what is in fact happening. Maybe change that message to > "Cannot visually display binary diffs. DeltaSize(%) = -10". > It is in principal an minimal change, but it is more invasive than it sounds because the parts which deal with "diffs" are 100% different, and separate from, those which deal with "deltas". The code which generates the diff, and outputs the "cannot..." message does not have enough info to know about the underlying delta (nor whether there is in fact any delta at all - there is not always one). -- - stephan beal http://wanderinghorse.net/home/stephan/ http://gplus.to/sgbeal "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of those who insist on a perfect world, freedom will have to do." -- Bigby Wolf ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
Really, I am only implying some minimal file statistic like 'DeltaSize(%)' or somesuch to show the user it is in fact compared internally. The current message contradicts what is in fact happening. Maybe change that message to "Cannot visually display binary diffs. DeltaSize(%) = -10". On Sun, Dec 22, 2013 at 3:15 PM, Stephan Beal wrote: > On Sun, Dec 22, 2013 at 9:06 PM, wrote: > >> Thanks. I didn't know how binary was handled given the Timeline diff >> response = "cannot compute difference between binary files". >> > > That message is a bit misleading. It really means a "visual difference." > There isn't a mechanism to show a textual diff for binaries, and fossil's > internal deltas and its text diffs are two completely different beasts. > > >> I think it would be cool if instead fossil listed some of the metrics >> used or determined in the binary delta operation. >> > > The diff-related pages don't actually use the delta code (though > diff/delta are logically similar, they are much different implementations). > A delta blob does in fact know (without expensive processing) the size of > the original blob and the size of the delta, so it might be feasible to do > that. The unsightly part is that fossil doesn't really know what's a binary > and what isn't (the delta algorithm is the same for all data). When > performing a textual diff and it runs into any binary-looking data, it > aborts the diff and assumes that it's binary. i.e. it would first need to > run through the text diff and, as a fallback, generate statistics for a > binary delta. Yeah, doable, but IMO horribly ugly because it would have to > be done as a fallback for the diff generation, making it more expensive > (computation/memory) than it really needs to be. > > -- > - stephan beal > http://wanderinghorse.net/home/stephan/ > http://gplus.to/sgbeal > "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of > those who insist on a perfect world, freedom will have to do." -- Bigby Wolf > > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users > > ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
On Sun, Dec 22, 2013 at 9:06 PM, wrote: > Thanks. I didn't know how binary was handled given the Timeline diff > response = "cannot compute difference between binary files". > That message is a bit misleading. It really means a "visual difference." There isn't a mechanism to show a textual diff for binaries, and fossil's internal deltas and its text diffs are two completely different beasts. > I think it would be cool if instead fossil listed some of the metrics used > or determined in the binary delta operation. > The diff-related pages don't actually use the delta code (though diff/delta are logically similar, they are much different implementations). A delta blob does in fact know (without expensive processing) the size of the original blob and the size of the delta, so it might be feasible to do that. The unsightly part is that fossil doesn't really know what's a binary and what isn't (the delta algorithm is the same for all data). When performing a textual diff and it runs into any binary-looking data, it aborts the diff and assumes that it's binary. i.e. it would first need to run through the text diff and, as a fallback, generate statistics for a binary delta. Yeah, doable, but IMO horribly ugly because it would have to be done as a fallback for the diff generation, making it more expensive (computation/memory) than it really needs to be. -- - stephan beal http://wanderinghorse.net/home/stephan/ http://gplus.to/sgbeal "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of those who insist on a perfect world, freedom will have to do." -- Bigby Wolf ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
Thanks. I didn't know how binary was handled given the Timeline diff response = "cannot compute difference between binary files". I think it would be cool if instead fossil listed some of the metrics used or determined in the binary delta operation. Thanks for Fossil! On Sun, Dec 22, 2013 at 2:51 PM, Stephan Beal wrote: > On Sun, Dec 22, 2013 at 8:44 PM, wrote: > >> Ah, is there a way to quantify the binary delta? >> If I have a 1MB binary file and commit a 1 byte change, what is the size >> of the computed binary delta? >> > > Very, very small: > > Create two binaries with a one-byte difference: > > [stephan@host:~/cvs/fossil/libfossil]$ cat f-tag > foo; echo -n x >> foo; > cat f-wiki >> foo > > [stephan@host:~/cvs/fossil/libfossil]$ cat f-tag > bar; echo -n y >> bar; > cat f-wiki >> bar > > > [stephan@host:~/cvs/fossil/libfossil]$ l foo bar > -rw-rw-r-- 1 stephan users 1281922 Dec 22 20:48 bar > -rw-rw-r-- 1 stephan users 1281922 Dec 22 20:48 foo > > The delta: > > [stephan@host:~/cvs/fossil/libfossil]$ ./f-delta foo bar > 4tz2 > 2Qjj@0,1:yO@0,2UEw@2Qk7,3wep03; > > Confirm that the delta actually does what's expected: > > [stephan@host:~/cvs/fossil/libfossil]$ ./f-delta foo bar > baz > [stephan@host:~/cvs/fossil/libfossil]$ ./f-delta -a baz foo > bar2 > [stephan@host:~/cvs/fossil/libfossil]$ cmp bar bar2 > [stephan@host:~/cvs/fossil/libfossil]$ echo $? > 0 > > To answer your question: 37 bytes: > > [stephan@host:~/cvs/fossil/libfossil]$ l baz > -rw-rw-r-- 1 stephan users 37 Dec 22 20:49 baz > > -- > - stephan beal > http://wanderinghorse.net/home/stephan/ > http://gplus.to/sgbeal > "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of > those who insist on a perfect world, freedom will have to do." -- Bigby Wolf > > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users > > ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
On Sun, Dec 22, 2013 at 8:44 PM, wrote: > Ah, is there a way to quantify the binary delta? > If I have a 1MB binary file and commit a 1 byte change, what is the size > of the computed binary delta? > Very, very small: Create two binaries with a one-byte difference: [stephan@host:~/cvs/fossil/libfossil]$ cat f-tag > foo; echo -n x >> foo; cat f-wiki >> foo [stephan@host:~/cvs/fossil/libfossil]$ cat f-tag > bar; echo -n y >> bar; cat f-wiki >> bar [stephan@host:~/cvs/fossil/libfossil]$ l foo bar -rw-rw-r-- 1 stephan users 1281922 Dec 22 20:48 bar -rw-rw-r-- 1 stephan users 1281922 Dec 22 20:48 foo The delta: [stephan@host:~/cvs/fossil/libfossil]$ ./f-delta foo bar 4tz2 2Qjj@0,1:yO@0,2UEw@2Qk7,3wep03; Confirm that the delta actually does what's expected: [stephan@host:~/cvs/fossil/libfossil]$ ./f-delta foo bar > baz [stephan@host:~/cvs/fossil/libfossil]$ ./f-delta -a baz foo > bar2 [stephan@host:~/cvs/fossil/libfossil]$ cmp bar bar2 [stephan@host:~/cvs/fossil/libfossil]$ echo $? 0 To answer your question: 37 bytes: [stephan@host:~/cvs/fossil/libfossil]$ l baz -rw-rw-r-- 1 stephan users 37 Dec 22 20:49 baz -- - stephan beal http://wanderinghorse.net/home/stephan/ http://gplus.to/sgbeal "Freedom is sloppy. But since tyranny's the only guaranteed byproduct of those who insist on a perfect world, freedom will have to do." -- Bigby Wolf ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
Ah, is there a way to quantify the binary delta? If I have a 1MB binary file and commit a 1 byte change, what is the size of the computed binary delta? You are correct of course, but I tend not to extend the spirit of fossil to binary files and images. It is their existence and not legacy that is critical in my application. An example would be icons used for toolbars and menus. If I tweak them or add layering, I sorta obsolete the originals. Here, I prefer the smaller repo. On Sun, Dec 22, 2013 at 1:46 PM, Mark Janssen wrote: > > > On Sun, Dec 22, 2013 at 7:37 PM, wrote: > >> I am curious what is stored in the repo for each new commit that includes >> a tiny change to a binary file. >> Whether a dll or an image file, is fossil storing each binary file >> compressed, uncompressed or some sort of delta? >> Over time(6mo's to 1yr), I would like to reduce my repo size by purging >> really old binary files. >> >> Thanks for fossil! >> > > Fossil does have delta encoding but I am not sure whether this is used > for binary files. However part of the design philosophy of Fossil is that > no history is ever lost. So reducing the repository size is generally not > possible. > > Mark > > ___ > fossil-users mailing list > fossil-users@lists.fossil-scm.org > http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users > > ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
On Sun, Dec 22, 2013 at 7:37 PM, wrote: > I am curious what is stored in the repo for each new commit that includes > a tiny change to a binary file. > Whether a dll or an image file, is fossil storing each binary file > compressed, uncompressed or some sort of delta? > Over time(6mo's to 1yr), I would like to reduce my repo size by purging > really old binary files. > > Thanks for fossil! > Fossil does have delta encoding but I am not sure whether this is used for binary files. However part of the design philosophy of Fossil is that no history is ever lost. So reducing the repository size is generally not possible. Mark ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
Re: [fossil-users] Question on repo size after repeated binary file commits?
On Sun, Dec 22, 2013 at 1:37 PM, wrote: > I am curious what is stored in the repo for each new commit that includes > a tiny change to a binary file. > Whether a dll or an image file, is fossil storing each binary file > compressed, uncompressed or some sort of delta? > A binary delta. > Over time(6mo's to 1yr), I would like to reduce my repo size by purging > really old binary files. > You can do this in Fossil. But doing so violates the spirit and intent of Fossil. The intent is that you only commit to a fossil repository what you want to keep forever. If you have transient files that you only want to keep for 6 to 12 months, you would do well to store them separately. -- D. Richard Hipp d...@sqlite.org ___ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
