On Thu, Oct 6, 2011 at 4:47 PM, Erlis Vidal <er...@erlisvidal.com> wrote:
> Do you know about "protocol buffers". I'm asking because I didn't know > about them since recently. > > > http://google-opensource.blogspot.com/2008/07/protocol-buffers-googles-data.html > Never heard of them. Thanks for the link :). > Could this help in order to allow binaries and improve the serialization > performance? > No idea just yet :). But i will take a look at that. One thing to remember is that JSON != JavaScript, meaning that if we include ANY algorithms for binary data, ALL clients which want to use fossil/json HAVE to implement those algorithms. e.g. if we decide on base64, ALL client languages have to support it if they want to use those features of the API. Because of this, i would like to stick "non-formatted" data where possible, falling back to non-JSON-specified algos only when absolutely necessary. My current thinking on binary is that it "probably should" be encoded the same way one encodes binary in sqlite: X'aabbccdd', i.e. a simple list of hex value pairs. The down-side is that it literally doubles the size, which means that those fossil users which version-control 500MB files (which is insane, IMO, but people do it) will have a memory cost of approximately 3x that file's size when turning the artifact into JSON (1x for the original copy and 2x for the X'...' encoding, and possibly even more for the underlying sqlite statement to/from that data). Try to commit a 2GB file and you can't - your machine (or the remote fossil server) will die with an OOM error. The up-side is that en/decoding it is absolutely trivial - even an absolute beginner could figure out how to encode/decode it and it can be done in any language which supports binary data (e.g. NOT in shell code, but in JS/Python/Perl/PHP/Java, etc). -- ----- stephan beal http://wanderinghorse.net/home/stephan/
_______________________________________________ fossil-users mailing list fossil-users@lists.fossil-scm.org http://lists.fossil-scm.org:8080/cgi-bin/mailman/listinfo/fossil-users
