On Thu, 31 Mar 2011 18:56:54 +0200 Christophe-Marie Duquesne <chm.duque...@gmail.com> wrote:
> Hi, > > I am currently writing a FUSE file system based on git-annex for > replicating binary files on several machines. I thought I could share > it here in order to get some ideas and contributors. > > What are your goals? > Seamless synchronization "à la dropbox". > Ability to use with big binary files such as mp3/movies. > Entirely decentralized. > Don't use unnecessary space > Keep it simple: avoid special VCS commands and keep a filesystem > interface as much as possible. you also need to do various git/git-annex commands, or am I missing something? > Why? > Because sparkleshare and dvcs-autosync are bad at versioning binary files I quite like dvcs-autosync, but it indeed lacks space-efficient storage of big files. I would like to try if we can use git-annex to support this in dvcs-autosync, although AFAIK git-annex is not transparent in the way regular git is transparent (i.e. it needs to explicitly copy files between locations), I assume this is the reason you need to go for a FUSE-based approach? or do you just prefer this over regular fs + inotify? > Because Unison needs disk space for each couple of hosts it > synchronizes and thus does not really scales for more than 2 hosts > Because Coda is not completely decentralized and it bothers me you actually tried coda? it's something I'm interested in, on paper it looks like an awesome, maybe-even-perfect open source dropbox-clone but the reality is probably different, I never tried it so I wouldn't know. > What do you have? > A python implementation. It is about 600 sloc, and you'll find it on > https://github.com/chmduquesne/sharebox > Be careful, it is very alpha and it still does not have a proper > conflict handler. > > Hey, but copying is slow! > On my machine, copying files to a sharebox fs is about 10 times slower > than copying it on a normal fs. All the time is spent in python's > os.write(): I guess the only way to work around this problem is to > rewrite the whole thing in C, but I am keeping this for later. hmm, writing files is i/o-bound, I doubt the language will have much effect here. check with top/vmstat if you get iowait, if so your storage medium is getting saturated and rewriting in C won't help. maybe a network/buffering/.. issue. > I am interested in: > - suggestions for the functional design (I have my ideas, but I'd love > to be challenged). in your REAMDE you suggest to use a crontab for synchronisation; maybe you can reuse/be inspired by the xmpp system dvcs-autosync uses; it works quite well, it's quite robust and it's instant :) Dieter _______________________________________________ vcs-home mailing list vcs-home@lists.madduck.net http://lists.madduck.net/listinfo/vcs-home
