https://bugzilla.wikimedia.org/show_bug.cgi?id=45646
--- Comment #5 from Tim Landscheidt <t...@tim-landscheidt.de> ---
I've looked at that recently ("it should be so easy!"), but the dumps process
as present isn't as straight-forward as you would think.
I assumed it would be enough to replace "rsync DIR" with "rsync DIR && ln -s
...", but the reality (cf.
http://git.wikimedia.org/tree/operations%2Fpuppet.git; "download::gluster" is
the class that feeds /public/datasets/public) is much more complicated: A list
of files (!) to sync is produced on the remote, that is rsync'ed to local and
then fed to several (!) rsync workers (limited by count of files and size to
transfer) that do the actually copying. This process is run continuously, so
there is no obvious point to hook into.
This complexity is probably due to the requirement to sync more than 4 TBytes
of data :-). I *think* it would be possible to have a cron job that just sets
the symlinks without upsetting rsync too much, but I'm definitely not sure
about that :-).
--
You are receiving this mail because:
You are on the CC list for the bug.
_______________________________________________
Wikibugs-l mailing list
Wikibugs-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikibugs-l
