Time rsYnc Machine (tym)

2012-07-26 Thread M. Carrasco
http://dragoman.org/tym Regards Tomas -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read: http://www.catb.org/~esr/faqs/smart-questions.html

Time rsYnc Machine (tym)

2012-07-28 Thread M. Carrasco
These are comments to several previous posting. Regards Tomas 1. Limitations tym is a wrap: the hard work is done by rsync, with whatever limitations it might have, in particular for "non-atomic". Each backup is a new directory; "--backup" is not used as no renaming is required. 2. Time mac

Time rsYnc Machine (tym)

2012-07-28 Thread M. Carrasco
Correction, tym is "push": rsync does all the remote manipulation. The worse case might look like this: - Run program foo-run.com - Log data foo-log.com - Source data foo-source1.com to foo-sourceN.com - Destination data foo-destination.com For this, one migh

Re: Time rsYnc Machine (tym)

2012-07-26 Thread josephj
No good deed goes unpunished ;) Very nicely coded script, but it's a bit dense. I'm good at bash and can survive in rsync, but could you provide a description of what it actually does so I don't have to spend a long time analysing the code? Does it keep multiple versions like the name implies?

Re: Time rsYnc Machine (tym)

2012-07-26 Thread M. Carrasco
Joe, You know programmers are not very good at documenting :-) On 26 Jul 2012, at 21:42, jose...@main.nc.us wrote: > No good deed goes unpunished ;) > > Very nicely coded script, but it's a bit dense. I'm good at bash and can > survive in rsync, My intention was to make something readable :-)

Re: Time rsYnc Machine (tym)

2012-07-27 Thread M. Carrasco
Joe, Your desires are orders :-) ... a proper (small) man page http://dragoman.org/tym Regards Tomas On 26 Jul 2012, at 21:42, jose...@main.nc.us wrote: > No good deed goes unpunished ;) > > Very nicely coded script, but it's a bit dense. I'm good at bash and can > survive in rsync, but co

Re: Time rsYnc Machine (tym)

2012-07-27 Thread josephj
Thanks so much. I'm going to try it out. It looks like all I need to do is add something (manual or script) to delete the oldest versions periodically. Joe > Joe, > > Your desires are orders :-) > > ... a proper (small) man page > > http://dragoman.org/tym > > Regards > Tomas > > On 26 Jul 201

Re: Time rsYnc Machine (tym)

2012-07-27 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Note that this stuff is a lot easier if you pull your backups rather than pushing them. That way your making of directories and symlinks and deleting of old backups are all done locally. On 07/27/12 16:04, jose...@main.nc.us wrote: > Thanks so much.

Re: Time rsYnc Machine (tym)

2012-07-28 Thread josephj
Could you explain push vs. pull? I haven't seen that before. TIA Joe > Note that this stuff is a lot easier if you pull your backups rather > than pushing them. That way your making of directories and symlinks > and deleting of old backups are all done locally. -- Please use reply-all for m

Re: Time rsYnc Machine (tym)

2012-07-28 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Push is when you run your backup program (rsync and whatever script) on the machine being backed up and you push/upload your data to the backup system. Pull is when you run your backup program on the backup system and pull/download the data from the m

Re: Time rsYnc Machine (tym)

2012-07-28 Thread Dan Stromberg
On Sat, Jul 28, 2012 at 3:48 AM, M. Carrasco wrote: > 3. Cron > It can run properly run from cron as it is demonized. > What's this about? I've never had problems running run of the mill scripts from cron, once the environment is adequately replicated. > "--hard-links" is not used and there is

Re: Time rsYnc Machine (tym)

2012-07-28 Thread josephj
He redirects stdout and stderr to files and doesn't require user interaction. Living on a notebook, almost all of my scripts don't do that, so they won't work from cron or any background situation unless I modify them with that in mind. Joe > On Sat, Jul 28, 2012 at 3:48 AM, M. Carrasco wrote:

Re: Time rsYnc Machine (tym)

2012-07-28 Thread Dan Stromberg
Ah. Not being interactive is important for running in cron; I believe stdin will probably immediately EOF. But redirecting stdout and stderr is unnecessary - the output just goes to a cron e-mail with most cron's. Sometimes it's better to redirect to a file, but that's more of a user preferen

Re: Time rsYnc Machine (tym)

2012-07-28 Thread M. Carrasco
> On Sat, Jul 28, 2012 at 3:48 AM, M. Carrasco wrote: > 3. Cron > It can run properly run from cron as it is demonized. > What's this about? I've never had problems running run of the mill scripts > from cron, once the environment is adequately replicated. Joe already answered, but one can also

Re: Time rsYnc Machine (tym)

2012-08-01 Thread Henri Shustak
> Push is when you run your backup program (rsync and whatever script) > on the machine being backed up and you push/upload your data to the > backup system. > > Pull is when you run your backup program on the backup system and > pull/download the data from the machine being backed up. You may al

Re: Time rsYnc Machine (tym)

2012-08-07 Thread Linda Walsh
Dan Stromberg wrote: FWIW, it might be nice to add a hardlink detecting bloom filter to rsync at some point. This makes the process of detecting hardlinks less expensive. Another way to narrow down the field is to just look at st_nlink. What's a bloom filter? and how / why wou

Re: Time rsYnc Machine (tym)

2012-08-07 Thread Linda Walsh
ARG! http://lmgtfy.com/?q=bloom+filter K S Braunsdorf wrote: Just google.com or ddg.gg "bloom filter" and you'd be more happy. --ksb Lassi Tuura wrote: Hey, I still have no clue what a bloom filter is?? ;-) Cluesticks anyone? www.google.com/search?q=bloom+filter L. -- Please use r

Re: Time rsYnc Machine (tym)

2012-08-09 Thread Dan Stromberg
I may be mistaken, but I heard at one time that rsync was noticeably slower when asked to preserve hard links. I'm guessing this is a matter of CPU requirements rather than I/O requirements, but that's just a guess. You can go a long way toward detecting hardlinks by just watching for st_nlink >

Re: Time rsYnc Machine (tym)

2012-08-09 Thread Linda Walsh
Dan Stromberg wrote: I may be mistaken, but I heard at one time that rsync was noticeably slower when asked to preserve hard links. I'm guessing this is a matter of CPU requirements rather than I/O requirements, but that's just a guess. --- I didn't realize at the time I wrote it,

Re: Time rsYnc Machine (tym)

2012-08-09 Thread M. Carrasco
Reading the rsync man page, it seems that the -H option with --link-dest is tricky. The price to pay on not using -H is that hard linked files are treated as separated files: I could not find any mechanism in rsync to improve the time machine effect; I would appreciate hits on how to improve it.

RE: Time rsYnc Machine (tym)

2012-08-10 Thread Voelker, Bernhard
M. Carrasco wrote (Friday, August 10, 2012 12:23 AM): > Reading the rsync man page, it seems that the -H option with > --link-dest is tricky. The price to pay on not using -H is that > hard linked files are treated as separated files: I could not > find any mechanism in rsync to improve the time m

Re: Time rsYnc Machine (tym)

2012-08-10 Thread Paul Slootman
On Thu 09 Aug 2012, Linda Walsh wrote: > > Anyway, thanks for the history update. I have a feeling rsync is afraid to > use > memory -- and really, it should try to use alot of memory to optimize > transfers, I have had rsync fail after using up 8GB memory + 4GB swap, so I'm very happy it doe

Re: Time rsYnc Machine (tym)

2012-08-10 Thread Linda Walsh
Paul Slootman wrote: On Thu 09 Aug 2012, Linda Walsh wrote: Anyway, thanks for the history update. I have a feeling rsync is afraid to use memory -- and really, it should try to use alot of memory to optimize transfers, I have had rsync fail after using up 8GB memory + 4GB swap, so I'm ver

Time rsYnc Machine (tym) - version 2

2012-08-08 Thread M. Carrasco
1. Version 2 It is at http://dragoman.org/tym - Compatible with version 1 - Some refinements - Options to monitor, show the logs, kill the process and delete old copies - Making the code more readable (Joe, please point out part that need more commenting) 2. Bloom filter tym is just a wrap fo