Re: rsync help the gramma
cp -a /home/orangeroot1000 /media/MaxtorUSBHDD/OrangeRoot1000Backups/Lappie$(date -u +%Y%m%d-%H:%M) Cron jobs are wonderful, if you are backing up any system files root will need to do the cron, otherwise $user is sufficient. The example above is using cp command but rsync would work the same for all the date stuff so you never have to mess with dates. The nice thing about rsync is you could continue a full backup each month, but the beauty behind rsync is that it does one MASTER backup then unless you delete it will always add too that original with new data, SAVING all the old that sometimes you find out 3 days after you deleted it that I really needed that information. This is why I personally only do a MANUAL delete of older files and never automatic. Have fun rsyncing.. vp --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
One thing I might point out though is that you (Betty) said you did the backup about once a month and your example shows rsyncing to a directory named for the month (October). That's fine but some people use rsync to update a single backup destination so they don't keep each backup separately. You choice which best suits you, but I thought it worth mentioning. Larry On Wed, Dec 8, 2010 at 11:56 PM, Steve Holmes st...@holmesgrown.com wrote: Yes, I commend you for going for the command line tools like that. Tackling rsync is no slouch by any means. Frankly, I can rarely remember all the command line options for rsync and have to either constantly look at the man page or previous working examples. I would also create little one or two line shell scripts with these commands in them and then just run the script each month or whenever. That's why Joseph's script example had the '$1' and '$2' thingies in there. those can then be substituted by just putting in the names of the paths and run the script. Also, I believe Joseph's example included the --delete option. However, I don't recall what is actually being deleted at that point. Might wanna check to be sure there. Also, if you get more adventuresome in the future, you might wanna look at a program called rdiff-backup. It uses rsync under the hood and allows for optional inclusions and exclusions like rsync does but allows for staging different backup sets over time and yes, each subsequent backup is just the differences. Plus then if you need to restore a file from the most recent backup, it is merely a mirror of your current environment so you can just copy the desired item to be restored. But rsync is a good start and an excellent way to sync up two computers over a network too. On Wed, Dec 08, 2010 at 10:18:01PM -0700, Alan Dayley wrote: Do not think yourself stupid! I know software developers who are afraid of anything on a command line. The fact that you are using rsync is a long ways from anything close to stupid! Excellent work. Alan On Wed, Dec 8, 2010 at 10:05 PM, betty nicepeng...@webcanine.com wrote: ok, great, so do i just do this; sto...@stormy-desktop:~$ rsync -avEHh /home/stormy/ /media/october or do i do this : sto...@stormy-desktop:~$ rsync -avEHh --delete-after --progress /home/stormy/ /media/october sorry to be so stupid, but i rely on you guys for this ;) thank you!! betty On 12/08/2010 08:32 PM, Joseph Sinclair wrote: You're pretty close to ideal there. I use rsync -avEHh --delete-after --progress $SOURCE $DESTINATION Generally no need to sudo for your own homedir, and -z is really only useful for network copying (it compresses in-transit, not on disk). That will only copy over changes between the source and destination, and will remove anything in destination that's no longer in source. Make sure the external drive is formatted EXT3 or XFS so you can preserve things like access controls and ownership. When you look at the final output of the command, there's usually a ratio listed. That's the amount that could be copied vs. the amount actually copied. If that's much larger than 1, then you're only copying changes. ==Joseph++ betty wrote: I have an ext hdd that i copy my stuff to every few months. I think that i am copying everything all over again each time. What I'd like to do is just copy files that have changed. This is the command i have been using sto...@stormy-desktop:~$ sudo rsync -azvH /home/stormy/ /media/october please don't suggest that i use dd or whatever else there is because i am not good at trying new things. i think i'm doing a lot just to do the backup every few months. any rsync suggestions greatly appreciated. if i am already using the correct command for just copying things that have changed, please let me know. THANKS, Merry Christmas to all pluggers ! -- betty i. www.webcanine.com information for people who care for dogs. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss -- Dazed_75 a.k.a. Larry The spirit of resistance to government is so valuable on certain
Re: rsync help the gramma
This is true - if you change the destination folder you do copy everything because rsync does not know you copied anything before. Rsync syncs. Thus if you wish to sync /my/files/ and /media/my/backupfiles/ it copies everything from files to back up files on the first run then every subsequent run it copies changes. But if I ask it to now sync my/files/ and /media/my/NEW/backupfiles/ it must first copy every thing again because non of the files in /my/files exist in /media/my/NEW/backupfiles/ If you want to have say /my/files backed up every day with that days changes but you also want to keep track up each change separately (aka have a snap shot for every month) so you can go back to prior versions, then you want to look at rsyncs cousin rsnapshot. Used in combination with rsync rsnapshot keeps track of changes by copying files that have changed and creating links to previously copied files that have not changed. Here is a quick blog post and a set of narrated presentation slides I did on rsync and rsnapshot a while back http://mlwtech.blogspot.com/2009/04/rsync-and-ssh-keys-presentation-on.html The audio auto-play and self progressing slide features are all messed up because I did the auto-play stuff on a windows box but if your interested you can play each slides associated mp3 manually. I keep thinking I am going to go fix this by simply web linking the associated auto play mp3 files or manually dealing with relative paths but I am a bit busy... But I digress! The post and associated slides are good and you may find the info very useful :) On Thu, Dec 9, 2010 at 1:26 AM, Dazed_75 lthiels...@gmail.com wrote: One thing I might point out though is that you (Betty) said you did the backup about once a month and your example shows rsyncing to a directory named for the month (October). That's fine but some people use rsync to update a single backup destination so they don't keep each backup separately. You choice which best suits you, but I thought it worth mentioning. Larry On Wed, Dec 8, 2010 at 11:56 PM, Steve Holmes st...@holmesgrown.com wrote: Yes, I commend you for going for the command line tools like that. Tackling rsync is no slouch by any means. Frankly, I can rarely remember all the command line options for rsync and have to either constantly look at the man page or previous working examples. I would also create little one or two line shell scripts with these commands in them and then just run the script each month or whenever. That's why Joseph's script example had the '$1' and '$2' thingies in there. those can then be substituted by just putting in the names of the paths and run the script. Also, I believe Joseph's example included the --delete option. However, I don't recall what is actually being deleted at that point. Might wanna check to be sure there. Also, if you get more adventuresome in the future, you might wanna look at a program called rdiff-backup. It uses rsync under the hood and allows for optional inclusions and exclusions like rsync does but allows for staging different backup sets over time and yes, each subsequent backup is just the differences. Plus then if you need to restore a file from the most recent backup, it is merely a mirror of your current environment so you can just copy the desired item to be restored. But rsync is a good start and an excellent way to sync up two computers over a network too. On Wed, Dec 08, 2010 at 10:18:01PM -0700, Alan Dayley wrote: Do not think yourself stupid! I know software developers who are afraid of anything on a command line. The fact that you are using rsync is a long ways from anything close to stupid! Excellent work. Alan On Wed, Dec 8, 2010 at 10:05 PM, betty nicepeng...@webcanine.com wrote: ok, great, so do i just do this; sto...@stormy-desktop:~$ rsync -avEHh /home/stormy/ /media/october or do i do this : sto...@stormy-desktop:~$ rsync -avEHh --delete-after --progress /home/stormy/ /media/october sorry to be so stupid, but i rely on you guys for this ;) thank you!! betty On 12/08/2010 08:32 PM, Joseph Sinclair wrote: You're pretty close to ideal there. I use rsync -avEHh --delete-after --progress $SOURCE $DESTINATION Generally no need to sudo for your own homedir, and -z is really only useful for network copying (it compresses in-transit, not on disk). That will only copy over changes between the source and destination, and will remove anything in destination that's no longer in source. Make sure the external drive is formatted EXT3 or XFS so you can preserve things like access controls and ownership. When you look at the final output of the command, there's usually a ratio listed. That's the amount that could be copied vs. the amount actually copied. If that's much larger than 1, then you're only copying changes. ==Joseph++ betty wrote:
Re: rsync help the gramma
Bryan wrote: This is true - if you change the destination folder you do copy everything because rsync does not know you copied anything before. This is true in the general case. However, you can tell rsync about the files you copied before with the --compare-dest, --copy-dest, or --link-dest options. Rsync syncs. Thus if you wish to sync /my/files/ and /media/my/backupfiles/ it copies everything from files to back up files on the first run then every subsequent run it copies changes. But if I ask it to now sync my/files/ and /media/my/NEW/backupfiles/ it must first copy every thing again because non of the files in /my/files exist in /media/my/NEW/backupfiles/ If you use rsync -a --link-dest=/media/my/backupfiles /my/files/ /media/my/NEW/backupfiles/ any files that are unchanged since the previous backup to /media/my/backupfiles will not be copied by rsync. Instead, they will be hardlinked from /media/my/backupfiles to /media/my/NEW/backupfiles. IMHO, rsync is simply amazing. Many scripts use this feature of rsync to give you incremental backups that look like full backups. Bryan mentioned rsnapshot, which I haven't used, but looks interesting. I use dirvish. I set it up to do an automated nightly backup of the several systems I have at home. Here is a list of my toplevel backup directories for one system: base $ ls ./ 20100103/ 20100801/ 20101017/ 20101125/ 20101202/ ../20100207/ 20100905/ 20101024/ 20101126/ 20101203/ 20081005/ 20100307/ 20100912/ 20101031/ 20101127/ 20101204/ 20090104/ 20100404/ 20100919/ 20101107/ 20101128/ 20101205/ 20090405/ 20100502/ 20100926/ 20101114/ 20101129/ 20101206/ 20090705/ 20100606/ 20101003/ 20101121/ 20101130/ 20101207/ 20091004/ 20100704/ 20101010/ 20101124/ 20101201/ 20101208/ Dirvish has left me nightly backups of the last 15 days and weekly backups of the last 3 months, monthly backups for the last year, and annual backups prior to that. (Dirvish backed up nightly for the last two years, but has culled older backups according to the schedule I gave it.) Each of the directories contains a full snapshot of the files at the time they were backed up. Note that there is no redundancy. There is only one copy of each file that hasn't been changed, with links to it appearing in multiple directories. -Dale --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
You might also check out the Déjà Dup tool. It also does incremental backups correctly (it is a front-end to duplicity). As long as you don't need some of the more advanced features of duplicity, rsnapshot, rdiff-backup, etc, it works quite well on the desktop. signature.asc Description: OpenPGP digital signature --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
I use -avczPW for switches. very rarely do I automate a delete and it is always AFTER a full copy. If you put a / in the wrong place you can nuke your whole set of backups. So beware... other than that rsync IS the way to backup. It is so featureful and once you set up your style makes it pretty brainless to use. Rsync get sym links, hard links, and permissions, mod dates, the whole schbang. The wonderful thing too about rsync is if it gets interrupted it doesn't corrupt your whole backup. So when you restart it it adjusts itself for source and destination and then just restarts the backup from the last completed file. gk --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
rsync help the gramma
I have an ext hdd that i copy my stuff to every few months. I think that i am copying everything all over again each time. What I'd like to do is just copy files that have changed. This is the command i have been using sto...@stormy-desktop:~$ sudo rsync -azvH /home/stormy/ /media/october please don't suggest that i use dd or whatever else there is because i am not good at trying new things. i think i'm doing a lot just to do the backup every few months. any rsync suggestions greatly appreciated. if i am already using the correct command for just copying things that have changed, please let me know. THANKS, Merry Christmas to all pluggers ! -- betty i. www.webcanine.com information for people who care for dogs. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
You're pretty close to ideal there. I use rsync -avEHh --delete-after --progress $SOURCE $DESTINATION Generally no need to sudo for your own homedir, and -z is really only useful for network copying (it compresses in-transit, not on disk). That will only copy over changes between the source and destination, and will remove anything in destination that's no longer in source. Make sure the external drive is formatted EXT3 or XFS so you can preserve things like access controls and ownership. When you look at the final output of the command, there's usually a ratio listed. That's the amount that could be copied vs. the amount actually copied. If that's much larger than 1, then you're only copying changes. ==Joseph++ betty wrote: I have an ext hdd that i copy my stuff to every few months. I think that i am copying everything all over again each time. What I'd like to do is just copy files that have changed. This is the command i have been using sto...@stormy-desktop:~$ sudo rsync -azvH /home/stormy/ /media/october please don't suggest that i use dd or whatever else there is because i am not good at trying new things. i think i'm doing a lot just to do the backup every few months. any rsync suggestions greatly appreciated. if i am already using the correct command for just copying things that have changed, please let me know. THANKS, Merry Christmas to all pluggers ! signature.asc Description: OpenPGP digital signature --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
ok, great, so do i just do this; sto...@stormy-desktop:~$ rsync -avEHh /home/stormy/ /media/october or do i do this : sto...@stormy-desktop:~$ rsync -avEHh --delete-after --progress /home/stormy/ /media/october sorry to be so stupid, but i rely on you guys for this ;) thank you!! betty On 12/08/2010 08:32 PM, Joseph Sinclair wrote: You're pretty close to ideal there. I use rsync -avEHh --delete-after --progress $SOURCE $DESTINATION Generally no need to sudo for your own homedir, and -z is really only useful for network copying (it compresses in-transit, not on disk). That will only copy over changes between the source and destination, and will remove anything in destination that's no longer in source. Make sure the external drive is formatted EXT3 or XFS so you can preserve things like access controls and ownership. When you look at the final output of the command, there's usually a ratio listed. That's the amount that could be copied vs. the amount actually copied. If that's much larger than 1, then you're only copying changes. ==Joseph++ betty wrote: I have an ext hdd that i copy my stuff to every few months. I think that i am copying everything all over again each time. What I'd like to do is just copy files that have changed. This is the command i have been using sto...@stormy-desktop:~$ sudo rsync -azvH /home/stormy/ /media/october please don't suggest that i use dd or whatever else there is because i am not good at trying new things. i think i'm doing a lot just to do the backup every few months. any rsync suggestions greatly appreciated. if i am already using the correct command for just copying things that have changed, please let me know. THANKS, Merry Christmas to all pluggers ! -- betty i. www.webcanine.com information for people who care for dogs. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
Do not think yourself stupid! I know software developers who are afraid of anything on a command line. The fact that you are using rsync is a long ways from anything close to stupid! Excellent work. Alan On Wed, Dec 8, 2010 at 10:05 PM, betty nicepeng...@webcanine.com wrote: ok, great, so do i just do this; sto...@stormy-desktop:~$ rsync -avEHh /home/stormy/ /media/october or do i do this : sto...@stormy-desktop:~$ rsync -avEHh --delete-after --progress /home/stormy/ /media/october sorry to be so stupid, but i rely on you guys for this ;) thank you!! betty On 12/08/2010 08:32 PM, Joseph Sinclair wrote: You're pretty close to ideal there. I use rsync -avEHh --delete-after --progress $SOURCE $DESTINATION Generally no need to sudo for your own homedir, and -z is really only useful for network copying (it compresses in-transit, not on disk). That will only copy over changes between the source and destination, and will remove anything in destination that's no longer in source. Make sure the external drive is formatted EXT3 or XFS so you can preserve things like access controls and ownership. When you look at the final output of the command, there's usually a ratio listed. That's the amount that could be copied vs. the amount actually copied. If that's much larger than 1, then you're only copying changes. ==Joseph++ betty wrote: I have an ext hdd that i copy my stuff to every few months. I think that i am copying everything all over again each time. What I'd like to do is just copy files that have changed. This is the command i have been using sto...@stormy-desktop:~$ sudo rsync -azvH /home/stormy/ /media/october please don't suggest that i use dd or whatever else there is because i am not good at trying new things. i think i'm doing a lot just to do the backup every few months. any rsync suggestions greatly appreciated. if i am already using the correct command for just copying things that have changed, please let me know. THANKS, Merry Christmas to all pluggers ! -- betty i. www.webcanine.com information for people who care for dogs. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
Re: rsync help the gramma
Yes, I commend you for going for the command line tools like that. Tackling rsync is no slouch by any means. Frankly, I can rarely remember all the command line options for rsync and have to either constantly look at the man page or previous working examples. I would also create little one or two line shell scripts with these commands in them and then just run the script each month or whenever. That's why Joseph's script example had the '$1' and '$2' thingies in there. those can then be substituted by just putting in the names of the paths and run the script. Also, I believe Joseph's example included the --delete option. However, I don't recall what is actually being deleted at that point. Might wanna check to be sure there. Also, if you get more adventuresome in the future, you might wanna look at a program called rdiff-backup. It uses rsync under the hood and allows for optional inclusions and exclusions like rsync does but allows for staging different backup sets over time and yes, each subsequent backup is just the differences. Plus then if you need to restore a file from the most recent backup, it is merely a mirror of your current environment so you can just copy the desired item to be restored. But rsync is a good start and an excellent way to sync up two computers over a network too. On Wed, Dec 08, 2010 at 10:18:01PM -0700, Alan Dayley wrote: Do not think yourself stupid! I know software developers who are afraid of anything on a command line. The fact that you are using rsync is a long ways from anything close to stupid! Excellent work. Alan On Wed, Dec 8, 2010 at 10:05 PM, betty nicepeng...@webcanine.com wrote: ok, great, so do i just do this; sto...@stormy-desktop:~$ rsync -avEHh /home/stormy/ /media/october or do i do this : sto...@stormy-desktop:~$ rsync -avEHh --delete-after --progress /home/stormy/ /media/october sorry to be so stupid, but i rely on you guys for this ;) thank you!! betty On 12/08/2010 08:32 PM, Joseph Sinclair wrote: You're pretty close to ideal there. I use rsync -avEHh --delete-after --progress $SOURCE $DESTINATION Generally no need to sudo for your own homedir, and -z is really only useful for network copying (it compresses in-transit, not on disk). That will only copy over changes between the source and destination, and will remove anything in destination that's no longer in source. Make sure the external drive is formatted EXT3 or XFS so you can preserve things like access controls and ownership. When you look at the final output of the command, there's usually a ratio listed. That's the amount that could be copied vs. the amount actually copied. If that's much larger than 1, then you're only copying changes. ==Joseph++ betty wrote: I have an ext hdd that i copy my stuff to every few months. I think that i am copying everything all over again each time. What I'd like to do is just copy files that have changed. This is the command i have been using sto...@stormy-desktop:~$ sudo rsync -azvH /home/stormy/ /media/october please don't suggest that i use dd or whatever else there is because i am not good at trying new things. i think i'm doing a lot just to do the backup every few months. any rsync suggestions greatly appreciated. if i am already using the correct command for just copying things that have changed, please let me know. THANKS, Merry Christmas to all pluggers ! -- betty i. www.webcanine.com information for people who care for dogs. --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss --- PLUG-discuss mailing list - PLUG-discuss@lists.plug.phoenix.az.us To subscribe, unsubscribe, or to change your mail settings: http://lists.PLUG.phoenix.az.us/mailman/listinfo/plug-discuss
