backup to NTFS USB disk

2012-04-12 Thread joop g
Hello, * I am setting up a backup on a Linux system with Windows XP workstations. The backup goes to three alternating usb drives, each of which is NTFS formatted. The disks should be virtually identical but they do not seem to be. First, my mount command is this (I edited a bit for brevity)

Re: backup to NTFS USB disk

2012-04-12 Thread joop g
On Thursday 12 April 2012 11:20:54 joop g wrote: (I forgot to include my rsync commands) mount -t ntfs-3g -o locale=nl_NL.iso-8859-1,silent /dev/disk/by-id/usb-DiskA \ /mnt/tmp || mount -t ntfs-3g -o locale=nl_NL.iso-9959-1,silent /dev/disk/by-id/usb-DiskB \ /mnt/tmp || mount -t ntfs-3g -o

Re: Run rsync even not connected

2012-04-12 Thread Chris Arnold
On Apr 11, 2012, at 10:48 PM, Brian K. White br...@aljex.com wrote: Now just run the rsync command. no nohup, no . Just so completely understand, it should be: rsync --options x no nohup no Or am I completely wrong? Btw, Brian K. White, that looks familiar. You a member of the opensuse or

Re: Run rsync even not connected

2012-04-12 Thread Brian K. White
On 4/12/2012 12:32 PM, Chris Arnold wrote: On Apr 11, 2012, at 10:48 PM, Brian K. Whitebr...@aljex.com wrote: Now just run the rsync command. no nohup, no. Just so completely understand, it should be: rsync --options x no nohup no Or am I completely wrong? No I was saying that if you

Rsync takes long time to finish

2012-04-12 Thread vijay patel
Hi Friends, I am using rsync to copy data from Production File Server to Disaster Recovery file server. I have 100Mbps link setup between these two servers. Folder structure is very deep. It is having path like /reports/folder1/date/folder2/file.tx, where we have 1600 directories like

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Several suggestions... Add a lockfile to your cron job so it doesn't run two instances at the same time and you don't have to predict the run time. Make sure you are running rsync version 3+ on both systems. It has significant performance benefits

RE: Rsync takes long time to finish

2012-04-12 Thread Stier, Matthew
And, although rsync does parallelize, nothing stops you from running multiple instances of rsync. I had to transfer files from system A to system B, and being limited by the processing power of a single thread of rsync, I drilled down one level, and ran rsync's against each the first level

Re: Rsync to a Remote NAS

2012-04-12 Thread Joachim Otahal (privat)
This is like mounting the remote drive via samba and then do a sync, this is like doing a normal copy job without the deltra transfer benefits of rsync. If at all possible you should run an rsync daemon on the NAS box and then run the rsync command on the other side of the VPN. rsync uses port

RE: Rsync takes long time to finish

2012-04-12 Thread Stier, Matthew
The first clause should read does not parallelize. -Original Message- From: rsync-boun...@lists.samba.org [mailto:rsync-boun...@lists.samba.org] On Behalf Of Stier, Matthew Sent: Thursday, April 12, 2012 3:07 PM To: Kevin Korb; rsync@lists.samba.org Subject: RE: Rsync takes long time to

Re: Run rsync even not connected

2012-04-12 Thread Joachim Otahal (privat)
it has not been mentioned: nohup ! screen is a bit complicated if you never used it. It only makes sense if you want to check back later and see life what is going on. at or cron would be my last restort. nohup would be my choice nohup (you command and options) Example: x@x:~ nohup ls -la

Re: Run rsync even not connected

2012-04-12 Thread Brian K. White
On 4/12/2012 3:36 PM, Joachim Otahal (privat) wrote: it has not been mentioned: nohup ! Yes it was. -- bkw -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options: https://lists.samba.org/mailman/listinfo/rsync Before posting, read:

Re: Run rsync even not connected

2012-04-12 Thread Joachim Otahal (privat)
Brian K. White schrieb: On 4/12/2012 3:36 PM, Joachim Otahal (privat) wrote: it has not been mentioned: nohup ! Yes it was. You are right, found your post now. -- Please use reply-all for most replies to avoid omitting the mailing list. To unsubscribe or change options:

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
Thanks friends. We are using Redhat Linux 5.8 on Production and Disaster Recovery side. By drilling down we have found out it is taking lot of time to check what has changed while data tranfer is very fast. As i mentioned data in these folders is very less (hardly 40GB) and whenever new

Re: Rsync to a Remote NAS

2012-04-12 Thread Chris Arnold
Are you saying the current way we are doing it does NOT support incremental backups after the first full backup? One of the NAS devices is a readynas duo rnd2100. In the backup section of the gui, it does say backup:remote::rsync but when i select that and fill in the info and click test

Re: Rsync to a Remote NAS

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 It depends on what you mean by incremental backups. There are 3 things involved with rsync and backups... The first is rsync's ability to not not backup the same file twice unless it has changed. This will work with any method as long as you sync

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 And make sure both systems are running rsync v3. It indexes in parallel to the copying. On 04/12/12 16:59, Karl O. Pinc wrote: On 04/12/2012 03:28:18 PM, vijay patel wrote: Thanks friends. We are using Redhat Linux 5.8 on Production and

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
I am getting following thing in 'tune2fs -l' : Filesystem features: has_journal resize_inode dir_index filetype needs_recovery sparse_super large_file Does this mean it is set? One more thing i am not using rsync as daemon (Because i am confused with its usage at the moment), will it

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, you have the feature in your filesystem. Good. If it is ext3 then converting it to ext4 would still help assuming your distro supports it. You are using rsync over ssh. This is my preference as well for security reasons. Using rsyncd would

Re: Rsync to a Remote NAS

2012-04-12 Thread Joachim Otahal (privat)
Chris Arnold schrieb: Are you saying the current way we are doing it does NOT support incremental backups after the first full backup? Oh, the rsync itself will do incremental, however if the filesystem below is going over VPN it won't work fast. rsync's design expects local and fast

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
yes both servers are having rsync 3.0.6. Date: Thu, 12 Apr 2012 17:19:47 -0400 From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 Yes, you have the feature in your filesystem. Good. If it is

Re: Rsync takes long time to finish

2012-04-12 Thread Dan Stromberg
I've heard lots of good suggestions already - another thing that I've not seen mentioned is, upgrading your kernel may help. Somewhere shortly before kernel 3.0, pathname lookups got noticeably faster. You could also try an alternative filesystem like xfs. It's supposed to be pretty good at

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 There was also a serious performance regression in 2.6.39. On 04/12/12 17:29, Dan Stromberg wrote: I've heard lots of good suggestions already - another thing that I've not seen mentioned is, upgrading your kernel may help. Somewhere shortly

RE: Rsync takes long time to finish

2012-04-12 Thread vijay patel
We are running Kernel 2.6.18-308.1.1.el5 which is latest in RHEL 5.8 on both the server. I think i might have to explore option of using ext4. Date: Thu, 12 Apr 2012 17:31:35 -0400 From: k...@sanitarium.net To: rsync@lists.samba.org Subject: Re: Rsync takes long time to finish

Re: Rsync takes long time to finish

2012-04-12 Thread Kevin Korb
-BEGIN PGP SIGNED MESSAGE- Hash: SHA1 That is a bit old for ext4. You need 2.6.28 as the bare minimum but there were a few early issues. I don't remember exactly when it stabilized but I think it was in the low 2.6.30s. Your 2.6.18 is from 2006. (Yes, I know, RedHat has been patching

Re: Rsync takes long time to finish

2012-04-12 Thread Karl O. Pinc
On 04/12/2012 04:36:44 PM, vijay patel wrote: We are running Kernel 2.6.18-308.1.1.el5 which is latest in RHEL 5.8 on both the server. I think i might have to explore option of using ext4. Before you do anything you want to figure out why it is slow so you can solve the real problem.

Re: Rsync takes long time to finish

2012-04-12 Thread Brian K. White
You can try to switch to faster filesystems (reiserfs/ext4/btrfs/zfs) and enable metadata performance options and do other tuning steps (dir_index, noatime) and upgrade disks and ram etc, but mostly, with a frankly unrealistic business requirement like that, you have to either tell business

Re: Rsync takes long time to finish

2012-04-12 Thread Matthias Schniedermeyer
On 12.04.2012 23:59, vijay patel wrote: Hi Friends, I am using rsync to copy data from Production File Server to Disaster Recovery file server. I have 100Mbps link setup between these two servers. Folder structure is very deep. It is having path like