On 3/27/07, Evren Yurtesen <[EMAIL PROTECTED]> wrote: > David Rees wrote: > > Evren, I didn't see that you mentioned a wall clock time for your > > backups? I want to know how many files are in a single backup, how > > much data is in that backup and how long it takes to perform that > > backup. > > I sent the status of the backups earlier today to mailing list?
Still missing wall-clock time. Though we can extrapolate that it may be taking over 6 hours for a full backup of the system below. Is that correct? > This is from 1 machine. > > Totals Existing Files New Files > Backup# Type #Files Size/MB MB/sec #Files Size/MB > #Files Size/MB > 112 full 149290 1957.8 0.10 149251 1937.9 4601 20.7 > I dont know if the problem is hard links. This is not a FreeBSD or Linux > problem. It exists on both. Just that people using ultra fast 5 disk > raid 5 setups are seeing 2mbytes/sec transfer rate means that backuppc > is very very inefficient. As mentioned earlier, RAID 5 is horrible for random small read/write performance. It is often worse than a single disk. But still, I have a client which has 1.5 million files and 80-100GB of data. A full backup takes 4-6 hours which is reasonable. Full backups average 4.5-5MB/s. > For example this guy is using Linux (problem is OS independent) > http://forum.psoft.net/showpost.php?p=107808&postcount=16 Your transfer rates are at least 5x slower than his and 10x slower than what most people here are getting. There is something wrong specifically with your setup. I suspect that if your backups were going 5x faster you'd be happy and 10x faster you'd be very happy. > On Linux with raid setup with async io etc. people are getting slightly > better results. I think ufs2 is just fine. I wonder if there is > something in my explanations...The problem is backuppc. People are > getting ~2mbytes/sec(was it 2 or 5?) speed with raid5 and 5 drives, > using Linux. It is a miracle that backup even finishes in 24 hours using > a standart ide drive. With ext2 the default is async IO. With ext3 (the default system now) the default is ordered which is similar to BSD's soft updates. > This is like the 'Contact' movie. The sphere took 30 seconds to download > but there were 18 hours of recording. If what you said was true and > backuppc would be backing up very small amount of files and skipping > most, then backups would probably take less time than 2-4 hours each. Using the rsync transfer method, it does require at least a stat of every single file to be backed up. For 150k files and 2GB of data, you'd really expect this to be done within a hour with nearly any machine. Are you using the --checksum-seed option? That will significantly speed up (the 3rd and later) backups as well. Are you also sure the backup server isn't swapping as well? Can you get us some good logs from vmstat or similar during a backup? I also suspect that there is something else in this case slowing your machine down. Unless you give us the information to help track it down, we will not be able to figure it out. I feel like I am pulling teeth. -Dave ------------------------------------------------------------------------- Take Surveys. Earn Cash. Influence the Future of IT Join SourceForge.net's Techsay panel and you'll get the chance to share your opinions on IT & business topics through brief surveys-and earn cash http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/backuppc-users http://backuppc.sourceforge.net/
