Evren Yurtesen wrote:
Totals Existing Files New Files
Backup# Type #Files Size/MB MB/sec #Files Size/MB
#Files Size/MB
245 full 152228 2095.2 0.06 152177 2076.9 108 18.3
246 incr 118 17.3 0.00 76 0.2 69 17.1
Can you post the duration that these backups took? All that these stats
tell us is how much of your data is churning, and how big your average
file size is.
I dont know if the problem is hard links. This is not a FreeBSD or Linux
problem. It exists on both. Just that people using ultra fast 5 disk
raid 5 setups are seeing 2mbytes/sec transfer rate means that backuppc
is very very inefficient.
For example this guy is using Linux (problem is OS independent)
http://forum.psoft.net/showpost.php?p=107808&postcount=16
Er, RAID-5 is slower than a single disk except on sustained reads, which
is not the typical case with BackupPC. I only use a single disk and get
between 0.5mb/s and 10mb/s, depending on the client and its file
size/file count distribution. The question is whether other people have
gotten the system to work correctly under the same or worse conditions,
and already that has been answered in the affirmative. Be prepared to
accept that the software isn't broken just because you haven't gotten it
working to your satisfaction in your specific situation... It could be
rsync, UFS, BSD, or any number of other factors that are causing you grief.
Whatever, this is not the problem here.The fact is that, according to
reiserfs developers reiserfs is more or less the same speed with ext2. I
dont think the problem is related to any filesystem as it occurs on both
Linux and FreeBSD
Your argument lacks logic. If a filesystem can be configured to be slow
on multiple OS's, does that mean BackupPC is failing to do its job? No,
it means multiple people have managed to set up a system that performs
badly using it. That's not so uncommon. BackupPC does not exist in a
vacuum: its performance is sensitive to the environment of the server
and its clients. Many people are using it without issue, right here on
this very list. The question you should be asking is, what makes your
system perform badly? Start by picking out things that you do
differently, or that definitely affect performance. Transport protocol
(rsync, tar). UFS. BSD. Sync on your filesystem. Start changing
those one at a time and measure the performance.
On Linux with raid setup with async io etc. people are getting slightly
better results. I think ufs2 is just fine. I wonder if there is
something in my explanations...The problem is backuppc. People are
getting ~2mbytes/sec(was it 2 or 5?) speed with raid5 and 5 drives,
using Linux. It is a miracle that backup even finishes in 24 hours using
a standart ide drive.
If you want people to help you, it's probably best if you refrain from
blaming the program until you have proof. So far, you have argued with
people who provide evidence that the system works fine. It puts people
on the defensive, and you may find people less willing to help you in
the future. We do know BackupPC behaves well on other file systems and
operating systems... maybe UFS or BSD is doing something poorly--how
would you know they aren't? We have less data points to draw from there.
I suspect it has a lot more to do with what the MB/s stats really mean.
Maybe Craig can give a precise definition?
This is like the 'Contact' movie. The sphere took 30 seconds to download
but there were 18 hours of recording. If what you said was true and
backuppc would be backing up very small amount of files and skipping
most, then backups would probably take less time than 2-4 hours each.
With rsync in incremental backups, it still has to check the metadata
for each file to determine the changed file set. With millions of
files, it will take a while. If your host or server is low on memory
for any reason, this may bog it down and start vm swapping. I would
recommend trying out tar to see if the protocol behavior matters. Try
different mount options that are for higher performance. Others have
made similar suggestions as well.
Good luck,
JH
-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
BackupPC-users mailing list
BackupPC-users@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/backuppc-users
http://backuppc.sourceforge.net/
