Ralf Gross wrote: > Hi, > > I'm faced with the growing storage demands in my department. In the > near future we will need several hundred TB. Mostly large files. ATM > we already have 80 TB of data with gets backed up to tape. > > Providing the primary storage is not the big problem. My biggest > concern is the backup of the data. One solution would be using a > NetApp solution with snapshots. On the other hand is this a very > expensive solution, the data will be written once, but then only read > again. Short: it should be a cheap solution, but the data should be > backed up. And it would be nice if we could abandon tape backups... > > My idea is to use some big RAID 6 arrays for the primary data, create > LUNs in slices of max. 10 TB with XFS filesystems. > > Backuppc would be ideal for backup, because of the pool feature (we > already use backuppc for a smaller amount of data). > > Has anyone experiences with backuppc and a pool size of >50 TB? I'm > not sure how well this will work. I see that backuppc needs 45h to > backup 3,2 TB of data right now, mostly small files. > > I don't like very large filesystems, but I don't see how this will > scale with either multiple backuppc server and smaller filesystems > (well, more than one server will be needed anyway, but I don't want to > run 20 or more server...) or (if possible) with multiple backuppc > instances on the same server, each with a own pool filesystem. > > So, anyone using backuppc in such an environment? >
In one way, and compared to some my backup set is pretty small (pool is 791.45GB). In another dimension, I think it is one of the larger (comprising 20874602 files). The breadth of my pool leads to... -bash-3.2$ df -i /data/ Filesystem Inodes IUsed IFree IUse% Mounted on /dev/drbd0 1932728448 47240613 1885487835 3% /data ...nearly 50 million inodes used (so somewhere close to 30 million hard links). XFS holds up surprisingly well to this abuse*, but the strain shows. Traversing the whole pool takes three days. Attempting to grow my tail (the number of backups I keep) causes serious performance degradation as I approach 55 million inodes. Just an anecdote to be aware of. > Ralf Chris * I have recently taken my DRBD mirror off-line and copied the BackupPC directory structure to both XFS-without-DRBD and an EXT4 file system for testing. Performance of the XFS file system was not much different with, or without DRBD (a fat fiber link helps there). The first traversal of the pool on the EXT4 partition is about 66% through the pool traversal after about 96 hours. ------------------------------------------------------------------------------ SOLARIS 10 is the OS for Data Centers - provides features such as DTrace, Predictive Self Healing and Award Winning ZFS. Get Solaris 10 NOW http://p.sf.net/sfu/solaris-dev2dev _______________________________________________ BackupPC-users mailing list [email protected] List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
