Re: [BackupPC-users] How scalable is backuppc?
On Fri, 2009-03-13 at 19:56 +0100, Matthias Meyer wrote: Mike Dresser wrote: Matthias Meyer wrote: Dear all, How scalable is backuppc? Where are the limits or what can produce performance bottlenecks? Here is a datapoint. I am backing up 2000+ user directories on 1600 mac workstations with a 'cluster' of 7 low end 1U servers via xtar (needed to pick up resource forks). Each server is connected to a Exastore NAS from which they mount the /var/lib/backuppc and /etc/backuppc directories. Each server is limited to 6 backups at a time and they average a load of 2 - 5. The cluster currently uses 4TB on the NAS which is roughly 6TB of data from the workstations. I have written a few glue scripts so that new workstations are added automatically to the 'cluster' and the load is distributed among machines in the 'cluster'. I say 'cluster' because it is not a true fail-over cluster in that each server node is responsible for its set of workstations and the cpool is not shared among all server nodes. For all intents there is no limit to how much this architecture can scale -- I just add on more server nodes as needed. cheers, ski -- When we try to pick out anything by itself, we find it connected to the entire universeJohn Muir Chris Ski Kacoroski, kacoro...@gmail.com, 206-501-9803 or ski98033 on most IM services -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] NAS / SAN and other storage devices
Hi all I am wondering if it is possible to store the backup datas, on non resident devices disks, with Backuppc ? Can I have a NAS or a disk in network wich is not inside the backuppc server ? IF YES where are the informations about those possibilities ? If NOT how do I do when all the possibilities of insides disks are full ? Do I have to add a second backuppc server ? Thanks for your help +-- |This was sent by lionel@equipement-agriculture.gouv.fr via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] NAS / SAN and other storage devices
On Wed, 2009-03-18 at 08:37 -0400, yodo64 wrote: Hi all I am wondering if it is possible to store the backup datas, on non resident devices disks, with Backuppc ? Can I have a NAS or a disk in network wich is not inside the backuppc server ? IF YES where are the informations about those possibilities ? If NOT how do I do when all the possibilities of insides disks are full ? Do I have to add a second backuppc server ? you can use any device that is a block device .. for the secondo question, I think that the best solution is to use lvm/evms or what ever logical volume manager you like... -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] NAS / SAN and other storage devices
If you are usig a SAN or NAS, it should look like 'just another disk' to your BackupPC server. So, yes, you can use them. The problem Ihave found is needing the backuppc backup storage to appear as a single file system. I guess you could use software striping across just about any kind of disk images (real disks, external RAID, SAN or NAS based drives) but it would be up to you to ensure redundancy in case one of the components failed (or just became unavailable temporarily). IHS ... Jack On Wed, Mar 18, 2009 at 7:37 AM, yodo64 backuppc-fo...@backupcentral.com wrote: Hi all I am wondering if it is possible to store the backup datas, on non resident devices disks, with Backuppc ? Can I have a NAS or a disk in network wich is not inside the backuppc server ? IF YES where are the informations about those possibilities ? If NOT how do I do when all the possibilities of insides disks are full ? Do I have to add a second backuppc server ? Thanks for your help +-- |This was sent by lionel@equipement-agriculture.gouv.fr via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] NAS / SAN and other storage devices
I have used BackupPC over NFS successfully in large scale installations. See my previous post. cheers, ski On Wed, 2009-03-18 at 08:37 -0400, yodo64 wrote: Hi all I am wondering if it is possible to store the backup datas, on non resident devices disks, with Backuppc ? Can I have a NAS or a disk in network wich is not inside the backuppc server ? IF YES where are the informations about those possibilities ? If NOT how do I do when all the possibilities of insides disks are full ? Do I have to add a second backuppc server ? Thanks for your help +-- |This was sent by lionel@equipement-agriculture.gouv.fr via Backup Central. |Forward SPAM to ab...@backupcentral.com. +-- -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- When we try to pick out anything by itself, we find it connected to the entire universeJohn Muir Chris Ski Kacoroski, kacoro...@gmail.com, 206-501-9803 or ski98033 on most IM services -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] FS and backuppc performance
Hi all! I'm running backuppc in several installations and sites and i'm very pleased with it, one of the sites has more than 3TB compressed data and about 6.000.000 files. Backupps run very well fast and reliable. My question is about FS performance. From what I've seen on the list there are some people using XFS, Ext3, and so on. What's your experience with the different file systems? For now I'm using ext3 and I don't have much of problems with one exception, some time ago backuppc server was rebooted for kernel and system security update (after being up for 8 months). And on boot a filecheck run on the backuppc data partition and it took almost 2 days to run, lots of inconsistency found and lots of corrections needed. Ext3 was running with noatime, nodiratime,and data mode is journaled. after that i tested some recoveries that went perfect and since then i don't have a prob, but to be sincere I didn't like to see the filecheck run like that and data getting corrupted like that too. BTW I'm using 8 sata drives in a hardware raid 5 (raid utilities say raid status is fine as well as the hard drives). I'm a SuSE fan and for years I used reiserfs that i loved and never game me problems the problem is that reiserfs is not maintained as it used to be... What FS do you guys use recommend/used and why? Cheers Pedro -- -- Pedro M. S. Oliveira IT Consultant Email: pmsolive...@gmail.com URL: http://pedro.linux-geex.com Cellular: +351 96 5867227 -- -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
[BackupPC-users] cloning the pool
Hi, I want to clone the pool to a local disk attached via USB. I can't made it with a dd because the pool is on a raid volume that don't contain the pool only. Does it exist best practice to do this? Thanks Matteo -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] cloning the pool
Matteo Sgalaberni wrote: Hi, I want to clone the pool to a local disk attached via USB. I can't made it with a dd because the pool is on a raid volume that don't contain the pool only. Does it exist best practice to do this? If you have a small pool and sufficient RAM, you may be able to copy with a file oriented approach (rsync -H, tar, cp -a, cpio, etc). but you have to take everything under the directory containing the pool and pc subdirectories at once to maintain the hardlinks. However, this is not an efficient process so there are limits to the size you can handle that way. Above that, you would have to partition the raid volume and dd the partition to a equal-sized partition on the external disk. -- Les Mikesell lesmikes...@gmail.com -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] cloning the pool
On Wed, 2009-03-18 at 18:43 +0100, Matteo Sgalaberni wrote: Does it exist best practice to do this? Hi, search for previous posts on the list for using rsync for doing this (or backuppc_tarpccopy maybe?).. stupid solution (depending on the size of your array..): dd and then rm ?? (kiss-logic) -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] cloning the pool
This link: http://backuppc.wiki.sourceforge.net/change+archive+directory has info on how to move the pool, but similar methods can be used to clone it. Dave -Original Message- From: Matteo Sgalaberni [mailto:sg...@sgala.com] Sent: Wednesday, March 18, 2009 10:43 AM To: backuppc-users@lists.sourceforge.net Subject: [BackupPC-users] cloning the pool Hi, I want to clone the pool to a local disk attached via USB. I can't made it with a dd because the pool is on a raid volume that don't contain the pool only. Does it exist best practice to do this? Thanks Matteo -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/ -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] FS and backuppc performance
On 03/18 05:48 , Pedro M. S. Oliveira wrote: What FS do you guys use recommend/used and why? I typically use XFS for backuppc data pools, and ext3 for the root filesystem. I don't want to run out of inodes like ext3 can do. :) -- Carl Soderstrom Systems Administrator Real-Time Enterprises www.real-time.com -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
Re: [BackupPC-users] cloning the pool
I want to clone the pool to a local disk attached via USB. I can't made it with a dd because the pool is on a raid volume that don't contain the pool only. We're about to do exactly the same thing. This to get ourselves a weekly off-site copy. We will use 500 GB external disks to rsync -aH the complete backuppc directory to this disk. We will use lvm and some encrypted filesystem for enhanced security. We'll have to test it out because the wiki is not very clear about it: rsync has different limitations than cp - don't ask me whether it's better or worse. It's simply something different to try. It might be nice to have some case studies / usage scenarios on the backuppc wiki ? I'll report our experiences after we tested it all out.. cheers stoffell -- Apps built with the Adobe(R) Flex(R) framework and Flex Builder(TM) are powering Web 2.0 with engaging, cross-platform capabilities. Quickly and easily build your RIAs with Flex Builder, the Eclipse(TM)based development software that enables intelligent coding and step-through debugging. Download the free 60 day trial. http://p.sf.net/sfu/www-adobe-com ___ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List:https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki:http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
