Les Mikesell wrote at about 09:50:25 -0500 on Tuesday, June 2, 2009: > Jeffrey J. Kosowsky wrote: > > > > In my mind the only major reason not to move to a database > > architecture is that it would require a substantial re-write of > > BackupPC as pointed out in my earlier note. > > Do you actually have any experience with large scale databases? I think > most installations that come anywhere near the size and activity of a > typical backuppc setup would require a highly experienced DBA to > configure and would have to be spread across many disks to have adequate > performance.
I am by no means a database expert, but I think you are way overstating the complexity issues. While the initial design would certainly need someone with experience, I don't know why each implementation would require a "highly experienced DBA" or why it "would have to be spread across many disks" any more than a standard BackupPC implementation. Modern databases are written to hide a lot of the complexity of optimization. Plus the database is large only in the sense of having lots of table entries but is otherwise not particularly complex nor do you have to deal with multiple simultaneous access queries which is usually the major bottleneck requiring optimization and performance tuning. Similarly the queries will in general be very simple and easily keyed relative to other real-world databases. Remember size != difficulty or complexity. > When you get down to the real issues, normal operation has > a bottleneck with disk head motion which a database isn't going to do > any better without someone knowing how to tune it across multiple disks. This seems like a red herring. The disk head motion issue applies whether the data is stored in a database or in a combination of a filesystem + attrib files. If anything, storage in a single database would be more efficient than having to find and individually load (and unpack) multiple attrib files since the database storage can be optimized to some degree automagically while even attrib files that are logically "sequential" could be scattered all over the disk leading to inefficient head movement. Also, the database could be stored on one disk and the pool on another but this would be difficult if not impossible to do on BackupPC where the pool, the links, and the attrib files are all on the same filesystem. > Also, while some database do offer remote replication, it isn't > magic either and keeping it working isn't a common skill. > Again a red herring. Jut having the ability to temporarily "throttle" BackupPC leaving the database in a consistent state would allow one to just simply copy (e.g., rsync) the database and the pool to a backup device. This copy would be much faster than today's BackupPC because you wouldn't have the hard link issue. Remote replication would be even better but not necessary to solve the common issue of copying the pool raised by so many people on this list. > -- > Les Mikesell > lesmikes...@gmail.com > > > > ------------------------------------------------------------------------------ > OpenSolaris 2009.06 is a cutting edge operating system for enterprises > looking to deploy the next generation of Solaris that includes the latest > innovations from Sun and the OpenSource community. Download a copy and > enjoy capabilities such as Networking, Storage and Virtualization. > Go to: http://p.sf.net/sfu/opensolaris-get > _______________________________________________ > BackupPC-users mailing list > BackupPC-users@lists.sourceforge.net > List: https://lists.sourceforge.net/lists/listinfo/backuppc-users > Wiki: http://backuppc.wiki.sourceforge.net > Project: http://backuppc.sourceforge.net/ > ------------------------------------------------------------------------------ OpenSolaris 2009.06 is a cutting edge operating system for enterprises looking to deploy the next generation of Solaris that includes the latest innovations from Sun and the OpenSource community. Download a copy and enjoy capabilities such as Networking, Storage and Virtualization. Go to: http://p.sf.net/sfu/opensolaris-get _______________________________________________ BackupPC-users mailing list BackupPC-users@lists.sourceforge.net List: https://lists.sourceforge.net/lists/listinfo/backuppc-users Wiki: http://backuppc.wiki.sourceforge.net Project: http://backuppc.sourceforge.net/
