Re: Amanda server selection advice
Gene Heskett wrote: So is offloading the compression duties to the client, which has the advantage of reducing the footprint on the networks bandwidth too. You do this in the dumptype with compress client best. It Just Works(TM) :-) Well, like I said in my other e-mail, and I believe this is an issue that other people are running into as well, the client machines are so slow that even my current backup server is faster than them. Believe me, I benchmarked the backup times with server vs client side compression, and the server side is still faster even with it being a huge bottleneck. Client side compression only Just Works if your clients are reasonably powered, which isn't always the case with legacy servers to back up. ;) Graeme
Re: Amanda server selection advice
On Friday 03 February 2006 09:14, Graeme Humphries wrote: Gene Heskett wrote: So is offloading the compression duties to the client, which has the advantage of reducing the footprint on the networks bandwidth too. You do this in the dumptype with compress client best. It Just Works(TM) :-) Well, like I said in my other e-mail, and I believe this is an issue that other people are running into as well, the client machines are so slow that even my current backup server is faster than them. Believe me, I benchmarked the backup times with server vs client side compression, and the server side is still faster even with it being a huge bottleneck. Client side compression only Just Works if your clients are reasonably powered, which isn't always the case with legacy servers to back up. ;) Graeme I'm not sure what you would classify as legacy stuff. Here I'm doing 3 machines including this as the server, and of course this is the fastest box, but the clients are a 500mhz K6-III box I use for firewall, and a 1400 mhz athlon box running my milling machine. This one claims to be an XP2800, but actually runs at 2.1ghz. So it pays me about an hour saved overall to let the other boxes do their own thing. Now if there was a dozen old 150 cyrix's out there, it still might pay because they'd all be running in parallel, but with only 2, yeah, I'd do it on the server. Because the firewall box doesn't really get a lot of updates etc, its often done and in the holding disk in time to be a quite low number in the file count of the printout. Its /lib dir was a level 0, compressed to about 35% last night and #6 in the order taped. Out or 49 dle's. But to say thats a clean install of RH7.3, no, it was my main box for several years. So its got a lot of trash on it that could go out with a fresh broom. Probably 20GB of trash. But I've miss-laid my round tuit in my dotage. :) -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Amanda server selection advice
Gene Heskett wrote: I'm not sure what you would classify as legacy stuff. Here I'm doing 3 machines including this as the server, and of course this is the fastest box, but the clients are a 500mhz K6-III box I use for firewall, and a 1400 mhz athlon box running my milling machine. This one claims to be an XP2800, but actually runs at 2.1ghz. So it pays me about an hour saved overall to let the other boxes do their own thing. Now if there was a dozen old 150 cyrix's out there, it still might pay because they'd all be running in parallel, but with only 2, yeah, I'd do it on the server. Yeah, that's the situation with us, in that we have a very small number of servers, and they're all quite a bit slower (CPU wise) than the backup server, so server side compression is still faster. It really depends on the mix you have, that's for sure. :) Graeme
Re: Amanda server selection advice
Jon LaBadie wrote: Al concurred that the Ultra 40 is the better choice and brought up a number of points in support of his decision that I'd not considered. Things like end of product life and benefits of moving to Sun's new filesystem, ZFS, in the near future. This is a very good point that I hadn't thought of, if you're using Solaris. I've been very excited about ZFS for quite a while, and I'll most likely be running it on my desktop (or triple booting to it, at least) as soon as GNU/Solaris goes beta. ;) Graeme
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 09:46:22AM -0600, Graeme Humphries wrote: stan wrote: I've narrowed the choice of machines down to one of 2 Sun Models. Either a W2100Z, or a Ultra 40. I plan on using an Ultrim tape drive, and doing compression on the server, so I need lot's of CPU power, and of course high I/O throughput. I was wondering what people thought about this selection? Is Sun a requirement in your environment? Just asking, as a dual-core AMD64 would most like be much cheaper, would still give you enough performance to max out the tape write speed, and runs Solaris x86 quite nicely from what I hear. Our current backup server is an old dual Athlon MP 2400+ (running Ubuntu), and *it* has no problem maxing out the tape I/O from two SATA drives in RAID-0. It does struggle on the server-side compression though, which is why we're looking at moving to a newer dual core AMD64. It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
Re: Amanda server selection advice
stan wrote: I've narrowed the choice of machines down to one of 2 Sun Models. Either a W2100Z, or a Ultra 40. I plan on using an Ultrim tape drive, and doing compression on the server, so I need lot's of CPU power, and of course high I/O throughput. I was wondering what people thought about this selection? Is Sun a requirement in your environment? Just asking, as a dual-core AMD64 would most like be much cheaper, would still give you enough performance to max out the tape write speed, and runs Solaris x86 quite nicely from what I hear. Our current backup server is an old dual Athlon MP 2400+ (running Ubuntu), and *it* has no problem maxing out the tape I/O from two SATA drives in RAID-0. It does struggle on the server-side compression though, which is why we're looking at moving to a newer dual core AMD64. The W2100Z is a SCSI machine, but I would still add a 2nd SCSI card dedicated to the tape drive. The Ultra 40 looks like it would have more CPU power, as the processors are dual core. If you need / want to stick with Sun hardware, I'd recommend the Ultra 40, as the server side compression will tax whatever CPU power you can throw at it. I plan on using Solaris 10, if that matters. Is SATA equal to SCSI for the disk I/O? Not equal, per-se, but more recent SATA drives have support for NCQ and other SCSI-type features, so they're very close. Under many server usage patterns, high-end 15k+ RPM SCSI drives will still be faster, but for the kind of I/O workload Amanda does SATA should be more than fast enough. If you run a holding partition that is striped across two (or more) SATA disks you will be able to max out your tape's write speed, or come very close, and you'll be at a tiny fraction of the cost of SCSI. Graeme -- Graeme Humphries ([EMAIL PROTECTED]) (306) 955-7075 ext. 485 My views are not the views of my employers.
Re: Amanda server selection advice
stan wrote: It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. Ahh well, I figured it'd be something like that. In any case, we're doing server side compression, and I can't stress enough that you'll need tons of CPU horsepower on the backup box if you're backing up a large number of systems. Usually, items from our disklist take about 1/4 of the time to blow out to tape that they take to actually dump to the holding disk, and the bottleneck is totally the server side compression. Luckily, fast processors are cheap these days. ;) Graeme
Re: Amanda server selection advice
Graeme Humphries wrote: stan wrote: It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. Ahh well, I figured it'd be something like that. In any case, we're doing server side compression, and I can't stress enough that you'll need tons of CPU horsepower on the backup box if you're backing up a large number of systems. Usually, items from our disklist take about 1/4 of the time to blow out to tape that they take to actually dump to the holding disk, and the bottleneck is totally the server side compression. Luckily, fast processors are cheap these days. ;) Graeme Any reason you don't do client compression? Not only does it give you more CPUs to compress with, it also cuts down on the network bandwidth needed to move the data from the clients to the server. Frank -- Frank Smith [EMAIL PROTECTED] Sr. Systems Administrator Voice: 512-374-4673 Hoover's Online Fax: 512-374-4501
Re: Amanda server selection advice
On Tuesday 31 January 2006 11:50, Graeme Humphries wrote: stan wrote: It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. Ahh well, I figured it'd be something like that. In any case, we're doing server side compression, and I can't stress enough that you'll need tons of CPU horsepower on the backup box if you're backing up a large number of systems. Usually, items from our disklist take about 1/4 of the time to blow out to tape that they take to actually dump to the holding disk, and the bottleneck is totally the server side compression. Luckily, fast processors are cheap these days. ;) Graeme So is offloading the compression duties to the client, which has the advantage of reducing the footprint on the networks bandwidth too. You do this in the dumptype with compress client best. It Just Works(TM) :-) -- Cheers, Gene People having trouble with vz bouncing email to me should add the word 'online' between the 'verizon', and the dot which bypasses vz's stupid bounce rules. I do use spamassassin too. :-) Yahoo.com and AOL/TW attorneys please note, additions to the above message by Gene Heskett are: Copyright 2006 by Maurice Eugene Heskett, all rights reserved.
Re: Amanda server selection advice
Frank Smith wrote: Any reason you don't do client compression? Not only does it give you more CPUs to compress with, it also cuts down on the network bandwidth needed to move the data from the clients to the server. The reason is simply because in our setup, it *doesn't* give us more CPUs to compress with. Most of the backup is being pulled off our main file server, which is a dual P3-500. When doing client side compression, this slows the fileserver to a crawl while backups are happening, and it takes longer than doing server side compression on the dual Athlon 2400+ backup server. ;) Graeme
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 10:50:35AM -0600, Graeme Humphries wrote: stan wrote: It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. Ahh well, I figured it'd be something like that. In any case, we're doing server side compression, and I can't stress enough that you'll need tons of CPU horsepower on the backup box if you're backing up a large number of systems. Usually, items from our disklist take about 1/4 of the time to blow out to tape that they take to actually dump to the holding disk, and the bottleneck is totally the server side compression. Luckily, fast processors are cheap these days. ;) Which sort of leads directly back to the original question. Which of the 2 bxes I mentioned originally would have the most CPU poweer? I'm failry certian it's the Ultra 40, but I could be wrong. -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
Re: Amanda server selection advice
stan wrote: Which sort of leads directly back to the original question. Which of the 2 bxes I mentioned originally would have the most CPU poweer? I'm failry certian it's the Ultra 40, but I could be wrong. I think I said in my original message that I'd suspect the Ultra 40 as well, but maybe someone who knows Sun hardware better can comment? Graeme
Re: Amanda server selection advice
Hi I would guess the Ultra 40 if you go for the dual core option, there may not be much of a difference in the single core options 2.8GHz vs 2.6GHz. Form the sun web site Ultra 40 Processor One or two AMD Opteron 940-pin, 200-series single-core CPUs that range from 2.0 GHz to 2.8 GHz (models 246, 250, and 254) and dual-core CPUs that range from 2.2 GHz to 2.4 GHz (models 275 and 280) with three 8-GBps HyperTransport interconnects per CPU W2100z Processor Two AMD Opteron 200 Series CPUs that range from Model 244 (1.8 GHz) to Model 252 (2.6 GHz) Anthony Worrall On Tue, 2006-01-31 at 17:28, stan wrote: On Tue, Jan 31, 2006 at 10:50:35AM -0600, Graeme Humphries wrote: stan wrote: It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. Ahh well, I figured it'd be something like that. In any case, we're doing server side compression, and I can't stress enough that you'll need tons of CPU horsepower on the backup box if you're backing up a large number of systems. Usually, items from our disklist take about 1/4 of the time to blow out to tape that they take to actually dump to the holding disk, and the bottleneck is totally the server side compression. Luckily, fast processors are cheap these days. ;) Which sort of leads directly back to the original question. Which of the 2 bxes I mentioned originally would have the most CPU poweer? I'm failry certian it's the Ultra 40, but I could be wrong. -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 11:15:16AM -0600, Frank Smith wrote: Graeme Humphries wrote: stan wrote: It's one of those corporate political corectness things. Management recognizes the nae, and if I sugest a non name brand, I have to a +lot_ more expalining. Ahh well, I figured it'd be something like that. In any case, we're doing server side compression, and I can't stress enough that you'll need tons of CPU horsepower on the backup box if you're backing up a large number of systems. Usually, items from our disklist take about 1/4 of the time to blow out to tape that they take to actually dump to the holding disk, and the bottleneck is totally the server side compression. Luckily, fast processors are cheap these days. ;) Graeme Any reason you don't do client compression? Not only does it give you more CPUs to compress with, it also cuts down on the network bandwidth needed to move the data from the clients to the server. Actually the system I'm upgrading _does_ do client compression. But we are upgrading the network from 10M to Gigabit. nd a lot of the clients are _really_ old machine (100MHZ SPARCS for instance), so I'm anxious to get that load off of them Thus the change. -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 11:42:26AM -0600, Graeme Humphries wrote: stan wrote: Which sort of leads directly back to the original question. Which of the 2 bxes I mentioned originally would have the most CPU poweer? I'm failry certian it's the Ultra 40, but I could be wrong. I think I said in my original message that I'd suspect the Ultra 40 as well, but maybe someone who knows Sun hardware better can comment? Ah, thnaks, I missed that. -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 06:15:31PM +, Anthony Worrall wrote: Hi I would guess the Ultra 40 if you go for the dual core option, there may not be much of a difference in the single core options 2.8GHz vs 2.6GHz. Form the sun web site Ultra 40 Processor One or two AMD Opteron 940-pin, 200-series single-core CPUs that range from 2.0 GHz to 2.8 GHz (models 246, 250, and 254) and dual-core CPUs that range from 2.2 GHz to 2.4 GHz (models 275 and 280) with three 8-GBps HyperTransport interconnects per CPU W2100z Processor Two AMD Opteron 200 Series CPUs that range from Model 244 (1.8 GHz) to Model 252 (2.6 GHz) Thanks. So the consensus seems to be get the higher powerd CPU unit, even though it's SATA vs SCSI for the other one. BTW, the machines in question are the top end of each of these 2 product lines. So it is a dual core versus single core choice. -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
Re: Amanda server selection advice
stan schrieb: Thanks. So the consensus seems to be get the higher powerd CPU unit, even though it's SATA vs SCSI for the other one. BTW, the machines in question are the top end of each of these 2 product lines. So it is a dual core versus single core choice. Errm .. Sorry, if I don't have the whole thread in my *stack* now: Read something about 100MHz-clients etc. My suggestion: Combine many DLEs with dumptypes like compress client fast/best with a AMANDA-server that is capable to store all those DLEs on its holdingdisk(s). For 100MHz-clients I assume that any of your mentioned choices would suffice, with proper AMANDA-setup given. IMO it will do NO DIFFERENCE choosing dual- vs. single-core-CPU here. -- I think we would need more background-information here: Why do you think you need a stronger CPU? Won't different DLE-definitions help here? What are your overall dumpsizes right now? What limits do you hit? Stefan
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 10:01:33AM -0500, stan wrote: I'm going to upgrade our Amanda tape/index server. I've narrowed the choice of machines down to one of 2 Sun Models. Either a W2100Z, or a Ultra 40. I plan on using an Ultrim tape drive, and doing compression on the server, so I need lot's of CPU power, and of course high I/O throughput. I was wondering what people thought about this selection? The W2100Z is a SCSI machine, but I would still add a 2nd SCSI card dedicated to the tape drive. The Ultra 40 looks like it would have more CPU power, as the processors are dual core. I plan on using Solaris 10, if that matters. Is SATA equal to SCSI for the disk I/O? My first impression was SATA is fine for holding and 4 threads of the Ultra 40 was a big plus. But I decided to ask someone, Al Hopper, from the Solaris x86 mailing list whose opinion I value. Al concurred that the Ultra 40 is the better choice and brought up a number of points in support of his decision that I'd not considered. Things like end of product life and benefits of moving to Sun's new filesystem, ZFS, in the near future. Its a little off topic, but I thought my question and Al's reply worth posting. On Tuesday, Jan 31 2006, Al Hooper replied: On Tue, 31 Jan 2006, Jon LaBadie wrote: Al, On another mailing list, dealing with amanda backups a poster asked about choices for a backup server. He was questioning the Ultra 40 vs the W2100z. Both would be dual processors, the Ultra 40 dual core as well, but at a slightly lower clock speed. Assuming you are unfamiliar with amanda, the backup server would be receiving multiple, simultaneous network connections receiving tar data. Each connection would be piped through two processes (one a gzip) and collected on a holding disk. This holding disk would be SCSI on the W2100z, SATA on the Ultra 40. Dumps that have completly collected on the holding disk are queue for transfer to tape, one at a time. This would be occuring at the same time as the collection of other dumps. This involves two other processes, but is essentially a dd to tape. The tape drive will be an LTO (2 or 3?) connected to its own add-in SCSI controller. LTO drives today seem to be able to saturate a controller's I/O capability. My intuition says the SATA drives of the Ultra 40 should be fine for collecting the data, so lack of SCSI here is no problem. And the ability to run 4 slightly slower threads vs 2 faster ones in the W2100z could be a big factor for lots of gzip'ping. Do you see anything strongly pushing one solution over another? Yes! But you knew that already! :) ZFS will ship with the next Solaris 10 Update - that'll be Update 2, and it should ship on-time. It's just possible (looking in my crystal ball) that you'll be able to boot off ZFS in update 2 - because I've seen blogs that are 2 months+ old saying they had ZFS booting. At that point in time - using anything other than cheap fast SATA disk drives just won't make sense IMHO. SCSI is EOL technology - as I've pointed out before on the list - the primary reason is that commands are sent over the bus in s-l-o-w legacy xfer rate (5Mb/Sec) 8-bits wide[0]. And since it takes on average 5 SCSI commands to do anything useful, you simply *cannot* get enough commands to the disk drive (even a single disk drive on a dedicated SCSI bus) to keep the disk drive busy. Therefore, you're wasting your money on SCSI and buying EOL technology. ZFS, even in its current, untuned, and CPU intensive form, will give you better performance - even though it won't saturate the SATA drives at this time. The next rev, after the code has been optimized, will probably double throughput IMHO (WAG etc). But in the first rev, right now via Solaris Express, it'll easily do 100M bytes/Sec[1] without even trying. In the meantime, assuming you won't be able to boot off ZFS, just boot off one SATA (dedicated OS) drive and you'll have 3 other SATA drives to play with using regular SVM - or ZFS if you want to load the next Solaris express release on the new box (build 31 or 32 should ship in the next week). When ZFS ships, use all 4 SATA drives in a raidz config for everything (data, OS, swap, home etc). This may sound conter-intutive - using a 4-drive raidz for everything, including your backup processes, but that is exactly what ZFS is designed to do. Also don't forget that you can setup a ZFS filesystem with compression enabled - so you could spool your incoming data stream to a ZFS compressed filesystem - then your read performance would be 2x - depending on the compressibility of your incoming data stream. Since the W1100 and W2100 are based on designs made by someone else, they'll be EOLed ASAP AFAICT. This is my personal opinion only. The Ultra40 is Suns long term solution to a deskside dualie tower and it will be better supported going forward IMHO. You
Re: Amanda server selection advice
On Tue, Jan 31, 2006 at 09:00:59PM -0500, Jon LaBadie wrote: On Tue, Jan 31, 2006 at 10:01:33AM -0500, stan wrote: I'm going to upgrade our Amanda tape/index server. I've narrowed the choice of machines down to one of 2 Sun Models. Either a W2100Z, or a Ultra 40. I plan on using an Ultrim tape drive, and doing compression on the server, so I need lot's of CPU power, and of course high I/O throughput. I was wondering what people thought about this selection? The W2100Z is a SCSI machine, but I would still add a 2nd SCSI card dedicated to the tape drive. The Ultra 40 looks like it would have more CPU power, as the processors are dual core. I plan on using Solaris 10, if that matters. Is SATA equal to SCSI for the disk I/O? My first impression was SATA is fine for holding and 4 threads of the Ultra 40 was a big plus. But I decided to ask someone, Al Hopper, from the Solaris x86 mailing list whose opinion I value. Al concurred that the Ultra 40 is the better choice and brought up a number of points in support of his decision that I'd not considered. Things like end of product life and benefits of moving to Sun's new filesystem, ZFS, in the near future. Thanks! My mind is now made up :-) He has very convincing arguments. And I may need to think about Amanda configs a year or so down the road. Let's see virtual tapes on big disk with the disk driver itself doing the compression.. Thanks for taking the time to explain this so well to your resource, and thank him for his excellent analasys! -- U.S. Encouraged by Vietnam Vote - Officials Cite 83% Turnout Despite Vietcong Terror - New York Times 9/3/1967
