Re: [Veritas-bu] Storage and Performance HP & Veritas Net Backup

2006-07-04 Thread Austin Murphy

On 7/1/06, Giblin Dean L. <[EMAIL PROTECTED]> wrote:

Backup Performance – The 8GB/Min Goal

 < roughly 140MB/sec, continuous for 8 hrs >


Our future storage space requirements were estimated to be 4 TB. Our
requirement is to complete the backups within an 8 hour window.  To meet
this objective required a backup solution that could provide an 8 GB/Min
solution.



Hardware:
DL580 G3 ,  Dual Processor, 4GB, LSI 320 to tape, HP RAID controller
MSA30 DB ,  14x  15K, 146 GB
MSL6030, Dual LTO 3



·HP SCSI drivers for Microsoft Windows have a 64K block size
limitation. All the test tools which could bypass this driver and set
communication parameters on the SCSI bus obtained increased performance. All of 
the applications which utilized the system driver suffered.


Supposedly there are Veritas drivers for windows that are faster.  See
previous discussions.


Our original design was to implement a RAID 10 solution using two 7 disk
RAID 0 arrays. This results in 490 GB of storage and approximately 1.8
GB/min data rate.


A 7x 146 GB RAID 5 array should EASILY give you 100MB/sec read
performance.  Probably much better.

The key to speed in NetBackup is SEPARATE I/O CHANNELS!  There is a
great performance document (which I can't find right now) that shows
how Oracle, Sun and Veritas demonstrated a 1TB/hr setup several years
ago.

Using 2 separate U320 SCSI channels you should be able to have both
halves of the MSA30-DB spitting out 100MB/sec or more.

Each LTO-3 tape drive needs its own U320 SCSI channel.  (The robotics
can piggyback on one of those channels.)

The system disks in the server itself need a separate SCSI channel as
well.  That brings us to a total of 5 separate high-speed SCSI
channels.  If you don't currently have 5 separate channels, you need
to add some of these: http://lsi.com/products/scsi_hbas/lsiu320.html

The other side of the equation is the network performance.  To backup
a NAS box over the network you need to optimize the network path.
Jumbo frames will probably help.  Aggregating 2 gigabit ports might
help too.   You didn't say how many other clients will be backing up
over the network.  If there are many clients, spread them across
multiple gigabit interfaces on the server.

To keep the LTO-3 drives happy, you need to feed them at least 40
MB/sec.  To meet your design goal of 8 GB/min you need to continuously
feed 2 drives @ 70 MB/sec each.  That will be fairly difficult and
will probably require 4 separate gigabit interfaces.  I suggest adding
one of these to the two built-in NICs:
http://intel.com/network/connectivity/products/pro1000mt_dual_server_adapter.htm

The difficulty lies in having the clients produce data that fast.  You
will most likely need to have the client data streams multiplexed @ 4
or 8 streams per tape drive.
Disk staging is useful for slow clients, but is probably a negative
factor for the NAS box.

Here's my summary:
- Add I/O channels for a total of 4 gigabit Ethernet and 5 U320 SCSI channels.
- Optimize the network between the NAS box and the backup server.
- Send the NAS data directly to a single tape drive.
- Configure 1 LTO-3 tape drive per SCSI channel
- Configure 1 RAID5 array per SCSI channel
- Separate the system disks from the backup SCSI channels
- Use a High MPX on client backups to tape.
- Use disk staging only for VERY slow clients.
- Use the Vertias drivers where available, not Microsoft or HP.

You need to be able to read or write at over 100 MB/sec on each SCSI
channel and you will need to read from the network at over 50MB/sec on
each Gigabit interface ALL AT THE SAME TIME.  With a Solid PCI-X
backplane (like the DL 580G3 has) this is entirely reasonable.

Austin Murphy

___
Veritas-bu maillist  -  Veritas-bu@mailman.eng.auburn.edu
http://mailman.eng.auburn.edu/mailman/listinfo/veritas-bu


[Veritas-bu] Storage and Performance HP & Veritas Net Backup

2006-07-01 Thread Giblin Dean L.










Backup
Performance – The 8GB/Min Goal

 

Our future storage space
requirements were estimated to be 4 TB. Our requirement is to complete the
backups within an 8 hour window.  To meet this objective required a backup
solution that could provide an 8 GB/Min solution. We researched several
technologies to obtain the required level of performance. After months of
research we decided to base our solution around an article written by HP,
“Getting the most performance from your HP StorageWorks Ultrium 960 tape
drive[1]”.
This white paper discussed several technologies and methodologies to obtain the
performance levels desired. The article also provide information for combining
our backup solution of choice, Veritas NetBackup[2] and
used hardware that had excellent reviews[3].
During this process we decided to use a DAS and NAS storage solution. We
decided to avoid SAN storage and disk to disk storage because of the overhead,
complexity, and cost associated with these technologies. In addition, we
decided to implement a single vendor solution to enhance support. Once our
design was in place with worked with multiple resellers and the manufacture, HP
directly to insure the equipment ordered with meet or exceed the performance
levels outlined in the HP performance article above. Once the equipment was in
place we were unable to obtain the results documented. The article highlights
some of our findings along the way. I think our findings will raise an eyebrow
or two and you will find it a worthy read. After our ordeal we did identify a
technique to obtain the performance desired. After reading this article if you
have experienced the same challenges or are equally disappointed with
HP’s position we would like to hear from you. Below is our tale.

 

Objective: Setup a simple, scalable, high performance single
vendor DAS backup solution.

 

Hardware:

 

DL580 G3

   Dual
Processor

   4
  GB RAM (2 GB per processor)

   4 
15K RPM drives

LSI Ultra 320 SCSI
Controller for Tape Storage Unit

   Max
SCSI cache memory and battery to enable write back cache

 

MSA500 G2

   
Model with dual Ultra 320 SCSI channel

   
14  15K RPM, 72 GB Drives for performance

 

MSA30

   
14  15K RPM, 72 GB Drives for performance

   
Model with dual Ultra 320 SCSI channels

 

MSL6030 

    
17 TB Backup storage capacity

   
Dual LTO 3 drives for performance and required backup operations

 




Software Tools Utilized:

Intel Iometer

HP PAT (Performance Analysis Tool)

HP Read Data

Windows Explorer file copy

HP Library & Tape tool Prebackup
test

 

Tests:

    Over several months working with HP we tried a
myriad of configurations and tests. 

 

These tests included but are not limited to the following. 

    Various Windows 2003 Server Service Packs &
Hotfixes

    BIOS updates for system, SCSI, Drives, Tape Lib,
Tape Drive etc.

    Driver updates and versions for system, array
controller, and tape

    Differing drivers to include both SCSI Mini-Port
and Non-Mini-Port drivers

    Changing strip size on drive arrays

    Changing RAID level from 0, 1, 5, and 10.

    Changing Read/Write cache Levels

    Changing SCSI driver parameters

    Changing software buffer, memory, and block size
parameters

    Using dual data channels to MSA500

    These tests were repeated using an MSA30.

    

Testing Results: 

·   
Stripe size, Number of drives and
RAID level made no significance impact on the performance.

·   
MSA 500 G2 does not use two independent SCSI
channels. The second SCSI interface is only for failover and redundancy. With
the same configuration MSA 30 outperforms MSA 500 G2 using single SCSI channel.

·   
 “Iometer” provides unusually high
through put results. We learned that this tool bypasses the Windows system
drivers. Therefore this tool is not an accurate measure of the performance.

·   
 “HP PAT” also bypasses Windows system
drivers and is not recommended for benchmarking.

·   
The test data created by “HP Create Data”
utility is euphoria data and is not realistic. The data sizes created are
perfect and no performance loss is realized with SCSI data packets that range
in sizes. Expect your results to be approximately 50% or worse then the
performance numbers obtained using this data. “HP Read data”,
“HP Prebackup test”, “Windows Explorer” file copy and
“VERITAS Netbackup” Null test provided comparable performance
results and were consistent with ‘real world’ findings.

·   
Performance was never any better than it would have
been with a single drive. It runs as fast as the single drive.

·   
Performance was related to number of logical units
(LUN). Results obtained were 3 to 4 times better when LUNs were increased.

·   
HP SCSI drivers for Microsoft Windows have a 64K
block size