split RAID1 during backups?

2005-10-24 Thread Jeff Breidenbach
Hi all, I have a two drive RAID1 serving data for a busy website. The partition is 500GB and contains millions of 10KB files. For reference, here's /proc/mdstat Personalities : [raid1] md0 : active raid1 sdc1[0] sdd1[1] 488383936 blocks [2/2] [UU] For backups, I set the md0 partition to

Re: split RAID1 during backups?

2005-10-24 Thread Jurriaan Kalkman
Hi all, I have a two drive RAID1 serving data for a busy website. The partition is 500GB and contains millions of 10KB files. For reference, here's /proc/mdstat Personalities : [raid1] md0 : active raid1 sdc1[0] sdd1[1] 488383936 blocks [2/2] [UU] For backups, I set the md0

Re: split RAID1 during backups?

2005-10-24 Thread Brad Campbell
Jeff Breidenbach wrote: So - I'm thinking of the following backup scenario. First, remount /dev/md0 readonly just to be safe. Then mount the two component paritions (sdc1, sdd1) readonly. Tell the webserver to work from one component partition, and tell the backup process to work from the

Re: split RAID1 during backups?

2005-10-24 Thread Jeff Breidenbach
First of all, if the data is mostly static, rsync might work faster. Any operation that stats the individual files - even to just look at timestamps - takes about two weeks. Therefore it is hard for me to see rsync as a viable solution, even though the data is mostly static. About 400,000 files

Re: split RAID1 during backups?

2005-10-24 Thread dean gaudet
On Mon, 24 Oct 2005, Jeff Breidenbach wrote: First of all, if the data is mostly static, rsync might work faster. Any operation that stats the individual files - even to just look at timestamps - takes about two weeks. Therefore it is hard for me to see rsync as a viable solution, even

Re: split RAID1 during backups?

2005-10-24 Thread John Stoffel
Jeff == Jeff Breidenbach jeff@jab.org writes: Jeff # mount | grep md0 Jeff /dev/md0 on /data1 type reiserfs (rw,noatime,nodiratime) Ah, you're using reiserfs on here. It may or may not be having problems with all those files per-directory that you have. Is there any way you can split them up

Re: split RAID1 during backups?

2005-10-24 Thread dean gaudet
On Mon, 24 Oct 2005, Jeff Breidenbach wrote: Dean, the comment about write-mostly is confusing to me. Let's say I somehow marked one of the component drives write-mostly to quiet it down. How do I get at it? Linux will not let me mount the component partition if md0 is also mounted. Do you

Re: split RAID1 during backups?

2005-10-24 Thread Thomas Garner
Should there be any consideration for the utilization of the gigabit interface that is passing all of this backup traffic, as well as the speed of the drive that is doing all of the writing during this transaction? Is the 18MB/s how fast the data is being copied over the network, or is it

Re: split RAID1 during backups?

2005-10-24 Thread Jeff Breidenbach
On 10/24/05, Thomas Garner [EMAIL PROTECTED] wrote: Should there be any consideration for the utilization of the gigabit interface that is passing all of this backup traffic, as well as the speed of the drive that is doing all of the writing during this transaction? Is the 18MB/s how fast