Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-24 Thread Chris Murphy
On Jul 22, 2014, at 11:13 AM, Chris Murphy wrote: > > It's been a while since I did a rebuild on HDDs, So I did this yesterday and day before with an SSD and HDD in raid1, and made the HDD do the rebuild. Baseline for this hard drive: hdparm -t 35.68 MB/sec dd if=/dev/zero of=/dev/rdisk2s

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-22 Thread TM
Stefan Behrens giantdisaster.de> writes: > TM, Just read the man-page. You could have used the replace tool after > physically removing the failing device. > > Quoting the man page: > "If the source device is not available anymore, or if the -r option is > set, the data is built only using the

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-22 Thread Chris Murphy
On Jul 21, 2014, at 8:51 PM, Duncan <1i5t5.dun...@cox.net> wrote: > >> It does not matter at all what the average file size is. > > … and the filesize /does/ matter. I'm not sure how. A rebuild is replicating chunks, not doing the equivalent of cp or rsync on files. Copying chunks (or strips

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-22 Thread Stefan Behrens
On Tue, 22 Jul 2014 14:43:45 + (UTC), Tm wrote: > Wang Shilong cn.fujitsu.com> writes: > > >> The latest btrfs-progs include man page of btrfs-replace. Actually, you >> could use it >> something like: >> >> btrfs replace start | >> >> You could use 'btrfs file show' to see missing device

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-22 Thread TM
Wang Shilong cn.fujitsu.com> writes: > The latest btrfs-progs include man page of btrfs-replace. Actually, you > could use it > something like: > > btrfs replace start | > > You could use 'btrfs file show' to see missing device id. and then run > btrfs replace. > Hi Wang, I physicall

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-21 Thread Duncan
ronnie sahlberg posted on Mon, 21 Jul 2014 09:46:07 -0700 as excerpted: > On Sun, Jul 20, 2014 at 7:48 PM, Duncan <1i5t5.dun...@cox.net> wrote: >> ashford posted on Sun, 20 Jul 2014 12:59:21 -0700 as excerpted: >> >>> If you assume a 12ms average seek time (normal for 7200RPM SATA >>> drives), an

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-21 Thread Wang Shilong
On 07/21/2014 10:00 PM, TM wrote: Wang Shilong cn.fujitsu.com> writes: Just my two cents: Since 'btrfs replace' support RADI10, I suppose using replace operation is better than 'device removal and add'. Another Question is related to btrfs snapshot-aware balance. How many snapshots did you h

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-21 Thread Wang Shilong
On 07/21/2014 10:00 PM, TM wrote: Wang Shilong cn.fujitsu.com> writes: Just my two cents: Since 'btrfs replace' support RADI10, I suppose using replace operation is better than 'device removal and add'. Another Question is related to btrfs snapshot-aware balance. How many snapshots did you h

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-21 Thread Chris Murphy
On Jul 21, 2014, at 10:46 AM, ronnie sahlberg wrote: > On Sun, Jul 20, 2014 at 7:48 PM, Duncan <1i5t5.dun...@cox.net> wrote: >> ashford posted on Sun, 20 Jul 2014 12:59:21 -0700 as excerpted: >> >>> If you assume a 12ms average seek time (normal for 7200RPM SATA drives), >>> an 8.3ms rotational

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-21 Thread ronnie sahlberg
On Sun, Jul 20, 2014 at 7:48 PM, Duncan <1i5t5.dun...@cox.net> wrote: > ashford posted on Sun, 20 Jul 2014 12:59:21 -0700 as excerpted: > >> If you assume a 12ms average seek time (normal for 7200RPM SATA drives), >> an 8.3ms rotational latency (half a rotation), an average 64kb write and >> a 100M

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-21 Thread TM
Wang Shilong cn.fujitsu.com> writes: > Just my two cents: > > Since 'btrfs replace' support RADI10, I suppose using replace > operation is better than 'device removal and add'. > > Another Question is related to btrfs snapshot-aware balance. > How many snapshots did you have in your system? >

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Duncan
ashford posted on Sun, 20 Jul 2014 12:59:21 -0700 as excerpted: > If you assume a 12ms average seek time (normal for 7200RPM SATA drives), > an 8.3ms rotational latency (half a rotation), an average 64kb write and > a 100MB/S streaming write speed, each write comes in at ~21ms, which > gives us ~4

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Wang Shilong
Hi, On 07/20/2014 04:45 PM, TM wrote: Hi, I have a raid10 with 4x 3TB disks on a microserver http://n40l.wikia.com/wiki/Base_Hardware_N54L , 8Gb RAM Recently one disk started to fail (smart errors), so I replaced it Mounted as degraded, added new disk, removed old Started yesterday I am monito

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread George Mitchell
On 07/20/2014 02:28 PM, Bob Marley wrote: On 20/07/2014 21:36, Roman Mamedov wrote: On Sun, 20 Jul 2014 21:15:31 +0200 Bob Marley wrote: Hi TM, are you doing other significant filesystem activity during this rebuild, especially random accesses? This can reduce performances a lot on HDDs. E.g.

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Bob Marley
On 20/07/2014 21:36, Roman Mamedov wrote: On Sun, 20 Jul 2014 21:15:31 +0200 Bob Marley wrote: Hi TM, are you doing other significant filesystem activity during this rebuild, especially random accesses? This can reduce performances a lot on HDDs. E.g. if you were doing strenous multithreaded r

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread ashford
This is the cause for the slow reconstruct. > I believe the problem here might be that a Btrfs rebuild *is* a strenuous > random read (+ random-ish write) just by itself. If you assume a 12ms average seek time (normal for 7200RPM SATA drives), an 8.3ms rotational latency (half a rotation), an ave

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Roman Mamedov
On Sun, 20 Jul 2014 21:15:31 +0200 Bob Marley wrote: > Hi TM, are you doing other significant filesystem activity during this > rebuild, especially random accesses? > This can reduce performances a lot on HDDs. > E.g. if you were doing strenous multithreaded random writes in the > meanwhile, I

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Bob Marley
On 20/07/2014 10:45, TM wrote: Hi, I have a raid10 with 4x 3TB disks on a microserver http://n40l.wikia.com/wiki/Base_Hardware_N54L , 8Gb RAM Recently one disk started to fail (smart errors), so I replaced it Mounted as degraded, added new disk, removed old Started yesterday I am monitoring /va

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread TM
whisperpc.com> writes: > > Finally, TM didn't mention anything about other I/O activity on the array, > which, regardless of the method of reconstruction, could have a > significant impact on the speed of a reconstruction. > > There are a LOT of parameters here that could impact throughput. S

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread TM
whisperpc.com> writes: > > Finally, TM didn't mention anything about other I/O activity on the array, > which, regardless of the method of reconstruction, could have a > significant impact on the speed of a reconstruction. > > There are a LOT of parameters here that could impact throughput. S

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread ashford
Tomasz, > On Sun, Jul 20, 2014 at 01:53:34PM +, Duncan wrote: >> TM posted on Sun, 20 Jul 2014 08:45:51 + as excerpted: >> >>> One week for a raid10 rebuild 4x3TB drives is a very long time. >>> Any thoughts? >>> Can you share any statistics from your RAID10 rebuilds? >> >> At a week, that

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Austin S Hemmelgarn
On 07/20/2014 10:00 AM, Tomasz Torcz wrote: > On Sun, Jul 20, 2014 at 01:53:34PM +, Duncan wrote: >> TM posted on Sun, 20 Jul 2014 08:45:51 + as excerpted: >> >>> One week for a raid10 rebuild 4x3TB drives is a very long time. >>> Any thoughts? >>> Can you share any statistics from your RAI

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Tomasz Torcz
On Sun, Jul 20, 2014 at 01:53:34PM +, Duncan wrote: > TM posted on Sun, 20 Jul 2014 08:45:51 + as excerpted: > > > One week for a raid10 rebuild 4x3TB drives is a very long time. > > Any thoughts? > > Can you share any statistics from your RAID10 rebuilds? > > > At a week, that's nearly

Re: 1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread Duncan
TM posted on Sun, 20 Jul 2014 08:45:51 + as excerpted: > One week for a raid10 rebuild 4x3TB drives is a very long time. > Any thoughts? > Can you share any statistics from your RAID10 rebuilds? Well, 3 TB is big and spinning rust is slow. Even using the smaller power-of-10 (1000) figures f

1 week to rebuid 4x 3TB raid10 is a long time!

2014-07-20 Thread TM
Hi, I have a raid10 with 4x 3TB disks on a microserver http://n40l.wikia.com/wiki/Base_Hardware_N54L , 8Gb RAM Recently one disk started to fail (smart errors), so I replaced it Mounted as degraded, added new disk, removed old Started yesterday I am monitoring /var/log/messages and it seems it wi