Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Chris Murphy
On Jan 9, 2014, at 6:31 PM, George Mitchell geo...@chinilu.com wrote: Jim, my point was that IF the drive does not successfully resolve the bad block issue and btrfs takes a write failure every time it attempts to overwrite the bad data, it is not going to remap that data, but rather it is

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Roman Mamedov
On Tue, 14 Jan 2014 12:13:09 -0700 Chris Murphy li...@colorremedies.com wrote: On Jan 9, 2014, at 6:31 PM, George Mitchell geo...@chinilu.com wrote: Jim, my point was that IF the drive does not successfully resolve the bad block issue and btrfs takes a write failure every time it

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 11:13 AM, Chris Murphy wrote: On Jan 9, 2014, at 6:31 PM, George Mitchell geo...@chinilu.com wrote: Jim, my point was that IF the drive does not successfully resolve the bad block issue and btrfs takes a write failure every time it attempts to overwrite the bad data, it is not

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Roman Mamedov
On Tue, 14 Jan 2014 12:29:28 -0800 George Mitchell geo...@chinilu.com wrote: what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. You can configure smartd (from smartmontools)

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Chris Murphy
On Jan 14, 2014, at 12:37 PM, Roman Mamedov r...@romanrm.net wrote: I vaguely remember having some drives that were not able to remap a single block on write, but doing that successfully if I overwrote a sizable area around (and including) that block, or overwrite the whole drive. And after

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Hugo Mills
On Wed, Jan 15, 2014 at 03:00:21AM +0600, Roman Mamedov wrote: That said, do not fall into a false sense of security relying on proprietary, barely if ever updated after the device has been shipped, and often very peculiar-behaving SMART routines inside the black-box HDD firmware as your most

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Chris Murphy
On Jan 14, 2014, at 1:29 PM, George Mitchell geo...@chinilu.com wrote: Chris, Please don't misunderstand me. I am not advocating that btrfs or any other filesystem should be dealing with bad blocks. I believe very strongly that if the drive firmware can't deal with that transparently

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Roman Mamedov
On Tue, 14 Jan 2014 14:05:11 -0700 Chris Murphy li...@colorremedies.com wrote: On Jan 14, 2014, at 12:37 PM, Roman Mamedov r...@romanrm.net wrote: I vaguely remember having some drives that were not able to remap a single block on write, but doing that successfully if I overwrote a

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Chris Murphy
On Jan 14, 2014, at 2:06 PM, Hugo Mills h...@carfax.org.uk wrote: On Wed, Jan 15, 2014 at 03:00:21AM +0600, Roman Mamedov wrote: That said, do not fall into a false sense of security relying on proprietary, barely if ever updated after the device has been shipped, and often very

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:00 PM, Roman Mamedov wrote: On Tue, 14 Jan 2014 12:29:28 -0800 George Mitchell geo...@chinilu.com wrote: what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. You

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:00 PM, Roman Mamedov wrote: On Tue, 14 Jan 2014 12:29:28 -0800 George Mitchell geo...@chinilu.com wrote: what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. You

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Chris Murphy
On Jan 14, 2014, at 2:19 PM, Roman Mamedov r...@romanrm.net wrote: On Tue, 14 Jan 2014 14:05:11 -0700 Chris Murphy li...@colorremedies.com wrote: On Jan 14, 2014, at 12:37 PM, Roman Mamedov r...@romanrm.net wrote: I vaguely remember having some drives that were not able to remap a

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Chris Murphy
On Jan 14, 2014, at 2:37 PM, Chris Murphy li...@colorremedies.com wrote: I've seen that happen on OS X Server (client doesn't produce SMART warnings in user space). Oops. It does, just not automatically, it seems you have to go look for this in Disk Utility. Chris Murphy -- To unsubscribe

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:14 PM, Chris Murphy wrote: On Jan 14, 2014, at 1:29 PM, George Mitchell geo...@chinilu.com wrote: And the key to monitoring hard drive health, in my opinion, is SMART and what we are lacking at this point is a SMART capability to provide visual notifications to the user when

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:14 PM, Chris Murphy wrote: On Jan 14, 2014, at 1:29 PM, George Mitchell geo...@chinilu.com wrote: And the key to monitoring hard drive health, in my opinion, is SMART and what we are lacking at this point is a SMART capability to provide visual notifications to the user when

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread Roman Mamedov
On Tue, 14 Jan 2014 14:37:46 -0700 Chris Murphy li...@colorremedies.com wrote: Reserve sectors are fundamental to ECC. If there are no more reserves, the status should be a failed drive, it can no longer do its own relocation of data experiencing transient read errors in this case. With the

Re: How does btrfs handle bad blocks in raid1?

2014-01-14 Thread George Mitchell
On 01/14/2014 01:14 PM, Chris Murphy wrote: And the key to monitoring hard drive health, in my opinion, is SMART and what we are lacking at this point is a SMART capability to provide visual notifications to the user when any hard drive starts to seriously degrade or suddenly fails. Gnome

Re: How does btrfs handle bad blocks in raid1?

2014-01-10 Thread Duncan
George Eleftheriou posted on Thu, 09 Jan 2014 17:49:48 +0100 as excerpted: I'm really looking forward to the day that typing: mkfs.btrfs -d raid10 -m raid10 /dev/sd[abcd] will do exactly what is expected to do. A true RAID10 resilient in 2 disks' failure. Simple and beautiful. We're

Re: How does btrfs handle bad blocks in raid1?

2014-01-10 Thread George Mitchell
On 01/10/2014 07:27 AM, Duncan wrote: George Eleftheriou posted on Thu, 09 Jan 2014 17:49:48 +0100 as excerpted: I'm really looking forward to the day that typing: mkfs.btrfs -d raid10 -m raid10 /dev/sd[abcd] will do exactly what is expected to do. A true RAID10 resilient in 2 disks'

Re: How does btrfs handle bad blocks in raid1?

2014-01-10 Thread Duncan
Chris Murphy posted on Thu, 09 Jan 2014 11:52:08 -0700 as excerpted: Understood. I'm considering a 2nd drive dying during rebuild (from a 1st drive dying) as essentially simultaneous failures. And in the case of raid10, the likelihood of a 2nd drive failure being the lonesome drive in a

How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Clemens Eisserer
Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be horribly unreliable. A 32GB microSDHC card reported bad blocks after 4 days, while a usb pen drive returns bogus data without any warning at all. So I wonder, how would btrfs

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Hugo Mills
On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be horribly unreliable. A 32GB microSDHC card reported bad blocks after 4 days, while a usb pen drive returns

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Duncan
Hugo Mills posted on Thu, 09 Jan 2014 10:42:47 + as excerpted: On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be horribly unreliable. A 32GB microSDHC

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Austin S Hemmelgarn
On 2014-01-09 07:41, Duncan wrote: Hugo Mills posted on Thu, 09 Jan 2014 10:42:47 + as excerpted: On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Mason
On Thu, 2014-01-09 at 12:41 +, Duncan wrote: Hugo Mills posted on Thu, 09 Jan 2014 10:42:47 + as excerpted: On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Duncan
Austin S Hemmelgarn posted on Thu, 09 Jan 2014 07:52:44 -0500 as excerpted: On 2014-01-09 07:41, Duncan wrote: Hugo Mills posted on Thu, 09 Jan 2014 10:42:47 + as excerpted: If a [btrfs ]block is read and fails its checksum, then the other copy (in RAID-1) is checked and used if it's

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Eleftheriou
Duncan, As a silent reader of this list (for almost a year)... As an anonymous supporter of the BAARF (Battle Against Any RAID Four/Five/Six/ Z etc...) initiative... I can only break my silence and applaud your frequent interventions referring to N-Way mirroring (searching the list for the

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Hugo Mills
On Thu, Jan 09, 2014 at 05:49:48PM +0100, George Eleftheriou wrote: Duncan, As a silent reader of this list (for almost a year)... As an anonymous supporter of the BAARF (Battle Against Any RAID Four/Five/Six/ Z etc...) initiative... I can only break my silence and applaud your frequent

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Murphy
On Jan 9, 2014, at 9:49 AM, George Eleftheriou ele...@gmail.com wrote: I'm really looking forward to the day that typing: mkfs.btrfs -d raid10 -m raid10 /dev/sd[abcd] will do exactly what is expected to do. A true RAID10 resilient in 2 disks' failure. Simple and beautiful. How is a

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Murphy
On Jan 9, 2014, at 5:52 AM, Austin S Hemmelgarn ahferro...@gmail.com wrote: Just a thought, you might consider running btrfs on top of LVM in the interim, it isn't quite as efficient as btrfs by itself, but it does allow N-way mirroring (and the efficiency is much better now that they have

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Eleftheriou
claiming that RAID-10 (with 2-way mirroring) is guaranteed to survive an arbitrary 2-device failure is incorrect. Yes, you are right. I didn't mean any 2 devices. I should have added from different mirrors :) -- To unsubscribe from this list: send the line unsubscribe linux-btrfs in the body

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Hugo Mills
On Thu, Jan 09, 2014 at 06:34:23PM +0100, George Eleftheriou wrote: claiming that RAID-10 (with 2-way mirroring) is guaranteed to survive an arbitrary 2-device failure is incorrect. Yes, you are right. I didn't mean any 2 devices. I should have added from different mirrors :) If you

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Eleftheriou
How is a resilient 2 disk failure possible with four disk raid10? __ ___ RAID0 __|__ __|__ ___ RAID1 | || | AB CD Loosing A+C / A+D / B+C / B+D is resilient. Loosing A+B or C+D is catastrophic. Sorry, it's my fault. In my urge to praise

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Murphy
On Jan 9, 2014, at 5:41 AM, Duncan 1i5t5.dun...@cox.net wrote: Having checksumming is good, and a second copy in case one fails the checksum is nice, but what if they BOTH do? I'd love to have the choice of (at least) three-way-mirroring, as for me that seems the best practical hassle/cost

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Austin S Hemmelgarn
On 2014-01-09 12:31, Chris Murphy wrote: On Jan 9, 2014, at 5:52 AM, Austin S Hemmelgarn ahferro...@gmail.com wrote: Just a thought, you might consider running btrfs on top of LVM in the interim, it isn't quite as efficient as btrfs by itself, but it does allow N-way mirroring (and the

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Austin S Hemmelgarn
On 2014-01-09 13:08, Chris Murphy wrote: On Jan 9, 2014, at 5:41 AM, Duncan 1i5t5.dun...@cox.net wrote: Having checksumming is good, and a second copy in case one fails the checksum is nice, but what if they BOTH do? I'd love to have the choice of (at least) three-way-mirroring, as for me

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Murphy
On Jan 9, 2014, at 3:42 AM, Hugo Mills h...@carfax.org.uk wrote: On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be horribly unreliable. A 32GB microSDHC

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Eleftheriou
Thanks Hugo, Since: -- i keep daily backups -- all 4 devices are of the same size I think I can test it (as soon as I have some time to spend in the transition to BTRFS) and verify your assumptions (...and get my wish) If you have an even number of devices and all the devices are the

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Murphy
On Jan 9, 2014, at 11:22 AM, Austin S Hemmelgarn ahferro...@gmail.com wrote: On 2014-01-09 13:08, Chris Murphy wrote: On Jan 9, 2014, at 5:41 AM, Duncan 1i5t5.dun...@cox.net wrote: Having checksumming is good, and a second copy in case one fails the checksum is nice, but what if they BOTH

RE: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Kyle Gates
On Thu, 9 Jan 2014 11:40:20 -0700 Chris Murphy wrote: On Jan 9, 2014, at 3:42 AM, Hugo Mills wrote: On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of, one write every 10s) workloads on cheap flash media which proved to be

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Chris Murphy
On Jan 9, 2014, at 12:13 PM, Kyle Gates kylega...@hotmail.com wrote: On Thu, 9 Jan 2014 11:40:20 -0700 Chris Murphy wrote: On Jan 9, 2014, at 3:42 AM, Hugo Mills wrote: On Thu, Jan 09, 2014 at 11:26:26AM +0100, Clemens Eisserer wrote: Hi, I am running write-intensive (well sort of,

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Mitchell
I really suspect a lot of bad block issues can be avoided by monitoring SMART data. SMART is working very well for me with btrfs formatted drives. SMART will detect when sectors silently fail and as those failures accumulate, SMART will warn in an obvious way that the drive in question is at

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread Clemens Eisserer
Hi George, I really suspect a lot of bad block issues can be avoided by monitoring SMART data. SMART is working very well for me with btrfs formatted drives. SMART will detect when sectors silently fail and as those failures accumulate, SMART will warn in an obvious way that the drive in

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Mitchell
Hello Clemens, On 01/09/2014 04:08 PM, Clemens Eisserer wrote: Hi George, I really suspect a lot of bad block issues can be avoided by monitoring SMART data. SMART is working very well for me with btrfs formatted drives. SMART will detect when sectors silently fail and as those failures

Re: How does btrfs handle bad blocks in raid1?

2014-01-09 Thread George Mitchell
On 01/09/2014 05:06 PM, Jim Salter wrote: On Jan 9, 2014 7:46 PM, George Mitchell geo...@chinilu.com wrote: I would prefer that the drive, even flash media type, would catch and resolve write failures. If it doesn't happen at the hardware layer, according to how I understand Hugo's answer,