Re: How to stress test raid6 on 122 disk array

On 2016-08-15 09:39, Martin wrote: That really is the case, there's currently no way to do this with BTRFS. You have to keep in mind that the raid5/6 code only went into the mainline kernel a few versions ago, and it's still pretty immature as far as kernel code goes. I don't know when (if

Re: How to stress test raid6 on 122 disk array

On Mon, Aug 15, 2016 at 7:38 AM, Martin wrote: >> Looking at the kernel log itself, you've got a ton of write errors on >> /dev/sdap. I would suggest checking that particular disk with smartctl, and >> possibly checking the other hardware involved (the storage controller

Re: How to stress test raid6 on 122 disk array

On 2016-08-15 09:38, Martin wrote: Looking at the kernel log itself, you've got a ton of write errors on /dev/sdap. I would suggest checking that particular disk with smartctl, and possibly checking the other hardware involved (the storage controller and cabling). I would kind of expect BTRFS

Re: How to stress test raid6 on 122 disk array

> That really is the case, there's currently no way to do this with BTRFS. > You have to keep in mind that the raid5/6 code only went into the mainline > kernel a few versions ago, and it's still pretty immature as far as kernel > code goes. I don't know when (if ever) such a feature might get

Re: How to stress test raid6 on 122 disk array

> Looking at the kernel log itself, you've got a ton of write errors on > /dev/sdap. I would suggest checking that particular disk with smartctl, and > possibly checking the other hardware involved (the storage controller and > cabling). > > I would kind of expect BTRFS to crash with that many

Re: How to stress test raid6 on 122 disk array

On Mon, Aug 15, 2016 at 6:19 AM, Martin wrote: > > I have now had the first crash, can you take a look if I have provided > the needed info? > > https://bugzilla.kernel.org/show_bug.cgi?id=153141 [337406.626175] BTRFS warning (device sdq): lost page write due to IO error

Re: How to stress test raid6 on 122 disk array

On 2016-08-15 08:19, Martin wrote: I'm not sure what Arch does any differently to their kernels from kernel.org kernels. But bugzilla.kernel.org offers a Mainline and Fedora drop down for identifying the kernel source tree. IIRC, they're pretty close to mainline kernels. I don't think they

Re: How to stress test raid6 on 122 disk array

On 2016-08-15 08:19, Martin wrote: The smallest disk of the 122 is 500GB. Is it possible to have btrfs see each disk as only e.g. 10GB? That way I can corrupt and resilver more disks over a month. Well, at least you can easily partition the devices for that to happen. Can it be done with

Re: How to stress test raid6 on 122 disk array

>> The smallest disk of the 122 is 500GB. Is it possible to have btrfs >> see each disk as only e.g. 10GB? That way I can corrupt and resilver >> more disks over a month. > > Well, at least you can easily partition the devices for that to happen. Can it be done with btrfs or should I do it with

Re: How to stress test raid6 on 122 disk array

>> I'm not sure what Arch does any differently to their kernels from >> kernel.org kernels. But bugzilla.kernel.org offers a Mainline and >> Fedora drop down for identifying the kernel source tree. > > IIRC, they're pretty close to mainline kernels. I don't think they have any > patches in the

Re: How to stress test raid6 on 122 disk array

On 2016-08-04 17:12, Chris Murphy wrote: On Thu, Aug 4, 2016 at 2:51 PM, Martin wrote: Thanks for the benchmark tools and tips on where the issues might be. Is Fedora 24 rawhide preferred over ArchLinux? I'm not sure what Arch does any differently to their kernels

Re: How to stress test raid6 on 122 disk array

Martin writes: > The smallest disk of the 122 is 500GB. Is it possible to have btrfs > see each disk as only e.g. 10GB? That way I can corrupt and resilver > more disks over a month. Well, at least you can easily partition the devices for that to happen. However, I

Re: How to stress test raid6 on 122 disk array

Excellent. Thanks. In order to automate it, would it be ok if I dd some zeroes directly to the devices to corrupt them, or do need to physically take the disks out while running? The smallest disk of the 122 is 500GB. Is it possible to have btrfs see each disk as only e.g. 10GB? That way I can

Re: How to stress test raid6 on 122 disk array

On Thu, Aug 4, 2016 at 2:51 PM, Martin wrote: > Thanks for the benchmark tools and tips on where the issues might be. > > Is Fedora 24 rawhide preferred over ArchLinux? I'm not sure what Arch does any differently to their kernels from kernel.org kernels. But

Re: How to stress test raid6 on 122 disk array

Thanks for the benchmark tools and tips on where the issues might be. Is Fedora 24 rawhide preferred over ArchLinux? If I want to compile a mainline kernel. Are there anything I need to tune? When I do the tests, how do I log the info you would like to see, if I find a bug? On 4 August 2016

Re: How to stress test raid6 on 122 disk array

On Thu, Aug 4, 2016 at 1:05 PM, Austin S. Hemmelgarn wrote: >Fedora should be fine (they're good about staying up to > date), but if possible you should probably use Rawhide instead of a regular > release, as that will give you quite possibly one of the closest >

Re: How to stress test raid6 on 122 disk array

On 2016-08-04 13:43, Martin wrote: Hi, I would like to find rare raid6 bugs in btrfs, where I have the following hw: * 2x 8 core CPU * 128GB ram * 70 FC disk array (56x 500GB + 14x 1TB SATA disks) * 24 FC or 2x SAS disk array (1TB SAS disks) * 16 FC disk array (1TB SATA disks) * 12 SAS disk