On 05/30/2012 10:53 PM, Richard Elling wrote: > On May 30, 2012, at 1:07 PM, Sašo Kiselkov wrote: > >> On 05/25/2012 08:40 PM, Richard Elling wrote: >>> See the soluion at https://www.illumos.org/issues/644 >>> -- richard >> >> And predictably, I'm back with another n00b question regarding this >> array. I've put a pair of LSI-9200-8e controllers in the server and >> attached the cables to the enclosure to each of the HBAs. As a result >> (why?) I'm getting some really strange behavior: >> >> * piss poor performance (around 5MB/s per disk tops) >> * fmd(1M) running one core at near 100% saturation each time something >> writes or reads from the pool >> * using fmstat I noticed that its the eft module receiving hundreds of >> fault reports every second >> * fmd is flooded by multipath failover ereports like: >> >> ... >> May 29 21:11:44.9408 ereport.io.scsi.cmd.disk.tran >> May 29 21:11:44.9423 ereport.io.scsi.cmd.disk.tran >> May 29 21:11:44.8474 ereport.io.scsi.cmd.disk.recovered >> May 29 21:11:44.9455 ereport.io.scsi.cmd.disk.tran >> May 29 21:11:44.9457 ereport.io.scsi.cmd.disk.dev.rqs.derr >> May 29 21:11:44.9462 ereport.io.scsi.cmd.disk.tran >> May 29 21:11:44.9527 ereport.io.scsi.cmd.disk.tran >> May 29 21:11:44.9535 ereport.io.scsi.cmd.disk.dev.rqs.derr >> May 29 21:11:44.6362 ereport.io.scsi.cmd.disk.recovered >> ... >> >> >> >> I suspect that multipath is something not exactly very happy with my >> Toshiba disks, but I have no idea what to do to make it work at least >> somehow acceptably. I tried messing with scsi_vhci.conf to try and set >> load-balance="none", change the scsi-vhci-failover-override for the >> Toshiba disks to f_asym_lsi, flashing the latest as well as old firmware >> in the cards, reseating them to other PCI-e slots, removing one cable >> and even removing one whole HBA, unloading the eft fmd module etc, but >> nothing helped so far and I'm sort of out of ideas. Anybody else got an >> idea on what I might try? > > Those ereports are consistent with faulty cabling. You can trace all of the > cables and errors using tools like lsiutil, sg_logs, kstats, etc. > Unfortunately, > it is not really possible to get into this level of detail over email, and it > can > consume many hours. > -- richard
That's actually a pretty good piece of information for me! I will try changing my cabling to see if I can get the errors to go away. Thanks again for the suggestions! Cheers -- Saso _______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
