On Mar 25, 2012, at 6:26 AM, Jeff Bacon wrote: >> In general, mixing SATA and SAS directly behind expanders (eg without >> SAS/SATA intereposers) seems to be bad juju that an OS can't fix. > > In general I'd agree. Just mixing them on the same box can be problematic, > I've noticed - though I think as much as anything that the firmware > on the 3G/s expanders just isn't as well-tuned as the firmware > on the 6G/s expanders, and for all I know there's a firmware update > that will make things better.
I haven't noticed a big difference in the expanders, does anyone else see issues with 6G expanders? > SSDs seem to be an exception, however. Several boxes have a mix of > Crucial C300, OCZ Vertex Pro, and OCZ Vertex-3 SSDs for the usual > purposes on the expander with the constellations, or in one case, > Cheetah 15ks. One box has SSDs and Cheetah 15ks/constellations on the same > expander under massive loads - the aforementioned box suffering from > 80k ZIO queues - with nary a blip. (The SSDs are swap drives, and > we were force-swapping processes out to disk as part of task management. > Meanwhile, the Java processes are doing batch import processing using > the Cheetahs as staging area, so those two expanders are under constant > heavy load. Yes that is as ugly as it sounds, don't ask, and don't do > this yourself. This is what happens when you develop a database without > clear specs and have to just throw hardware underneath it guessing > all the way. But to give you an idea of the load they were/are under.) Sometime over beers we can trade war stories... many beers... :-) > > The SSDs were chosen with an eye towards expander-friendliness, and tested > relatively extensively before use. YMMV of course and this is nowhere > to skimp on a-data or Kingston; buy what Anand says to buy and you > seem to do very well. Yes. Be aware that companies like Kingston rebadge drives from other, reputable suppliers. And some reputable suppliers have less-than-perfect models. > > I would say, never do it on LSI 3G/s expanders. Be careful with using > SATA spindles. Test the hell out of any SSD you use first. But you seem > to be able to get away with the better consumer-class SATA SSDs. > > (I realize that many here would say that if you are going to use > SSD in an enterprise config, you shouldn't be messing with anything > short of Deneva or the SAS-based SSDs. I'd say there are simply > a bunch of caveats with the consumer MLC SSDs in such situations > to consider and if you are very clear about them up front, then > they can be just fine. > > I suspect the real difficulty in these situations is in having > a management chain that is capable of both grokking the caveats up > front and remembering that they agreed to them when something > does go wrong. :) As in this case I am the management chain, > it's not an issue. This is of course not the usual case.) We'd like to think that given the correct information, reasonable people will make the best choice. And then there are PHBs. -- richard -- DTrace Conference, April 3, 2012, http://wiki.smartos.org/display/DOC/dtrace.conf ZFS Performance and Training richard.ell...@richardelling.com +1-760-896-4422
_______________________________________________ zfs-discuss mailing list zfs-discuss@opensolaris.org http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
