On Tue, 2 Feb 2010, Miles Nordin wrote:
"fc" == Frank Cusack <frank+lists/z...@linetwo.net> writes:
fc> by FCoE are you talking about iSCSI?
FCoE is an L2 design where ethernet ``pause'' frames can be sent
specific to one of the seven CoS levels instead of applying to the
entire port, which makes PAUSE abuseable for other purposes than their
Please redirect this encyclopedia contribution to WikiPedia, where it
belongs.
Thanks,
Bob
former one. CoS is an L2 priority/QoS tag inside the VLAN header.
Before this hack, pause frames are not useful for congestion
management because they cause head-of-line blocking, so serious
switches only send them in response to backplane congestion, and for
example serious hosts might send them for PCI contention, if clever
enough. With the hack, the HOL-blocking effect of a PAUSE still
spreads further than you might ideally like but can be constrained to
one of the seven CoS planes in your fabric (probably, the Storage
plane). This lets you have an HOL-blocking, lossless storage fabric
in parallel with a buffered TCP fabric that is not lossless (uses
packet drops for congestion control like normal Ethernet). You will
find some squirrely language from FCoE proponents around these issues
because they are trying to convince you that you have every desireable
buzzword in every part of your network, while in fact what you're
doing is making the same wise trade-off that every other non-Ethernet
LAN fabric has always made. My parallel point is that the
HOL-blocking lossless fabric is *CHEAPER* to create, not nmore
expensive. It is less capable. It has no buffers and therefore no
QoS. It just happens to be what's best for storage. so, they want
you to pay the prices of a multi-queued QoSed WRED big-buffered
non-blocking fabric suitable for transit traffic even though you
mostly just need to push storage bits: classic upsell, just like all
those ``XL'' PFC's they try to push off to customers who are not even
in the DFZ.
FCoE also includes a bunch of expensive hocus-pokus to bridge these
frames onto a traditional FC-switched network and do a bunch of other
things I don't understand like FC zoning and F-SPF. Most of the pitch
dwells on this, trying to convince you they've made things ``simpler''
for you because it's once piece of wire. This seems like an
anti-feature to me: wire's cheap while understanding things is hard,
and now everyone's forced to catch up and learn Fibre Channel before
it's safe to touch anything. Good in the long run, absolutely.
Cheaper, fuck no.
but the legitimate pitch for FCoE over iSCSI, to my view right now,
comes from not from this management baloney but from the seven CoS
levels, and the possibility some can be blocking and others buffered.
Internet *transit* traffic (as opposed to end systems), and anything
high-rtt, *must* be buffered, while within the LAN my current thinking
is that you're better off with a 10Gbit/s HOL-blocking bufferless link
than a 1Gbit/s non-blocking buffered link. The latter applies double
for storage traffic which, made up of UDP-like reads and writes where
you are stuck trying to perfect TCP to avoid blowing the buffers of
normal switches while still getting yourself out of slow-start before
the transaction's over andn doing all this in an environment where you
cannot even convince thick-skulled netadmins they NEED to provide RED,
not this bullshit ``weighted tail drop'' of 3560 u.s.w., and which
besides really need backpressure from the fabric so they can be QoS'ed
in the initiator's stack ahead of the network so that for example
scrubs don't slow down pools (don't you find this happens more over
iSCSI than over SAS?). I'm saaying, um...shit...saying, ``You need to
think, about what you are trying to accomplish,'' and that Sun might
have a suite of protocols based on ancient IB stuff that accomplishes
more than FCoE, and does it cheaper (to them) and more simply, so,
following their usual annoying plan, step 2 charge FCoE prices minus
<smallnumber>, step 3 profit.
meanwhile mellanox, having forseen all this and built open standards
to solve it, is out there desperately trying to push some baloney
called Etherband or something because all you bank admins are too daft
to buy anything that does not have Ether in the name. :(
--
Bob Friesenhahn
bfrie...@simple.dallas.tx.us, http://www.simplesystems.org/users/bfriesen/
GraphicsMagick Maintainer, http://www.GraphicsMagick.org/
_______________________________________________
zfs-discuss mailing list
zfs-discuss@opensolaris.org
http://mail.opensolaris.org/mailman/listinfo/zfs-discuss
