Jon,
Original Message
----------------
>> I am seeking a few points of clarification:
>>
>> 1. Fibre Channel folks have attempted to explain to me why TCP/IP could =
>> NEVER be a viable interconnect for block level storage operations. They =
>> claim:
>> a. TCP is too CPU intensive and creates too much latency for storage =
>> I/O operations.
>> b. The IP stack is too top heavy and processing packet headers is too =
>> slow to support storage I/O operations.
There is a lot of work to show that this is not true. Check out Van
Meter's 1998 ASPLOS paper "VISA - Netstations virtual internet SCSI
adaptor."
Perhaps more importantly, there are many companies that are building
TCP in silicon ASICs. This should make TCP's performance comparable
to Fibre Channel. Both TCP/IP and FC provide about the same
functionality ... reliable, in-order transmission.
The bottom line is that FC is done in hardware while TCP has
traditionally been done in software. Therefore, previous performance
numbers are not going to be fair. Once TCP is in silicon, its
performance should be roughly equal to FC.
>> c. The maximum throughput of a GE TCP/IP connection is 768 Mps, which =
>> is too slow to support storage I/O operations.
I believe there are higher numbers (especially with Jumbo
Frames). Alteon's web site show's 920 Mbps. Microsoft and Duke
University have both shown TCP performance o 1Gb+/s performance over
other networks.
BTW, why is 768 Mbps too slow for storage. Many apps (e.g.,
transaction workloads) are I/O's per second bound, not bandwidth
bound. Also, even if storage over IP/ether is a bit slower than FC,
the benefits of leveraging IP's infrastructure (i.e., routers,
switches, NICs, network management, networking people) is a huge
advantage.
There is also the issue of SCSI over TCP/IP in the SAN vs. the
LAN/WAN. Some companies, focusing on the SAN, are building
SCSI/lightweight transport/IP while others, focusing on the WAN,
propose SCSI/TCP/IP. It may be the case that SAN and WAN traffic use
different transport protocols to gain a bit of extra performance in
the SAN.
>> Is any of this true?
>>
>> 2. Adaptec has posited a replacement for TCP called STP for use as a =
>> transport for storage. Does anyone know anything about this?
From Paul von Stamwitz's posting to the ips mailing list ...
The link to the SEP draft is
http://www.ietf.org/internet-drafts/draft-wilson-sep-00.txt
The press release is at:
http://www.adaptec.com/adaptec/press/release000504.html
The demo shows a Gb ethernet controller transporting SCSI traffic to several
targets through an off-the-shelf 100TX switch with a Gb uplink. The targets
are ethernet to U160 SCSI bridges with one or more SCSI drives attached. The
host controller runs under NT4.0 at appears to the OS as a SCSI host bus
adapter.
The architecture is based on Adaptec's SCSI Encapsulation Protocol
(SEP). SEP is mapped on top of TCP/IP or a light-weight transport
protocol specifically designed for SANs.
An SEP overview was presented at the IPS BOF in Adelaide last month and an
internet draft on SEP was submitted to IETF this week. I will forward the
link as soon as it becomes available. This draft is informational
only and intended to aid in this group's work toward an industry
standard SCSI transport protocol over IP networks.
>> 3. Current discussions of the SCSI over IP protocol seem to ignore the =
>> issue of TCP or any other transport protocol. Does anyone know =
>> definitively what transport is being suggested by the IBM/Cisco crowd?
Current SCSI over IP discussions are not ignoring TCP ... they are
definitely considering TCP as the primary transport. See the ips
web site at:
http://www.ece.cmu.edu/~ips
>>
>> 4. Another storage company is looking at Reliable UDP as a substitute =
>> for TCP in storage data transfers. Where can I learn more about this =
>> protocol, which I am told was introduced many years ago by Cisco?
Companies to look at include:
nishansystems.com
interprophet.com
san.com
arkresearch.com
Also, I believe that the IETF IP over FC working group is now
looking at FC over IP.
dave...........
David Nagle
Director, Parallel Data Lab
Senior Reseach Computer Scientist
School of Computer Science
Carnegie Mellon University
Pittsburgh, PA 15213
412-268-3898 (office)
412-268-3890 (fax)
http://www.ece.cmu.edu/~bassoon
