[This message was posted by Dale Wilson of Object Computing, Inc
<[email protected]> to the "FAST Protocol" discussion forum at
http://fixprotocol.org/discuss/46. You can reply to it on-line at
http://fixprotocol.org/discuss/read/95b5d551 - PLEASE DO NOT REPLY BY MAIL.]
In a recent post on another thread, Hanno said:
> I'm working together with the core mdowg group to improve documentation,
> presentation and training materials to improve the learning curve for
> new-comers, as well as encouraging existing implementers to be able to
> reap the full benefit of FAST.
>
> We welcome any comments and suggestions on what to include in the work
> that we currently do
As I developed QuickFAST [see my sig], I spent a lot of time with the FAST
documentation. In general, I have found it clear and unambiguous as a
specification, but not as a tutorial. This is a good thing. Reference manuals
and tutorials should be separate.
The FAST protocol is moderately complex, but the complexity is obviously
targeted at the goal achieving a high degree of practical (as opposed to
theoretical) lossless data compression. The learning curve is steep but it
makes sense once you get it.
The proposed extensions to encoding in FAST 1.2 continue this clarity. FAST
1.2 adds some useful new data types while supporting backward compatibility
such that a properly written FAST 1.2 decoder should be able to handle a FAST
1.1 data stream without problems.
I had a minor concern about the FAST 1.2 bit map field. As originally proposed
it did not allow representation of an empty set of bits. However when I
expressed this concern and the response on this list was positive.
I also have some concerns about the proposed modifications to the XML template
definition language, but they are cosmetic rather than functional. For
example, rather than:
[field] [type name="abc"/][/field]
I would much prefer
[field type="abc"/]
However, that's certainly not a show-stopper. (substitute angle brackets for []
where appropriate)
One thing I appreciate about the FAST specification is the separation of the
payload-data-encoding-decoding issues from the packet framing and session
management issues. This was a severe drawback to the FIX standard in which
message framing, session management and application data issues were muddled
together. FAST has avoided this by focusing completely on the
transcoding/compression issues and trusting that other layers of the software
will address other issues.
Unfortunately at the moment these "other layers" are left mostly unspecified.
I say "mostly" because Session Control Protocol (SCP) attempts to address some
of the session layer issues , and the FAST specification does mention the
possibility of "blocking" the FAST encoded messages.
Blocking is optional and does not appear to have been accepted by the industry
which seems to be developing a number of ad hoc and incompatible alternatives
to handle transmission of FAST on a (logically) streaming medium (TCP) as
opposed to a logically packetized medium(UDP and Multicast) . This to me is
the most important issue that needs to be addressed -- preferably by a solution
that is clearly distinct from the encoding/decoding concerns of the FAST
protocol.
Some concerns that could be addressed at the message framing level include:
-- Resynchronization after "something goes wrong."
Suppose for example, a fast decoder is delivering data to the application code
on-the-fly as opposed to assembling a complete message before delivering it to
the application (there are some obvious latency advantages in doing this.) If
the application were to throw an exception, it would be very useful to have a
way to find the next "clean point" from which decoding could continue.
There is more involved in this than just identifying the next message because
the issue of resetting the data dictionaries needs to be addressed as well. On
a multicast data feed this is simply a matter of restarting the decoder at the
next packet. However this is an artifact of the lossy and packetized nature of
multicast. On a reliable transport or a streaming transport the problem is
unsolved.
-- Providing "filtering" information at the framing level to allow an
application to determine that this message is not of interest and skip forward
to the next message in the stream of data with minimal decoding overhead.
Again the issue of resetting the dictionaries would have to be considered.
(CME's FIX/FAST 2.0 attempts to address this.)
It might be argued that having a framing protocol that addresses these needs
defeats some of the advantages of FAST encoding, however one thing to consider
is there is already "framing" information inherent in the IP protocol. For
every packet on the wire there are somewhere between 20 and 40 bytes of
"overhead" information on the wire (IP and TCP and/or UDP headers, etc.). A
well designed framing protocol that addresses the above concerns with, say, 8
to 12 bytes of overhead could easily be an overall win in effective, usable,
throughput and reduced latency.
Of course this is speculation on my part, but I think it's an issue worth
looking into.
And finally there is the session management layer. I won't even try to get
into that in this other than to say that in my opinion SCP needs a lot of work
before it addresses the practical session management issues that are likely to
be encountered as FAST is used in a wide variety of circumstances.
Dale Wilson
Principal Software Engineer
Object Computing, Inc.
http://www.ociweb.com/
Lead developer of the QuickFAST open source implementation of FAST.
http://www.quickfast.org/
[You can unsubscribe from this discussion group by sending a message to
mailto:[email protected]]
--
You received this message because you are subscribed to the Google Groups
"Financial Information eXchange" group.
To post to this group, send email to [email protected].
To unsubscribe from this group, send email to
[email protected].
For more options, visit this group at
http://groups.google.com/group/fix-protocol?hl=en.
