Hey Sylvain, Thanks for explaining the rationale. When you look at from the perspective of the use cases you mention, it makes sense to be able to supply the reader with the frame size up front.
I've opted to go for serializing the frame into a buffer. Although this could materialize an arbitrarily large amount of memory, ultimately the driving application has control of the degree to which this can occur, so in the grander scheme of things, you can still maintain streaming semantics. Thanks for the heads up. Cheers, Ben On Tue, Jan 8, 2013 at 4:08 PM, Sylvain Lebresne <sylv...@datastax.com>wrote: > Mostly this is because having the frame length is convenient to have in > practice. > > Without pretending that there is only one way to write a server, it is > common > to separate the phase "read a frame from the network" from the phase > "decode > the frame" which is often simpler if you can read the frame upfront. Also, > if > you don't have the frame size, it means you need to decode the whole frame > before being able to decode the next one, and so you can't parallelize the > decoding. > > It is true however that it means for the write side that you need to > either be > able to either pre-compute the frame body size or to serialize it in memory > first. That's a trade of for making it easier on the read side. But if you > want > my opinion, on the write side too it's probably worth parallelizing the > message > encoding (which require you encode it in memory first) since it's an > asynchronous protocol and so there will likely be multiple writer > simultaneously. > > -- > Sylvain > > > > On Tue, Jan 8, 2013 at 12:48 PM, Ben Hood <0x6e6...@gmail.com> wrote: > >> Hi, >> >> I've read the CQL wire specification and naively, I can't see how the >> frame length length header is used. >> >> To me, it looks like on the read side, you know which type of structures >> to expect based on the opcode and each structure is TLV encoded. >> >> On the write side, you need to encode TLV structures as well, but you >> don't know the overall frame length until you've encoded it. So it would >> seem that you either need to pre-calculate the cumulative TLV size before >> you serialize the frame body, or you serialize the frame body to a buffer >> which you can then get the size of and then write to the socket, after >> having first written the count out. >> >> Is there potentially an implicit assumption that the reader will want to >> pre-buffer the entire frame before decoding it? >> >> Cheers, >> >> Ben >> > >
