> The actual reason I beefed up Buffer a bit lately is *because* I need
> to do some protocol decoding of a byte stream.
>
> Now I see IOBuffer arrive. In order to avoid code bloat, wouldn't it
> be a better idea to integrate the functionality of IOBuffer into Buffer
> and just keep one?
Sorry about the timing, I have had IOBuffer on the way for some time
(I am still wondering where to put it, however, that has, believe it
or not, been a blocker for me. Perhaps Stdio.Buffer? I will create a
buffered stream that reads to and writes from said object, without
creating pike strings)
Unfortunately it is not possible to make String.Buffer even close to
as efficient as long as it uses a stringbuilder. And not using a
stringbuilder slows some things down (sprintf comes to mind) and makes
others more or less impossible (wide strings) without excessive code
duplication.
The whole reason for IOBuffer is that it uses pointer arithmetics on
guaranteed 8bit strings to be fast at both reading from the beginning
and writing to the end at the same time (I am, by the way, considering
converting it to be a circular buffer to avoid the one memmove it does
at times), and it is also efficient at creating sub-buffers.
The fact that it is guaranteed to only contain 8bit characters helps a
lot too.
> Or is the performance difference so strikingly great that this sounds
> like a bad idea?
As things stands now, yes.
Things like subbuffers is unfortunately actually impossible when using
a stringbuilder.
I guess I might outline the plan for IOBuffer some more (I actually
did this during the last pike conference, but it has been a while. :))
o Add support for reading to and writing from file objects to it.
Either add support in Stdio.File (also add
System.Memory + String.Buffer?) or do it the other way
around (that way lies madness, however, see also: Shuffler)
The main goal here is to do one copy less and avoid the pike string
creation
o Add support for System.Memory & String.Buffer to add()
o Add support for reading other things than "binary holerith" and integers
+ line
+ word
+ json object
+ encode_value?
o Add support for "throw mode".
It is rather useful to be able to change what happens when you try
to read data that is not in the buffer.
| void read_callback()
| {
| inbuffer->add( new_data );
| // Read next packet
| while( Buffer packet = inbuffer->read_hbuffer(2) )
| {
| packet->set_error_mode(Buffer.THROW_ERROR);
| if( mixed e = catch(handle_packet( packet ) ) )
| if( e->buffer_error )
| protocol_error(); // illegal data in packet
| else
| throw(e); // the other code did something bad
| }
| }
The handle_packet function then just assumes the packet is well
formed, and reads without checking if the read succeed (since it
will, or an error will be thrown).
This code snipplet also demonstrates why the subbuffers are handy,
read_hbuffer does not actually copy any data, the returned buffer
just keeps a reference to the data in the original buffer.
--
Per Hedbor
