Thanks Tim I've checked https://github.com/pgriess/node-msgpack, but I'm afraid that's not exact the thing I'm looking for.
The problem I'm facing is: In our node app, we use socket to deal with large number of real-time binary data sent from multiple clients. For performance and memory consideration, we left those message in buffer and continuously read data segments from the buffer to detect whether the app should process further segments in the buff or stop and drop the buffer if condition not met. Since the Buffer object is Nodejs is not resizable. So I just wonder why not provide a postion property for the Buffer object for streaming reading. Our current solution is a bit painful. To streamy read the buff, we inject an position property into each Buff instance and use a set of static helper method to process the buffer in order to push the position going forward. For example, to continuously read a 32bit uint from the buf, we can not do buf.readUInt32BE, we have to do BufHelper.readUInt32(buf) Regards, ty 2012/7/27 Tim Caswell <[email protected]> > If you're talking about a protocol parser, I find state machines work > best. Do you need a streaming parser or one that first deframes > messages and then parses pre-buffered messages? The techniques are > similar but not quite the same. > > An example of a non-streaming binary parser would be my msgpack > decoder or Felix's mysql protocol parser. > > Most streams have arbitrary chunk lengths that don't match up with > message lengths. If you're using a pre-existing protocol (like mysql) > then the framing is probably included in the wire protocol. > > On Thu, Jul 26, 2012 at 11:04 PM, Yi Tan <[email protected]> wrote: > > Hi all, > > > > I'm working on a node project to process binary stream send/received via > > socket connection. > > > > I'm new to Node, I found in the Buffer class, there is no property to > > indicate current stream read position. And this cause a lot of difficult > > when parsing complex binary stream. > > > > Could you give me some hint on how to implement a position property for > > Buffer as well as to keep the best performance. > > > > Many thanks, > > > > ty > > > > -- > > Job Board: http://jobs.nodejs.org/ > > Posting guidelines: > > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > > You received this message because you are subscribed to the Google > > Groups "nodejs" group. > > To post to this group, send email to [email protected] > > To unsubscribe from this group, send email to > > [email protected] > > For more options, visit this group at > > http://groups.google.com/group/nodejs?hl=en?hl=en > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > To unsubscribe from this group, send email to > [email protected] > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en > -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
