I had to solve this problem recently as well, and I built myself a simple TCP-backed JSON framing protocol (null-delimited) — perhaps you could use something like this? I could give you my (unfinished, untested, unstable) library as an example, if you'd like.
-- Peter Lejeck Sent with Sparrow (http://www.sparrowmailapp.com/?sig) On Friday, July 27, 2012 at 1:50 PM, Baz wrote: > Hello, > > I have a process where a 3rd party service sends a message to a pre-defined > endpoint in my system. This triggers a series of actions on my side: > Invoke a webservice to pull a large batch of items that need to be processed > Iterate through the large batch and queue up a number of smaller batches to > process > Process each smaller batch in semi-parallel: > Run calculations > Save result to db > Dynamically update any web clients "watching" the affected data when complete > > > It seems there are two discrete parts here: that which listens for, and > processes messages - and a second part that dynamically renders changes to > the client based on changes in the backend. > > > I would love some advice, thoughts or suggestions on how to architect any of > this, especially the first part regarding listening for and processing > messages. Some basic thoughts: > > I need an endpoint, that's no problem, but then it seems I need some sort of > queuing system. Should I roll my own? What about Amazon SQS? Is it easy to > get node to respond to messages and spawn more instances if need be? What are > some important considerations? > > For the front-end, should I be basically looking at Socket.io > (http://Socket.io)? > > Thanks for any help, > Baz > > -- > Job Board: http://jobs.nodejs.org/ > Posting guidelines: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > You received this message because you are subscribed to the Google > Groups "nodejs" group. > To post to this group, send email to [email protected] > (mailto:[email protected]) > To unsubscribe from this group, send email to > [email protected] > (mailto:[email protected]) > For more options, visit this group at > http://groups.google.com/group/nodejs?hl=en?hl=en -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
