On Thu, Feb 16, 2012 at 1:48 PM, HG <[email protected]> wrote: > Hi! > > On Thu, Feb 16, 2012 at 3:38 PM, Dean Landolt <[email protected]> > wrote: > > > > > > On Thu, Feb 16, 2012 at 7:09 AM, HG <[email protected]> wrote: > >> > >> Hi! > >> > >> I created a small counting algorithm first in JavaScript. Seems to > >> work ok. But this is of course single threaded and I could only store > >> snapshots of it to DB. So I started thinking that what if I use Redis > >> directly as the memory. Then I could have multiple threads or even > >> servers running against it. > >> > >> However, the algorithm limits the stuff that is kept in memory. So, > >> basically I need to do something like this: > >> > >> redis.zincrby(key, 1, str, function (err, reply) { > >> redis.zcard(key, function (err, N) { > >> If (N > maxSize) { > >> // do stuff with the keys, including delete some (this is a > >> few callbacks here...) > >> } > >> } > >> I.e. I add counters (zincrby ands new one if it doesn't exists), but > >> if there are too many as a result of that, I need to do some cleaning. > >> > >> But obviously, the whole thing works in threaded situation only if > >> that code is blocking. So, I can't run multiple servers against redis > >> like this or this whole code should block redis (I guess I could store > >> this kind of procedure to MongoDB...). But is there even a way to > >> block single node.js event loop for this? I know that would not be > >> wise (for keeping up the speed), but I'm just trying to learn how to > >> do this kind of stuff? Should all cases where I need blocking with the > >> DB done with stored procedures in the DB? > > > > > > > > Technically you can do this with a blocking child process call, but I'm > > pretty sure you don't want to do that. Can you be a little more specific > > with what you're trying to do? Are you looking for a priority queue? > > I'm keeping count of the most frequent strings that I get as input. > Basically, this is the main part of the JS implementation that I > thought I could do with redis: > > var top = function (size) { > var items = []; > var counts = []; > > return { > newItem: function (str) { > n++; > var ind = items.indexOf(str); > if (ind >= 0){ > // boost the one > counts[ind] += 1; > } else { > // not there, so add it > if (items.length < size){ > items.push(str); > counts.push(1); > } else { > // too many, lower counts and remove if zero > for (var i = 0; i < items.length; i++) { > counts[i] -= 1; > if (counts[i] === 0){ > counts.splice(i,1); > items.splice(i,1); > i -= 1; > } > } > } > } > }, > ... > > And in the input loop, I call this with top.newItem(input_str). But > with this implementation, I can only support one thread. >
You want a lowest-first priority queue. Check github, I'm sure there's at least a few implementations available. But what specifically do you mean by "thread" here? Do you mean "process", or are you doing something fancy w/in c/c++land? Regardless, you'll have to serialize access somehow. -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to the Google Groups "nodejs" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/nodejs?hl=en?hl=en
