Hi!
On Thu, Feb 16, 2012 at 3:38 PM, Dean Landolt <[email protected]> wrote:
>
>
> On Thu, Feb 16, 2012 at 7:09 AM, HG <[email protected]> wrote:
>>
>> Hi!
>>
>> I created a small counting algorithm first in JavaScript. Seems to
>> work ok. But this is of course single threaded and I could only store
>> snapshots of it to DB. So I started thinking that what if I use Redis
>> directly as the memory. Then I could have multiple threads or even
>> servers running against it.
>>
>> However, the algorithm limits the stuff that is kept in memory. So,
>> basically I need to do something like this:
>>
>> redis.zincrby(key, 1, str, function (err, reply) {
>> redis.zcard(key, function (err, N) {
>> If (N > maxSize) {
>> // do stuff with the keys, including delete some (this is a
>> few callbacks here...)
>> }
>> }
>> I.e. I add counters (zincrby ands new one if it doesn't exists), but
>> if there are too many as a result of that, I need to do some cleaning.
>>
>> But obviously, the whole thing works in threaded situation only if
>> that code is blocking. So, I can't run multiple servers against redis
>> like this or this whole code should block redis (I guess I could store
>> this kind of procedure to MongoDB...). But is there even a way to
>> block single node.js event loop for this? I know that would not be
>> wise (for keeping up the speed), but I'm just trying to learn how to
>> do this kind of stuff? Should all cases where I need blocking with the
>> DB done with stored procedures in the DB?
>
>
>
> Technically you can do this with a blocking child process call, but I'm
> pretty sure you don't want to do that. Can you be a little more specific
> with what you're trying to do? Are you looking for a priority queue?
I'm keeping count of the most frequent strings that I get as input.
Basically, this is the main part of the JS implementation that I
thought I could do with redis:
var top = function (size) {
var items = [];
var counts = [];
return {
newItem: function (str) {
n++;
var ind = items.indexOf(str);
if (ind >= 0){
// boost the one
counts[ind] += 1;
} else {
// not there, so add it
if (items.length < size){
items.push(str);
counts.push(1);
} else {
// too many, lower counts and remove if zero
for (var i = 0; i < items.length; i++) {
counts[i] -= 1;
if (counts[i] === 0){
counts.splice(i,1);
items.splice(i,1);
i -= 1;
}
}
}
}
},
...
And in the input loop, I call this with top.newItem(input_str). But
with this implementation, I can only support one thread.
--
HG.
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to [email protected]
To unsubscribe from this group, send email to
[email protected]
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
