Re: [nodejs] How do you execute a function asynchronously?

[Harsh Dev]

Hi Jeremy,
I've been looking into TAGG as well so if possible, could you give us an 
overview of what your results are/were?

Thanks,
Harsh

On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:
>
> Thanks Jorge!
>
> Fantastic information, it sounds like threads-a-gogo is the way to go. 
> I'll use it and let you all know of my results.
>
> Thanks again!
> Jeremy
>
> On 4/11/2012 3:05 PM, Jorge wrote:
> > On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:
> >> On 4/10/2012 6:09 PM, Jorge wrote:
> >>> That's exactly what threads_a_gogo is for:
> >>>
> >>> 
> >>>
> >>> It lets you run these calculations in parallel, in javascript, in 
> threads, and using all the available CPU cores. Then the main thread will 
> receive the results in a callback or via an event.
> >> Hi all,
> > Hi Jeremy,
> >
> >> threads_a_gogo sounds cool, but what are the technical details of using 
> it?
> >>
> >> 1. What does it use in the back-end to schedule / manage tasks? 
> child_process?
> > It creates p-threads and runs v8 isolates in them.
> >
> >> 2. child_process takes some 30 ms to spawn a child thread. How much 
> time does GoGo take?
> > When I run this:
> >
> > $ time node -e "require('threads_a_gogo').createPool(100).destroy()"
> >
> > I get:
> >
> > real0m0.387s
> > user0m0.390s
> > sys0m0.286s
> >
> > That's 3.87 ms per thread
> >
> > But note that these 0.387s include as well the time to start node, the 
> time to require('threads_a_gogo'), and the time to destroy the 100 threads, 
> so it must be a bit less than 3.87 ms per thread.
> >
> >> 3. How does it internally work? Are there constantly running threads + 
> scheduler? Or does it spawn a new thread per task?
> > You could create/destroy the threads on demand, but I would rather 
> create a thread pool of twice or 3x the number of cores and install in them 
> only once the functions that do the calculations, then reuse them again and 
> again via a pool.any.eval(program, cb). Each thread takes only ~2MB, and 
> when idle it uses exactly 0% cpu.
> >
> >> 4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / 
> stats to show performance with / without GoGo?
> > Threads_a_gogo lets you
> >
> > -run these blocking calls in parallel so
> > -they won't block node's main thread
> > -you'll be exploiting all the cpu cores in the machine
> >
> > With an n cores machine, you'll be serving results n times faster, but 
> even in a single core machine, you should use threads_a_gogo simply to 
> avoid blocking node.
> >
> > WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice 
> as fast as in node's main thread, see: 
>  it's also a good example because it's very similar to your use case, istm.
> >
> >> Thanks,
> >> Jeremy Rudd
> > Cheers,
>
>

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Hi Harsh,

I did not have the time/need to try that one out. "threads_a_gogo" 
sounded the most useful though.


Jeremy

On 8/7/2012 2:49 AM, Harsh Dev wrote:

Hi Jeremy,
I've been looking into TAGG as well so if possible, could you give us 
an overview of what your results are/were?


Thanks,
Harsh

On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:

Thanks Jorge!

Fantastic information, it sounds like threads-a-gogo is the way to
go.
I'll use it and let you all know of my results.

Thanks again!
Jeremy



--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[greelgorke]

If you can split your calculation to partials, like recursive approach, you 
can use process.nextTick to execute the partial asynchronously. Check out 
this example https://gist.github.com/3284767



Am Dienstag, 7. August 2012 11:28:23 UTC+2 schrieb Jeremy Rudd:
>
>  Hi Harsh,
>
> I did not have the time/need to try that one out. "threads_a_gogo" sounded 
> the most useful though.
>
> Jeremy
>
> On 8/7/2012 2:49 AM, Harsh Dev wrote: 
>
> Hi Jeremy, 
> I've been looking into TAGG as well so if possible, could you give us an 
> overview of what your results are/were?
>
>  Thanks,
> Harsh
>
> On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote: 
>>
>> Thanks Jorge! 
>>
>> Fantastic information, it sounds like threads-a-gogo is the way to go. 
>> I'll use it and let you all know of my results.
>>
>> Thanks again!
>> Jeremy
>>  
>  
>  

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[George Stagas]

https://github.com/stagas/bitch allows you to define a function that
is forked away in a new process and allows you to transparently call
it whenever you want and kill it when you don't need it any more.

2012/8/7 greelgorke :
> If you can split your calculation to partials, like recursive approach, you
> can use process.nextTick to execute the partial asynchronously. Check out
> this example https://gist.github.com/3284767
>
>
>
> Am Dienstag, 7. August 2012 11:28:23 UTC+2 schrieb Jeremy Rudd:
>>
>> Hi Harsh,
>>
>> I did not have the time/need to try that one out. "threads_a_gogo" sounded
>> the most useful though.
>>
>> Jeremy
>>
>> On 8/7/2012 2:49 AM, Harsh Dev wrote:
>>
>> Hi Jeremy,
>> I've been looking into TAGG as well so if possible, could you give us an
>> overview of what your results are/were?
>>
>> Thanks,
>> Harsh
>>
>> On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:
>>>
>>> Thanks Jorge!
>>>
>>> Fantastic information, it sounds like threads-a-gogo is the way to go.
>>> I'll use it and let you all know of my results.
>>>
>>> Thanks again!
>>> Jeremy
>>
>>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jorge]

On Apr 9, 2012, at 12:34 PM, Jeremy Rudd wrote:

> Hi all,
> 
> I've got a couple of long-running functions that use BigIntegers and the like 
> for some financial calculations. These take a few seconds to execute. Now a 
> few seconds of CPU on the main node thread means all other activity grinds to 
> a halt. What's the easiest way to call these functions asynchronously? ie, on 
> another thread?
> 
> Should I be making a module? Is it possible without a module? Should I be 
> using child_process? 
> 
> Any pointers or tips appreciated.
> Thank you.

That's exactly what threads_a_gogo is for:



It lets you run these calculations in parallel, in javascript, in threads, and 
using all the available CPU cores. Then the main thread will receive the 
results in a callback or via an event.
-- 
Jorge.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Hi all,

threads_a_gogo sounds cool, but what are the technical details of using it?

1. What does it use in the back-end to schedule / manage tasks? child_process?
2. child_process takes some 30 ms to spawn a child thread. How much time does 
GoGo take?
3. How does it internally work? Are there constantly running threads + 
scheduler? Or does it spawn a new thread per task?
4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to 
show performance with / without GoGo?

Thanks,
Jeremy Rudd

On 4/10/2012 6:09 PM, Jorge wrote:

That's exactly what threads_a_gogo is for:



It lets you run these calculations in parallel, in javascript, in threads, and 
using all the available CPU cores. Then the main thread will receive the 
results in a callback or via an event.


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jorge]

On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:
> On 4/10/2012 6:09 PM, Jorge wrote:
>> That's exactly what threads_a_gogo is for:
>> 
>> 
>> 
>> It lets you run these calculations in parallel, in javascript, in threads, 
>> and using all the available CPU cores. Then the main thread will receive the 
>> results in a callback or via an event.
> 

> Hi all,

Hi Jeremy,

> threads_a_gogo sounds cool, but what are the technical details of using it?
> 
> 1. What does it use in the back-end to schedule / manage tasks? child_process?

It creates p-threads and runs v8 isolates in them.

> 2. child_process takes some 30 ms to spawn a child thread. How much time does 
> GoGo take?

When I run this:

$ time node -e "require('threads_a_gogo').createPool(100).destroy()"

I get:

real0m0.387s
user0m0.390s
sys 0m0.286s

That's 3.87 ms per thread

But note that these 0.387s include as well the time to start node, the time to 
require('threads_a_gogo'), and the time to destroy the 100 threads, so it must 
be a bit less than 3.87 ms per thread.

> 3. How does it internally work? Are there constantly running threads + 
> scheduler? Or does it spawn a new thread per task?

You could create/destroy the threads on demand, but I would rather create a 
thread pool of twice or 3x the number of cores and install in them only once 
the functions that do the calculations, then reuse them again and again via a 
pool.any.eval(program, cb). Each thread takes only ~2MB, and when idle it uses 
exactly 0% cpu.

> 4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats 
> to show performance with / without GoGo?

Threads_a_gogo lets you 

-run these blocking calls in parallel so
-they won't block node's main thread
-you'll be exploiting all the cpu cores in the machine

With an n cores machine, you'll be serving results n times faster, but even in 
a single core machine, you should use threads_a_gogo simply to avoid blocking 
node.

WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice as fast 
as in node's main thread, see:  it's also a 
good example because it's very similar to your use case, istm.

> Thanks,
> Jeremy Rudd

Cheers,
-- 
Jorge.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Thanks Jorge!

Fantastic information, it sounds like threads-a-gogo is the way to go. 
I'll use it and let you all know of my results.


Thanks again!
Jeremy

On 4/11/2012 3:05 PM, Jorge wrote:

On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:

On 4/10/2012 6:09 PM, Jorge wrote:

That's exactly what threads_a_gogo is for:



It lets you run these calculations in parallel, in javascript, in threads, and 
using all the available CPU cores. Then the main thread will receive the 
results in a callback or via an event.

Hi all,

Hi Jeremy,


threads_a_gogo sounds cool, but what are the technical details of using it?

1. What does it use in the back-end to schedule / manage tasks? child_process?

It creates p-threads and runs v8 isolates in them.


2. child_process takes some 30 ms to spawn a child thread. How much time does 
GoGo take?

When I run this:

$ time node -e "require('threads_a_gogo').createPool(100).destroy()"

I get:

real0m0.387s
user0m0.390s
sys 0m0.286s

That's 3.87 ms per thread

But note that these 0.387s include as well the time to start node, the time to 
require('threads_a_gogo'), and the time to destroy the 100 threads, so it must 
be a bit less than 3.87 ms per thread.


3. How does it internally work? Are there constantly running threads + 
scheduler? Or does it spawn a new thread per task?

You could create/destroy the threads on demand, but I would rather create a 
thread pool of twice or 3x the number of cores and install in them only once 
the functions that do the calculations, then reuse them again and again via a 
pool.any.eval(program, cb). Each thread takes only ~2MB, and when idle it uses 
exactly 0% cpu.


4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to 
show performance with / without GoGo?

Threads_a_gogo lets you

-run these blocking calls in parallel so
-they won't block node's main thread
-you'll be exploiting all the cpu cores in the machine

With an n cores machine, you'll be serving results n times faster, but even in 
a single core machine, you should use threads_a_gogo simply to avoid blocking 
node.

WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice as fast as in 
node's main thread, see:  it's also a good 
example because it's very similar to your use case, istm.


Thanks,
Jeremy Rudd

Cheers,


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en