Re: [nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Hi Harsh,

I did not have the time/need to try that one out. "threads_a_gogo" 
sounded the most useful though.


Jeremy

On 8/7/2012 2:49 AM, Harsh Dev wrote:

Hi Jeremy,
I've been looking into TAGG as well so if possible, could you give us 
an overview of what your results are/were?


Thanks,
Harsh

On Wednesday, April 11, 2012 11:15:50 PM UTC-7, Jeremy Rudd wrote:

Thanks Jorge!

Fantastic information, it sounds like threads-a-gogo is the way to
go.
I'll use it and let you all know of my results.

Thanks again!
Jeremy



--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Secure distribution of NodeJS applications

[Jeremy Rudd]

*What:* Can NodeJS apps be distributed as binary? ie. you compile the .js 
app via V8 into its native binary, and distribute the binary to our 
clients? ... or is minifying the code all you can do?

*Why:* We build serverside applications in NodeJS for clients, that have 
often to be hosted on the client's servers. Distributing source code means 
clients can easily steal our solution and stop paying licensing fees. This 
opens up the possibility of easy reverse-engineering or reuse of our apps 
without our awareness.

*Shamelessly cross posted on*: 
http://stackoverflow.com/questions/9413123/secure-distribution-of-nodejs-applications

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] NodeJS downloadable Docs

[Jeremy Rudd]

Hi,

I couldn't find a downloadable version of the NodeJS docs anywhere, so I 
grabbed a copy of the "docs" portion of the website using a sitegrabber, 
and I've uploaded the resulting file in case anyone else is interested. 
Please note that only links under the "docs" section work. Click the "docs" 
menu button to return to the docs homepage, then click a link to dig deeper.

*NodeJS Docs.chm <http://www.box.com/s/hpp75sg16lrmeehu4lpa>* (200 KB)

Could some admin upload this on the nodeJS site under the "docs" section? 
It could be a header on top somewhere; I'm sure there are people who would 
like offline docs.

Thanks,
Jeremy Rudd

PS: In case you cannot click the link above, copy this 
: http://www.box.com/s/hpp75sg16lrmeehu4lpa

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] Secure distribution of NodeJS applications

[Jeremy Rudd]

Don't know much about C++ addons. Can they access NodeJS objects as usual? 
All classes and functions?

Anyways, about the VM thing, is it possible to capture the snapshot the V8 
engine generates, and then use that to run the app?

Regardless, if you're saying that the Node/V8 engines keep the source code 
in memory, that means less security. So I'll have to resort to source code 
obfuscation / uglification.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] NodeJS downloadable Docs

[Jeremy Rudd]

Sorry dtan, CHM is a windows-only Compiled HTML package. I'll output the 
raw HTML for Mac/Unix users. Here:

*NodeJS Docs.zip * (220 KB)

PS: If you cannot see the link: http://www.box.com/s/4efuqosleeign5p2om7m

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] NodeJS downloadable Docs

[Jeremy Rudd]

I attached a ZIP if you prefer the browser.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: NodeJS downloadable Docs

[Jeremy Rudd]

Can someone please upload my CHM/ZIP on the Node website under to Docs
folder?

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: NodeJS downloadable Docs

[Jeremy Rudd]

Thanks TC,

But .markdown files cannot be natively viewed on usual computers. HTML/
CHM is a more friendly format.

JR

On Feb 23, 11:19 pm, "T.C. Hollingsworth" 
wrote:
> On Thu, Feb 23, 2012 at 9:50 AM, Jeremy Rudd  
> wrote:
> > Hi,
>
> > I couldn't find a downloadable version of the NodeJS docs anywhere, so I
> > grabbed a copy of the "docs" portion of the website using a sitegrabber, and
> > I've uploaded the resulting file in case anyone else is interested. Please
> > note that only links under the "docs" section work. Click the "docs" menu
> > button to return to the docs homepage, then click a link to dig deeper.
>
> The documentation is included in the "doc" folder of the tarball and in git.
>
> While it's still probably useful to have it separate and/or in CHM
> format, at least you won't have to scrape the website to do it on the
> next update.  ;-)
>
> > NodeJS Docs.chm (200 KB)
>
> > Could some admin upload this on the nodeJS site under the "docs" section? It
> > could be a header on top somewhere; I'm sure there are people who would like
> > offline docs.
>
> > Thanks,
> > Jeremy Rudd
>
> > PS: In case you cannot click the link above, copy this
> > : http://www.box.com/s/hpp75sg16lrmeehu4lpa
>
> -T.C.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: Secure distribution of NodeJS applications

[Jeremy Rudd]

On Feb 24, 11:52 pm, Lalo Martins  wrote:

> And please don't co-opt the word “secure” for it.

I'm a dev, not management. I understand the field internally and so
I'm trying to start discussions to securely distribute NodeJS apps.
Anyone wants to vote if C++ EXE's are harder to decompile than .NET
EXE's? Just throw the damn thing into Reflector and viola!. With C++
you never really get great code.


> Distribute your product with source. If your clients have enough
> technical ability to “steal” it, they also have enough to help you fix
> bugs.

I work with assembly and IL. Source code can be compiled perfectly.
Decompiled code almost NEVER compiles perfectly, the program crashes
and/or strange errors appear. Binaries force the reverse engineer to
work with assembly at a low level. Source code allows any idiot
programmer to have a go at it.


> And then make it a big deal in your promotion material that it comes with
> sources.

We're trying to protect our application from clients, not GIVE it to
them. We're distributing applications that are run internally in
organizations. We won't even KNOW if they are running hundreds of
instances/copies of our software.


> If somebody “steals” your work, sue them. Contract law is more than
> sufficient for that, and if your clients are determined to “steal” your
> work, binaries won't stop them.

Binaries will slow them down. Enough to be helpful.


> If the problem is more about especially clever things you do to solve
> their problem -- again, contracts will protect you much better than
> binaries or obscured code, but you also have the option of patents.

Patents give away the techniques. Ever wonder why McDonald machines
are not patented? Did you know that in China, for instance, the govt
openly allows local companies to copy techniques described in patents?
So much for patents.



> Odds are, though, the thing you're trying to protect isn't 10% as valuable as
> you think it is.)

That's not for you to judge :) .. and not for me to judge either. I'm
a dev, not a forecaster.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: Secure distribution of NodeJS applications

[Jeremy Rudd]

On Feb 26, 7:18 pm, manimal45  wrote:
> It would be awesome if 'customers' could download node.js apps as
> executable.
> This would open the road for many new applications where a central
> "cloud" server instance could communicate with customers
> infrastructure via agents.

I second this. There needs to be a way to 'compile' nodeJS apps into
its binary format. AND a way to distribute this as a self-extracting
installer, look at Firefox, I mean they don't ask you to install from
source. Custom branding would also improve the business-value of
distributing NodeJS apps. It looks more pro.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: Secure distribution of NodeJS applications

[Jeremy Rudd]

On Feb 26, 8:09 pm, mscdex  wrote:
> It'd be neat to be able to have specific modules' (bundled in the node
> executable) exported functions not show their source code
> when .toString()'ed, kind of like how native C++ functions show
> "[native code]" when .toString()'ed.

Exactly. I mean which application framework on earth allows you to
"view source" from the BINARIES?! (except HTML/JS!) The toString()
function should just return a blank string from binary NodeJS apps.
This would be really useful.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: Secure distribution of NodeJS applications

[Jeremy Rudd]

On Feb 26, 7:18 pm, manimal45  wrote:
> It would be awesome if 'customers' could download node.js apps as
> executable.

I've opened a new issue for this. Add your comments there!

https://github.com/joyent/node/issues/2835

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: Secure distribution of NodeJS applications

[Jeremy Rudd]

> There is no such thing as a javascript binary. The code included with node
> is stored in string format in the node executable.

No worries, what if the code was compiled via V8, and then stored as a
data file?
Whatever format it is in, if you could just load the same datafile you
would'nt need
the source code again.


> In my opinion, this is not a Node problem to solve. Obfuscation/source code
> hiding is an opportunity for a third party to make a native module to
> encrypt/decrypt source files.

You don't need to encrypt the source if you can just work with the
intermediate data format. See above.


> Most of us are standing on the shoulders of giants. This is why node is so
> great. But yet, we are selling software touting features that are really a
> gift from the Node community, and that we got for mostly free.

I'm sorry to sound capitalistic, but sometimes you need to make
commercial products for clients that are willing to pay. If my work
could be protected in such a case, it would help me. And yes, nodeJS
is a great gift, and I would look into giving whatever else I create
back.

> Bitching cause the Node creators were generous enough to share their work
> with you, but  not  cheap enough to lock it down is ludicrous.

Sorry if I sounded like I was bitching, I was just trying to make
NodeJS a more professional platform: .NET, Java, Flash all allow you
to 'compile' your app in some way and so I was just asking if
something like this could be implemented for node.

Best Regards

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Hi all,

I've got a couple of long-running functions that use BigIntegers and the 
like for some financial calculations. These take a few seconds to execute. 
Now a few seconds of CPU on the main node thread means all other activity 
grinds to a halt. What's the easiest way to call these functions 
asynchronously? ie, on another thread?

Should I be making a module? Is it possible without a module? Should I be 
using child_process? 

Any pointers or tips appreciated.
Thank you.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] Re: How do you execute a function asynchronously?

[Jeremy Rudd]

I've gone through what you said, and here are my results:

- Child_process is slow to startup (30 ms) and consumes 10 MB base since 
its a whole new V8 instance. Maybe if these are started and not stopped 
and used alongside a job queue it could work. But it cannot work like 
"async function calls" where you start one for each small calc you need 
to do.


- WebWorkers seem to be an HTML5 integration API, and extra bloat is not 
needed


- Cannot find anything like "backgrounder"

- Gearman-node seems more suited to longer running jobs, and not 1 sec 
tasks.


- Resque seems to be a reliable queue, should one be needed. -- 
https://github.com/blog/542-introducing-resque


Currently I'll just run my calc on the main CPU and see what happens. 
Any ideas anybody?


- Should I use gearman?
- Or should I go with child_process since I currently have the task code 
also written in NodeJS, and not C++.



On 4/9/2012 4:15 PM, Oleg Efimov (Sannis) wrote:
Yes, you can execute them in child_process. Also where may be already 
existing modules, try look for maintained forks of "webworkers" or 
"backgrounder".

Also you can use queues: kue or geaman-node


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Hi all,

threads_a_gogo sounds cool, but what are the technical details of using it?

1. What does it use in the back-end to schedule / manage tasks? child_process?
2. child_process takes some 30 ms to spawn a child thread. How much time does 
GoGo take?
3. How does it internally work? Are there constantly running threads + 
scheduler? Or does it spawn a new thread per task?
4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to 
show performance with / without GoGo?

Thanks,
Jeremy Rudd

On 4/10/2012 6:09 PM, Jorge wrote:

That's exactly what threads_a_gogo is for:

<https://github.com/xk/node-threads-a-gogo>

It lets you run these calculations in parallel, in javascript, in threads, and 
using all the available CPU cores. Then the main thread will receive the 
results in a callback or via an event.


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] How do you execute a function asynchronously?

[Jeremy Rudd]

Thanks Jorge!

Fantastic information, it sounds like threads-a-gogo is the way to go. 
I'll use it and let you all know of my results.


Thanks again!
Jeremy

On 4/11/2012 3:05 PM, Jorge wrote:

On Apr 11, 2012, at 9:40 AM, Jeremy Rudd wrote:

On 4/10/2012 6:09 PM, Jorge wrote:

That's exactly what threads_a_gogo is for:

<https://github.com/xk/node-threads-a-gogo>

It lets you run these calculations in parallel, in javascript, in threads, and 
using all the available CPU cores. Then the main thread will receive the 
results in a callback or via an event.

Hi all,

Hi Jeremy,


threads_a_gogo sounds cool, but what are the technical details of using it?

1. What does it use in the back-end to schedule / manage tasks? child_process?

It creates p-threads and runs v8 isolates in them.


2. child_process takes some 30 ms to spawn a child thread. How much time does 
GoGo take?

When I run this:

$ time node -e "require('threads_a_gogo').createPool(100).destroy()"

I get:

real0m0.387s
user0m0.390s
sys 0m0.286s

That's 3.87 ms per thread

But note that these 0.387s include as well the time to start node, the time to 
require('threads_a_gogo'), and the time to destroy the 100 threads, so it must 
be a bit less than 3.87 ms per thread.


3. How does it internally work? Are there constantly running threads + 
scheduler? Or does it spawn a new thread per task?

You could create/destroy the threads on demand, but I would rather create a 
thread pool of twice or 3x the number of cores and install in them only once 
the functions that do the calculations, then reuse them again and again via a 
pool.any.eval(program, cb). Each thread takes only ~2MB, and when idle it uses 
exactly 0% cpu.


4. Is it recommended for 1-10 ms tasks as well? Do you have any idea / stats to 
show performance with / without GoGo?

Threads_a_gogo lets you

-run these blocking calls in parallel so
-they won't block node's main thread
-you'll be exploiting all the cpu cores in the machine

With an n cores machine, you'll be serving results n times faster, but even in 
a single core machine, you should use threads_a_gogo simply to avoid blocking 
node.

WRT to performance, a fibonacci(40) in a thread_a_gogo runs about twice as fast as in 
node's main thread, see:<https://gist.github.com/2018811>  it's also a good 
example because it's very similar to your use case, istm.


Thanks,
Jeremy Rudd

Cheers,


--
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en