Re: [nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

With all do respect you are in over your head :) If you want to take a stab at this for real take a peek at google-caja On Mon, Jul 2, 2012 at 9:27 PM, Will Riley wrote: > Hi, > > Right now I'm working on a sandbox library for node.js. I'd most likely be > using vm.runInNewContext to prevent an

[nodejs] Re: getting Error: Addinig a cache directory to the cache will make the world implode whenever npm install

Hmm, well I didn't figure it out exactly but I had my configuration as follows: c:\nodex>npm config list ; cli configs ; userconfig C:\Users\u0123444\.npmrc cache = "c:\\projects\\npm" prefix = "c:\\projects\\npm_global" tmp = "c:\\projects\\npm_temp" ; builtin config C:\Program Files\nodejs\no

Re: [nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

This is a talk from 2012 RubyConfIndia where two guys talk about sandboxing Ruby on the server: http://www.youtube.com/watch?v=ntIzf9onRqA Some of the stuff is Ruby specific, but there is a lot of good Linux stuff in there too. On Monday, July 2, 2012 8:20:42 PM UTC-7, Fedor Indutny wrote: > >

[nodejs] Re: getting Error: Addinig a cache directory to the cache will make the world implode whenever npm install

oops sorry, am going through google groups and not used to their new interface, anyway thought I should mention that I am on windows and I set up my npm directories to be outside of the user directory, I have to do that since my permissions to do not include those necessary for npm to work und

[nodejs] Re: getting Error: Addinig a cache directory to the cache will make the world implode whenever npm install

On Tuesday, July 3, 2012 6:24:51 AM UTC+2, bryan rasmussen wrote: > > It seems whenever I do npm install now I get the error Error: Adding a > cache directory to the cache will make the world implode, are there any > steps I should take in particular - I would prefer not having to reinstall >

[nodejs] getting Error: Addinig a cache directory to the cache will make the world implode whenever npm install

It seems whenever I do npm install now I get the error Error: Adding a cache directory to the cache will make the world implode, are there any steps I should take in particular - I would prefer not having to reinstall everything. Thanks, Bryan Rasmussen -- Job Board: http://jobs.nodejs.org/

Re: [nodejs] Web scraping and Memory leaking issue

+1 for Cheerio. On Tue, Jul 3, 2012 at 9:42 AM, rhasson wrote: > Have you looked at Cheerio (https://github.com/MatthewMueller/cheerio) ? > I've been using it over JSDom and it's faster and lightweight. If you're > doing heaving scraping I would recommend checking it out. > > > On Monday, July

Re: [nodejs] Web scraping and Memory leaking issue

Have you looked at Cheerio (https://github.com/MatthewMueller/cheerio) ? I've been using it over JSDom and it's faster and lightweight. If you're doing heaving scraping I would recommend checking it out. On Monday, July 2, 2012 10:30:25 AM UTC-4, ec.developer wrote: > > Charles, Thanks for you

[nodejs] require-dir: exactly what the name says

Had been writing little index.js files to gather directories of routes/utilities/helpers/etc. for a while, so finally got around to making a generalized module for it: https://github.com/aseemk/requireDir Hope some of you find it useful too! Aseem -- Job Board: http://jobs.nodejs.org/ Posting

Re: [nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

Hi, OneJS (http://github.com/azer/onejs) aims to do a very similar job by bundling. It wraps all the modules in your project and provides them custom implementations of global NodeJS functions & objects (require, process, console etc..); https://github.com/azer/onejs/blob/master/templates/module.

Re: [nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

Well, you may use separate node process, but you should run in in chroot or jail if you want a real safity here. Cheers, Fedor. On Mon, Jul 2, 2012 at 8:16 PM, Will Riley wrote: > Ah, I see. I saw in the docs that it says "safely running untrusted code > requires a separate process", but coul

Re: [nodejs] Re: Pusher/Pubnub versus Socket.io

Hi Mathew! You are welcome for the response. I am happy to answer more questions. 1. Cross Platform AES Symmetric Key Cryptography - Read Blog Article 1. Basically Full Round-trip Cryptography of Data (Beyond

Re: [nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

Ah, I see. I saw in the docs that it says "safely running untrusted code requires a separate process", but could you elaborate on that? Would it be better to use vanilla v8 for something like this? On Monday, July 2, 2012 10:53:26 PM UTC-4, Ben Noordhuis wrote: > > On Tue, Jul 3, 2012 at 4:27 AM

Re: [nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

On Tue, Jul 3, 2012 at 4:27 AM, Will Riley wrote: > Hi, > > Right now I'm working on a sandbox library for node.js. I'd most likely be > using vm.runInNewContext to prevent any unwanted methods (eg process.kill) > from becoming accessible, and the untrusted code would run in a different > node pro

[nodejs] Sandboxing using 'vm' module & wrapping require()/process.binding()

Hi, Right now I'm working on a sandbox library for node.js. I'd most likely be using vm.runInNewContext to prevent any unwanted methods (eg process.kill) from becoming accessible, and the untrusted code would run in a different node process. I'm looking into the possibility of enabling code to

Re: [nodejs] Re: trying to wrap my head around "promises" - async vs Q

I saw your slides and I cant agree more with you. The other day I did some thinking about all the javascript code I have been writing and I came to this conclussio (bear with me, please): - for me the problem with CPS (continuation passing style) for asynchronous code is not the cascade of nested c

[nodejs] Re: many events transmitting small amounts of data compared to one final event transmitting all the data

On Jul 2, 7:28 pm, Adeel Qureshi wrote: > now with this approach there is only one event that gets fired at the end > and all the matched data is returned. this seems like would cause less > network traffic but takes the realtime effect away also i am concerned that > if there were a lot of matche

Re: [nodejs] Re: trying to wrap my head around "promises" - async vs Q

On Sun, Jul 01, 2012 at 04:04:12PM -0700, Alexey Petrushin wrote: > There are some good use cases for control flow libraries, but, sadly, > in most cases the end result is even worse than without it. > > It seems there's no good solution to this problem - code looks ugly no > matter what You do -

Re: [nodejs] WARNING: don't use IOWatcher, it'll be removed in v0.9

On Mon, Jul 2, 2012 at 11:53 PM, Marc wrote: > Hi, > > I'm using the https://github.com/JustinTulloss/zeromq.node module. I already > forked the module (https://github.com/snrmwg/zeromq.node) to migrate to > libuv for node >= 0.7.11. > Everything works fine. Since node 0.8 I get following message

[nodejs] WARNING: don't use IOWatcher, it'll be removed in v0.9

Hi, I'm using the https://github.com/JustinTulloss/zeromq.node module. I already forked the module (https://github.com/snrmwg/zeromq.node) to migrate to libuv for node >= 0.7.11. Everything works fine. Since node 0.8 I get following message in the console: WARNING: don't use IOWatcher, it'll b

[nodejs] Re: many events transmitting small amounts of data compared to one final event transmitting all the data

Compromise is probably your best option. Send it in chunks like pagination APIs, gather up X number of lines and return results in an event. You can move X to the number of lines that gives you the best performance for your needs. I would do something like 1000 lines at a time and send the resul

[nodejs] many events transmitting small amounts of data compared to one final event transmitting all the data

Hi One of the functionality that my nodejs application provides is the ability to search thru log files which can be huge. My first approach was to read the files line by line and make a regular expression match and whenever a match is found, emit an event with the matched data something like

Re: [nodejs] Re: Testing: best practices

highkick: http://github.com/azer/highkick lowkick: http://github.com/azer/lowkick On Mon, Jul 2, 2012 at 1:57 PM, Alexey Petrushin wrote: > - mochajs for server. > - mochajs + zombiejs or phantomjs for client JS & acceptance testing. > > Don't like to stub DB calls, in most cases it's overkill, j

Re: [nodejs] Re: How would you serve 100,000 simultaneous comet requests with Node?

On Wednesday, June 6, 2012 7:34:11 PM UTC-3, Ben Noordhuis wrote: > > On Thu, Jun 7, 2012 at 12:30 AM, Marcel Mitsuto F. S. > wrote: > > I can spawn as many connections as the kernel TCP stack permits, but > then > > node crashes when it gets to the limit. > > Can you specify 'crash'? If you

[nodejs] cradle - upload images

What would be the best practise to upload images and video clips to CouchDB using cradle and node? (I'm using iriscouch as hosting service for CouchDB) -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this

[nodejs] Re: process.title on OSX (not a PS question)

On Jul 2, 3:17 pm, Garrett Johnson wrote: > I am aware setting *process.title* is ignored in OSX, but it appears > starting in 0.8.x the changes are ignored on the process object too. > Perhaps this is debatable for testing purposes and/or should at least be > added to the migration wiki? > > http

[nodejs] Re: Testing: best practices

- mochajs for server. - mochajs + zombiejs or phantomjs for client JS & acceptance testing. Don't like to stub DB calls, in most cases it's overkill, just clear it before every test, it's fast enough. -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki

[nodejs] Re: trying to wrap my head around "promises" - async vs Q

The article http://www.infoq.com/articles/surviving-asynchronous-programming-in-javascript compares seven libraries (including asynch, but not q). -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this messa

[nodejs] process.title on OSX (not a PS question)

I am aware setting *process.title* is ignored in OSX, but it appears starting in 0.8.x the changes are ignored on the process object too. Perhaps this is debatable for testing purposes and/or should at least be added to the migration wiki? https://github.com/joyent/node/blob/master/deps/uv/src/

Re: [nodejs] Re: Testing: best practices

I've been happy with mocha.js + should.js + sinon.js sinon.js is fantastic - easy way to mock, stub, spy anything. I like to put mocha.js into watch mode: file save, re-runs tests, and pops up a growl notification on pass/fail I like should.js syntactic assertion sugar on top of any object On M

Re: [nodejs] Re: [OT] The Color Code Test - Commandline Edition

Hence the OT prefix (meaning off-topic). They're just some of the common personality profiles, but done on the commandline. If you wanted to create your own survey-type thing, it's a decent example. I plan to put it to a web page as well. But really it's just something silly / fun - Most useful i

[nodejs] Re: Testing: best practices

mocha + chai (assert style) for both node.js and browser side testing On Monday, 2 July 2012 03:51:40 UTC-5, Evgeny Bogdanov wrote: > > Few questions on testing for node projects > > I m super happy with unit testing in Node. > I unit test Client/Server code by mocking external dependencies for ev

Re: [nodejs] How to end node js execution in case of a API usage error

On Mon, Jul 2, 2012 at 4:17 PM, Benjamin Schneider wrote: > I am currently writing a node js library. Now I came up with the question > what would be the best approach to indicate a usage error by the programmer. > > For example if I got the following function in my public API: > > function do

Re: [nodejs] Web scraping and Memory leaking issue

Charles, Thanks for your suggestion. About global links_grabbed - I am sure there could be a better solution, but in my case it is not so significant. I have tried, just for testing, to store 200 thousands of large links in an array, and then I've outputed the used memory, and it's amount is ver

Re: [nodejs] Web scraping and Memory leaking issue

I have totally refactored the code. I've removed the node-scraper (it uses now the request and jsdom modules). Plus I've added some parallel stuff on sending http requests. After checking 6000 pages, the RSS amount is about ~370MB. ps this is very strange that 'window' object is not destroyed by

Re: [nodejs] Testing: best practices

I usually use sinon.js to mock my ajax calls and i write my asserts against a DOM which is restored for every test... . 2012/7/2 Evgeny Bogdanov > > I don't have a clear answer how it is best to unit test client code that > has jQuery calls. > I mock jQuery calls, but it s

[nodejs] How to end node js execution in case of a API usage error

I am currently writing a node js library. Now I came up with the question what would be the best approach to indicate a usage error by the programmer. For example if I got the following function in my public API: function doSomething(aNumber) { // ... } and I need to make sure t

Re: [nodejs] Web scraping and Memory leaking issue

Hi, I had a play with your code and found a couple of things. It's probably worth trying to avoid the global variable links_grabbed as it's just getting larger and larger as you crawl. I know you need it to avoid parsing the same site twice, but perhaps you could find a more lightweight data struc

Re: [nodejs] Re: Testing: best practices

I use Nodeunit with some mocking tools On Mon, Jul 2, 2012 at 7:42 PM, Bradley Meck wrote: > I am lazy when it comes to browser testing and use Testling > http://testling.com/ . > > For Node testing I use vows. > > -- >

[nodejs] Re: Testing: best practices

I am lazy when it comes to browser testing and use Testling http://testling.com/ . For Node testing I use vows. -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this message because you are subscribed to

Re: [nodejs] Web scraping and Memory leaking issue

do you heavily rely on node-scraper? or can you use pure jsdom? Not sure where it leaks though, but didnt see much of a memory usage after closing the windows with jsdom. On Mon, Jul 2, 2012 at 3:51 PM, ec.developer wrote: > Ahhh, brilliant! Thank you. window.close() - minimized significantly th

Re: [nodejs] Web scraping and Memory leaking issue

Ahhh, brilliant! Thank you. window.close() - minimized significantly the memory usage. But it still leaks. Before closing the window I was able to check ~1000 pages. Now I can check over 1 pages, but after a while I got again the memory allocation error. On Monday, July 2, 2012 4:32:14 PM U

Re: [nodejs] Web scraping and Memory leaking issue

https://github.com/tmpvar/jsdom#how-it-works jsdom.env(html, function(`errors`, `window`) { // free memory associated with the window window.close(); }); On Mon, Jul 2, 2012 at 3:30 PM, tim sebastian wrote: > node-scraper doesnt seem to be closing the jsdom window it creates. > And honestly

Re: [nodejs] Web scraping and Memory leaking issue

node-scraper doesnt seem to be closing the jsdom window it creates. And honestly dont see a way to do so expect you play around with the node-scraper module yourself to fix this issue. Not even sure if that is the problem, but i had a similar issue working with plain jsdom, and not closing the "wi

[nodejs] Web scraping and Memory leaking issue

Hi all, I've created a small app, which searches for Not Found [404] exceptions on a specified website. I use the node-scraper module ( https://github.com/mape/node-scraper/), which uses native node's request module and jsdom for parsing the html). My app recursively searches for links on the e

Re: [nodejs] redis-client

it uses an error first callback style, so reply is actually the variable slot for the error.. Which is undefined because there isn't an error.. do: rclient.hincrby "SERVERS", "IP", 1, (err, reply) -> console.log reply instead On Monday, July 2, 2012 at 2:53 PM, Angelo Chen wrote: > Hi, > > t

[nodejs] redis-client

Hi, this works, but why reply is undefined? rclient.hincrby "SERVERS", "IP", 1, (reply) -> console.log reply thanks, angelo -- Job Board: http://jobs.nodejs.org/ Posting guidelines: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines You received this m

[nodejs] Re: Text rendering speed reduces dramatically when view on different page using Chrome

Maybe you are right. This is the very simple snippet that receives data and display them on the page: *socket.addEventListener("message", function(event) { // handle open event var data = event.data; }); * Do you know how to improve speed. I need help desperately. I thought it was data pus

[nodejs] Convention for supporting proxying of outbound HTTP traffic from Node libraries

OpenID for Node.js (node-openid, http://github.com/havard/node-openid) and node-oauth (and possibly others) have received a suggestion and pull request to support proxying outbound HTTP(S) traffic through honoring a HTTP_PROXY environment variable (see https://github.com/havard/node-openid/issu

Re: [nodejs] Re: [OT] The Color Code Test - Commandline Edition

I have no idea what these things are or why they're relevant to me as a developer who uses nodejs... except perhaps as an example of how to do certain things in nodejs? If that's your intention, for which nodejs-related programming problems do these packages exemplify good solutions? On Jul 2,

[nodejs] Testing: best practices

Few questions on testing for node projects I m super happy with unit testing in Node. I unit test Client/Server code by mocking external dependencies for every function (DB, etc.) I don't have a clear answer how it is best to unit test client code that has jQuery calls. I mock jQuery calls, but

Re: [nodejs] [libuv] thread, uv_loop_new and uv_accept dangerous cocktail

Hi, Thanks for the tip! I'll implement this solution this evening and I'll see if I'm able to increase the number of request/s of this HTTP server can handle. Cheers, Mathieu On Monday, July 2, 2012 3:05:45 AM UTC+2, Ben Noordhuis wrote: > > On Mon, Jul 2, 2012 at 1:39 AM, Mathieu Garaud >

[nodejs] Re: Hook.io performance

Hello, perhaps you could give a try at tinyhook ( https://github.com/sergeyksv/tinyhook), which aims differents use case than hook.io (simpler, lighter, with less features, no self-healing, etc) but seems to better perform in raw performances. It doesn't use dnode. Its api is quite compatible wi

[nodejs] Re: [OT] The Color Code Test - Commandline Edition

I'm on a roll: https://github.com/coolaj86/the-5-love-languages AJ ONeal On Sun, Jul 1, 2012 at 1:41 AM, AJ ONeal wrote: > Ever wanted to know your color but were too afraid to ask? > Does knowing that 'a' means red, 'b' means, blue, etc make it too hard for > you to not cheat the system? > >