[nodejs] new framework for json apis

[Gregg Caines]

Hey all... I've been working on a new framework for restful json apis for a 
while, and was wondering if anyone might be interested in taking a look, 
giving feedback, etc.

http://percolatorjs.com 

It's definitely a bit weird, but of course I think it's the best way to 
create JSON APIs.  I didn't really have any desire to duplicate what's 
already been done.

It's also not production ready ( because I'm not using it in production 
myself for another month or so), so I'm not really trying to spread it far 
and wide at this point, but I'd love to hear any 
comments/criticism/questions from the big brains here about what I have so 
far.

Thanks,

G

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] new framework for json apis

[Gregg Caines]

Kinda-sorta-almost.  That draft makes it look more complicated than it is
though.  And nothing forces you to follow that format either.  You can
still make old-school arbitrary json apis like twitter and facebook do, if
you really want to.


On Sun, Dec 16, 2012 at 11:24 AM, Scott Elcomb  wrote:

> On Sun, Dec 16, 2012 at 1:38 PM, Gregg Caines  wrote:
> > Hey all... I've been working on a new framework for restful json apis
> for a
> > while, and was wondering if anyone might be interested in taking a look,
> > giving feedback, etc.
> >
> > http://percolatorjs.com
>
> Is it fair to assume that percolatorjs is an implementation of your
> application/vnd.hyper+JSON draft at
> <http://caines.ca/blog/programming/hyperjson-a-first-draft/> ?
>
> --
>   Scott Elcomb
>   @psema4 on Twitter / Identi.ca / Github & more
>
>   Atomic OS: Self Contained Microsystems
>   http://code.google.com/p/atomos/
>
>   Member of the Pirate Party of Canada
>   http://www.pirateparty.ca/
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] new framework for json apis

[Gregg Caines]

I assume you mean the standard HTTP methods?  I agree... That would indeed
be a no-starter if I'd left those out.  :)

Check the docs at http://percolatorjs.com/documentation.html#server-route .
 They show the use of multiple methods on one resource.  You can add others
as well as you'd no doubt expect.

This is different from express/sinatra in that I try to promote keeping
every method of a single route together in a single place.  This makes for
better code organization and re-use, and lets the framework handle some
HTTP errors more correctly.

I still need to document better and make this stuff more obvious, so thanks
for the feedback.

G



On Sun, Dec 16, 2012 at 12:11 PM, Tiger Nassau Inc <
john.tigernas...@gmail.com> wrote:

> Does not seem to use standard REST calls - that is a no start - needs to
> be as simple as php slim, then would be really nice
> Sent from my LG Mobile
>
> Scott Elcomb  wrote:
>
> >On Sun, Dec 16, 2012 at 1:38 PM, Gregg Caines  wrote:
> >> Hey all... I've been working on a new framework for restful json apis
> for a
> >> while, and was wondering if anyone might be interested in taking a look,
> >> giving feedback, etc.
> >>
> >> http://percolatorjs.com
> >
> >Is it fair to assume that percolatorjs is an implementation of your
> >application/vnd.hyper+JSON draft at
> ><http://caines.ca/blog/programming/hyperjson-a-first-draft/> ?
> >
> >--
> >  Scott Elcomb
> >  @psema4 on Twitter / Identi.ca / Github & more
> >
> >  Atomic OS: Self Contained Microsystems
> >  http://code.google.com/p/atomos/
> >
> >  Member of the Pirate Party of Canada
> >  http://www.pirateparty.ca/
> >
> >--
> >Job Board: http://jobs.nodejs.org/
> >Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >You received this message because you are subscribed to the Google
> >Groups "nodejs" group.
> >To post to this group, send email to nodejs@googlegroups.com
> >To unsubscribe from this group, send email to
> >nodejs+unsubscr...@googlegroups.com
> >For more options, visit this group at
> >http://groups.google.com/group/nodejs?hl=en?hl=en
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] new framework for json apis

[Gregg Caines]

Totally agree, but I'm not even done the server-side yet. :)  I'm not even
entirely sure what the client library should do yet (beyond the standard
http stuff like request or super-agent do), and the only consumers of the
API I've written with it are in ruby and erlang, so I'm even a little
language agnostic for client libs.


On Sun, Dec 16, 2012 at 12:00 PM, Jonathan Chayce Dickinson <
jonathand...@gmail.com> wrote:

> Good work! Vannevar Bush would be proud of you :).
>
> A client library to consume this would be nice...
>
> Jonathan
>
> On Sun, Dec 16, 2012 at 9:24 PM, Scott Elcomb  wrote:
>
>> On Sun, Dec 16, 2012 at 1:38 PM, Gregg Caines  wrote:
>> > Hey all... I've been working on a new framework for restful json apis
>> for a
>> > while, and was wondering if anyone might be interested in taking a look,
>> > giving feedback, etc.
>> >
>> > http://percolatorjs.com
>>
>> Is it fair to assume that percolatorjs is an implementation of your
>> application/vnd.hyper+JSON draft at
>> <http://caines.ca/blog/programming/hyperjson-a-first-draft/> ?
>>
>> --
>>   Scott Elcomb
>>   @psema4 on Twitter / Identi.ca / Github & more
>>
>>   Atomic OS: Self Contained Microsystems
>>   http://code.google.com/p/atomos/
>>
>>   Member of the Pirate Party of Canada
>>   http://www.pirateparty.ca/
>>
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nodejs@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>
>  --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] new framework for json apis

[Gregg Caines]

Ohh good questions...

> There doesn't seem to be any support for streams which sucks.

Check out http://percolatorjs.com/documentation.html#context-req and
http://percolatorjs.com/documentation.html#context-res .  Those are the
unaltered request and response objects, so you can still stream as well you
ever could from the node standard library.  There's no synchronous
middleware system like connect here either, so you basically get the stream
as soon as node does too.

> Using `onX(cb)` instead of `.on("x", cb)` is going against the grain of
node for no reason.

I used the word 'on' in onRequest(), onJson() and onBody(), but those are
not really for streams or even general events and should only ever have 1
handler/listener, so I didn't feel the need to try to emulate the streams
or events API.  Let me know if you think they're more 'eventy' than I do
though.  I could certainly name them setRequestPreprocessor(), withJson(),
and withBody() too if that leads to less confusion with event/stream APIs.
 I'll have to throw this around in my brain a bit.

> And other then the Hyper+JSON and JSONmodule thing there doesn't seem to
be something here that isn't covered by other libraries.

I'm going to go out on a limb and assume you're basically saying this
could've been implemented on top of express.  That's not the case though
once you get into the details:

* Percolator comes with completely different routing that lets you build
re-usable components (like that JSONModule).  Building that on top of
express (where two methods of the same resource know nothing of one
another) would have been gnarly and not really got me much more than node's
standard http server does.  The standard lib does a heck of a lot.
* I didn't care to have a synchronous middleware system like connect,
because I've yet to find a case for a middleware system like that that
can't be handled by a 1-liner lazily, and on-demand ($.onJson() is a good
example of that -- 80% of my requests never need a bodyParser, and the ones
that do get it with 1 line).
* I didn't want to have any dynamic html generating functionality at all
(though it's easily added to an app).  If your app is more website-y and
not a single-page-app-y, then express is simply the better choice.  If you
have a single page app with static assets backed by an API, then you're in
Percolator's sweet spot.  I generally think PJAX should die in a fire
though, so I don't care to contribute to it.  I said this was an
opinionated framework, right?  :)

There are a whole lot of things outside of json apis that express can do
that Percolator will never do either, so I'd never try to compare this to
express in general (express is better at basically everything else, even in
my biased opinion).  Percolator was more inspired by webmachine (
https://github.com/basho/webmachine/wiki ) and its use-cases than sinatra
though, so it naturally doesn't use express.

Thanks for the input!


On Sun, Dec 16, 2012 at 3:04 PM, Jake Verbaten  wrote:

> Using `onX(cb)` instead of `.on("x", cb)` is going against the grain of
> node for no reason.
>
> I recommend you change that api.
>
> There doesn't seem to be any support for streams which sucks.
>
> And other then the Hyper+JSON and JSONmodule thing there doesn't seem to
> be something here that isn't covered by other libraries.
>
> Which means I recommend you take those two concepts and make them seperate
> open source libraries
>
>
> On Sun, Dec 16, 2012 at 1:28 PM, Gregg Caines  wrote:
>
>> Totally agree, but I'm not even done the server-side yet. :)  I'm not
>> even entirely sure what the client library should do yet (beyond the
>> standard http stuff like request or super-agent do), and the only consumers
>> of the API I've written with it are in ruby and erlang, so I'm even a
>> little language agnostic for client libs.
>>
>>
>> On Sun, Dec 16, 2012 at 12:00 PM, Jonathan Chayce Dickinson <
>> jonathand...@gmail.com> wrote:
>>
>>> Good work! Vannevar Bush would be proud of you :).
>>>
>>> A client library to consume this would be nice...
>>>
>>> Jonathan
>>>
>>> On Sun, Dec 16, 2012 at 9:24 PM, Scott Elcomb  wrote:
>>>
>>>> On Sun, Dec 16, 2012 at 1:38 PM, Gregg Caines  wrote:
>>>> > Hey all... I've been working on a new framework for restful json apis
>>>> for a
>>>> > while, and was wondering if anyone might be interested in taking a
>>>> look,
>>>> > giving feedback, etc.
>>>> >
>>>> > http://percolatorjs.com
>>>>
>>>> Is it fair to assum

Re: [nodejs] new framework for json apis

[Gregg Caines]

> req.on("end", handler) is an event that only happens once and and will
only have a single handler in most cases. Just make request, json and body
events and make your $ thing an event emitter.

I think you've sold me on that.  Good points.

> CRUDCollection (its annoying to rename your concepts a day after you
publish your ideas on the mailing list) could be something returns a (req,
res) function I can pass to any router I want. For example

Sorry. It was actually renamed before I published to the list and I missed
updating the one spot in the docs that you found.  You helped me find a
documentation bug if it makes you feel any better. :)

> What's im trying to suggest is don't write a JSON framework. Write
modules that make writing a JSON API server easier. Then later build
whatever opinionated framework you want on top of it.

Alright so there are a couple of problems that I ran into with this
approach mainly revolving around modularity.  Percolator lets you create
node modules out of the objects that handle the requests for methods of a
single 'resource'.  These modules need more than req and res though if
they're going to share functionality server-wide.  Functionality like error
handling, server configuration information, inevitable global variables,
etc.  There are a number of aspects that can't just be require()ed in and
must be passed at runtime.  I'd considered a third parameter for that
stuff, or monkey-patching req and res like express does, but neither seemed
very pure from a node perspective, so I went with what would be easiest on
the app developer, and would ultimately make creating a consistent http api
easiest.

One good example in CRUDCollection is the triple-duty that fetch()
provides.  That method is called for GET, PUT, and DELETE to determine a
404 scenario automatically before the handlers for GET, PUT and DELETE are
even called.  The system-wide error handler is called, with the current
request url to put out a nice 404 message in the case that the thing could
not be retrieved.  If it could be retrieved, you don't have to make that
database call again in GET to send it to the user, it's simply available as
$.fetched .

These might seem like small things, but these small things are basically
death-by-1000-cuts when trying to write a real API.  See the difference in
length between the express version and the Percolator version for example (
http://percolatorjs.com/examples.html ).  There's no one place where the
express version is obviously wordy, and yet it ends up being twice as long.
 What's additionally not shown is that the linksCollection in the
percolator example could have been require()ed in from a separate file
because it doesn't rely on closures to share state like the express version
does.

I'm interested to hear if you have a better a solution for those problems
though.  Thanks for the feedback again.

G










On Mon, Dec 17, 2012 at 1:58 AM, Jake Verbaten  wrote:

> > I used the word 'on' in onRequest(), onJson() and onBody(), but those
> are not really for streams or even general events and should only ever have
> 1 handler/listene
>
> req.on("end", handler) is an event that only happens once and and will
> only have a single handler in most cases. Just make request, json and body
> events and make your $ thing an event emitter.
>
> > assume you're basically saying this could've been implemented on top of
> express.
>
> Hell no. You can implement it on top of http.
>
> CRUDCollection (its annoying to rename your concepts a day after you
> publish your ideas on the mailing list) could be something returns a (req,
> res) function I can pass to any router I want. For example
>
> var module = CRUDCollection({
>   ...
> })
>
> arbitaryRouter.addRoute("/myCollection", module.handler)
> arbitaryRouter.addRoute("/myCollection/:memberid", module.wildcard)
>
> > I didn't care to have a synchronous middleware system
>
> Agreed middleware systems are poor.
>
> What's im trying to suggest is don't write a JSON framework. Write modules
> that make writing a JSON API server easier. Then later build whatever
> opinionated framework you want on top of it.
>
> I've listed a few of those modules (
> https://github.com/Raynos/routil#routil ), [npm-www](
> https://github.com/isaacs/npm-www) demonstrates a bunch more of those
> modules.
>
>
> On Sun, Dec 16, 2012 at 4:21 PM, Gregg Caines  wrote:
>
>> Ohh good questions...
>>
>> > There doesn't seem to be any support for streams which sucks.
>>
>> Check out http://percolatorjs.com/documentation.html#context-req and
>> http://percolatorjs.com/documentation.html#context-res .  Those are the
>> unaltered 

Re: [nodejs] new framework for json apis

[Gregg Caines]

Very cool, Shaun...  Feel free to give me a shout anytime if you have
questions, or find bugs.

G


On Mon, Dec 17, 2012 at 3:55 PM, shaun etherton wrote:

>
> Hey Gregg
>
> FWIW. I like it so far and have started to use it.
>
> I'm mainly experimenting with it at this point, but your framework is a
> good fit for what i was looking for.
>
> Thank you.
>
>

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] new framework for json apis

[Gregg Caines]

I mostly agree, but I personally think it's worse than you make it out to
be.  Take a look at the Percolator and express examples at
http://percolatorjs.com/examples.html and you'll see that most of the
reason that the express version is ~2x longer is due to error handling.  I
find setting up error-handlers to be boring as hell and frankly don't want
to do it anymore.  Also see the absolutely brilliant webmachine state
machine diagram at
https://raw.github.com/wiki/basho/webmachine/images/http-headers-status-v3.pngto
see all the kinds of errors that can occur, many before you even
route.
 And when a good REST API responds with a 404, for example, it should do so
in exactly the same way every time regardless of whether the 404 is found
in the router (no route exists) or in the resource (the requested item
doesn't exist in the database).

Real-world apps need to share a lot more across response handlers than just
error-handling though.  In just my current app I need to share these common
services amoung the different response handlers:  statsd, the database, a
logger, and a message queue.

And any time you want to share any centralized resource, it either needs to
be:
1. explicitly passed (breaking the function(req, res) signature)
OR
2. monkey-patched on either of the req or res objects
OR
3. you're stuck with closures implicitly pulling in external state.

I threw out #3 immediately because relying on closures and external state
keeps me from splitting up my response handlers into separate
modules/files.  I also found #2 to be kind of unsanitary, and only
superficially more consistent with the node http server than #1.  Let's
face it, if req and res aren't exactly what node gives back, then you've
lost backward compatibility anyway.  It's not enough for them to be named
the same and contain mostly the same stuff if the code inside makes use of
any of the differences (and the entire point is to make use of the
differences).

Let me know if you have better ideas for solving these problems though...

G



On Mon, Dec 17, 2012 at 5:53 PM, Jake Verbaten  wrote:

> The problem of how do I write a generic request handler but still let the
> user decide how to handle errors is frustratingly hard.
>
> The best solution I have come up with is.
>
> router.addRoute("/thing", require("route-handler")({ error:
> myErrorFramework }))
>
> which is annoying at best. you hard couple the signature of the error
> callback to a particular neutral signature that your error framework
> handles.
>
> But really it's not that bad. You configure your application wide error
> handling system, then configure all your routes in routes.js and pass in
> the error handler to each route handler.
>
> When building things like these I've only found error handling to be that
> annoying thing that you need everywhere and only be configured once.
>
>
>
> On Mon, Dec 17, 2012 at 7:36 AM, Gregg Caines  wrote:
>
>> > req.on("end", handler) is an event that only happens once and and will
>> only have a single handler in most cases. Just make request, json and body
>> events and make your $ thing an event emitter.
>>
>> I think you've sold me on that.  Good points.
>>
>> > CRUDCollection (its annoying to rename your concepts a day after you
>> publish your ideas on the mailing list) could be something returns a (req,
>> res) function I can pass to any router I want. For example
>>
>> Sorry. It was actually renamed before I published to the list and I
>> missed updating the one spot in the docs that you found.  You helped me
>> find a documentation bug if it makes you feel any better. :)
>>
>> > What's im trying to suggest is don't write a JSON framework. Write
>> modules that make writing a JSON API server easier. Then later build
>> whatever opinionated framework you want on top of it.
>>
>> Alright so there are a couple of problems that I ran into with this
>> approach mainly revolving around modularity.  Percolator lets you create
>> node modules out of the objects that handle the requests for methods of a
>> single 'resource'.  These modules need more than req and res though if
>> they're going to share functionality server-wide.  Functionality like error
>> handling, server configuration information, inevitable global variables,
>> etc.  There are a number of aspects that can't just be require()ed in and
>> must be passed at runtime.  I'd considered a third parameter for that
>> stuff, or monkey-patching req and res like express does, but neither seemed
>> very pure from a node perspective, so I went with what would be easiest on
>> the app developer, and would ultimately 

[nodejs] Re: Is it wise to host nodejs apps on third party machines with source code? Can't we compile into machine code before hosting?

[Gregg Caines]

You shouldn't look to compilation to solve this problem, because 
compilation is not encryption.  Even if it was compiled C, anyone who can 
open a hexeditor can see your username and password in there.  I bet you 
could even see it from notepad.

An encryption scheme won't be bulletproof either, because you've got to put 
the decryption logic on the machine as well, or your app won't actually be 
able to use the decrypted username/password.

Of course you could encrypt anyway; that's at least mildly harder than 
reading plain text (but still pretty trivial to decrypt -- they just need 
to console.log(decryptedPassword) before you use it in your code).

I think you've either got to trust your 3rd-party hosts, or stop using 
3rd-party hosts.  As with most scenarios, if someone malicious has full 
access to your machine, there's not much you can do.

G



On Wednesday, January 9, 2013 2:31:55 AM UTC-8, Koti Kanna wrote:
>
> Hi all,
>
> I am using npm module imap to fetch emails. where it requires username and 
> password to fetch mails. This mail client i am hosting on a third party 
> machine. My worry is, if any body open this file they can actually see my 
> user name and password. Is there any way to precompile this module into 
> machine code?
>
> Do you have any plans to support executables for nodejs Apps, before it 
> touches version 1.0?
>
> Thanks
> Koti
>
> P.S.: The one and only language right now i know is JavaScript.
>

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] auto-reconnect strategy question

[Gregg Caines]

Hey all, I'm working on https://github.com/cainus/Prozess, which is a 
client library for Kafka and I'm looking to try to implement some sort of 
auto-reconnect capability.

What are the usual strategies for that with regards to handling additional 
writes while the client reconnects?

I was thinking about 2 ways to handle when app code tries to write while 
the client is reconnecting:
(A) immediately return a "reconnecting" error in my callback so that app 
code can know that it can retry shortly.
OR
(B) continue to take new writes from the app code, but just make them wait 
for the reconnect, and fail each of them if a single reconnect attempt fails

I'd like to ensure a reliable write either way, so I'm not considering 
solutions like "just queue up the write data until it can be sent".

Does anyone have a better third option, or recommendation on one of these 
above?  Is there a standard way that other libraries are doing this?

Thanks!

G

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] ANN: coveralls.io coverage reporting for node.js

[Gregg Caines]

Hey all... I just published a rudimentary package for exporting node.js 
code coverage data to http://coveralls.io/ which lets you add a nifty code 
coverage button to your package's README, as seen here:  
https://github.com/cainus/urlgrey#readme and gives you nice reporting like 
https://coveralls.io/r/cainus/urlgrey .

Here's the package: https://npmjs.org/package/coveralls 

You need mocha's jscov reporter (or anything that reports in the same 
format), so you probably need tests in mocha to use this.  It's certainly 
possible that other coverage reporters use the same format though.  You 
also need a build running in travis-ci to use this.

If you have a different coverage report format, let me know about it, and I 
can probably add it.

Any questions/comments/etc are welcome.

G

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] ANN: coveralls.io coverage reporting for node.js

[Gregg Caines]

If you send me example json output from istanbul, I can make it work pretty
easily... I couldn't figure out what istanbul was outputting from its
README other than html.


On Sun, Mar 24, 2013 at 10:56 AM, Daniel Rinehart wrote:

> Will this with work with the reports generated by Istanbul?
>
> https://github.com/gotwarlost/istanbul
>
>
> -- Daniel R.  [http://danielr.neophi.com/]
>
>
> On Sat, Mar 23, 2013 at 10:50 PM, Gregg Caines  wrote:
>
>> Hey all... I just published a rudimentary package for exporting node.js
>> code coverage data to http://coveralls.io/ which lets you add a nifty
>> code coverage button to your package's README, as seen here:
>> https://github.com/cainus/urlgrey#readme and gives you nice reporting
>> like https://coveralls.io/r/cainus/urlgrey .
>>
>> Here's the package: https://npmjs.org/package/coveralls
>>
>> You need mocha's jscov reporter (or anything that reports in the same
>> format), so you probably need tests in mocha to use this.  It's certainly
>> possible that other coverage reporters use the same format though.  You
>> also need a build running in travis-ci to use this.
>>
>> If you have a different coverage report format, let me know about it, and
>> I can probably add it.
>>
>> Any questions/comments/etc are welcome.
>>
>> G
>>
>> --
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nodejs@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>  --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/npo8HaNI5Wk/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] ANN: coveralls.io coverage reporting for node.js

[Gregg Caines]

Actually... Istanbul looks like it can generate an lcov .info file.  That
should be fine.  I'll give it a go.

G


On Sun, Mar 24, 2013 at 10:56 AM, Daniel Rinehart wrote:

> Will this with work with the reports generated by Istanbul?
>
> https://github.com/gotwarlost/istanbul
>
>
> -- Daniel R.  [http://danielr.neophi.com/]
>
>
> On Sat, Mar 23, 2013 at 10:50 PM, Gregg Caines  wrote:
>
>> Hey all... I just published a rudimentary package for exporting node.js
>> code coverage data to http://coveralls.io/ which lets you add a nifty
>> code coverage button to your package's README, as seen here:
>> https://github.com/cainus/urlgrey#readme and gives you nice reporting
>> like https://coveralls.io/r/cainus/urlgrey .
>>
>> Here's the package: https://npmjs.org/package/coveralls
>>
>> You need mocha's jscov reporter (or anything that reports in the same
>> format), so you probably need tests in mocha to use this.  It's certainly
>> possible that other coverage reporters use the same format though.  You
>> also need a build running in travis-ci to use this.
>>
>> If you have a different coverage report format, let me know about it, and
>> I can probably add it.
>>
>> Any questions/comments/etc are welcome.
>>
>> G
>>
>> --
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nodejs@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>  --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/npo8HaNI5Wk/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] ANN: coveralls.io coverage reporting for node.js

[Gregg Caines]

I'm pretty sure that istanbul's lcov output should work now, in the latest
version of coveralls.  Can you let me know if you try it and see any
problems?

Thanks!

G


On Sun, Mar 24, 2013 at 10:56 AM, Daniel Rinehart wrote:

> Will this with work with the reports generated by Istanbul?
>
> https://github.com/gotwarlost/istanbul
>
>
> -- Daniel R.  [http://danielr.neophi.com/]
>
>
> On Sat, Mar 23, 2013 at 10:50 PM, Gregg Caines  wrote:
>
>> Hey all... I just published a rudimentary package for exporting node.js
>> code coverage data to http://coveralls.io/ which lets you add a nifty
>> code coverage button to your package's README, as seen here:
>> https://github.com/cainus/urlgrey#readme and gives you nice reporting
>> like https://coveralls.io/r/cainus/urlgrey .
>>
>> Here's the package: https://npmjs.org/package/coveralls
>>
>> You need mocha's jscov reporter (or anything that reports in the same
>> format), so you probably need tests in mocha to use this.  It's certainly
>> possible that other coverage reporters use the same format though.  You
>> also need a build running in travis-ci to use this.
>>
>> If you have a different coverage report format, let me know about it, and
>> I can probably add it.
>>
>> Any questions/comments/etc are welcome.
>>
>> G
>>
>> --
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nodejs@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>  --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/npo8HaNI5Wk/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] Re: Getting Text Summary, Image or Video from link

[Gregg Caines]

It's the open graph protocol: http://ogp.me/ 

I don't have experience with these packages, but they might help:
https://npmjs.org/package/ogp
https://npmjs.org/package/og-meta-extract

G



On Saturday, April 13, 2013 3:21:01 AM UTC-7, nikkubhai wrote:
>
> In facebook ,twitter and google+ , when you paste a link : you get a 
> summary is its a blog, a picture if its a picture and a video if its 
> youtube linkHow do they do it?
>
> Is there a nodejs module for it?
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] ANN: Prozess, a node.js lib for the kafka distributed messaging queue

[Gregg Caines]

https://github.com/cainus/Prozess  /  https://npmjs.org/package/prozess

There are lots of TODOs still on this, but we've got a Kafka lib working in 
node.js that handles Kafka 0.7 messages.  We're not using it in production 
yet, but it's on its way and we'll be fleshing out more features as it goes.

Kafka is a distributed pub/sub style message queue released by linkedin.

This is useful if you want to do some really heavy-duty message-queuing or 
pubsub.

Feedback or contributions are appreciated of course!


-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

Re: [nodejs] ANN: Prozess, a node.js lib for the kafka distributed messaging queue

[Gregg Caines]

It's the only one AFAIK that can read / write 0.7+ messages.  Ideally
we would've forked a different lib, but didn't find them to be under
very active development, or using normal node idioms.

-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

[nodejs] Re: ANN: Pure Javascript ZooKeeper Client

[Gregg Caines]

There are a bunch of these ( https://npmjs.org/search?q=zookeeper ).  How 
is yours different?  I'm asking because I'm in the market for one.

Thanks!

G



On Wednesday, May 1, 2013 10:43:31 AM UTC-7, Alex Guan wrote:
>
> Hi All,
>
> I just published a pure Javascript ZooKeeper client to npm:
>
> https://npmjs.org/package/node-zookeeper-client
> https://github.com/alexguan/node-zookeeper-client
>
> This module is designed to resemble the ZooKeeper Java client API but with 
> tweaks to follow the convention of Node.js modules. Developers that are 
> familiar with the ZooKeeper Java client would be able to pick it up quickly.
>
> This module has been tested to work with ZooKeeper version 3.4.*.
>
> Suggestions/comments are appreciated!
>
> Best,
> Alex
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] api in npm for getting the number or list of dependent packages?

[Gregg Caines]

Hey all, does anyone know if npm has an API (or couchdb view) for getting 
the list (or just the count) of dependent packages of a given package ?  I 
suspect it does since npmjs.org has the info.

I know there's an npm mailing list, but it mysteriously won't let me post.  
Thanks!

G

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] api in npm for getting the number or list of dependent packages?

[Gregg Caines]

I'm actually looking for the opposite... the dependants, not the
dependencies.  For a given package x, it's the list of all the packages
that depend on x.

G


On Sat, May 4, 2013 at 1:33 PM, Martin Cooper  wrote:

> The simplest way is to grab the package metadata from the registry with
> this URL:
>
> http://registry.npmjs.org//latest
>
> That will get you a JSON version of the same object you see with 'npm view
> ', at which point you can just pluck out the 'dependencies' object.
>
> --
> Martin Cooper
>
>
>
>
> On Sat, May 4, 2013 at 12:52 PM, Gregg Caines  wrote:
>
>> Hey all, does anyone know if npm has an API (or couchdb view) for getting
>> the list (or just the count) of dependent packages of a given package ?  I
>> suspect it does since npmjs.org has the info.
>>
>> I know there's an npm mailing list, but it mysteriously won't let me
>> post.  Thanks!
>>
>> G
>>
>> --
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nodejs@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>>
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>  --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/_H8sH6qWozs/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] Re: Having trouble replicating with npm repo (couchdb) - anyone tried it lately and/or seen this error?

[Gregg Caines]

I ran into the same issue, and this is going to sound crazy, but I ended up 
writing my own replication (which is amazingly easy for anyone who's comfy 
with making http requests programmatically; couch spits out a page-able 
list of all changes in json format at 
http://isaacs.iriscouch.com/registry/_changes).  It might also be worth 
trying https://npmjs.org/package/replicate .

I can't remember what the root cause of your problem is, but I think it's 
something to do with different couch versions and large attachments or 
something.

G

On Thursday, May 16, 2013 9:22:21 PM UTC-7, andy wrote:
>
> Kevin,
>
> Unfortunately, no. We tried a few of the tips mentioned here on the 
> CouchDB list (
> http://mail-archives.apache.org/mod_mbox/couchdb-user/201305.mbox/%3cCAL+Y1nuP=wBwXn8eM7MBzZg2v3nKChTEVmo=bntwhf5ukfi...@mail.gmail.com%3e)
>  
> - for example, we didn't have an admin user set up, so we tried that and it 
> looked like it would work...but we restarted replication with a new DB (we 
> want this to be a repeatable process) and it failed after only 500 or so 
> documents. We were still trying Couch 1.3 so we're gonna drop down to 1.2.1 
> and see how that goes.
>
> So, no real idea what is wrong. If anyone has more tips on replicating 
> with the public npm repo, or maybe wants to zip up their .couch file and 
> put it on bittorrent, I'm all ears, haha. 
>
> I'm really hoping StrongLoops additions to NPM work out well (see 
> http://blog.strongloop.com/whats-new-in-strong-loop-node-beta-3-private-repositories/)
>  
> and someone will create an 'enterprise' repo a la Nexus/Artifactory. 
>
> andy
>
> On Thu, May 16, 2013 at 6:50 PM, Kevin Sawicki 
> 
> > wrote:
>
>> Hi Andy,
>>
>> I'm also seeing the exact same issue trying to replicate 
>> isaacs.iriscouch.com to another iriscouch.com database, it gets stuck at 
>> 17,286 documents (16gb) and those errors start to appear in the log.
>>
>> Have you found any more details about this issue?
>>
>> Sincerely,
>> Kevin
>>
>>
>> On Monday, May 13, 2013 8:16:51 PM UTC-7, andy wrote:
>>>
>>> Based on the awesome feedback I got from https://groups.google.**
>>> com/d/msg/nodejs/sX4mbsRPwls/**WtDDE-To2o4J,
>>>  
>>> we tried replicating the npm repo so we could use it in an offline 
>>> environment.
>>>
>>> We're essentially following the instructions at http://clock.co.uk/tech-
>>> **blogs/how-to-create-a-private-**npmjs-repositorybut
>>>  replication fails after syncing about 17k documents.
>>>
>>> We've tried reinstalling couch (found one issue that suggested using a 
>>> patched version of SpiderMonkey) but the same thing keeps happening, even 
>>> after restarting replication several times.
>>>
>>> Here's our setup:
>>>
>>> CentOS 6.4
>>> CouchDB 1.3
>>> SpiderMonkey 1.8.5-7 
>>>
>>> Replication works fine for over 17,000 documents, then we see this error 
>>> and can't get past it:
>>>
>>> [Sat, 11 May 2013 00:55:39 GMT] [error] [<0.12970.4>] Replicator: 
>>> couldn't write document `bufferhelper`, revision `19-**
>>> d339684ee7f5eaf4cc18d84da75383**2d`, to target database `registry`. 
>>> Error: `unauthorized`, reason: `Please log in before writing to the db`.
>>>
>>> Any ideas?
>>>
>>> Thanks,
>>>
>>> Andy
>>>
>>  -- 
>> -- 
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines: 
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nod...@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+un...@googlegroups.com 
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>  
>> --- 
>> You received this message because you are subscribed to the Google Groups 
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to nodejs+un...@googlegroups.com .
>> For more options, visit https://groups.google.com/groups/opt_out.
>>  
>>  
>>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] http parsing in node?

[Gregg Caines]

Does anyone know if the core lib exposes the http parser somehow?  Or if 
there's a package on npm worth using for http parsing?  I saw a few 
packages but they look mostly abandoned.

I'm looking for something smart enough to handle real (and trickier) 
requests, like chunked encoding.

Thanks!

G

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] http parsing in node?

[Gregg Caines]

Good start.  Is there any reason you don't try for node-style request and
response streams as an interface?

Also, this just looks like the server side of things (decoding requests /
encoding responses).  Is that right?  Any plans for the client side?

Thanks,

G


On Mon, Jun 10, 2013 at 7:04 PM, Tim Caswell  wrote:

> The core http parser is exposed via the private APIs just look at the
> source of http.js in node's source.  I will warn however that it does seem
> to change between versions as the node core team tries to make every
> release faster than the last.
>
> I've written a couple pure-js http parsers, they are not complete yet
> though.  If you use my latest one, I'll keep on working on it since I need
> it for my js-git project.
>
> https://github.com/creationix/http-codec
>
>
> On Mon, Jun 10, 2013 at 6:09 PM, Daniel Rinehart wrote:
>
>> One of the common recommended packages is:
>> https://github.com/substack/node-parsley
>>
>>
>> -- Daniel R.  [http://danielr.neophi.com/]
>>
>>
>> On Mon, Jun 10, 2013 at 6:07 PM, Gregg Caines  wrote:
>>
>>> Does anyone know if the core lib exposes the http parser somehow?  Or if
>>> there's a package on npm worth using for http parsing?  I saw a few
>>> packages but they look mostly abandoned.
>>>
>>> I'm looking for something smart enough to handle real (and trickier)
>>> requests, like chunked encoding.
>>>
>>> Thanks!
>>>
>>> G
>>>
>>> --
>>> --
>>> Job Board: http://jobs.nodejs.org/
>>> Posting guidelines:
>>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To post to this group, send email to nodejs@googlegroups.com
>>> To unsubscribe from this group, send email to
>>> nodejs+unsubscr...@googlegroups.com
>>> For more options, visit this group at
>>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>>
>>> ---
>>> You received this message because you are subscribed to the Google
>>> Groups "nodejs" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to nodejs+unsubscr...@googlegroups.com.
>>> For more options, visit https://groups.google.com/groups/opt_out.
>>>
>>>
>>>
>>
>>  --
>> --
>> Job Board: http://jobs.nodejs.org/
>> Posting guidelines:
>> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
>> You received this message because you are subscribed to the Google
>> Groups "nodejs" group.
>> To post to this group, send email to nodejs@googlegroups.com
>> To unsubscribe from this group, send email to
>> nodejs+unsubscr...@googlegroups.com
>> For more options, visit this group at
>> http://groups.google.com/group/nodejs?hl=en?hl=en
>>
>> ---
>> You received this message because you are subscribed to the Google Groups
>> "nodejs" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to nodejs+unsubscr...@googlegroups.com.
>> For more options, visit https://groups.google.com/groups/opt_out.
>>
>>
>>
>
>  --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/riob-diPHSw/unsubscribe?hl=en.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] Stream Reliability

[Gregg Caines]

Hey all,

>From my understanding, the stream abstraction doesn't currently concern 
itself with (big-R) Reliability.  Is that correct?

For example, what if I wanted to send a text file over TCP (maybe with a 
capitalization transform in between for good measure), where the entire 
script could resume properly from being interrupted and restarted?

I'd need to maintain the byte offset of the last file-read somewhere of 
course that could survive the restart, but it seems like the internal 
'drain' mechanism can't really guarantee that even if I've successfully 
read up to that byte, it's been successfully written to TCP.  It could have 
been in motion through the streams and their buffers (even with 
highWaterMark = 1) at the time that script was stopped.  

I could imagine an implementation where no stream can emit a 'drain' event 
until all of its down-streams have emitted 'drain' events, but otherwise it 
seems that the in-memory buffers sort of make that impossible.  

I'm just asking because I'm trying to determine if streams are the right 
abstraction for some reliable messaging work that I'm doing.  It would be 
nice if the answer was yes, but it seems like no.  

Can anyone throw me a clue?

Thanks,

G

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] Re: Browserable REST API framework?

[Gregg Caines]

I wrote an api framework that has something like that: 
 http://percolatorjs.com .  This blog post explains the browser part a 
little bit:  
http://caines.ca/blog/programming/so-i-wrote-a-json-api-framework-and-the-framework-was-the-least-interesting-part/
 . 
 

I personally never want to use a regular web API again.

G



On Friday, August 9, 2013 3:00:59 PM UTC-7, Victor Hooi wrote:
>
> Hi,
>
> I'm currently using the Django Rest Framework (
> http://django-rest-framework.org/), which is, as the name suggests, a 
> REST framework for Django.
>
> It has a very neat browserable API - you can see a demo here:
>
> http://restframework.herokuapp.com/
>
> Basically, if you go to the REST endpoints with a browser, it offers a 
> nice HTML interface, and if it's a writeable endpoint, you can also use a 
> webform to POST/PUT etc.
>
> It's quite nice for discovery, and also great for testing/debugging.
>
> I'm just curious if anybody knows of something similar or equivalent in 
> Node.js?
>
> Cheers,
> Victor
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] Re: how to create a singleton

[Gregg Caines]

The singleton pattern is actually unnecessary in most languages outside of 
java, including javascript.  You should be wary of any javascript book that 
tries to teach you singletons at all.  Many of those gang of four patterns 
simply don't translate outside of java and c++ (eg if you want to implement 
command, strategy, or factory patterns, you should really first check out 
javascript's first class functions.).

So how do you achieve the same effect in javascript?  In the browser, you 
have globals.  If you want just one instance of a thing, create it, and set 
it to a global variable.  You can use that global variable everywhere.  (If 
you're thinking "but global variables are bad!", I mostly agree.  This is 
one of the reasons that the singleton itself is actually considered an 
anti-pattern by many.

In node, the module system is a global namespace that can maintain state, 
so that's the appropriate way to achieve the same effect.  I do this in 
node applications all the time.  For example, if I have a module for 
emailing with a send() method on it, I don't have it export a constructor; 
I have it export an object.  That object might maintain some state or it 
might not.  When the module is subsequently require()'d, it will have any 
state that it has accumulated since.

All of this is a bit hairy because we're talking about global state (which 
is the main impetus for singletons, when you get right down to it).  
Whatever you decide to do though, keep in mind that writing idiomatic code 
in any language means using the features of that language, and not 
translating java/C++ idioms into it.  It will be an extremely rare 
javascripter that will want to work on your code if you've got singletons 
in it.

G



On Monday, November 18, 2013 3:38:09 PM UTC-8, Reza Razavipour wrote:
>
> A newbie question...
>
> I have an app that connects and reuses the same connection to a remote 
> database and a connection to a remote soap server.
> I want to implement a singleton pattern for each of these. I am used to 
> doing that in C++ and Java but want to know what the standard 
> implementation for a Singleton pattern is in node.js.
>
> Any recommendations or references.
>
>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] Re: how to create a singleton

[Gregg Caines]

Yeah... that's what I'm saying :)

G


On Thu, Nov 21, 2013 at 7:33 AM, Kamil Leszczuk  wrote:

> > For example, if I have a module for emailing with a send() method on it,
> I don't have it export a constructor; I have it export an object.  That
> object might maintain some state or it might not.  When the module is
> subsequently require()'d, it will have any state that it has accumulated
> since.
>
> For.most of the time, that's unnecessary - multiple require() calls for
> the same module return same, cached module, so you can store state just by
> using local variables in that module.
>
> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/GmUto9AN47U/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] Re: how to create a singleton

[Gregg Caines]

I'm not 100% sure of your requirements, but this is what I assume you're
after:

-
file : soapClient.js

var memoizedClient;
var soap = require('soap');
var create = function (cb){
  if (memoizedClient){
return cb(null, memoizedClient);
  }
  var url = "http://172.31.19.39/MgmtServer.wsdl";;
  var endPoint = "https://172.31.19.39:9088";;
  var options = {};
  options.endpoint = endPoint;
  soap.createClient(url, options, function(err, result) {
if (err) {
  return cb(null, memoizedClient);   // pass errors properly! :)
}
setSecurity(new soap.BasicAuthSecurity('admin-priv', 'password'));
memoizedClient = result;
cb(null, result);
  });
exports.create = create;
exports.client = memoizedClient;  // I don't do this, but I think it's what
you're looking for

-
file: someClientCode.js

var client;
require('soapClient').create(function(err, client){
  // do stuff here
});
-

ALSO:
If you're somehow sure that you've already called create() previously, you
could just do:


var client = require('soapClient').client;
// do stuff here

I know that's a hell of a lot prettier, but I usually don't do that because
I like to not
have to worry about whether i've called create() or not.  I just call
create() everytime, and
suck up the fact that there's a callback.  In the cases that I'd use this
pattern like you are
using it, it's ONLY because I'd like to not have to incur the network cost
of the asynch call
everywhere I use it, and not because I want fewer indentations in my code.
I do this with
database connection pools as well, for example, because those shouldn't be
created over
and over for every use.  So in short, ALL of this hinges on my assumption
that
"soap.createClient" involves a network call that you don't want to repeat
everytime you
want to make a soap request.  Otherwise, there's no good reason to memoize
(cache state).

Often I put a setter on the memoizedClient as well because it works nicely
for dependency
injection in unit tests.

S... I think I have to admit in retrospect that this is indeed a
singleton.  At least this
way you can be more idiomatic than any general javascript singleton code
I've seen.

(I never ran any of this code, so it's probably broken in 14 ways)

Hope that helps,
G



On Thu, Nov 21, 2013 at 9:03 AM, Reza Razavipour
wrote:

> Ok so this is the way i have it constructed... SoapClient is my module and
> I want it to be used as a singleton, i.e. if it is initialized, it just
> returns the connection, or SoapClient.instance...
>
> var SoapClient = function() {
>
>> var soap = require('soap');
>> this.init = function (){
>> var url = "http://172.31.19.39/MgmtServer.wsdl";;
>> var endPoint = "https://172.31.19.39:9088";;
>> var options = {};
>> options.endpoint = endPoint;
>> soap.createClient(url, options, function(err, result) {
>> if (err) {
>> console.log('soap client create failed ' + err);
>> return;
>> }
>> console.log('init is called ready');
>> SoapClient.instance = result;
>> SoapClient.instance.setSecurity(new soap.BasicAuthSecurity(
>> 'admin-priv', 'password'));
>> });
>> };
>> };
>> SoapClient.getInstance  = function () {
>> if (SoapClient.instance) {
>> return SoapClient.instance;
>> }
>> new SoapClient().init();
>> return SoapClient.instance;
>> };
>> SoapClient.instance = null;
>> module.exports = SoapClient;
>
>
> Thoughts
>
> On Thursday, November 21, 2013 8:34:14 AM UTC-8, Reza Razavipour wrote:
>>
>> this I get the jist of the conversation but my Javascript skills, less
>> than 2 months, does not allow me to be able to code this up.
>> Also, as a starter in the new language the last thing I want to do is to
>> miss out on the language correct way of things and force lets say the Java
>> way of doing things in JS.
>>
>> Can you show me a skeleton of such function and a tiny consumer of that?
>> The reason I say that is the fact that all of the examples are all done in
>> the same JS file and not setup as module and when I try to change to a
>> module and a consumer, I run into syntax problems and 
>>
>> I do not understand the difference between a class exporting an object as
>> opposed to a constructor
>>
>>
>> Thank you so much for showing me the correct way
>> On Thursday, November 21, 2013 7:37:32 AM UTC-8, Kamil Leszczuk wrote:
>>>

Re: [nodejs] Re: how to create a singleton

[Gregg Caines]

Sure that's the common explanation, but don't fall for it: globality has
everything to do with it.  If the singleton didn't care about creating
global state, it could just have `this.created = true;` for its *only*
internal state and have thrown exceptions from the constructor for any
subsequent calls.  That's the simplest way to ensure that only one instance
gets constructed.

In the browser, where there's no module system to stop your global
variables from spraying everywhere, any singleton can just as safely be
replaced by a simple single global instance instantiated normally.  There's
absolutely no difference in control or globality whatsoever, and you get
better performance by not making completely unnecessary getInstance() calls
just to get your instance.

The singleton pattern is really nothing more than a hack to get globals
into java via the global class namespace.  Even in java, it's pretty common
to solve the same problems with IoC containers now instead of singletons,
because global state is generally considered A Bad Thing (unless used
extremely judiciously).

G






On Tue, Nov 26, 2013 at 9:50 PM, dhtml  wrote:

>
>
> On Thursday, November 21, 2013 7:13:48 AM UTC-8, Gregg Caines wrote:
>
>
>> So how do you achieve the same effect in javascript?  In the browser, you
>> have globals.
>>
>
> The global object has nothing to do with web browsers.
>
>
>> If you want just one instance of a thing, create it, and set it to a
>> global variable.  You can use that global variable everywhere.  (If you're
>> thinking "but global variables are bad!", I mostly agree.  This is one of
>> the reasons that the singleton itself is actually considered an
>> anti-pattern by many.
>>
>> Singleton is necessary when the program must have at most one instance of
> an object; where having two would be a problem. In javascript, it's
> well-used where the initialization of that one object needs some variables
> or configuration to initialize itself.
>
> --
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
> ---
> You received this message because you are subscribed to a topic in the
> Google Groups "nodejs" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/nodejs/GmUto9AN47U/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> nodejs+unsubscr...@googlegroups.com.
> For more options, visit https://groups.google.com/groups/opt_out.
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] State of the art for request isolation in http servers?

[Gregg Caines]

Hey all... I'm wondering if anyone can point me to the current 
best-practice for isolating requests in a web app.  In general I'm trying 
to solve the problem of keeping the server running despite bad code in a 
particular request.  Are domains my only shot?  Do they completely solve 
it?  Does anyone have existing code?

I'm on a somewhat large team, working on a somewhat large codebase, and 
until now I've been just logging restarts and combing logs for these types 
of errors, then fixing them (which I'll always do), but I'm starting to 
feel a bit silly with PHP having solved this 10 years ago.  ;)  When a bug 
does get through, it would be nice to not lose the whole server and the 
possible 10,000+ customer requests attached to it, while I scramble to fix 
it.

Thanks for any ideas or pointers!

G

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

[nodejs] Re: State of the art for request isolation in http servers?

[Gregg Caines]

Well even though all the responses so far would require some pretty 
non-standard solutions (and therefore major changes to our current app), I 
really do appreciate them.  We have logging, metrics and alerts on server 
restarts, so we know about and fix restarts as fast as possible I believe, 
but losing 10,000+ user requests at once (per server!  and we have dozens 
of servers running!) due to one bad api endpoint is just not worth the risk 
of running like this anymore.  I'm definitely forced to consider the 
weirder solutions if there isn't a standard one.

There have got to be others working on a standard yet somewhat large 
deployment that have similar concerns though.  How is everyone else 
managing this?   (And if your answer is "Be more careful", I'm going to 
assume you're not in the same situation.  Also: we've got a staging 
environment we test in first and nearly 100% test coverage  )

G 


On Tuesday, January 14, 2014 7:40:51 PM UTC-8, tjholowaychuk wrote:
>
> check out Koa http://koajs.com/ you won't get separate stacks like you do 
> with node-fibers but similar otherwise (built with generators)
>
> On Tuesday, 14 January 2014 12:28:52 UTC-8, Gregg Caines wrote:
>>
>> Hey all... I'm wondering if anyone can point me to the current 
>> best-practice for isolating requests in a web app.  In general I'm trying 
>> to solve the problem of keeping the server running despite bad code in a 
>> particular request.  Are domains my only shot?  Do they completely solve 
>> it?  Does anyone have existing code?
>>
>> I'm on a somewhat large team, working on a somewhat large codebase, and 
>> until now I've been just logging restarts and combing logs for these types 
>> of errors, then fixing them (which I'll always do), but I'm starting to 
>> feel a bit silly with PHP having solved this 10 years ago.  ;)  When a bug 
>> does get through, it would be nice to not lose the whole server and the 
>> possible 10,000+ customer requests attached to it, while I scramble to fix 
>> it.
>>
>> Thanks for any ideas or pointers!
>>
>> G
>>
>

-- 
-- 
Job Board: http://jobs.nodejs.org/
Posting guidelines: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en

--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
For more options, visit https://groups.google.com/groups/opt_out.

Re: [nodejs] Re: State of the art for request isolation in http servers?

[Gregg Caines]

 similar flavors of the 
> same strategy, but the trycatch API is dead simple and that might appeal to 
> you more than domains, which do have some complexity to them. Adam also has 
> a very different philosophy than the Node core team, where he wants to keep 
> Node processes running and actually recover from errors more often than 
> not, and trycatch reflects that philosophy.
>
> 4. In addition to the control flow alternatives others have mentioned, 
> some people like the way that promises compose for error-handling. I 
> personally find using streams with promises a little awkward, but if your 
> web service has any sort of pipeline (pull some data -> do something with 
> the data -> cache it in e.g. redis -> render a template -> shove it out the 
> response), promises might be a way to DRY up your error-handling in a way 
> that allows you to confine the consequences of exceptions. Also, if you use 
> Bluebird, you probably won't even pay that much of a performance penalty.
>
> Just trying to be extra-careful is probably not going to ever feel very 
> satisfying. There's a lot that can go wrong, and as much as the core tries 
> to be consistent about *either* throwing synchronously *or* emitting 
> 'error' events / passing Error objects to callbacks, there are a lot of 
> gotchas that only time, experience, and production crashes will make clear. 
> Domains are core's general solution to this problem, along with the 
> philosophy (that's been articulated here and elsewhere many times) that the 
> sane thing to do when your Node process encounters an error is to shut down 
> and restart the process.
>
> Another piece of this is to partition your services such that you don't 
> have the problem of 10,000 clients being at risk if one request crashes. 
> Think about scaling your app horizontally (using cluster or something 
> similar) to keep each process dealing with a smaller number of clients if 
> you can. PHP handles this better (in terms of your problem -- there are 
> always tradeoffs!) because each PHP script is being run in its own context 
> (which for all intents and purposes is a process -- if one PHP handler, it 
> has no effect on the others), which is just a fundamentally different model 
> from Node.
>
>  
> On Tue, Jan 14, 2014 at 9:43 PM, Gregg Caines 
> > wrote:
>
>> Well even though all the responses so far would require some pretty 
>> non-standard solutions (and therefore major changes to our current app), I 
>> really do appreciate them.  We have logging, metrics and alerts on server 
>> restarts, so we know about and fix restarts as fast as possible I believe, 
>> but losing 10,000+ user requests at once (per server!  and we have dozens 
>> of servers running!) due to one bad api endpoint is just not worth the risk 
>> of running like this anymore.  I'm definitely forced to consider the 
>> weirder solutions if there isn't a standard one.
>>
>> There have got to be others working on a standard yet somewhat large 
>> deployment that have similar concerns though.  How is everyone else 
>> managing this?   (And if your answer is "Be more careful", I'm going to 
>> assume you're not in the same situation.  Also: we've got a staging 
>> environment we test in first and nearly 100% test coverage  )
>>
>> G 
>>
>>
>> On Tuesday, January 14, 2014 7:40:51 PM UTC-8, tjholowaychuk wrote:
>>>
>>> check out Koa http://koajs.com/ you won't get separate stacks like you 
>>> do with node-fibers but similar otherwise (built with generators)
>>>
>>> On Tuesday, 14 January 2014 12:28:52 UTC-8, Gregg Caines wrote:
>>>>
>>>> Hey all... I'm wondering if anyone can point me to the current 
>>>> best-practice for isolating requests in a web app.  In general I'm trying 
>>>> to solve the problem of keeping the server running despite bad code in a 
>>>> particular request.  Are domains my only shot?  Do they completely solve 
>>>> it?  Does anyone have existing code?
>>>>
>>>> I'm on a somewhat large team, working on a somewhat large codebase, and 
>>>> until now I've been just logging restarts and combing logs for these types 
>>>> of errors, then fixing them (which I'll always do), but I'm starting to 
>>>> feel a bit silly with PHP having solved this 10 years ago.  ;)  When a bug 
>>>> does get through, it would be nice to not lose the whole server and the 
>>>> possible 10,000+ customer requests attached to it, while I scramble to fix 
>>>> 

[nodejs] where-used analysis?

[Gregg Caines]

Hey all... does anyone have a solution for detecting where/if a file is 
required in a large codebase?  Manually grepping for it gets old pretty 
quickly when there can be a difference in relative paths.  I can't find 
anything like this in npm so I was thinking I could write such a thing 
using https://github.com/substack/module-deps , but I thought I'd check 
here first.

Thanks!

G 

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/66a6e364-1ee7-4d3c-9d0d-f691d878897b%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.

[nodejs] event loop lag hunting

[Gregg Caines]

Hey all,

Does anyone have any experience hunting down causes of event-loop lag? 
 I've determined that our app is heavily affected by it (See the attached 
png where I graph api response time in red against event-loop lag in blue), 
but now I'm at a bit of a loss for tracking down the source(s).   If I 
understand the nature of the cause correctly, these will be spots in the 
code that are cpu-intensive without relegating back to the event loop.  Has 
anyone got an easy way to hunt the cases down? 

Thanks!

G 

-- 
Job board: http://jobs.nodejs.org/
New group rules: 
https://gist.github.com/othiym23/9886289#file-moderation-policy-md
Old group rules: 
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
--- 
You received this message because you are subscribed to the Google Groups 
"nodejs" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to nodejs+unsubscr...@googlegroups.com.
To post to this group, send email to nodejs@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/nodejs/00a2487f-dd92-41ea-9263-f44f5b17a6b0%40googlegroups.com.
For more options, visit https://groups.google.com/d/optout.