Re: [nodejs] Re: The future of node streams. We need to talk

2013-08-17 Thread Tim Caswell
I'm on vacation this week, but decided to see how the node.js list is going. First I want to reply to Eldar's concerns., Yes, I'm also frustrated with streams in node.js. But, I do *not* advocate we all stop using node streams in node modules. Isaac is right that node has reached a level of

[nodejs] Re: The future of node streams. We need to talk

2013-08-15 Thread greelgorke
the error handling is something domains should do. i haven't tried them yet, but generally i think it's a good idea to separate error handling from the business logic. I don't see a good point to let errors bubble up the pipe, you need to transform them in most cases anyway. but it would be

[nodejs] Re: The future of node streams. We need to talk

2013-08-15 Thread Bruno Jouhier
Isaac's message is clear. Simple streams won't make it into core. So the solution is probably in userland. Tim's module looks like a good starting point and if it gets popular it could become a de facto simple stream standard API. My comments on it: * Would be nice to also include the

[nodejs] Re: The future of node streams. We need to talk

2013-08-15 Thread Bruno Jouhier
pipe should have a continuation callback. a.pipe(b, function(err) { ... }); This is the simplest way to hook up code which will be executed after the pipe operation: no need to trap two events (one for success, one for error), a good old callback does the job. With a callback API, you don't

Re: [nodejs] Re: The future of node streams. We need to talk

2013-08-15 Thread Jake Verbaten
* Would be nice to also include the writable side in the spec, with a symmetrical API (a write / abort pair). For simplicity you can get away with a Writable stream to just be `function (simpleStream) { /* consume it */ }` Then you don't need a central `pipe()` or `pump()` implementation and

Re: [nodejs] Re: The future of node streams. We need to talk

2013-08-15 Thread Bruno Jouhier
On Thursday, August 15, 2013 7:43:30 PM UTC+2, Raynos wrote: * Would be nice to also include the writable side in the spec, with a symmetrical API (a write / abort pair). For simplicity you can get away with a Writable stream to just be `function (simpleStream) { /* consume it */ }`

Re: [nodejs] Re: The future of node streams. We need to talk

2013-08-15 Thread Jake Verbaten
Yes, but why not do things the natural way: This an imperative vs functional style mismatch. the natural way depends on your background ```js var fileStream = fs.read(/foo.txt) var result = fs.write(/bar.txt, fileStream) result(function (err) { // entire fileStream has been flushed to disk })

[nodejs] Re: The future of node streams. We need to talk

2013-08-14 Thread Floby
file.pipe(res) does clean the resources on error because node completely shuts down, doesn't it? On Tuesday, 13 August 2013 12:08:30 UTC+2, Eldar wrote: file.pipe(res) On response error (say client clicked close window button) the file will remain open. And vice versa on file error

[nodejs] Re: The future of node streams. We need to talk

2013-08-14 Thread tjholowaychuk
Yeah I just realized the other day that errors are not propagated either, that makes them even less enjoyable to work with. Streams as they are today are barely usable, and things like object mode just really don't belong in core, things with objects should be emitters not streams. I'd love to

[nodejs] Re: The future of node streams. We need to talk

2013-08-14 Thread Eldar
Node completely shutdowns because emitter.emit('error') triggers exception when there is no listeners for error event. The .pipe() listens for errors but doesn't change that behavior. So strictly speaking above server will shutdown on file error because of uncaught exception and leak on

[nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Jakeb Rosoman
Aren't simple-stream's still a bit problematic if you pass them around like values? Because multiple readers would still interfere with each other wouldn't they? On Tuesday, August 13, 2013 4:05:02 AM UTC+12, Eldar wrote: If you want to create some function which accepts a stream and can

[nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Floby
ok. I didn't understand the part about `.pipe()` doesn't cleanup any resources. Everything works fine until the streaming process completes successfully. Is there an example where that becomes obvious? On Monday, 12 August 2013 18:05:02 UTC+2, Eldar wrote: If you want to create some

[nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Eldar
file.pipe(res) On response error (say client clicked close window button) the file will remain open. And vice versa on file error response will hang. So the correct way to pipe is file.pipe(res) file.on('error', res.destroy.bind(res)) res.on('error', file.destroy.bind(res)) вторник, 13

[nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Eldar
Than don't pass them to multiple readers? вторник, 13 августа 2013 г., 8:23:25 UTC+4 пользователь Jakeb Rosoman написал: Aren't simple-stream's still a bit problematic if you pass them around like values? Because multiple readers would still interfere with each other wouldn't they? On

Re: [nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Nathan White
file.pipe(res) file.on('error', res.destroy.bind(res)) res.on('error', file.destroy.bind(res)) What is wrong with this? I don't see an issue with streams/pipe. You could argue that docs and tutorials need to be clearer about error handling. I would argue this is an education problem,

[nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Jakeb Rosoman
Sure but while we are talking about fixing stuff.. On Tuesday, August 13, 2013 10:16:01 PM UTC+12, Eldar wrote: Than don't pass them to multiple readers? вторник, 13 августа 2013 г., 8:23:25 UTC+4 пользователь Jakeb Rosoman написал: Aren't simple-stream's still a bit problematic if you

Re: [nodejs] Re: The future of node streams. We need to talk

2013-08-13 Thread Eldar
However, the end result is a higher level abstraction of streams that should live in user land not core. The end result is not a higher level abstraction just because you can convert simple-stream to 100% compatible core stream and vice versa. I would argue this is an education problem, not a