mark, just implement a stream shaped wrapper for your db client thing, (and
make sure you publish it to npm!)
On Sun, Oct 14, 2012 at 4:24 AM, Isaac Schlueter <i...@izs.me> wrote:
> Mark,
>
> The overall performance impact hasn't been fully established yet. I
> suspect that it'll be slight, but it's unclear whether it'll be an
> improvement (owing most likely to greater hidden class optimization
> from having all streams share more of the same code), or a regression
> (owing to the fact that there's just more stuff being done).
>
> The syscall/IO footprint is identical, though. Past experience in
> this area has shown that the total time spent in JS is usually a
> pretty small part of the overall latency unless the code is very hot
> or doing something very stupid. We'll see soon.
>
>
> On Sat, Oct 13, 2012 at 5:52 PM, Mark Hahn <m...@hahnca.com> wrote:
> > So using it in the backwards-compatible way doesn't cause any performance
> > loss? If so I can choose which to use in every situation.
> >
> >
> > On Sat, Oct 13, 2012 at 5:31 PM, Isaac Schlueter <i...@izs.me> wrote:
> >>
> >> Mark,
> >>
> >> Well... yes. If you want to siphon out the data as fast as possible,
> >> and it's not going to a writable stream interface of some sort, then
> >> you have to read() in a tight loop on every readable event. That's
> >> actually not much different than the 'data' event scheme.
> >>
> >> Note that if you attach a 'data' event handler, then it'll do this for
> >> you. The backwards-compatible API is exactly the one you're used to.
> >> The major difference is that, in 0.10, if you're using 'data' events,
> >> then pause and resume actually work in a non-surprising way (ie, you
> >> won't get 'data' events happening while it's in a paused state), and
> >> all streams in core will have the same set of events and methods
> >> (instead of each of them implementing 90% of the API in subtly
> >> different ways).
> >>
> >>
> >> On Sat, Oct 13, 2012 at 5:25 PM, Mark Hahn <m...@hahnca.com> wrote:
> >> > But pipe only works if the writes are to another stream. If they are
> to
> >> > a
> >> > db driver or something without pipe support then I have to do my own
> >> > reads.
> >> > Or am I missing something here?
> >> >
> >> >
> >> > On Sat, Oct 13, 2012 at 5:19 PM, Nathan Rajlich <
> nat...@tootallnate.net>
> >> > wrote:
> >> >>
> >> >> Mark, to pump at max rate you'd use .pipe().
> >> >>
> >> >> On Sat, Oct 13, 2012 at 5:16 PM, Mark Hahn <m...@hahnca.com> wrote:
> >> >> >> There is no 'data' event. There is a read() method, and a
> >> >> >> 'readable'
> >> >> >> event. You call read() until it returns null, and then wait for a
> >> >> >> readable
> >> >> >> event to tell you it's time to read() more.
> >> >> >
> >> >> > So, if we want to pump it at max rate we would run a tight loop to
> >> >> > read
> >> >> > and
> >> >> > write in the beginning and then on every readable event? It seems
> >> >> > like
> >> >> > more work and a lot messier compared to the old data event scheme.
> >> >> >
> >> >> >
> >> >> > On Sat, Oct 13, 2012 at 5:08 PM, Alexey Petrushin
> >> >> > <alexey.petrus...@gmail.com> wrote:
> >> >> >>
> >> >> >> Thanks for help, especially You Isaac for such a detailed answer.
> >> >> >>
> >> >> >> As far as I understand it's possible to wrap existing evented
> stream
> >> >> >> API
> >> >> >> into callback interface (with in-memory data buffers to handle
> >> >> >> mismatch
> >> >> >> between explicit/implicit control flow).
> >> >> >> But probably it won't worth it, it will be more easy to just use
> it
> >> >> >> as
> >> >> >> it's supposed to be used (with pipes) and wait untill those
> changes
> >> >> >> in
> >> >> >> 0.10.
> >> >> >> The new API seems to be very similar to what I asked for.
> >> >> >>
> >> >> >> P.S.
> >> >> >>
> >> >> >> As for the question and why do I need it - I'm working on
> >> >> >> application
> >> >> >> that
> >> >> >> uses custom streams and though that maybe I can cheat and simplify
> >> >> >> my
> >> >> >> work a
> >> >> >> little by not implementing complex evented interface :).
> >> >> >>
> >> >> >> I once used such abstraction for working with streams in ruby:
> >> >> >>
> >> >> >> to.write do |writer|
> >> >> >> from.read{|buff| writer.write buff}
> >> >> >> end
> >> >> >>
> >> >> >> Files are open and closed properly, buffer also have some default
> >> >> >> size,
> >> >> >> so
> >> >> >> the code is very simple to use (more details
> >> >> >> http://alexeypetrushin.github.com/vfs ).
> >> >> >> Basically by implementing just those two methods You get ability
> to
> >> >> >> stream
> >> >> >> from any stream into any stream (fs, s3, sftp, ...).
> >> >> >>
> >> >> >> I tried to do something similar with asynchronous streams.
> >> >> >>
> >> >> >> --
> >> >> >> Job Board: http://jobs.nodejs.org/
> >> >> >> Posting guidelines:
> >> >> >>
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> >> >> You received this message because you are subscribed to the Google
> >> >> >> Groups "nodejs" group.
> >> >> >> To post to this group, send email to nodejs@googlegroups.com
> >> >> >> To unsubscribe from this group, send email to
> >> >> >> nodejs+unsubscr...@googlegroups.com
> >> >> >> For more options, visit this group at
> >> >> >> http://groups.google.com/group/nodejs?hl=en?hl=en
> >> >> >
> >> >> >
> >> >> > --
> >> >> > Job Board: http://jobs.nodejs.org/
> >> >> > Posting guidelines:
> >> >> >
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> >> > You received this message because you are subscribed to the Google
> >> >> > Groups "nodejs" group.
> >> >> > To post to this group, send email to nodejs@googlegroups.com
> >> >> > To unsubscribe from this group, send email to
> >> >> > nodejs+unsubscr...@googlegroups.com
> >> >> > For more options, visit this group at
> >> >> > http://groups.google.com/group/nodejs?hl=en?hl=en
> >> >>
> >> >> --
> >> >> Job Board: http://jobs.nodejs.org/
> >> >> Posting guidelines:
> >> >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> >> You received this message because you are subscribed to the Google
> >> >> Groups "nodejs" group.
> >> >> To post to this group, send email to nodejs@googlegroups.com
> >> >> To unsubscribe from this group, send email to
> >> >> nodejs+unsubscr...@googlegroups.com
> >> >> For more options, visit this group at
> >> >> http://groups.google.com/group/nodejs?hl=en?hl=en
> >> >
> >> >
> >> > --
> >> > Job Board: http://jobs.nodejs.org/
> >> > Posting guidelines:
> >> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> > You received this message because you are subscribed to the Google
> >> > Groups "nodejs" group.
> >> > To post to this group, send email to nodejs@googlegroups.com
> >> > To unsubscribe from this group, send email to
> >> > nodejs+unsubscr...@googlegroups.com
> >> > For more options, visit this group at
> >> > http://groups.google.com/group/nodejs?hl=en?hl=en
> >>
> >> --
> >> Job Board: http://jobs.nodejs.org/
> >> Posting guidelines:
> >> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> >> You received this message because you are subscribed to the Google
> >> Groups "nodejs" group.
> >> To post to this group, send email to nodejs@googlegroups.com
> >> To unsubscribe from this group, send email to
> >> nodejs+unsubscr...@googlegroups.com
> >> For more options, visit this group at
> >> http://groups.google.com/group/nodejs?hl=en?hl=en
> >
> >
> > --
> > Job Board: http://jobs.nodejs.org/
> > Posting guidelines:
> > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> > You received this message because you are subscribed to the Google
> > Groups "nodejs" group.
> > To post to this group, send email to nodejs@googlegroups.com
> > To unsubscribe from this group, send email to
> > nodejs+unsubscr...@googlegroups.com
> > For more options, visit this group at
> > http://groups.google.com/group/nodejs?hl=en?hl=en
>
> --
> Job Board: http://jobs.nodejs.org/
> Posting guidelines:
> https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
> You received this message because you are subscribed to the Google
> Groups "nodejs" group.
> To post to this group, send email to nodejs@googlegroups.com
> To unsubscribe from this group, send email to
> nodejs+unsubscr...@googlegroups.com
> For more options, visit this group at
> http://groups.google.com/group/nodejs?hl=en?hl=en
>
--
Job Board: http://jobs.nodejs.org/
Posting guidelines:
https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines
You received this message because you are subscribed to the Google
Groups "nodejs" group.
To post to this group, send email to nodejs@googlegroups.com
To unsubscribe from this group, send email to
nodejs+unsubscr...@googlegroups.com
For more options, visit this group at
http://groups.google.com/group/nodejs?hl=en?hl=en
