I'm using objectMode. Do I need to do anything differently? I sometimes get a 502 error during the transform process, which proves to me that the web server handling the HTTP request is taking too much that it can handle. So, is this a performance tuning task that should be done on the web server side of things? - Like increasing the web server response timeout?
If I increase the web server response timeout, will the client wait for it to finish? Thanks Zlatko On Wed, Apr 27, 2016 at 10:21 AM, Zlatko <[email protected]> wrote: > Depending on how you implemented the transform stream, the pipeline should > be balancing itself automatically. How did you implement the stream? Do you > use object mode? Did you try to do something with highWaterMark? > > > > On Tuesday, April 26, 2016 at 6:34:05 PM UTC+2, Kayode Odeyemi wrote: >> >> Hi, >> >> I'm streaming data using the following data flow: >> >> Source(Database) -> Transform (An HTTP request) -> Sink(Database) >> >> Obviously the HTTP request task is a blocking task. At the moment, it >> seems the Source is faster than the Transform process. >> >> How do I make the Transform process delay or better still work in tandem >> with the Source or much better make the source delay until the current >> transform task finishes? >> >> Appreciate your help. >> >> Thanks >> >> -- > Job board: http://jobs.nodejs.org/ > New group rules: > https://gist.github.com/othiym23/9886289#file-moderation-policy-md > Old group rules: > https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines > --- > You received this message because you are subscribed to the Google Groups > "nodejs" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To post to this group, send email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/nodejs/791f9e72-c9e6-458f-bad7-04496ffd9505%40googlegroups.com > <https://groups.google.com/d/msgid/nodejs/791f9e72-c9e6-458f-bad7-04496ffd9505%40googlegroups.com?utm_medium=email&utm_source=footer> > . > For more options, visit https://groups.google.com/d/optout. > -- Job board: http://jobs.nodejs.org/ New group rules: https://gist.github.com/othiym23/9886289#file-moderation-policy-md Old group rules: https://github.com/joyent/node/wiki/Mailing-List-Posting-Guidelines --- You received this message because you are subscribed to the Google Groups "nodejs" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/nodejs/CAPbrmCu1B3iFeXQNgKTX9WAQV6PTgEvhTdfXo6cGH%3D0t6ZPG7g%40mail.gmail.com. For more options, visit https://groups.google.com/d/optout.
