Willem, thanks for taking this up.
Ralf On 25.07.14 10:30, Willem.Jiang [via Camel] wrote: > Hi, > > You just bring up an interesting topic of avoiding extra copy of > response input stream. > It could save us lots of the memory and time if the response is a big > and chunked message. > I just created a JIRA[1] to track this issue. We may create some sub > tasks as we need to work on five different http client implement at > the time. > > I prefer to avoid close the input stream in a Synchronization > callback. It could be OK in a proxy route, as the input stream can be > consumed when the exchange int the route is completed. But if the > input stream is used in the other route, it’s hard for camel to know > if the stream is in use unless we have a reference count for it. > > [1]https://issues.apache.org/jira/browse/CAMEL-7638 > > -- > Willem Jiang > > Red Hat, Inc. > Web: http://www.redhat.com > Blog: http://willemjiang.blogspot.com (English) > http://jnn.iteye.com (Chinese) > Twitter: willemjiang > Weibo: 姜宁willem > > > > On July 25, 2014 at 2:36:14 PM, rsteppac2 ([hidden email] > </user/SendEmail.jtp?type=node&node=5754438&i=0>) wrote: > > > I am trying to implement a memory efficient http reverse proxy that is > > working on streams only to handle large HTTP requests and responses. > This > > should be as easy as described here: > > > > > http://camel.apache.org/how-to-use-camel-as-a-http-proxy-between-a-client-and-server.html > > > > > > The Jetty consumer places the socket input stream with the http POST > body > > into the exchange and I can hook it up with a producer to forward the > > request. No problem here. > > > > The problem is the response. All http producers that I am aware of > (Jetty, > > http4, netty-http) read the response stream into a byte-array,wrap > it in an > > InputStream and place that in the exchange body. Thus a large http > response > > blows my proxy out of the water. None of the producers seem to offer an > > option to make them put the original socket input stream into the > exchange, > > just as the Jetty consumer does. > > > > http4: Everything happens in > > org.apache.camel.component.http4.HttpProducer::process() -> > > populateResponse(..) -> extractResponseBody(..) -> > > doExtractResponseBodyAsStream(); here the original stream is copied > into an > > instance of CachedOutputStream, backed by a byte-array. > > > > Jetty: org.eclipse.jetty.client.AsyncHttpConnection::handle() -> > > org.eclipse.jetty.http.HttpParser::parseNext() will fill a > byte-array in > > org.eclipse.jetty.client.ContentExchange which is a CachedExchange > which is > > a HttpExchange. > > > > netty-http: Builds a pipeline that assembles the HttpResponse > content as a > > composite ChannelBuffer. The wrapped channel buffers make up the > complete > > response stream. > > > > Is there a way to configure any of the clients to place the socket > input > > stream into the exchange body? > > > > This thread describes the same problem and proposes a patch, which I > think > > has not made it into the Camel code base as of yet: > > > http://camel.465427.n5.nabble.com/Chunking-issue-with-http-producer-td5735075.html > > > > > > The question is how to close the socket stream of the client. If I > should > > have to replicate the suggested patch, would a Synchronization > call-back > > registered with the exchange via > Exchange::addOnCompletion(Synchronization) > > be the correct place to do it? I have tried with a FileInputStream > and that > > seems to work. I.e. the call-back is triggered after the stream > contents > > have been written out to the socket. > > > > > > Thanks! > > Ralf > > > > > > > > -- > > View this message in context: > http://camel.465427.n5.nabble.com/Stream-only-reverse-proxy-with-minimal-memory-footprint-tp5754424.html > > > > Sent from the Camel - Users mailing list archive at Nabble.com. > > > > > > ------------------------------------------------------------------------ > If you reply to this email, your message will be added to the > discussion below: > http://camel.465427.n5.nabble.com/Stream-only-reverse-proxy-with-minimal-memory-footprint-tp5754424p5754438.html > > > To unsubscribe from Stream-only reverse proxy with minimal memory > footprint, click here > <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=unsubscribe_by_code&node=5754424&code=cmFsZkBzdGVwcGFjaGVyLm5hbWV8NTc1NDQyNHwtMjc3MjkzMDM=>. > NAML > <http://camel.465427.n5.nabble.com/template/NamlServlet.jtp?macro=macro_viewer&id=instant_html%21nabble%3Aemail.naml&base=nabble.naml.namespaces.BasicNamespace-nabble.view.web.template.NabbleNamespace-nabble.view.web.template.NodeNamespace&breadcrumbs=notify_subscribers%21nabble%3Aemail.naml-instant_emails%21nabble%3Aemail.naml-send_instant_email%21nabble%3Aemail.naml> > > -- View this message in context: http://camel.465427.n5.nabble.com/Stream-only-reverse-proxy-with-minimal-memory-footprint-tp5754424p5754440.html Sent from the Camel - Users mailing list archive at Nabble.com.
