Hi Carlo,
Starting with 1.8.0, you can only have 16 outstanding messages to a
given client at a time, then mongrel2 closes the connection. You ought
to be able to connect to the control port and determine how much data
has gone to each client to rate limit your handler though. You can also
up the limit on the number of outstanding messages if you want, 16 is
completely arbitrary, but know that the data has to exist *somewhere*
until it goes out the socket, and if it's not in your handler's RAM then
it's in mongrel2's RAM.
-Jason
On 18:45 Thu 04 Oct , Carlo Pires wrote:
> When I try to send in chunks of 1024 I got the folowing error in
> mongrel2:
> Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/handler.c:126: errno: None)
> Error sending raw message to HTTP listener on FD 47, closing them.
> Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/handler.c:152: errno: None)
> Failed to deliver to connection 5 on socket 47
> Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/io.c:171: errno: Bad file
> descriptor) Error sending SSL data.
> Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/io.c:673: errno: None) Write
> error when sending all.
> Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/connection.c:925: errno:
> None) Error delivering to MSG listener on FD -1, closing them.
> I have SSL enabled. The code I'm using to send is (python):
> def http_chunked_response(body, code, status, headers):
> Â Â Â assert 'Content-Length' in headers, 'Chunked response must have
> Content-Length in headers'
> Â Â Â
> Â Â Â payload = {'code': code, 'status': status, 'body': ''}
> Â Â Â payload['headers'] = "\r\n".join('%s: %s' % (k,v) for k,v in
> headers.items())
> Â Â Â yield HTTP_FORMAT % payload
> Â Â Â
> Â Â Â for chunk_body in body:
> Â Â Â Â Â Â for chunk in chunks(cStringIO.StringIO(chunk_body),
> 1024):
> Â Â Â Â Â Â Â Â Â yield chunk
> In reply_http:
> for msg in http_chunked_response(body, code, status, headers):
> Â Â Â self.reply(req, msg)
> Am I missing something?
>
> 2012/10/4 Justin Karneges <[1][email protected]>
>
> On Thursday, October 04, 2012 09:09:26 PM Loic d'Anterroches wrote:
> > On 2012-10-04 19:33, Carlo Pires wrote:
> > > Hi,
> > >
> > > Mongrel2 is great in receiving large uploads. It just stream them
> to
> > > disk and notify the handler.
> > >
> > > But the inverse seems to be problematic. How can I stream a big
> file
> > > (>4G) to be served by mongrel2 without compromising the memory?
> >
> > In a way or another you will to kind of evaluate what size is the
> pipe
> > between mongrel2 and the client (the control port can help you), then
> > you send your file in many parts, each part in a single message. For
> > example, you could send ~4000 messages, each being 1MB in size. This
> > way, you never allocate a lot of memory.
>
> I'd like to see a credits-based flow control added to the protocol,
> so that
> Mongrel2 can throttle handlers, at least optionally. Since PUB
> sockets never
> block, IMO it is not very appropriate to use it for large streaming
> responses
> without some kind of out of band flow control.
> This is how I deal with the problem in Zurl.
> Justin
>
> --
> Â Carlo Pires
>
> References
>
> 1. mailto:[email protected]
