When I try to send in chunks of 1024 I got the folowing error in mongrel2:
Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/handler.c:126: errno: None)
Error sending raw message to HTTP listener on FD 47, closing them.
Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/handler.c:152: errno: None)
Failed to deliver to connection 5 on socket 47
Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/io.c:171: errno: Bad file
descriptor) Error sending SSL data.
Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/io.c:673: errno: None) Write
error when sending all.
Thu, 04 Oct 2012 21:32:09 GMT [ERROR] (src/connection.c:925: errno: None)
Error delivering to MSG listener on FD -1, closing them.
I have SSL enabled. The code I'm using to send is (python):
def http_chunked_response(body, code, status, headers):
assert 'Content-Length' in headers, 'Chunked response must have
Content-Length in headers'
payload = {'code': code, 'status': status, 'body': ''}
payload['headers'] = "\r\n".join('%s: %s' % (k,v) for k,v in
headers.items())
yield HTTP_FORMAT % payload
for chunk_body in body:
for chunk in chunks(cStringIO.StringIO(chunk_body), 1024):
yield chunk
In reply_http:
for msg in http_chunked_response(body, code, status, headers):
self.reply(req, msg)
Am I missing something?
2012/10/4 Justin Karneges <[email protected]>
> On Thursday, October 04, 2012 09:09:26 PM Loic d'Anterroches wrote:
> > On 2012-10-04 19:33, Carlo Pires wrote:
> > > Hi,
> > >
> > > Mongrel2 is great in receiving large uploads. It just stream them to
> > > disk and notify the handler.
> > >
> > > But the inverse seems to be problematic. How can I stream a big file
> > > (>4G) to be served by mongrel2 without compromising the memory?
> >
> > In a way or another you will to kind of evaluate what size is the pipe
> > between mongrel2 and the client (the control port can help you), then
> > you send your file in many parts, each part in a single message. For
> > example, you could send ~4000 messages, each being 1MB in size. This
> > way, you never allocate a lot of memory.
>
> I'd like to see a credits-based flow control added to the protocol, so that
> Mongrel2 can throttle handlers, at least optionally. Since PUB sockets
> never
> block, IMO it is not very appropriate to use it for large streaming
> responses
> without some kind of out of band flow control.
>
> This is how I deal with the problem in Zurl.
>
> Justin
>
--
Carlo Pires
