[
https://issues.apache.org/jira/browse/COUCHDB-1583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13488134#comment-13488134
]
Adam Kocoloski commented on COUCHDB-1583:
-----------------------------------------
Thanks for opening this ticket Bob. Sorry for not firing off a topic branch
sooner. When we started using the new replicator internally at Cloudant we
noticed that long-running replications would ultimately consume a ton of memory
on the node mediating the replication and crash. I tracked it down to the
mailbox for the process consuming the _changes feed over HTTP, and from there
to a misuse of ibrowse's stream_next API. Admittedly the API is
super-confusing; it should have a big read warning box telling you that you may
receive multiple messages for each invocation of the function.
The patch is working well for us, and given that the commit landed directly on
master I'm going to go ahead and resolve this one.
> improve the flow control in ibrowse streams
> -------------------------------------------
>
> Key: COUCHDB-1583
> URL: https://issues.apache.org/jira/browse/COUCHDB-1583
> Project: CouchDB
> Issue Type: Improvement
> Components: Replication
> Reporter: Bob Dionne
> Assignee: Adam Kocoloski
> Fix For: 1.3
>
>
> The flow control in ibrowse's async response streams is tricky. We call
> stream_next to pull more data off the socket, but it seems that ibrowse
> will sometimes split that data into multiple messages. If we call
> stream_next for each message we process we end up with an overflowing
> mailbox.
> This commit[1] changes the consumer so that it clears out the mailbox before
> calling stream_next.
> [1] 8ccf696f833a0f0a453d
--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira
