[
https://issues.apache.org/jira/browse/HADOOP-4802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=12654992#action_12654992
]
Raghu Angadi commented on HADOOP-4802:
--------------------------------------
yes, other buffering improvements are more work and better suited in a new
jira. Btw, we don't need the check '{{buf != initialBuffer}}'.
Couple of advantages to current patch :
# we can clearly say this won't change performance for common case. Though I
agree that creating new stream (i.e. two more allocations) each time won't be
noticeable (except ay be more frequent partial G/C on benchmarks).
# Subclassing BAOS (or a new output stream) will be required for other
buffering improvements any way.
> RPC Server send buffer retains size of largest response ever sent
> ------------------------------------------------------------------
>
> Key: HADOOP-4802
> URL: https://issues.apache.org/jira/browse/HADOOP-4802
> Project: Hadoop Core
> Issue Type: Bug
> Components: ipc
> Affects Versions: 0.18.2, 0.19.0
> Reporter: stack
> Attachments: 4802-v2.patch, 4802.patch
>
>
> The stack-based ByteArrayOutputStream in Server.Hander is reset each time
> through the run loop. This will set the BAOS 'size' back to zero but the
> allocated backing buffer is unaltered. If during an Handlers' lifecycle, any
> particular RPC response was fat -- Megabytes, even -- the buffer expands
> during the write to accommodate the particular response but then never
> shrinks subsequently. If a hosting Server has had more than one 'fat
> payload' occurrence, the resultant occupied heap can provoke memory woes (See
> https://issues.apache.org/jira/browse/HBASE-900?focusedCommentId=12654009#action_12654009
> for an extreme example; occasional payloads of 20-50MB with 30 handlers
> robbed the heap of 700MB).
--
This message is automatically generated by JIRA.
-
You can reply to this email to add a comment to the issue online.
