[
https://issues.apache.org/jira/browse/HTRACE-308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15034617#comment-15034617
]
Colin Patrick McCabe commented on HTRACE-308:
---------------------------------------------
Hmm, the patch has those lines as:
{code}
// Maximum length of HRPC message body
-const MAX_HRPC_BODY_LENGTH = 64 * 1024 * 1024
+const MAX_HRPC_BODY_LENGTH = 32 * 1024 * 1024
{code}
Not sure why the MAX_HRPC_BODY_LENGTH was missing when you applied the patch...
weird. Maybe need to sync up since we've been getting a lot done in master
lately...
The GC improvement here is huge... definitely something we need for 4.1. This
patch makes us stable on 300 nodes :D
bq. +1 after fixing above if it an issue.
thx
> Deserialize WriteSpans requests incrementally rather than all at once to
> optimize GC
> ------------------------------------------------------------------------------------
>
> Key: HTRACE-308
> URL: https://issues.apache.org/jira/browse/HTRACE-308
> Project: HTrace
> Issue Type: Improvement
> Components: htraced
> Affects Versions: 4.0
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Fix For: 4.1
>
> Attachments: HTRACE-308.001.patch, HTRACE-308.002.patch,
> HTRACE-308.003.patch, HTRACE-308.004.patch
>
>
> We should deserialize WriteSpans requests incrementally rather than all at
> once. Currently, we can deserialize 63 MB of spans all at once, which
> immediately creates somewhere between 60k and 600k spans, depending on span
> size. This is hard on the garbage collector because it's a lot of
> allocations all at once, and because it allocates a very large array to hold
> it all.
> It would be better to deserialize spans one at a time and feed them into the
> datastore via the BatchIngestor. This will ensure that we don't have to
> allocate giant arrays of spans all at once. If the datastore lags behind the
> rate of span ingestion, this will avoid us needing to allocate a bunch of
> memory "up front" which can lead to further slowdowns due to GC.
> Also, we should reuse buffers for the RPC handlers, and use buffering while
> deserializing to avoid making lots of small reads from the socket.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
