[
https://issues.apache.org/jira/browse/HTRACE-237?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14944543#comment-14944543
]
Colin Patrick McCabe commented on HTRACE-237:
---------------------------------------------
{{htrace-core4/src/main/java/org/apache/htrace/core/Span.java}}: sort the trace
info map by key name before outputting it, so that we generate the same JSON
each time from the same Span object. Formerly, we were iterating through a
hash map so the order was undefined. That makes unit tests harder because you
can't compare against a fixed string.
{{htrace-core4/src/test/java/org/apache/htrace/util/TestUtil.java}}: Add some
test utilities for generating random spans.
{{htrace-htraced/go/src/org/apache/htrace/common/rest.go}}: In htraced, I found
it confusing that there were two files named {{rest.go}}. Since
{{common/rest.go}} was really just some structure definitions for what was sent
via the REST API, it seems better to just put it with the other stuff in
{{common/rpc.go}}.
{{htrace-htraced/go/src/org/apache/htrace/client/client.go}}: Rather than
accepting input in this form:
{code}
span1(newline)
span2(newline)
span3(newline)
{code}
Accept input in this form:
{code}
{ Spans: [ span1, span2, span ] }
{code}
Basically, we now accept a "real JSON object" rather than a
whitespace-separated set of span JSON objects. This makes it easier to add
more fields to the JSON later. It also means that the msgpack structure can be
exactly the same as the JSON one, but just serialized a little differently.
{{htrace-htraced/pom.xml}}: Add msgpack dependency and shade it.
{{htrace-htraced/src/main/java/org/apache/htrace/impl/Conf.java}}: The Conf
object is about caching some of the configuration values retrieved from the
{{HTraceConfiguration}} object. It's just easier to put it all in one class
rather than scattering it around. The JSON stuff here is really just making a
nice toString method so we can log the full configuration on startup.
{{htrace-htraced/src/main/java/org/apache/htrace/impl/RateLimitedLogger.java}}:
A logger object that only logs at a configurable interval, to avoid flooding
the user's log with htrace info. This is a utility class used by the
SpanReceiver.
{{htrace-htraced/src/test/java/org/apache/htrace/impl/DataDir.java}}: this was
moved from {{htrace-htraced/src/test/java/org/apache/htrace/util/DataDir.java}}
and simplified a little bit.
{{htrace-htraced/src/test/java/org/apache/htrace/impl/HTracedProcess.java}}:
this was moved and not changed very much
{{htrace-htraced/src/main/java/org/apache/htrace/impl/HTracedSpanReceiver.java}}:
big ideas:
* Allow both msgpack or REST to be used, depending on configuration.
* Buffer a few megabytes of spans and use double buffering to send the buffer
without blocking incoming spans
* Avoid memory allocations; hang on to our buffers forever. Serialize spans to
the buffer immediately after receiving them so that they can be GCed quickly.
> Optimize htraced span receiver
> ------------------------------
>
> Key: HTRACE-237
> URL: https://issues.apache.org/jira/browse/HTRACE-237
> Project: HTrace
> Issue Type: Improvement
> Affects Versions: 4.1
> Reporter: Colin Patrick McCabe
> Assignee: Colin Patrick McCabe
> Attachments: HTRACE-237.001.patch, HTRACE-237.002.patch,
> HTRACE-237.003.patch, HTRACE-237.004.patch
>
>
> Optimize htraced span receiver so that it supports double buffering and both
> the msgpack and REST transports.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
