[
https://issues.apache.org/jira/browse/HTRACE-51?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14283362#comment-14283362
]
Colin Patrick McCabe commented on HTRACE-51:
--------------------------------------------
Great... this new implementation looks a lot more compact and maintainable than
the old one.
{code}
53 * <p>Create an instance by doing:
54 * <code>SpanReceiver receiver =
55 *
HTracedRESTReceiver.builder().hostname(HTRACED_HOSTNAME).port(HTRACED_PORT).build();</code>
56 * Then call <code>receiver.receiveSpan(Span);</code> to send spans to
htraced. This method
{code}
I think we should create {{HTracedRESTReceiver}} objects via the "normal
method"-- basically using {{SpanReceiverBuilder}}. This will avoid the need to
add special-case logic in every client for each different {{SpanReceiver}}
type. All that it requires is that you provide a constructor for
{{HTracedRESTReceiver}} that takes a single argument, an
{{HTraceConfiguration}}, and pulls the relevant configuration from there.
This makes configuration a lot easier since basically Hadoop / HBase / etc. can
just use their normal configuration mechanism and HTrace grabs the keys it
needs via an object implementing {{HTraceConfiguration}}. Basically the
intention is that the Hadoop key "htrace.hadoop.foo.bar" shows up as "foo.bar"
to HTrace. Hadoop {{Configuration}} keys that don't start with "htrace.hadoop"
don't show up at all to HTrace.
{code}
88 /**
89 * Simple queue to hold spans between periodic runs of the httpclient.
90 * This queue is unbounded but it gets serviced and elements drained
91 * on a period. The draining thread, even if it dies,
ScheduledExecutorService
92 * launches a new one (as I read it -- TODO: test).
93 */
94 private final ConcurrentLinkedQueue<Span> queue = new
ConcurrentLinkedQueue<Span>();
{code}
I'm a little concerned about an unbounded queue. I feel like we need to set a
bound, even if we don't expect that it will ever get met in practice.
Otherwise we might hit an out of memory error if some trace spans get created
in a tight loop. Something like
http://docs.oracle.com/javase/7/docs/api/java/util/concurrent/ArrayBlockingQueue.html
would be helpful here. It lets you set a capacity in the constructor. If you
try to "put" when it's full, it blocks-- otherwise, it won't. We can set a
pretty large upper bound, maybe even a million elements (8MB array). But let's
not be unbounded because that road leads to OOMs and frustration. Also arrays
are faster than linked lists B)
{{PostSpans#run}}: I think we ought to put an upper limit on the number of
spans we'll send in a single request. It looks like right now we'll just keep
adding spans to the request as long as there are spans to add. This might not
even ever end if spans are being generated faster than we can add them to the
request!
bq. To finish, need to add a means of start/stop of htraced. Once I have that,
will build unit tests for this client.
Yeah, absolutely. I think the best way to do it is just to have JUnit spawn an
{{htraced}} process. {{htraced}} starts up really quickly. You can pass
configuration keys to {{htraced}} on the command-line via
{{\-Dmy.key=my.value}} (or via generating a config file, but that might be more
work).
Long-term, I also assume that we'll want to move the go code to htrace-htraced/
so that the server and client can be in the same package. I think this is a
pretty easy move, but maybe it's easier to get in some of our outstanding
htraced patches before we do it, to avoid manually rebasing everything.
> htraced java REST client (a.k.a java SpanReceiver for htraced)
> --------------------------------------------------------------
>
> Key: HTRACE-51
> URL: https://issues.apache.org/jira/browse/HTRACE-51
> Project: HTrace
> Issue Type: New Feature
> Reporter: stack
> Assignee: stack
> Attachments: 51.txt, 51.txt
>
>
> Add a java REST client for htraced so we can build a receiver that writes
> htraced via REST from a java process.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
