[
https://issues.apache.org/jira/browse/HADOOP-15566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Siyao Meng updated HADOOP-15566:
--------------------------------
Attachment: HADOOP-15566.000.WIP.patch
Status: Patch Available (was: In Progress)
Posting this work-in-progress patch for proof-of-concept purpose.
Based on [Carlos's
implementation|https://github.com/carlosalberto/hadoop/commits/ot_initial_integration],
I created a shim layer to greatly reduce trace hook API changes in existing
code, and then switched tracer to Jaeger.
To test this:
1. Apply the patch on trunk and compile
{code}
mvn install -Pdist -DskipTests -e -Dmaven.javadoc.skip=true
-Denforcer.skip=true -DskipShade
{code}
2. Download Jaeger binary from https://www.jaegertracing.io/download/
3. Set up environment variables for Jaeger to work properly, and to let Jaeger
always send traces (for testing)
{code}
export JAEGER_AGENT_HOST=127.0.0.1
export JAEGER_AGENT_PORT=6831
export JAEGER_SAMPLER_TYPE=const
export JAEGER_SAMPLER_PARAM=1
# export JAEGER_REPORTER_LOG_SPANS=true # Uncomment this if you want to print
a debug log every time a span is sent
{code}
4. Run the all-in-one binary (good for testing) in the background
{code}
./jaeger-1.12.0-darwin-amd64/jaeger-all-in-one --log-level debug
{code}
5. Launch NN and DN, locally or remotely, but be sure to include the
environment variables in Step 3.
6. Run an HDFS shell command and you will see the trace in
http://127.0.0.1:16686/. For example:
{code}
hdfs dfs -ls /
{code}
This is still under development. The shape of the shim Tracer/Span/TraceScope
class isn't finalized. Comments are welcome!
> Support Opentracing
> -------------------
>
> Key: HADOOP-15566
> URL: https://issues.apache.org/jira/browse/HADOOP-15566
> Project: Hadoop Common
> Issue Type: New Feature
> Components: metrics, tracing
> Affects Versions: 3.1.0
> Reporter: Todd Lipcon
> Assignee: Siyao Meng
> Priority: Major
> Labels: security
> Attachments: HADOOP-15566.000.WIP.patch, Screen Shot 2018-06-29 at
> 11.59.16 AM.png, ss-trace-s3a.png
>
>
> The HTrace incubator project has voted to retire itself and won't be making
> further releases. The Hadoop project currently has various hooks with HTrace.
> It seems in some cases (eg HDFS-13702) these hooks have had measurable
> performance overhead. Given these two factors, I think we should consider
> removing the HTrace integration. If there is someone willing to do the work,
> replacing it with OpenTracing might be a better choice since there is an
> active community.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
---------------------------------------------------------------------
To unsubscribe, e-mail: common-issues-unsubscr...@hadoop.apache.org
For additional commands, e-mail: common-issues-h...@hadoop.apache.org
