Thanks for trying this, Mastake. I've got HDFS working on my cluster
with tracing and LocalFileSpanReceiver. Did you try using HBase +
HDFS with LocalFileSpanReceiver? Be sure to use a build including
HTRACE-112 since LFSR was kind of busted prior to that.
I'm going to do a longer writeup about getting HDFS + HBase working
with other span receivers just as soon as I finish stomping a few more
bugs.
best,
Colin
On Tue, Feb 24, 2015 at 12:04 PM, Masatake Iwasaki
iwasak...@oss.nttdata.co.jp wrote:
Hi,
Thanks for trying this. I am sorry for late reply.
I tried this today
by hbase-1.0.1-SANPSHOT built with {{-Dhadoop-two.version=2.7.0-SNAPSHOT}}
in pseudo distributed cluster
but failed to get end-to-end trace.
I checked that
* tracing works for both of hbase and hdfs,
* hbase runs with 2.7.0-SNAPSHOT jar of hadoop.
When I did do put with tracing on,
I saw span named FSHLog.sync with annotations such as
syncing writer and writer synced.
The code for tracing in FSHLog worked at least.
I'm still looking into this.
If it turned out that tracing spans are not reached to
actual HDFS writer thread in HBase, I will file a JIRA.
# We need hadoop-2.6.0 or higher in order to trace HDFS.
# Building hbase from source with {{-Dhadoop-two.version=2.6.0}}
# is straight forward way to do this
# because the binary release of hbase-1.0.0 bundles hadoop-2.5.1 jars.
Masatake
On 2/11/15 08:56, Nick Dimiduk wrote:
Hi Joshua,
In theory there's nothing special for you to do. Just issue your query to
HBase with tracing enabled. The active span will go through HBase, down
into HDFS, and back again. You'll need both systems collecting spans into
the same place so that you can report on the complete trace tree.
I've not recently tested the end-to-end, but I believe it's all there. If
not, it's a bug -- this is an intended use case. Can you give it a try
and let us know how it goes?
FYI, 0.99.x are preview releases of HBase and not for production use. Just
so you know :)
-n
On Wednesday, February 11, 2015, Chunxu Tang chunxut...@gmail.com wrote:
Hi all,
Now I’m exploiting HTrace to trace request level data flows in HBase and
HDFS. I have successfully traced HBase and HDFS by using HTrace,
respectively.
After that, I combine HBase and HDFS together and I want to just send a
PUT/GET request to HBase, but to trace the whole data flow in both HBase
and HDFS. In my opinion, when I send a request such as Get to HBase, it
will at last try to read the blocks on HDFS, so I can construct a whole
data flow tracing through HBase and HDFS. While, the fact is that I can
only get tracing data of HBase, with no data of HDFS.
Could you give me any suggestions on how to trace the data flow in both
HBase and HDFS? Does anyone have similar experience? Do I need to modify
the source code? And maybe which part(s) should I touch? If I need to
modify the code, I will try to create a patch for that.
Thank you.
My Configurations:
Hadoop version: 2.6.0
HBase version: 0.99.2
HTrace version: htrace-master
OS: Ubuntu 12.04
Joshua
