Thanks for this Mike! It should be helpful. I've been planning on doing a blog post about getting htraced set up, but there are a few bugs right now that I'm trying to fix first.
Good writeup. One note is that I think most Linux distros newer than CentOS 6 will be able to do "yum install leveldb-devel" or similar (maybe even CentOS 7?) Also, brew is a nice option on MacOS if you are already using that. One other note is that we automatically install the "godep" program. So it should not be necessary to install that manually. The BUILDING.txt is wrong here (it says that godep needs to be installed by hand.) I have filed HTRACE-126 to correct this part of BUILDING.txt. > 1) Is REST the only interface for getting span info in and out of htraced? Yes. I wrote some code to do a protobuf interface, but I haven't posted it yet. I think we should stabilize the other stuff before going for the extra performance that a non-REST interface would give. > 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be > used? I didn't see it anywhere in the Java clients. Span Receivers are basically "sinks" where spans can go. Client code, such as Hadoop or HBase, doesn't need to care about which span receiver the spans are going to. But you will need to put the htrace-htraced jar on your CLASSPATH to use it. > 3) Is it expected that there will be one htraced process running. I am > thinking of it like the collector in > Zipkin, http://twitter.github.io/zipkin/Architecture.html. Is that correct? > Is there a similar architecture > diagram for HTrace to help me see the big picture? Happy to help make one > once I figure it out. Yes, that's right. There is currently one htraced per cluster. Maybe later if load becomes a problem we'll shard it. You are correct that it is similar to the collector in zipkin. I am going to make some architecture slides for my talk at ApacheCon US, I will post those as soon as they exist (I'll also do a short writeup on setting up) Thanks for checking out htrace! Colin On Sat, Feb 28, 2015 at 8:04 PM, Michael Wall <[email protected]> wrote: > Thanks for the replies. Not sure if this is useful or not, but here are > some notes on how I got htraced running. I missed the > htrace-core/src/go/BUILDING.txt file before, which mentioned these items > already. > > - Install Go > Download 1.3.1 from https://golang.org/dl/ > Unzip somewhere, like /local/opt/go > export GOROOT=/local/opt/go > export GOPATH="${GOROOT}/packages" > add $GOROOT/bin and $GOPATH/bin to $PATH > Seems like GOROOT may no longer be needed for newer versions of Go > > - Install godep > Run > go get github.com/tools/godep > Puts it in GOPATH > > - Install leveldb > Download 1.13.1 from > https://code.google.com/p/leveldb/downloads/detail?name=leveldb-1.13.0.tar.gz > extract, cd into leveldb directory and run make > as root from leveldb directory setup lib and include for compiler > -- Linux CentOS 6.5 > cp --preserve=links libleveldb.* /usr/lib64 > cp -r include/leveldb /usr/include/ > ldconfig > -- Mac 10.10 > xcode-select --install # to make sure /usr/local/include is included > cp -a libleveldb.* /usr/local/lib > cp -r include/leveldb /usr/local/include/ > > - Install maven > Already had 3.0.5 installed > > - Get HTrace > git clone http://git-wip-us.apache.org/repos/asf/incubator-htrace.git > mvn clean package > > I looked the code a little and still have some questions. > > 1) Is REST the only interface for getting span info in and out of htraced? > 2) How is HTracedRESTReciever in htrace-htraced/ used or how will it be > used? I didn't see it anywhere in the Java clients. > 3) Is it expected that there will be one htraced process running. I am > thinking of it like the collector in Zipkin, > http://twitter.github.io/zipkin/Architecture.html. Is that correct? Is > there a similar architecture diagram for HTrace to help me see the big > picture? Happy to help make one once I figure it out. > > Thanks in advance for the help. Once I understand more, I'll look at the > open Jira's if that is the best way to get involved. > > Mike > > > On Thu, Feb 19, 2015 at 9:40 PM, Colin McCabe <[email protected]> > wrote: > >> Hi Michael, >> >> It's great that you're looking at the project! >> >> The build requirements are in BUILDING.txt, which says "Requires go >> version 1.3.1 or 1.4." In practice, other versions of go will >> probably work, we just haven't tested them. If you are using any >> modern version of Linux, you should be able to use "apt-get install >> golang" or "zypper install golang", etc. etc. Please do let us know >> if there are any difficulties building, though... we want this to be >> very accessible. >> >> As Nick commented, you don't need golang to build the Java client >> library. You might find it helpful to use the "LocalFileSpanReceiver" >> which just writes trace spans to a local file. The go code is used in >> htraced, which is a daemon which aggregates the spans all in one >> place. >> >> I'm doing some testing now myself with HTrace+Hadoop+HBase and I'll >> try to send out my notes about how I set things up. It will probably >> take a week to do that though since there is other stuff on my plate. >> But stay tuned. >> >> re: compiling without golang. We are planning on moving the htraced >> stuff into the htace-htraced subproject rather than htrace-core, where >> it is now. Once that move happens you should be able to just compile >> the maven subproject you want. I'm not sure if we need a maven >> profile or not since basically the profile would be equivalent to just >> not compiling that subproject. >> >> best, >> Colin >> >> On Thu, Feb 19, 2015 at 12:26 PM, Michael Wall <[email protected]> wrote: >> > Hi, >> > >> > I am interested in learning more about HTrace and helping. Hopefully >> this >> > is the correct list, I didn't see a users alias. >> > >> > Specifically, I use Accumulo everyday and am investigating how HTrace can >> > supplement or replace Accumulo's built-in tracing. >> > >> > I just checked out the code and tried to build it. To my surprise, I >> need >> > to install Golang. What version should I be using? Is there somewhere I >> > can read about why this was chosen and which modules/functionality Go >> will >> > be used for? >> > >> > Thanks >> > >> > Mike Wall >>
