Hi Lewis, Good questions. I would say HTrace differs from TRACE logging (or other single-node metrics, JMX, audit logs, etc.) in that it pulls together information from across the cluster. This is something that is a major pain point when using a distributed system such as HDFS. Just to diagnose a slow write, you might have to match up logs from a client log and the logs of 3 different datanodes. The big idea behind htrace is two things: integrating those logging sources, and using sampling to instrument performance in production. The main thing htrace deals with is "spans" which are lengths of time.
We're working on a web UI that will allow people to search for spans by time, duration, and name (among other things). It's not quite finished now (hoping to have something usable in HTrace 3.2.0 or maybe 3.3.0... but abe can comment more on that.) Here's an early screenshot (probably way out of date now): https://issues.apache.org/jira/secure/attachment/12689757/Search%20page%20skeleton%20-%200.png There is also a plan to create a visualization of parent/child relationships on the web UI, by using the d3 library (which can draw graphs, and do many other things besides.) In the meantime, there's an option to product a graphviz file from a file containing span JSON. That way you can draw a graph of parent/child relationships with the "dot" tool, available on Linux. Uh... unfortunately it's broken right now... let me file a JIRA for that :P This is a very new feature, got added earlier this week. The web UI is a great place to get involved right now... there is a lot of work going on there and we've been adding new contributors. Colin On Thu, Feb 26, 2015 at 1:46 PM, Lewis John Mcgibbney <[email protected]> wrote: > Hi Nick, > > Grand. Thank you > > What is visualization looking like right now? It there currently a > mechanism for visualizing HTrace structures? > Is it worth considering posting something like this as a GSoC project is > one does not currently exist? > Thanks > Lewis > > > On Thu, Feb 26, 2015 at 1:31 PM, Nick Dimiduk <[email protected]> wrote: > >> Hi Lewis, >> >>
