Lewis, Thought I'd add my two cents as well :).
As Colin stated, we're working on presenting a graph for visualizing the parent/child relationships. We're going to continue refining the search page and are still looking for the right look and feel. A GSoC project would be cool. Adding new visualizations (such as a histogram or generate charts for spans by description, start time, end time, etc.) might work. If you're interested, feel free to share some ideas/thoughts on visualizations that would be useful to you. -Abe On Fri, Feb 27, 2015 at 1:31 PM, Lewis John Mcgibbney < [email protected]> wrote: > Thanks ll for the info here. > I am going to press forward with investigating HTrace for the Nutch 2.X > Search Stack. > Lewis > > > On Fri, Feb 27, 2015 at 11:58 AM, Colin P. McCabe <[email protected]> > wrote: > > > Hi Lewis, > > > > Good questions. I would say HTrace differs from TRACE logging (or > > other single-node metrics, JMX, audit logs, etc.) in that it pulls > > together information from across the cluster. This is something that > > is a major pain point when using a distributed system such as HDFS. > > Just to diagnose a slow write, you might have to match up logs from a > > client log and the logs of 3 different datanodes. The big idea behind > > htrace is two things: integrating those logging sources, and using > > sampling to instrument performance in production. The main thing > > htrace deals with is "spans" which are lengths of time. > > > > We're working on a web UI that will allow people to search for spans > > by time, duration, and name (among other things). It's not quite > > finished now (hoping to have something usable in HTrace 3.2.0 or maybe > > 3.3.0... but abe can comment more on that.) > > > > Here's an early screenshot (probably way out of date now): > > > > > https://issues.apache.org/jira/secure/attachment/12689757/Search%20page%20skeleton%20-%200.png > > > > There is also a plan to create a visualization of parent/child > > relationships on the web UI, by using the d3 library (which can draw > > graphs, and do many other things besides.) > > > > In the meantime, there's an option to product a graphviz file from a > > file containing span JSON. That way you can draw a graph of > > parent/child relationships with the "dot" tool, available on Linux. > > Uh... unfortunately it's broken right now... let me file a JIRA for > > that :P This is a very new feature, got added earlier this week. > > > > The web UI is a great place to get involved right now... there is a > > lot of work going on there and we've been adding new contributors. > > > > Colin > > > > On Thu, Feb 26, 2015 at 1:46 PM, Lewis John Mcgibbney > > <[email protected]> wrote: > > > Hi Nick, > > > > > > Grand. Thank you > > > > > > What is visualization looking like right now? It there currently a > > > mechanism for visualizing HTrace structures? > > > Is it worth considering posting something like this as a GSoC project > is > > > one does not currently exist? > > > Thanks > > > Lewis > > > > > > > > > On Thu, Feb 26, 2015 at 1:31 PM, Nick Dimiduk <[email protected]> > > wrote: > > > > > >> Hi Lewis, > > >> > > >> > > > > > > -- > *Lewis* >
