Hi Chris, Thanks for sharing memex project, I will definitely take a look first.
Meanwhile, I am planning to start with Neo4j and Gephy. Will definitely keep you posted. Visualization of graph, especially big graphs (like most crawls) is challenging. Bin On Mon, May 2, 2016 at 1:27 PM, Mattmann, Chris A (3980) < [email protected]> wrote: > Bin I completely agree. > > My team built the following: > > 1. Memex Explorer (http://github.com/memex-explorer/memex-explorer) > but not actively developed anymore that used Bokeh.js and streaming > publishing from Nutch under development to publish events and visualize > crawls > > 2. We are using D3.js in my team to visualize the Nutch crawl graph. > Lots under development. > > Are you interested in collaborating? > > Cheers, > Chris > > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Chris Mattmann, Ph.D. > Chief Architect > Instrument Software and Science Data Systems Section (398) > NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA > Office: 168-519, Mailstop: 168-527 > Email: [email protected] > WWW: http://sunset.usc.edu/~mattmann/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > Director, Information Retrieval and Data Science Group (IRDS) > Adjunct Associate Professor, Computer Science Department > University of Southern California, Los Angeles, CA 90089 USA > WWW: http://irds.usc.edu/ > ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++ > > > > > > > > > > On 5/2/16, 9:26 AM, "Bin Wang" <[email protected]> wrote: > > >Hi there, > > > >Is there a state of the art visualization tool that is Nutch friendly? > > > >I am planning to get the crawldb information into a better format that can > >be digested by Neo4j or Gephi for analysis. However, I have read here > >< > http://grokbase.com/t/nutch/user/124fbmankh/how-to-do-detailed-postmortem-analysis-and-visualization-of-nutch-crawl-data > > > >and there <http://wiki.apache.org/nutch/bin/nutch%20webgraph> about the > >demand but I don't see any solid tutorial or documentation regarding the > >visualization. > > > >I don't think visualization is a necessity for Nutch but something out of > >the box will be interesting to have. (people love graphs) > > > >Bin >
