Hyperic is a great monitoring suite as well, as it auto discovers all sorts of daemons and lets you monitor them without much work at all. It's great.
A plugin to collect information from Hadoop itself also exists: < http://github.com/hyperic/hq-hadoop/tree/master> Alex On Fri, Jun 5, 2009 at 9:37 AM, Brian Bockelman <[email protected]>wrote: > Hey Anthony, > > I'd add you can look at our Ganglia stats: > > http://rcf.unl.edu/ganglia/?c=red-workers > > > http://rcf.unl.edu/ganglia/?c=red-workers&h=node123&m=load_one&r=hour&s=descending&hc=4 > > The second one is the node-specific view, and you can see all the Hadoop > metrics integrated nicely with the system metrics. > > Brian > > > On Jun 5, 2009, at 11:33 AM, Matt Massie wrote: > > Anthony- >> >> The ganglia web site is at http://ganglia.info/ with documentation in a >> wiki at http://ganglia.wiki.sourceforge.net/. There is also a good wiki >> page at IBM as well >> http://www.ibm.com/developerworks/wikis/display/WikiPtype/ganglia . >> Ganglia packages are available for most distributions to help with >> installation so make sure to grep for ganglia with your favorite package >> manager (e.g. aptitude, yum, etc). Ganglia will give you more information >> about your cluster than just Hadoop metrics. You'll get CPU, load, memory, >> disk and network monitoring as well for free. >> >> You can see live demos of ganglia at http://ganglia.info/?page_id=69. >> >> Good luck. >> >> -Matt >> >> On Jun 5, 2009, at 7:10 AM, Brian Bockelman wrote: >> >> Hey Anthony, >>> >>> Look into hooking your Hadoop system into Ganglia; this produces about 20 >>> real-time statistics per node. >>> >>> Hadoop also does JMX, which hooks into more "enterprise"-y monitoring >>> systems. >>> >>> Brian >>> >>> On Jun 5, 2009, at 8:55 AM, Anthony McCulley wrote: >>> >>> Hey all, >>>> I'm currently tasked to come up with a web/flex-based >>>> visualization/monitoring system for a cloud system using hadoop as part >>>> of a >>>> university research project. I was wondering if I could elicit some >>>> feedback from all of you with regards to: >>>> >>>> >>>> - If you were an engineer of a cloud system running hadoop, what >>>> information would you be interested in capturing, viewing, monitoring, >>>> etc? >>>> - Is there any sort of real-time stats or monitoring currently available >>>> for hadoop? if so, is in a web-friendly format? >>>> >>>> Thanks in advance, >>>> >>>> - Anthony >>>> >>> >>> >
