+2 Awesome job, Jesse! On Tuesday, July 1, 2014, Eli Levine <[email protected]> wrote:
> +1 for only supporting Hadoop2 initially (great work, Jesse!). It's > important that Phoenix started implementing monitoring and metrics > features. An incremental approach is the way to go IMHO. > > Thanks, > > Eli > > > On Tue, Jul 1, 2014 at 11:13 AM, James Taylor <[email protected] > <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: > >> Seems like an excellent feature to have in 4.1 which I'm hoping we can >> do by end of the month. I'd rather the feature make it in and only >> support hadoop2 than have it not make the release. >> >> Any one else have thoughts on this? >> >> On Tue, Jul 1, 2014 at 8:01 PM, Jesse Yates <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> > That's certainly possible, though somewhat unsatisfying :) Its not >> terribly >> > difficult to the Hadoop1 code - I've modularized a majority of the code >> so >> > all that is really needed is replicating the reading of the Hadoop1 >> metrics2 >> > classes to something the phoenix writer understands (just like in the >> > Hadoop2 impl). >> > >> > Happy to push the code somewhere so people can take a look... or they >> just >> > wait a couple weeks :) >> > >> > ------------------- >> > Jesse Yates >> > @jesse_yates >> > jyates.github.com >> > >> > >> > On Tue, Jul 1, 2014 at 10:53 AM, James Taylor <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> >> > wrote: >> >> >> >> How about if we make it a hadoop2 only feature? >> >> >> >> >> >> On Tuesday, July 1, 2014, Jesse Yates <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> wrote: >> >>> >> >>> I was working on a patch to support using Cloudera's HTrace >> >>> (https://github.com/cloudera/htrace) library for phoenix queries. >> There was >> >>> a preliminary patch for the 2.X series, but it never got committed. >> Two >> >>> weeks ago I almost finished porting it to the 4.X line, but haven't >> finished >> >>> the piece that will support Hadoop1*. I can't seem to find the JIRA >> right >> >>> now though... >> >>> >> >>> I've gotten sidetracked on some internal work, so probably not going >> to >> >>> have time to wrap it up for a couple weeks. >> >>> >> >>> -J >> >>> >> >>> * It uses the Hadoop metrics2 library to collect the traces and then >> has >> >>> a default implementation (as a metrics sink) that writes them to a >> phoenix >> >>> table so it can be analyzed later. Because of the changes between >> Hadoop1 >> >>> and 2, there is some reflection funkiness that is necessary to >> support both, >> >>> but just haven't finished the hadoop1 side of it. >> >>> ------------------- >> >>> Jesse Yates >> >>> @jesse_yates >> >>> jyates.github.com >> >>> >> >>> >> >>> On Tue, Jul 1, 2014 at 9:29 AM, Stephen Sprague <[email protected] >> <javascript:_e(%7B%7D,'cvml','[email protected]');>> >> >>> wrote: >> >>>> >> >>>> following up to my own message i did see reference to something >> called >> >>>> "Hannibal" (https://github.com/sentric/hannibal) in the Phoenix >> doc. Looks >> >>>> like this helps with identifying imbalance within your cluster. >> >>>> >> >>>> I'll keep searching around though - a Phoenix log file looks like >> >>>> perhaps the best way to monitor phoenix queries at this point as i >> don't >> >>>> think there is any self-aware queries to ask of it. That's a jdbc >> driver >> >>>> issue anyway - not phoenix. >> >>>> >> >>>> Cheers, >> >>>> Stephen. >> >>>> >> >>>> >> >>>> >> >>>> On Mon, Jun 30, 2014 at 10:17 PM, Stephen Sprague < >> [email protected] <javascript:_e(%7B%7D,'cvml','[email protected]');>> >> >>>> wrote: >> >>>>> >> >>>>> Hi guys, >> >>>>> >> >>>>> May i ask what tools you use to monitor (active) Phoenix queries? >> >>>>> >> >>>>> looking at the web ui for hbase master and region servers it isn't >> >>>>> exactly intuitive as to what *phoenix* is doing. Its probably a >> stretch to >> >>>>> ask if there's something like "show processlist" (ala mysql) out >> there >> >>>>> somewhere. >> >>>>> >> >>>>> so is there a log file with metrics in it that perhaps i could tail >> or >> >>>>> analyze in realtime? perhaps even feed it back into hbase? :) >> hey, wait a >> >>>>> sec, anything in those system tables that is realtime-ish? >> >>>>> >> >>>>> regardless, kinda curious how people monitor the Phoenix plant. >> >>>>> >> >>>>> thanks, >> >>>>> Stephen. >> >>>>> >> >>>>> >> >>>> >> >>> >> > >> > >
