Do you mean want to connect the JobTracker using .Net ? If so, I'm afraid I have no idea how to this. The rpc of hadoop is language dependent.
On Thu, Feb 4, 2010 at 7:18 PM, Mark N <nipen.m...@gmail.com> wrote: > could you please elaborate on this ( * hint to get started as am very new > to hadoop? ) > So far I could succesfully read all the default and custom counters. > > Currently we are having a .net client. > > thanks in advance. > > > On Thu, Feb 4, 2010 at 4:53 PM, Jeff Zhang <zjf...@gmail.com> wrote: > > > Well, you can create a proxy of JobTracker in client side, and then you > can > > use the API of JobTracker to get the information of jobs. The Proxy take > > the > > responsibility of communication with the Master Node. Read the source > > code > > of JobClient can help you. > > > > > > On Thu, Feb 4, 2010 at 6:59 PM, Mark N <nipen.m...@gmail.com> wrote: > > > > > Ye currently am using jobclient to read these counters. > > > > > > But We are not able to use *webservices *because the jar which is used > to > > > read the counters from running hadoop job is itself a Hadoop program > > > > > > If we could have pure Java Api which is run without hadoop command then > > we > > > could return the counter variable into webservices and show in UI. > > > > > > Any help or technique to show thsese counters in the UI would be > > > appreciated ( not necessarily using web service ) > > > > > > > > > I am using webservices because I am having .net VB client > > > > > > thanks > > > > > > > > > > > > On Wed, Feb 3, 2010 at 8:33 PM, Jeff Zhang <zjf...@gmail.com> wrote: > > > > > > > I think you can use JobClient to get the counters in your web > service. > > > > If you look at the shell script bin/hadoop, you will find that > actually > > > > this > > > > shell use the JobClient to get the counters. > > > > > > > > > > > > > > > > On Wed, Feb 3, 2010 at 4:34 AM, Mark N <nipen.m...@gmail.com> wrote: > > > > > > > > > We have a hadoop job running and have used custom counters to track > > > few > > > > > counters ( like no of successfully processed documents matching > > certain > > > > > conditions) > > > > > > > > > > > > > > > Since we need to get this counters even while the Hadoop job is > > running > > > , > > > > > we > > > > > wrote another Java program to read these counters > > > > > > > > > > > > > > > *Counter reader program *will do the following : > > > > > > > > > > > > > > > 1) List all the running jobs. > > > > > > > > > > 2) Get the running job using Job name > > > > > > > > > > 2) Get all the counter for individual running jobs > > > > > > > > > > 3) Set this counters in variables. > > > > > We could successfully read these counters , but since we > need > > > to > > > > > show these counters to custom UI , how can we show these counters? > > > > > > > > > > we looked into various options to read these counters to > show > > in > > > > UI > > > > > as following : > > > > > > > > > > 1. Dump these counters to database , however this may be > > overhead > > > > > 2. Write web service and UI will invoke the functions from > > these > > > > > service to show in UI ( However since we need to run "*Counter > reader > > > > > program " *with Hadoop command it might not be feasible to write > web > > > > > service ? ) > > > > > > > > > > so the question is can we achive to read the counters using > > simple > > > > > Java APIs ? Does anyone have idea how does the default jobtracker > JSP > > > > works > > > > > ? we wanted to built something similar to this > > > > > > > > > > thanks > > > > > > > > > > > > > > > > > > > > -- > > > > > Nipen Mark > > > > > > > > > > > > > > > > > > > > > -- > > > > Best Regards > > > > > > > > Jeff Zhang > > > > > > > > > > > > > > > > -- > > > Nipen Mark > > > > > > > > > > > -- > > Best Regards > > > > Jeff Zhang > > > > > > -- > Nipen Mark > -- Best Regards Jeff Zhang
