Re: How to Access a Mapper and Reducer Counters (cont'd???)

2012-11-02 Thread Lewis John Mcgibbney
Hi, So I was working on this again. To display all counters from a currently running job I've been doing CollectionString mapTaskCounterGroups = currentJob.getCounters().getGroupNames(); When I print these values to std out this gives me the following; [FileSystemCounters,

Re: Error: INFO ipc.Client: Retrying connect to server: /192.168.100.11:8020. Already tried 0 time(s).

2012-11-02 Thread shriparv
This just tells that the service which you are trying to conned is not running on that address or port no. just try netstat -nlrt | grep 8020 and check if it showing something. what i can find out from the error that the server is not running or if it running then its running on some other port

Re: cluster set-up / a few quick questions - SOLVED

2012-11-02 Thread Nitin Pawar
you can get the script from hadoop codebase at http://svn.apache.org/viewcvs.cgi/hadoop/commonhttp://svn.apache.org/viewcvs.cgi/hadoop/common/trunk/ On Fri, Nov 2, 2012 at 12:41 AM, Kartashov, Andy andy.kartas...@mpac.cawrote: People, While I did not find start-balancer.sh script on my

RE: jobtracker page @50030 timeout or take very long time.

2012-11-02 Thread Kartashov, Andy
... another thought... do you happened to have some ungracefully terminated jobs still running in the background.. Try hadoop job -list Sometimes, when I hardstop a job and restart a new one, I notice a slow down until I kill those jobs gracefully by running hadoop job -kill job-id . The

Re: OutputFormat and Reduce Task

2012-11-02 Thread Dhruv
Thanks Harsh, just to be clear--if I have a large key set and if I run with just one reducer which is the default, the OutputFormat and the RecordWriter will be constructed only once? On Thu, Nov 1, 2012 at 8:14 PM, Harsh J ha...@cloudera.com wrote: Hi Dhruv, Inline. On Fri, Nov 2, 2012

Re: OutputFormat and Reduce Task

2012-11-02 Thread Harsh J
Yes, only once per task attempt. On Fri, Nov 2, 2012 at 11:05 PM, Dhruv dhru...@gmail.com wrote: Thanks Harsh, just to be clear--if I have a large key set and if I run with just one reducer which is the default, the OutputFormat and the RecordWriter will be constructed only once? On Thu,