Re: troubles with HBase unit tests using MiniMRCluster on 0.23.1-SNAPSHOT

2012-01-12 Thread Mahadev Konar
fix or would it? thanks mahadev On Thu, Jan 12, 2012 at 6:02 PM, Mahadev Konar wrote: > HI Andrew, >  Sorry got side tracked with something else. Taking a look now. > > mahadev > > On Thu, Jan 12, 2012 at 3:23 PM, Andrew Purtell wrote: >> Hi Mahadev, >> &

Re: troubles with HBase unit tests using MiniMRCluster on 0.23.1-SNAPSHOT

2012-01-12 Thread Mahadev Konar
ng back. - Piet Hein > (via Tom White) > > > From: Andrew Purtell > To: "mapreduce-user@hadoop.apache.org" > Cc: Mahadev Konar > Sent: Wednesday, January 11, 2012 9:52 AM > > Subject: Re: troubles with HBase unit tests u

Re: troubles with HBase unit tests using MiniMRCluster on 0.23.1-SNAPSHOT

2012-01-10 Thread Mahadev Konar
Hi Andrew, Thanks for filing MAPREDUCE-3654. Ill take a look. As for the filenotfound issue mvn -Dhadoop.profile=23 clean test -Dtest=org.apache.hadoop.hbase.mapreduce.TestTableMapReduce on hbase 0.92. (latest svn branch) should be able to recreate this? (Assuming your patch applied on MR-3654)

Re: How to remove ending tab separator in streaming output

2011-12-26 Thread Mahadev Konar
Bccing common-user and ccing mapred-user. Please use the correct mailing lists for your questions. You can use -Dstream.map.output.field.separator= for specifying the seperator.   The link below should have more information. http://hadoop.apache.org/common/docs/r0.20.205.0/streaming.html#Custom

Re: Hadoop with Spring / Guice

2011-12-26 Thread Mahadev Konar
Eyal, Did you check out the spring data project? http://www.springsource.org/spring-data/hadoop That might offer some help. thanks mahadev On Mon, Dec 26, 2011 at 2:12 AM, Eyal Golan wrote: > Hi everyone, > This is my first post in this list, as I am a newb with Hadoop. > I am looking in t

Re: how does Hadoop Yarn support different programming models?

2011-12-26 Thread Mahadev Konar
Hi Bing, These links should give you more info: http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/YARN.html http://hadoop.apache.org/common/docs/r0.23.0/hadoop-yarn/hadoop-yarn-site/WritingYarnApplications.html Hope that helps. thanks mahadev On Mon, Dec 26, 2011 at 1:5

Re: Are job statistic logs right in the 0.23 version of Hadoop?

2011-12-19 Thread Mahadev Konar
Moving it to mapreduce-list. Sophie, This could just be a bug a 0.23. 0.23 does not have jobtrackers/tasktrackers. Could you see if you can recreate this? If yes, please do file a jira on this. thanks mahadev On Mon, Dec 19, 2011 at 12:22 PM, Raj V wrote: > Sophie > > > Are the clocks in syn

Re: Running YARN on top of legacy HDFS (i.e. 0.20)

2011-12-05 Thread Mahadev Konar
Avery, Currently we have only tested 0.23 MRv2 with 0.23 hdfs. I might be wrong but looking at the HDFS apis' it doesnt look like that it would be a lot of work to getting it to work with 0.20 apis. We had been using filecontext api's initially but have transitioned back to the old API's. Hope th

Re: Parsing job history logs [Hadoop 23]

2011-12-05 Thread Mahadev Konar
RD, You can take a look at JobHistoryUtils in org.apache.hadoop.mapreduce.v2.jobhistory. This might be helpful for programmatic api to get the jobhistory information. mahadev On Sat, Dec 3, 2011 at 12:37 PM, RD wrote: > For Parsing job history logs in H23, One way I see is to figure out the >

Re: MRv2 with other hadoop projects

2011-11-27 Thread Mahadev Konar
Stephen, Coming up PIG 0.9.2 and 0.10 are supposed to work with 0.23 (mrv2). Hive - I am not too sure of, you might want to check there mailing lists. As for HBase, there is some work needed (see: HBASE-4813 and MR-3169) to make it work with MRv2. Hope that helps. thanks mahadev On Nov 26,

Re: JobTracker API Hooks

2011-11-12 Thread Mahadev Konar
Moving it to mapreduce-user. Ronnie, Is jobclient.getalljobs() something that you are looking for? thanks mahadev On Sat, Nov 12, 2011 at 7:08 AM, Ronnie Dove wrote: > Hello, > > Is it possible to connect to an already running JobTracker via socket through > the Hadoop Java API?  I see that

Re: Why jobtracker.jsp can safely call a non-thread-safe method of JT?

2011-02-16 Thread Mahadev Konar
These are all getter apis which if you look through the implementations call some synchronized api's. Its just that they dont need to synchronized across all the completed/retired/running jobs for the web ui. thanks mahadev On Wed, Feb 16, 2011 at 12:47 AM, Min Zhou wrote: > Anyone can help me w

Re: get Map tasks info in command line

2011-02-13 Thread Mahadev Konar
you should be able to use hadoop job -events to get the task completion events from the job tracker. Here is a link: http://hadoop.apache.org/common/docs/current/commands_manual.html#job thanks mahadev On Sun, Feb 13, 2011 at 8:45 AM, Pedro Costa wrote: > Hi, > > 1 - How do I get the name of t

Re: location awareness on RT tasks?

2011-02-08 Thread Mahadev Konar
Hi Pedro, You can read abt the hdfs placement policy at: http://hadoop.apache.org/common/docs/r0.20.2/hdfs_design.html thanks mahadev On Fri, Feb 4, 2011 at 7:06 AM, Pedro Costa wrote: > Hi, > > When hadoop is running in cluster, the output of the Reducers are > saved in HDFS. The MapReduce h