Re: 10 nodes how to build the topological graph

2012-02-02 Thread alo alt
Hi Rock, you mean rack awareness. http://hadoop.apache.org/common/docs/r0.17.2/hdfs_user_guide.html#Rack+Awareness Here you find examples: http://wiki.apache.org/hadoop/topology_rack_awareness_scripts best, Alex -- Alexander Lorenz http://mapredit.blogspot.com On Feb 3, 2012, at 8:03 AM, Ji

10 nodes how to build the topological graph

2012-02-02 Thread Jinyan Xu
Hi , I have 10 machines to build a small cluster, how to build the topological graph between these machines? I mean how to build the rack use 10 machines. Thanks ! -Rock The information and any attached documents contained in this message may be confidential a

Problem when starting datanode

2012-02-02 Thread hadoop hive
hey folks , I m getting an when i starting my datanode. can any1 have the idea what this error about. 2012-02-03 11:57:02,947 ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: DatanodeRegistration( 10.0.3.31:50010, storageID=DS-1677953808-10.0.3.31-50010-1318330317888, infoPort=50075, ipcPo

2 Failures in unit test of Hadoop 1.0.0 version

2012-02-02 Thread Grace
Hi all, Downloading the release version Hadoop 1.0.0. from the apache web site. After several hours' test period(by executing "ant test"), I found there are totally 2 failures in TestCLI(60 failed) and TestSaslRPC cases respectively. I wonder if there is any missing steps for me. Is there someone

Re: Can't achieve load distribution

2012-02-02 Thread Mark Kerzner
And that is exactly what I found. I have a "hack" for now - give all files on the command line - and I will wait for the next release in some distribution. Thank you, Mark On Thu, Feb 2, 2012 at 9:55 PM, Harsh J wrote: > New API NLineInputFormat is only available from 1.0.1, and not in any > o

Re: Can't achieve load distribution

2012-02-02 Thread Harsh J
New API NLineInputFormat is only available from 1.0.1, and not in any of the earlier 1 (1.0.0) or 0.20 (0.20.x, 0.20.xxx) vanilla Apache releases. On Fri, Feb 3, 2012 at 7:08 AM, Praveen Sripati wrote: > Mark, > > NLineInputFormat was not something which was introduced in 0.21, I have > just sent

Re: Can't achieve load distribution

2012-02-02 Thread Praveen Sripati
Mark, NLineInputFormat was not something which was introduced in 0.21, I have just sent the reference to the 0.21 url FYI. It's in 0.20.205, 1.0.0 and 0.23 releases also. Praveen On Fri, Feb 3, 2012 at 1:25 AM, Mark Kerzner wrote: > Praveen, > > this seems just like the right thing, but it's AP

[ANNOUNCE] Apache MRUnit 0.8.0-incubating released

2012-02-02 Thread Brock Noland
The Apache MRUnit team is pleased to announce the release of MRUnit 0.8.0-incubating from the Apache Incubator. This is the second release of Apache MRUnit, a Java library that helps developers unit test Apache Hadoop map reduce jobs. The release is available here: http://www.apache.org/dyn/close

Any ever run Hadoop on Tier 3?

2012-02-02 Thread Keith Wiley
I'm looking for any experience I can benefit from in running Hadoop on Tier 3? I'll probably be using Cloudera's CDH3, which provides Hadoop-0.20.2+923.194. Would anyone anticipate any gotchas with such a scenario?

Re: Can't achieve load distribution

2012-02-02 Thread Mark Kerzner
Praveen, this seems just like the right thing, but it's API 0.21 (I googled about the problems with it), so I have to use either the next Cloudera release, or Hortonworks, or something, am I right? Mark On Thu, Feb 2, 2012 at 7:39 AM, Praveen Sripati wrote: > > I have a simple MR job, and I wan

Re: Execute a Map/Reduce Job Jar from Another Java Program.

2012-02-02 Thread Robert Evans
What happens? Is there an exception, does nothing happen? I am curious. Also how did you launch your other job that is trying to run this one. The hadoop script sets up a lot of environment variables classpath etc to make hadoop work properly, and some of that may not be set up correctly to

Re: setting multi node in local machine

2012-02-02 Thread Anil Gupta
If u use VMware and create vm's then you can do it. Best Regards, Anil On Feb 1, 2012, at 8:22 PM, Arun Prakash wrote: > I have windows machine,i am trying to install hadoop with multiple data > node like cluster in single machine .Is it possible? > > > Best Regards > Arun Prakash C.K > > Ke

Re: setting multi node in local machine

2012-02-02 Thread Harsh J
This conversation ought to help you get started: http://search-hadoop.com/m/a4klk28NUr12 On Thu, Feb 2, 2012 at 9:52 AM, Arun Prakash wrote: > I have windows machine,i am trying to install hadoop with multiple data > node like cluster in single machine .Is it possible? > > > Best Regards >  Arun

Re: org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException as:hdfs

2012-02-02 Thread Harsh J
$ sudo rm /var/lib/hadoop-0.20/cache/hdfs/dfs/data And then start DN again regularly, all should be ok. On Thu, Feb 2, 2012 at 5:36 AM, Vijayakumar Ramdoss wrote: > Hi All, > > When I am try to start the  datanode, namenode and secondarynodes, its > throwing the org.apache.hadoop.security.UserGr

Re: Execute a Map/Reduce Job Jar from Another Java Program.

2012-02-02 Thread Harsh J
Moving to common-user. Common-dev is for project development discussions, not user help. Could you elaborate on how you used RunJar? What arguments did you provide, and is the target jar a runnable one or a regular jar? What error did you get? On Thu, Feb 2, 2012 at 8:44 PM, abees muhammad wrote

Re: Can't achieve load distribution

2012-02-02 Thread Praveen Sripati
> I have a simple MR job, and I want each Mapper to get one line from my input file (which contains further instructions for lengthy processing). Use the NLineInputFormat class. http://hadoop.apache.org/mapreduce/docs/r0.21.0/api/org/apache/hadoop/mapreduce/lib/input/NLineInputFormat.html Pravee