try hadoop 1.1.2.. Best Regards Amir Sanjar
System Management Architect PowerLinux Open Source Hadoop development lead IBM Senior Software Engineer Phone# 512-286-8393 Fax# 512-838-8858 From: Mohammad Mustaqeem <3m.mustaq...@gmail.com> To: common-dev <common-dev@hadoop.apache.org>, Date: 04/09/2013 08:18 AM Subject: Re: Building Hadoop from source code @Ling, i am very new. I was about to ask you the same. Which version is easy to handle? I want to change something in HDFS. Please, suggest. On Tue, Apr 9, 2013 at 6:29 PM, Ling Kun <lkun.e...@gmail.com> wrote: > @Mohammad > > Sorry, since the internet connection rate I am using is tooooooo low, It > takes me a looooong time to download the tar file you showed above. > > however, here is some of my opinion based on my local develop env, and i > am also a fresh hadooper here. > > 1. what you have download from the trunk is Hadoop 3.0, which you can > see from the tar.gz filename. While the link above is about Hadoop 1.0, > and 2.0. So it seems hadoop developers have adjust the distributed tar > file. > > 2. When looking into the bin/hadoop script in both tar.gz file. I think > that you will find out that most of them are the same. This file will make > a MR job start. > > 3. What version of hadoop do you plan to work on? 2.0 or 3.0? > > Ling Kun > > > On Tue, Apr 9, 2013 at 8:25 PM, Mohammad Mustaqeem > <3m.mustaq...@gmail.com>wrote: > > > @Ling, Is it the same file as we download from > > http://apache.techartifact.com/mirror/hadoop/common/ ?? > > But the tree structure of the build tar file is not same as downloaded > from > > the site. > > > > @Ling plz guide me. I have been stuck at this state from 1 month . > > I didn't find link that explain in detail. The material that I found is > > very old that uses ant. > > I want to just build hadoop from source code. > > Plz, tell > > > > > > On Tue, Apr 9, 2013 at 5:37 PM, Ling Kun <lkun.e...@gmail.com> wrote: > > > > > @Mohammad: > > > 1. have you successfully run the command Steve suggested? "mvn > package > > > -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip=true" > > > 2. If yes, and then you can use the command " find . -name "*tar*" " > to > > > find the tar file, which is in the path > > > "./hadoop-dist/target/hadoop-3.0.0-SNAPSHOT.tar.gz" > > > > > > 3. Sorry, I make a mistake about the tar file name, It is tar.gz > > > > > > > > > @ maisnam ns: > > > I am afraid the version you are using is not the same as Mohammad > > used. > > > > > > > > > Ling Kun > > > > > > > > > On Tue, Apr 9, 2013 at 7:58 PM, Mohammad Mustaqeem > > > <3m.mustaq...@gmail.com>wrote: > > > > > > > @maisnam, I am using ubuntu and after running svn I get hadoop folder > > as > > > > given in screenshot. > > > > I didn't finf any build folder > > > > > > > > > > > > On Tue, Apr 9, 2013 at 5:22 PM, maisnam ns <maisnam...@gmail.com> > > wrote: > > > > > > > >> @@Mohammad Mustaqeem please ignore the earlier mail as the > screenshots > > > >> where not copied in the mail I have attached a doc file. > > > >> > > > >> You can follow the steps earlier using the screenshots I have > > attached. > > > >> > > > >> > > > >> On Tue, Apr 9, 2013 at 5:15 PM, maisnam ns <maisnam...@gmail.com> > > > wrote: > > > >> > > > >>> @Mohammad Mustaqeem please follow the screenshots one by one > > > >>> > > > >>> > > > >>> cd build you will find > > > >>> > > > >>> > > > >>> Now cd hadoop-0.20.3-dev you will find the below screenshot > > > >>> > > > >>> > > > >>> Now the bin contains all the scripts you may need to start hadoop > > > >>> 1. Go to that bin directory and format your namenode > > > >>> 2. configure you site-xmls > > > >>> 3. start-all.sh note there are other ways also > > > >>> 4. do jps to see if all daemons are running > > > >>> > > > >>> > > > >>> > > > >>> On Tue, Apr 9, 2013 at 4:54 PM, Mohammad Mustaqeem < > > > >>> 3m.mustaq...@gmail.com> wrote: > > > >>> > > > >>>> Please, anyone tell me where is the build hadoop?? > > > >>>> that can be used to install the hadoop cluster. > > > >>>> > > > >>>> > > > >>>> On Tue, Apr 9, 2013 at 2:32 PM, Mohammad Mustaqeem > > > >>>> <3m.mustaq...@gmail.com>wrote: > > > >>>> > > > >>>> > @Ling, you mean to say to run "find -name *tar*"?? > > > >>>> > I run but don't know which file will be used to install Hadoop. > > > >>>> > > > > >>>> > @Niranjan > > > >>>> > I haven't changed anything I just executed "svn checkout > > > >>>> > > http://svn.apache.org/repos/asf/hadoop/common/trunk/hadoop-trunk"; > > > >>>> to get > > > >>>> > the Hadoop source code in the "hadoop-trunk" directory. > > > >>>> > After that I executed the "cd hadoop-trunk" and finally executed > > > "mvn > > > >>>> > package -Pdist -Dtar -DskipTests -Dmaven.javadoc.skip=true". > > > >>>> > I want to ask what is location of tar file created which can be > > used > > > >>>> to > > > >>>> > install the Hadoop ?? > > > >>>> > > > > >>>> > > > > >>>> > On Tue, Apr 9, 2013 at 2:08 PM, maisnam ns < > maisnam...@gmail.com> > > > >>>> wrote: > > > >>>> > > > > >>>> >> @Mohammad Mustaqeem - Once you create the patch as given in the > > > 'How > > > >>>> to > > > >>>> >> contribute to Hadoop' and apply the patch. The changes will be > > > >>>> reflected > > > >>>> >> in the files you have modified. > > > >>>> >> > > > >>>> >> 1.Now you trigger the build ant or maven. I tried with ant so > the > > > ant > > > >>>> >> command I gave is ant clean compile bin-package. Don't forget > to > > > >>>> download > > > >>>> >> ivy.jar and copy into you ant home/ lib folder. Once the build > is > > > >>>> >> triggered > > > >>>> >> Hadoop should get built along with the changes you made. > > > >>>> >> > > > >>>> >> If , I am not mistaken , you modified some hadoop files say > > > >>>> >> BlockLocation.java, in your > > > >>>> >> Hadoopx.x\src\core\org\apache\hadoop\fs\BlockLocation.java. > > > >>>> >> > > > >>>> >> The jar will be in > Hadoopx.x\build\hadoop-0.20.3-dev-core.jar(In > > my > > > >>>> >> version) > > > >>>> >> > > > >>>> >> Hope this clears your doubt. > > > >>>> >> > > > >>>> >> Regards > > > >>>> >> Niranjan Singh > > > >>>> >> > > > >>>> >> > > > >>>> >> On Tue, Apr 9, 2013 at 1:38 PM, Mohammad Mustaqeem > > > >>>> >> <3m.mustaq...@gmail.com>wrote: > > > >>>> >> > > > >>>> >> > @Steve > > > >>>> >> > I am new to Hadoop developement. > > > >>>> >> > Can you please tell me, whats is the location of tar file?? > > > >>>> >> > > > > >>>> >> > > > > >>>> >> > On Tue, Apr 9, 2013 at 12:09 AM, Steve Loughran < > > > >>>> ste...@hortonworks.com > > > >>>> >> > >wrote: > > > >>>> >> > > > > >>>> >> > > On 8 April 2013 16:08, Mohammad Mustaqeem < > > > >>>> 3m.mustaq...@gmail.com> > > > >>>> >> > wrote: > > > >>>> >> > > > > > >>>> >> > > > Please, tell what I am doing wrong?? > > > >>>> >> > > > Whats the problem?? > > > >>>> >> > > > > > > >>>> >> > > > > > >>>> >> > > a lot of these seem to be network-related tests. You can > turn > > > >>>> off all > > > >>>> >> the > > > >>>> >> > > tests; look in BUILDING.TXT at the root of the source tree > > for > > > >>>> the > > > >>>> >> > various > > > >>>> >> > > operations, then add -DskipTests to the end of every > command, > > > >>>> such as > > > >>>> >> > > > > > >>>> >> > > mvn package -Pdist -Dtar -DskipTests > > > >>>> >> > > > > > >>>> >> > > to build the .tar packages > > > >>>> >> > > > > > >>>> >> > > mvn package -Pdist -Dtar -DskipTests > > -Dmaven.javadoc.skip=true > > > >>>> >> > > to turn off the javadoc creation too, for an even faster > > build > > > >>>> >> > > > > > >>>> >> > > > > >>>> >> > > > > >>>> >> > > > > >>>> >> > -- > > > >>>> >> > *With regards ---* > > > >>>> >> > *Mohammad Mustaqeem*, > > > >>>> >> > M.Tech (CSE) > > > >>>> >> > MNNIT Allahabad > > > >>>> >> > 9026604270 > > > >>>> >> > > > > >>>> >> > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > -- > > > >>>> > *With regards ---* > > > >>>> > *Mohammad Mustaqeem*, > > > >>>> > M.Tech (CSE) > > > >>>> > MNNIT Allahabad > > > >>>> > 9026604270 > > > >>>> > > > > >>>> > > > > >>>> > > > > >>>> > > > >>>> > > > >>>> -- > > > >>>> *With regards ---* > > > >>>> *Mohammad Mustaqeem*, > > > >>>> M.Tech (CSE) > > > >>>> MNNIT Allahabad > > > >>>> 9026604270 > > > >>>> > > > >>> > > > >>> > > > >> > > > > > > > > > > > > -- > > > > *With regards ---* > > > > *Mohammad Mustaqeem*, > > > > M.Tech (CSE) > > > > MNNIT Allahabad > > > > 9026604270 > > > > > > > > > > > > > > > > > > > > > -- > > > http://www.lingcc.com > > > > > > > > > > > -- > > *With regards ---* > > *Mohammad Mustaqeem*, > > M.Tech (CSE) > > MNNIT Allahabad > > 9026604270 > > > > > > -- > http://www.lingcc.com > -- *With regards ---* *Mohammad Mustaqeem*, M.Tech (CSE) MNNIT Allahabad 9026604270
