Re: Why is hadoop build I generated from a release branch different from release build?
Thanks for all the replies. It turns out that build generated by ant has bin & conf etc folders in one level above. And I looked at hadoop scripts and apparently it looks for right jars both in root directory and root/build/ directory as well. so I think I am covered for now. Thanks again! On Thu, Mar 8, 2012 at 4:15 PM, Leo Leung wrote: > Hi Pawan, > > ant -p (not for 0.23+) will tell you the available build targets. > > Use mvn (maven) for 0.23 or newer > > > > -Original Message- > From: Matt Foley [mailto:mfo...@hortonworks.com] > Sent: Thursday, March 08, 2012 3:52 PM > To: common-user@hadoop.apache.org > Subject: Re: Why is hadoop build I generated from a release branch > different from release build? > > Hi Pawan, > The complete way releases are built (for v0.20/v1.0) is documented at >http://wiki.apache.org/hadoop/HowToRelease#Building > However, that does a bunch of stuff you don't need, like generate the > documentation and do a ton of cross-checks. > > The full set of ant build targets are defined in build.xml in the top > level of the source code tree. > "binary" may be the target you want. > > --Matt > > On Thu, Mar 8, 2012 at 3:35 PM, Pawan Agarwal >wrote: > > > Hi, > > > > I am trying to generate hadoop binaries from source and execute hadoop > > from the build I generate. I am able to build, however I am seeing > > that as part of build *bin* folder which comes with hadoop > > installation is not generated in my build. Can someone tell me how to > > do a build so that I can generate build equivalent to hadoop release > > build and which can be used directly to run hadoop. > > > > Here's the details. > > Desktop: Ubuntu Server 11.10 > > Hadoop version for installation: 0.20.203.0 (link: > > http://mirrors.gigenet.com/apache//hadoop/common/hadoop-0.20.203.0/) > > Hadoop Branch used build: branch-0.20-security-203 Build Command used: > > "Ant maven-install" > > > > Here's the directory structures from build I generated vs hadoop > > official release build. > > > > *Hadoop directory which I generated:* > > pawan@ubuntu01:/hadoop0.20.203.0/hadoop-common/build$ ls -1 ant > > c++ > > classes > > contrib > > examples > > hadoop-0.20-security-203-pawan > > hadoop-ant-0.20-security-203-pawan.jar > > hadoop-core-0.20-security-203-pawan.jar > > hadoop-examples-0.20-security-203-pawan.jar > > hadoop-test-0.20-security-203-pawan.jar > > hadoop-tools-0.20-security-203-pawan.jar > > ivy > > jsvc > > src > > test > > tools > > webapps > > > > *Official Hadoop build installation* > > pawan@ubuntu01:/hadoop0.20.203.0/hadoop-common/build$ ls /hadoop -1 > > bin build.xml > > c++ > > CHANGES.txt > > conf > > contrib > > docs > > hadoop-ant-0.20.203.0.jar > > hadoop-core-0.20.203.0.jar > > hadoop-examples-0.20.203.0.jar > > hadoop-test-0.20.203.0.jar > > hadoop-tools-0.20.203.0.jar > > input > > ivy > > ivy.xml > > lib > > librecordio > > LICENSE.txt > > logs > > NOTICE.txt > > README.txt > > src > > webapps > > > > > > > > Any pointers for help are greatly appreciated? > > > > Also, if there are any other resources for understanding hadoop build > > system, pointers to that would be also helpful. > > > > Thanks > > Pawan > > >
RE: Why is hadoop build I generated from a release branch different from release build?
Hi Pawan, ant -p (not for 0.23+) will tell you the available build targets. Use mvn (maven) for 0.23 or newer -Original Message- From: Matt Foley [mailto:mfo...@hortonworks.com] Sent: Thursday, March 08, 2012 3:52 PM To: common-user@hadoop.apache.org Subject: Re: Why is hadoop build I generated from a release branch different from release build? Hi Pawan, The complete way releases are built (for v0.20/v1.0) is documented at http://wiki.apache.org/hadoop/HowToRelease#Building However, that does a bunch of stuff you don't need, like generate the documentation and do a ton of cross-checks. The full set of ant build targets are defined in build.xml in the top level of the source code tree. "binary" may be the target you want. --Matt On Thu, Mar 8, 2012 at 3:35 PM, Pawan Agarwal wrote: > Hi, > > I am trying to generate hadoop binaries from source and execute hadoop > from the build I generate. I am able to build, however I am seeing > that as part of build *bin* folder which comes with hadoop > installation is not generated in my build. Can someone tell me how to > do a build so that I can generate build equivalent to hadoop release > build and which can be used directly to run hadoop. > > Here's the details. > Desktop: Ubuntu Server 11.10 > Hadoop version for installation: 0.20.203.0 (link: > http://mirrors.gigenet.com/apache//hadoop/common/hadoop-0.20.203.0/) > Hadoop Branch used build: branch-0.20-security-203 Build Command used: > "Ant maven-install" > > Here's the directory structures from build I generated vs hadoop > official release build. > > *Hadoop directory which I generated:* > pawan@ubuntu01:/hadoop0.20.203.0/hadoop-common/build$ ls -1 ant > c++ > classes > contrib > examples > hadoop-0.20-security-203-pawan > hadoop-ant-0.20-security-203-pawan.jar > hadoop-core-0.20-security-203-pawan.jar > hadoop-examples-0.20-security-203-pawan.jar > hadoop-test-0.20-security-203-pawan.jar > hadoop-tools-0.20-security-203-pawan.jar > ivy > jsvc > src > test > tools > webapps > > *Official Hadoop build installation* > pawan@ubuntu01:/hadoop0.20.203.0/hadoop-common/build$ ls /hadoop -1 > bin build.xml > c++ > CHANGES.txt > conf > contrib > docs > hadoop-ant-0.20.203.0.jar > hadoop-core-0.20.203.0.jar > hadoop-examples-0.20.203.0.jar > hadoop-test-0.20.203.0.jar > hadoop-tools-0.20.203.0.jar > input > ivy > ivy.xml > lib > librecordio > LICENSE.txt > logs > NOTICE.txt > README.txt > src > webapps > > > > Any pointers for help are greatly appreciated? > > Also, if there are any other resources for understanding hadoop build > system, pointers to that would be also helpful. > > Thanks > Pawan >
Re: Why is hadoop build I generated from a release branch different from release build?
Hi Pawan, The complete way releases are built (for v0.20/v1.0) is documented at http://wiki.apache.org/hadoop/HowToRelease#Building However, that does a bunch of stuff you don't need, like generate the documentation and do a ton of cross-checks. The full set of ant build targets are defined in build.xml in the top level of the source code tree. "binary" may be the target you want. --Matt On Thu, Mar 8, 2012 at 3:35 PM, Pawan Agarwal wrote: > Hi, > > I am trying to generate hadoop binaries from source and execute hadoop from > the build I generate. I am able to build, however I am seeing that as part > of build *bin* folder which comes with hadoop installation is not generated > in my build. Can someone tell me how to do a build so that I can generate > build equivalent to hadoop release build and which can be used directly to > run hadoop. > > Here's the details. > Desktop: Ubuntu Server 11.10 > Hadoop version for installation: 0.20.203.0 (link: > http://mirrors.gigenet.com/apache//hadoop/common/hadoop-0.20.203.0/) > Hadoop Branch used build: branch-0.20-security-203 > Build Command used: "Ant maven-install" > > Here's the directory structures from build I generated vs hadoop official > release build. > > *Hadoop directory which I generated:* > pawan@ubuntu01:/hadoop0.20.203.0/hadoop-common/build$ ls -1 > ant > c++ > classes > contrib > examples > hadoop-0.20-security-203-pawan > hadoop-ant-0.20-security-203-pawan.jar > hadoop-core-0.20-security-203-pawan.jar > hadoop-examples-0.20-security-203-pawan.jar > hadoop-test-0.20-security-203-pawan.jar > hadoop-tools-0.20-security-203-pawan.jar > ivy > jsvc > src > test > tools > webapps > > *Official Hadoop build installation* > pawan@ubuntu01:/hadoop0.20.203.0/hadoop-common/build$ ls /hadoop -1 > bin > build.xml > c++ > CHANGES.txt > conf > contrib > docs > hadoop-ant-0.20.203.0.jar > hadoop-core-0.20.203.0.jar > hadoop-examples-0.20.203.0.jar > hadoop-test-0.20.203.0.jar > hadoop-tools-0.20.203.0.jar > input > ivy > ivy.xml > lib > librecordio > LICENSE.txt > logs > NOTICE.txt > README.txt > src > webapps > > > > Any pointers for help are greatly appreciated? > > Also, if there are any other resources for understanding hadoop build > system, pointers to that would be also helpful. > > Thanks > Pawan >
