Harsh, Thanks so much for your thorough explanation. John
-----Original Message----- From: Harsh J [mailto:ha...@cloudera.com] Sent: Wednesday, June 05, 2013 8:18 PM To: <user@hadoop.apache.org> Subject: Re: Hadoop JARs and Eclipse Hi John, On Thu, Jun 6, 2013 at 1:21 AM, John Lilley <john.lil...@redpoint.net> wrote: > -- From where will it fetch the Hadoop JARs? >From the Maven Central repository (we publish our jars and dependencies are >also available there), or a custom defined repository if you lack internet >access. > -- If I have built Hadoop from source, how do I tell it to find my just-built > JARs instead of those from the official release site? If you built Hadoop locally with "mvn install", then their version is added to your local repository (~/.m2) and can be "referred' to after a successful "mvn install". > -- Is it necessary to build Hadoop from source so that the Eclipse debugger > can locate source files and display them if I need to step in? No, see below. > -- Step into the source code of referenced Hadoop code in the debugger. This one is easy with the Eclipse Maven IDE plugin, see http://stackoverflow.com/questions/310720/get-source-jar-files-attached-to-eclipse-for-maven-managed-dependencies. This lets you view and also debug-step into sources of dependencies. > -- Make changes to the Hadoop source and rebuild it if things are not working > as expected (e.g. to add logging). > -- Switch to building from SVN instead of a release snapshot, and apply > patches. If you're wanting to use the branch-2 tip or the trunk tip, then they carry their own version (2.1.0-beta-SNAPSHOT, or 3.0.0-SNAPSHOT and such). Referring these versions in the pom.xml of your project would be the first step (you may also have to enable discovery of snapshot releases?). Second would be to make the changes to the hadoop source and running a global "mvn install" to update the local repository (~/.m2) versions to the patched ones. Subsequently, your project will be able to find it. What am saying is you can do this without having to manage Hadoop as an eclipse project - just patch and build (plus install) on CLI, but mark your project to reference their exact SNAPSHOT version so it discovers the new built jars. -- Harsh J
