Re: Compile Just a Subproject
--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter. Thanks, Curtis On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth cnaur...@hortonworks.com wrote: Hi Curtis, I handle this by running mvn install -DskipTests once from the root of the whole hadoop project to install the sub-modules in my local Maven repository. Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module. Full details are in the BUILDING.txt file in the root of the hadoop project. Look for the section titled Building components separately. I hope this helps. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich curtuller...@gmail.com wrote: What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that thishttp://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772would have worked but no changes seem to take effect after building this way. Thanks, Curtis
Re: Compile Just a Subproject
I've executed the commands as you've said, and the jar that is run is unchanged. Here's exactly what I did. C:\l\ start-dfs C:\l\ start-yarn C:\l\ cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshellhadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami ...output... ...modify logging code in C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshellmvn clean install -DskipTests C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshellhadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami ...identical output to before... Do you see anything wrong with what I've done? Thanks, Curtis On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi ojo...@hortonworks.com wrote: Hi Curtis, where are you picking your jar file from? once you run above command you will see the updated jar file in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar I hope you are not using below jar file /hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar What chris has said is right. once you have taken latest code, you should follow below steps 1) mvn clean install -DskipTests (clean to remove previously generated code) 2) now say you are updating distributed shell client code. then go to /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/ and then run mvn clean install and use jar from target sub folder. Thanks, Omkar Joshi *Hortonworks Inc.* http://www.hortonworks.com On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich curtuller...@gmail.comwrote: Hi Chris, I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows): cd %hadoop_install_dir% mvn clean package -DskipTests mvn install -DskipTests --- modify the code in distributed shell's Client.java --- cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell mvn clean mvn package -DskipTests mvn install -DskipTests Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter. Thanks, Curtis On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth cnaur...@hortonworks.com wrote: Hi Curtis, I handle this by running mvn install -DskipTests once from the root of the whole hadoop project to install the sub-modules in my local Maven repository. Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module. Full details are in the BUILDING.txt file in the root of the hadoop project. Look for the section titled Building components separately. I hope this helps. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich curtuller...@gmail.com wrote: What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that thishttp://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772would have worked but no changes seem to take effect after building this way. Thanks, Curtis
Re: Compile Just a Subproject
-applications-distributedshell hadoop org.apache.hadoop.yarn.applications.distributedshell.Client -jar target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar -shell_command whoami ...identical output to before... Do you see anything wrong with what I've done? Thanks, Curtis On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi ojo...@hortonworks.com wrote: Hi Curtis, where are you picking your jar file from? once you run above command you will see the updated jar file in /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar I hope you are not using below jar file /hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar What chris has said is right. once you have taken latest code, you should follow below steps 1) mvn clean install -DskipTests (clean to remove previously generated code) 2) now say you are updating distributed shell client code. then go to /hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/ and then run mvn clean install and use jar from target sub folder. Thanks, Omkar Joshi Hortonworks Inc. On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich curtuller...@gmail.com wrote: Hi Chris, I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows): cd %hadoop_install_dir% mvn clean package -DskipTests mvn install -DskipTests --- modify the code in distributed shell's Client.java --- cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell mvn clean mvn package -DskipTests mvn install -DskipTests Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter. Thanks, Curtis On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth cnaur...@hortonworks.com wrote: Hi Curtis, I handle this by running mvn install -DskipTests once from the root of the whole hadoop project to install the sub-modules in my local Maven repository. Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module. Full details are in the BUILDING.txt file in the root of the hadoop project. Look for the section titled Building components separately. I hope this helps. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich curtuller...@gmail.com wrote: What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that this would have worked but no changes seem to take effect after building this way. Thanks, Curtis
Re: Compile Just a Subproject
can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module. Full details are in the BUILDING.txt file in the root of the hadoop project. Look for the section titled Building components separately. I hope this helps. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich curtuller...@gmail.com wrote: What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that thishttp://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772would have worked but no changes seem to take effect after building this way. Thanks, Curtis
Re: Compile Just a Subproject
Hi Curtis, I handle this by running mvn install -DskipTests once from the root of the whole hadoop project to install the sub-modules in my local Maven repository. Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module. Full details are in the BUILDING.txt file in the root of the hadoop project. Look for the section titled Building components separately. I hope this helps. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich curtuller...@gmail.comwrote: What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that thishttp://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772would have worked but no changes seem to take effect after building this way. Thanks, Curtis
Re: Compile Just a Subproject
Hi Chris, I really appreciate the response. What you described is what I initially tried. The changes never seem to take effect though. Here's what I've done (this is Windows): cd %hadoop_install_dir% mvn clean package -DskipTests mvn install -DskipTests --- modify the code in distributed shell's Client.java --- cd hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell mvn clean mvn package -DskipTests mvn install -DskipTests Then I run the jar, just as before. I've just been changing log statements to see if the changes worked. They don't--the output doesn't change after doing this rebuild. I've also tried clearing the files put on HDFS in my user directory in case that was the issue. I've taken a more thorough look at BUILDING.txt and I seemed to be consistent with the procedures described there. Am I missing anything else? I've tried restarting yarn and dfs, though I didn't think that would matter. Thanks, Curtis On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth cnaur...@hortonworks.comwrote: Hi Curtis, I handle this by running mvn install -DskipTests once from the root of the whole hadoop project to install the sub-modules in my local Maven repository. Then, you can change your working directory to any sub-module (like distributed shell) and run a successful Maven build for just that sub-module. Full details are in the BUILDING.txt file in the root of the hadoop project. Look for the section titled Building components separately. I hope this helps. Chris Nauroth Hortonworks http://hortonworks.com/ On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich curtuller...@gmail.comwrote: What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that thishttp://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772would have worked but no changes seem to take effect after building this way. Thanks, Curtis
Compile Just a Subproject
What is the way to compile just a subproject of Hadoop with Maven (without rebuilding parent modules)? In my case I'm wondering about DistributedShell. I thought that thishttp://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772would have worked but no changes seem to take effect after building this way. Thanks, Curtis
