I should clarify that the build doesn't actually fail (so that bug doesn't
seem to be the issue). It reports success. But then the output from the
just-build jar doesn't change.
Here's my output from building:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
clean package -DskipTests
[INFO] Scanning for projects...
[INFO]
[INFO]
------------------------------------------------------------------------
[INFO] Building hadoop-yarn-applications-distributedshell 3.0.0-SNAPSHOT
[INFO]
------------------------------------------------------------------------
[INFO]
[INFO] --- maven-clean-plugin:2.4.1:clean (default-clean) @
hadoop-yarn-applications-distributedshell ---
[INFO] Deleting
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target
[INFO]
[INFO] --- maven-antrun-plugin:1.6:run (create-testdirs) @
hadoop-yarn-applications-distributedshell ---
[INFO] Executing tasks
main:
[mkdir] Created dir:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-dir
[INFO] Executed tasks
[INFO]
[INFO] --- maven-resources-plugin:2.2:resources (default-resources) @
hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:compile (default-compile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 3 source files to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\classes
[INFO]
[INFO] --- maven-resources-plugin:2.2:testResources (default-testResources)
@ hadoop-yarn-applications-distributedshell ---
[INFO] Using default encoding to copy filtered resources.
[INFO]
[INFO] --- maven-compiler-plugin:2.5.1:testCompile (default-testCompile) @
hadoop-yarn-applications-distributedshell ---
[INFO] Compiling 1 source file to
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\test-classes
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default) @
hadoop-yarn-applications-distributedshell ---
[INFO] Building jar:
C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell\target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
[INFO]
[INFO] --- maven-surefire-plugin:2.12.3:test (default-test) @
hadoop-yarn-applications-distributedshell ---
[INFO] Tests are skipped.
[INFO]
[INFO] --- maven-jar-plugin:2.3.1:jar (default-jar) @
hadoop-yarn-applications-distributedshell ---
[INFO]
[INFO] --- maven-site-plugin:3.0:attach-descriptor (attach-descriptor) @
hadoop-yarn-applications-distributedshell ---
[INFO]
------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO]
------------------------------------------------------------------------
[INFO] Total time: 3.487s
[INFO] Finished at: Fri Jun 21 16:08:02 PDT 2013
[INFO] Final Memory: 19M/367M
[INFO]
------------------------------------------------------------------------
On Fri, Jun 21, 2013 at 3:57 PM, Arpit Agarwal <aagar...@hortonworks.com>wrote:
> Curtis, if you are running 'mvn clean' in a subproject you may be hitting
> this bug:
> https://issues.apache.org/jira/browse/HADOOP-9383
>
> You might have better luck if you include install along with package in
> the same invocation.
>
> -Arpit
>
>
> On Fri, Jun 21, 2013 at 3:12 PM, Curtis Ullerich
> <curtuller...@gmail.com>wrote:
>
>> I've executed the commands as you've said, and the jar that is run is
>> unchanged. Here's exactly what I did.
>>
>> C:\l\> start-dfs
>> C:\l\> start-yarn
>> C:\l\> cd
>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
>> org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...output...
>> ...modify logging code in
>> C:/l/hadoop-yarn-applications-distributedshell/src/main/java/org/apache/hadoop/yarn/applications/distributedshell/Client.java
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>mvn
>> clean install -DskipTests
>>
>> C:\l\hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell>hadoop
>> org.apache.hadoop.yarn.applications.distributedshell.Client -jar
>> target\hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar
>> -shell_command whoami
>> ...identical output to before...
>>
>> Do you see anything wrong with what I've done?
>>
>> Thanks,
>> Curtis
>>
>>
>>
>>
>> On Thu, Jun 20, 2013 at 7:17 PM, Omkar Joshi <ojo...@hortonworks.com>wrote:
>>
>>> Hi Curtis,
>>>
>>> where are you picking your jar file from? once you run above command you
>>> will see the updated jar file in
>>>
>>> "..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/target/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>> I hope you are not using below jar file
>>>
>>> "..../hadoop-dist/target/hadoop-3.0.0-SNAPSHOT/share/hadoop/yarn/hadoop-yarn-applications-distributedshell-3.0.0-SNAPSHOT.jar"
>>>
>>>
>>> What chris has said is right.
>>>
>>> once you have taken latest code, you should follow below steps
>>> 1) mvn clean install -DskipTests (clean to remove previously generated
>>> code)
>>> 2) now say you are updating distributed shell client code. then go to
>>> ""..../hadoop-yarn-project/hadoop-yarn/hadoop-yarn-applications/hadoop-yarn-applications-distributedshell/"
>>> and then run "mvn clean install" and use jar from target sub folder.
>>>
>>> Thanks,
>>> Omkar Joshi
>>> *Hortonworks Inc.* <http://www.hortonworks.com>
>>>
>>>
>>> On Thu, Jun 20, 2013 at 11:47 AM, Curtis Ullerich <
>>> curtuller...@gmail.com> wrote:
>>>
>>>> Hi Chris,
>>>>
>>>> I really appreciate the response. What you described is what I
>>>> initially tried. The changes never seem to take effect though. Here's what
>>>> I've done (this is Windows):
>>>>
>>>> cd %hadoop_install_dir%
>>>> mvn clean package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> --- modify the code in distributed shell's Client.java ---
>>>>
>>>>
>>>> cd
>>>> hadoop-yarn-project\hadoop-yarn\hadoop-yarn-applications\hadoop-yarn-applications-distributedshell
>>>> mvn clean
>>>> mvn package -DskipTests
>>>> mvn install -DskipTests
>>>>
>>>> Then I run the jar, just as before. I've just been changing log
>>>> statements to see if the changes worked. They don't--the output doesn't
>>>> change after doing this rebuild. I've also tried clearing the files put on
>>>> HDFS in my user directory in case that was the issue. I've taken a more
>>>> thorough look at BUILDING.txt and I seemed to be consistent with the
>>>> procedures described there. Am I missing anything else? I've tried
>>>> restarting yarn and dfs, though I didn't think that would matter.
>>>>
>>>> Thanks,
>>>> Curtis
>>>>
>>>>
>>>>
>>>>
>>>> On Thu, Jun 20, 2013 at 11:17 AM, Chris Nauroth <
>>>> cnaur...@hortonworks.com> wrote:
>>>>
>>>>> Hi Curtis,
>>>>>
>>>>> I handle this by running "mvn install -DskipTests" once from the root
>>>>> of the whole hadoop project to install the sub-modules in my local Maven
>>>>> repository. Then, you can change your working directory to any sub-module
>>>>> (like distributed shell) and run a successful Maven build for just that
>>>>> sub-module. Full details are in the BUILDING.txt file in the root of the
>>>>> hadoop project. Look for the section titled "Building components
>>>>> separately". I hope this helps.
>>>>>
>>>>> Chris Nauroth
>>>>> Hortonworks
>>>>> http://hortonworks.com/
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Jun 19, 2013 at 4:53 PM, Curtis Ullerich <
>>>>> curtuller...@gmail.com> wrote:
>>>>>
>>>>>> What is the way to compile just a subproject of Hadoop with Maven
>>>>>> (without rebuilding parent modules)? In my case I'm wondering about
>>>>>> DistributedShell. I thought that
>>>>>> this<http://stackoverflow.com/questions/1114026/maven-modules-building-a-single-specific-module/3899772#3899772>would
>>>>>> have worked but no changes seem to take effect after building this
>>>>>> way.
>>>>>>
>>>>>> Thanks,
>>>>>> Curtis
>>>>>>
>>>>>
>>>>>
>>>>
>>>
>>
>
