You can use one per build processor, that reduces concurrent updates but still keeps the cache function. And then try to avoid using install.
-- http://bernd.eckenfels.net -----Original Message----- From: Andrew Wang <[email protected]> To: "[email protected]" <[email protected]> Cc: Andrew Bayer <[email protected]>, Sangjin Lee <[email protected]>, Lei Xu <[email protected]>, [email protected] Sent: Fr., 18 Sep. 2015 20:42 Subject: Re: Local repo sharing for maven builds I think each job should use a maven.repo.local within its workspace like abayer said. This means lots of downloading, but it's isolated. If we care about download time, we could also bootstrap with a tarred .m2/repository after we've run a `mvn compile`, so before it installs the hadoop artifacts. On Fri, Sep 18, 2015 at 11:02 AM, Ming Ma <[email protected]> wrote: > +hadoop common dev. Any suggestions? > > > On Fri, Sep 18, 2015 at 10:41 AM, Andrew Bayer <[email protected]> > wrote: > > > You can change your maven call to use a different repository - I believe > > you do that with -Dmaven.repository.local=path/to/repo > > On Sep 18, 2015 19:39, "Ming Ma" <[email protected]> wrote: > > > >> Hi, > >> > >> We are seeing some strange behaviors in HDFS precommit build. It seems > >> like it is caused by the local repo on the same machine being used by > >> different concurrent jobs which can cause issues. > >> > >> In HDFS, the build and test of "hadoop-hdfs-project/hdfs" depend on > >> "hadoop-hdfs-project/hdfs-client"'s hadoop-hdfs-client-3.0.0- > >> SNAPSHOT.jar. HDFS-9004 adds some new method to > hadoop-hdfs-client-3.0.0-SNAPSHOT.jar. > >> In the precommit build for HDFS-9004, unit tests for > "hadoop-hdfs-project/hdfs" > >> complain the method isn't defined > >> https://builds.apache.org/job/PreCommit-HDFS-Build/12522/testReport/. > >> Interestingly sometimes it just works fine > >> https://builds.apache.org/job/PreCommit-HDFS-Build/12507/testReport/. > >> > >> So we are suspecting that there is another job running at the same time > >> that published different version of > hadoop-hdfs-client-3.0.0-SNAPSHOT.jar > >> which doesn't have the new methods defined to the local repo which is > >> shared by all jobs on that machine. > >> > >> If the above analysis is correct, what is the best way to fix the issue > >> so that different jobs can use their own maven local repo for build and > >> test? > >> > >> Thanks. > >> > >> Ming > >> > > >
