Re: Not able to read jars in repo intermittently
A week ago I asked about an issue where multiple builds were downloading to the shared local repo and causing corruption in random jars. My company's Jenkins team are offering the solution of running my own private repo to avoid this, but this radically increases build times because each build has to download all required jars to the local repo from our proxy. The Jenkins team have set up their build servers to run on 4 hosts. I don't know much about Jenkins but I figure that it must be automated - is there a maven operation they could carry out to clone at least the most heavily used artefacts in the Artifactory repo to the local hard drive? That would eliminate the individual build job clashes on the jars in the local repo like commons-io. -- Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Not able to read jars in repo intermittently
Can I just verify the problem? These affected jars are frequently used, and have been there for a long time, e.g. /home/jenkins/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar There should be no writes going on to this or other affected jars by any processes on the maven repo. Are these errors are caused by poor file handling by the underlying OS (Windows here), or is it maven itself? Is there a known issue for what's being discussed? Regards Adam -- Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Not able to read jars in repo intermittently
For the record, it seems like the "EXECUTOR_NUMBER" variable ( https://issues.jenkins-ci.org/browse/JENKINS-31391 ) should be available to be able to easily specify a per-executor repo. So something such as this should do the job: mvn -Dmaven.repo.local=$JENKINS_HOME/.m2/$EXECUTOR_NUMBER/ ... -- Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Not able to read jars in repo intermittently
I regularly have the same problem on my Jenkins build server: either "corrupted zip file" or "cannot find symbol" and the only explanation for me was concurrent build. I has never thought of having Maven local repo specific to an executor, thanks Bernd Eckenfels, that's a really good point ! I was really worried about the *huge* disk impact on disk space per-job repositories would have but had never thought about per-executor ! Thanks. To tell you the truth, I was thinking about a system to have a file-system layer to mutualize all the non-SNAPSHOT jars and another per-job layer to store the SNAPSHOT jars... whatever, this would have been complicated and the per-executor seems like a good trade-of between simplicity and disk space. Francois -- Sent from: http://maven.40175.n5.nabble.com/Maven-Users-f40176.html - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Not able to read jars in repo intermittently
UIt is correct that maven does not like it when your run install jobs concurrently with jobs reading the same artifacts. In order to avoid this you can have a workspace specific Repository („Local to workspace“ this might have some advantages for isolating different jobs but consumes a lot of disk) or you can configure the local maven Repository to be unique for the jenkins built executors (“Local to Executor“). This way you have only a fixed multiplier for disk space usage. What you also can do is to reduce the need to install (snapshots) at the end of a built job. For example by only using deploy or have Jenkins deploy the resulting artifacts After verify. Gruss Bernd -- http://bernd.eckenfels.net From: Adam Hardy <adam.ha...@cyberspaceroad.com> Sent: Monday, March 26, 2018 7:44:17 PM To: users@maven.apache.org Subject: Not able to read jars in repo intermittently I use a Jenkins server which is administered by another team. I get the following errors intermittently. The team claim the issue is "the maven repository is not multi write safe. It's possible several jobs are updating the jar files at the same time and corrupt them. If that's the case, the only option available at the moment is to use a repository local to the workspace" These are reads, and the writes to create these jars will only ever be done once - isn't that correct? I have my own settings.xml to use but I have not configured a local repo for my own project because i thought that would not be my responsibility as a jenkins client. Is there an obvious source of this error, and alternatively should I configure my own repo in my settings.xml? Here's the error: [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] error reading /home/jenkins/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-pool/commons-pool/1.6/commons-pool-1.6.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-pool/commons-pool/1.6/commons-pool-1.6.jar; error in opening zip file [ERROR] /home/jenkins/workspace/rdis_gem-tardis-rest_master-V5NBFAKC4NMCAXWRJ7MGOI5IW5OWL6UESQFP6S4T4AW2CRGDV65A/src/main/java/com/bp/gem/util/servlet/RequestLoggingInnerFilter.java:[6,36] package org.apache.commons.io.output does not exist [ERROR] /home/jenkins/workspace/rdis_gem-tardis-rest_master-V5NBFAKC4NMCAXWRJ7MGOI5IW5OWL6UESQFP6S4T4AW2CRGDV65A/src/main/java/com/bp/gem/util/servlet/RequestLoggingInnerFilter.java:[243,23] cannot find symbol symbol: class TeeOutputStream location: class com.bp.gem.util.servlet.RequestLoggingInnerFilter.TeeServletOutputStream [ERROR] /home/jenkins/workspace/rdis_gem-tardis-rest_master-V5NBFAKC4NMCAXWRJ7MGOI5IW5OWL6UESQFP6S4T4AW2CRGDV65A/src/main/java/com/bp/gem/util/servlet/RequestLoggingInnerFilter.java:[246,32] cannot find symbol symbol: class TeeOutputStream location: class com.bp.gem.util.servlet.RequestLoggingInnerFilter.TeeServletOutputStream [INFO] 9 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 3.216 s [INFO] Finished at: 2018-03-23T15:12:08+00:00 [INFO] Final Memory: 32M/366M [INFO] - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
Re: Not able to read jars in repo intermittently
What jars are you overwriting in your Maven repo? The SNAPSHOT system allows things to be written in at the same time as other jobs depending on the previous version. Is this a problem with SNAPSHOT jars or non SNAPSHOT jars? It looks like you're saying non SNAPSHOT: org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar being your example. Why do you need to publish that more than once ever? - Paul
Not able to read jars in repo intermittently
I use a Jenkins server which is administered by another team. I get the following errors intermittently. The team claim the issue is "the maven repository is not multi write safe. It's possible several jobs are updating the jar files at the same time and corrupt them. If that's the case, the only option available at the moment is to use a repository local to the workspace" These are reads, and the writes to create these jars will only ever be done once - isn't that correct? I have my own settings.xml to use but I have not configured a local repo for my own project because i thought that would not be my responsibility as a jenkins client. Is there an obvious source of this error, and alternatively should I configure my own repo in my settings.xml? Here's the error: [INFO] - [ERROR] COMPILATION ERROR : [INFO] - [ERROR] error reading /home/jenkins/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-pool/commons-pool/1.6/commons-pool-1.6.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/org/hamcrest/hamcrest-core/1.3/hamcrest-core-1.3.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-io/commons-io/2.4/commons-io-2.4.jar; error in opening zip file [ERROR] error reading /home/jenkins/.m2/repository/commons-pool/commons-pool/1.6/commons-pool-1.6.jar; error in opening zip file [ERROR] /home/jenkins/workspace/rdis_gem-tardis-rest_master-V5NBFAKC4NMCAXWRJ7MGOI5IW5OWL6UESQFP6S4T4AW2CRGDV65A/src/main/java/com/bp/gem/util/servlet/RequestLoggingInnerFilter.java:[6,36] package org.apache.commons.io.output does not exist [ERROR] /home/jenkins/workspace/rdis_gem-tardis-rest_master-V5NBFAKC4NMCAXWRJ7MGOI5IW5OWL6UESQFP6S4T4AW2CRGDV65A/src/main/java/com/bp/gem/util/servlet/RequestLoggingInnerFilter.java:[243,23] cannot find symbol symbol: class TeeOutputStream location: class com.bp.gem.util.servlet.RequestLoggingInnerFilter.TeeServletOutputStream [ERROR] /home/jenkins/workspace/rdis_gem-tardis-rest_master-V5NBFAKC4NMCAXWRJ7MGOI5IW5OWL6UESQFP6S4T4AW2CRGDV65A/src/main/java/com/bp/gem/util/servlet/RequestLoggingInnerFilter.java:[246,32] cannot find symbol symbol: class TeeOutputStream location: class com.bp.gem.util.servlet.RequestLoggingInnerFilter.TeeServletOutputStream [INFO] 9 errors [INFO] - [INFO] [INFO] BUILD FAILURE [INFO] [INFO] Total time: 3.216 s [INFO] Finished at: 2018-03-23T15:12:08+00:00 [INFO] Final Memory: 32M/366M [INFO] - To unsubscribe, e-mail: users-unsubscr...@maven.apache.org For additional commands, e-mail: users-h...@maven.apache.org
