Fwd: maven 3.0.3 out of memory error, version ranges, lots of maven meta downloads
Can someone confirm on the maven users list that they have received this email? I did not but it does appear in the mail archive Thanks P Original Message Subject: maven 3.0.3 out of memory error, version ranges, lots of maven meta downloads Date: Mon, 20 Jun 2011 18:54:41 +0100 From: Paul French paul.fre...@kirona.com Organisation: Kirona To: users@maven.apache.org CC: Ian Jones ian.jo...@kirona.com Below is some filtered output from a maven build (showing maven meta data being downloaded for one artifact). All my dependencies use version ranges of the form [1.0.0.SNAPSHOT,2.0.0.SNAPSHOT) In general the build fails with out of memory. I can increase the heap size and the build is successful, but a relatively simple build is now requiring 500MB heap size. First question is why is maven re-checking and downloading the same meta data over and over again? If I add additional version ranged dependencies to my pom then the problem seems to get exponentially worse. For a larger build so many requests are made so quickly to the nexus remote repository that I start to see Error transferring file: Address already in use: connect. I can circumvent this problem by setting my repos to have an updatePolicy of interval:10 so during a build maven does not re-download the same maven meta data file. However not very complicated builds are still failing with out of memory even with 1024M of heap space. So this workaround just speeds up the inevitable out of memory error appearing. Any ideas how to reduce the memory usage? Do not say do not use version ranges. We use semantic versioning enforced by API analysis tools for each of our modules (OSGi bundles) so we require version ranges. If I change all version range dependencies to an exact dependency (a serious amount of pom editing) then all is fine. As you can tell below we have 3 main internal remote repos setup (kirona, kirona-snapshot and third.party.libraries). We do not specify version ranges on libraries that are not in our control e.g. log4j etc etc Downloading: http://fuji:8081/nexus/content/repositories/third-party/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml Downloaded: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml (354 B at 21.6 KB/sec) Downloaded: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml (318 B at 6.6 KB/sec) Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona-knopflerfish/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/third-party/com/kirona/server/third.party.libraries/maven-metadata.xml Downloaded: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml (318 B at 10.0 KB/sec) Downloaded: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml (354 B at 11.2 KB/sec) Downloading: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/third-party/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona-knopflerfish/com/kirona/server/third.party.libraries/maven-metadata.xml Downloaded: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml (318 B at 9.7 KB/sec) Downloaded: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml (354 B at 10.8 KB/sec) Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/serve... The above problems renders eclipse useless with m2e, it causes eclipse to hang on start up (build automatically set to true) and eventually die!! I'm starting to come to the conclusion that between maven and m2e version ranges just do not work very well. I hope I am wrong!! Thanks Ian
Re: Fwd: maven 3.0.3 out of memory error, version ranges, lots of maven meta downloads
To be clear the problem is in maven. I am running maven from the command line. Thanks P On 21/06/2011 11:16, Paul French wrote: Can someone confirm on the maven users list that they have received this email? I did not but it does appear in the mail archive Thanks P Original Message Subject: maven 3.0.3 out of memory error, version ranges, lots of maven meta downloads Date: Mon, 20 Jun 2011 18:54:41 +0100 From: Paul French paul.fre...@kirona.com Organisation: Kirona To: users@maven.apache.org CC: Ian Jones ian.jo...@kirona.com Below is some filtered output from a maven build (showing maven meta data being downloaded for one artifact). All my dependencies use version ranges of the form [1.0.0.SNAPSHOT,2.0.0.SNAPSHOT) In general the build fails with out of memory. I can increase the heap size and the build is successful, but a relatively simple build is now requiring 500MB heap size. First question is why is maven re-checking and downloading the same meta data over and over again? If I add additional version ranged dependencies to my pom then the problem seems to get exponentially worse. For a larger build so many requests are made so quickly to the nexus remote repository that I start to see Error transferring file: Address already in use: connect. I can circumvent this problem by setting my repos to have an updatePolicy of interval:10 so during a build maven does not re-download the same maven meta data file. However not very complicated builds are still failing with out of memory even with 1024M of heap space. So this workaround just speeds up the inevitable out of memory error appearing. Any ideas how to reduce the memory usage? Do not say do not use version ranges. We use semantic versioning enforced by API analysis tools for each of our modules (OSGi bundles) so we require version ranges. If I change all version range dependencies to an exact dependency (a serious amount of pom editing) then all is fine. As you can tell below we have 3 main internal remote repos setup (kirona, kirona-snapshot and third.party.libraries). We do not specify version ranges on libraries that are not in our control e.g. log4j etc etc Downloading: http://fuji:8081/nexus/content/repositories/third-party/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml Downloaded: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml (354 B at 21.6 KB/sec) Downloaded: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml (318 B at 6.6 KB/sec) Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona-knopflerfish/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/third-party/com/kirona/server/third.party.libraries/maven-metadata.xml Downloaded: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml (318 B at 10.0 KB/sec) Downloaded: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml (354 B at 11.2 KB/sec) Downloading: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/third-party/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml Downloading: http://fuji:8081/nexus/content/repositories/kirona-knopflerfish/com/kirona/server/third.party.libraries/maven-metadata.xml Downloaded: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/server/third.party.libraries/maven-metadata.xml (318 B at 9.7 KB/sec) Downloaded: http://fuji:8081/nexus/content/repositories/kirona-snapshots/com/kirona/server/third.party.libraries/maven-metadata.xml (354 B at 10.8 KB/sec) Downloading: http://fuji:8081/nexus/content/repositories/kirona/com/kirona/serve... The above problems renders eclipse useless with m2e, it causes eclipse to hang on start up (build automatically set to true) and eventually die!! I'm starting to come to the conclusion that between maven and m2e version ranges just do not work very well. I hope I am wrong!! Thanks Ian
