On Jun 22, 2011, at 12:09 PM, Russell E Glaue wrote:
> I am wanting to update the G3 dev wiki to help first-time builders get
> started.
> How would you suggest a build of G3 could become faster?
>
> The only tip I am aware of for speeding up trunk (G3) builds is to install a
> maven repository proxy like nexus. This is eluded to in the wiki already.
> Are there any other tips?
You can build without tests. E.g.: 'mvn clean install -DskipTests=true'. If I'm
simply running a build, I'll frequently turn off tests ('-DskipTests=true' I
think '-DskipTests' is equivalent). If I'm making code changes, I run with
tests.
You can also build in an offline mode (e.g. 'mvn -o clean install
-DskipTests'). This avoids searching of maven repos, all together. So, you do
need one full online build, before this will work. Due to the current
development work going on in OpenWebBeans, OpenEJB, Aries, and Geronimo (I
think these are the most frequently changing projects, ATM), you run the risk
of getting out-of-date between the projects.
FYI, we are running Continuous Integration builds. E.g. --
http://ci.apache.org/builders/geronimo-server-trunk/builds/106
I encourage people to build from source. However, if you really want pre-built
binaries of unreleased versions of Geronimo, assemblies from our CI builds are
uploaded to the apache snapshot repository. So, it is possible to download
SNAPSHOT builds from the SNAPSHOT repository. For example:
https://repository.apache.org/content/repositories/snapshots/org/apache/geronimo/assemblies/geronimo-tomcat7-javaee6/3.0-SNAPSHOT/
>
>
> I have been able to successfully build G3 from trunk several times now but
> with
> differing build time results.
>
> The best build time I have had is 13 hours.
> The worst build time I have had is 39 hours. (this is from my latest build)
Ouch. My build times are nowhere near this long. Are you running with an empty
local maven repository?
>
> I am using a very basic build procedure.
> 1. checkout G3 trunk
> 2. JAVA_HOME="/path/to/jdk/1.6.0_22"
> 3. download and install a fresh maven 2.2.1
> 4. MAVEN_OPTS="-Xmx968m -XX:MaxPermSize=500m -XX:ReservedCodeCacheSize=64m"
My settings are similar. I don't specify '-XX:ReservedCodeCacheSize=64m'.
I run with the following. I know there are others running with larger max
heaps. I haven't :
MAVEN_OPTS="-Xmx1024m -XX:MaxPermSize=512m -XX:+HeapDumpOnOutOfMemoryError"
> 5. mvn clean install
> 6. 13 to 39 hours later == BUILD SUCCESS
>
>
> From observing the build process, it would seem the most used (seemingly
> paused)
> time is right before the build process moves on to the next plugin or
> assembly.
>
> This "seemingly paused time" after the assembly I assume is the packaging
> process, because the resulting output is a huge collection.
>
> [INFO] Geronimo Assemblies ............................... SUCCESS [3.995s]
> [INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [8:15.210s]
> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8 SUCCESS
> [1:57:17.311s]
> [INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS
> [1:02:47.374s]
> [INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [8:08.838s]
> [INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7 SUCCESS
> [1:59:16.619s]
> [INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS
> [1:08:13.854s]
>
> That is ~1.5 hours for the "Java EE 6 Web Profile + Jetty8" and "Java EE 6 Web
> Profile + Tomcat7" assemblies.
Here are my build times. This is running without tests. Which would affect the
overall build time. Assembly times would be unaffected -- we don't run any
tests when building assemblies.
[INFO] Geronimo Assemblies ............................... SUCCESS [0.163s]
[INFO] Geronimo Assemblies :: Minimal + Jetty8 ........... SUCCESS [42.253s]
[INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Jetty8 SUCCESS
[1:04.295s]
[INFO] Geronimo Assemblies :: Java EE 6 + Jetty8 ......... SUCCESS [1:33.097s]
[INFO] Geronimo Assemblies :: Minimal + Tomcat7 .......... SUCCESS [41.587s]
[INFO] Geronimo Assemblies :: Java EE 6 Web Profile + Tomcat7 SUCCESS
[1:14.409s]
[INFO] Geronimo Assemblies :: Java EE 6 + Tomcat7 ........ SUCCESS [1:46.333s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 28:15.199s
[INFO] Finished at: Wed Jun 22 12:38:10 EDT 2011
[INFO] Final Memory: 633M/1011M
[INFO] ------------------------------------------------------------------------
>
>
> For the plugins, I would assume the same, except that I notice that, of
> course,
> the latest snapshot dependencies are being downloaded, and the poms in the
> maven
> SNAPSHOT repositories are continually being accessed.
>
> I realize that because trunk (G3) largely depends on 3rd-party artifact
> SNAPSHOTS that these repositories are checked during every build, and new
> SNAPSHOT libraries are always being downloaded (as is the nature of snapshot
> releases).
Correct. A local maven proxy (e.g. Nexus) can reduce the overhead.
>
> As it is not a simple process for a user to setup a maven repo proxy, and
> could
> be a deterrent to someone getting started if it were necessary to install a
> proxy, I am looking at other methods and tips that might increase the speed of
> building G3.
I've tried running with a local maven proxy. However, I've found it to be too
much trouble to maintain/configure.
>
> Is this feasible? Or would we just say, that because of the nature of SNAPSHOT
> builds and their dependencies on 3rd-party SNAPSHOT artifacts, there is no
> real
> means to improving build speed unless a proxy is utilized?
> If the later is true, I would like to add this note in the wiki's build
> documentation. And also warn those who choose to build without a proxy may
> experience a long build time.
While I'm thinking about it -- what is your network environment? The focus on
proxies imply that your slow build times are predominantly network issue. Are
you certain that is the case?
--kevan
