On 4 April 2014 14:41, Fraser Adams <[email protected]> wrote:
> On 04/04/14 13:30, Rob Godfrey wrote: > >> No... "install in the local repo" is what the mvn clean install will >> do... So, you literally just have to follow Robbie's instructions: check >> out the qpid-parent-pom/trunk directory, run mvn clean install from >> wherever you checked it out to... then go back to the broker and the maven >> build should complete ok. Hope this helps, Rob >> > Cheers Rob, > I ended up figuring out that myself after staring at the instructions for > a while and deciding that I was reading way too much into things :-) > > I've now got the Java Broker etc. built using Maven but before I play with > the QMF things I wouldn't mind answers to a few Maven questions - I'm still > very much at the "Burn the Witch" stage wrt. my trust of Maven ;-) > > I've now ended up with a directory > /home/fadams/.m2/repository > > Filled with stuff. > > I'm not clear by what witchcraft Maven decides what to shove there - Maven has the concept of a local repo and a remote repo, analogous to e.g. your installed packages in a linux distro, and the repository the package manager got them from. The default local repo localtion is ~/.m2/repository, and the default remote repo is maven central. Any dependencies it needs to download will go in the local repo, so it doenst necessarily need to download them again next time they are used (unles e.g they are snapshots and new versions came out). Additionally, anything you run "mvn install" on will go in the local repo configured at the time (can be overriden per-command for example) to allow it to be used by other things you build later. 'Any dependencies' above includes the dependencies of a particular artifact you are using, the [plugin] dependencies of anything needed by your build process to do what you are asking, and any [plugin] dependencies needed by maven itself. (Additional reading, there are different dependency scopes that can influence if/when they are needed/used etc: http://maven.apache.org/guides/introduction/introduction-to-dependency-mechanism.html#Dependency_Scope ) > for example I eyeballed the parent pom.xml and saw the dependencies, The parent pom actually has no real dependencies (except on its parent, the apache pom). What you likely saw was the pluginManagement and dependencyManagement sections, which help fix down and/or consistently specify the versions of particular dependencies and plugins that might be used by the child modules and their build steps. These dont actually mean things have a dependency on those artifacts, just which version etc they would get by default if they should happen to specify such a dependency or plugin to use. This helps keep things using consistent versions of artifacts, and in the case of plugins helps ensure reproducable builds by locking down the versions in use. > but when I looked at the repository after running mvn clean install on > that I was kind of expecting the directories in the repository to pretty > much match what was in the dependencies, but it definitely didn't after > running mvn on the parent pom.xml, though to be fair after building the > main things it does all appear to match (though with a ton of other stuff > too). > What you would have got after doing 'mvn install' on the qpid-parent pom would be mostly only the plugins needed by maven itself to carry out those actions. What they ship in the core maven release is augmented by downloading/installing things to the local repo as they are used both by itself and by people declaring they want to use a specific plugin in their build. > > When I build the Java Broker etc. it seemed to take an age, when I was > using ant it took under a minute on my system, but with Maven it reports a > total time of 5:18 min and it looks like it was downloading half the > Internet :-D I'm *guessing* that this is a one-off cost as it fills up my > local repo with stuff? > > Yes, the first time you would have incurred the cost of grabbing any plugins the build needed that you didnt already have and also the dependencies of the things being built (the later of which happens on your first Ant build too - go look in ~/.ivy2/cache, then try deleting it and see what happens to the Ant build time). Run it again and it will be a lot faster going forward. On my machine using the head of trunk (I just committed a bunch of tweaks earlier), 'ant clean build' vs 'mvn clean package -DskipTests=true' has the maven build being 1 or 2sec slower (32 vs 33/34), which is actually surprisingly close since using those commands we currenty have the maven build doing a bunch of extra packaging and enforcement that the ant build doesnt. It may also be quicker if told to build 'offline' to ensure it didnt check any repos for updates. (It takes a further 2mins 21sec on top of that for me to use the Ant build to generate the maven artifacts we currently publish to central...a generation step we obviously wont need to do for the maven build as they are already there inherantly.) > Where do you set CLASSPATH when you build using Maven? With the ant build > I used to have: > <qpid>qpid/java/build/lib/qpid-all.jar > > You dont set CLASSPATH, basically. Supplying dependency information is all handled through the poms, which maven uses to build the classpath for each module as it builds. If you want to build something else that uses e.g the qpid client, the thing your are building should have a dependency for qpid-client in its pom. You can make that dependency be either a published release [or snapshot] version available in central [or the apache snapshots repo], or you can 'mvn install' your own modified copy if you want to use an unpublished client release. If you dont want to use maven / want to use basic javac commands for the other thing you are compiling against the client, then you would need to make the client jars available somewhere and add the individual jars to your classpath currently. We aren't recreating qpid-all at this time because while it may be convenient in cases, it is actually a horrible horrible thing, which I'll talk a bit more to later when covering the build directory :) We have a qpid-all file in every release artifact made by the ant build, each having different content, and all different from the one that gets created in the build dir. Even if we wanted to recreate it in the maven build, we would need to give it a unique name for every component to ever publish it anyway. We could instead do something like add the classpath manifest entries previously contained in qpid-all.jar into the main qpid-client, qpid-broker etc jars so you could do 'java -jar qpid-foo' etc on them instead. > Which was nice and convenient, if possibly a bit sloppy, from what I can > see each qpid/java subdirectory seems to have a target directory e.g. > qpid/java/client/target though in there there is > qpid-client-0.28-SNAPSHOT.jar which seems less convenient that > qpid-client.jar if I want to set up CLASSPATH in my .bashrc. Am I missing > something? > > Another thing that I'm not clear on is that in the Maven repository there > seems to be a bunch of jars installed - as an example > org/eclipse/jetty/jetty-websocket/8.1.14.v20131031/jetty-websocket-8.1.14.v20131031.jar. > In the "olden days" Qpid pulled in jetty and the Jar was available in > qpid/java/build/lib so when I was messing around with Jetty on another > project I had it handily on my CLASSPATH - I can't seem to see jetty > anywhere now except in the Maven repository? So how does the Broker see it? > The broker sees it because it says it has a runtime dependency on the management-http module in its pom, and the management-http module says it has a compile time dependency on Jetty in its pom, and so maven then knows the broker transitively needs jetty. The actual jars never leave the local repo unless we instruct the build to do something that requires that (e.g. package a binary assembly containing it, or copying it somewhere) in the pom or you execute a command manually that does (e.g. a quick Google says 'mvn dependency:copy-dependencies' requests the dependency plugin to run its copy-dependencies goal, which create a dir in target/dependency with everything in it). The only reason the Ant build still puts things in the lib dir (by using Ivy to grab thigns from the maven central repo, i.e roughly the same thing Maven is doing to download things) is so that I didnt need to rewrite even more of the Ant build than I already was when I modified it to allow removing the jars from the lib dir, where they were actually committed into the svn repo historically (I refer you to my previous mails comment about the Ant build being a pain in the ass and me being the one best placed to speak best to this point :P). > In the olden days when I just did "ant" at the end of that I ended up with > something that would run 'cause I had > <qpid-trunk>qpid/java/build/bin > > on my path and could simply do "qpid-server" but now there's no nice > convenient build directory. > The existing build directory is horrible and should burn :P I'll admit, it can be convenient, but its also a complete jumble with a little bit of extra mess on top, mixing up all the different components together in lib so you cant tell which bits need what (qpid-all.jar in the build directory being worst of all, as you have everything on the classpath manifest in there) and doing ugly copies of stuff into subdirs of lib to seperate them for later use precisely because you cant tell which bits need what. I would have tried to kill it a long time ago if it didnt involve yet more 'rewriting the ant build' fun, which I just couldnt bring myself to subject me to :) Maven deals with modules, which each have their own target dir for their intermediate work and final output, and if you want something that aggregates multiple modules then there are varous ways to do that, avoiding the global build dir mess while still letting you compose a larger unit from the individual bits. It is certainly different than what our Ant build does, but in this case I think its a good thing. We can always do things to make life easier, more below. > I noticed in > qpid/java/broker/target/qpid-broker-0.28-SNAPSHOT-bin.tar.gz that archive > seems to contain qpid-server, do I *really* have to now have to faff around > copying and unpacking archives everytime I want to update a build? That > seems an awful lot less convenient than simply doing "ant clean" then "ant". > > Have I missed something? I hope so 'cause if not it seems a step backwards > from a "just works" POV > What you say is true *currently*, to run the broker built by the maven build you would need to unpack that tar file. We can certainly add functionality to remove that need, we just havent got to those final few 'nice to have' things yet. There are a multitude of ways we could do it, for example we could use a plugin and have the maven build itself start the broker (e.g see what i did with the 'tests' in the QMF tree: those actually run without even creating a jar file), or add some build config to allow output of an unpackaged version of it during the main build, or..<insert idea>. > > This might all be second nature to folks familiar with Maven, but I'm a > bit "old skool" and I quite like knowing what's installed on my system, > where it is installed, and why it's there so I'd quite like a bit of > reassurance :-) > > Frase > > > > --------------------------------------------------------------------- > To unsubscribe, e-mail: [email protected] > For additional commands, e-mail: [email protected] > >
