On Fri, Jul 26, 2013 at 3:47 PM, Richard Bair <richard.b...@oracle.com>wrote:
I'm assuming here you're talking about publishing real builds (at least
OpenJFX ones) and not on a local developers machine ('cause there'd be no
advantage to that alone). But maybe you can help me understand another part
of this problem, which is that suppose we have two developers, A and B. A
is on some code two weeks old. B is completely up to date. B does some fix
and pushes it. The build server builds the artifacts and puts them in the
repo. The next time A does a build, it grabs the latest built artifacts for
the code A isn't building (WebView, for instance) and there is a
compile/link error because the new binaries from B are out of sync with the
2 week old code that A is building with.
Normally you version for things like this, but in this case we're talking
about shared libraries that are unversioned -- they're SNAPSHOT. But one
snapshot is not equal to another. How to handle this? Right now in the
closed builds we have an explicit "ant update" step you have to run to get
the latest binaries.
I've had similar situations to what you're facing, where I might have a
number of core/utility libraries in an organisation and then lots of
projects that use these. If one team changes the utilities but the other
team doesn't want those changes, it gets messy.
I'll give you the overview of solutions I've come up to this and you can
take from it what you like. I haven't used Gradle though so I'll have to
talk in terms of Maven. Since Maven is more restrictive than Gradle you can
hopefully extract what you need. Others might be able to chime in more or
different opinions (I'd love that personally).
In Maven, you would have each of your modules as a separate Maven module,
with it's own POM and it's own coordinates (where it is deployed to in the
repo). e.g. using groupId:artifactId:version you would have stuff like:
- net.java.openjfx:openjfx-base:8.0.1
- net.java.openjfx:openjfx-controls:8.0.1
- etc
Each of the native modules would also be their own modules with their own
POM files and coordinates, so you would have:
- net.java.openjfx:jsl-decora:8.0.1
- net.java.openjfx:jsl-prism:8.0.1
- net.java.openjfx:native-font:8.0.1
- etc
Maven typically works best when you group these in a folder hierarchy,
with parent POMs as needed. Gradle is more flexible, but I suspect it would
still benefit from a the standard hierarchy here, this could look something
like (cut down, just indicative):
- openjfx
- base
- controls
- graphics
- graphics-core
- jsl-decora
- jsl-prism
Each of those directories would have it's own POM and in the leaf cases a
src/main/java (or native equivalent such as src/main/c++). I've introduced
graphics-core, since currently graphics is one big blob of Java and native
artifacts, making it hard to work with and deploy, etc. It works better if
they are separate and in Maven a parent module (such as graphics) would not
typically have code in it as well.
This directory structure allows you to build each module, or group of
modules stand alone. You could go into the controls directory and run 'mvn
clean install' and it would build only that, or into the graphics directory
and build only the graphics child modules. The top level openjfx module
would have no source code but would just provide an uber parent where you
could just build all the sub-modules in one command mvn clean install
(maybe adding a -Pwin64 to trigger profiles to build the OS specific
versions).
Each of these modules would then be deployed to its own unique coordinates
in your Maven repo (your self-hosted artifactory or whatever). So
jsl-decora is in its own directory in the repo and is then referenced as a
dependency by graphics-core (or whatever needs to use it). Additionally the
JavaDoc and source code for each module would be deployed with each. Each
module is a totally stand-alone deliverable - even if it only exists to be
used inside a bigger module.
This is where it gets nice, since if jsl-decora is available in a repo I
have access to, I never need to build it, Maven will just pick up that one.
Similar for all the pure Java modules as well - if I just want to build
'controls' but not 'base', Maven will pick up base from the repo. This is
good modularisation - you have this on the code level now but not on your
build level as far as I can see.
Additionally you can open an individual POM file in IntelliJ or Eclipse as
its own project. So you could open just the 'controls' project and it
should be able to build and run the unit tests, etc, of just that module.
Or if you open the graphics parent module it would open all the child
projects as well. And opening the top level openjavafx POM, would open all
the child projects and descendants for everything. (As an aside, with Maven
I never check in my intellij project files, and tend to get my developers
to open the POM file directly - the POM file becomes the source of truth
and is IDE agnostic).
Obviously this would all make it massively easier for contributors, since
we often would be playing only in one module (like controls), so the build
would be simple and quick, drawing bits we aren't messing with straight out
of your repo, and we never even need to open these other bits in our IDE.
This is extra useful given all the nasty native stuff that JFX has - as a
contributor I don't want to have to build 'glass' just to add a ComboBox
fix, etc.
If you have really good clean modularisation, then versioning gets easier.
I don't really know how your internal build works, how often it happens,
etc, and how you structure milestones. I've heard Agile mentioned but I'd
guess it's psuedo-agile. But to give us something to work with, let's
assume a pretty standard Agile practice, maybe something like an automated
nightly build that compiles all the source code for all the platforms and
runs the unit tests, and reports back any failures. Then at the end of a
milestone you have a manually triggered build that does all of the same
again, tags it in SVN, and your testing team then runs their full set of
integration/regression tests on this. Maybe the reality is a bit different
for you guys, but hopefully its close enough you can extrapolate.
My strategy here would likely be to have the automated nightly builds
deploy a SNAPSHOT release (say 8.0.1.SNAPSHOT) which is always the latest
and greatest of 8.0.1. I would then also have these automated builds deploy
the exact same code as a versioned build - probably something like
8.0.1-2013-07-29 where the last bit is the date it was deployed. Note that
when I say 'deploy' here, I mean into your own self-hosted Maven repo
(Artifactory, or whatever you want). You wouldn't deploy nightly builds
into Maven Central.
By default, the checked-in code would reference the SNAPSHOT version, so
in every POM file that referenced another module (e.g. controls would have
a dependency onto base), it would use 8.0.1.SNAPSHOT as the version (or
more likely a variable like ${project.version}). Anyone who checks out and
builds would automatically be building against the latest, which is what
you would expect.
Ideally everyone should be constantly updating their code and making sure
it compiles against the latest always. You mention two week old code - this
isn't exactly something you'd expect to see in an Agile process. Agile
tasks are typically 4 to 16 hours and the conclusion of each should involve
a check-in and update to the latest and then be included in the nightly
build and unit tested against, etc. If a task is bigger than that, you're
probably drifting from Agile best practices (and typically would break your
task down into smaller chunks). Your nightly builds should always run off
head and compile so everyone should be working towards that for their
milestone (and milestones should be 1 to 4 weeks in a good agile process,
with all code checked in by the end of the milestone, then tagged and
released for user feedback).
But assuming you had reasons for these out of date or very long tasks,
then to pin your code to a particular version, there are a couple of
options.
The first option is the simplest: just have the entire openjfx project
open in your IDE. When you have a module open and included in the IDE build
(in IntelliJ at least, I don't know what they others do) then your copy of
the code is always used. Even if a newer snapshot is deployed, it's not
used by IntelliJ (I'm pretty sure of this, but it could use double
checking). So if you have all the modules open (i.e. you opened the top
level openjfx POM, which is what I assume most of you do) then you control
what versions you are using, simply by controlling the source control
updates. If you don't want a newer version of 'base' don't update it from
SVN or update to a specific SVN revision.
The second option is more explicit. Just change the dependencies in your
POMs between modules to reference a specific version. So if you are on the
controls team and you need to hook into a specific version of base, you
just edit the controls POM and where it references base as a dependency put
in the version number for the day you want to pin to, e.g. version
8.0.1-2013-07-29. You are then guaranteed to be fixed against that version
and never have to worry about a change being forced on you.
The versioned nightly builds are purely to allow for the second option. If
you didn't need the second option, you could just use snapshots. If you do
want these versioned nightly builds then they are going to build up over
time and given the size of JFX this might get to be a problem after a
while. Personally I would keep these daily versions around only for a
milestone. At the end of a milestone I would do a formal milestone release,
like 8.0.1-m23 (where m23 is the milestone number) and deploy that version.
People really shouldn't be sticking to older code for long (it's not
healthy) so I would delete all the old daily builds (you can easily do this
in your own hosted repository, unlike Maven Central). Anyone who really
needs to stick to that milestone would at least have to upgrade to the
final version of that milestone. If this is too harsh for your needs, you
could keep the last 3 (or 5 or 20 or 100) milestones worth of nightly
builds.
I assume you do several internal milestones releases for each actual
release to the public. If so, I probably wouldn't release milestones to
Maven Central but I'd consider putting each 'release' into there (e.g.
ea-b96 is a release). Or maybe you just wait until you get out of 'ea' and
do actual formal releases such as 8.0.1. These non-ea releases definitely
should be synched to Maven Central - this is the bit where we run into
trouble with Oracle policy on self-hosting, but if you had your own private
repo where you were releasing as per all of the above, with POMs all setup
and source code and javadoc included, it would be a fairly trivial task for
us in the community to then deploy your formal releases to Maven Central
for you.
If any of that is unclear or needs further elaboration, let me know and
I'll do my best. It's a lengthy topic and difficult to talk through like
this.
As usual this is all my experience, my opinions and my info that I have.
Take and use anything you want, ignore the rest. And if people out there
have better ways of doing things, I'd be just as keen as anyone to know
about them.
