And one other thing I forgot to mention on this topic, the practice of
having platform specific jars that contain both the non-platform specific
stuff (usually 90%+) and the platform specific stuff does not fit that well
into Maven repo deployment.
Maven would prefer to have something like
- jfxrt.jar - contains all the non-platform specific code
- jfxrt-win.jar - only contains the additional bits for windows
- jfxrt-osx.jar - only contains the additional bits for OSx
- etc
You would then deploy these all under net.java.openjfx:jfxrt:8.0.1 but the
main jar would have no classifier and the other jars would have a
classifier for their platform. The source (and javadoc) however would be
one zip for all them (deployed under the same coordinates but with the
classifier 'source' or 'javadoc').
Currently I'm stuck in the deployment of the 78 backport as Sonatype won't
let me close a release without a non-classified JAR but we have only
operating specific ones that I am deploying with a classifier. I'm waiting
to hear from Sonatype what they recommend in this case,
On Mon, Jul 29, 2013 at 11:34 PM, Daniel Zwolenski <zon...@gmail.com> wrote:
> On Fri, Jul 26, 2013 at 3:47 PM, Richard Bair <richard.b...@oracle.com>wrote:
>
> I'm assuming here you're talking about publishing real builds (at least
>> OpenJFX ones) and not on a local developers machine ('cause there'd be no
>> advantage to that alone). But maybe you can help me understand another part
>> of this problem, which is that suppose we have two developers, A and B. A
>> is on some code two weeks old. B is completely up to date. B does some fix
>> and pushes it. The build server builds the artifacts and puts them in the
>> repo. The next time A does a build, it grabs the latest built artifacts for
>> the code A isn't building (WebView, for instance) and there is a
>> compile/link error because the new binaries from B are out of sync with the
>> 2 week old code that A is building with.
>>
>> Normally you version for things like this, but in this case we're talking
>> about shared libraries that are unversioned -- they're SNAPSHOT. But one
>> snapshot is not equal to another. How to handle this? Right now in the
>> closed builds we have an explicit "ant update" step you have to run to get
>> the latest binaries.
>>
>
> I've had similar situations to what you're facing, where I might have a
> number of core/utility libraries in an organisation and then lots of
> projects that use these. If one team changes the utilities but the other
> team doesn't want those changes, it gets messy.
>
> I'll give you the overview of solutions I've come up to this and you can
> take from it what you like. I haven't used Gradle though so I'll have to
> talk in terms of Maven. Since Maven is more restrictive than Gradle you can
> hopefully extract what you need. Others might be able to chime in more or
> different opinions (I'd love that personally).
>
> In Maven, you would have each of your modules as a separate Maven module,
> with it's own POM and it's own coordinates (where it is deployed to in the
> repo). e.g. using groupId:artifactId:version you would have stuff like:
>
> - net.java.openjfx:openjfx-base:8.0.1
> - net.java.openjfx:openjfx-controls:8.0.1
> - etc
>
> Each of the native modules would also be their own modules with their own
> POM files and coordinates, so you would have:
>
> - net.java.openjfx:jsl-decora:8.0.1
> - net.java.openjfx:jsl-prism:8.0.1
> - net.java.openjfx:native-font:8.0.1
> - etc
>
> Maven typically works best when you group these in a folder hierarchy,
> with parent POMs as needed. Gradle is more flexible, but I suspect it would
> still benefit from a the standard hierarchy here, this could look something
> like (cut down, just indicative):
>
> - openjfx
> - base
> - controls
> - graphics
> - graphics-core
> - jsl-decora
> - jsl-prism
>
>
> Each of those directories would have it's own POM and in the leaf cases a
> src/main/java (or native equivalent such as src/main/c++). I've introduced
> graphics-core, since currently graphics is one big blob of Java and native
> artifacts, making it hard to work with and deploy, etc. It works better if
> they are separate and in Maven a parent module (such as graphics) would not
> typically have code in it as well.
>
> This directory structure allows you to build each module, or group of
> modules stand alone. You could go into the controls directory and run 'mvn
> clean install' and it would build only that, or into the graphics directory
> and build only the graphics child modules. The top level openjfx module
> would have no source code but would just provide an uber parent where you
> could just build all the sub-modules in one command mvn clean install
> (maybe adding a -Pwin64 to trigger profiles to build the OS specific
> versions).
>
> Each of these modules would then be deployed to its own unique coordinates
> in your Maven repo (your self-hosted artifactory or whatever). So
> jsl-decora is in its own directory in the repo and is then referenced as a
> dependency by graphics-core (or whatever needs to use it). Additionally the
> JavaDoc and source code for each module would be deployed with each. Each
> module is a totally stand-alone deliverable - even if it only exists to be
> used inside a bigger module.
>
> This is where it gets nice, since if jsl-decora is available in a repo I
> have access to, I never need to build it, Maven will just pick up that one.
> Similar for all the pure Java modules as well - if I just want to build
> 'controls' but not 'base', Maven will pick up base from the repo. This is
> good modularisation - you have this on the code level now but not on your
> build level as far as I can see.
>
> Additionally you can open an individual POM file in IntelliJ or Eclipse as
> its own project. So you could open just the 'controls' project and it
> should be able to build and run the unit tests, etc, of just that module.
> Or if you open the graphics parent module it would open all the child
> projects as well. And opening the top level openjavafx POM, would open all
> the child projects and descendants for everything. (As an aside, with Maven
> I never check in my intellij project files, and tend to get my developers
> to open the POM file directly - the POM file becomes the source of truth
> and is IDE agnostic).
>
> Obviously this would all make it massively easier for contributors, since
> we often would be playing only in one module (like controls), so the build
> would be simple and quick, drawing bits we aren't messing with straight out
> of your repo, and we never even need to open these other bits in our IDE.
> This is extra useful given all the nasty native stuff that JFX has - as a
> contributor I don't want to have to build 'glass' just to add a ComboBox
> fix, etc.
>
> If you have really good clean modularisation, then versioning gets easier.
>
> I don't really know how your internal build works, how often it happens,
> etc, and how you structure milestones. I've heard Agile mentioned but I'd
> guess it's psuedo-agile. But to give us something to work with, let's
> assume a pretty standard Agile practice, maybe something like an automated
> nightly build that compiles all the source code for all the platforms and
> runs the unit tests, and reports back any failures. Then at the end of a
> milestone you have a manually triggered build that does all of the same
> again, tags it in SVN, and your testing team then runs their full set of
> integration/regression tests on this. Maybe the reality is a bit different
> for you guys, but hopefully its close enough you can extrapolate.
>
> My strategy here would likely be to have the automated nightly builds
> deploy a SNAPSHOT release (say 8.0.1.SNAPSHOT) which is always the latest
> and greatest of 8.0.1. I would then also have these automated builds deploy
> the exact same code as a versioned build - probably something like
> 8.0.1-2013-07-29 where the last bit is the date it was deployed. Note that
> when I say 'deploy' here, I mean into your own self-hosted Maven repo
> (Artifactory, or whatever you want). You wouldn't deploy nightly builds
> into Maven Central.
>
> By default, the checked-in code would reference the SNAPSHOT version, so
> in every POM file that referenced another module (e.g. controls would have
> a dependency onto base), it would use 8.0.1.SNAPSHOT as the version (or
> more likely a variable like ${project.version}). Anyone who checks out and
> builds would automatically be building against the latest, which is what
> you would expect.
>
> Ideally everyone should be constantly updating their code and making sure
> it compiles against the latest always. You mention two week old code - this
> isn't exactly something you'd expect to see in an Agile process. Agile
> tasks are typically 4 to 16 hours and the conclusion of each should involve
> a check-in and update to the latest and then be included in the nightly
> build and unit tested against, etc. If a task is bigger than that, you're
> probably drifting from Agile best practices (and typically would break your
> task down into smaller chunks). Your nightly builds should always run off
> head and compile so everyone should be working towards that for their
> milestone (and milestones should be 1 to 4 weeks in a good agile process,
> with all code checked in by the end of the milestone, then tagged and
> released for user feedback).
>
> But assuming you had reasons for these out of date or very long tasks,
> then to pin your code to a particular version, there are a couple of
> options.
>
> The first option is the simplest: just have the entire openjfx project
> open in your IDE. When you have a module open and included in the IDE build
> (in IntelliJ at least, I don't know what they others do) then your copy of
> the code is always used. Even if a newer snapshot is deployed, it's not
> used by IntelliJ (I'm pretty sure of this, but it could use double
> checking). So if you have all the modules open (i.e. you opened the top
> level openjfx POM, which is what I assume most of you do) then you control
> what versions you are using, simply by controlling the source control
> updates. If you don't want a newer version of 'base' don't update it from
> SVN or update to a specific SVN revision.
>
> The second option is more explicit. Just change the dependencies in your
> POMs between modules to reference a specific version. So if you are on the
> controls team and you need to hook into a specific version of base, you
> just edit the controls POM and where it references base as a dependency put
> in the version number for the day you want to pin to, e.g. version
> 8.0.1-2013-07-29. You are then guaranteed to be fixed against that version
> and never have to worry about a change being forced on you.
>
> The versioned nightly builds are purely to allow for the second option. If
> you didn't need the second option, you could just use snapshots. If you do
> want these versioned nightly builds then they are going to build up over
> time and given the size of JFX this might get to be a problem after a
> while. Personally I would keep these daily versions around only for a
> milestone. At the end of a milestone I would do a formal milestone release,
> like 8.0.1-m23 (where m23 is the milestone number) and deploy that version.
> People really shouldn't be sticking to older code for long (it's not
> healthy) so I would delete all the old daily builds (you can easily do this
> in your own hosted repository, unlike Maven Central). Anyone who really
> needs to stick to that milestone would at least have to upgrade to the
> final version of that milestone. If this is too harsh for your needs, you
> could keep the last 3 (or 5 or 20 or 100) milestones worth of nightly
> builds.
>
> I assume you do several internal milestones releases for each actual
> release to the public. If so, I probably wouldn't release milestones to
> Maven Central but I'd consider putting each 'release' into there (e.g.
> ea-b96 is a release). Or maybe you just wait until you get out of 'ea' and
> do actual formal releases such as 8.0.1. These non-ea releases definitely
> should be synched to Maven Central - this is the bit where we run into
> trouble with Oracle policy on self-hosting, but if you had your own private
> repo where you were releasing as per all of the above, with POMs all setup
> and source code and javadoc included, it would be a fairly trivial task for
> us in the community to then deploy your formal releases to Maven Central
> for you.
>
> If any of that is unclear or needs further elaboration, let me know and
> I'll do my best. It's a lengthy topic and difficult to talk through like
> this.
>
> As usual this is all my experience, my opinions and my info that I have.
> Take and use anything you want, ignore the rest. And if people out there
> have better ways of doing things, I'd be just as keen as anyone to know
> about them.
>
>
