Comments inline. TL;DR +1 on a small number of supported versions with different classifiers that only span a limited api skew to avoid a mountain of reflection. Along with that, support for the builds via jenkins testing.
Any further dependency resoluton should be considered 'external projects' and handled via their own maven setttings.xml which can be in external repos by people who want hbase to support other versions of our dependencies (and possibly have a branch of hbase with the appropriate modifications). Any new dependency versions we want to support should be heavily vetted for ease of integration and stability. -1 on keeping code in the POMs for things we don't directly release as that means more potential maintaince for things we (as a community) don't care that much about (ala current Avro support). ------------------- Jesse Yates @jesse_yates jyates.github.com On Wed, May 16, 2012 at 2:09 PM, Andrew Purtell <apurt...@apache.org> wrote: > On Wed, May 16, 2012 at 1:00 PM, Roman Shaposhnik <r...@apache.org> wrote: > > Now, it seems like in Bigtop we're going to soon expose the Maven repo > > with all of the Maven artifacts constituting a particular Bigtop > "stack". You > > could think of it as a transitive closure of all of the deps. built > against > > each other. This, of course, will not tackle an issue of a random > combination > > of components (we only support the versions of components as > > specified in our own BOM for each particular Bigtop release) but it will > > provide a pretty stable body of Maven artifacts that are KNOWN (as > > in tested) to be compiled against each other. > > I think HBase should consider having a single blessed set of > dependencies and only one build for a given release, This would be really nice, but seems a bit unreasonable given that we are the "hadoop database" (if not in name, at least by connotation). I think limiting our support to the latest X versions (2-3?) is reasonable given consistent APIs - we should be very careful in picking which new versions we support and when. A lot of the pain with the hadoop distributions has been then wildly shifting apis, making a lot of work painful for handling different versions (distcp/backup situations come to mind here, among other things. > but also several > Jenkins projects set up to insure that release also builds against > some larger set of additional dependencies according to contributor > needs, Definitely a necessity if we support more than 1 version. Only problem here is that we then have to worry about multiple builds, which seemed to be a problem in the past. If we are going to support more than 1 version, we need to have full support for that version/permutation of options (eg. Hadoop X with Zookeeper Y) > and otherwise the user is welcome to mvn > -Ddependency.version=foo. I'd prefer not to have pieces in the code that are not being regularly tested/used. If we find we have a lot of people using a given version and willing to support it, then we should roll it in (like with other external dependencies, like the Avro stuff that we are stuck with). The mvn command you recommend above is already quite close to what we are doing already, with just specifying the hadoop version as a profile, eg (or close enough) -Dhadoop.version=0.23 +1 on the idea of having classifiers for the different versions we actually release as proper artifacts, and should be completely reasonable to enable via profiles. I'd have to double check as to _how_ people would specify that classifier/version of hbase from the maven repo, but it seems entirely possible (my worry here is about the collison with the -tests and -sources classifiers, which are standard mvn conventions for different builds). Otherwise, with maven it is very reasonable to have people hosting profiles for versions that they want to support - generally, this means just another settings.xml file that includes another profile that people can activate on their own, when they want to build against their own version. > A project like BigTop could separately > handle a broader set of combinations according to "distribution > consumer" demand, we could point potential users at that if it's an > option. > > Best regards, > > - Andy > > Problems worthy of attack prove their worth by hitting back. - Piet > Hein (via Tom White) >
