On Wed, 2006-12-06 at 11:57 -0500, Jason van Zyl wrote:
> On 6 Dec 06, at 11:38 AM 6 Dec 06, Carl Trieloff wrote:
>
> > I have spoken with a few committers over IRC, ApacheCon etc about
> > this so here it comes. Some of us would like to include maven into
> > Fedora distributions. There are two components to this, one
> > technical and the other process similar to the Apache incubator
> > process in that you need a sponsor, need to pass review etc...
> >
>
> To be frank, I'm not so sure this would be a good thing for Maven
> users. It's not terribly hard to install and here are some of the
> problems I see:
>
> 1. Who is going to maintain this? It seems rather complicated when we
> have something that works pretty easily.
>
The patch's goals are twofold.
1: Ensure full offline mode (the -o switch still allows maven to try and
check for plugin updates and what not in valid repositories)
2: Ensure that projects can build against a single version of a
dependency, rather than multiple.
> 2. On Redhat would Maven be laid out differently then our stock
> install? For documentation, and learning purposes having Maven be
> laid out different and work differently on different platforms is not
> something I would like to see. It will just make it more difficult
> for Maven users. Maven does more then Java but we are about platform
> neutrality and having something that works in a specific way on
> Redhat that doesn't translate to working on OS/X is not a good thing.
>
Maven is laid out differently by the rpm. However, this has nothing to
do with Red Hat specifically. The RPM adheres to the Filesystem
Heirarchy Standard (FHS), which a reference for install locations for
*nix systems: http://www.pathname.com/fhs/
> 3. What about multiple versions? This would make Maven bound to the
> OS which is also not a good thing. I have often run into many problem
> where multiple versions are not allowed. I run 4 different versions
> of Maven and using RPMs here would make this difficult.
>
As I mentioned in point 2 in #1 above, this is actually one of the
things we want (optionally, ofcourse). JPackage (used by Red Hat, Suse,
Mandriva, etc.) currently does not support multiple version
installations of a package in parallel. Due to this limitation, we need
a maven there that can work with 1 version of each package.
> I would be extremely hesitant to make this the standard offering for
> Maven users on Redhat. I would be hesitant to off anything that did
> not work in a standard way, or as much of a standard way as possible,
> between all platforms. I do not see this as a great benefit to Maven
> users.
>
All of the stuff done by the patch is optional, and not enabled by
default. Everything is controlled by properties (e.g.
-Dmaven2.ignore.versions has to be there for maven to ignore versions
and go with 1 version per dependency, and so on). Whatever the patch
does, is intended to be used on a range of distributions, not just in
Fedora.
Cheers,
Deepak
> Jason.
>
> >
> > A few of us have been working on working out all the issues to meet
> > the Fedora and Linux distro guidelines. Thanks to Deepak he has
> > worked out most of this and has created a set of patches for maven
> > in addition to identifying the work that needs to be done each time
> > an update needs to be included into Fedora. It would be ideal if we
> > could create a maven target with each release that met these
> > guidelines and work in a Linux distro build environment. Once we
> > get through the technical, Deepak, and I can help drive the
> > process with Fedora.
> >
> > Please take a look, Deepak can JIRA all the patches and a few of us
> > can
> > help with this.
> >
> > Regards
> > Carl, Deepak
> >
> >
> > The following provided by Deepak
> >
> >
> > Since there is a strict "must build offline" requirement, the
> > maven2 rpm
> > is designed to build completely disconnected from the Internet. To
> > this
> > end, the rpm is self-contained -- it contains poms, necessary jars for
> > bootstrapping, etc.
> >
> > Here is where all the maven2 sources are (explanation of source
> > files is
> > at the end):
> > http://people.redhat.com/dbhole/maven/
> >
> > It is strongly recommended that whoever plans to help out with this,
> > reads the maven2-jpp-readme.html file, as it gives a brief idea as to
> > how the mapping system works. Changes from maven2-jpprepolayout.patch
> > (or their equivalent), and new files
> > maven2-JPackageRepositoryLayout.java and maven2-
> > MavenJPackageDepmap.java
> > (or their equivalent) need to go upstream in order to make everything
> > work with jpackage smoothly.
> >
> > In order to put maven2 into Fedora, here are the brief set of steps
> > for
> > all dependencies, and maven itself:
> >
> > 1. Get rpm from jpackage. Check if it is set to build natively.
> > If not,
> > Build it, run it through spec-gcj-convert (file attached), verify
> > new spec, rebuild it, make sure it conforms to Fedora guidelines
> > for everything except the release number and all, that should be
> > jpackage specific. Then upload this to jpackage.
> > If it is:
> > Do nothing
> >
> > 2. Then, use the latest spec from jpackage after doing #1, update the
> > release to match fc guidelines, merge changelog, build into Fedora.
> >
> >
> > A brief explanation what the files are for:
> > -> m2_jar_repo.tar.gz - An archive of certain jar files needed for
> > building (some maven2 dependencies need maven2 to build -- so maven2
> > has a bootstrap mode where pre-built jars are used. Once such a
> > maven
> > is built, it is then used to build all dependencies, and then maven
> > is re-built in non-bootstrap mode against those dependencies)
> > -> m2_pom_repo.tar.gz - An archive of all the pom files needed for
> > building
> > -> maven2-addjdom-depmap.xml - Adds a jdom dependency to maven.
> > jdom is
> > used by the jpp mapping system
> > -> maven2-addjdomtobootstrappath.patch - Adds jdom to the bootstrap
> > classpath (see above note)
> > -> maven2-buildallplugins.patch - Patch to enable plugins that maven
> > disables by default (we always build as much as possible in our
> > packages).
> > -> maven2-disable-itests.patch - Patch to disable integration
> > tests. The
> > tests take a long time to run, so this patch is used to disable them
> > (unless the --with itests switch is given)
> > -> maven2-empty-dep.jar/pom - Certain jars in maven repositories are
> > monolithic and not needed when building against JPackage packages.
> > e.g. velocity-dep, which contains all of velocity's dependencies.
> > Since the jpp mapping system can only map one-to-one, we need to
> > maven dependencies such as velocity-dep to something... that
> > something is this empty-dep which contains no classes.
> > -> maven2-enable-unbuilt-modules.patch - As with plugins, we also
> > try to
> > build as many modules as possible. This patch enables it such
> > modules.
> > -> maven2-it-jppfixes.patch - This patch was originally needed with
> > the
> > old rpm. In theory, only the first chunk of patch should be needed
> > (to enable the -s switch), but I am not certain -- it needs to be
> > investigated.
> > -> maven2-JPackageRepositoryLayout.java -> The "JPackage layout".
> > In the
> > Jpackage layout, a file location is $groupId/$artifactId.jar with
> > any
> > "."'s in the groupId converted to "/"'s first. *This file, or an
> > equivalent needs to go upstream*
> > -> maven2-jpp-readme.html - A readme for how packages should be built
> > around this maven.
> > -> maven2-jpprepolayout.patch - The most critical component of all.
> > This
> > patch wires in usage of the jpackage repository. Additionally, it
> > also ensures that maven will not go online -- not for dependencies,
> > not for checking plugin updates, not for anything. It also adds
> > the jpackage repository (/usr/share/maven2/repository) to the
> > default
> > list. *The changes in here are the ones that need to go upstream*
> > -> maven2-jpp-script - The init script for jpackage building. It is a
> > wrapper around the mvn script -- it calls mvn with certain
> > properties
> > that causes maven to behave in an offline manner,
> > use /usr/share/maven2/repository, etc.
> > -> maven2-MavenJPackageDepmap.java - A helper package that loads the
> > dependency map, provides the value based on key requested, etc.
> > *This
> > file, or an equivalent needs to go upstream*
> > -> maven2-maven-site-plugin.tar.gz - The maven-site-plugin. The
> > version
> > in the plugins tarball cannot build against the new doxia.
> > -> maven2-model-v3.tar.gz - The model v3 class
> > -> maven2-plugins-060420-src.tar.gz - The plugins tarball.
> > See .spec to
> > see how it was generated
> > -> maven2-plugins-plexus151.patch - Disabled a code stub that causes
> > build to fail against plexus-utils 1.5.1+
> > -> maven2-run-it-tests.sh - Script to run integration tests
> > -> maven2-script - The mvn wrapper script
> > -> maven2-settings.xml - I don't think this is used anymore..
> > -> maven2-src.tar.gz - The maven2-2.0.4 source tarball from svn.
> >
> >
> > Cheers,
> > Deepak
> >
> >
> > #!/usr/bin/env python
> >
> > # spec-convert-gcj
> > # ================
> > #
> > # Create a JPackage spec file that contains GCJ support from a
> > JPackage
> > # SRPM and its corresponding RPMs. spec-convert-gcj uses the
> > binary RPMs
> > # to determine each sub-package's list of jar files.
> > #
> > # Example:
> > # spec-convert-gcj ~/rpmbuild/SRPMS/bsh-*1.3.0-6jpp* ~/rpmbuild/
> > RPMS/noarch/bsh-*1.3.0-6jpp*
> >
> > import sys
> > import os
> > import warnings
> >
> > class Error(Exception):
> > pass
> >
> > # parsing state
> > state = "PREAMBLE"
> >
> > partList = [ "PREAMBLE",
> > "%package",
> > "%prep",
> > "%build",
> > "%install",
> > "%check",
> > "%clean",
> > "%preun",
> > "%postun",
> > "%pre",
> > "%post",
> > "%files",
> > "%changelog",
> > "%description",
> > "%triggerpostun",
> > "%triggerun",
> > "%triggerin",
> > "%trigger",
> > "%verifyscript",
> > "END" ]
> >
> > packages = {}
> >
> > macroizedpackagenames = {}
> >
> > currentpackage = None
> >
> > doneuptofiles = False
> >
> > mainpackageempty = False
> >
> > NAME = "spec-convert-gcj"
> > VERSION = "1.1"
> >
> > class Package:
> > name = ""
> > jarfilenames = []
> > postdone = False
> > postundone = False
> >
> > def __init__(self, name, jarfilenames):
> > self.name = name
> > self.jarfilenames = jarfilenames
> >
> > def contains_jars(self):
> > return len(self.jarfilenames) != 0
> >
> > def get_name(self):
> > return self.name
> >
> > def get_jar_file_names(self):
> > return self.jarfilenames
> >
> > def set_post_done(self):
> > self.postdone = True
> >
> > def get_post_done(self):
> > return self.postdone
> >
> > def set_postun_done(self):
> > self.postundone = True
> >
> > def get_postun_done(self):
> > return self.postdone
> >
> > def __str__(self):
> > return self.name
> >
> > def get_package_name(rpmfilename):
> > rpmoutput = os.popen("rpm --queryformat %{name} -qp " +
> > rpmfilename)
> > packagename = rpmoutput.read()
> > status = rpmoutput.close()
> > if status is not None or packagename == "":
> > raise Error, "spec file name lookup failed"
> > else:
> > print "spec file name lookup passed"
> > return packagename
> >
> > def extract_files(rpmfilename, filenamepattern):
> > status = os.system("rpm2cpio " + rpmfilename + " | cpio -id " +
> > filenamepattern)
> > if status > 0:
> > raise Error, "%s exited with code %d" % ("cpio", status)
> > elif status < 0:
> > raise Error, "%s killed by signal %d" % ("cpio", -status)
> > else:
> > print "rpm2cpio passed"
> >
> > def get_jar_file_names_callback(jarfilenames, dirname, fnames):
> > for fname in fnames:
> > if fname.endswith(".jar") \
> > and os.path.isfile(os.path.join(dirname, fname)) \
> > and not os.path.islink(os.path.join(dirname, fname)):
> > jarfilenames.append(fname)
> > os.remove(os.path.join(dirname, fname))
> >
> > def note_package(rpmfilename):
> > global packages
> > unmacroizedpackagename = get_package_name(rpmfilename)
> > if unmacroizedpackagename != mainpackagename + "-debuginfo":
> > if unmacroizedpackagename == mainpackagename:
> > packagename = mainpackagename
> > else:
> > packagename = macroizedpackagenames
> > [unmacroizedpackagename]
> > jarfilenames = []
> > extract_files(rpmfilename, "*.jar")
> > os.path.walk(".", get_jar_file_names_callback, jarfilenames)
> > packages[packagename] = Package(packagename, jarfilenames)
> >
> > def filter_preamble(line):
> > if line.lower().startswith("buildarch:"):
> > if line.lower().split()[1] == "noarch":
> > add_buildarch(line)
> > else:
> > outputspecfile.write(line)
> > else:
> > outputspecfile.write(line)
> >
> > def add_buildarch(line):
> > outputspecfile.write("%if ! %{gcj_support}\n")
> > outputspecfile.write(line)
> > outputspecfile.write("%endif\n")
> >
> > def add_requires():
> > outputspecfile.write("%if %{gcj_support}\n")
> > outputspecfile.write("BuildRequires:\t\tjava-gcj-compat-devel\n")
> > outputspecfile.write("Requires(post):\t\tjava-gcj-compat\n")
> > outputspecfile.write("Requires(postun):\tjava-gcj-compat\n")
> > outputspecfile.write("%endif\n")
> > outputspecfile.write("\n")
> >
> > def add_aot_compile_rpm():
> > outputspecfile.write("%if %{gcj_support}\n")
> > outputspecfile.write("%{_bindir}/aot-compile-rpm\n")
> > outputspecfile.write("%endif\n")
> > outputspecfile.write("\n")
> >
> > def add_so_and_db_files():
> > outputspecfile.write("%if %{gcj_support}\n")
> > outputspecfile.write("%dir %attr(-,root,root) %{_libdir}/gcj/%
> > {name}\n")
> > for jarfilename in currentpackage.get_jar_file_names():
> > outputspecfile.write("%attr(-,root,root) %{_libdir}/gcj/%
> > {name}/" + jarfilename + ".*\n")
> > outputspecfile.write("%endif\n")
> > outputspecfile.write("\n")
> >
> > def add_rebuild_gcj_db():
> > outputspecfile.write("%if %{gcj_support}\n")
> > add_rebuild_gcj_db_no_condition()
> > outputspecfile.write("%endif\n")
> > outputspecfile.write("\n")
> >
> > def add_rebuild_gcj_db_no_condition():
> > outputspecfile.write("if [ -x %{_bindir}/rebuild-gcj-db ]\n")
> > outputspecfile.write("then\n")
> > outputspecfile.write(" %{_bindir}/rebuild-gcj-db\n")
> > outputspecfile.write("fi\n")
> >
> > def add_name(name):
> > if name.startswith(mainpackagename):
> > if name != mainpackagename:
> > outputspecfile.write(" " + name.replace(mainpackagename +
> > "-", "", 1))
> > else:
> > # full name
> > outputspecfile.write(" -n " + name)
> > outputspecfile.write("\n")
> >
> > def add_post(name):
> > outputspecfile.write("%if %{gcj_support}\n")
> > outputspecfile.write("%post")
> > add_name(name)
> > add_rebuild_gcj_db_no_condition()
> > outputspecfile.write("%endif\n")
> > outputspecfile.write("\n")
> >
> > def add_postun(name):
> > outputspecfile.write("%if %{gcj_support}\n")
> > outputspecfile.write("%postun")
> > add_name(name)
> > add_rebuild_gcj_db_no_condition()
> > outputspecfile.write("%endif\n")
> > outputspecfile.write("\n")
> >
> > def add_before_part(state):
> > global doneuptofiles
> >
> > if state == "%files":
> > if doneuptofiles == False:
> > for packagename in packages:
> > package = packages[packagename]
> > if package.contains_jars():
> > if not package.get_post_done():
> > add_post(package.get_name())
> > if not package.get_postun_done():
> > add_postun(package.get_name())
> > doneuptofiles = True
> >
> > def add_to_part(state):
> > if state == "PREAMBLE":
> > if currentpackage is not None \
> > and currentpackage.contains_jars():
> > add_requires()
> > if state == "%package":
> > if currentpackage is not None \
> > and currentpackage.contains_jars():
> > add_requires()
> > elif state == "%install":
> > add_aot_compile_rpm()
> > elif state == "%files":
> > if currentpackage is not None \
> > and currentpackage.contains_jars():
> > add_so_and_db_files()
> > elif state == "%post":
> > if currentpackage is not None \
> > and currentpackage.contains_jars():
> > add_rebuild_gcj_db()
> > currentpackage.set_post_done()
> > elif state == "%postun":
> > if currentpackage is not None \
> > and currentpackage.contains_jars():
> > add_rebuild_gcj_db()
> > currentpackage.set_postun_done()
> >
> > def set_currentpackage(line):
> > global currentpackage
> >
> > fields = line.split()
> > if len(fields) == 1:
> > if not mainpackageempty:
> > try:
> > currentpackage = packages[mainpackagename]
> > except KeyError, e:
> > pass
> > else:
> > currentpackage = None
> > else:
> > if fields[1] == "-n":
> > # full name specified
> > try:
> > currentpackage = packages[fields[2]]
> > except KeyError, e:
> > pass
> > else:
> > try:
> > currentpackage = packages[mainpackagename + "-" +
fields[1]]
> > except KeyError, e:
> > pass
> >
> > def set_macroized_package_names():
> > global macroizedpackagenames
> >
> > packagelines = os.popen("cat " + specfilename + " | grep '^%
> > package'")
> > packagenames = os.popen("rpm -q --specfile --queryformat='%
> > {name}\n' " + specfilename + " | grep -v " + mainpackagename + "-
> > debuginfo")
> >
> > packageline = packagelines.readline()
> > # skip main package name
> > packagename = packagenames.readline()
> > packagename = packagenames.readline()
> >
> > if packagename != "":
> > while packageline != "":
> > fields = packageline.split()
> > if len(fields) == 2:
> > macroizedpackagenames[packagename.strip()] =
> > mainpackagename + "-" + fields[1]
> > else:
> > if len(fields) == 3:
> > macroizedpackagenames[packagename.strip()] =
> > fields[2]
> > packageline = packagelines.readline()
> > packagename = packagenames.readline()
> >
> > packagelines.close()
> > packagenames.close()
> >
> > if packageline != "" or packagename !="":
> > raise Error, "number of Name: and %package lines does not
> > match number of names returned by spec file query"
> > else:
> > print "macroized package names creation passed"
> >
> > print "map of unmacroized names to macroized names:"
> > print macroizedpackagenames
> >
> > def next_state(line):
> > global state
> > if line == "":
> > add_to_part(state)
> > state = "END"
> > return
> > for part in partList:
> > if line.startswith(part):
> > add_to_part(state)
> > state = part
> > add_before_part(state)
> > set_currentpackage(line)
> >
> > def print_version():
> > print NAME, VERSION
> > sys.exit(0)
> >
> > def print_usage():
> > print "Usage: " + NAME + " SRPM RPMS"
> > print "Create a JPackage spec file that contains GCJ support
> > from a JPackage"
> > print "SRPM and its corresponding RPMs. " + NAME + " uses the
> > binary RPMs"
> > print "to determine each sub-package's list of jar files."
> > print ""
> > print "Example:"
> > print " " + NAME + " ~/rpmbuild/SRPMS/bsh-*1.3.0-6jpp* ~/
> > rpmbuild/RPMS/noarch/bsh-*1.3.0-6jpp*"
> > sys.exit(0)
> >
> > if __name__ == "__main__":
> > try:
> >
> > options = sys.argv[1:]
> >
> > if len(options) == 0:
> > print NAME + ": no input files"
> > print_usage()
> >
> > origdir = os.getcwd()
> >
> > # find SRPM within arguments
> > srpmfilename = ""
> > rpmfilenames = []
> > for option in options:
> > if option.endswith(".src.rpm"):
> > if srpmfilename != "":
> > raise Error, "more than one SRPM specified"
> > else:
> > srpmfilename = os.path.join(origdir, option)
> > elif option.endswith(".rpm"):
> > rpmfilenames.append(os.path.join(origdir, option))
> > elif option == "--version":
> > print_version()
> > elif option == "--help":
> > print_usage()
> >
> > warnings.filterwarnings("ignore", message="tmpnam is a
> > potential security risk to your program")
> > tmpdir = os.path.basename(os.tmpnam())
> > os.mkdir(tmpdir)
> > os.chdir(tmpdir)
> >
> > # get spec file name: assumes that the spec file is called
> > # %{name}.spec
> > print "processing: " + srpmfilename
> > mainpackagename = get_package_name(srpmfilename)
> > specfilename = mainpackagename + ".spec"
> >
> > # extract spec file
> > print "extracting spec file: " + specfilename
> > extract_files(srpmfilename, specfilename)
> >
> > # check for existing gcj support
> > checkconverted = os.popen("grep gcj_support " + specfilename)
> > checksupport = checkconverted.readline()
> > if checksupport != "":
> > raise Error, "these rpms already support gcj"
> > status = checkconverted.close()
> > if status is None:
> > raise Error, "failed to close grep pipe"
> > else:
> > print "no gcj support detected"
> >
> > # calculate unmacroized name-to-macroized name mapping
> > set_macroized_package_names()
> >
> > for rpmfilename in rpmfilenames:
> > note_package(rpmfilename)
> >
> > # open spec file for reading
> > specfile = open(specfilename, 'r')
> >
> > # open output spec file for writing
> > outputspecfile = open(os.path.join(origdir, mainpackagename
> > + "-gcj.spec"), 'w')
> >
> > # check if there is a main package or only sub-packages
> > if mainpackagename in packages:
> > currentpackage = packages[mainpackagename]
> > else:
> > mainpackageempty = True
> >
> > print "main package:", currentpackage
> >
> > line = specfile.readline()
> > next_state(line)
> >
> > # skip license block
> > while line.startswith("#"):
> > outputspecfile.write(line)
> > line = specfile.readline()
> >
> > outputspecfile.write("\n%define gcj_support %{?_with_gcj_support:
> > 1}%{!?_with_gcj_support:%{?_without_gcj_support:0}%{!?
> > _without_gcj_support:%{?_gcj_support:%{_gcj_support}}%{!?
> > _gcj_support:0}}}\n")
> >
> > while line != "":
> > if state == "PREAMBLE":
> > filter_preamble(line)
> > else:
> > outputspecfile.write(line)
> >
> > line = specfile.readline()
> > next_state(line)
> >
> > outputspecfile.close()
> >
> > os.chdir(origdir)
> >
> > except Error, e:
> > print >> sys.stderr, "%s: error: %s" % (
> > os.path.basename(sys.argv[0]), e)
> > sys.exit(1)
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
>
---------------------------------------------------------------------
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
