On Wed, 2006-12-06 at 11:57 -0500, Jason van Zyl wrote: > On 6 Dec 06, at 11:38 AM 6 Dec 06, Carl Trieloff wrote: > > > I have spoken with a few committers over IRC, ApacheCon etc about > > this so here it comes. Some of us would like to include maven into > > Fedora distributions. There are two components to this, one > > technical and the other process similar to the Apache incubator > > process in that you need a sponsor, need to pass review etc... > > > > To be frank, I'm not so sure this would be a good thing for Maven > users. It's not terribly hard to install and here are some of the > problems I see: > > 1. Who is going to maintain this? It seems rather complicated when we > have something that works pretty easily. >
The patch's goals are twofold. 1: Ensure full offline mode (the -o switch still allows maven to try and check for plugin updates and what not in valid repositories) 2: Ensure that projects can build against a single version of a dependency, rather than multiple. > 2. On Redhat would Maven be laid out differently then our stock > install? For documentation, and learning purposes having Maven be > laid out different and work differently on different platforms is not > something I would like to see. It will just make it more difficult > for Maven users. Maven does more then Java but we are about platform > neutrality and having something that works in a specific way on > Redhat that doesn't translate to working on OS/X is not a good thing. > Maven is laid out differently by the rpm. However, this has nothing to do with Red Hat specifically. The RPM adheres to the Filesystem Heirarchy Standard (FHS), which a reference for install locations for *nix systems: http://www.pathname.com/fhs/ > 3. What about multiple versions? This would make Maven bound to the > OS which is also not a good thing. I have often run into many problem > where multiple versions are not allowed. I run 4 different versions > of Maven and using RPMs here would make this difficult. > As I mentioned in point 2 in #1 above, this is actually one of the things we want (optionally, ofcourse). JPackage (used by Red Hat, Suse, Mandriva, etc.) currently does not support multiple version installations of a package in parallel. Due to this limitation, we need a maven there that can work with 1 version of each package. > I would be extremely hesitant to make this the standard offering for > Maven users on Redhat. I would be hesitant to off anything that did > not work in a standard way, or as much of a standard way as possible, > between all platforms. I do not see this as a great benefit to Maven > users. > All of the stuff done by the patch is optional, and not enabled by default. Everything is controlled by properties (e.g. -Dmaven2.ignore.versions has to be there for maven to ignore versions and go with 1 version per dependency, and so on). Whatever the patch does, is intended to be used on a range of distributions, not just in Fedora. Cheers, Deepak > Jason. > > > > > A few of us have been working on working out all the issues to meet > > the Fedora and Linux distro guidelines. Thanks to Deepak he has > > worked out most of this and has created a set of patches for maven > > in addition to identifying the work that needs to be done each time > > an update needs to be included into Fedora. It would be ideal if we > > could create a maven target with each release that met these > > guidelines and work in a Linux distro build environment. Once we > > get through the technical, Deepak, and I can help drive the > > process with Fedora. > > > > Please take a look, Deepak can JIRA all the patches and a few of us > > can > > help with this. > > > > Regards > > Carl, Deepak > > > > > > The following provided by Deepak > > > > > > Since there is a strict "must build offline" requirement, the > > maven2 rpm > > is designed to build completely disconnected from the Internet. To > > this > > end, the rpm is self-contained -- it contains poms, necessary jars for > > bootstrapping, etc. > > > > Here is where all the maven2 sources are (explanation of source > > files is > > at the end): > > http://people.redhat.com/dbhole/maven/ > > > > It is strongly recommended that whoever plans to help out with this, > > reads the maven2-jpp-readme.html file, as it gives a brief idea as to > > how the mapping system works. Changes from maven2-jpprepolayout.patch > > (or their equivalent), and new files > > maven2-JPackageRepositoryLayout.java and maven2- > > MavenJPackageDepmap.java > > (or their equivalent) need to go upstream in order to make everything > > work with jpackage smoothly. > > > > In order to put maven2 into Fedora, here are the brief set of steps > > for > > all dependencies, and maven itself: > > > > 1. Get rpm from jpackage. Check if it is set to build natively. > > If not, > > Build it, run it through spec-gcj-convert (file attached), verify > > new spec, rebuild it, make sure it conforms to Fedora guidelines > > for everything except the release number and all, that should be > > jpackage specific. Then upload this to jpackage. > > If it is: > > Do nothing > > > > 2. Then, use the latest spec from jpackage after doing #1, update the > > release to match fc guidelines, merge changelog, build into Fedora. > > > > > > A brief explanation what the files are for: > > -> m2_jar_repo.tar.gz - An archive of certain jar files needed for > > building (some maven2 dependencies need maven2 to build -- so maven2 > > has a bootstrap mode where pre-built jars are used. Once such a > > maven > > is built, it is then used to build all dependencies, and then maven > > is re-built in non-bootstrap mode against those dependencies) > > -> m2_pom_repo.tar.gz - An archive of all the pom files needed for > > building > > -> maven2-addjdom-depmap.xml - Adds a jdom dependency to maven. > > jdom is > > used by the jpp mapping system > > -> maven2-addjdomtobootstrappath.patch - Adds jdom to the bootstrap > > classpath (see above note) > > -> maven2-buildallplugins.patch - Patch to enable plugins that maven > > disables by default (we always build as much as possible in our > > packages). > > -> maven2-disable-itests.patch - Patch to disable integration > > tests. The > > tests take a long time to run, so this patch is used to disable them > > (unless the --with itests switch is given) > > -> maven2-empty-dep.jar/pom - Certain jars in maven repositories are > > monolithic and not needed when building against JPackage packages. > > e.g. velocity-dep, which contains all of velocity's dependencies. > > Since the jpp mapping system can only map one-to-one, we need to > > maven dependencies such as velocity-dep to something... that > > something is this empty-dep which contains no classes. > > -> maven2-enable-unbuilt-modules.patch - As with plugins, we also > > try to > > build as many modules as possible. This patch enables it such > > modules. > > -> maven2-it-jppfixes.patch - This patch was originally needed with > > the > > old rpm. In theory, only the first chunk of patch should be needed > > (to enable the -s switch), but I am not certain -- it needs to be > > investigated. > > -> maven2-JPackageRepositoryLayout.java -> The "JPackage layout". > > In the > > Jpackage layout, a file location is $groupId/$artifactId.jar with > > any > > "."'s in the groupId converted to "/"'s first. *This file, or an > > equivalent needs to go upstream* > > -> maven2-jpp-readme.html - A readme for how packages should be built > > around this maven. > > -> maven2-jpprepolayout.patch - The most critical component of all. > > This > > patch wires in usage of the jpackage repository. Additionally, it > > also ensures that maven will not go online -- not for dependencies, > > not for checking plugin updates, not for anything. It also adds > > the jpackage repository (/usr/share/maven2/repository) to the > > default > > list. *The changes in here are the ones that need to go upstream* > > -> maven2-jpp-script - The init script for jpackage building. It is a > > wrapper around the mvn script -- it calls mvn with certain > > properties > > that causes maven to behave in an offline manner, > > use /usr/share/maven2/repository, etc. > > -> maven2-MavenJPackageDepmap.java - A helper package that loads the > > dependency map, provides the value based on key requested, etc. > > *This > > file, or an equivalent needs to go upstream* > > -> maven2-maven-site-plugin.tar.gz - The maven-site-plugin. The > > version > > in the plugins tarball cannot build against the new doxia. > > -> maven2-model-v3.tar.gz - The model v3 class > > -> maven2-plugins-060420-src.tar.gz - The plugins tarball. > > See .spec to > > see how it was generated > > -> maven2-plugins-plexus151.patch - Disabled a code stub that causes > > build to fail against plexus-utils 1.5.1+ > > -> maven2-run-it-tests.sh - Script to run integration tests > > -> maven2-script - The mvn wrapper script > > -> maven2-settings.xml - I don't think this is used anymore.. > > -> maven2-src.tar.gz - The maven2-2.0.4 source tarball from svn. > > > > > > Cheers, > > Deepak > > > > > > #!/usr/bin/env python > > > > # spec-convert-gcj > > # ================ > > # > > # Create a JPackage spec file that contains GCJ support from a > > JPackage > > # SRPM and its corresponding RPMs. spec-convert-gcj uses the > > binary RPMs > > # to determine each sub-package's list of jar files. > > # > > # Example: > > # spec-convert-gcj ~/rpmbuild/SRPMS/bsh-*1.3.0-6jpp* ~/rpmbuild/ > > RPMS/noarch/bsh-*1.3.0-6jpp* > > > > import sys > > import os > > import warnings > > > > class Error(Exception): > > pass > > > > # parsing state > > state = "PREAMBLE" > > > > partList = [ "PREAMBLE", > > "%package", > > "%prep", > > "%build", > > "%install", > > "%check", > > "%clean", > > "%preun", > > "%postun", > > "%pre", > > "%post", > > "%files", > > "%changelog", > > "%description", > > "%triggerpostun", > > "%triggerun", > > "%triggerin", > > "%trigger", > > "%verifyscript", > > "END" ] > > > > packages = {} > > > > macroizedpackagenames = {} > > > > currentpackage = None > > > > doneuptofiles = False > > > > mainpackageempty = False > > > > NAME = "spec-convert-gcj" > > VERSION = "1.1" > > > > class Package: > > name = "" > > jarfilenames = [] > > postdone = False > > postundone = False > > > > def __init__(self, name, jarfilenames): > > self.name = name > > self.jarfilenames = jarfilenames > > > > def contains_jars(self): > > return len(self.jarfilenames) != 0 > > > > def get_name(self): > > return self.name > > > > def get_jar_file_names(self): > > return self.jarfilenames > > > > def set_post_done(self): > > self.postdone = True > > > > def get_post_done(self): > > return self.postdone > > > > def set_postun_done(self): > > self.postundone = True > > > > def get_postun_done(self): > > return self.postdone > > > > def __str__(self): > > return self.name > > > > def get_package_name(rpmfilename): > > rpmoutput = os.popen("rpm --queryformat %{name} -qp " + > > rpmfilename) > > packagename = rpmoutput.read() > > status = rpmoutput.close() > > if status is not None or packagename == "": > > raise Error, "spec file name lookup failed" > > else: > > print "spec file name lookup passed" > > return packagename > > > > def extract_files(rpmfilename, filenamepattern): > > status = os.system("rpm2cpio " + rpmfilename + " | cpio -id " + > > filenamepattern) > > if status > 0: > > raise Error, "%s exited with code %d" % ("cpio", status) > > elif status < 0: > > raise Error, "%s killed by signal %d" % ("cpio", -status) > > else: > > print "rpm2cpio passed" > > > > def get_jar_file_names_callback(jarfilenames, dirname, fnames): > > for fname in fnames: > > if fname.endswith(".jar") \ > > and os.path.isfile(os.path.join(dirname, fname)) \ > > and not os.path.islink(os.path.join(dirname, fname)): > > jarfilenames.append(fname) > > os.remove(os.path.join(dirname, fname)) > > > > def note_package(rpmfilename): > > global packages > > unmacroizedpackagename = get_package_name(rpmfilename) > > if unmacroizedpackagename != mainpackagename + "-debuginfo": > > if unmacroizedpackagename == mainpackagename: > > packagename = mainpackagename > > else: > > packagename = macroizedpackagenames > > [unmacroizedpackagename] > > jarfilenames = [] > > extract_files(rpmfilename, "*.jar") > > os.path.walk(".", get_jar_file_names_callback, jarfilenames) > > packages[packagename] = Package(packagename, jarfilenames) > > > > def filter_preamble(line): > > if line.lower().startswith("buildarch:"): > > if line.lower().split()[1] == "noarch": > > add_buildarch(line) > > else: > > outputspecfile.write(line) > > else: > > outputspecfile.write(line) > > > > def add_buildarch(line): > > outputspecfile.write("%if ! %{gcj_support}\n") > > outputspecfile.write(line) > > outputspecfile.write("%endif\n") > > > > def add_requires(): > > outputspecfile.write("%if %{gcj_support}\n") > > outputspecfile.write("BuildRequires:\t\tjava-gcj-compat-devel\n") > > outputspecfile.write("Requires(post):\t\tjava-gcj-compat\n") > > outputspecfile.write("Requires(postun):\tjava-gcj-compat\n") > > outputspecfile.write("%endif\n") > > outputspecfile.write("\n") > > > > def add_aot_compile_rpm(): > > outputspecfile.write("%if %{gcj_support}\n") > > outputspecfile.write("%{_bindir}/aot-compile-rpm\n") > > outputspecfile.write("%endif\n") > > outputspecfile.write("\n") > > > > def add_so_and_db_files(): > > outputspecfile.write("%if %{gcj_support}\n") > > outputspecfile.write("%dir %attr(-,root,root) %{_libdir}/gcj/% > > {name}\n") > > for jarfilename in currentpackage.get_jar_file_names(): > > outputspecfile.write("%attr(-,root,root) %{_libdir}/gcj/% > > {name}/" + jarfilename + ".*\n") > > outputspecfile.write("%endif\n") > > outputspecfile.write("\n") > > > > def add_rebuild_gcj_db(): > > outputspecfile.write("%if %{gcj_support}\n") > > add_rebuild_gcj_db_no_condition() > > outputspecfile.write("%endif\n") > > outputspecfile.write("\n") > > > > def add_rebuild_gcj_db_no_condition(): > > outputspecfile.write("if [ -x %{_bindir}/rebuild-gcj-db ]\n") > > outputspecfile.write("then\n") > > outputspecfile.write(" %{_bindir}/rebuild-gcj-db\n") > > outputspecfile.write("fi\n") > > > > def add_name(name): > > if name.startswith(mainpackagename): > > if name != mainpackagename: > > outputspecfile.write(" " + name.replace(mainpackagename + > > "-", "", 1)) > > else: > > # full name > > outputspecfile.write(" -n " + name) > > outputspecfile.write("\n") > > > > def add_post(name): > > outputspecfile.write("%if %{gcj_support}\n") > > outputspecfile.write("%post") > > add_name(name) > > add_rebuild_gcj_db_no_condition() > > outputspecfile.write("%endif\n") > > outputspecfile.write("\n") > > > > def add_postun(name): > > outputspecfile.write("%if %{gcj_support}\n") > > outputspecfile.write("%postun") > > add_name(name) > > add_rebuild_gcj_db_no_condition() > > outputspecfile.write("%endif\n") > > outputspecfile.write("\n") > > > > def add_before_part(state): > > global doneuptofiles > > > > if state == "%files": > > if doneuptofiles == False: > > for packagename in packages: > > package = packages[packagename] > > if package.contains_jars(): > > if not package.get_post_done(): > > add_post(package.get_name()) > > if not package.get_postun_done(): > > add_postun(package.get_name()) > > doneuptofiles = True > > > > def add_to_part(state): > > if state == "PREAMBLE": > > if currentpackage is not None \ > > and currentpackage.contains_jars(): > > add_requires() > > if state == "%package": > > if currentpackage is not None \ > > and currentpackage.contains_jars(): > > add_requires() > > elif state == "%install": > > add_aot_compile_rpm() > > elif state == "%files": > > if currentpackage is not None \ > > and currentpackage.contains_jars(): > > add_so_and_db_files() > > elif state == "%post": > > if currentpackage is not None \ > > and currentpackage.contains_jars(): > > add_rebuild_gcj_db() > > currentpackage.set_post_done() > > elif state == "%postun": > > if currentpackage is not None \ > > and currentpackage.contains_jars(): > > add_rebuild_gcj_db() > > currentpackage.set_postun_done() > > > > def set_currentpackage(line): > > global currentpackage > > > > fields = line.split() > > if len(fields) == 1: > > if not mainpackageempty: > > try: > > currentpackage = packages[mainpackagename] > > except KeyError, e: > > pass > > else: > > currentpackage = None > > else: > > if fields[1] == "-n": > > # full name specified > > try: > > currentpackage = packages[fields[2]] > > except KeyError, e: > > pass > > else: > > try: > > currentpackage = packages[mainpackagename + "-" + fields[1]] > > except KeyError, e: > > pass > > > > def set_macroized_package_names(): > > global macroizedpackagenames > > > > packagelines = os.popen("cat " + specfilename + " | grep '^% > > package'") > > packagenames = os.popen("rpm -q --specfile --queryformat='% > > {name}\n' " + specfilename + " | grep -v " + mainpackagename + "- > > debuginfo") > > > > packageline = packagelines.readline() > > # skip main package name > > packagename = packagenames.readline() > > packagename = packagenames.readline() > > > > if packagename != "": > > while packageline != "": > > fields = packageline.split() > > if len(fields) == 2: > > macroizedpackagenames[packagename.strip()] = > > mainpackagename + "-" + fields[1] > > else: > > if len(fields) == 3: > > macroizedpackagenames[packagename.strip()] = > > fields[2] > > packageline = packagelines.readline() > > packagename = packagenames.readline() > > > > packagelines.close() > > packagenames.close() > > > > if packageline != "" or packagename !="": > > raise Error, "number of Name: and %package lines does not > > match number of names returned by spec file query" > > else: > > print "macroized package names creation passed" > > > > print "map of unmacroized names to macroized names:" > > print macroizedpackagenames > > > > def next_state(line): > > global state > > if line == "": > > add_to_part(state) > > state = "END" > > return > > for part in partList: > > if line.startswith(part): > > add_to_part(state) > > state = part > > add_before_part(state) > > set_currentpackage(line) > > > > def print_version(): > > print NAME, VERSION > > sys.exit(0) > > > > def print_usage(): > > print "Usage: " + NAME + " SRPM RPMS" > > print "Create a JPackage spec file that contains GCJ support > > from a JPackage" > > print "SRPM and its corresponding RPMs. " + NAME + " uses the > > binary RPMs" > > print "to determine each sub-package's list of jar files." > > print "" > > print "Example:" > > print " " + NAME + " ~/rpmbuild/SRPMS/bsh-*1.3.0-6jpp* ~/ > > rpmbuild/RPMS/noarch/bsh-*1.3.0-6jpp*" > > sys.exit(0) > > > > if __name__ == "__main__": > > try: > > > > options = sys.argv[1:] > > > > if len(options) == 0: > > print NAME + ": no input files" > > print_usage() > > > > origdir = os.getcwd() > > > > # find SRPM within arguments > > srpmfilename = "" > > rpmfilenames = [] > > for option in options: > > if option.endswith(".src.rpm"): > > if srpmfilename != "": > > raise Error, "more than one SRPM specified" > > else: > > srpmfilename = os.path.join(origdir, option) > > elif option.endswith(".rpm"): > > rpmfilenames.append(os.path.join(origdir, option)) > > elif option == "--version": > > print_version() > > elif option == "--help": > > print_usage() > > > > warnings.filterwarnings("ignore", message="tmpnam is a > > potential security risk to your program") > > tmpdir = os.path.basename(os.tmpnam()) > > os.mkdir(tmpdir) > > os.chdir(tmpdir) > > > > # get spec file name: assumes that the spec file is called > > # %{name}.spec > > print "processing: " + srpmfilename > > mainpackagename = get_package_name(srpmfilename) > > specfilename = mainpackagename + ".spec" > > > > # extract spec file > > print "extracting spec file: " + specfilename > > extract_files(srpmfilename, specfilename) > > > > # check for existing gcj support > > checkconverted = os.popen("grep gcj_support " + specfilename) > > checksupport = checkconverted.readline() > > if checksupport != "": > > raise Error, "these rpms already support gcj" > > status = checkconverted.close() > > if status is None: > > raise Error, "failed to close grep pipe" > > else: > > print "no gcj support detected" > > > > # calculate unmacroized name-to-macroized name mapping > > set_macroized_package_names() > > > > for rpmfilename in rpmfilenames: > > note_package(rpmfilename) > > > > # open spec file for reading > > specfile = open(specfilename, 'r') > > > > # open output spec file for writing > > outputspecfile = open(os.path.join(origdir, mainpackagename > > + "-gcj.spec"), 'w') > > > > # check if there is a main package or only sub-packages > > if mainpackagename in packages: > > currentpackage = packages[mainpackagename] > > else: > > mainpackageempty = True > > > > print "main package:", currentpackage > > > > line = specfile.readline() > > next_state(line) > > > > # skip license block > > while line.startswith("#"): > > outputspecfile.write(line) > > line = specfile.readline() > > > > outputspecfile.write("\n%define gcj_support %{?_with_gcj_support: > > 1}%{!?_with_gcj_support:%{?_without_gcj_support:0}%{!? > > _without_gcj_support:%{?_gcj_support:%{_gcj_support}}%{!? > > _gcj_support:0}}}\n") > > > > while line != "": > > if state == "PREAMBLE": > > filter_preamble(line) > > else: > > outputspecfile.write(line) > > > > line = specfile.readline() > > next_state(line) > > > > outputspecfile.close() > > > > os.chdir(origdir) > > > > except Error, e: > > print >> sys.stderr, "%s: error: %s" % ( > > os.path.basename(sys.argv[0]), e) > > sys.exit(1) > > > > > > --------------------------------------------------------------------- > > To unsubscribe, e-mail: [EMAIL PROTECTED] > > For additional commands, e-mail: [EMAIL PROTECTED] > --------------------------------------------------------------------- To unsubscribe, e-mail: [EMAIL PROTECTED] For additional commands, e-mail: [EMAIL PROTECTED]
