A few of us have been working on working out all the issues to
meet the Fedora and Linux distro guidelines. Thanks to Deepak he
has worked out most of this and has created a set of patches for
maven in addition to identifying the work that needs to be done
each time an update needs to be included into Fedora. It would be
ideal if we could create a maven target with each release that
met these guidelines and work in a Linux distro build
environment. Once we get through the technical, Deepak, and I can
help drive the process with Fedora.
Please take a look, Deepak can JIRA all the patches and a few of
us can
help with this.
Regards
Carl, Deepak
The following provided by Deepak
Since there is a strict "must build offline" requirement, the
maven2 rpm
is designed to build completely disconnected from the Internet.
To this
end, the rpm is self-contained -- it contains poms, necessary
jars for
bootstrapping, etc.
Here is where all the maven2 sources are (explanation of source
files is
at the end):
http://people.redhat.com/dbhole/maven/
It is strongly recommended that whoever plans to help out with this,
reads the maven2-jpp-readme.html file, as it gives a brief idea
as to
how the mapping system works. Changes from maven2-
jpprepolayout.patch
(or their equivalent), and new files
maven2-JPackageRepositoryLayout.java and maven2-
MavenJPackageDepmap.java
(or their equivalent) need to go upstream in order to make
everything
work with jpackage smoothly.
In order to put maven2 into Fedora, here are the brief set of
steps for
all dependencies, and maven itself:
1. Get rpm from jpackage. Check if it is set to build natively.
If not,
Build it, run it through spec-gcj-convert (file attached), verify
new spec, rebuild it, make sure it conforms to Fedora guidelines
for everything except the release number and all, that should be
jpackage specific. Then upload this to jpackage.
If it is:
Do nothing
2. Then, use the latest spec from jpackage after doing #1, update
the
release to match fc guidelines, merge changelog, build into Fedora.
A brief explanation what the files are for:
-> m2_jar_repo.tar.gz - An archive of certain jar files needed for
building (some maven2 dependencies need maven2 to build -- so maven2
has a bootstrap mode where pre-built jars are used. Once such a
maven
is built, it is then used to build all dependencies, and then maven
is re-built in non-bootstrap mode against those dependencies)
-> m2_pom_repo.tar.gz - An archive of all the pom files needed for
building
-> maven2-addjdom-depmap.xml - Adds a jdom dependency to maven.
jdom is
used by the jpp mapping system
-> maven2-addjdomtobootstrappath.patch - Adds jdom to the bootstrap
classpath (see above note)
-> maven2-buildallplugins.patch - Patch to enable plugins that maven
disables by default (we always build as much as possible in our
packages).
-> maven2-disable-itests.patch - Patch to disable integration
tests. The
tests take a long time to run, so this patch is used to disable them
(unless the --with itests switch is given)
-> maven2-empty-dep.jar/pom - Certain jars in maven repositories are
monolithic and not needed when building against JPackage packages.
e.g. velocity-dep, which contains all of velocity's dependencies.
Since the jpp mapping system can only map one-to-one, we need to
maven dependencies such as velocity-dep to something... that
something is this empty-dep which contains no classes.
-> maven2-enable-unbuilt-modules.patch - As with plugins, we also
try to
build as many modules as possible. This patch enables it such
modules.
-> maven2-it-jppfixes.patch - This patch was originally needed
with the
old rpm. In theory, only the first chunk of patch should be needed
(to enable the -s switch), but I am not certain -- it needs to be
investigated.
-> maven2-JPackageRepositoryLayout.java -> The "JPackage layout".
In the
Jpackage layout, a file location is $groupId/$artifactId.jar with
any
"."'s in the groupId converted to "/"'s first. *This file, or an
equivalent needs to go upstream*
-> maven2-jpp-readme.html - A readme for how packages should be
built
around this maven.
-> maven2-jpprepolayout.patch - The most critical component of
all. This
patch wires in usage of the jpackage repository. Additionally, it
also ensures that maven will not go online -- not for dependencies,
not for checking plugin updates, not for anything. It also adds
the jpackage repository (/usr/share/maven2/repository) to the
default
list. *The changes in here are the ones that need to go upstream*
-> maven2-jpp-script - The init script for jpackage building. It
is a
wrapper around the mvn script -- it calls mvn with certain
properties
that causes maven to behave in an offline manner,
use /usr/share/maven2/repository, etc.
-> maven2-MavenJPackageDepmap.java - A helper package that loads the
dependency map, provides the value based on key requested, etc.
*This
file, or an equivalent needs to go upstream*
-> maven2-maven-site-plugin.tar.gz - The maven-site-plugin. The
version
in the plugins tarball cannot build against the new doxia.
-> maven2-model-v3.tar.gz - The model v3 class
-> maven2-plugins-060420-src.tar.gz - The plugins tarball.
See .spec to
see how it was generated
-> maven2-plugins-plexus151.patch - Disabled a code stub that causes
build to fail against plexus-utils 1.5.1+
-> maven2-run-it-tests.sh - Script to run integration tests
-> maven2-script - The mvn wrapper script
-> maven2-settings.xml - I don't think this is used anymore..
-> maven2-src.tar.gz - The maven2-2.0.4 source tarball from svn.
Cheers,
Deepak
#!/usr/bin/env python
# spec-convert-gcj
# ================
#
# Create a JPackage spec file that contains GCJ support from a
JPackage
# SRPM and its corresponding RPMs. spec-convert-gcj uses the
binary RPMs
# to determine each sub-package's list of jar files.
#
# Example:
# spec-convert-gcj ~/rpmbuild/SRPMS/bsh-*1.3.0-6jpp* ~/rpmbuild/
RPMS/noarch/bsh-*1.3.0-6jpp*
import sys
import os
import warnings
class Error(Exception):
pass
# parsing state
state = "PREAMBLE"
partList = [ "PREAMBLE",
"%package",
"%prep",
"%build",
"%install",
"%check",
"%clean",
"%preun",
"%postun",
"%pre",
"%post",
"%files",
"%changelog",
"%description",
"%triggerpostun",
"%triggerun",
"%triggerin",
"%trigger",
"%verifyscript",
"END" ]
packages = {}
macroizedpackagenames = {}
currentpackage = None
doneuptofiles = False
mainpackageempty = False
NAME = "spec-convert-gcj"
VERSION = "1.1"
class Package:
name = ""
jarfilenames = []
postdone = False
postundone = False
def __init__(self, name, jarfilenames):
self.name = name
self.jarfilenames = jarfilenames
def contains_jars(self):
return len(self.jarfilenames) != 0
def get_name(self):
return self.name
def get_jar_file_names(self):
return self.jarfilenames
def set_post_done(self):
self.postdone = True
def get_post_done(self):
return self.postdone
def set_postun_done(self):
self.postundone = True
def get_postun_done(self):
return self.postdone
def __str__(self):
return self.name
def get_package_name(rpmfilename):
rpmoutput = os.popen("rpm --queryformat %{name} -qp " + rpmfilename)
packagename = rpmoutput.read()
status = rpmoutput.close()
if status is not None or packagename == "":
raise Error, "spec file name lookup failed"
else:
print "spec file name lookup passed"
return packagename
def extract_files(rpmfilename, filenamepattern):
status = os.system("rpm2cpio " + rpmfilename + " | cpio -id " +
filenamepattern)
if status > 0:
raise Error, "%s exited with code %d" % ("cpio", status)
elif status < 0:
raise Error, "%s killed by signal %d" % ("cpio", -status)
else:
print "rpm2cpio passed"
def get_jar_file_names_callback(jarfilenames, dirname, fnames):
for fname in fnames:
if fname.endswith(".jar") \
and os.path.isfile(os.path.join(dirname, fname)) \
and not os.path.islink(os.path.join(dirname, fname)):
jarfilenames.append(fname)
os.remove(os.path.join(dirname, fname))
def note_package(rpmfilename):
global packages
unmacroizedpackagename = get_package_name(rpmfilename)
if unmacroizedpackagename != mainpackagename + "-debuginfo":
if unmacroizedpackagename == mainpackagename:
packagename = mainpackagename
else:
packagename = macroizedpackagenames[unmacroizedpackagename]
jarfilenames = []
extract_files(rpmfilename, "*.jar")
os.path.walk(".", get_jar_file_names_callback, jarfilenames)
packages[packagename] = Package(packagename, jarfilenames)
def filter_preamble(line):
if line.lower().startswith("buildarch:"):
if line.lower().split()[1] == "noarch":
add_buildarch(line)
else:
outputspecfile.write(line)
else:
outputspecfile.write(line)
def add_buildarch(line):
outputspecfile.write("%if ! %{gcj_support}\n")
outputspecfile.write(line)
outputspecfile.write("%endif\n")
def add_requires():
outputspecfile.write("%if %{gcj_support}\n")
outputspecfile.write("BuildRequires:\t\tjava-gcj-compat-devel\n")
outputspecfile.write("Requires(post):\t\tjava-gcj-compat\n")
outputspecfile.write("Requires(postun):\tjava-gcj-compat\n")
outputspecfile.write("%endif\n")
outputspecfile.write("\n")
def add_aot_compile_rpm():
outputspecfile.write("%if %{gcj_support}\n")
outputspecfile.write("%{_bindir}/aot-compile-rpm\n")
outputspecfile.write("%endif\n")
outputspecfile.write("\n")
def add_so_and_db_files():
outputspecfile.write("%if %{gcj_support}\n")
outputspecfile.write("%dir %attr(-,root,root) %{_libdir}/gcj/%
{name}\n")
for jarfilename in currentpackage.get_jar_file_names():
outputspecfile.write("%attr(-,root,root) %{_libdir}/gcj/%{name}/"
+ jarfilename + ".*\n")
outputspecfile.write("%endif\n")
outputspecfile.write("\n")
def add_rebuild_gcj_db():
outputspecfile.write("%if %{gcj_support}\n")
add_rebuild_gcj_db_no_condition()
outputspecfile.write("%endif\n")
outputspecfile.write("\n")
def add_rebuild_gcj_db_no_condition():
outputspecfile.write("if [ -x %{_bindir}/rebuild-gcj-db ]\n")
outputspecfile.write("then\n")
outputspecfile.write(" %{_bindir}/rebuild-gcj-db\n")
outputspecfile.write("fi\n")
def add_name(name):
if name.startswith(mainpackagename):
if name != mainpackagename:
outputspecfile.write(" " + name.replace(mainpackagename + "-",
"", 1))
else:
# full name
outputspecfile.write(" -n " + name)
outputspecfile.write("\n")
def add_post(name):
outputspecfile.write("%if %{gcj_support}\n")
outputspecfile.write("%post")
add_name(name)
add_rebuild_gcj_db_no_condition()
outputspecfile.write("%endif\n")
outputspecfile.write("\n")
def add_postun(name):
outputspecfile.write("%if %{gcj_support}\n")
outputspecfile.write("%postun")
add_name(name)
add_rebuild_gcj_db_no_condition()
outputspecfile.write("%endif\n")
outputspecfile.write("\n")
def add_before_part(state):
global doneuptofiles
if state == "%files":
if doneuptofiles == False:
for packagename in packages:
package = packages[packagename]
if package.contains_jars():
if not package.get_post_done():
add_post(package.get_name())
if not package.get_postun_done():
add_postun(package.get_name())
doneuptofiles = True
def add_to_part(state):
if state == "PREAMBLE":
if currentpackage is not None \
and currentpackage.contains_jars():
add_requires()
if state == "%package":
if currentpackage is not None \
and currentpackage.contains_jars():
add_requires()
elif state == "%install":
add_aot_compile_rpm()
elif state == "%files":
if currentpackage is not None \
and currentpackage.contains_jars():
add_so_and_db_files()
elif state == "%post":
if currentpackage is not None \
and currentpackage.contains_jars():
add_rebuild_gcj_db()
currentpackage.set_post_done()
elif state == "%postun":
if currentpackage is not None \
and currentpackage.contains_jars():
add_rebuild_gcj_db()
currentpackage.set_postun_done()
def set_currentpackage(line):
global currentpackage
fields = line.split()
if len(fields) == 1:
if not mainpackageempty:
try:
currentpackage = packages[mainpackagename]
except KeyError, e:
pass
else:
currentpackage = None
else:
if fields[1] == "-n":
# full name specified
try:
currentpackage = packages[fields[2]]
except KeyError, e:
pass
else:
try:
currentpackage = packages[mainpackagename + "-" + fields[1]]
except KeyError, e:
pass
def set_macroized_package_names():
global macroizedpackagenames
packagelines = os.popen("cat " + specfilename + " | grep '^%
package'")
packagenames = os.popen("rpm -q --specfile --queryformat='%{name}
\n' " + specfilename + " | grep -v " + mainpackagename + "-
debuginfo")
packageline = packagelines.readline()
# skip main package name
packagename = packagenames.readline()
packagename = packagenames.readline()
if packagename != "":
while packageline != "":
fields = packageline.split()
if len(fields) == 2:
macroizedpackagenames[packagename.strip()] = mainpackagename +
"-" + fields[1]
else:
if len(fields) == 3:
macroizedpackagenames[packagename.strip()] = fields[2]
packageline = packagelines.readline()
packagename = packagenames.readline()
packagelines.close()
packagenames.close()
if packageline != "" or packagename !="":
raise Error, "number of Name: and %package lines does not match
number of names returned by spec file query"
else:
print "macroized package names creation passed"
print "map of unmacroized names to macroized names:"
print macroizedpackagenames
def next_state(line):
global state
if line == "":
add_to_part(state)
state = "END"
return
for part in partList:
if line.startswith(part):
add_to_part(state)
state = part
add_before_part(state)
set_currentpackage(line)
def print_version():
print NAME, VERSION
sys.exit(0)
def print_usage():
print "Usage: " + NAME + " SRPM RPMS"
print "Create a JPackage spec file that contains GCJ support from
a JPackage"
print "SRPM and its corresponding RPMs. " + NAME + " uses the
binary RPMs"
print "to determine each sub-package's list of jar files."
print ""
print "Example:"
print " " + NAME + " ~/rpmbuild/SRPMS/bsh-*1.3.0-6jpp* ~/rpmbuild/
RPMS/noarch/bsh-*1.3.0-6jpp*"
sys.exit(0)
if __name__ == "__main__":
try:
options = sys.argv[1:]
if len(options) == 0:
print NAME + ": no input files"
print_usage()
origdir = os.getcwd()
# find SRPM within arguments
srpmfilename = ""
rpmfilenames = []
for option in options:
if option.endswith(".src.rpm"):
if srpmfilename != "":
raise Error, "more than one SRPM specified"
else:
srpmfilename = os.path.join(origdir, option)
elif option.endswith(".rpm"):
rpmfilenames.append(os.path.join(origdir, option))
elif option == "--version":
print_version()
elif option == "--help":
print_usage()
warnings.filterwarnings("ignore", message="tmpnam is a potential
security risk to your program")
tmpdir = os.path.basename(os.tmpnam())
os.mkdir(tmpdir)
os.chdir(tmpdir)
# get spec file name: assumes that the spec file is called
# %{name}.spec
print "processing: " + srpmfilename
mainpackagename = get_package_name(srpmfilename)
specfilename = mainpackagename + ".spec"
# extract spec file
print "extracting spec file: " + specfilename
extract_files(srpmfilename, specfilename)
# check for existing gcj support
checkconverted = os.popen("grep gcj_support " + specfilename)
checksupport = checkconverted.readline()
if checksupport != "":
raise Error, "these rpms already support gcj"
status = checkconverted.close()
if status is None:
raise Error, "failed to close grep pipe"
else:
print "no gcj support detected"
# calculate unmacroized name-to-macroized name mapping
set_macroized_package_names()
for rpmfilename in rpmfilenames:
note_package(rpmfilename)
# open spec file for reading
specfile = open(specfilename, 'r')
# open output spec file for writing
outputspecfile = open(os.path.join(origdir, mainpackagename + "-
gcj.spec"), 'w')
# check if there is a main package or only sub-packages
if mainpackagename in packages:
currentpackage = packages[mainpackagename]
else:
mainpackageempty = True
print "main package:", currentpackage
line = specfile.readline()
next_state(line)
# skip license block
while line.startswith("#"):
outputspecfile.write(line)
line = specfile.readline()
outputspecfile.write("\n%define gcj_support %{?_with_gcj_support:
1}%{!?_with_gcj_support:%{?_without_gcj_support:0}%{!?
_without_gcj_support:%{?_gcj_support:%{_gcj_support}}%{!?
_gcj_support:0}}}\n")
while line != "":
if state == "PREAMBLE":
filter_preamble(line)
else:
outputspecfile.write(line)
line = specfile.readline()
next_state(line)
outputspecfile.close()
os.chdir(origdir)
except Error, e:
print >> sys.stderr, "%s: error: %s" % (
os.path.basename(sys.argv[0]), e)
sys.exit(1)
--------------------------------------------------------------------
-
To unsubscribe, e-mail: [EMAIL PROTECTED]
For additional commands, e-mail: [EMAIL PROTECTED]
