Hello!
I'd like to discuss an issue we have with "Tizen on Yocto": how do we
handle conditional compilation?
In Tizen, the preferred approaches are bcond and %ifarch checks in a
single .spec file used by all profiles. A new profile can be created
based on these .spec files without modifying them, just by changing the
OBS build config of the profile.
In Yocto, similar checks are possible with PACKAGECONFIG. In fact, there
is (currently unused) code in the meta-translator which maps these
yes/no decisions and their effect on configure flags and dependencies to
bcond.
But architecture and distro (with "Yocto distro" matching the "Tizen
profile") specific changes are already handled such that detecting the
influence of changing some build config options (typically MACHINE,
DISTRO_FEATURES and/or TUNE_FLAGS) is harder. For example, there is
embedded Python code checking these variables.
What works is a brute-force approach: configure all combinations of
architecture, profile and key features (like wayland vs. x11), run the
conversion in each build config, then compare the outcome. Due to the
way how the converter works (it runs as a class inside a single build
config), this is best done as a post-processing step.
This approach also works when different build configurations require
different layers which add recipes or modify shared ones (for example,
meta-tizen-ivi is only used when building packages for an IVI image and
is meant to have a .bbappend file which changes the shared tlm recipe
such that it installs IVI specific configuration files).
There is one problem inherent to this approach: introducing a new build
configuration, be it with a new profile, architecture, or combination of
features, requires defining the build with Yocto meta data first, then
updating the converter setup and re-creating the .spec files derived
from Yocto recipes. It is no longer possible to create new profiles only
in OBS if these derived recipes need to be compiled differently than in
an existing build.
That the comparison between two builds for different profiles ends up
creating "if %{profile} == foo" checks is part of that problem.
But I can't think of a better solution, so that's all that I can offer.
Better suggestions are welcome :-/
I have a prototype script ready which already implements the merging
of .spec files. The handling of the source and patch files is still
missing. I'm attaching the current state of the script because it
contains a description of my setup and the resulting .spec file for
cairo. Cairo is a good example because it contains both arch-specific
changes (--disable-some-floating-point is used by Yocto only on ARM) and
feature checks (optional X11 features).
The current script intentionally only generates "if <something> endif"
sections. Only code common to all build configs is used unconditionally.
In particular there is no "else" default branch, because it is not clear
which side should be the default (x86 or ARM? wayland or x11?).
The if checks are unnecessarily complex at the moment because they get
created mechanically, without any logic for simplifying them. I intend
to add that shortly and will post an update when done, so please don't
get put off by that complexity in the meantime.
--
Best Regards, Patrick Ohly
The content of this message is my personal opinion only and although
I am an employee of Intel, the statements I make here in no way
represent Intel's position on the issue, nor am I authorized to speak
on behalf of Intel on this matter.
%bcond_with wayland
%bcond_with x11
Name: cairo
Version: 1.12.16
Release: 0
License: MPL-1.0 and LGPL-2.1
Summary: The Cairo 2D vector graphics library
Group: Graphics & UI Framework/Libraries
Url: http://cairographics.org
Source0: http://cairographics.org/releases/cairo-1.12.16.tar.xz
Source1: tizen-default.manifest
Source2: file-list-generator.py
Patch0: 0001-Remove-LTO-support.patch
BuildRequires: autoconf
BuildRequires: automake
%if "%{profile}" == "common"
%if %{with x11}
BuildRequires: libX11-devel
BuildRequires: libsm-devel
%endif
%endif
BuildRequires: libtool
%if "%{profile}" == "common"
%if %{with x11}
BuildRequires: libxext-devel
BuildRequires: libxrender-devel
%endif
%endif
BuildRequires: pkg-config
BuildRequires: pkgconfig(egl)
BuildRequires: pkgconfig(fontconfig)
BuildRequires: pkgconfig(glesv2)
BuildRequires: pkgconfig(glib-2.0)
BuildRequires: pkgconfig(libpng)
BuildRequires: pkgconfig(pixman-1)
BuildRequires: python
BuildRequires: zlib-devel
%description
Cairo is a multi-platform library providing anti-aliased vector-based
rendering for multiple target backends. Paths consist of line segments and
cubic splines and can be rendered at any width with various join and cap
styles. All colors may be specified with optional translucence
(opacity/alpha) and combined using the extended Porter/Duff compositing
algebra as found in the X Render Extension.
%package -n libcairo
Summary: The Cairo 2D vector graphics library
%description -n libcairo
Cairo is a multi-platform library providing anti-aliased vector-based
rendering for multiple target backends. Paths consist of line segments and
cubic splines and can be rendered at any width with various join and cap
styles. All colors may be specified with optional translucence
(opacity/alpha) and combined using the extended Porter/Duff compositing
algebra as found in the X Render Extension.
%package -n cairo-gobject
Summary: The Cairo library GObject wrapper library
Group: Graphics & UI Framework/Libraries
%description -n cairo-gobject
A GObject wrapper library for the Cairo API.
%package -n cairo-script-interpreter
Summary: The Cairo library script interpreter
Group: Graphics & UI Framework/Libraries
%description -n cairo-script-interpreter
The Cairo script interpreter implements CairoScript. CairoScript is used
by tracing utilities to enable the ability to replay rendering.
%package -n cairo-perf-utils
Summary: The Cairo library performance utilities
Group: Graphics & UI Framework/Libraries
%description -n cairo-perf-utils
The Cairo library performance utilities
%package -n cairo-devel
Summary: The Cairo 2D vector graphics library - Development files
Group: Graphics & UI Framework/Libraries
Requires: libcairo = %{version}-%{release}
%description -n cairo-devel
Cairo is a multi-platform library providing anti-aliased vector-based
rendering for multiple target backends. Paths consist of line segments and
cubic splines and can be rendered at any width with various join and cap
styles. All colors may be specified with optional translucence
(opacity/alpha) and combined using the extended Porter/Duff compositing
algebra as found in the X Render Extension. This package contains symbolic
links, header files, and related items necessary for software development.
%package -n cairo-doc
Summary: The Cairo 2D vector graphics library - Documentation files
Group: Graphics & UI Framework/Libraries
%description -n cairo-doc
Cairo is a multi-platform library providing anti-aliased vector-based
rendering for multiple target backends. Paths consist of line segments and
cubic splines and can be rendered at any width with various join and cap
styles. All colors may be specified with optional translucence
(opacity/alpha) and combined using the extended Porter/Duff compositing
algebra as found in the X Render Extension. This package contains
documentation.
%package -n cairo-locale
Summary: The Cairo 2D vector graphics library
Group: Graphics & UI Framework/Libraries
%description -n cairo-locale
Cairo is a multi-platform library providing anti-aliased vector-based
rendering for multiple target backends. Paths consist of line segments and
cubic splines and can be rendered at any width with various join and cap
styles. All colors may be specified with optional translucence
(opacity/alpha) and combined using the extended Porter/Duff compositing
algebra as found in the X Render Extension.
%prep
%setup -q -n %{name}-%{version}
%patch0 -p1
cp %{SOURCE1} .
%build
export ac_cv_lib_bfd_bfd_openr="no"
export ac_cv_lib_lzo2_lzo2a_decompress="no"
export CFLAGS=" -ffat-lto-objects"
do_configure() {
autotools_do_configure
}
autotools_do_configure() {
if [ -e %{_builddir}/cairo-1.12.16/configure.in -o -e %{_builddir}/cairo-1.12.16/configure.ac ]; then
if [ x"default" = xdefault ]; then
acpaths=
for i in `find %{_builddir}/cairo-1.12.16 -maxdepth 2 -name \*.m4|grep -v 'aclocal.m4'| \
grep -v 'acinclude.m4' | grep -v 'aclocal-copy' | sed -e 's,\(.*/\).*$,\1,'|sort -u`; do
acpaths="$acpaths -I $i"
done
else
acpaths="default"
fi
autoreconf --verbose --install --force --exclude=autopoint $acpaths
fi
if [ -e %{_builddir}/cairo-1.12.16/configure ]; then
%configure \
%if "%{profile}" == "common"
%if %{with wayland}
%ifarch armv7l i586
%ifarch armv7l
--disable-some-floating-point \
%endif
%endif
%endif
%if %{with x11}
%ifarch armv7l i586
%ifarch armv7l
--disable-some-floating-point \
%endif
%endif
%endif
%endif
--enable-tee \
--enable-egl=yes \
--enable-glesv2 \
--disable-valgrind \
%if "%{profile}" == "common"
%if %{with wayland}
--without-x \
%endif
%if %{with x11}
--with-x=yes \
%endif
%endif
%if "%{profile}" == "ivi"
--without-x \
%endif
fi
}
# Now configure
do_configure
cd %{_builddir}/%{name}-%{version}
export ac_cv_lib_bfd_bfd_openr="no"
export ac_cv_lib_lzo2_lzo2a_decompress="no"
export CFLAGS=" -ffat-lto-objects"
do_compile() {
base_do_compile
}
base_do_compile() {
make %{?_smp_mflags}
}
# Now build
do_compile
cd %{_builddir}/%{name}-%{version}
%install
export ac_cv_lib_bfd_bfd_openr="no"
export ac_cv_lib_lzo2_lzo2a_decompress="no"
export CFLAGS=" -ffat-lto-objects"
do_install() {
autotools_do_install
rm -rf %{buildroot}%{_bindir}/cairo-sphinx
rm -rf %{buildroot}%{_libdir}/cairo/cairo-fdr*
rm -rf %{buildroot}%{_libdir}/cairo/cairo-sphinx*
rm -rf %{buildroot}%{_libdir}/cairo/.debug/cairo-fdr*
rm -rf %{buildroot}%{_libdir}/cairo/.debug/cairo-sphinx*
}
autotools_do_install() {
%make_install
}
# Now install
do_install
cd %{_builddir}/%{name}-%{version}
cat >sub-cairo-gobject-file.list <<EOF
%{_libdir}/libcairo-gobject.so.*
EOF
cat >sub-cairo-script-interpreter-file.list <<EOF
%{_libdir}/libcairo-script-interpreter.so.*
EOF
cat >sub-cairo-perf-utils-file.list <<EOF
%{_bindir}/cairo-trace
%{_libdir}/cairo/libcairo-trace.so.*
EOF
cat >sub-cairo-devel-file.list <<EOF
%{_includedir}
/lib/lib*.so
%{_libdir}/lib*.so
%{_libdir}/*.la
%{_libdir}/*.o
%{_libdir}/pkgconfig
%{_datadir}/pkgconfig
%{_datadir}/aclocal
/lib/*.o
%{_libdir}/cairo/*.la
/lib/*.la
%{_libdir}/cairo/*.la
%{_libdir}/cairo/*.so
EOF
cat >sub-cairo-doc-file.list <<EOF
%{_datadir}/doc
%{_mandir}
%{_infodir}
%{_datadir}/gtk-doc
%{_datadir}/gnome/help
EOF
cat >sub-cairo-locale-file.list <<EOF
%{_datadir}/locale
EOF
cat >main-file.list <<EOF
%{_libdir}/libcairo.so.*
EOF
%{__python} $RPM_SOURCE_DIR/file-list-generator.py -d "--compress=/(info|man)/" %{buildroot} sub-cairo-gobject sub-cairo-script-interpreter sub-cairo-perf-utils sub-cairo-devel sub-cairo-doc sub-cairo-locale main
%post -n cairo-gobject -p /sbin/ldconfig
%postun -n cairo-gobject -p /sbin/ldconfig
%post -n cairo-script-interpreter -p /sbin/ldconfig
%postun -n cairo-script-interpreter -p /sbin/ldconfig
%post -n libcairo -p /sbin/ldconfig
%postun -n libcairo -p /sbin/ldconfig
%files -n libcairo -f main-spec.list
%manifest tizen-default.manifest
%files -n cairo-gobject -f sub-cairo-gobject-spec.list
%manifest tizen-default.manifest
%files -n cairo-script-interpreter -f sub-cairo-script-interpreter-spec.list
%manifest tizen-default.manifest
%files -n cairo-perf-utils -f sub-cairo-perf-utils-spec.list
%manifest tizen-default.manifest
%files -n cairo-devel -f sub-cairo-devel-spec.list
%manifest tizen-default.manifest
%files -n cairo-doc -f sub-cairo-doc-spec.list
%manifest tizen-default.manifest
%files -n cairo-locale -f sub-cairo-locale-spec.list
%manifest tizen-default.manifest
#! /usr/bin/python
#
# Permission is hereby granted, free of charge, to any person obtaining a copy
# of this software and associated documentation files (the "Software"), to deal
# in the Software without restriction, including without limitation the rights
# to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
# copies of the Software, and to permit persons to whom the Software is
# furnished to do so, subject to the following conditions:
#
# The above copyright notice and this permission notice shall be included in
# all copies or substantial portions of the Software.
#
# THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
# IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
# FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
# AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
# LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
# OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN
# THE SOFTWARE.
# After running the meta-translator in different build configurations, run
# this tool to combine the results with one spec file per recipe. The combined
# spec file uses '%ifarch <arch>', '%if "%{profile}" == "<profile>"' and
# "%bcond_with foo %if %{with foo}" to make lines specific to certain build
# configurations.
#
# To use it, organize build directories hierarchically such that all directories
# in the same parent directory only differ by one particular attribute, for
# example the target architecture.
#
# In Tizen, one might build for common vs. ivi, armv7l vs. x86_64 vs. i586,
# wayland vs x11 and organize that as follows:
# build/srpm/
# common/
# wayland/
# armv7l
# i586
# x86_64
# x11/
# armv7l
# i586
# x86_64
# ivi/
# armv7l
# i586
# x86_64
#
# The "ivi" profile only supports wayland, so there is no need for
# that distinction underneath it.
#
# Then run the script with parameters telling it what the if check
# for each directory needs to be when merging at that level, with
# --check=<path wildcard>:<check>[:<value>].
#
# Each if check is applied to all directories matching it. The value
# to check for is optional and the same as the directory name if not
# given.
#
# For the setup above, that leads to these if checks below when running inside
# the "build" directory. Here is the full example:
#
# merge-srpm.py --check=*/common:profile \
# --check=*/ivi:profile \
# --check=*/wayland:bcond \
# --check=*/x11:bcond \
# --check=*/armv7l:arch \
# --check=*/i586:arch \
# --check=*/x86_64:arch \
# --input=srpm \
# --output=merged \
# pango cairo expat
import difflib
import fnmatch
import glob
import logging
import optparse
import os
import re
import sys
CHECK_METAVAR="<path shell pattern>:profile|bcond|arch[:<value>]"
parser = optparse.OptionParser(usage="usage: %prog [options] recipe-1 recipe-2 ...")
parser.add_option("-q", "--quiet", action="store_true", default=False,
help="suppress regular output")
parser.add_option("-d", "--debug", action="store_true", default=False,
help="enable debugging output, overrides --quiet")
parser.add_option("-c", "--check", metavar=CHECK_METAVAR, action="append",
help="Apply the specified check to directories matching the pattern.")
parser.add_option("", "--ignore-missing", action="store_true", default=False,
help="Do not treat missing .spec files as error.")
parser.add_option("-i", "--input", default=None, metavar="DIRECTORY",
help="Find build directories underneath the given directory.")
parser.add_option("-o", "--output", default=None, metavar="DIRECTORY|-",
help="Create one directory per recipe underneath that directory or write merged .spec files to stdout.")
(options, recipes) = parser.parse_args()
if options.debug:
level=logging.DEBUG
elif options.quiet:
level=logging.WARNING
else:
level=logging.INFO
logging.basicConfig(level=level,
format='%(levelname)s: %(message)s')
# Parse checks.
for i, check in enumerate(options.check):
t = check.split(':')
if len(t) < 2 or len(t) > 3:
logging.error('--check "%s" does not have the format %s' % (check, CHECK_METAVAR))
exit(1)
options.check[i] = t
if t[1] not in ('profile', 'bcond', 'arch'):
logging.error('--check "%s" uses invalid check: %s' % (check, t[1]))
exit(1)
# Global variables for the current recipe: bcond header and running
# counter for each new conditional section.
bconds = []
prefixcounter = 0
# Figure out what check to use for a certain directory. May update the
# list of glob bconds.
def findcheck(root, dir):
tocheck = os.path.join(os.path.relpath(root, options.input), dir)
logging.debug('Looking up check for directory %s.' % tocheck)
for check in options.check:
if fnmatch.fnmatch(tocheck, check[0]):
value = check[2] if len(check) > 2 else os.path.basename(tocheck)
check = check[1]
if check == 'profile':
return ['%if', '"%%{profile}" == "%s"' % value]
elif check == 'bcond':
header = '%%bcond_with %s\n' % value
if header not in bconds:
bconds.append(header)
return ['%if', '%%{with %s}' % value]
elif check == 'arch':
return ['%ifarch', value]
logging.error('No check configured for directory %s.' % tocheck)
exit(1)
# Now process each recipe.
def merge(recipe, root):
if os.path.isdir(os.path.join(root, 'conf')):
# Found an individual build. Return the .spec file, line by line.
pattern = os.path.join(root, 'tmp*', 'deploy', 'rpm', 'specs', recipe + '.spec')
specs = glob.glob(pattern)
if not specs:
if options.ignore_missing:
return None
else:
logging.error('.spec file not found: ' + pattern)
exit(1)
elif len(specs) > 1:
logging.error('.spec file ambiguous: ' + pattern)
exit(1)
logging.debug('Reading %s.' % specs[0])
return open(specs[0]).readlines()
else:
# Need to merge result from each sub-directory.
dirs = [x for x in os.listdir(root) if os.path.isdir(os.path.join(root, x))]
dirs.sort()
logging.debug('Merging %s from directories %s in %s.' % (recipe, dirs, root))
specs = {}
for d in dirs:
spec = merge(recipe, os.path.join(root, d))
# Ignore empty directories.
if spec is not None:
specs[d] = spec
# No result at all from this directory?
if not specs:
return None
dirs = specs.keys()
dirs.sort()
# First one is the default, others need to be merged in.
merged = specs[dirs[0]]
firstcheck = findcheck(root, dirs[0])
for i in range(1, len(dirs)):
left = merged
right = specs[dirs[i]]
secondcheck = findcheck(root, dirs[i])
diff = difflib.SequenceMatcher(None, left, right).get_matching_blocks()
merged = []
lastleft = 0
lastright = 0
def emitcond(original, last, current, length, check):
global prefixcounter
if last < current:
# When doing further diffs, we must not match against this conditional
# section because the lines might not be active. Using a unique prefix
# prevents such undesired matches.
prefix = '*%04d: ' % prefixcounter
prefixcounter += 1
merged.append(prefix + ' '.join(check) + '\n')
merged.extend([prefix + line for line in original[last:current]])
merged.append(prefix + '%endif\n')
return current + length
for l, r, n in diff:
# Handle differences.
lastleft = emitcond(left, lastleft, l, n, firstcheck)
lastright = emitcond(right, lastright, r, n, secondcheck)
# Common block. Guaranteed to be emitted at the end.
merged.extend(left[l:l+n])
# Update the check for the merged spec file, in case we compare
# it against more spec files.
if firstcheck[0] != secondcheck[0]:
logging.error('%s: two different kinds of checks needed, which is not supported: %s != %s' %
(root, firstcheck, secondcheck))
exit(1)
if firstcheck[0] == '%ifarch':
firstcheck.extend(secondcheck[1:])
else:
# TODO: Not sure whether this valid syntax for %if.
firstcheck.extend(['||'] + secondcheck[1:])
return merged
for recipe in recipes:
bconds = []
prefixcounter = 0
spec = merge(recipe, options.input)
if spec is None:
logging.error('No .spec file found for %s.' % recipe)
exit(1)
# Strip out the special conditional line prefixes that were used
# to prevent overlapping conditional sections.
strip = re.compile(r'^(\*\d+: )+')
spec = [strip.sub('', x) for x in spec]
if bconds:
bconds.sort()
spec[0:0] = bconds + ['\n']
if options.output == '-':
sys.stdout.writelines(spec)
else:
output = os.path.join(options.output, recipe)
if os.path.isdir(output):
# TODO: update directory
pass
else:
os.mkdir(output)
specfile = open(os.path.join(output, recipe + '.spec'), 'w').writelines(spec)
specfile.close()
_______________________________________________
Dev mailing list
[email protected]
https://lists.tizen.org/listinfo/dev
