Hello again
I'd like to summary recent additions related to my changes:
- feature:install and feature:repo-add won't install/add blacklisted
feature/repository
- etc/org.apache.karaf.features.cfg may use property placeholders and
these will be resolved from bundle context properties and from
etc/versions.properties (configurable). The idea is to have e.g.,:
<bundleReplacements>
<bundle
replacement="mvn:commons-io/commons-io/${version.commons-io}" />
</bundleReplacements>
and just list all the versions we need in etc/versions.properties. So when
fixed bundle comes out, it's a matter of changing property in single file
and all features will use overriden version.
- implemented overriding feature definitions (replace entire feature
definition)
- implemented overriding dependency="true|false" flag in bundles. If we
know that some external features XML should use this flag and it doesn't,
we can configure it externally without trying to have our PR merged in
upstream project.
- profiles can use "blacklisted.bundle.xxx", "blacklisted.feature.xxx" and
"blacklisted.repository.xxx" properties (KARAF-5339)
- features specified in karaf-maven-plugin:assembly configuration may use
wildcards (KARAF-5273)
I still hope to get some comments/review ;) I know it's not quick fix. If
you think I went in wrong direction, please let me know.
best regards
Grzegorz Grzybek
2017-12-02 21:27 GMT+01:00 Grzegorz Grzybek <gr.grzy...@gmail.com>:
> Hello
>
> I want to report about my progress with https://issues.apache.org/
> jira/browse/KARAF-5376 (Processor mechanism for feature definitions
> (a.k.a. "better overrides")).
>
> *Goal* - to replace (or provide more flexible alternative) of "overrides"
> and "blacklisting" mechanisms
>
> *Idea* - Karaf features come (normally) from features XML files,
> according to XML Schema specified by Karaf itself. My idea was that we can
> "intercept" the process of loading the repositories and alter the
> definitions. Effectively we could then affect the definitions which may be
> out of our influence, or use some bundles we'd rather like to replace with
> others. Some feature repositories include other repositories (e.g., Camel
> 2.19.2 features referencing CXF 3.1.11 features) and we'd rather like to
> avoid it.
>
> *Problems with blacklisting and overrides* - blacklisting is purely
> static thing - applied at distro creation time. Overrides are a bit awkward
> to use ("range" clause) and have some implicit behavior (requiring runtime
> access to resource, to verify version and symbolic name).
>
> *What I did*
>
> In short words - I've created a mechanism steered by
> etc/org.apache.karaf.features.xml (name matching
> etc/org.apache.karaf.features.cfg PID) that looks roughly like this:
>
> <featuresProcessing xmlns="http://karaf.apache.
> org/xmlns/features-processing/v1.0.0"
> xmlns:f="http://karaf.apache.org/xmlns/features/v1.5.0";>
>
> <blacklistedRepositories>
> <repository>mvn:org.hibernate/hibernate-validator-osgi-
> karaf-features/[5,*)/xml/features</repository>
> <!-- ... -->
> </blacklistedRepositories>
> <blacklistedFeatures>
> <feature>*jetty*</feature>
> <feature version="[2,3)">*jclouds*</feature>
> <!-- ... -->
> </blacklistedFeatures>
> <blacklistedBundles>
> <bundle>mvn:commons-logging/*</bundle>
> </blacklistedBundles>
>
> <overrideBundleDependency>
> <!-- Override "dependency" flag for all bundles of all features
> for given repository URI(s) -->
> <repository
> uri="mvn:org.ops4j.pax.cdi/pax-cdi-features/*/xml/features"
> dependency="true" />
> <repository uri="mvn:*/xml/features" dependency="true" />
> <!-- Override "dependency" flag for all bundles of given
> feature(s) -->
> <feature name="jclouds*" version="[1,3)" dependency="true" />
> <!-- Override "dependency" flag for given bundle(s) -->
> <bundle uri="mvn:javax.annotation/javax.annotation-api/*"
> dependency="true" />
> ...
> </overrideBundleDependency>
>
> <bundleReplacements>
> <bundle originalUri="mvn:commons-beanutils/commons-beanutils/[
> 1.9,2)"
> replacement="mvn:commons-beanutils/commons-beanutils/1.9.3"
> />
> <bundle
> replacement="mvn:commons-collections/commons-collections/3.2.2"
> />
> <bundle originalUri="mvn:org.eclipse.jetty.orbit/javax.servlet/[3,
> 4)"
>
> replacement="mvn:org.apache.geronimo.specs/geronimo-servlet_3.0_spec/1.0"
> mode="maven" />
> <!-- ... -->
> </bundleReplacements>
>
> <featureReplacements>
> <replacement mode="replace">
> <feature name="pax-jsf-resources-support"
> description="Provide sharing of resources according to Servlet 3.0 for OSGi
> bundles and JSF" version="6.0.7">
> <f:feature version="[6.0,6.1)">pax-jsf-support</f:feature>
> <f:bundle dependency="true">mvn:org.ops4j.pax.web/pax-web-
> resources-extender/6.0.7</f:bundle>
> <f:bundle>mvn:org.ops4j.pax.web/pax-web-resources-jsf/6.0.
> 7</f:bundle>
> </feature>
> </replacement>
> <!-- ... -->
> </featureReplacements>
>
> </featuresProcessing>
>
> Description:
>
> - blacklistedRepositories, blacklistedFeatures and blacklistedBundles
> mean what they mean - mark (instead of removing) given item kind from JAXB
> model of loaded features XML. This information is preserved at runtime, and
> for example is displayed with "features:list" command
>
> - overrideBundleDependency (not yet implemented) - is a method of
> overriding "dependency="true|false"" flags in feature definitions
>
> - bundleReplacements (implemented) - is an extension of "overrides"
> mechanism. In "osgi" mode, it roughly works like before - requires access
> to runtime ResourceImpl, to check headers and compare versions and symbolic
> names. In "maven" mode it's just static change of given bundle URI - any
> feature declaring <bundle>originalUri</bundle> will be seen as declaring
> <bundle>replacement</bundle>. This allows for replacement of e.g., SMX
> specs bundle with javax.* bundle (or vice versa).
>
> - featureReplacements (not yet implemented) - is a way of changing (or
> altering) any features by name (or pattern) and version simply by providing
> different definition of given feature. This could be useful if we as Karaf
> users know "better" than authors of original features, for whom OSGi and
> Karaf were rather an afterthought.
>
> Both bundle/feature/repository blacklisting and bundle/feature override
> use two new classes to "match" candidate to replace/override:
> - org.apache.karaf.features.LocationPattern - matches URI (of bundle or
> repository) in two modes - for non mvn: URI, it just uses a string with
> possible "*" glob. For mvn: URI, it splits the URI into well known
> components (groupId, artifactId, version, type and classifier). For version
> it can use version range (e.g, "[2.1,2.2)") and for other components it can
> use "*" glob.
> - org.apache.karaf.features.FeaturePattern - matches feature by name
> (possibly containing "*" glob) and version range (org.apache.felix.utils.
> version.VersionRange)
>
> Universally applicable:
>
> FeaturesProcessor generally postprocesses
> org.apache.karaf.features.internal.model.Features
> instance(s) and marks items as blacklisted if they match given pattern and
> changes the model (according to overrides/replacements).
>
> The behavior after processing JAXB model of features is exactly the same
> both when using FeaturesServiceImpl at runtime and when using profile
> Builder during custom assembly creation. For example changed flow of
> karaf-maven-plugin:asembly invocation looks like this:
>
> [INFO] --- karaf-maven-plugin:4.2.0-SNAPSHOT:assembly (default-assembly)
> @ test-karaf-plugins ---
> [INFO] Using repositories:
> [INFO] https://repo.maven.apache.org/maven2@id=central
> ...
> [INFO] Custom startup KAR found: mvn:grgr.test.karaf.plugins/
> my-kar/0.1.0.BUILD-SNAPSHOT/kar
> ...
> [INFO] Unzipping kars
> [INFO] processing KAR: mvn:grgr.test.karaf.plugins/
> my-kar/0.1.0.BUILD-SNAPSHOT/kar
> [INFO] found repository: mvn:grgr.test.karaf.plugins/
> my-kar/0.1.0.BUILD-SNAPSHOT/xml/features
> [INFO] Found features processor configuration: ../classes/etc/org.apache.
> karaf.features.xml
> ...
> [INFO] Loading profiles from:
> [INFO] file:///home/ggrzybek/sources/_testing/grgr-test-karaf-
> plugins/src/main/profiles/1
> [INFO] jar:mvn:test/profiles/42!/profiles/2
> [INFO] Found profiles: p1a, p1b-another, p2b, p2c-two, p2c-one-two, p2a
> ...
> [INFO] Generating features processor configuration:
> etc/org.apache.karaf.features.xml
> [INFO] Startup stage
> [INFO] Loading startup repositories
> [INFO] Resolving startup features and bundles
> [INFO] Features: undertow, my-startup-feature
> ...
> [INFO] skipping blacklisted maven bundle: mvn:org.ops4j.pax.tipi/org.
> ops4j.pax.tipi.undertow.websocket-jsr/1.3.25.1
> ...
> [INFO] Boot stage
> [INFO] Loading boot repositories
> [INFO] adding feature repository: mvn:org.ops4j.pax.web/pax-web-
> features/6.0.7/xml/features
> [INFO] adding feature repository: mvn:grgr.test.karaf.plugins/
> my-kar/0.1.0.BUILD-SNAPSHOT/xml/features
> [INFO] Feature undertow/1.3.25.1 is defined as a boot feature
> ...
> [INFO] adding overriden maven artifact:
> mvn:org.jboss.logging/jboss-logging/3.3.1.Final
> (original location: mvn:org.jboss.logging/jboss-logging/3.3.0.Final)
> ...
> [INFO] skipping blacklisted maven artifact:
> mvn:org.ops4j.pax.tipi/org.ops4j.pax.tipi.undertow.websocket-jsr/1.3.25.1
> [INFO] adding overriden maven artifact:
> mvn:javax.servlet/javax.servlet-api/3.1-b09
> (original location: mvn:javax.servlet/javax.servlet-api/3.1.0)
> ...
> [INFO] Install stage
> [INFO] Loading installed repositories
>
> overrides and blacklisting information may come from karaf-maven-plugin
> configuration (in POM), from externally provided
> org.apache.karaf.features.xml (e.g., in resource KAR or in POM:
> <configuration>/<featuresProcessing>) or from profiles.
>
> There are still few things to polish, but I'll appreciate any feedback to
> check if I misunderstood something or there were other plans related to
> these mechanisms. The changes are in https://github.com/apache/
> karaf/commits/KARAF-5376-overrides_v2 branch.
>
> best regards
> Grzegorz Grzybek
>
