Right. Not sure it's worth it, but why not. I still think the best location for the osgi-compatible version resolver is in pax-url-aether, as it's really the only place it will be used afaik, but again, it's no big deal.
2017-02-01 14:14 GMT+01:00 Łukasz Dywicki <[email protected]>: > What I was thinking about was something like that: > AetherBasedResolver.class.getResources("META-INF/pax- > customization.properties") > then just parsing it and pushing into ServiceLocator instance created > by MavenRepositorySystemUtil. We can do that because > DefaultServiceLocator implementation is mutable as Grzegorz pointed > out. Since we will be in internal package getResources should behave > properly in both cases. > > Cheers, > Lukasz > > > > 2017-02-01 12:54 GMT+01:00 Guillaume Nodet <[email protected]>: > > 2017-02-01 12:35 GMT+01:00 Łukasz Dywicki <[email protected]>: > > > >> @Guillaume you are right on findEntries part, this will break with > >> maven execution. > >> > > > > What findEntries part are you talking about ? > > I was just referring to the fact that a modified version of > > MavenRepositorySystemUtil class would be better embedded directly into > > pax-url-aether. Else, you'd have to provide 2 alternative ways to use it, > > one by using a fragment and the other one by ensuring that the hacked > class > > is loaded first in the classloader (in case of a simple classloader). > > It just seems easier to me to just move that class in the shaded > > pax-url-aether and hack it so that the behavior can be easily controlled > > using a boolean on the usual pax-url-aether configuration. > > > > > > > >> > >> @Grzegorz I do not negate that version, metadata or range resolving > >> works. It does with some small exceptions (see PAXURL-342). What > >> doesn't work is re-use of ranges from maven build into karaf runtime. > >> When you use maven <dependency> and aether to build custom assembly > >> feature resolver in runtime will not work for certain cases because of > >> version edges. > >> When you will take a look on these two files: > >> https://github.com/splatch/maven-osgi-resolver/blob/ > >> master/shared/src/test/java/org/code_house/maven/osgi/ > >> resolver/shared/version/OsgiVersionRangeTest.java > >> https://github.com/splatch/maven-osgi-resolver/blob/ > >> master/test/src/test/java/org/eclipse/aether/util/version/ > >> GenericVersionRangeTest.java > >> and diff between them https://www.diffchecker.com/eyzDSSXU you will > >> find quite big area for troubles. > >> > >> We about to start using ranges in build because we do not want to have > >> 3 different minor versions installed without need. Using ranges in > >> both places acutally might speed up running integration tests because > >> metadata is already fetched and present in local repository. Otherwise > >> our integration tests bootstrap takes 2 minutes just to scan remotes > >> for new versions (see above pax url issue). > >> > >> Cheers, > >> Lukasz > >> > >> 2017-02-01 12:02 GMT+01:00 Grzegorz Grzybek <[email protected]>: > >> > Hello, > >> > > >> > Current pax-url-aether has already some custom services DInjected into > >> > RepositorySystem, like here[1]. Also in Fabric8v1 and in Karaf I did > some > >> > tricks to implement non-canonical "update releases" scenario[2]. > >> > So I think adding configuration options for pax-url to modify the way > >> > RepositorySystem is configured should not be a problem. > >> > > >> > What is the ultimate problem you want to solve? Is it (at lowest > level) > >> the > >> > ability to handle the below URLs?: > >> > > >> > osgi:install mvn:groupId/artifactId/[lowerBound, upperBound) > >> > > >> > Currently (pax-url 2.5.2) LATEST, RELEASE and SNAPSHOT versions > should be > >> > handled correctly[3]: > >> > - LATEST instructs AetherBasedResolver to fetch group/artifact > >> > metadata.xml and pick latest release OR snapshot > >> > - RELEASE instructs AetherBasedResolver to fetch group/artifact > >> > metadata.xml and pick latest release > >> > - SNAPSHOT instructs AetherBasedResolver to fetch > group/artifact/version > >> > metadata.xml and pick latest snapshot > >> > > >> > e.g., in Fabric8v1 I added custom org.eclipse.aether.impl. > >> MetadataResolver > >> > that is able to resolve metadata ("maven-metadata.xml") even in local > >> > repositories into which a SNAPSHOT was installed using `mvn clean > >> install` > >> > - so the metadata is stored in "maven-metadata-local.xml" file - but > the > >> > repo is used as remote repository (expecting to return > >> "maven-metadata.xml" > >> > file. > >> > > >> > Are you using same version ranges in POM and in features.xml? (I don't > >> > argue with the fact that version ranges are used at all in POM :). > >> > > >> > regards > >> > Grzegorz > >> > === > >> > [1]: > >> > https://github.com/ops4j/org.ops4j.pax.url/blob/master/pax- > >> url-aether/src/main/java/org/ops4j/pax/url/mvn/internal/ > >> AetherBasedResolver.java#L1168-L1169 > >> > [2]: > >> > https://ops4j1.jira.com/browse/PAXURL-322? > focusedCommentId=37006&page= > >> com.atlassian.jira.plugin.system.issuetabpanels:comment- > >> tabpanel#comment-37006 > >> > [3]: http://ggrzybek.blogspot.com/2016/10/using-maven-with-osgi- > >> part-3.html > >> > > >> > 2017-02-01 11:44 GMT+01:00 Guillaume Nodet <[email protected]>: > >> > > >> >> 2017-02-01 11:31 GMT+01:00 Łukasz Dywicki <[email protected]>: > >> >> > >> >> > Thanks for your repiles. If we will manage to get pax-url accepting > >> >> > different version range resolving than maven default then I think > we > >> >> > will not have any troubles with features left. What I was thinking > >> >> > about is moving my maven-osgi-resolver to karaf tooling and > extending > >> >> > pax-url in the way it could pick up version range resolver > >> >> > implementation from fragment bundle. This way we could keep current > >> >> > behavior which might be used by someone but also let others use > end to > >> >> > end range support. There are more "extension points" built into > >> >> > Aerther which gets normally wired by IoC. Since we can't and we do > not > >> >> > want to embed yet-another-ioc-tool for low level stuff we would > just > >> >> > need to make aether's ServiceLocator entries customizable. It is > >> >> > simple Map between role and implementation classes thus would not > >> >> > require anything more than bundle.findEntries. This way we could > also > >> >> > solve pax-url troubles with wagon not loaded up properly. > >> >> > > >> >> > >> >> Won't that make things a bit more complicated for the karaf maven > >> plugin ? > >> >> It does not run in OSGi, so the fragment stuff won't work. If the > >> problem > >> >> is the compatibility, it may still be easier to put the code in > >> >> pax-url-eather, and only have a flag to turn the version resolver > into > >> an > >> >> OSGi compatible one, so that the default would be unchanged. > >> >> I honestly don't mind, I'm just trying to find the best way to handle > >> that. > >> >> > >> >> > >> >> > @Guillaume, we don't need to handle RELEASE flag because this part > is > >> >> > not subject of version range resolution but VersionResolver. This > is > >> >> > piece of logic we would not (hopefully) need to amend. > >> >> > > >> >> > If you will take a look on my current implementation there is mixed > >> >> logic: > >> >> > https://github.com/splatch/maven-osgi-resolver/blob/ > >> >> > master/compatible/src/main/java/org/code_house/maven/ > >> >> > osgi/resolver/compatible/CompatibleOsgiVersionRangeReso > lver.java#L87 > >> >> > https://github.com/splatch/maven-osgi-resolver/blob/ > >> >> > master/strict/src/main/java/org/code_house/maven/osgi/ > >> resolver/strict/ > >> >> > StrictOsgiVersionRangeResolver.java#L80 > >> >> > > >> >> > First implementation uses Maven ordering of versions meaning it > >> >> > preffers releases over snapshots in selected range. Second > >> >> > implementation behaves as OSGi, meaning it will ignore snapshot and > >> >> > use regular qualifier comparision but more importantly it will also > >> >> > accept just 3.4.0 as a range without upper bound. > >> >> > > >> >> > Best regards, > >> >> > Lukasz > >> >> > > >> >> > 2017-02-01 9:44 GMT+01:00 Jean-Baptiste Onofré <[email protected]>: > >> >> > > Hi Lukasz, > >> >> > > > >> >> > > Thanks for your detailed e-mail and I fully agree with you. > >> >> > > > >> >> > > I guess the first step would be to improve the version range > >> support in > >> >> > > Maven URL, and after in the feature resolver. > >> >> > > > >> >> > > Correct ? > >> >> > > > >> >> > > Regards > >> >> > > JB > >> >> > > > >> >> > > > >> >> > > On 02/01/2017 02:05 AM, Łukasz Dywicki wrote: > >> >> > >> > >> >> > >> Dear receivers, > >> >> > >> I would like to summarize my research and fight to align version > >> range > >> >> > >> handling in different parts of karaf related projects. As some > of > >> you > >> >> > >> might not know version ranges are working differently depending > on > >> >> > >> context we are working in. In general most of logic stays the > same > >> >> > >> while there are some edge cases which breaks up everything. But > >> let me > >> >> > >> start from begining. > >> >> > >> > >> >> > >> Karaf is OSGi related project which keeps very nice integration > >> with > >> >> > >> maven based repositories thanks to pax-url. Both environments do > >> >> > >> support ranges in quite different way, an example of maven range > >> >> > >> understanding is described in maven enforcer plugin > documentation > >> [1]. > >> >> > >> Reason why ranges are working differently here and there is a > maven > >> >> > >> snapshot version and understanding of released version. Osgi > >> framework > >> >> > >> does not distinguish any of these. It has knowledge of major, > minor > >> >> > >> and micro parts of an version and uses them for comparision but > the > >> >> > >> qualifier is just a text which might be used for sorting > artifacts > >> >> > >> with same number. This means that for Maven 3.0-SNAPSHOT > version is > >> >> > >> lower than 3.0. In maven there is also knowledge of alpha, beta, > >> rc, > >> >> > >> cr, milestone, ga and sp (service pack) release types [2]. > >> >> > >> > >> >> > >> Now lets come to places which are using or might be using > version > >> >> > >> ranges in typical Karaf based project: > >> >> > >> - OSGi framework for wiring in packages > >> >> > >> - pax-url-mvn for installing maven artifacts > >> >> > >> - karaf feature core for choosing dependant features > >> >> > >> - maven for including dependant artifacts (ie. feature sets/KARs > >> etc) > >> >> > >> - karaf-maven-plugin for building assemblies > >> >> > >> > >> >> > >> When any of range definitions is crossing osgi-maven world > problems > >> >> > >> starts to happen. For example range such [2.18, 2.19) in maven > will > >> >> > >> accept 2.19.0-SNAPSHOT while in OSGi it will not. This lead to > >> >> > >> situations that these two code parts behave completely > differently > >> >> > >> (assuming that camel-core feature is just one bundle): > >> >> > >> <bundle>mvn:org.apache.camel/camel-core/${camel.version}</ > bundle> > >> >> > >> <feature version="${camel.version}">camel-core</feature> > >> >> > >> This will behave like above but not like bundle statement: > >> >> > >> > >> >> > >> <repository>mvn:org.apache.camel.karaf/features/${camel. > >> >> > version}/xml/features</repository> > >> >> > >> > >> >> > >> There are some attempts to work around that by using versions > >> starting > >> >> > >> from ie 2.18.1 so version beginning works just fine but still > >> there is > >> >> > >> problem of range end. To exclude 2.19-SNAPSHOT in maven you must > >> use > >> >> > >> "2.19.min" which in osgi will acceptversion 2.19.. Obviously > there > >> is > >> >> > >> also no way to influence 3rd party so they do not release > version > >> >> > >> 4.1.0 but 4.1.1 just for our environment pleasure. > >> >> > >> > >> >> > >> For me it's quite big issue because hitting us on daily basis. > We > >> have > >> >> > >> quite few modules (around 400) which are usualy moving together > but > >> >> > >> they should be keeping contract/interfaces on micro versions. > This > >> >> > >> inconsistency lives in Karaf and Pax Url since very long time > and > >> >> > >> current project infrastructure is not ready to changing that. > From > >> >> > >> other hand keeping this inconsistent will lead to ultimate fail > >> some > >> >> > >> day and users frustration as well (see KARAF-4105 [3]). Worth to > >> point > >> >> > >> that this issue pointed out brieefly this issue but didn't solve > >> cause > >> >> > >> but aligned just one place to maven's logic while keeping all > >> others > >> >> > >> the same. > >> >> > >> > >> >> > >> I took my chance and managed to get maven understanding osgi > >> version > >> >> > >> ranges thanks to core extensions mechanism they have [4]. I also > >> >> > >> managed to correct shaded aether inside pax-url [5] so it use > >> version > >> >> > >> ranges in same way as maven. What I completely failed is making > a > >> >> > >> custom distro built with my pax-url. Since pax-url-mvn is a > startup > >> >> > >> bundle I can't use overrides for changing it's version and I > can't > >> >> > >> influence its classes using fragment bundle (yet). To get my own > >> >> > >> pax-url I would ned to get rid of framework, but then I have to > >> copy > >> >> > >> bunch of resources. It would be fine for temporary prosthesis > but I > >> >> > >> can't rely on it forever. I also got into troubles with > >> >> > >> karaf-maven-plugin when setting extra dependency with "my own > >> aether". > >> >> > >> > >> >> > >> As you now know - there is lots of troubles with version ranges > >> making > >> >> > >> their usage in end-to-end build very difficult. I would love to > get > >> >> > >> this solved as soon as possible in 4.1 without holding current > >> >> > >> release. Aligning all these version range handling is > definitelly > >> >> > >> doable because from Maven/Aether perspective there is an SPI for > >> that. > >> >> > >> We just need to deliver it our own VersionRangeResolver > interface > >> [6]. > >> >> > >> Open question is shall we keep ordering of versions same as > maven > >> >> > >> breaking up a little osgi range understanding here. > >> >> > >> > >> >> > >> [1] > >> >> > >> http://maven.apache.org/components/enforcer/enforcer- > >> >> > rules/versionRanges.html > >> >> > >> [2] > >> >> > >> https://github.com/eclipse/aether-core/blob/1.0.x/aether- > >> >> > util/src/main/java/org/eclipse/aether/util/version/ > >> >> > GenericVersion.java#L183 > >> >> > >> [3] https://issues.apache.org/jira/browse/KARAF-4105 > >> >> > >> [4] http://markmail.org/message/z6x27umabwqhdjvy > >> >> > >> [5] > >> >> > >> https://github.com/splatch/maven-osgi-resolver/blob/ > >> >> > master/compatible-pax/pom.xml > >> >> > >> [6] > >> >> > >> https://github.com/splatch/maven-osgi-resolver/blob/ > >> >> > master/compatible-locator/src/main/java/org/apache/maven/ > >> >> > repository/internal/MavenRepositorySystemUtils.java#L78 > >> >> > >> > >> >> > >> Kind regards, > >> >> > >> Lukasz > >> >> > >> -- > >> >> > >> Apache Karaf Committer & PMC > >> >> > >> Twitter: @ldywicki > >> >> > >> Blog: http://dywicki.pl > >> >> > >> Code-House - http://code-house.org > >> >> > >> > >> >> > > > >> >> > > -- > >> >> > > Jean-Baptiste Onofré > >> >> > > [email protected] > >> >> > > http://blog.nanthrax.net > >> >> > > Talend - http://www.talend.com > >> >> > > >> >> > >> >> > >> >> > >> >> -- > >> >> ------------------------ > >> >> Guillaume Nodet > >> >> > >> > > > > > > > > -- > > ------------------------ > > Guillaume Nodet > -- ------------------------ Guillaume Nodet
