Hi I got the OSGi tests to work with karaf 4 and they are all passing now for the features tests. There is a few that has been @Ignored because of known issues. eg we need a new SMX bundle for them.
You can run these tests reliable using a bash script cd tests/camel-itest-karaf ./run-tests.sh You can also try running using mvn with mvn clean install But the latter tend to break on my computer after 20 or so tests. The scripts runs reliable and it also cleanup dangling karaf JVMs. But generally its been safer to run one mvn goal per test it seems. I also fixed the mistakes in karaf features and the tests so they are working again. Raul you should be able to use that on your branch to give the osgi tests a go. On Sat, Mar 26, 2016 at 5:47 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: > I managed to get the CI jobs to complete now with setting higher memory. > > The snapshots are now deployed and the CI jobs for osgi is triggered > > Triggering a new build of Camel.trunk.itest.karaf > Triggering a new build of Camel.trunk.itest.osgi > Finished: SUCCESS > > > > On Sat, Mar 26, 2016 at 12:36 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: >> Oh I forgot the CI jobs are here >> https://builds.apache.org/view/A-D/view/Camel/ >> >> I got the link from Mueller. Beforehand it was hard/impossible to find >> Camel from >> https://builds.apache.org/ >> >> So I was for a longer time not able to see all these jobs, and just >> got the CI emails to look at. >> >> But the OSGi jobs rely on the SNAPSHOT of Camel has been built prior >> to that, and that job is unreliable. I think we need to look at making >> the -notest also publish the SNAPSHOT to the snapshot mvn repo, so the >> CI jobs can pickup these new JARs. >> >> If we get that more reliable then Camel end users can also easier use >> SNAPSHOT instead of having to build locally to be sure they get a >> clean build of it, otherwise you can end up with mixed builds or >> stale/old snapshots. >> >> >> >> On Sat, Mar 26, 2016 at 12:33 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: >>> On Sat, Mar 26, 2016 at 12:12 PM, Raul Kripalani <ra...@apache.org> wrote: >>>> Agree with your points. >>>> >>>> 1. If you use lambdas on 2.18, you can't backport the code. >>>> >>>> 2. I'll spend some time this weekend installing the bundles on Karaf. Karaf >>>> 4 is OK, or are we sticking with older versions for 2.x? >>>> >>> >>> Karaf 2.x is dropped. Use Karaf 4.x as the primary karaf version. >>> We may want to add a -Pkaraf3 profile to run against older Karaf 3.x >>> containers. >>> >>> >>>> 3. I'll also run our OSGi itests, but it may be worth having Jenkins do all >>>> this. Could you configure a job in JKS to build this branch? I don't have >>>> job admin rights I believe. >>>> >>> >>> We already have some CI jobs to run 2 osgi based tests >>> >>> - tests/camel-itest-karaf >>> - tests/camel-itest-osgi >>> >>> The former tests that the Camel component can be installed, it does >>> this using pax-exam to bootup karaf and install the camel-xxx feature >>> and find the component and then tear down the test. >>> >>> You can run these tests with the -Pkaraf4 profile. >>> But mind I had trouble with these tests may stop after 20 or so tests >>> and leave some karaf JVMs dangling. There is a kill-karaf.sh script to >>> kill em. >>> >>> The camel-itest-osgi is likely in some state where some tests would >>> fail even for older branches. But these are unit tests that run Camel >>> apps in karaf and do some testing stuff like we do in regular >>> component tests. This has not yet been migrated to karaf 4, and run on >>> karaf 2.x. But you can give it a go and see how bad/good it is. It >>> used to be able to run almost all tests and pass. And we had also had >>> periods where everything was green. >>> >>> However people dont write so many osgi tests so this module do not >>> have so much love. And there are plenty of components that are not >>> being tested. >>> >>> >>>> 4. It's necessary to configure the maven-jar-plugin to explicitly use a >>>> manifest file, otherwise the plugin generates its own. Originally I tried >>>> adding the config to the build plugins in the parent POM, but it failed for >>>> artifacts of type=maven-plugin. Hence I placed it in pluginManagement and >>>> then incorporated it in those intermediate POMs (components, examples, >>>> etc.) whose children built normal JARs. However, I ended up tracing the >>>> issue to a conflict introduced by the config of the maven-jar-plugin that >>>> maven-plugin injects, and solved it with a couple >>>> of combine.self="override". Hence, I think I might be able to push our >>>> maven-jar-plugin config back to the parent POM and to the build section. >>>> >>> >>> Ah okay. I am okay with we can do "anything" we need to build all the >>> Camel components and whatnot. >>> >>> Its more what the impact is for end users. If we can make out examples >>> and maven archetypes simpler and easier that would be cool. But mind >>> that not all examples are osgi examples. >>> >>> But it looks promising your work - thanks for starting this hard work. >>> >>> >>> >>>> Cheers, >>>> >>>> *Raúl Kripalani* >>>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and >>>> Messaging Engineer >>>> http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani >>>> Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> >>>> >>>> On Sat, Mar 26, 2016 at 10:12 AM, Claus Ibsen <claus.ib...@gmail.com> >>>> wrote: >>>> >>>>> Hi >>>>> >>>>> Good to hear that we may drop the <bundle> extension, nice work. >>>>> >>>>> For camel-core I have thought of build the manifest.mf by hand so we >>>>> can better control it. It seems the bundle plugin has its issues with >>>>> different versions to build it, as you say. >>>>> >>>>> Have you tried testing any of these JARs in OSGi? It would be good if >>>>> you try that and install some of the examples in a karaf 4 container >>>>> and see how it runs. >>>>> >>>>> For lambda code in the existing source code then we cannot do that for >>>>> code that need to be backported to 2.17 and 2.16 branches. We can only >>>>> use lambdas in new code that is not to be backported. >>>>> >>>>> For Camel 3.0 we can then covert the code to use lambads all over >>>>> where applicable. There is likely some tools in IDEA or Eclipse etc >>>>> that can assist with that. >>>>> >>>>> >>>>> And Camel end users can use lamdas today with their own code, also >>>>> with older Camel versions on java 8. >>>>> >>>>> It may be that those people are using a new bundle plugin, or more >>>>> likely not using OSGi at all. >>>>> >>>>> >>>>> I would like to see more OSGi testing on this branch before any merge. >>>>> OSGi is complex and pain to maintain and build modules. So we need to >>>>> be sure we are on a path that we can do this. There is 250+ maven >>>>> modules that are build as OSGi so it better have to work for us. >>>>> >>>>> But I really like that the bundle plugin is just generating a >>>>> MANIFEST.MF file that the JAR plugin just slurps in. Can it not be >>>>> configured somehow so the generated MANIFEST.MF is not replace so end >>>>> users do not need to do anything with a JAR plugin? >>>>> >>>>> >>>>> >>>>> >>>>> >>>>> On Sat, Mar 26, 2016 at 1:09 AM, Raul Kripalani <ra...@apache.org> wrote: >>>>> > There is good news, bad news, and good news again ;-) >>>>> > >>>>> > *Good news:* Now that Camel 2.18 is officially on JDK8, we can use >>>>> lambdas >>>>> > in our code. Woohoo! >>>>> > >>>>> > *Bad news:* Our current Maven build breaks when doing so. >>>>> > >>>>> > This is what happens: >>>>> > >>>>> > 1. maven-bundle-plugin version 2.3.x cannot parse JDK8 lambdas. [1] >>>>> > >>>>> > 2. Upgrading it to the latest version (3.0.1) makes the plugin fail >>>>> > miserably (at least on camel-core). >>>>> > >>>>> > 3. Using the latest 2.x version of the maven-bundle-plugin (2.5.4) >>>>> makes >>>>> > the maven-shade-plugin fail with this error in turn: >>>>> > >>>>> > [ERROR] The project main artifact does not exist. This could have the >>>>> > following >>>>> > [ERROR] reasons: >>>>> > [ERROR] - You have invoked the goal directly from the command line. This >>>>> is >>>>> > not >>>>> > [ERROR] supported. Please add the goal to the default lifecycle via an >>>>> > [ERROR] <execution> element in your POM and use "mvn package" to have >>>>> it >>>>> > run. >>>>> > [ERROR] - You have bound the goal to a lifecycle phase before "package". >>>>> > Please >>>>> > [ERROR] remove this binding from your POM such that the goal will be >>>>> run >>>>> > in >>>>> > [ERROR] the proper phase. >>>>> > >>>>> > 4. Upgrading the shade plugin doesn't help. >>>>> > >>>>> > 5. The situation is dirty :P >>>>> > >>>>> > The problem is the usage of the 'bundle' custom packaging type enabled >>>>> > by >>>>> > extensions='true' in the maven-bundle-plugin. >>>>> > >>>>> > *Good news again:* >>>>> > >>>>> > We can avoid the problem by reverting back to the standard 'jar' >>>>> packaging >>>>> > type, which even puts us in a less brittle situation wrt. to future >>>>> > Maven >>>>> > plugins we may want to add/upgrade. This implies: >>>>> > >>>>> > 1. Calling the maven-bundle-plugin:manifest goal (instead of the >>>>> bundle >>>>> > goal) to only generate the MANIFEST.MF. >>>>> > >>>>> > 2. Adding it to the JAR via an option in the the maven-jar-plugin: >>>>> > >>>>> > <plugin> >>>>> > <groupId>org.apache.maven.plugins</groupId> >>>>> > <artifactId>maven-jar-plugin</artifactId> >>>>> > <version>${maven-jar-plugin-version}</version> >>>>> > <configuration> >>>>> > <archive> >>>>> > >>>>> > >>>>> <manifestFile>${project.build.outputDirectory}/META-INF/MANIFEST.MF</manifestFile> >>>>> > </archive> >>>>> > </configuration> >>>>> > </plugin> >>>>> > >>>>> > I've made the change. But since it's a big one, I didn't push to master >>>>> but >>>>> > to a branch for us to review first: jdk8-lambdas. >>>>> > >>>>> > The build passes (yay!) but OSGi battle testing is a must to ensure >>>>> > there >>>>> > are no regressions. >>>>> > >>>>> > Could you have a quick look? If no feedback is received until Monday >>>>> > EOD, >>>>> > I'll merge to master. If feedback is positive, we can merge earlier to >>>>> > enable people to develop with lambdas finally. >>>>> > >>>>> > --- >>>>> > >>>>> > Just in case you're not aware, the OSGi legend Neil Bartlett has just >>>>> > released a new bnd-maven-plugin [2] that claims to overcome such >>>>> > dysfunctions, as well as others, of the Felix plugin. He complained >>>>> > about >>>>> > how the custom packaging type breaks integration with other plugins >>>>> > downstream (what we're experiencing). >>>>> > >>>>> > I see no reason to migrate to his bnd-maven-plugin, but we should keep >>>>> > it >>>>> > in mind for the future. >>>>> > >>>>> > [1] https://issues.apache.org/jira/browse/FELIX-4005 >>>>> > [2] http://njbartlett.name/2015/03/27/announcing-bnd-maven-plugin.html >>>>> > >>>>> > Cheers, >>>>> > >>>>> > *Raúl Kripalani* >>>>> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data >>>>> > and >>>>> > Messaging Engineer >>>>> > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani >>>>> > Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> >>>>> >>>>> >>>>> >>>>> -- >>>>> Claus Ibsen >>>>> ----------------- >>>>> http://davsclaus.com @davsclaus >>>>> Camel in Action 2: https://www.manning.com/ibsen2 >>>>> >>> >>> >>> >>> -- >>> Claus Ibsen >>> ----------------- >>> http://davsclaus.com @davsclaus >>> Camel in Action 2: https://www.manning.com/ibsen2 >> >> >> >> -- >> Claus Ibsen >> ----------------- >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 > > > > -- > Claus Ibsen > ----------------- > http://davsclaus.com @davsclaus > Camel in Action 2: https://www.manning.com/ibsen2 -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
