On Tue, Mar 29, 2016 at 1:04 AM, Raul Kripalani <ra...@apache.org> wrote: > Thanks a lot Claus! > > Given that we now make no distinction between bundles and standard JARs > through the packaging type, I was having the OSGi MANIFEST generation > triggered for bundles where its irrelevant, e.g. camel-grape, > camel-spring-boot, many examples, etc. So I added a profile which is > activated by default and is disabled through a new POM property > camel.osgi.skip, which I added to the POMs of such modules. >
Can we flip the switch so you have to enable it on the maven modules that you want to be an osgi bundle. I am asking because people who are not using OSGi should really not see camel.osgi.skip=true in the examples / camel-spring-boot-starter etc. They should be clean and without any osgi stuff. Also I would rather make it explicit that this maven module is built as an osgi bundle if it has camel.osgi=true. > I was also able to upgrade the version of the maven-bundle-plugin to the > latest 3.0.1. > > Hopefully I'll be able to finish my tests tomorrow and merge the branch > into master for a full test in JKS. > > From then on... let the lambda fun begin (for features we don't intend to > backport)! > > Cheers, > > *Raúl Kripalani* > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data and > Messaging Engineer > http://about.me/raulkripalani | http://www.linkedin.com/in/raulkripalani > Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> > > On Sun, Mar 27, 2016 at 5:24 PM, Claus Ibsen <claus.ib...@gmail.com> wrote: > >> Hi >> >> I got the OSGi tests to work with karaf 4 and they are all passing now >> for the features tests. There is a few that has been @Ignored because >> of known issues. eg we need a new SMX bundle for them. >> >> You can run these tests reliable using a bash script >> >> cd tests/camel-itest-karaf >> ./run-tests.sh >> >> You can also try running using mvn with >> >> mvn clean install >> >> But the latter tend to break on my computer after 20 or so tests. The >> scripts runs reliable and it also cleanup dangling karaf JVMs. But >> generally its been safer to run one mvn goal per test it seems. >> >> I also fixed the mistakes in karaf features and the tests so they are >> working again. >> >> Raul you should be able to use that on your branch to give the osgi tests >> a go. >> >> >> >> >> >> >> On Sat, Mar 26, 2016 at 5:47 PM, Claus Ibsen <claus.ib...@gmail.com> >> wrote: >> > I managed to get the CI jobs to complete now with setting higher memory. >> > >> > The snapshots are now deployed and the CI jobs for osgi is triggered >> > >> > Triggering a new build of Camel.trunk.itest.karaf >> > Triggering a new build of Camel.trunk.itest.osgi >> > Finished: SUCCESS >> > >> > >> > >> > On Sat, Mar 26, 2016 at 12:36 PM, Claus Ibsen <claus.ib...@gmail.com> >> wrote: >> >> Oh I forgot the CI jobs are here >> >> https://builds.apache.org/view/A-D/view/Camel/ >> >> >> >> I got the link from Mueller. Beforehand it was hard/impossible to find >> >> Camel from >> >> https://builds.apache.org/ >> >> >> >> So I was for a longer time not able to see all these jobs, and just >> >> got the CI emails to look at. >> >> >> >> But the OSGi jobs rely on the SNAPSHOT of Camel has been built prior >> >> to that, and that job is unreliable. I think we need to look at making >> >> the -notest also publish the SNAPSHOT to the snapshot mvn repo, so the >> >> CI jobs can pickup these new JARs. >> >> >> >> If we get that more reliable then Camel end users can also easier use >> >> SNAPSHOT instead of having to build locally to be sure they get a >> >> clean build of it, otherwise you can end up with mixed builds or >> >> stale/old snapshots. >> >> >> >> >> >> >> >> On Sat, Mar 26, 2016 at 12:33 PM, Claus Ibsen <claus.ib...@gmail.com> >> wrote: >> >>> On Sat, Mar 26, 2016 at 12:12 PM, Raul Kripalani <ra...@apache.org> >> wrote: >> >>>> Agree with your points. >> >>>> >> >>>> 1. If you use lambdas on 2.18, you can't backport the code. >> >>>> >> >>>> 2. I'll spend some time this weekend installing the bundles on Karaf. >> Karaf >> >>>> 4 is OK, or are we sticking with older versions for 2.x? >> >>>> >> >>> >> >>> Karaf 2.x is dropped. Use Karaf 4.x as the primary karaf version. >> >>> We may want to add a -Pkaraf3 profile to run against older Karaf 3.x >> containers. >> >>> >> >>> >> >>>> 3. I'll also run our OSGi itests, but it may be worth having Jenkins >> do all >> >>>> this. Could you configure a job in JKS to build this branch? I don't >> have >> >>>> job admin rights I believe. >> >>>> >> >>> >> >>> We already have some CI jobs to run 2 osgi based tests >> >>> >> >>> - tests/camel-itest-karaf >> >>> - tests/camel-itest-osgi >> >>> >> >>> The former tests that the Camel component can be installed, it does >> >>> this using pax-exam to bootup karaf and install the camel-xxx feature >> >>> and find the component and then tear down the test. >> >>> >> >>> You can run these tests with the -Pkaraf4 profile. >> >>> But mind I had trouble with these tests may stop after 20 or so tests >> >>> and leave some karaf JVMs dangling. There is a kill-karaf.sh script to >> >>> kill em. >> >>> >> >>> The camel-itest-osgi is likely in some state where some tests would >> >>> fail even for older branches. But these are unit tests that run Camel >> >>> apps in karaf and do some testing stuff like we do in regular >> >>> component tests. This has not yet been migrated to karaf 4, and run on >> >>> karaf 2.x. But you can give it a go and see how bad/good it is. It >> >>> used to be able to run almost all tests and pass. And we had also had >> >>> periods where everything was green. >> >>> >> >>> However people dont write so many osgi tests so this module do not >> >>> have so much love. And there are plenty of components that are not >> >>> being tested. >> >>> >> >>> >> >>>> 4. It's necessary to configure the maven-jar-plugin to explicitly use >> a >> >>>> manifest file, otherwise the plugin generates its own. Originally I >> tried >> >>>> adding the config to the build plugins in the parent POM, but it >> failed for >> >>>> artifacts of type=maven-plugin. Hence I placed it in pluginManagement >> and >> >>>> then incorporated it in those intermediate POMs (components, examples, >> >>>> etc.) whose children built normal JARs. However, I ended up tracing >> the >> >>>> issue to a conflict introduced by the config of the maven-jar-plugin >> that >> >>>> maven-plugin injects, and solved it with a couple >> >>>> of combine.self="override". Hence, I think I might be able to push our >> >>>> maven-jar-plugin config back to the parent POM and to the build >> section. >> >>>> >> >>> >> >>> Ah okay. I am okay with we can do "anything" we need to build all the >> >>> Camel components and whatnot. >> >>> >> >>> Its more what the impact is for end users. If we can make out examples >> >>> and maven archetypes simpler and easier that would be cool. But mind >> >>> that not all examples are osgi examples. >> >>> >> >>> But it looks promising your work - thanks for starting this hard work. >> >>> >> >>> >> >>> >> >>>> Cheers, >> >>>> >> >>>> *Raúl Kripalani* >> >>>> PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big Data >> and >> >>>> Messaging Engineer >> >>>> http://about.me/raulkripalani | >> http://www.linkedin.com/in/raulkripalani >> >>>> Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> >> >>>> >> >>>> On Sat, Mar 26, 2016 at 10:12 AM, Claus Ibsen <claus.ib...@gmail.com> >> wrote: >> >>>> >> >>>>> Hi >> >>>>> >> >>>>> Good to hear that we may drop the <bundle> extension, nice work. >> >>>>> >> >>>>> For camel-core I have thought of build the manifest.mf by hand so we >> >>>>> can better control it. It seems the bundle plugin has its issues with >> >>>>> different versions to build it, as you say. >> >>>>> >> >>>>> Have you tried testing any of these JARs in OSGi? It would be good if >> >>>>> you try that and install some of the examples in a karaf 4 container >> >>>>> and see how it runs. >> >>>>> >> >>>>> For lambda code in the existing source code then we cannot do that >> for >> >>>>> code that need to be backported to 2.17 and 2.16 branches. We can >> only >> >>>>> use lambdas in new code that is not to be backported. >> >>>>> >> >>>>> For Camel 3.0 we can then covert the code to use lambads all over >> >>>>> where applicable. There is likely some tools in IDEA or Eclipse etc >> >>>>> that can assist with that. >> >>>>> >> >>>>> >> >>>>> And Camel end users can use lamdas today with their own code, also >> >>>>> with older Camel versions on java 8. >> >>>>> >> >>>>> It may be that those people are using a new bundle plugin, or more >> >>>>> likely not using OSGi at all. >> >>>>> >> >>>>> >> >>>>> I would like to see more OSGi testing on this branch before any >> merge. >> >>>>> OSGi is complex and pain to maintain and build modules. So we need to >> >>>>> be sure we are on a path that we can do this. There is 250+ maven >> >>>>> modules that are build as OSGi so it better have to work for us. >> >>>>> >> >>>>> But I really like that the bundle plugin is just generating a >> >>>>> MANIFEST.MF file that the JAR plugin just slurps in. Can it not be >> >>>>> configured somehow so the generated MANIFEST.MF is not replace so end >> >>>>> users do not need to do anything with a JAR plugin? >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> >> >>>>> On Sat, Mar 26, 2016 at 1:09 AM, Raul Kripalani <ra...@apache.org> >> wrote: >> >>>>> > There is good news, bad news, and good news again ;-) >> >>>>> > >> >>>>> > *Good news:* Now that Camel 2.18 is officially on JDK8, we can use >> >>>>> lambdas >> >>>>> > in our code. Woohoo! >> >>>>> > >> >>>>> > *Bad news:* Our current Maven build breaks when doing so. >> >>>>> > >> >>>>> > This is what happens: >> >>>>> > >> >>>>> > 1. maven-bundle-plugin version 2.3.x cannot parse JDK8 lambdas. >> [1] >> >>>>> > >> >>>>> > 2. Upgrading it to the latest version (3.0.1) makes the plugin >> fail >> >>>>> > miserably (at least on camel-core). >> >>>>> > >> >>>>> > 3. Using the latest 2.x version of the maven-bundle-plugin >> (2.5.4) >> >>>>> makes >> >>>>> > the maven-shade-plugin fail with this error in turn: >> >>>>> > >> >>>>> > [ERROR] The project main artifact does not exist. This could have >> the >> >>>>> > following >> >>>>> > [ERROR] reasons: >> >>>>> > [ERROR] - You have invoked the goal directly from the command >> line. This >> >>>>> is >> >>>>> > not >> >>>>> > [ERROR] supported. Please add the goal to the default lifecycle >> via an >> >>>>> > [ERROR] <execution> element in your POM and use "mvn package" to >> have >> >>>>> it >> >>>>> > run. >> >>>>> > [ERROR] - You have bound the goal to a lifecycle phase before >> "package". >> >>>>> > Please >> >>>>> > [ERROR] remove this binding from your POM such that the goal >> will be >> >>>>> run >> >>>>> > in >> >>>>> > [ERROR] the proper phase. >> >>>>> > >> >>>>> > 4. Upgrading the shade plugin doesn't help. >> >>>>> > >> >>>>> > 5. The situation is dirty :P >> >>>>> > >> >>>>> > The problem is the usage of the 'bundle' custom packaging type >> enabled by >> >>>>> > extensions='true' in the maven-bundle-plugin. >> >>>>> > >> >>>>> > *Good news again:* >> >>>>> > >> >>>>> > We can avoid the problem by reverting back to the standard 'jar' >> >>>>> packaging >> >>>>> > type, which even puts us in a less brittle situation wrt. to >> future Maven >> >>>>> > plugins we may want to add/upgrade. This implies: >> >>>>> > >> >>>>> > 1. Calling the maven-bundle-plugin:manifest goal (instead of the >> >>>>> bundle >> >>>>> > goal) to only generate the MANIFEST.MF. >> >>>>> > >> >>>>> > 2. Adding it to the JAR via an option in the the >> maven-jar-plugin: >> >>>>> > >> >>>>> > <plugin> >> >>>>> > <groupId>org.apache.maven.plugins</groupId> >> >>>>> > <artifactId>maven-jar-plugin</artifactId> >> >>>>> > <version>${maven-jar-plugin-version}</version> >> >>>>> > <configuration> >> >>>>> > <archive> >> >>>>> > >> >>>>> > >> >>>>> >> <manifestFile>${project.build.outputDirectory}/META-INF/MANIFEST.MF</manifestFile> >> >>>>> > </archive> >> >>>>> > </configuration> >> >>>>> > </plugin> >> >>>>> > >> >>>>> > I've made the change. But since it's a big one, I didn't push to >> master >> >>>>> but >> >>>>> > to a branch for us to review first: jdk8-lambdas. >> >>>>> > >> >>>>> > The build passes (yay!) but OSGi battle testing is a must to >> ensure there >> >>>>> > are no regressions. >> >>>>> > >> >>>>> > Could you have a quick look? If no feedback is received until >> Monday EOD, >> >>>>> > I'll merge to master. If feedback is positive, we can merge >> earlier to >> >>>>> > enable people to develop with lambdas finally. >> >>>>> > >> >>>>> > --- >> >>>>> > >> >>>>> > Just in case you're not aware, the OSGi legend Neil Bartlett has >> just >> >>>>> > released a new bnd-maven-plugin [2] that claims to overcome such >> >>>>> > dysfunctions, as well as others, of the Felix plugin. He >> complained about >> >>>>> > how the custom packaging type breaks integration with other plugins >> >>>>> > downstream (what we're experiencing). >> >>>>> > >> >>>>> > I see no reason to migrate to his bnd-maven-plugin, but we should >> keep it >> >>>>> > in mind for the future. >> >>>>> > >> >>>>> > [1] https://issues.apache.org/jira/browse/FELIX-4005 >> >>>>> > [2] >> http://njbartlett.name/2015/03/27/announcing-bnd-maven-plugin.html >> >>>>> > >> >>>>> > Cheers, >> >>>>> > >> >>>>> > *Raúl Kripalani* >> >>>>> > PMC & Committer @ Apache Ignite, Apache Camel | Integration, Big >> Data and >> >>>>> > Messaging Engineer >> >>>>> > http://about.me/raulkripalani | >> http://www.linkedin.com/in/raulkripalani >> >>>>> > Blog: raul.io | twitter: @raulvk <https://twitter.com/raulvk> >> >>>>> >> >>>>> >> >>>>> >> >>>>> -- >> >>>>> Claus Ibsen >> >>>>> ----------------- >> >>>>> http://davsclaus.com @davsclaus >> >>>>> Camel in Action 2: https://www.manning.com/ibsen2 >> >>>>> >> >>> >> >>> >> >>> >> >>> -- >> >>> Claus Ibsen >> >>> ----------------- >> >>> http://davsclaus.com @davsclaus >> >>> Camel in Action 2: https://www.manning.com/ibsen2 >> >> >> >> >> >> >> >> -- >> >> Claus Ibsen >> >> ----------------- >> >> http://davsclaus.com @davsclaus >> >> Camel in Action 2: https://www.manning.com/ibsen2 >> > >> > >> > >> > -- >> > Claus Ibsen >> > ----------------- >> > http://davsclaus.com @davsclaus >> > Camel in Action 2: https://www.manning.com/ibsen2 >> >> >> >> -- >> Claus Ibsen >> ----------------- >> http://davsclaus.com @davsclaus >> Camel in Action 2: https://www.manning.com/ibsen2 >> -- Claus Ibsen ----------------- http://davsclaus.com @davsclaus Camel in Action 2: https://www.manning.com/ibsen2
