Hi Dave
Dave Reynolds wrote:
Interesting. I'd like to hear your impressions of that. I've been
wondering how much real momentum and investment there is around OSGi.
The fact that an entire track on Friday at ApacheCon was all dedicated
to OSGi is a signal, these were the talks:
- OSGi for mere mortals, Bertrand Delacretaz
- Apache Celix - Universal OSGi?, Alexander Broekhuis
- Deployment With Apache Karaf and ACE, Jean-Baptiste Onofré
- Provisioning distributed OSGi applications in a cloud, Guillaume Nodet
- Using OSGi to Build Better Software: Lessons from a Telemedicine
Software for Smartphones and Desktop Systems, Doreen Seider
Of these, I attended only the talk on Apache Karaf.
With 5 tracks at the same time the chance of finding two or more
interesting talks at the same time is high.
However, I have been talking with developers using Apache projects for
OSGi and working for a couple of the ApacheCon sponsors which are using
OSGi heavily with all of their customers.
My overall impression is that OSGi usage is expanding and it's not
anymore limited to big projects such as Eclipse and/or J2EE containers
such as Apache Geronimo.
It requires changing the way you work, but it certainly a big advantage
to be able to fix a small bug in a bundle and push it live without
restarting servers as suppose to: remove a server from the load balancing
pool, shut the server down, deploy the new version, restart, add the
server back to the load balancing pool. All this can be automated with
tools such as Puppet, Chef or similar projects. However, is not as flexible
and tuned towards Java as OSGi is. This is what it seems to me one of the
biggest advantages which OSGi provides.
(I'll share more on ApacheCon when I am back).
Are you still using OSGi?
Yes. Though I expect it to be just one tool we use for some components
rather than the fundamental architecture we use all the time.
Can I ask you which OSGi "runtime"(?) do you use?
- http://eclipse.org/equinox/
- http://felix.apache.org/
- http://karaf.apache.org/
- other?
Primarily equinox. Felix is the other easy one to work with given the
toolchain I use, indeed we mostly use the Felix component bundles.
However, we also use jRuby with its OSGi integration and I have only got
that to run reliably under equinox.
The demo I see about Karaf what quite effective and Karaf works both with
Felix and Equinox:
"Apache Karaf is a small OSGi based runtime which provides a lightweight
container onto which various components and applications can be deployed.
[...] Supports the latest OSGi 4.2 containers: Apache Felix Framework 3.0
and Eclipse Equinox 3.6" -- http://karaf.apache.org/
See also:
- https://cwiki.apache.org/KARAF/how-do-i-choose-between-felix-and-equinox.html
You strongly advocated for the one jar (=bundle) approach and provided
strong evidence for this. I am convinced by your arguments.
How important this is depends on the tool chain. There's increased
support for using repositories, including Maven repositories for dynamic
bundle resolution so the benefits of one bundle may be reducing. But for
min config, get something work out of the box quickly, I still feel it
has benefits.
However, I wonder if others might benefit from having OSGi bundles for
the single Jena components (or just Jena without the dependencies which
do not come from Jena, probably this is better...?). I also found this:
http://repo1.maven.org/maven2/org/apache/servicemix/bundles/
Have you been using it?
No, though there are several bundle repositories around.
I still need to get my head around all these projects and how they
related each others and how they might relate to Apache Jena in the
context of OSGi.
Sure.
Another fact is that two other projects in the incubator which are
currently using Apache Jena (directly or indirectly) are Clerezza and
Stanbol. They might benefit from us providing Jena OSGi bundles directly.
I need to look at the details and what they currently do (and how they
use Jena), but it might sense to deal with OSGi ourselves directly.
I wanted control over generating bundles to match Jena snapshots so
doing our own internal releases made most sense.
Yep, it makes sense.
The reason why I mentioned Clerezza and Stanbol was different though.
They both use Jena directly (as Clerezza) or indirectly (as Stanbol,
via Crelezza). I did not mentioned them to say: "you/we should use
them when we need Jena as OSGi bundle", but to say: "if they both use
OSGi, it would make sense for us to provide them the Jena bundles
directly and to all others who might want/need to use Jena in an OSGi
context".
Finally, (this is me being curious) it would be interesting to know what
is your main driver in relation to OSGi. What's the feature/capability
which is a big attractor for you?
Modularity plus dynamism.
I get the dynamism.
But, I am not sure I understand the modularity advantage... but it's
because I know very little about OSGi.
What I do not understand in relation to modularity is what OSGi gives
more in comparison to have modules for your projects (as we do for Jena)
for example. Aren't your OSGi bundles the same as the Jena modules?
I wanted to be able to package modular components I was working on as
some sort of plugin and be able to upgrade plugins with minimum hassle.
The idea was to have a general platform which had standard APIs for
things like RDF data repositories, data conversion services, web
presentation services and API-over-RDF services and then be able to
freely plugin new implementations of any of these with the same ease
with which you can extend or upgrade eclipse. OSGi is a great platform
for that.
What's more OSGi allows you to do this dynamically. So for example, with
the base platform running if you decide you want to provision a new
store (DataRepository) its just a matter of dropping a config file into
the right directory and it is detected, read, provisioned and event
notifications are sent with a minimum of code support needed and no
restart. If there's a defect in one your components you edit it, save
and the updated component autodeploys to the running platform in real
time - no need to restart servers (well mostly).
It hasn't all been plain sailing though - the problems have been
tooling, startup sequencing and mindset.
Tooling? This is me again not knowing enough about OSGi and tools for
Eclipse... but the question which came into my mind is: "what kind of
capabilities/features do you need at development time when you package
your jar as an OSGi bundle as supposed to as a normal jar?
For development we used bndtools in Eclipse. On its own that was
fantastic, such an easy tool with very quick development cycles compared
to the world of Maven and WTP.
With Maven one thing that gets in the way with multiple modules is the
fact that if you do a refactoring in one module you will not see changes
in the other modules using that module unless you properly configure a
dependency between Eclipse projects (which we, unfortunately, do not do
for Jena). Do you think we should/could change that?
Now the workflow is: change module A, mvn install, mvn compile for module
B. Going via the local repository just to see the consequences on dependent
modules of a refactoring of code in another module is a pain.
The trouble is that it wasn't very Maven
compatible and as Jena and our other projects have got sucked ever
deeper into the Maven black hole I ended up having to use bndtools and
Maven in combination. We managed the make that work (e.g. Brian
developed a maven plugin for deploying bundles to a local bndtools
repository) but it isn't very satisfying and requires use of m2e. This
may improve, the bndtools folks had an intensive get together a few
weeks ago and Maven compatibility was very high on their agenda but the
new version hasn't been released yet.
Again, I do not have experience with what you describe. Are there things
we can/should change in the pom.xml files to make things easier?
The startup issues are my fault rather than OSGi's but it is part of the
OSGi challenge to handle the high levels of concurrency. When you start
up a runtime you have limited control over what order bundles become
available. Some of the facilities you use for configuring and
provisioning services (config admin, Felix file install) are themselves
services that are dynamically starting up. The once file install is up
it starts trying to install the things it finds, often at the same time
as the other services are still trying to start up. While the runtimes
give you a basic notion of runlevel we've had some problems guaranteeing
clean startup, particularly when it comes to embedded scripting like
jRuby. While you are debugging in Eclipse on a decent machine all seems
fine, but shift to a slow machine and it's easy to find there's a subtle
race condition you hadn't thought of.
The "mindset" issue is a subtle one. Thinking in terms of OSGi services
and modules encourages you to think about components running in the same
JVM and having rich interfaces. There are distributed OSGi solutions
(Apache CXF, Paremus) but they feel very RPC like. Whereas for our work
I'd rather be thinking in terms of web style REST APIs, with built in
assumptions of distribution for each API. Having a nice pluggable
framework makes it easy to think of each service you add as just another
plug-in and only later discover it would have been better thought of as
a distributable web services.
Conclusion, next week (when I am back) I want to help Andy with the
testing of what we have for the first Apache Jena release. Fix problems
we might (or might not!) find. After we have rolled out a couple of
releases and we things/processes are set, I want to come back to this
and OSGi and look at how best we can make Apache Jena available to those
who might use it or are already using it via OSGi bundles. This in
practice might turn up something very easy to do (probably via a small
additional Maven plug-in).
The existing Maven plug-in for OSGi is very capable.
This one:
http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html ?
Cheers,
Paolo
Dave
