Hi Dave,
thanks for your answer (which I found useful).
Dave Reynolds wrote:
Hi Paolo,
On Sat, 2011-11-12 at 18:23 -0800, Paolo Castagna wrote:
My overall impression is that OSGi usage is expanding and it's not
anymore limited to big projects such as Eclipse and/or J2EE containers
such as Apache Geronimo.
Interesting. I've certainly come across people who use it routinely on
projects of all sizes but it wasn't clear to me it was growing rather
than shrinking, that's good to know.
Finally, (this is me being curious) it would be interesting to know what
is your main driver in relation to OSGi. What's the feature/capability
which is a big attractor for you?
Modularity plus dynamism.
I get the dynamism.
But, I am not sure I understand the modularity advantage... but it's
because I know very little about OSGi.
What I do not understand in relation to modularity is what OSGi gives
more in comparison to have modules for your projects (as we do for Jena)
for example. Aren't your OSGi bundles the same as the Jena modules?
The modularity is at a finer grain size than a bundle. Working in OSGi
encourages you to structure your solution as a set of separate component
that will be externally wired together. In particular, I make heavy use
of the Component Services Framework and the bnd annotation framework for
that which makes it very easy to declare components and component
dependencies (singleton/multiple, factories, static/dynamic).
For example to add a new RDF store type in Modal (the modular framework
I've been working on) I can just declare a new factory implementation
and drop it in then my existing code can find and invoke the factory.
Contrast that with ModelFactory which either needs extending when you
add a new Model type or you have to provide alternative factory
interfaces.
Now I understand.
Of course, this is all perfectly possible using a dependency injection
framework or a set of conventions on top of plain Java. There is no
magic. It is simply a style that the platform supports well.
Yep. Indeed, we use Google Guice (via ServiceLoader [1]) for the things
you described above.
[1] http://download.oracle.com/javase/6/docs/api/java/util/ServiceLoader.html
For a project such Apache Jena where the speed and/or number of new "things"
coming in is not very high these things might perceived just an unnecessary
overhead. However, now that Jena is in Apache, it would be good to focus on
extension points and plan for making as easy as possible to:
- add new readers/writers for RDF IO and/or SPARQL result sets
- add new storage/query layers
- add new custom indexes and made them available to users via SPARQL
- ...
From my limited experience, these things are not as easy and they often
requires changes to the Jena code itself.
It would be good to be able to do this via simple drop-in extensions and
without the need to change or touch the Jena codebase. This IMHO could add
value to the project and help growing the community of contributors to the
project.
It hasn't all been plain sailing though - the problems have been
tooling, startup sequencing and mindset.
Tooling? This is me again not knowing enough about OSGi and tools for
Eclipse... but the question which came into my mind is: "what kind of
capabilities/features do you need at development time when you package
your jar as an OSGi bundle as supposed to as a normal jar?
Packaging as a bundle is trivial. The universal tool for that is bnd
which you can drive directly, via Eclipse plugins like bndtools or via
the Maven bundle plugin.
For me the incremental development approach support by bndtools is by
far the most attractive, especially with its support for multi-bundle
projects. All of that is lost of you touch Maven with its
one-artefact-per-project approach.
Ack.
For development we used bndtools in Eclipse. On its own that was
fantastic, such an easy tool with very quick development cycles compared
to the world of Maven and WTP.
With Maven one thing that gets in the way with multiple modules is the
fact that if you do a refactoring in one module you will not see changes
in the other modules using that module unless you properly configure a
dependency between Eclipse projects (which we, unfortunately, do not do
for Jena). Do you think we should/could change that?
No.
Ack.
Eclipse project dependencies are the most fragile part of m2e in my
experience, the advice I've had has been to keep that switched off and
that certainly seems to help with some of the Maven instabilities.
Ok.
Now the workflow is: change module A, mvn install, mvn compile for module
B. Going via the local repository just to see the consequences on dependent
modules of a refactoring of code in another module is a pain.
Quite. In bndtools without Maven all that is dynamic, the cross project
dependencies just worked. Lovely. In bndtools *with* Maven then you end
up having to make sure the modules you depend on are visible in
bndtools' idea of a repository as well so you are back to the piecemeal
workflow you describe but with more opportunities for things to go
wrong.
Things are clearly not perfect and different people have different preferences,
opinions, working styles, experience different pain points etc. There are also
different communities to whom we feed our software modules, etc.
You cannot imagine how much I would love to make everybody happy on this and
remove all the pain points. But, unfortunately, not always I know how to do
it or sometimes it is not even possible considering the status we are in with
the tools we have available.
Having said that, not everything is bad... if I look back a few years, I would
say that the situation is certainly improved and many of the things who used
to be done manually are now automated and we do not even think to them anymore
(after some initial effort). From my point of view the cost of maintaining
modules has been reduced as well as the unnecessary diversity.
Next week, I am going to focus on what we have and testing things for the
first Apache Release. After we have done a couple of releases in Apache,
we should come back to OSGi and explore what we can do to make life of
Apache Jena and OSGi users easier and as close as possible to "pain free".
OSGi is another important channel (as well as the Maven repository) to better
connect with advanced Apache Jena users (and therefore we should IMHO take
care of that).
The trouble is that it wasn't very Maven
compatible and as Jena and our other projects have got sucked ever
deeper into the Maven black hole I ended up having to use bndtools and
Maven in combination. We managed the make that work (e.g. Brian
developed a maven plugin for deploying bundles to a local bndtools
repository) but it isn't very satisfying and requires use of m2e. This
may improve, the bndtools folks had an intensive get together a few
weeks ago and Maven compatibility was very high on their agenda but the
new version hasn't been released yet.
Again, I do not have experience with what you describe. Are there things
we can/should change in the pom.xml files to make things easier?
No, this is not specific to Jena.
Ack.
The existing Maven plug-in for OSGi is very capable.
This one:
http://felix.apache.org/site/apache-felix-maven-bundle-plugin-bnd.html ?
Yes.
Ok.
Thanks,
Paolo
Cheers,
Dave
