Re: Maven Build

[Michał Kłeczek (XPro Sp. z o. o.)]

Indeed - it is possible to specify your dependencies as tightly as you 
want in OSGi. The issue is that:
1) It defeats one of the main purposes of OSGi in the first place
2) Nobody does that which would mean the only objects you can send over 
the wire would be instances of classes from specially crafted bundles

I agree the same problems are present in standard Java class loading - 
this is one of the reasons why I say River class loading is broken and 
needs to be fixed :)

The discussion about JBoss Modules goals is not really important in this 
context. I am only using it as an implementation of non-hierarchical 
class loading. Module identification and dependency resolution needs to 
be River specific anyway. After doing some experiments and analysis I 
dumped OSGi (which was my first choice and I am really willing to 
discuss OSGi integration - would love to see it done properly). JBoss 
Modules is a "bare bones" library that provides the mechanism while not 
imposing any policy. OSGi container can be (and is) implemented on top 
of it. River can be implemented on top of it as well.

The requirements for me generally are:
- downloaded code must not be executed before it is verified
- it should be easy to provide "composite services" - for example 
RemoteEventListener implementation wrapping JavaSpace proxy and 
publishing events to the space. At this moment this is impossible and in 
general requires non-hierarchical class loading.
- programmers should not be required to provide any River specific 
information inside jar files or any River specific "descriptors". This 
is difficult so if any such descriptor is absolutely necessary - build 
tools should be provided that automate this process.
- no specific deployment format and/or structure should be imposed

Two first are must have and two later - nice to have (or be as close as 
possible).

Some design decisions (in no particular order):

- codebase is an implementation of an interface (instead of being a 
String). This will make the mechanism extensible and - hopefully - 
future proof.
- codebase implementation classes are going to be resolved recursively 
using exactly the same algorithms (eat your own dog food). Recursion 
depth limited by a (configurable?) constant.
- codebase identity based on object equality (codebase implementation 
required to implement equals and hashcode). The implementation should be 
(directly or indirectly) based on cryptographic hashes equality. It is 
important not to base codebase identity on names only.
- since codebases are objects - they can be verified before use so no 
downloaded code is executed before use. What's more - since classes of 
objects are known to be trusted already - objects may safely verify 
themselves! What it means in general is that all TrustVerifier concept 
and implementation is not needed anymore in River. Objects can simply 
use built in serialization/ObjectInputValidation mechanisms and/or 
implement a River specific interface for this.
- JBoss Modules as an implementation of non-hierarchical class loading - 
BUT it is only one of possible implementations. Implementation based on 
PreferredClassLoader is possible as well (and might be provided as 
legacy fallback - but not that important at this moment since it is 
broken and I see the whole effort as River 3.0 breaking backwards 
compatibility anyway).
- basically "CodeBase" implementation needs to provide only one method: 
"ClassLoader createClassLoader()" (some more might be required to handle 
related issues like granting permissions to codebases so that they can 
create class loaders)

Some outstanding design questions:
- how to handle "private" codebases: for example my RemoteEventListener 
wraps a JavaSpace proxy and we want to treat the JavaSpace proxy 
codebase as "local" to the wrapper - it should get a separate 
ClassLoader even if the actual code is the same and is downloaded from 
the same location. It basically means location or content itself is not 
enough to establish identity. We also need something more to distinguish 
them. Changing the identity of the codebase might cause "lost codebase" 
issues so it has to be done properly.
- How to implement permission grants properly? Granting to class loaders 
requires duplicating class loaders just to have separate permission sets 
per object. Maybe something else is required based on object (not class) 
identity. That would also allow solving the problem above.

Thanks,
Michal
Niclas Hedhman <mailto:nic...@hedhman.org>
November 16, 2016 at 11:53 PM
On Wed, Nov 16, 2016 at 8:43 PM, "Michał Kłeczek (XPro Sp. z o. o.)"<
michal.klec...@xpro.biz>  wrote:

3. My comment about OSGi being "non-deterministic" in resolving
dependencies means that the
same bundle installed in two different environments is going to be linked
with different dependent
bundles. It basically means the same object graph valid in one VM
instance cannot be deserialized
in another VM instance unless:

Well, technically speaking this is true in standard Jini classloading
approach as well, since you have no control of which classes are being used
from parent classloaders.

a) all bundles have their dependencies specified in such a way that exact
same dependency graph
is going to be present everywhere (meaning no Package-Imports, no version
ranges etc)

This is incorrect. You can narrow his down with additional constraints
(typically package attributes). Not very well-known feature, but completely
possible, and whether you want version ranges or not, is up to you. Again,
traditional RMI can be set up to either use versioned or non-versioned
classes, in the dynamic loading part. I don't see much of a difference.
(Sorry, I can't comment on JBoss Modules as I have no experience)

b) all JVMs have the same set of bundles installed (ie - there is no
dynamic code downloading at all
- the code is preloaded)

I never worked with Paremus implementation, but I vaguely recall Richard
explaining that dynamically loaded bundles would be garbage collected when
not needed anymore. Sounds like dynamic code downloading to me.

c) classes of serialized objects are not loaded by OSGi framework class
loaders but by other
non-related class loading framework (which is what original old OSGi-Jini
spec did)

Well, that spec was dropped in OSGi Release 4 because it could not deal
with multiple versions of classes, that Release 4 introduced. The bundle
would both require DynamicImport-Package, as well as have no means to
export "new" functionality discovered, which is why I think it was dropped
(I wasn't privvy of the discussion back then)

d) classes of all objects in the object graph are from the same bundle
(special case of a) )

Yes, but not in itself a requirement per se.

In general all efforts I am aware of regarding "remoting" in Java didn't
really even try to solve issues of
dynamically downloaded code. They were based on all parties having the
same code installed and/or
having a central authority providing a shared consistent view of all
available software.

Yes, I agree. It is an age old issue with any remoting technology, and I
think the main reason Jini didn't succeed (except for awkward initial
licensing) was that people were weary of "Java only" solutions, effectively
making Java a virus. ;-)

I am not saying it is wrong. I am only saying that IMHO it is not
something River should do -
it is simply a solved problem and there is not point of re-doing it (JERI
is cool but it is simply yet-
another-RPC-stack)

JERI is cool because it tries to preserve security contexts. Yes, security
is seldom used in Java applications, mainly because we think we trust all
the code that is running. I think this assumption is slowly eroding away,
as we rely more and more on third-party code, which we can't vet in full to
see what is going on. Usage of containers is effectively taking a similar
stance at a lower level.


Regarding JBoss Modules:
I am not really advocating this particular library - it is just that it
is the only (non-OSGi - see above :) )
implementation of non-hierarchical class loading that I am aware of which
is of good quality and
actively maintained. I wouldn't want to reimplement it by myself. Thought
about ClassWorlds but it
doesn't seem to be too active and had some performance problems in the
past. JBoss Modules
is Apache licensed so it can be used by River.

The only thing I see in the quick browse of documentation is that it was
intended for loading JARs inside WARs inside EARs, and figure out how to
establish a working classloading mechanism inside such a cluster. Maybe I
am missing something really important, but it doesn't seem to be
particularly oriented towards ensuring the same classes being present
(which seems to be your beef with OSGi) in both JVMs.

I am not saying that your effort is bad, and I have no problem it being
donated to Apache River (or published elsewhere). I am only criticizing
your claims about OSGi, which I find to be not true.


Cheers
--
Niclas Hedhman, Software Developer
http://zest.apache.org - New Energy for Java

Richard Nicholson <mailto:richard.nichol...@paremus.com>
November 16, 2016 at 10:07 AM
Agree with Niclas. I don’t understand the resolution comment.

Not only is OSGi <> Jini integration feasible but it is a historical 
fact. Paremus did this - and a pretty good job we did - in … something 
like 2006 :-/

If the River community decided that there is interest in OSGi - then 
I’d suggest reading the Remote Service and Remote Service Admin 
specifications and thinking about how Jini concepts might enhance that 
world view. There are well over 10 million OSGi enabled IoT gateways 
out there!

Sorry - I see no compelling technical or commercial arguments for the 
JBoss Module route.



Niclas Hedhman <mailto:nic...@hedhman.org>
November 16, 2016 at 9:11 AM
I am curious, what do you mean by "non-deterministic dependency 
resolution"
? You can make it as predictable as you wish with attributes and
directives.

Cheers
Niclas

On Wed, Nov 16, 2016 at 4:07 PM, Michał Kłeczek <michal.klec...@xpro.biz>



Michał Kłeczek <mailto:michal.klec...@xpro.biz>
November 16, 2016 at 9:07 AM
While non-hierarchical class loading is crucial, OSGI with its 
non-deterministic dependency resolution is very difficult ( if not 
impossible ) to target.
I'm working on JBoss Module based class loading for River which I'm 
going to propose as contribution soon.

Thanks,
Michal

On Wednesday, 16 November 2016, Dawid Loubser <da...@travellinck.com 
<mailto:da...@travellinck.com>> wrote:
Dawid Loubser <mailto:da...@travellinck.com>
November 16, 2016 at 7:18 AM
+1 for OSGi providing the best solution to the class resolution problem,
though I think some work will have to be done around trust, as you say.