On 15/02/2022 8:29 pm, Michał Kłeczek wrote:
Hi Peter,
JGDMS uses a new implementation of a subset of the Java
Serialization’s stream format, with input validation and defenses
against malicious data (all connections are first authenticated when
using secure endpoints). Codebase annotations are no longer
appended in serialization streams, this feature is deprecated but it
can still be enabled.
How the client knows the code needed to deserialise?
The service provides this information, typically in a services
configuration, by default this is a space separated list of URI, similar
to a codebase annotation, but it doesn't have to be. JERI manages the
deserialization of code through a default ProxyCodebaseSpi
implementation, the client applies constraints, to ensure that input
validation is used as well as any other constraints, such as principals,
or encryption strength. ProxyCodebaseSpi can be customized by the
client, so the client may implement ProxyCodebaseSpi if it wants to do
something different, eg use OSGi or Maven to manage dependency resolution.
Does it require to have it installed in advance? If so - how?
Only JGDMS platform and JERI.
How is the following scenario handled:
class JavaSpaceEventPublisher implements RemoteEventListener,
Serializable {
private final JavaSpace space;
//… publish event in JavaSpace implementation
}
The smart proxy class has dependencies on RemoteEventListener and on
JavaSpace. How do you properly resolve classes in this case?
Typically the client has the ServiceAPI it needs already installed
locally, however this may not always be the case, depending on how you
want to resolve the proxy classes and how much you want to share with
the client, you can include additional jar files in the annotation, and
use preferred.list or you can use Maven or OSGi to resolve dependencies
and provision the ClassLoader used for proxy deserialization.
This paper documents the problems with this approach:
https://dl.acm.org/doi/pdf/10.5555/1698139
JGDMS provisions a ClassLoader at each Endpoint, the ClassLoader is
solely responsible for class resolution, once it has been assigned to
the relevant ObjectEndpoint. A provider mechanism allows customization.
JGDMS doesn't suffer from codebase annotation loss, nor class
resolution issues. But it did have to give up some functionality;
it cannot resolve classes that do not belong to a service proxy or
its service api and are not resolvable from the Endpoint ClassLoader,
if they are not present on the remote machine. The solution is to
always use a service, for parameters passed to a service, if they are
not part of the service api, eg the client overrides the type of
parameter arguments for a service. This means that if the parameter
is not an interface, you cannot create a service that implements it
and pass it as an argument. That’s why its still possible, but not
recommended to use codebase annotations appended to the serialization
stream. The solution is to create service api that uses only
interfaces for parameter arguments. For example a remote events and
listeners use this pattern. To prevent unexpected breakages, either
use interfaces, or final classes, or both, for service api remote
method parameters. Then you won’t get into the situation where you
need codebase annotations appended in the stream.
I am not sure I follow but...
What I am trying to achieve is exactly the opposite - place as little
constraints as possible on service implementors and make the whole
thing “magically work” :)
JGDMS only does this with AtomicILFactory by default, you aren't
constrained to using that, you can enable codebase annotations in the
stream, or override BasicILFactory, if you want to do something
different, or just use BasicILFactory as is, you can avoid applying this
restriction to service parameter arguments, but then you have to accept
the compromises that come with that such as codebase annotation loss,
which can spoil the magic.
For me it is simpler to use interface types for service method arguments
and provide a final implementation class as part of the Service API,
this allows the client to either use the default ServiceAPI classes or
implement the interface with another service.
If you want to use non final classes for your service method arguments
and allow clients to override these classes, then you will need to
enable codebase annotations in AtomicILFactory in your configuration.
The caveat is there is no guarantee, the service will be able to resolve
these classes at the server endpoint, or that codebase annotation loss
won't occur, it will try using existing mechanisms, such as
RMIClassLoaderSPI, which is probably fine for seasoned Jini vets, but
not so user friendly for the newbie, who now has to debug
ClassNotFoundExceptions.
It's like Java serialization, magic comes with compromises.
For example if a service proxy is serialized within a serialization
stream, it will be replaced by a proxy serializer and it will be
assigned its own independent stream, with ClassLoader, independent of
the stream in which it was serialized. This is based on the
ObjectEndpoint identity, so it will always resolve to the same
ClassLoader. Note that ProxyCodebaseSpi can be a provider or OSGi
service.
Does it mean you cannot provide services that don’t have any
ObjectEndpoint (ie. local only)?
This would be IMO unacceptable constraint. For example:
- How do you provide the above mentioned JavaSpaceEventPublisher
- How would you provide a java.sql.DataSource as a service?
If you don't have an ObjectEndpoint, then there is no one to
authenticate, you only have bytes to de-serialize, so how do you
establish trust, ie who did the bytes come from? However it is possible
to have a service that has an ObjectEndpoint and only uses it for
authentication of the proxy serializer and codebase provisioning after
which the deserialized bytes become objects and don't make remote method
calls. I think that would be an acceptable alternative; someone needs
to vouch for the serialized bytes.
Now the proxy serializer is itself a service (bootstrap proxy), that
is authenticated when using secure endpoints. You could quite easily
add an interface to the proxy serializer to return your object
annotation.
Note that I use a string, because I also use it in secure multicast
discovery protocols (typically IPv6), which don't include objects,
for authentication and provisioning a ClassLoader for a lookup
service proxy prior to any Object de-serialization.
https://www.iana.org/assignments/ipv6-multicast-addresses/ipv6-multicast-addresses.xhtml
Summing up to simplify JGDMS and solve some very difficult issues, it
had to give up:
1. Support for circular references in serialized object graphs, was
dropped.
My solution supports any object graph.
What capability do you need circular references for? For example
Throwable creates a circular object graph. These circular relationships
can be reconstructed during deserialization by code, rather than as part
of the serialized bytes. Allowing circular object graphs, sounds like
a neat feature, however it is not possible to validate circular
relationships atomically, in which case if validation checks fail, you
have fully constructed objects than an attacker might manipulate for
gadget attacks. Atomic input validation prevents object creation, so
no gadget attack. This is a low risk if you are using authentication,
but it is still nice to harden the api, in the event that the identity
of someone you trust has been stolen.
1.
2. Extensible classes in service api method parameters are not advised.
Yes - this one is tricky although my solution supports that as well
(there are edge cases though).
2.
3. ProxyTrust - deprecated and replaced with secure authentication
and httpmd (SHA-256) or signer certificates using ProxySerializer.
Deprecated in my solution as well: code is validated _before_ execution.
Good decision, it wasn't a good solution and causes unnecessary
complexity for zero benefit.
4. Untrusted machines are not allowed in a djinn, some level of
trust is required, with authentication and authorisation constraints.
Not necessary in my solution.
In general I think the differences are caused by the different
perspective:
I see software distribution (ie. mobile code) as orthogonal to networking.
You see River primarily as networking solution with mobile code
dependent on it.
It would be great to be able to merge the ideas and work on common
solution.
The question is whether it should be Apache River project…
I think the PMC has already decided River's fate, and I tend to agree
with their decision, the problem is that historically, it hasn't been
possible to innovate inside the Apache River project, innovation has
been forced to happen elsewhere and it wasn't just what I was doing,
there was an attempt to do some container work in River but that also
got shut down. People had trouble in the past agreeing on Rivers
direction and there are no currently active developers. It is still
possible to get a group of people together to create an Apache project,
but I don't think the code needs it. github and other sites like it
are better for loose collaboration, where developers can feed off each
others ideas and innovations and the best solutions survive.
BTW the name of the software I am working on is CodeSpaces (and yes I
am aware of MS/GitHub product named the same but I came up with it
earlier and even registered a domain net.codespaces for this purpose).
Michal
I'll keep an eye out for your work. I think you'll be able to add this
functionality extensible to JGDMS as a downstream project, dependent on
a few JGDMS modules, that way you can focus on what you need to achieve
without worrying about managing the whole repository. There may be non
JGDMS forks of River out there too, but I don't think they will support
IPv6 discovery and other features that will be of benefit, such as
support for modern stateless TLS v1.3 session tickets.
Cheers,
Peter.
