I probably should have vetted this before hitting send... let me know if
you need any clarifications.
Cheers,
Peter.
On 23/08/2019 12:59 PM, Peter Firmstone wrote:
"...since at the time the industry believed that distributed objects
were going to save us from complexity.) Many of the sins of
serialization were committed in the desire to get that last .1%, but
the cost and benefit of that last .1% are woefully out of balance."
The following are probably a non goals, but something to consider or
keep in mind, relating to distributed objects:
The are three types of distributed objects:
1. Immutable value / data Object types.
2. Shared Mutable Objects.
3. Unshared Mutable Objects.
4. Remote Objects / Services (best for managing shared mutable state).
The second type of distributed object causes much pain and should be
discouraged. The first three types of distributed objects can have
class resolution issues, but these are solveable.
A lot of folks also have problems with deserialization Objects when
class visibility is different at both ends, I'm guessing this would be
the same for value types.
For example OSGi folk recommend using primitive parameter types for
remote OSGi services.
RMI annotates streams with codebase annotations. Jini Extensible
Remote Invocation used to do that too.
The problem with RMI codebase and Jini codebase annotations is if you
resolve your classes locally, you lose the codebase annotations, when
re-serializing data and because class visibility can be different at
different endpoints, end up with all sorts of class resolution
issues. "Class Loading Issues in Java™ RMIand Jini™ Network
Technology" by Michael Warres
https://pdfs.semanticscholar.org/143f/468fcbdafd20f2b8c27fe5e0a869913b641a.pdf
The solution of course is simple, ensure that you deserialize into the
same module that you serialized from, especially when deserializing in
another jvm, so class resolution is identical.
We serialize a lot of complex object graphs, none are circular. The
module used for serialization should have visiblity of the entire
graph of object classes.
So if we're using OSGi modules, and provide a network / remote service
(not to be confused with an OSGi remote service) we ensure the proxy's
for these services have the same module installed at the client and
server endpoints. The service is represented by a Java interface and
the client makes calls on the interfaces methods. This interface may
be implemented by what is called a smart proxy, which is encapsulated
by a module which is dynamically downloaded at runtime, or a
reflection Proxy using an InvocationHandler that is generated
dynamically.
We still provide an option for codebase annotations for client
parameter objects, where a client subclasses parameter types and pass
them to the service, but this is discouraged, it is provided for
backward compatibility only. Where the parameters are also
interfaces, the client can implement a remote object and pass it as a
parameter instead, in our system, this will cause a module to be
loaded in the server identical to that at the client to resolve the
remote object classes, without using stream codebase annotations.
Incidentally, if you're curious how this happens, a proxy is sent {I
guess you can call it a serialization proxy :) } and authenticated by
the remote end, security constraints applied, then the remote end asks
the proxy for a codebase URL,which is loaded into a ClassLoader with
controlled visibility, this is extensible using a ServiceProvider or
OSGi service, then the proxy is deserialized into this by calling a
method on the serialization proxy.
By limiting scope, we can still have 99% of the benefits of
distributed objects, without the pain.
Incidentally apart from the complexity of class resolution, what
really limited distributed computing was IPv4. IPv6 removes the
network addressing limitations placed on distributed computing.
So I'd make the following qualifications:
1. Use only primitive types when serializing between different
languages.
2. Serialize Java language Object types and primitives only between
jvm's when class visibility is uncontrolled.
3. When serializing other object types, ensure they are immutable if
shared and that class visibility is identical and managed at both
endpoints.
4. Do not serialize objects whose classes may not be resolveable
(when you need to depend on annotated streams and uncontrolled
class resolution for example), find another way to solve the
problem.
We've had a 20 years to iron out the wrinkles. :)
Regards,
Peter.
On 23/08/2019 7:36 AM, Peter Firmstone wrote:
Hi Sean,
Regarding the section entitled "Why not write a new serialization
library?", unlike the serialization libraries listed, our purpose was
to be able to securely deserialize untrusted data, while maintaining
backward serial form compatibility with Java Serialization, provided
it didn't compromise security.
We don't use blacklists or whitelists, we use permissions to grant
DeserializationPermission, it doesn't have the granularity of white
lists, but then, classes that implement @AtomicSerial are supposed to
be hardened implementations in any case.
If it can be of use, feel free to experiment with it, hopefully it
might help with some of your design decisions:
https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/jgdms-platform/src/main/java/org/apache/river/api/io
Much of the code on this site provides implementation examples as well.
Regards,
Peter.
On 20/08/2019 7:55 AM, Sean Mullan wrote:
Brian Goetz (copied) has done a lot of thinking in the serialization
area, so I have copied him. Not sure if you have seen it but he
recently posted a document about some of his ideas and possible
future directions for serialization:
http://cr.openjdk.java.net/~briangoetz/amber/serialization.html
--Sean
On 8/17/19 10:22 PM, Peter Firmstone wrote:
Thanks Sean,
You've gone to some trouble to answer my question, which
demonstrates you have considered it.
I donate some time to help maintain Apache River, derived from
Sun's Jini. Once Jini depended on RMI, today, not so much, it
still has some dependencies on some RMI interfaces, but doesn't
utilise JRMP although it provides some backward compatibilty enable
it.
But my point is, we heavily utilise java Serialization, and have an
independant implementation of a subset of Java Serialization
(originating from Apache Harmony). We do this for security as we
use an annotated serialization constructor. Serial form is
unchanged, we have Serializers for commonly used java library
objects, for example, we have a "PermissionSerializer", but we
don't have a "PermissionCollectionSerializer" or
"PermissionsSerializer" (for java.security.Permissions).
Incidentally, we have found we do not need the ability to serialize
circular object graphs. Throwable is an object that has a
circular object graph, but that circular object graph can be linked
up after deserialization.
Permission implementing Serializable is probably not too much of a
threat, as these objects are effectively immutable after lazy
initialization.
ProtectionDomain calls java.security.Permissions::setReadOnly
during it's construction.
ProtectionDomain::getPermissions returns internal
java.security.Permissions. If this is serialized, then the
readOnly internal state can be written to as the internal object
references are accessible from within the stream.
Admitedly, the attacker would already need to have some privilege,
to have access to a ProtectionDomain, so it's a path of privilege
escallation. I'm not talking about gadget attacks and
deserialization of untrusted data, I'm talking about breaking
encapsulation.
Even though we are heavily dependant on Java Serialization, we are
very careful when we implement it, and avoid implementing it when
possible. Hindsight is 20:20, but given we are now seeing some Java
SE backward compatibility breakages, perhaps it might be worth
considering breaking serialization. I don't mean we need to
necessarily break object serial form, but making the Java
serialization API explicit with subset of existing api features,
that makes long term maintenace and security less of a burden and
removing support for Serialization of some objects, where it is
seldom used, perhaps using a JEP that requests developers to
consider which library objects actually need to be serializable.
Something we do in our Java Serialization API is require that
mutable deserialized objects are defensively copied during object
construction (serial fields are deserialized before an object is
constructed, the deserialized fields are accessible via a parameter
passed in during construction. We have tools that assist
developers to check deserialized Java Collections contain the
expected object types for example, so during object construction
the developer has to replace the Collection with a new instance and
copy the contents to the new Collection after checking the type of
each object contained therein. Also we don't actually serialize
Java Collections, we have standard serial forms for List, Set and
Map, so these serial forms are equal, similar to the List, Set and
Map contracts. By doing this, Collections don't actually need to
implement Serializable at all, as a Serializer becomes responsible
for their serialization. This also means that all Collections
must be accessed by interfaces, rather than implementation classes,
so the deserialization constructor, must defensively copy them into
their preferred Collection instance. It's a bit like dependency
injection.
I know it would take time, and there would be some pain, but long
term it would save a lot of maintenance developer time.
Regards,
Peter.
On 17/08/2019 12:50 AM, Sean Mullan wrote:
On 8/15/19 8:18 PM, Peter Firmstone wrote:
Hi Roger,
+1 for writeReplace
Personally I'd like to see some security classes break backward
compatibility and remove support for serialization as it allows
someone to get references to internal objects, especially since
these classes are cached by the JVM. Which makes
PermissionCollection.setReadOnly() very easy to bypass, by adding
permissions to internal collections once you have a reference to
them.
Does anyone have any use cases for serializing these objects?
These objects are easy to re-create by sending or recieving and
parsing strings, because they are built from text based policy
files, and when you do that, you are validating input, so I never
did fully understand why they were made serializable.
This is briefly explained on page 61 in the "Inside Java 2
Platform Security" book [1]:
"The Permission class implements two interfaces:
java.security.Guard and java.io.Serializable. For the latter, the
intention is that Permission objects may be transported to remote
machines, such as via Remote Method Invocation (RMI), and thus a
Serializable representation is useful."
The Permission class was introduced in Java SE 1.2 so there were
different motivations back then :)
--Sean
[1] https://www.oracle.com/technetwork/java/javaee/index-141918.html
