Some performance figures, atomic de-serialization using loopback local
network sockets, codebase downloads, dynamic class loading, with
security manager and policies in force and Jini extensible remote method
invocation, there's communication going on between 4 different jvm
instances, the one
A simpler example of production code modified to support @AtomicSerial
that satisfies invariants follows. Note how we can't type check
Generics with the static validator method, but the compiler does it for
us with the constructor :)
I'm wondering if there's scope for secure alternatives of
Thanks Chris,
see below...
On 4/02/2015 12:14 AM, Chris Hegarty wrote:
Hi Peter,
On 2 Feb 2015, at 11:16, Peter Firmstone wrote:
As mentioned I've been experimenting with an invariant validating
ObjectInputStream, that's backward and forward compatible with Java's Object
Serialization str
Hi Peter,
On 2 Feb 2015, at 11:16, Peter Firmstone wrote:
> As mentioned I've been experimenting with an invariant validating
> ObjectInputStream, that's backward and forward compatible with Java's Object
> Serialization stream protocol.
>
> No changes have been made to ObjectOutputStream.
>
As mentioned I've been experimenting with an invariant validating
ObjectInputStream, that's backward and forward compatible with Java's
Object Serialization stream protocol.
No changes have been made to ObjectOutputStream.
ObjectInputStream has been overridden and reading from the stream has
I decided to sample cpu load (see attached), with debugging enabled for
the validating ObjectInputStream and JERI, so heaps of output to the
console.
There are no performance optimisations with stream validation, I've just
focused on correctness and security.
Thank you HotSpot developers, ni
Two examples:
1. of a child class with super class invariants and
2. Checking List and Map contents for type correctness.
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for additional
For the sharper eyed, you'll have noticed the readObjectNoData() method
I forgot to check:
private static boolean check(GetArg arg) throws IOException{
// If this class is not in the heirarchy of classes the stream
// may have been tampered with, see the Serialization Specification
/
Another example of intra dependencies, turns out we have a lot of these
intra class dependencies in our project.
Cheers,
Peter.
/*
* Licensed to the Apache Software Foundation (ASF) under one
* or more contributor license agreements. See the NOTICE file
* distributed with this work for a
Logging output from failed validation:
NonActGrp-out: Jan 29, 2015 7:36:16 PM
net.jini.jeri.BasicInvocationHandler invoke
NonActGrp-out: Jan 29, 2015 7:36:16 PM
net.jini.jeri.BasicInvocationHandler invoke
NonActGrp-out: FAILED: outbound call
net.jini.core.event.RemoteEventListener.notify remot
Just a quick note, to avoid stream corruption, object that aren't
instantiated are replaced by an enum marker, Reference.DISCARDED, and
null is returned in their place, fields are read and any trailling
writeObject data is discarded. Object that are not allowed to be
deserialized are still re
Although not directly relevant to this discussion, here are some
functional examples of deserialization constructors, I now have a fully
functional validating ObjectInputStream.
Unfortunately in our project we have intra object dependencies as
demonstrated by this example; a static validator,
I have attempted to capture some of the ideas that we have discussed so far.
https://bugs.openjdk.java.net/browse/JDK-8071471
-Chris.
On 21/01/15 21:43, David M. Lloyd wrote:
At some point, the responsibility *must* fall on the author of the
serializable class in question to avoid constructs
And that is the point of this discussion, to determine how to do just that.
But considering it's such a difficult problem, we haven't figured out how to
yet.
Peter.
- Original message -
>
> Message: 2
> Date: Wed, 21 Jan 2015 15:43:13 -0600
> From: "David M. Lloyd"
>
> At some point,
At some point, the responsibility *must* fall on the author of the
serializable class in question to avoid constructs which are exploitable
- like finalizers, and classes that can have side-effects before
validation can be completed.
On 01/21/2015 02:26 PM, Peter Firmstone wrote:
Don't forget
Don't forget that "null" may also be an invalid state.
What I have learnt so far:
1. An attacker can craft a stream to obtain a reference to any object
created during deserialization, using finalizers or circular links
(there may be yet other undiscovered methods).
2. An attacker c
On 20/01/15 20:22, Peter Levart wrote:
Hi Chris and Peter,
It just occurred to me that the following scheme would provide failure
atomicity for the whole Object regardless of whether readObject methods
are used or not for various classes in hierarchy:
- allocate uninitialized instance
- call d
Hi Chris and Peter,
It just occurred to me that the following scheme would provide failure
atomicity for the whole Object regardless of whether readObject methods
are used or not for various classes in hierarchy:
- allocate uninitialized instance
- call default accessible constructor of the
- Original message -
> On 15/01/15 20:33, Peter Firmstone wrote:
> > Thanks Chris,
> >
> > WRT circular references, is it possible to detect and delay setting
> > these until after all verifiers run?
>
> It is possible to detect the circular reference.
>
> Currently you can retrieve a ci
On 15/01/15 20:33, Peter Firmstone wrote:
> Thanks Chris,
>
> WRT circular references, is it possible to detect and delay setting
> these until after all verifiers run?
It is possible to detect the circular reference.
Currently you can retrieve a circular reference ( in readObject ) from
readFi
Thanks Chris,
WRT circular references, is it possible to detect and delay setting these until
after all verifiers run?
An option might be to provide a protected method in ObjectInputStream to opt
out or switch off support for circular references.
Apart from Serialization, another common use fo
On 15 Jan 2015, at 12:25, Peter Firmstone wrote:
> Chris,
>
> Can you explain some of the detail in your failure atomicity work?
Certainly.
The current implementation of defaultReadObject sets non-primitive field values
one at a time, without first checking that all their types are assignable
On 15 January 2015 at 13:01, Peter Firmstone
wrote:
> A third option that hasn't yet been considered would be to investigate an
> api that various already existing frameworks can implement providers for, so
> they no longer need to find creative ways to construct instances of java
> platform class
On 13/01/2015 1:56 AM, Chris Hegarty wrote:
On 10/01/15 07:00, Peter Firmstone wrote:
As shown in my earlier example, intra class invariants can be enforced
using Serialization constructors, from which static methods are called
to check invariants prior to super class constructors being called.
Chris,
Can you explain some of the detail in your failure atomicity work?
Thanks,
Peter.
The following is a summary of our exploration so far:
We have narrowed down to two options:
A. constructors with static invariant check methods.
B. Continue to use readObject(), we have a proposed solution to the
final field problem and could modify the jvm to register for
finalization lat
On 01/14/2015 12:37 PM, Chris Hegarty wrote:
What do you think?
I agree completely.
An API at the level that you are proposing will provide the necessary
functionality and flexibility that is required to do validation in
readObject. As you clearly stated, and is already the case, validation
It's only top down if there are intra class invariants, in this case invariants
are checked more than once.
This done simply because we can't move up the constructor call chain untill all
child class invariants have been satisfied.
For simpler class relationships, invariant check order is botto
On 14/01/15 12:58, Peter Firmstone wrote:
Hi Chris,
Sorry, no.
Currently when an ObjectStreamClass is read in from the stream, the
framework searches for the first zero arg constructor of a non
serializable class and creates and instance of the class read and
resolved from the stream, however i
Hi Chris,
Sorry, no.
Currently when an ObjectStreamClass is read in from the stream, the framework
searches for the first zero arg constructor of a non serializable class and
creates and instance of the class read and resolved from the stream, however it
does so using a super class constructor
On 13/01/15 00:51, Peter Firmstone wrote:
- Original message -
> On 10/01/15 07:00, Peter Firmstone wrote:
> > Again, thank you all for engaging in discussion of this very difficult
> > topic.
> >
> > While we can't presently check intra object dependencies during
> > deserializatio
On 13/01/15 10:00, Peter Levart wrote:
Hi Chris,
I stepped back and asked myself what problem(s) we are searching
solution(s) for and tried to see how existing API solves or doesn't
solve them. I think that existing readObject() + GetFields API already
solves most of the problems discussed so fa
Peter F,
I am still struggling with the basic concept of you proposal. Let me see
if I understand it correctly. Does the following describe a similar
scenario as you envisage:
1) For each Serializable type, T, in the deserialized types
hierarchy, starting with the top most ( closest to j
Could we use a static validator method and generate bytecode for constructors
dynamically?
The developer can optionally implement the constructors.
static GetField invariantCheck(GetField f);
Create a caller sensitive GetField implementation and add a two new methods to
GetField:
abstract Obj
Hi Chris,
I stepped back and asked myself what problem(s) we are searching
solution(s) for and tried to see how existing API solves or doesn't
solve them. I think that existing readObject() + GetFields API already
solves most of the problems discussed so far as it:
- provides encapsulation (us
- Original message -
> On 10/01/15 07:00, Peter Firmstone wrote:
> > Again, thank you all for engaging in discussion of this very difficult
> > topic.
> >
> > While we can't presently check intra object dependencies during
> > deserialization with readObject(), the examples I provide
> Am 12.01.2015 um 17:15 schrieb Stephen Colebourne :
>
> On 12 January 2015 at 11:37, Chris Hegarty wrote:
>> For clarity, I would like to describe how things currently work.
>>
>> 1) Allocate a new instance of the deserialized type.
>> 2) Call the first non-Serializable types no-arg construct
On 12 January 2015 at 11:37, Chris Hegarty wrote:
> For clarity, I would like to describe how things currently work.
>
> 1) Allocate a new instance of the deserialized type.
> 2) Call the first non-Serializable types no-arg constructor
> ( may be j.l.Object ).
> 3) For each type in the dese
On 10/01/15 07:00, Peter Firmstone wrote:
Again, thank you all for engaging in discussion of this very difficult
topic.
While we can't presently check intra object dependencies during
deserialization with readObject(), the examples I provide can do this.
I have replied to Davids mail with a sm
On 01/12/2015 05:51 AM, Chris Hegarty wrote:
On 08/01/15 22:03, David M. Lloyd wrote:
private static void validate(GetField fields) {
if (fields.getInt("lo") > fields.getInt("hi")) { ... }
}
...
In fact you cannot validate invariants across multiple objects at
On 08/01/15 22:03, David M. Lloyd wrote:
private static void validate(GetField fields) {
if (fields.getInt("lo") > fields.getInt("hi")) { ... }
}
...
In fact you cannot validate invariants across multiple objects at all
using this method *or* readObject() (exis
On 08/01/15 20:10, Brian Goetz wrote:
1) Validate invariants
A clear and easy to understand mechanism that can validate the
deserialized
fields. Does not prevent the use of final fields, as the
serialization framework
will be responsible for setting them. Something along the lines
Again, thank you all for engaging in discussion of this very difficult
topic.
While we can't presently check intra object dependencies during
deserialization with readObject(), the examples I provide can do this.
ObjectInputValidation is sufficient for enforcing business rules,
however it ca
On 01/08/2015 02:10 PM, Brian Goetz wrote:
1) Validate invariants
A clear and easy to understand mechanism that can validate the
deserialized
fields. Does not prevent the use of final fields, as the
serialization framework
will be responsible for setting them. Something along the
1) Validate invariants
A clear and easy to understand mechanism that can validate the deserialized
fields. Does not prevent the use of final fields, as the serialization
framework
will be responsible for setting them. Something along the lines of what
David
suggested:
Peter, David
I would like to see how far we can push the existing Serialization mechanism,
before embarking on a road that may lead us to substantially different one.
Whether that be constructor based, to otherwise ( we know we will need a
replacement for Unsafe.allocateInstance, for third-part
On 01/08/2015 02:32 AM, Peter Firmstone wrote:
Thank you all for participating in this discussion.
Initially a constructor signature for deserialization was proposed to
enforce invariants and encapsulation, however there appears to be
interest in using alternative methods, although they appear t
Thank you all for participating in this discussion.
Initially a constructor signature for deserialization was proposed to
enforce invariants and encapsulation, however there appears to be
interest in using alternative methods, although they appear to be
improvements over the status quo, I'm ha
On 01/07/2015 10:02 AM, Peter Levart wrote:
On 01/07/2015 03:54 PM, Chris Hegarty wrote:
On 06/01/15 17:49, Peter Levart wrote:
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(Obje
On 01/07/2015 03:54 PM, Chris Hegarty wrote:
On 06/01/15 17:49, Peter Levart wrote:
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNo
On 06/01/15 17:49, Peter Levart wrote:
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
ObjectInputStream.Ge
On 01/06/2015 06:21 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 15:06, Peter Levart wrote:
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
ObjectInputStream.GetField fields = in.readFields(); // thi
On 6 Jan 2015, at 15:06, Peter Levart wrote:
> On 01/06/2015 04:03 PM, Peter Levart wrote:
>> private void readObject(ObjectInputStream in) throws IOException,
>> ClassNotFoundException {
>>ObjectInputStream.GetField fields = in.readFields(); // this already
>> validates the types
>
On 01/06/2015 04:03 PM, Peter Levart wrote:
private void readObject(ObjectInputStream in) throws IOException,
ClassNotFoundException {
ObjectInputStream.GetField fields = in.readFields(); // this
already validates the types
Well, not true currently. But type validation could be added
Hi Chris,
On 01/06/2015 12:10 PM, Chris Hegarty wrote:
On 6 Jan 2015, at 07:27, Peter Levart wrote:
Hi Brian,
On 01/05/2015 09:51 PM, Brian Goetz wrote:
Thanks Peter(s): I think it's just about time to re-sync on the goals, since
there are an infinitude of issues with serialization, only som
On 6 January 2015 at 11:46, Chris Hegarty wrote:
> With shallow/no hierarchies, and others, the serialization proxy pattern
> works quite well. The overhead of course being the creation of the proxy
> itself, but this is typically a minimal container with just the state needed
> to create the “
On 6 Jan 2015, at 11:13, Stephen Colebourne wrote:
On 6 January 2015 at 10:25, Chris Hegarty wrote:
>> On 6 Jan 2015, at 08:31, Stephen Colebourne wrote:
>>> I've thought on a number of occasions that what I wanted from
>>> serializable was a merger of readObject and readResolve
>>>
>>> private
On 5 Jan 2015, at 20:51, Brian Goetz wrote:
> Thanks Peter(s): I think it's just about time to re-sync on the goals, since
> there are an infinitude of issues with serialization, only some of which we
> can address.
>
> To that end, let me toss out some of the results of our recent exploration
- Original message -
> > > On 5/01/2015 6:51 PM, Peter Firmstone wrote:
> > > > Thanks Dave,
> > > >
> > > > That's another way of checking invariants, you could expose A's
> > > > check method but you'd also need a couple of additional static
> > > > methods to obtain integers upper and l
On 6 January 2015 at 10:25, Chris Hegarty wrote:
> On 6 Jan 2015, at 08:31, Stephen Colebourne wrote:
>> I've thought on a number of occasions that what I wanted from
>> serializable was a merger of readObject and readResolve
>>
>> private Object readObjectAndResolve(ObjectInputStream in) throws
On 6 Jan 2015, at 07:27, Peter Levart wrote:
Hi Brian,
>
> On 01/05/2015 09:51 PM, Brian Goetz wrote:
>> Thanks Peter(s): I think it's just about time to re-sync on the goals, since
>> there are an infinitude of issues with serialization, only some of which we
>> can address.
>>
>> To that end
On 6 Jan 2015, at 08:31, Stephen Colebourne wrote:
> On 6 January 2015 at 07:27, Peter Levart wrote:
>> readObject() can be used on both ends of the spectrum or anywhere
>> in-between. It can be used for explicit deserialization or just for
>> validation of default deserialized state. Would sepa
On 6 January 2015 at 07:27, Peter Levart wrote:
> readObject() can be used on both ends of the spectrum or anywhere
> in-between. It can be used for explicit deserialization or just for
> validation of default deserialized state. Would separate validation method
> give us anything more or simplify
Hi Brian,
On 01/05/2015 09:51 PM, Brian Goetz wrote:
Thanks Peter(s): I think it's just about time to re-sync on the goals,
since there are an infinitude of issues with serialization, only some
of which we can address.
To that end, let me toss out some of the results of our recent
exploratio
On 5/01/2015 6:51 PM, Peter Firmstone wrote:
Thanks Dave,
That's another way of checking invariants, you could expose A's check
method but you'd also need a couple of additional static methods to
obtain integers upper and lower from ReadSerial, so that B can ensure
its invariant.
Not sure I fo
Thanks Peter(s): I think it's just about time to re-sync on the goals,
since there are an infinitude of issues with serialization, only some of
which we can address.
To that end, let me toss out some of the results of our recent
explorations with improving serialization, where we started out o
On 01/05/2015 03:17 PM, David M. Lloyd wrote:
Would something like this prevent Finalizer attacks?
- leave finalization registration the way it is (at object allocation
time).
This was written incorrectly: "after Object default constructor completes"
- provide internal API with which a previ
On 01/03/2015 02:29 PM, Peter Firmstone wrote:
- Original message -
Wouldn't it be better to "register" for finalization just those instances that
complete their
construction or deserialization normally? I'm just trying to understand
why it is the way it is.
Perhaps that might be an o
On 01/03/2015 09:36 AM, Peter Levart wrote:
On 01/03/2015 01:38 PM, Peter Firmstone wrote:
> Hi,
>
> I would like to know what are the potential issues with simple
> constructor chaining where each constructor checks the invariant of its
> class state (with the already initialized state of sup
Hi Peter and others,
A assume your approach described here (to use constructors for
deserialization) is motivated mainly by the fact that only constructors
are allowed to set the final fields. Otherwise the explicit features of
your ReadSerial API are more or less accessible even now by using
My mistake, thank you.
Peter.
On 5/01/2015 9:57 PM, David Holmes wrote:
Hi Peter,
Did you send this to the list? I haven't seen it turn up yet.
David
On 5/01/2015 6:51 PM, Peter Firmstone wrote:
Thanks Dave,
That's another way of checking invariants, you could expose A's check
method but y
On 3/01/2015 9:24 PM, Peter Firmstone wrote:
Thanks Brian,
Those are good questions, some thoughts and examples inline:
- Original message -
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a view
of i
- Original message -
>
> On 01/04/2015 02:48 AM, Peter Firmstone wrote:
> > class B extends A {
> >
> > public final int cur;
> >
> > private static ReadSerial check(ReadSerial rs) {
> > A a = new A(rs);
>
> What to do if A is abstract?
:)
Create an anonymous instance, or create a sta
On 01/04/2015 02:48 AM, Peter Firmstone wrote:
class B extends A {
public final int cur;
private static ReadSerial check(ReadSerial rs) {
A a = new A(rs);
What to do if A is abstract?
Regards, Peter
int cur = rs.getInt("cur");
if ( a.lower> cur || cur> a.upper )
On 4/01/2015 9:55 AM, Peter Levart wrote:
- Original message -
>
> As Brian points out, this scheme can only validate intra-class
> invariants. It can't validate class-against-subclass state.
Sorry, I meant it can't validate class -against-superclass state.
Did he say that?
Not dir
On 01/03/2015 09:29 PM, Peter Firmstone wrote:
- Original message -
>
> As Brian points out, this scheme can only validate intra-class
> invariants. It can't validate class-against-subclass state.
Sorry, I meant it can't validate class -against-superclass state.
Did he say that?
N
P.S. Thanks for engaging this difficult subject.
It's worth remembering the finalizer attack isn't the only issue, a subclass
will have a reference after construction completes, it has a thread of
execution and if the superclass hasn't checked invarients, because circular
links haven't been wir
- Original message -
>
> As Brian points out, this scheme can only validate intra-class
> invariants. It can't validate class-against-subclass state.
Did he say that?
It's true that a superclass can't validate subclass state, it can't be expected
to know much about it, but it can validat
Just a quick comment about the finalization aspect ( as I have been thinking
about this too ).
On 3 Jan 2015, at 15:36, Peter Levart wrote:
> On 01/03/2015 01:38 PM, Peter Firmstone wrote:
>>
>> > Hi,
>> >
>> > I would like to know what are the potential issues with simple
>> > constructor chai
On 01/03/2015 01:38 PM, Peter Firmstone wrote:
> Hi,
>
> I would like to know what are the potential issues with simple
> constructor chaining where each constructor checks the invariant of its
> class state (with the already initialized state of superclass(es)).
Finalizer attack; a subclass c
> Hi,
>
> I would like to know what are the potential issues with simple
> constructor chaining where each constructor checks the invariant of its
> class state (with the already initialized state of superclass(es)).
Finalizer attack; a subclass can override the finalize method and
receive a t
Thanks Brian,
Those are good questions, some thoughts and examples inline:
- Original message -
> Overall the direction seems promising. Poking at it a bit...
>
> - ReadSerial methods are caller-sensitive and only show a class a view
> of its own fields.
> - Invariant checking i
On 01/02/2015 11:53 PM, Brian Goetz wrote:
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a
view of its own fields.
- Invariant checking is separate from deserialization, and does not
seem entirely built-in -- su
Overall the direction seems promising. Poking at it a bit...
- ReadSerial methods are caller-sensitive and only show a class a view
of its own fields.
- Invariant checking is separate from deserialization, and does not
seem entirely built-in -- subclass constructors seem responsible for
ask
Subclass example:
class SubFoo extends BaseFoo {
public static ReadSerial check(ReadSerial rs){
if (rs.getInt("y") > 128) throw Exception("too big");
return rs;
}
private final int y;
public SubFoo( int x , int y) {
super(x);
this.y = y;
}
Not quite, the constructor signature is the same for super and child classes.
ReadSerial is a container for each serialized Object in an ObjectInputStream
that provides and controlls access to serial fields, identified by name, type
and calling class, such that each class has it's own parameters
Just to note that there is some overlap between improving
serialization and the meta-model for Java (Beans v2.0) work I'm
looking at [1][2]. The key tool that a meta-model provides is to
enable class authors to select which pieces of their state form the
published properties. In most cases, the pro
So, if I understand this correctly, the way this would get used is:
class BaseFoo implements Serializable {
private final int x;
public BaseFoo(ReadSerial rs) {
this(rs.getInt("x"));
}
public BaseFoo(int x) {
this.x = x;
}
}
Right?
What happens with subclas
Is there any interest in developing an explicit API for Serialization?:
1. Use a public constructor signature with a single argument,
ReadSerialParameters (read only, writable only by the
serialization framework) to recreate objects, subclasses (when
permitted) call this first fr
89 matches
Mail list logo
