Hi Victor,
Some more thoughts, nothing concrete.
Class versioning and evolution, at least for our implementation needed
to be managed during construction, adding versioning flexibility into
schema is more challenging for unmarshalling since it provides an
attacker with choices. However schema flexibility is a mechanism
marshalling protocols use to handle class evolution, but I suspect at
some point in future, this will become a pain point or seen as a design
mistake and some other way to do versioning will be utilised, but I
could be wrong too, for now it seems to be standard practise and would
need support for that reason, I haven't given any design to whether an
Marshalling API can place constraints on schema, to allow client to
chose to discard unknown fields, place a limit on the number of unknown
fields, or bail out completely, but it's probably sound design if the
class developer has some control over serial form and its evolution,
rather than making it completely a schema responsibility / decision.
Relevant for schema security:
* Protocol Buffers — Field Presence
<https://protobuf.dev/programming-guides/field_presence/>
* Kirk McDonald — Unknown Fields in Protobuf
<https://kmcd.dev/posts/protobuf-unknown-fields/>
* CVE-2026-54270 — protobufjs memory amplification via unknown fields
<https://advisories.gitlab.com/npm/protobufjs/CVE-2026-54270/>
* CWE-230: Improper Handling of Missing Values
<https://cwe.mitre.org/data/definitions/230.html>
* CWE-232: Improper Handling of Undefined Values
<https://cwe.mitre.org/data/definitions/232.html>
In our system, we wire up both endpoints with the same classes, the same
class visibility with a deliberately inflexible schema that travels
prior to sending canonicalised introspect-able marshalled object bytes;
the constructors support evolution, the unmarshaller can read different
schema's and the serialization API allows each class to select
parameters during its construction, but any difference between the
schema and marshalled object bytes will cause an exception. It
supports serial form evolution, including refactoring from inheritance
to composition and inheritance hierarchies, where each class in an
object hierarchy has private namespace visibility, to avoid clashes with
parameter names, and encapsulation. It is extensible to other protocols
and has allowed us to migrate from the Java Serialization Protocol to
DER ASN.1, with minimal changes to serial form, the exceptions were
directly reading bytes from the stream via readObject, and circular
object graphs; they couldn't be supported for security reasons.
--
Regards,
Peter
On 31/08/2026 7:36 pm, Viktor Klang wrote:
Hi Peter,
>An SPI inferred from the stream format? Perhaps with an allow list
for stream formats?
No, I didn't mean inferring the format from the input, but rather the
reader deciding which format it is intending to read (up front).
>We're considering generating record classes dynamically from schema,
when they don't exist locally, to support existing implementations,
but with transparent protocols, it's not necessary as as the data can
be introspected.
I'd suspect class derivation from external transparent format has
questionable value as code to process it would have to be dynamic
anyway (since the static type cannot be known), so sort of equivalent
of programming with maps and lists.
On 2026-08-29 13:21, Peter Firmstone wrote:
Hi Victor, >OIS relies on the context ClassLoader, or a stack walk
when attempting to resolve class types. A string allow list is
simpler, but you might also consider permitting the OIS to be given a
specific ClassLoader for type resolution,
Hi Victor,
>OIS relies on the context ClassLoader, or a stack walk when
attempting to resolve class types. A string allow list is simpler,
but you might also consider permitting the OIS to be given a
specific ClassLoader for type resolution, since these are different
functions where a ClassLoader can perform the function of the allow
list, but perhaps shouldn't, but still be responsible for class type
resolution.
Given that Marshalling is envisioned to support use-site pluggable
input/output formats, we're having to consider the common
capabilities of formats to ensure the possibility of adapting
to/from each of them. Having implemented several such bridges over
the past couple of years it seems tractable. Of course there are a
multitude of different risks to account for and design around.
An SPI inferred from the stream format? Perhaps with an allow list
for stream formats?
String identity could be an attack surface for stream format
selection, an encrypted digest might be a better identifier, although
the risk is low, it might distinguish between versions, where the
versioning system is controlled by others? A developer may want to
constrain or limit not only stream formats, but versions or
implementations of the format.
It would be nice if there were hooks in there for checking for an
authenticated Subject with caller-constrained principals.
> We needed to support existing inheritance hierarchies of
Serializable objects, and we had code that is compiled separately
and comes together at runtime, so we couldn't rely on the compiler
checks for generic collections and built collection type validators
to address that.
Perhaps unsurprisingly, variants and especially dynamic type
restrictions tend to be challenging to encode in a non-cumbersome way.
We're considering generating record classes dynamically from schema,
when they don't exist locally, to support existing implementations,
but with transparent protocols, it's not necessary as as the data can
be introspected.
Cheers,
Peter.
On 2026-08-28 09:08, Peter Firmstone wrote:
Hi Victor,
Indeed, the ability to limit the types permitted to be loaded
(contextually to what is unmarshalled) is important to minimize
unmarshalling-associated risks. In my work on Marshalling I
currently rely on an allow-list (essentially String -> Class, as I
currently find ClassLoader to have a rather large API surface for
this purpose) since parsing a schema descriptor is resolving types.
Transparent protocols allow for more inspection than opaque
protocols, the latter limits the allow list to classes. Our
current wire protocol allows deep introspection, but I haven't
given this enough thought at this stage as to how an API might look
for a transparent protocol, vs opaque. OIS relies on the context
ClassLoader, or a stack walk when attempting to resolve class
types. A string allow list is simpler, but you might also consider
permitting the OIS to be given a specific ClassLoader for type
resolution, since these are different functions where a ClassLoader
can perform the function of the allow list, but perhaps shouldn't,
but still be responsible for class type resolution.
For schemaless payloads, the schema needs to be either known in
advance, or be determinable from the shape of the data itself.
I did consider this, and it really depends on the implementation,
but addressing transparency and deterministic behaviour is a
simplification that provides more certainty around security,
favouring the developer, or using inference which favours attackers
by providing options, similar to how a TLS protocol can be
downgraded to a less secure version. I didn't consider my
capability sufficient to implement the latter.
Current designs of Marshalling are essentially T -> record ->
output and input -> record -> T where the record serves both as
schema definition (using the record components as both names and
types are present at runtime). This encoding also permits
versioning (structure-as-version) as well as translation
(record-to-enum-value, record-to-cache-lookup, or equivalent).
This has the benefit of type safety, and since records don't
support inheritance hierarchies, it's an elegant solution. We
needed to support existing inheritance hierarchies of Serializable
objects, and we had code that is compiled separately and comes
together at runtime, so we couldn't rely on the compiler checks for
generic collections and built collection type validators to address
that.
I've been trialling AI since April this year. I've developed
standards I can't share on this list, since I've had AI agents
assist with their documentation, they're available for viewing on
GitHub and anything prior to April is AI free, and all AI
contributions are documented in git commits.
I've attached a text file that demonstrates the capabilities of the
serialisation protocol. I haven't invented a new protocol; it's
DER ASN.1, with canonical ordering rules for Collection interfaces
that allow for cross-language collection types.
JGDMS/JGDMS/examples/wire-protocol-showcase at trunk ·
pfirmstone/JGDMS
<https://urldefense.com/v3/__https://github.com/pfirmstone/JGDMS/tree/trunk/JGDMS/examples/wire-protocol-showcase__;!!ACWV5N9M2RV99hQ!MqgtZbcN1_ipspAsTMpGWItHkyKBYYoU81RUl-QOlbPQL73cCDYgHxbY-Z-qCq0VF3jZkQcFaOL6j_df-MuAp7b11Kuv_eA$>
Note the adversarial test code that produced the attachment was
written with AI assistance, genuinely attacking it to find
weaknesses in the protocol.
>We don't think of the object's serial form as serialised fields;
they are serialised parameter arguments used to create new objects.
That resonates with how I view it as well.
:)
Cheers,
Peter.
--
Cheers,
√
Viktor Klang
Software Architect, Java Platform Group
Oracle
--
Cheers,
√
Viktor Klang
Software Architect, Java Platform Group
Oracle
