AI read of your response below. It suggested a whole pile of changes
to make "on-by-default" work - I need some review time still, but it's
comments included below.
Cheers, Paul.
----
Your taxonomy is exactly the right cut, and it lines up almost
one-to-one with where
the implementation has landed, so let me answer against it concretely.
A1 (no free names) and A2 (lexical locals/params): both pack today, in
both modes.
A2 is the interesting one implementation-wise — captures are threaded
by value when
read-only and via the existing shared groovy.lang.Reference when
written, i.e. the
"reference based logic" Closure already uses, just passed as leading
parameters of the
hoisted method instead of constructor-injected fields. (Hardening
note: a read-only
capture that a NESTED closure re-captures must also be
holder-threaded, because the
nested closure class's constructor expects the shared Reference —
found and fixed by
brute force, see below.)
A3 (field/property names): agreed, and this is now enforced exactly as
you state it —
in fact you predicted a bug we then found empirically. We ran Groovy's
entire test
suite with packing forced on globally — core, tests, and everything
the tests evaluate
at runtime — precisely to find where theory and MOP reality diverge.
One of the finds
was an A3 hole: a bare name bound to an owner field is not a
DynamicVariable, so the
syntactic check was packing { foo + bar } against owner fields — and a later
DELEGATE_FIRST delegate was silently ignored (ClosureResolvingTest
caught it). Dynamic
mode now declines any field/property-bound or dynamic name (and
explicit this.x, which
routes through getProperty where EMC and Map-owner semantics live);
static mode packs
only with the type checker's owner-resolution proof — including treating names a
type-checking extension makeDynamic'd (mixed-mode builder DSLs) as unproven.
B1/B2: same split, same outcome — implicit-this calls decline
dynamically and pack
under the CS proof. You're right that "this" is captured already (the
hoisted method
is an instance method of the owner, so it IS this), and that
@DelegatesTo-targeted
calls are the open end — they currently decline in both modes, which
is the sound
default.
A4 / B2-with-delegate, "free names as inputs": I think this is more
within reach than
it first looks, because the machinery half-exists. The hoisted method
already takes
the captured environment as leading parameters — the dispatch tables pass
[receiver, captured..., args...] as plain parameters. Free names as
additional inputs
is the same shape: the compiler knows the free-name SET syntactically
(it's exactly
what the independence analysis collects today to decide declines), so
it could emit
them as parameters and leave the question of who supplies them to the
caller side.
Your "template" framing is apt — and the strict/proven flag the adapter already
carries is in effect a degenerate version of the adapter you describe
(strict = "no
one may supply meanings", proven = "statically shown empty").
On OpenClosure: I'd say the runtime we now have contains that split
behaviourally,
just not yet as structure. The adapter carries a latched
"MOP-pristine" bit; while
pristine, call() goes straight through the dispatch tables (your
"adapter is empty"
case — this is what static compilation gets unconditionally), and any
perturbation
(setMetaClass, categories) routes the call through the full MOP
instead. That latch is
exactly your adapter boundary, expressed as a branch rather than an
object. Reifying
it — one core that binds lexical names directly, one pluggable resolver for free
names, no owner/this/strategy triple — would be the honest
architecture for what the
runtime already does.
And the hardening campaign says the core can bear that weight. The
forced-global-flag
run started at 39 failing test classes (~140 failures) and is now down to ONE (a
single context-dependent case that passes in every standalone reproduction).
Everything in between was fixed by widening the mechanism, not narrowing it: the
adapter is now a small fixed-arity family (each member declares
exactly the doCall
signatures its generated-class counterpart would, so class-level
introspection — SAM
overload selection by closure arity, for instance — behaves
identically); argument
adaptation was pinned empirically against classed closures and
replicated exactly
(arity MMEs, single-List destructuring including Tuple0, vararg collection, the
metaclass coercion matrix, Closure-to-SAM); dehydrate()/rehydrate() keep packed
closures callable (dispatch holds its own receiver); and the
escape/serialization
gates grew two teeth the suite demanded (field initialisers are field
stores the old
walk couldn't see; a local passed to writeObject one hop later
declines). Tests that
assert the pre-pack bytecode shape now skip under the flag with named
reasons, and
the closure-serialization tests run against explicitly classed fixtures — so the
whole behavioural suite runs packed and green.
Which brings me to a retrospective I owe you on your ORIGINAL question
("in what way
are types a required deciding factor?"). My answer then — types only
matter to PROVE
the free-name subset; the trivial subset is sound by syntax — had the
right headline,
but doing the work revised it in two ways. First, "no free names" is a subtler
predicate than it reads: field-bound names, explicit this-properties, and
extension-forced dynamic calls under static compilation all turned out
to be free in
the sense that matters, exactly along the lines of your A3/A4. Second, and more
honestly: types were never the main event. The majority of the
engineering went to a
question orthogonal to types entirely — whether ONE adapter class can
impersonate
per-literal classes to a MOP that has assumed class-per-literal for
twenty years:
class-level arity introspection, method-selection semantics, argument
adaptation,
GString's parameter-count contract, serialization machinery, and
contexts the adapter
simply cannot inhabit (constructor special calls, trait helpers,
intersection casts).
So "can this be a PackedClosure?" is really two independent questions: is name
RESOLUTION sound (types-or-syntax, as we discussed), and can the
REPRESENTATION be
faithful (nothing to do with types). Your OpenClosure sketch is, I
think, the answer
to the second question stated as architecture: make the faithful core
the primitive
and the MOP a resolver at a defined boundary, instead of teaching a
shared class to
imitate many.
Concretely I'd suggest: the 6.0 slice stays as proposed (opt-in, the proven-safe
territory). Your OpenClosure sketch and the A4-template idea I'd like
to capture in
GEP-27 as the Groovy 7 direction — it slots naturally next to the
existing "unify the
writers" and "MethodClosure via table ids" items, and it gives default-on a
destination rather than just a flag flip. Want me to draft that
section, or would you
rather write it up in your own words first?
On Wed, Jul 15, 2026 at 9:46 PM Jochen Theodorou <[email protected]> wrote:
>
> On 7/15/26 08:39, Paul King wrote:
> > AI read below - I will push for further free names support.
> >
> > ----
> >
> > You're right and my sentence was sloppy. The real split is:
> >
> > * Closure with no free names (the { x -> x+1 } class): provably
> > delegate-independent
> > by inspection. Types irrelevant.
> > * Closure WITH a free name (foo(), or a bare bar): you have to know
> > whether that name
> > resolves to the owner (safe to bind at compile time) or could be a
> > delegate member
> > (unsafe). @CompileStatic's type checker resolves each one and
> > records where it landed
> > (the IMPLICIT_RECEIVER path) — so it can *prove* the answer
> > automatically. Dynamic
> > compilation doesn't resolve those names at compile time, so it
> > cannot prove it — which
> > is why the dynamic path uses the @PackedClosures trust assertion
> > plus the runtime
> > guard for that subset.
> >
> > That is the actual role of types: they let the compiler auto-*prove*
> > soundness for the
> > free-name subset. For the trivial subset there is nothing to prove.
>
> Let me express it like this:
>
> (A) variables
> (A1) no free variables like { x -> x+1 }. Easy, already mentioned
> (A2) x is from lexical scope of method. {x+1}, where x is a local
> variable or parameter. Here Closure already uses its reference based
> logic. These are also possible PackedClosure candidates.
> (A3) x is from field or property. Here only static compilation
> guarantees that the name "x" actually means the field or property and
> will not be intercepted or redirected by some MetaClass or Closure strategy
> (A4) x is not associated with a field or property, nor from the lexical
> scope of the method. x is then a truly free variable that gets its
> meaning only by setting a Closure strategy or by manipulating meta
> classes. If this is used as method argument, we can in static
> compilation still see if the matching method provides a delegatesTo or
> something like that to give the free variable a meaning after all.
> Otherwise it will fail compilation in static mode. Still works in
> dynamic mode. But yes, here only static compilation can find if that is
> a possible PackedClosure. But then it would still have to be called
> differently to actually work.
>
> For case A4 it would mean that the PackedClosure still could work, but
> needs the free variables as input. That deviates from the Closure idea
> and makes it more like a template - meaning the dynamic part is kept out
> of the construct and becomes part of the delegate or it becomes directly
> part of the call/doCall method. It is a question of how far we want to
> go here.
>
> But imagine we can split a Closure into something that binds the lexical
> names directly and then has a mechanism to deal with the free names
> only, then we have a concept Closure is actually using already today,
> but maybe we can migrate that concept and make the MOP part for only the
> resolution of the free variables. Then there would be only a single
> delegate, now owner and this, and no resolution strategy. All that would
> move out of the Closure. This new Closure could then become the core for
> the old Closure maybe. And maybe we would have a compiler option that
> will switch from the old to the new Closure... just ideas
>
> (B) methods
> Besides free names for variables/fields/properties/dynamic, there are
> also unqualified method calls. Here we have less variants
> (B1) dynamic mode: a method call is resolved according to the strategy
> and MOP, no direct binding. Example {foo()}, here foo is resolved
> dynamically and the meta class system goes to quite some lengths to try
> to resolve it.
> (B2) static mode: foo is resolved by the compiler, no MOP involved.
> PackedClosure could be possible. "this" is captured I think already,
> which means the more advanced cases where the parameter annotation
> determines the target of such a method call is maybe still open. Again
> this would probably require a delegate and thus maybe a different way to
> make the call. Similar to A4.
>
>
> To sum it up. My ambition is to make something like PackedClosure the
> core of modern Groovy. With a split of a static part and a dynamic part,
> with only one instance per Closure and no inner classes at all. With
> only one delegate, that is something like an MOP adapter, which takes
> care of the dynamic part. In case of static compilation that is empty.
> How exactly that MOP adapter looks like is not fully clear to me yet. I
> needs the relevant dynamic data and the metaclass. Just to give a
> temporary name let me call it OpenClosure (which is a bad name since it
> is kind of a contraction). A classic Closure would then consist of the
> OpenClosure and the adapter and inside reroute the calls to them. A
> PackedClosure would be almost identical to OpenClosure and can probably
> be replaced by it.
>
> bye Jochen
