[
https://issues.apache.org/jira/browse/GROOVY-12365?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18112491#comment-18112491
]
Paul King commented on GROOVY-12365:
------------------------------------
Future work beyond GraalVM, recorded here so it is not lost when this issue
closes.
Everything in this issue targets GraalVM's {{native-image}}, and it applies
unchanged to every distribution of it (Oracle GraalVM, GraalVM Community,
Mandrel, Liberica NIK, and the WebAssembly backend): they all read the same
{{META-INF/native-image}} files. For other ahead-of-time and snapshot
technologies the picture is that the *design* transfers and the *files* do not.
||Target||What applies||What does not||
|OpenJDK Leyden AOT cache (JDK 24+)|The reflection-free shapes: adapters
constructed rather than looked up by name, the VM plugin as a constructor
reference, every remaining reflective site enumerated by family. Those are what
a training run can archive.|The metadata itself. HotSpot prunes nothing, so
there is nothing to declare.|
|CRaC checkpoint/restore|Nothing from this issue is needed.|Its concern is live
state: threads, executors, open files.|
|Android via R8/ProGuard|The knowledge: the list of what the runtime reflects
over is exactly a keep-rule list.|The file format.|
|AppCDS, jlink, jpackage|Indifferent to all of it.| |
What made the runtime work in a native image was making it legible to static
analysis, and that is the property every one of these wants. The one rule worth
keeping for new runtime code is "no lookup by name inside the runtime";
{{tests-native}}'s {{checkNativeMetadata}} enforces it regardless of which tool
eventually consumes the result.
h3. Suggested follow-ups, in priority order
# *Android: emit ProGuard/R8 keep rules from the same generator.*
{{NativeImageMetadataGenerator}} already knows every class the runtime reflects
over and why. Writing the same families as a rules file into
{{META-INF/proguard/}} in the jar gives R8 users automatic keep rules the way
the metadata gives {{native-image}} users automatic registration; Android
tooling consumes that directory on its own. The adapter change already helps
shrinking, since {{DgmProxyFactory}} references every adapter statically and no
keep rule is needed for them, and an {{-assumevalues}} rule on
{{DgmProxyFactoryConfig.ENABLED}} is the R8 analogue of the
{{groovy.dgm.factory}} build-time switch. Small and mechanical; the most
directly reusable piece.
# *Leyden: a measurement pass.* Groovy has a specific exposure: Leyden archives
resolved {{invokedynamic}} sites only for the JDK's own bootstraps (lambdas,
string concatenation), so call sites bootstrapped by {{IndyInterface}} re-link
on every start even from a warm cache. A training run of the {{tests-native}}
probe on JDK 25 with {{-XX:AOTCache}} would show how much class loading and
linking is recovered, whether anything in the bootstrap misbehaves during
training, and whether the gap motivates work of a different kind (there is no
metadata to write).
# *CRaC: a checklist run.* Groovy holds live state a snapshot must survive: the
async runtime's virtual-thread executor and default pool, the reference-cleanup
threads behind {{ClassInfo}}, Grape's file locks when used. A
checkpoint-and-restore of the probe answers whether restore reinitialises or
resumes them. Any hook would sit behind the existing platform detection, since
core cannot depend on the CRaC API.
# *{{groovy-concurrent-java}}.* The async family (the executor and scoped-value
lookups) is registered in the core jar under conditions on classes that live in
the concurrent module, so a pure-Java user of that module in a native image
does not get it. A copy of that family into the concurrent-java jar closes the
gap.
# *Module coverage in the in-repo probe.* {{tests-native}} exercises the core
only; the modules' metadata is currently checked by the MCP server joint
validation alone. A NIO or XML call in the probe would give the modules a
per-PR guard, at the price of the probe subproject depending on them.
None of these blocks the current change.
> native: Ship Groovy's own reachability metadata in the groovy jar
> -----------------------------------------------------------------
>
> Key: GROOVY-12365
> URL: https://issues.apache.org/jira/browse/GROOVY-12365
> Project: Groovy
> Issue Type: Improvement
> Reporter: Paul King
> Priority: Major
>
> Groovy 6 dynamic code runs in a GraalVM native image thanks to the AOT link
> mode (GROOVY-12234), but building the image still requires every user to run
> the {{native-image-agent}} first, purely to capture the reflection and
> resource access Groovy's *own* runtime performs. The Iris example README says
> it plainly: without the recorded metadata the image builds and then fails
> initialising {{GroovySystem}}. Logging frameworks and most libraries solved
> this years ago by shipping their reachability metadata inside the jar
> (log4j-core carries
> {{META-INF/native-image/org.apache.logging.log4j/log4j-core/reflect-config.json}}
> and {{resource-config.json}}); {{native-image}} picks such files up
> automatically from the class path, and the GraalVM Gradle/Maven plugins do
> too. Groovy ships nothing, and the central [GraalVM reachability-metadata
> repository|https://github.com/oracle/graalvm-reachability-metadata] has no
> entry for {{org.apache.groovy:groovy}} either (its {{groovy-all}} entry is
> marked not-for-native-image and points at a {{groovy}} entry that does not
> exist), so plugin users get nothing there as well.
> h3. What the agent records for Groovy itself
> Two unrelated dynamic applications (the Iris deep-learning example and a
> small logging probe, both on GraalVM CE 25.2.4) produce almost the same
> Groovy-owned set: about 180 reflection entries and a handful of resources.
> * Runtime bootstrap types under {{groovy.lang}},
> {{org.codehaus.groovy.runtime}}, {{org.codehaus.groovy.reflection}},
> {{org.codehaus.groovy.vmplugin}}: {{GroovySystem}},
> {{MetaClassRegistryImpl}}, {{Closure}}, {{GString}}, {{GroovyObjectSupport}},
> {{ExpandoMetaClass}}, ranges, {{NullObject}}, the VM plugin, etc.
> * The {{Introspector}} probes the metaclass performs for every class it
> introspects: {{<Type>BeanInfo}} and {{<Type>Customizer}} lookups for each
> Groovy runtime type above (these are lookups of classes that do not exist;
> they still need to be registered so the lookup fails quietly rather than with
> {{MissingReflectionRegistrationError}}).
> * {{org.codehaus.groovy.runtime.dgm$N}} proxy classes (30 to 32 per
> application), loaded by name from {{GeneratedMetaMethod$Proxy.createProxy}}
> when a DGM method is first selected. This set is application dependent: a DGM
> method not exercised during the agent run fails at run time with
> {{ClassNotFoundException: org.codehaus.groovy.runtime.dgm$1175}} (seen when a
> code path was added after the agent run).
> * Resources: {{META-INF/dgminfo}},
> {{META-INF/groovy/org.codehaus.groovy.runtime.ExtensionModule}},
> {{META-INF/services/org.codehaus.groovy.runtime.ExtensionModule}}, plus a
> {{.class}} resource lookup ({{groovy/concurrent/AsyncScope.class}}).
> h3. Proposal
> Ship the Groovy-owned part of that metadata in the {{groovy}} jar under
> {{META-INF/native-image/org.apache.groovy/groovy/}} (and the equivalent for
> modules that need it, e.g. {{groovy-json}}, {{groovy-xml}}), following the
> log4j-core layout, so that a dynamic Groovy application builds with plain
> {{native-image -cp ...}} and no agent step:
> * {{reflect-config.json}} / {{reachability-metadata.json}} with the bootstrap
> types, the {{BeanInfo}} / {{Customizer}} probe names, and the DGM proxies.
> For the proxies the safe choice is to register all of them (about 2,000 small
> classes, all of which are in the jar anyway) rather than a sample, since the
> used set cannot be known ahead of time; alternatively
> {{GeneratedMetaMethod$Proxy}} could stop resolving them by name (e.g. a
> generated switch or a {{MethodHandle}} table), which would remove the entries
> entirely.
> * {{resource-config.json}} for {{META-INF/dgminfo}}, the extension-module
> descriptors and the service files.
> * A {{native-image.properties}} with {{Args}} carrying what the README
> currently asks users to type by hand and what the AOT link mode assumes:
> {{--initialize-at-run-time=org.codehaus.groovy.vmplugin.v8.IndyInterface}}
> (and the other runtime classes that must not be initialised at build time).
> This file is also the one place that can set a build-time system property for
> everybody, which matters for the JUL caller-location workaround documented
> under GROOVY-12354 ({{-Djdk.logger.packages=...}} is captured by a build-time
> static initialiser and ignored at run time).
> The metadata should be generated by the build rather than hand-maintained:
> run the agent over the existing native-image parity corpus (or a small
> dedicated script) as part of the performance/native tests, diff against the
> checked-in files, and fail on drift. Once the jar ships it, an entry in the
> reachability-metadata repository becomes unnecessary (the repository prefers
> jar-embedded metadata), but its dangling {{groovy-all}} replacement should be
> corrected or an {{org.apache.groovy:groovy}} entry added pointing at the
> shipped files.
> h3. Out of scope
> Caller-location accuracy in native images (GraalVM's visible
> MethodHandle-interpreter frames) is tracked under GROOVY-12354's
> documentation and the GraalVM issue; the fatal dynamic call to a
> {{@CallerSensitive}} method and the failing MOP {{super}} call in native
> images have their own issues.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
