[
https://issues.apache.org/jira/browse/GROOVY-12711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18139694#comment-18139694
]
ASF GitHub Bot commented on GROOVY-12711:
-----------------------------------------
daniellansun opened a new pull request, #3242:
URL: https://github.com/apache/groovy/pull/3242
## Summary
Joint compilation can keep javac `CLASS_OUTPUT` in memory when `memStub` is
enabled ([GROOVY-12711](https://issues.apache.org/jira/browse/GROOVY-12711)).
`InMemoryJavaFileManager` wraps a standard file manager and stores class
files on the heap. The new joint option `memClassOutput`
(`CompilerConfiguration.MEM_CLASS_OUTPUT`) takes effect only together with
`memStub`. In that mode `JavaAwareCompilationUnit` does not require `stubDir`
or a filesystem target directory: Groovy resolves the javac bytes from memory,
and Groovy's own class files are published to the same class loader instead of
being written to disk.
Disk `stubDir` and target-directory output stay the default when the option
is off. `groovy.mem.stub` still enables in-memory stubs only.
New API is marked `@since 7.0.0`, matching this line (`7.0.0-SNAPSHOT`).
## Test plan
- `./gradlew :test --tests
org.codehaus.groovy.tools.javac.MemClassOutputTest`
- `./gradlew :test --tests org.codehaus.groovy.classgen.InterfaceTest`
- `./gradlew checkstyleGate codenarcGate
:binary-compatibility:checkBinaryCompatibility`
> Joint compilation: keep javac CLASS_OUTPUT in memory when memStub is enabled
> ----------------------------------------------------------------------------
>
> Key: GROOVY-12711
> URL: https://issues.apache.org/jira/browse/GROOVY-12711
> Project: Groovy
> Issue Type: Improvement
> Components: Compiler
> Reporter: Daniel Sun
> Priority: Major
>
> h2. Background
> Apache Groovy's joint compilation pipeline already supports generating Java
> stubs in memory via {{CompilerConfiguration.MEM_STUB}} ({{"memStub"}}) and
> {{MemJavaFileObject}} (since 3.0.0). {{JavaAwareCompilationUnit}} emits those
> stubs at {{Phases.CONVERSION}} (including GEP-21 CONVERSION-phase
> transform/stubber contributions) and invokes javac at the
> {{SEMANTIC_ANALYSIS}} boundary.
> What remains disk-bound is javac *class output*: {{JavacJavaCompiler}} /
> consumers such as {{groovy-lsp}} still rely on a filesystem target directory
> (or an owner-only temporary directory) for {{CLASS_OUTPUT}}. Embeddings that
> need a clean, low-latency, no-workspace-pollution joint compile—language
> services, tests, and future agent tooling—therefore cannot yet run a fully
> heap-resident Groovy↔Java cycle through the public core APIs alone.
> This ticket is intentionally *not* a request to add an MCP server or LSP host
> to groovy-core. Optional modules {{groovy-lsp}} and {{groovy-mcp}} already
> provide protocol adapters over {{GroovyLanguageEngine}}. Core should supply
> reusable joint-compilation primitives those modules (and third parties) can
> adopt later.
> h2. Motivation
> * Agents and IDEs that patch Groovy and Java together need joint validation
> without writing stubs or {{.class}} files into the project tree or into temp
> directories that must be permission-hardened and cleaned up.
> * Disk stub/class I/O adds latency, races with file watchers, and
> false-positive failures when overlays disagree with on-disk sources.
> * Completing in-memory *class* output—on top of existing in-memory *stub*
> sources—closes the remaining gap in the joint pipeline and unblocks follow-on
> introspection work without widening the core attack surface.
> h2. Goals
> * Provide a reusable in-memory {{javax.tools.JavaFileManager}} (wrapping
> {{StandardJavaFileManager}} as needed) that stores javac {{CLASS_OUTPUT}}
> bytes in heap.
> * Integrate it into the joint-compilation path behind an *explicit opt-in*
> when {{memStub}} is enabled (recommended joint option constant alongside
> {{MEM_STUB}}, e.g. {{MEM_CLASS_OUTPUT}} / {{"memClassOutput"}}—final name
> bikeshed OK if documented).
> * Allow {{JavaAwareCompilationUnit}} + {{JavacJavaCompiler}} to complete a
> mixed Groovy/Java compile *without* requiring {{stubDir}} or a filesystem
> {{targetDirectory}} for class output when that opt-in is active.
> * Preserve today's disk-based {{stubDir}} / target-directory behavior as the
> default for Ant/Gradle-style joint builds.
> * Document the option next to {{MEM_STUB}}; mark new APIs {{@since 6.0.0}}
> (or the assigned 6.0.x).
> * Add focused unit tests proving a bidirectional Groovy↔Java reference
> compiles successfully with no stub or class files on disk, plus a regression
> that the classic disk path still works.
> h2. Non-Goals
> * No MCP JSON-RPC tools, daemon, or dependency on an MCP SDK in core.
> * No LSP protocol changes and no requirement to modify {{groovy-lsp}} /
> {{groovy-mcp}} in the same change set (follow-up tickets may consume the API).
> * No Overlay VFS process, no multi-tenant sandbox service.
> * No silent forced {{StaticTypeCheckingVisitor}} on dynamic Groovy.
> * No change to default joint-compilation outputs for existing builds that do
> not opt in.
> * No attempt to run all AST transforms before stub emission (GEP-21 stubbers
> remain the mechanism for pre-javac signature contributions).
> * No "zero hallucination" product claims; success is measured by tests,
> compatibility, and API clarity.
> h2. Proposed Approach
> # Add {{InMemoryJavaFileManager}} under {{org.codehaus.groovy.tools.javac}}
> that forwards to the standard manager for inputs/classpath but redirects
> {{getJavaFileForOutput(..., CLASS_OUTPUT, ...)}} to an in-memory
> {{JavaFileObject}} (e.g. {{MemClassFileObject}}), exposing produced bytecode
> by binary name.
> # Teach {{JavacJavaCompiler}} to install that manager when {{memStub}} is
> true *and* the new mem-class-output option is enabled (and/or when no target
> directory is configured—behavior must be documented and tested).
> # Adjust {{JavaAwareCompilationUnit}} configuration so the Groovy class
> loader can resolve types from the in-memory class bytes after javac without
> requiring {{targetDirectory}} on disk in that mode. Prefer the smallest
> robust hook; if a dedicated {{ClassNodeResolver}} helper is too large for one
> review, land bytecode retrieval + tested {{defineClass}}/loader integration
> first and file a follow-up for resolver sugar.
> # Keep {{MemJavaFileObject}} stub generation as-is; this ticket does not
> replace GEP-21.
> # Update package documentation for joint compilation accordingly.
> h2. Acceptance Criteria
> * Standards: Uses public {{javax.tools}} APIs only (no illegal reflective
> access / {{--add-exports}} requirements beyond what Groovy already needs for
> joint compile).
> * Compatibility: Existing {{stubDir}} joint compiles without the new option
> behave as today (regression test).
> * Functional: With {{memStub}} + mem class output, a fixture where Java
> calls a Groovy stubbed API and Groovy references a Java type compiles
> successfully with no stub directory and no class-output directory.
> * Maintainability: New types live in {{org.codehaus.groovy.tools.javac}},
> are documented, and do not entangle protocol modules.
> * Reusability: File manager API is usable by embeddings outside Ant (tests
> demonstrate construction without {{FileSystemCompiler}}).
> * Robustness: Compiler errors surface through existing
> collectors/diagnostics; failures do not leave orphan temp dirs (because none
> are required in the mem path).
> * Performance: Small fixture completes reliably under CI; mem path must not
> be pathologically slower than disk path for that fixture (guard with a
> generous timeout rather than a flaky microbenchmark).
> * Security: Output path does not write class bytes to the filesystem; does
> not introduce network I/O, script evaluation, or Grab; does not expand
> classpath resolution beyond existing joint-compile behavior. Vuln/scanner
> surface must not increase.
> h2. Risks / Compatibility
> * Binary compatibility: additive APIs only; run the project's
> binary-compatibility checks.
> * Class loader / resolution ordering: in-memory classes must become visible
> to Groovy resolution the same way disk {{targetDirectory}} classes are today;
> mishandling could break joint cycles—tests must cover the bidirectional case.
> * Opt-in flag vs implied behavior when {{targetDirectory == null}}: choose
> one documented rule to avoid surprising Ant users.
> * Follow-on modules (LSP) currently create owner-only temp dirs; migrating
> them is *out of scope* here but should become easier once this lands.
> h2. Test Plan
> * New unit/integration tests under the existing joint-compilation /
> tools.javac test areas:
> ** memStub + mem class output: Groovy↔Java cycle, assert success, assert no
> stubDir/class output dir required.
> ** intentional type error: failure reported cleanly.
> ** disk stubDir path unchanged when option off.
> * Run standard core test task(s) and binary-compatibility checks required
> for the touched modules.
> h2. Notes for implementers / reviewers
> * Related in-tree pieces: {{MemJavaFileObject}},
> {{JavaStubGenerator#generateMemStub}}, {{CompilerConfiguration.MEM_STUB}},
> {{JavaAwareCompilationUnit}}, GEP-21 {{StubberSupport}},
> {{subprojects/groovy-lsp}} ({{LspJavaCompiler}}, temp CLASS_OUTPUT),
> {{subprojects/groovy-mcp}} (protocol only—do not expand in this ticket).
> * Long-term "Semantic Oracle" agent tools (FQN contract inspect, multi-file
> dry-run, diagnostic enrichment) depend on *this* foundation plus later
> library introspection APIs; track those as separate issues after this
> Improvement merges.
--
This message was sent by Atlassian Jira
(v10.3.26#10030026)
