wenjin272 opened a new pull request, #1305: URL: https://github.com/apache/flink-agents/pull/1305
Linked issues: #939, #1234, #1235 ### Purpose of change Close five gaps found while validating the Operations guide for 0.4: reject the unsupported batch state backend even when Flink resolves the mode later, attribute child Agent Tool metrics correctly, count failures from the default SLF4J file output, clarify token metric conditions, and accept the documented uppercase logger values in Python. #### Runtime flow - Operator state initialization checks the actual keyed-state backend before opening Agent resources or recovering Actions. The existing graph-construction check still rejects explicit BATCH configurations early. - Each Action execution binds its component metric listener to its own resource cache. Tool registration follows the child-to-parent lookup chain without instantiating resources. - The automatically created Log4j2 FileAppender propagates write errors to the existing TraceLogWriter handler, which counts and absorbs them. - Python LoggerType resolves unmatched string values case-insensitively; configuration reads continue to use the existing typed conversion. #### Key decisions Validate the resolved backend because checking the configured execution mode alone misses AUTOMATIC and later configuration changes. Bind Tool lookup when creating the listener because asynchronous reports can arrive after the shared runner context switches Agents. Retain existing Action/Tool metric names and aggregation rather than adding an Agent dimension. Keep lowercase Python enum values to preserve serialization compatibility. ### Behavioral Semantics The following describes the runtime and Python changes; token accounting itself is unchanged and receives only a documentation correction. #### Interaction decisions | Conditions | Result | | --- | --- | | Explicit BATCH + batch state backend enabled at graph construction | Existing early rejection remains. | | Bounded AUTOMATIC, or mode changed after connecting the Agent + actual batch state backend | Initialization fails before Agent inputs are processed. | | STREAMING, unbounded AUTOMATIC, or batch backend disabled | This backend guard permits execution; this is not a claim of full BATCH support. | | Tool registered locally or inherited in the calling Action's scope | Attribute metrics to that Tool name. | | Tool registered only in a different scope | Attribute the unresolved call to unknown. | | Same Action and Tool names in parent and child | Share counters; resource selection remains scoped to the caller. | #### Behavioral contracts 1. Unsupported batch state backends cannot bypass the guard through deferred mode resolution or later configuration changes. 2. The supported test workflows retain all outputs with compatible backends. 3. Child-local and inherited Tools receive named metrics; missing Tools remain unknown in the caller's scope. 4. Equal Action and Tool names continue to aggregate within one operator. 5. Detected default-appender write failures each increment traceLogWriteFailures without escaping into Event processing; the first diagnostic is WARN and subsequent diagnostics are DEBUG. 6. Uppercase, lowercase and mixed-case FILE/SLF4J values load through Python's typed configuration API; enum serialization remains lowercase. 7. Invalid Python logger values remain errors. #### Failure behavior The backend guard raises with STREAMING/backend configuration guidance. Trace writes remain best effort, without a new retry policy; backend-swallowed errors remain unobservable. Unknown Tool names retain existing failure handling and unknown metric attribution. Invalid logger strings and unsupported non-string values still raise ValueError through the configuration API. ### Tests | Contract | Coverage | | --- | --- | | 1 | CompileUtilsTest: rejectsExplicitBatchModeWithDefaultBatchStateBackend; rejectsBatchStateBackendAtRuntime | | 2 | CompileUtilsTest: processesAllKeysWithCompatibleStateBackend; allowsAutomaticModeWithUnboundedSource | | 3 | ReActAgentSubagentTest: toolCallLoopRunsInsideTheChildScope (local/inherited); missingToolsRemainUnknownInTheCallingScope | | 4 | ReActAgentSubagentTest: parentAndChildToolsWithTheSameNameShareMetrics | | 5 | Slf4jTraceLoggerFailureTest: fileWriteFailuresAreCountedWithoutEscapingAndWarnOnlyOnce (fault injection and recovery) | | 6 | test_trace_log_output_type_from_yaml (six spellings, YAML read and JSON round trip) | | 7 | test_trace_log_output_type_rejects_invalid_values | Validation on the PR head: JDK 11 targeted suite: 161 passed, 7 JDK-21-only tests skipped; JDK 21 metrics/resource/sub-agent/recovery suite: 39 passed, none skipped. Python configuration suite: 49 passed. Ruff and diff checks passed. Hugo 0.110.0 built successfully; all 69 internal links/anchors on Deployment and Monitoring passed. Coverage includes real MiniCluster mode resolution, operator-level sub-agent execution, real Log4j2 appender fault injection, and Python configuration parsing. Not verified: the full repository suite, remote cluster deployment, live LLM services, or comprehensive BATCH compatibility. <details> <summary>Implementation invariants and reproduction commands</summary> The listener-specific recorders share call timing state, so normal Action terminal records still clean up unfinished calls; BuiltInMetricsTest.actionTerminalDropsOnlyItsUnpairedChildLatencyState covers both listener paths. Resource lookup does not create Tools. The appender test restores global logger configuration and verifies successful writes after recovery. Java: `mvn --batch-mode --no-transfer-progress -pl runtime -am test '-Dtest=CompileUtilsTest,**/metrics/*Test,ReActAgentSubagentTest,**/tracelog/*Test' -Dsurefire.failIfNoSpecifiedTests=false` Python: `python -m pytest flink_agents/plan/tests/test_configuration.py -q`, from python/ with PYTHONPATH set to the active environment's purelib. </details> ### API No existing public signatures are removed or changed. Python LoggerType accepts more spellings while keeping its existing values and defaults. Runtime metric names remain stable, but child Tool calls previously counted as unknown now appear under their names. Previously accepted unsupported jobs now fail explicitly during initialization. ### Documentation Deployment explains default STREAMING and the backend restriction. Monitoring describes Tool naming/aggregation, observable Trace Log failures, and the chat flow and response fields required for token metrics. - [ ] `doc-needed` - [ ] `doc-not-needed` - [x] `doc-included` ### Was this patch authored or co-authored using generative AI tooling? - [x] Yes - [ ] No Generated-by: Codex CLI 0.153.4 (gpt-6-astra) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
