wenjin272 opened a new issue, #1346:
URL: https://github.com/apache/flink-agents/issues/1346
### Search before asking
- [x] I searched the existing issues and did not find a duplicate of this
call-API problem. #1130 defines the terminal response event contract; this
issue concerns exposing that contract through `ctx.chat()`.
### Description
Java's `RunnerContext.chat()` currently returns `DurableFuture<ChatResult>`.
`ChatDurableFuture.await()` obtains the terminal `ChatResponseEvent` and calls
`response.getResponse()` before returning it.
This loses two capabilities of the event API:
- A terminal `ChatResponseEvent.failed(...)` is automatically converted into
`ChatResponseException`. The caller cannot inspect the failed response and
decide whether to continue, use a fallback, or throw.
- Structured output is stored on `ChatResponseEvent`, separately from
`ChatResult`. Call-style users cannot access `getStructuredOutput()`;
response-level metadata such as retry statistics is also unavailable.
This behavior already exists on main; it is not introduced by the
Future/gather refactoring in #1338.
#### Proposed direction
Return `DurableFuture<ChatResponseEvent>` from the Java chat API, and align
the Python call API with the same semantics. Awaiting a completed chat should
return its terminal response event, including ordinary chat-level failures,
without implicitly calling `getResponse()`.
For example:
```java
ChatResponseEvent response = ctx.chat(request).await();
if (response.isFailed()) {
// The application decides whether to fall back, continue, or throw.
} else {
ChatResult result = response.getResponse();
Object structuredOutput = response.getStructuredOutput();
}
```
Cancellation/interruption and framework faults must retain their exception
propagation semantics. Returning a failed chat response should not turn those
faults into ordinary results.
#### Acceptance criteria
- Successful and failed terminal chat responses are returned intact to
call-style users.
- Structured output and response metadata remain accessible, including after
durable replay.
- A failed response does not automatically throw while awaiting the chat
Future; explicitly accessing its success payload may retain the existing
throwing behavior.
- Java/Python APIs, bridges, callers, tests, and documentation are aligned,
with the return-type compatibility change documented.
Track this as a separate follow-up to #1338. It does not change retry policy
or define when failed outcomes should be persisted.
Related: #1086, #1130.
### How to reproduce
1. In a custom Java Action, call `ctx.chat(request).await()` for a request
with structured output configured. The returned `ChatResult` provides no access
to the parsed `ChatResponseEvent.getStructuredOutput()`.
2. Let a chat request finish with `ChatResponseEvent.failed(...)`. Awaiting
the call throws `ChatResponseException` instead of returning the failed event.
Source:
[ChatDurableFuture.await()](https://github.com/apache/flink-agents/blob/main/runtime/src/main/java/org/apache/flink/agents/runtime/chat/ChatDurableFuture.java),
[ChatResponseEvent](https://github.com/apache/flink-agents/blob/main/api/src/main/java/org/apache/flink/agents/api/event/ChatResponseEvent.java).
### Version and environment
Current main, Java call-style chat API (identified by source inspection).
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
