weiqingy commented on issue #280: URL: https://github.com/apache/flink-agents/issues/280#issuecomment-4686943165
Thanks @wenjin272, and agreed on the final-output-only scope — I'll leave the intermediate react loop unconstrained and handle the agent wiring in a follow-up. I went through the two references you shared. Two points stood out that shape the API surface: 1. `with_structured_output()` in LangChain auto-selects between a model's native methods (tool/function calling and JSON/schema mode) with an explicit `method=` override. The prompt-and-parser path for models without native support sits *outside* that call as a separate layer — so I'd treat native auto-selection and the prompt fallback as two distinct layers rather than one seamless fallback chain. 2. LangGraph's `create_react_agent` applies `response_format` as a dedicated post-loop node (`generate_structured_response`) that makes one additional `with_structured_output` call over the final message history — it does not bind a format tool into the react loop. That cleanly avoids the "virtual format tool competing with real tools" concern raised earlier above, at the cost of one extra model call. Proposed split: - PR #1 (this issue): add native structured output at the chat-model / connection layer — auto-select the provider's native mechanism (tool calling or JSON/schema mode) with an explicit override, falling back to the existing prompt-engineered path for models without native support. Java + Python parity. - Follow-up: wire final-output structuring into `ReActAgent` as a post-loop step (mirroring LangGraph's dedicated node), so only the final output is constrained. For capability detection I'd keep it declarative — each integration statically declares which native methods it supports — rather than runtime probing, to avoid extra calls and keep it testable. Does this match what you had in mind? Happy to adjust the API surface before I open the PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
