weiqingy commented on issue #912: URL: https://github.com/apache/flink-agents/issues/912#issuecomment-5467100686
Recording a documentation requirement that came out of the Ollama work in #981, so it is not lost between there and here. `supportsNativeStructuredOutput()` reports capable for every model on that connection, and no capability check can improve on that. Ollama applies the grammar in the server's sampler rather than the model, and neither `/api/show` nor `/api/version` exposes a schema-related signal to key on. Three deployments break the guarantee, and none of them is distinguishable at request-build time: - A server below v0.5.0 rejects `format` with HTTP 400. - Ollama Cloud accepts the request without enforcing the schema. - The MLX runner accepts the field and discards it. The last two return an unconstrained response and no error, so a caller cannot tell the difference from a successful constrained call. That makes this a documentation and fallback concern rather than a detection one. Once the strategy resolution and fallback behavior in this issue are in place, the user-facing documentation should cover the supported server versions, the known Cloud and MLX limitations, and what the fallback actually does when the native path is unavailable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
