ruanweihong-aaa commented on PR #13797: URL: https://github.com/apache/apisix/pull/13797#issuecomment-6356400657
Thanks for working on this. The same issue (#13769) has already been addressed by #13793, merged as 5c5e1b2cb54564d817cbd27f18b223678a792a39 and included in both 3.18.0 and 3.19.0. The merged implementation deep-copies the cached request body inside the retry loop, so each attempt starts from the original client body rather than the previous instance's modified table: https://github.com/apache/apisix/blob/5c5e1b2cb54564d817cbd27f18b223678a792a39/apisix/plugins/ai-proxy/base.lua#L179-L190 The client model is already captured by `detect_request_type` in the access phase: https://github.com/apache/apisix/blob/5c5e1b2cb54564d817cbd27f18b223678a792a39/apisix/plugins/ai-proxy/base.lua#L147-L150 #13793 also removed the redundant `request_llm_model` reassignment from `before_proxy`. The per-attempt model is now recorded only in `llm_model` at this point: https://github.com/apache/apisix/blob/5c5e1b2cb54564d817cbd27f18b223678a792a39/apisix/plugins/ai-proxy/base.lua#L245-L250 [The added regression test](https://github.com/apache/apisix/blob/5c5e1b2cb54564d817cbd27f18b223678a792a39/t/plugin/ai-proxy-multi-retry.t#L287-L309) checks that fallback retains `request_llm_model=client-model` while `llm_model=upstream-model-B`, and that instance A's `temperature` does not leak into instance B. Could we close this PR as superseded by #13793? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
