moonming commented on issue #13740: URL: https://github.com/apache/apisix/issues/13740#issuecomment-5068594055
Hi @MattJackson, thanks for doing this so carefully — and for reading the code instead of guessing. You've already got the key point right in `gateway.sh`: APISIX is a general-purpose API gateway, and the LLM support is the `ai-proxy` plugin running on top of it. So for APISIX your board is essentially measuring "general-purpose gateway + a plugin," which is a fair thing to call out. Your capability-matrix reasoning matches what's in the tree too (the single `anthropic → openai` converter, no native Gemini/Cohere provider at 3.17.0). If you'd like a purpose-built LLM-gateway entrant for a more like-for-like comparison, take a look at [api7/aisix](https://github.com/api7/aisix). It's an AI gateway with a Rust data plane, so protocol translation, streaming, and multi-provider routing are first-class rather than a plugin layer — much closer to what onthebench.ai is actually measuring. I'd be happy to help get its `gateway.sh` right, and I'm genuinely curious how it lands on the same rig. Disclosure, in the same spirit as yours: I started APISIX, and AISIX is built by our team at API7. Everything's open and re-runnable. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
