shreemaan-abhishek opened a new pull request, #13654:
URL: https://github.com/apache/apisix/pull/13654
### Description
`ai-protocols`' `get_messages()` flattens structured message content
(images, tool calls) down to plain text. That silently broke `ai-cache`
fidelity at **both** cache layers:
1. **L1 exact key** — `key.fingerprint` hashes the `get_messages` output.
After flattening, a `[{text}, {image_url}]` prompt and a plain-text prompt
carrying the same text collapse to the identical string, so they collide on one
L1 key and the multimodal request is wrongly served the text-only response.
2. **L2 semantic bypass** — `window_has_nontext` only tripped on
`type(content) == "table"`, which no longer exists after flattening, so
multimodal prompts stopped bypassing the semantic layer and could hit a
same-text vector.
### Fix
- `key.lua` — fold the raw `body.messages` into the L1 fingerprint so a
text+image prompt stays distinct from a text-only one. (`params` already
preserves every other body field verbatim, so `input`-based protocols were
never affected; only `messages` lost fidelity.)
- `semantic.lua` — replace `window_has_nontext` with `body_has_nontext`,
which scans the **raw** body and bypasses L2 whenever any block is non-text. It
also guards non-table message items and `content` shaped as a single block
object.
### Why it looked flaky
`t/plugin/ai-cache-semantic.t` TEST 64 asserts a multimodal prompt is a
MISS. Whether the collision surfaces depends on TEST 63's async log-phase L2
write-back racing TEST 64's read (`--- wait: 0.5`), so it reproduces
intermittently. It reproduces deterministically against a local RediSearch.
### Tests
- `t/plugin/ai-cache-semantic.t` — TEST 64 (end-to-end MISS) now passes;
added TEST 66 unit-testing the malformed-input handling. Full file green (215
tests), luacheck clean.
Checklist:
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change (N/A: internal
bugfix, no behavior/schema change)
- [x] I have verified that this change is backward compatible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
