AlinsRan opened a new pull request, #13581:
URL: https://github.com/apache/apisix/pull/13581
### Description
Fixes #11244
**Root cause**
`core.response.hold_body_chunk` accumulated every response-body chunk into a
Lua table (`{chunk, n, bytes, ...}`) and, when the size limit was reached or on
`eof`, concatenated the whole table into one string with `table.concat`. For
large bodies (the issue reports ~128K), this keeps *both* the list of
individual chunks *and* the final concatenated string alive simultaneously, and
each `table.concat` allocates a brand new string. The result is excessive
allocation/retention that shows up as memory growth when loggers hold large
response bodies.
**Fix**
Rewrite the accumulation to use a LuaJIT `string.buffer`
(`require("string.buffer")`), reviving the approach from the abandoned #12035
and adapting it to the current `hold_body_chunk` shape:
- chunks are appended with `buffer:put(chunk)` instead of being stored in a
growing Lua table;
- the final body is produced with `buffer:get()` (full body) or
`buffer:get(max_resp_body_bytes)` (truncated), which returns exactly the first
`max_resp_body_bytes` bytes and avoids materializing an intermediate
concatenated copy before truncation;
- the wrapper table is kept only for the `bytes` counter and the `done` flag
so the buffer object can carry per-`buffer_key` state.
All existing semantics are preserved:
- the `hold_the_copy` flag (still flushes `arg[1]` on non-final chunks when
false);
- `max_resp_body_bytes` truncation (returns exactly `max_resp_body_bytes`);
- the `done` flag (later chunks and `eof` return `nil` after a truncated
body was already returned);
- the single-chunk-that-is-also-eof path;
- per-`body_buffer_key` / `ctx._plugin_name` storage.
**Test**
Adds `t/core/response-hold-body.t` covering:
1. multi-chunk accumulation of a 128K body (full body returned at eof);
2. truncation at `max_resp_body_bytes` across multiple chunks;
3. the `done` flag — later chunks and eof return `nil` after truncation;
4. a single chunk that is also eof, truncated to `max_resp_body_bytes`;
5. small-body single-chunk passthrough (mirrors the existing
`t/core/response.t` TEST 8 mock pattern).
**Verification**
- `luacheck apisix/core/response.lua` passes (0 warnings / 0 errors).
- All inline Lua blocks in the new `.t` parse cleanly.
- The rewritten function was exercised end-to-end via a standalone harness
that drives the same `ngx.arg[1]/[2]` chunk/eof protocol; all six scenarios
above pass and return byte-identical bodies (e.g. truncation returns exactly
131072 bytes).
- A memory micro-benchmark inside test-nginx is not reliable, so the
rationale is documented above; the `string.buffer` path avoids the simultaneous
chunk-table + concatenated-string retention that caused the growth.
> Note: the full `prove` test-nginx run was not executed in my local
environment because the available OpenResty build (1.21.4.4) does not support
the `quic`/`http3` listen directives that `t/APISIX.pm` emits unconditionally
on master; this is an environment limitation, not a test failure. CI runs
against a QUIC-capable OpenResty.
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [ ] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
