tigran90 opened a new pull request, #14052:
URL: https://github.com/apache/apisix/pull/14052
### Description
This PR adds optional ProtoJSON support to `grpc-transcode` and fixes
several issues in the existing transcoding implementation.
By default, `grpc-transcode` serializes values decoded by lua-protobuf,
which differs from the ProtoJSON format used by Go `protojson`, grpc-gateway,
and other gRPC transcoders.
The new `protojson` option enables standard JSON mapping for protobuf
messages, including well-known types, 64-bit integers, field names, and
presence handling.
**The option is disabled by default.** Existing routes keep their legacy
request and response mapping. Some error cases and invalid gRPC responses are
handled differently to avoid returning corrupted data or leaving protobuf state
inconsistent.
The changes are organized into four commits: codec, plugin integration and
stability fixes, tests, and documentation.
### Changes
**Reliability improvements**
- Restore `pb.state()` after errors during encoding and decoding. A Lua
error while mapping a request returns `400` instead of `500`.
- Reject requests with `503` when the proto descriptor cannot be loaded,
instead of forwarding them untranscoded.
- Reset protobuf options correctly when switching between routes, without
re-applying unchanged options or sorting the route configuration in place.
- Validate gRPC frames instead of blindly stripping the frame header.
Responses with several or compressed messages are aborted; streaming methods
that return a single message keep working. The `X-Grpc-Web` request header no
longer bypasses frame parsing; gRPC-Web clients should use the `grpc-web`
Plugin.
- Buffer responses up to `max_resp_body_size` without truncating protobuf
data. Oversized responses get `502` when the upstream `Content-Length` is
known; otherwise the downstream response is aborted.
- Abort malformed or empty successful responses instead of returning
misleading JSON with `200`.
- Handle repeated `Content-Type` headers instead of failing with a Lua
error, and avoid logging the compiled proto on every request.
**ProtoJSON support**
When `protojson: true` is enabled:
- Use protobuf JSON field names, including custom `json_name`; requests
accept both forms.
- Preserve 64-bit integers as decimal strings.
- Support standard enum, bytes, float, and double representations.
- Support well-known types such as `Timestamp`, `Duration`, `Struct`,
`Value`, `ListValue`, wrappers, and `Any`.
- Encode protobuf maps and handle field presence using descriptor metadata.
- Validate request values while allowing unknown fields for forward
compatibility.
- With `show_status_in_body`, return error bodies as JSON, with typed status
details in ProtoJSON.
The implementation also caches descriptor metadata and avoids unnecessary
allocations in the LuaJIT hot path.
**New attributes**
| Attribute | Default | Description |
|---|---|---|
| `protojson` | `false` | Enables the ProtoJSON mapping. |
| `max_req_body_size` | not set | Optional request-body limit; returns `413`
when exceeded. |
| `error_response_format` | `legacy` | `flat` returns `{"code": 5,
"message": "Not found"}` instead of `{"error": {"code": 5, "message": "Not
found"}}`. |
| `any_empty_format` | `spec` | `go` writes `"value": {}` for an `Any`
holding `Empty`, as Go `protojson` does. |
### Tests
Tested with Apache APISIX Test::Nginx:
```bash
FLUSH_ETCD=1 prove --timer -Itest-nginx/lib -I./ t/plugin/grpc-transcode*.t
```
**Result: PASS - 6 files, 262 tests.**
This includes the existing grpc-transcode tests, which pass without
modification, and the new ProtoJSON and regression tests.
Related admin, schema, and gRPC suites also passed. The changed Lua files
passed `luacheck` and `lj-releng`; test reindexing produced no changes.
### Performance
Benchmarked against APISIX `master` (`e14c6e93`) with the same
configuration: one APISIX worker, access logging disabled, local
`t/grpc_server_example` upstream, and `wrk` (2 threads, 32 connections, 5 s
warm-up, 20 s measurement). Master and this branch alternated between runs.
| Scenario | Master | Legacy mode | ProtoJSON |
|---|---:|---:|---:|
| Plus | 3,871 | 3,702 | 3,826 |
| Multi | 1,997 | 1,943 | 1,966 |
| Struct | 2,427 | 2,411 | 3,375 |
*Requests per second; median of three alternating runs.*
In a separate six-round comparison of the default mode (mean ± standard
deviation), results stayed close to `master`: Plus 3,828 ± 73 vs 3,760 ± 88
(−1.8%); Multi 1,913 ± 48 vs 1,954 ± 59 (+2.1%).
The higher throughput for `Struct` is primarily due to smaller ProtoJSON
payloads, so those numbers should not be interpreted as an isolated codec
speedup.
### Documentation
Updated the English and Chinese plugin documentation with the new options,
JSON mapping examples, error handling, and limitations.
### Compatibility notes
The legacy mapping remains the default. The main intentional behavioral
changes concern invalid inputs, malformed gRPC responses, response size limits,
and the old `X-Grpc-Web` handling.
ProtoJSON is opt-in and does not require existing routes to change their
configuration.
#### Which issue(s) this PR fixes:
None linked.
### Checklist
- [x] I have explained the need for this PR and the problem it solves
- [x] I have explained the changes or the new features added to this PR
- [x] I have added tests corresponding to this change
- [x] I have updated the documentation to reflect this change
- [x] I have verified that this change is backward compatible (If not,
please discuss on the [APISIX mailing
list](https://github.com/apache/apisix/tree/master#community) first)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
