lizining1231 opened a new issue, #3781: URL: https://github.com/apache/dubbo-go/issues/3781
### Summary Triple servers used to compress **every** response whenever the client negotiated compression, including 200 to 500 byte responses. For small payloads the fixed cost of compression dominates: each response initializes a deflate state with its dictionaries and hash tables, flushes a complete compressed stream, and the compressed size is often close to or larger than the original. On a 400 byte echo workload at concurrency 100, **33.5%** of server CPU landed in `compress/flate` and `compress/gzip` (`putCompressor` 24.0%, `getCompressor` 9.5%). This proposal adds a server response compression threshold: **responses whose serialized size is below the threshold skip compression, responses at or above it keep the negotiated compression**, with a default of 512 bytes. The threshold applies to every registered compressor, with gzip as the built-in one. Request decompression, client defaults and streaming behavior are unchanged. ### Relevant Code 1. [server.go](https://github.com/apache/dubbo-go/blob/develop/protocol/triple/server.go#L369-L417) (`resolveHandlerOptions`): resolves the threshold from the service parameter, then TripleConfig, then the 512 byte default, and fails export on negative values 2. [server.go](https://github.com/apache/dubbo-go/blob/develop/protocol/triple/server.go#L421-L445) (`serverCompressMinBytes` and `resolveServerSendMaxBytes`): resolves the send limit once and caps the threshold at that limit plus one 3. [triple_config.go](https://github.com/apache/dubbo-go/blob/develop/global/triple_config.go#L27-L33): adds the `ServerCompressMinBytes *int` field with the `server-compress-min-bytes` key, and `Clone` deep copies the pointer 4. [options.go](https://github.com/apache/dubbo-go/blob/develop/protocol/triple/options.go#L58-L71): adds the `triple.server-compress-min-bytes` service parameter and the `WithServerCompressMinBytes` option ### Proposed Change **1. Configuration surface and precedence** | Source | Form | Precedence | | --- | --- | --- | | Service parameter | `triple.server-compress-min-bytes=256` | Highest | | Protocol config | `server-compress-min-bytes: 256` in YAML or JSON, or `WithServerCompressMinBytes(256)` | Middle | | Default | Not configured | 512 bytes | Semantics: - Absent or `nil` uses the 512 byte default. - An explicit `0` compresses every response, which matches the historical behavior and serves as the rollback switch. - A negative value fails service export with an error that names the source (default, protocol config or service parameter) instead of silently falling back. **2. Safe cap against the send limit** A threshold above the server send limit (`MaxServerSendMsgSize`) creates a trap: a response may exceed the limit uncompressed but fit once compressed. Skipping compression under the threshold would then reject that response. The threshold is therefore capped at `sendMaxBytes + 1`, so only responses that cannot fit even when compressed skip compression. The send limit is resolved once in `resolveHandlerOptions` and reused for both `WithSendMaxBytes` and the cap, so the same configuration values are not parsed twice. **3. Reuse existing capability, no new protocol field** `compressMinBytes` already exists on both the gRPC and Triple encoding paths. This change only connects the server configuration to the existing option: no wire format change, no client change, no new switch, and no change to the compression negotiation flow. The authoritative signals stay the compressed flag on the gRPC envelope and `Content-Encoding` on Triple responses, so readability does not depend on the `Grpc-Encoding` response header. ### Test **Planned unit tests** | File | Test | Coverage | | --- | --- | --- | | `global/triple_compression_config_test.go` | `TestTripleCompressionConfigRoundTrip` | YAML and JSON round-trip with `absent`, `null`, `0` and `512`, plus pointer independence in `Clone` | | `protocol/triple/response_compression_config_test.go` | `TestResolveHandlerCompressMinBytesValidation` | Zero rollback, configured threshold, service zero, negative global and service values, malformed `1KiB` | | `protocol/triple/response_compression_default_test.go` | `TestServerDefaultCompressionWire` | No config, `nil`, empty config, default instance, protocol 0 and 1024, service 0, 256 and 512, both wires, both request gzip variants, sizes 0, 255, 256, 511, 512, 513, 1023 and 1024 | | `protocol/triple/response_compression_default_test.go` | `TestServerDefaultCompressionServiceIsolation` | A service-level zero leaves other services on the same listener untouched and resolution does not mutate shared config | | `protocol/triple/response_compression_safety_test.go` | `TestServerCompressMinBytes`, `TestResolveServerSendMaxBytes` | Cap function table including the `int` boundary; send limit precedence and invalid value fallback | | `protocol/triple/response_compression_safety_test.go` | `TestServerCompressionSendLimitSafetyWire` | 64 byte limit with thresholds 0, 256 and 2048; bodies of 63, 64, 65 and 128 bytes; both wires; request compression on and off | | `protocol/triple/response_compression_safety_test.go` | `TestServerCompressionSendLimitStillEnforced` | Incompressible content, explicit identity and missing gzip accept still return `resource_exhausted` | | `protocol/triple/response_compression_safety_test.go` | `TestServerCompressionSendLimitSafetyStream` | Mixed sizes inside one server stream, confirming the per message decision | | `protocol/triple/triple_protocol/compression_threshold_test.go` | `TestCompressionThresholdDefaultsUnchanged` | Low-level handler and client defaults stay zero, and explicit thresholds are preserved | | `protocol/triple/server_test.go` | `Test_buildServerHandlerOptions` (signature adaptation) | Handler option count changes from 6 to 5 once the send limit is resolved once | **CPU profile** <img width="1916" height="877" alt="Image" src="https://github.com/user-attachments/assets/828b43e8-8b5b-4bdd-87a5-bd6529d6dad5" /> ### Prior Art 1. Caddy: `minimum_length` on the `encode` directive, defaults to 512 bytes, matching this default 2. nginx: `gzip_min_length`, defaults to 20 bytes, decides on response length alone 3. connect-go: `WithCompressMinBytes`, defaults to 0, and dubbo-go already ports the same low-level option 4. gRPC-Go: no server side compression threshold, it provides registration and negotiation only ### References 1. Caddy encode directive: https://caddyserver.com/docs/caddyfile/directives/encode 2. nginx gzip module: https://nginx.org/en/docs/http/ngx_http_gzip_module.html#gzip_min_length 3. connect-go compression options: https://github.com/connectrpc/connect-go/blob/main/option.go 4. gRPC compression guide: https://grpc.io/docs/guides/compression/ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
