mochengqian opened a new pull request, #965:
URL: https://github.com/apache/dubbo-go-pixiu/pull/965
## Summary
Closes #956.
Every LLM request resolves a `cooldownKey` at least once (more on the
retry/fallback path) through `endpointCredentialHash`. That function ran
`fmt.Sprintf("%x", sum)` and allocated a fresh 64-byte hex string that escapes
to the heap. The hex string is pure overhead — `credentialHash` is only ever
used as a comparable component of the `cooldownKey` map key, never displayed or
logged.
This PR uses the raw `[32]byte` sha256 output directly as the key component.
A `[32]byte` array is comparable and valid in a struct map key, so the hex
encoding and its heap allocation are removed while the cooldown identity
semantics stay byte-for-byte identical.
`sha256` itself stays per-call (nanoseconds for short keys); the allocation
was the dominant cost. Caching the hash on the endpoint was explicitly **not**
done — it would pollute the model struct and complicate snapshot cloning for a
~200ns win, as the issue notes.
## Change
- `cooldownKey.credentialHash`: `string` → `[32]byte`
- `endpointCredentialHash(...)`: returns `[32]byte` (zero value for
no-credential endpoints), dropping `fmt.Sprintf`
`cooldownKey` is unexported, so this is a purely internal change with no
upstream-visible API impact.
## Benchmark
`BenchmarkCooldown_EndpointInCooldown` (added in this PR), 100 LLM endpoints
each with an API key, on the hot `endpointInCooldown` path:
| | ns/op | B/op | allocs/op |
|---|---|---|---|
| before | 300.0 | 168 | 6 |
| after | 163.2 | 40 | 3 |
3 fewer allocs/op and ~46% faster, exceeding the issue's "at least one fewer
alloc/op" target.
## Test plan
- [x] `go test -race ./pkg/filter/llm/proxy/...` — existing cooldown tests
pass unchanged with the `[32]byte` zero value
- [x] `go vet ./pkg/filter/llm/proxy/` clean
- [x] `go build ./...` clean
- [x] New `BenchmarkCooldown_EndpointInCooldown` documents the before/after
alloc delta
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
