AlinsRan opened a new pull request, #13958:
URL: https://github.com/apache/apisix/pull/13958
Follows #13941, which covered the general data plane capability. This one
teaches it about Kubernetes, and settles the naming.
## What was missing
An upstream backed by Kubernetes discovery already ramps: endpoints reach
the balancer as ordinary nodes, a new one is new to the node set, and it warms
up. What slow start could not see is *which workload* answers on an address.
A cluster recycles addresses quickly, so both halves of that were wrong:
- a Pod that took over an address another Pod had just released inherited
its maturity and went straight to full weight, which is exactly the burst slow
start exists to prevent;
- a Pod that moved to another address was ramped again from the minimum
although it had been serving all along.
## Pod identity
`apisix/discovery/kubernetes/core.lua` now carries `targetRef.uid` in the
node metadata, for EndpointSlice and for the older Endpoints API, and `node_id`
in `apisix/slow_start.lua` prefers it:
| endpoint | lifecycle identity |
|---|---|
| backed by a Pod | `targetRef.uid` and the port |
| no `targetRef` (an external address, say) | host and port, as before |
| a configured node | host and port, a domain node its hostname |
Everything else follows from that: a Pod changing address keeps its ramp, an
address changing Pod starts a new one. The uid also makes the node set change
when a Pod is replaced in place, so the picker is rebuilt rather than kept from
the previous version.
## `slow_start.default_weight`
Kubernetes endpoints carry no weight of their own, so the registry gives
every one of them the same `default_weight`. An upstream that wants its ramp to
end somewhere else can now say so:
```json
{
"type": "roundrobin",
"service_name": "default/orders:http",
"discovery_type": "kubernetes",
"slow_start": {
"slow_start_time_seconds": 300,
"min_weight_percent": 1,
"default_weight": 100
}
}
```
It is rejected on anything but a `discovery_type: kubernetes` upstream.
Elsewhere the weights a registry hands out mean something - a Nacos service
with a 10/20/70 split, for one - and silently flattening them would be worse
than not offering the field.
## Deliberately out of scope
Two things the RFC lists for this stage are not here, and I would rather say
so than leave them looking forgotten:
- **Sharing endpoint observation times between upstreams.** The RFC records
`first_observed_at` at the discovery layer so two upstreams referencing the
same Service agree on when an endpoint appeared. Each upstream already keeps
its own lifecycle, and the only case the shared record changes is an upstream
created while the Service was already running, whose nodes are a mature
baseline either way.
- **Making the LIST transactional.** A relist republishes slices one at a
time, so a node can briefly leave the set. The tombstone window already covers
that: it comes back inside one slow start window and resumes its ramp rather
than restarting it. Buffering a whole LIST before publishing it is a change to
the discovery module's own contract, not to slow start, and belongs with a
change that has a reason of its own.
## Naming
@shreemaan-abhishek's review of #13941 pointed out that the feature answered
to two names: the configuration was `warm_up_conf`, while the module, the
shared dict, the log lines and the test files all said slow start - and the key
held a field called `slow_start_time_seconds`, so both names met inside one
object.
Slow start is what the rest of the world calls this. Envoy has
`slow_start_config`, NGINX Plus a `slow_start` parameter on an upstream server,
HAProxy `slowstart`. The weight curve here is Envoy's and two of the field
names already match it, so the configuration key becomes `slow_start` and
everything reads the same way. It also drops a `_conf` suffix that no other
upstream field carries.
The field names inside it do not change, and nothing has been released with
the old key, so there is nothing to migrate.
## Tests
- `t/node/upstream-slow-start.t` — a Pod keeps its ramp across an address
change, an address taken over by a different Pod ramps again, and
`default_weight` is the target a Kubernetes endpoint ramps to while a Nacos
upstream keeps its own weights.
- `t/admin/upstream-slow-start.t` — `default_weight` accepted on a
Kubernetes upstream and rejected anywhere else.
- `t/kubernetes/discovery/kubernetes2.t` — an endpoint with a `targetRef`
carries the Pod uid, one without it carries none. Runs in the `ci-kubernetes`
job, which needs a cluster.
Everything except the `t/kubernetes` case was run locally; that one needs
the kind cluster the CI job brings up.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
