nic-6443 opened a new pull request, #14032: URL: https://github.com/apache/apisix/pull/14032
### Description Consul discovery can fail a whole fetch round with a bogus resty.http error when the catalog and health indexes change together. This PR stops killing the watchers, which is what causes it. `connect()` spawns the catalog and health watchers as light threads, waits for the first one and kills both. The watchers go through `resty.consul`, which reads the response body via resty.http, and resty.http reads a body in a child coroutine. If the watcher that gets killed is still reading its body (typically the health watch streaming a large `/v1/health/state/any` response while the catalog watch returns first), `ngx.thread.kill` doesn't cancel the read pending in that child coroutine. When it completes, it resumes `connect()` at whatever it's waiting on, usually the status-line receive of the next catalog fetch, and that fetch fails with: ``` fetch_services_from_server(): connect consul: ... by sub url: /catalog/services, got catalog result: ".../resty/http.lua:479: bad argument #1 to 'str_sub' (string expected, got boolean)" ``` Now each index has its own watcher timer, and one run is one blocking query, so nothing gets killed and every run frees its memory when it ends. The watcher that sees a change runs the round itself: validate, fetch, back off on errors, same code as before. Rounds don't overlap: a watcher that sees a change while another round is running parks, and gets re-armed together with the round's watcher when it ends, so it queries again against the updated indexes and only triggers another fetch if something is still new. `watch_catalog` / `watch_health` now run a single query and return nothing when the index didn't change; the retry moved to the timer, so `stop_flag` and worker exit are checked between queries. What stays the same: indexes are only committed after a successful fetch, the retry/backoff progression is still one per server, and without keepalive the rounds are still paced by `fetch_interval`. What changes: a watcher that didn't report a change keeps its blocking query open across rounds instead of being killed and restarted; each server runs two timers instead of one timer with two light threads; and if a timer can't be created, only that watcher stops instead of the whole loop. `t/discovery/consul-watch-race.t` reproduces the race with a mock Consul that bumps both indexes together, answers the catalog watch shortly after the health watch starts streaming a slow chunked body, and delays the non-blocking catalog read so the health body finishes while the fetch is waiting. It fails on master with the `got boolean` error and a 503, and passes with this change. #### Which issue(s) this PR fixes: N/A ### Checklist - [x] I have explained the need for this PR and the problem it solves - [x] I have explained the changes or the new features added to this PR - [x] I have added tests corresponding to this change - [ ] I have updated the documentation to reflect this change - [x] I have verified that this change is backward compatible (If not, please discuss on the [APISIX mailing list](https://github.com/apache/apisix/tree/master#community) first) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
