JoniJnm commented on issue #9026:
URL: https://github.com/apache/apisix/issues/9026#issuecomment-2119945958
I'm using the apisix helm chart 2.7.0 (currently, the latest) and sometimes
I have this same error.
Just deleting the apisix-ingress-controller pod it starts to work again.
The apisix-ingress-controller pod output:
```
I0519 16:17:41.437043 1 leaderelection.go:250] attempting to acquire
leader lease apisix-gateway/ingress-apisix-leader...
I0519 16:17:41.451338 1 leaderelection.go:260] successfully acquired
lease apisix-gateway/ingress-apisix-leader
2024-05-19T16:17:41+08:00 warn providers/controller.go:220 found a
new leader apisix-ingress-controller-7b6f897698-rv4wq
2024-05-19T16:17:41+08:00 warn apisix/cluster.go:423 waiting cluster
default to ready, it may takes a while
2024-05-19T16:17:41+08:00 error apisix/route.go:90 failed to list
routes: unexpected status code 503; error message: {"error_msg":"has no healthy
etcd endpoint available"}
2024-05-19T16:17:41+08:00 error apisix/cluster.go:298 failed to list
routes in APISIX: unexpected status code 503; error message: {"error_msg":"has
no healthy etcd endpoint available"}
2024-05-19T16:17:43+08:00 error apisix/route.go:90 failed to list
routes: unexpected status code 503; error message: {"error_msg":"has no healthy
etcd endpoint available"}
2024-05-19T16:17:43+08:00 error apisix/cluster.go:298 failed to list
routes in APISIX: unexpected status code 503; error message: {"error_msg":"has
no healthy etcd endpoint available"}
2024-05-19T16:17:45+08:00 error apisix/route.go:90 failed to list
routes: unexpected status code 503; error message: {"error_msg":"has no healthy
etcd endpoint available"}
2024-05-19T16:17:45+08:00 error apisix/cluster.go:298 failed to list
routes in APISIX: unexpected status code 503; error message: {"error_msg":"has
no healthy etcd endpoint available"}
2024-05-19T16:17:47+08:00 error apisix/route.go:90 failed to list
routes: unexpected status code 503; error message: {"error_msg":"has no healthy
etcd endpoint available"}
2024-05-19T16:17:47+08:00 error apisix/cluster.go:298 failed to list
routes in APISIX: unexpected status code 503; error message: {"error_msg":"has
no healthy etcd endpoint available"}
2024-05-19T16:17:49+08:00 error apisix/route.go:90 failed to list
routes: unexpected status code 503; error message: {"error_msg":"has no healthy
etcd endpoint available"}
2024-05-19T16:17:49+08:00 error apisix/cluster.go:298 failed to list
routes in APISIX: unexpected status code 503; error message: {"error_msg":"has
no healthy etcd endpoint available"}
2024-05-19T16:17:49+08:00 error apisix/cluster.go:258 failed to sync
cache {"cost_time": "8.019457909s", "cluster": "default"}
2024-05-19T16:17:49+08:00 error providers/controller.go:419 failed
to wait the default cluster to be ready: unexpected status code 503; error
message: {"error_msg":"has no healthy etcd endpoint available"}
2024-05-19T16:17:49+08:00 error apisix/route.go:90 failed to list
routes: Get
"http://apisix-admin.apisix-gateway.svc.cluster.local:9180/apisix/admin/routes":
context canceled
2024-05-19T16:17:49+08:00 error apisix/plugin.go:46 failed to list
plugins' names: Get
"http://apisix-admin.apisix-gateway.svc.cluster.local:9180/apisix/admin/plugins?all=true":
context canceled
2024-05-19T16:17:49+08:00 error apisix/cluster.go:483 failed to list
plugin names in APISIX: Get
"http://apisix-admin.apisix-gateway.svc.cluster.local:9180/apisix/admin/plugins?all=true":
context canceled
2024-05-19T16:17:49+08:00 error apisix/cluster.go:298 failed to list
routes in APISIX: Get
"http://apisix-admin.apisix-gateway.svc.cluster.local:9180/apisix/admin/routes":
context canceled
2024-05-19T16:17:49+08:00 error apisix/cluster.go:446 failed to sync
schema: Get
"http://apisix-admin.apisix-gateway.svc.cluster.local:9180/apisix/admin/plugins?all=true":
context canceled
2024-05-19T16:17:49+08:00 error apisix/cluster.go:258 failed to sync
cache {"cost_time": "84.421µs", "cluster": "default"}
```
No errors on etc.
Some "errors" on apisix pod:
```
2024/05/20 07:35:17 [error] 50#50: *1482951 upstream timed out (110:
Connection timed out) while connecting to upstream, client: 10.0.0.87, server:
_, request: "POST /api/email/xxxxxx HTTP/1.1", upstream:
"http://10.16.0.4:8080/api/email/xxxxxxx";, host: "xxxxxx"
```
My current config:
```yaml
replicaCount: 2
resources:
requests:
cpu: 20m
memory: 200Mi
limits:
cpu: 1
memory: 512Mi
ingress-controller:
enabled: true
gateway:
tls:
enabled: true
config:
logLevel: "warn"
apisix:
serviceNamespace: apisix-gateway
resources:
requests:
memory: 50Mi
etcd:
logLevel: warn
replicaCount: 1
pdb:
create: false
service:
loadBalancerIP: xxxxx
type: LoadBalancer
dashboard:
enabled: false
apisix:
enableServerTokens: false
ssl:
enabled: true
pluginAttrs:
redirect:
https_port: 443
nginx:
logs:
enableAccessLog: true
accessLogFormatEscape: json
accessLogFormat:
'{"datetime":"$time_iso8601","http_referer":"$http_referer","host":"$host","remote_addr":"$remote_addr","request_method":"$request_method","request_time":"$request_time","request_uri":"$request_uri","http_status":"$status"}'
metrics:
serviceMonitor:
enabled: true
interval: 30s
labels:
release: 'kube-prometheus-stack'
```
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: notifications-unsubscr...@apisix.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
