GuyT2002 commented on issue #628: URL: https://github.com/apache/apisix-helm-chart/issues/628#issuecomment-1738616124
Thanks, it solves the issue, however now when I install the helm chart I receive the following stacktrace in the apisix-etcd pods: ``` [38;5;6metcd [38;5;5m07:20:52.18 [0m [38;5;6metcd [38;5;5m07:20:52.19 [0m[1mWelcome to the Bitnami etcd container[0m [38;5;6metcd [38;5;5m07:20:52.20 [0mSubscribe to project updates by watching [1mhttps://github.com/bitnami/containers[0m [38;5;6metcd [38;5;5m07:20:52.20 [0mSubmit issues and feature requests at [1mhttps://github.com/bitnami/containers/issues[0m [38;5;6metcd [38;5;5m07:20:52.20 [0m [38;5;6metcd [38;5;5m07:20:52.20 [0m[38;5;2mINFO [0m ==> ** Starting etcd setup ** [38;5;6metcd [38;5;5m07:20:52.22 [0m[38;5;2mINFO [0m ==> Validating settings in ETCD_* env vars.. [38;5;6metcd [38;5;5m07:20:52.22 [0m[38;5;2mINFO [0m ==> Initializing etcd [38;5;6metcd [38;5;5m07:20:52.22 [0m[38;5;2mINFO [0m ==> Generating etcd config file using env variables [38;5;6metcd [38;5;5m07:20:52.24 [0m[38;5;2mINFO [0m ==> Detected data from previous deployments [38;5;6metcd [38;5;5m07:20:56.33 [0m[38;5;3mWARN [0m ==> Cluster not responding! [38;5;6metcd [38;5;5m07:20:56.34 [0m[38;5;3mWARN [0m ==> Disaster recovery is disabled, the cluster will try to recover on it's own [38;5;6metcd [38;5;5m07:20:56.35 [0m[38;5;2mINFO [0m ==> ** etcd setup finished! ** [38;5;6metcd [38;5;5m07:20:56.38 [0m[38;5;2mINFO [0m ==> ** Starting etcd ** {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_ADVERTISE_CLIENT_URLS","variable-value":"http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2379,http://apisix-etcd.ingress-apisix.svc.cluster.local:2379"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_AUTH_TOKEN","variable-value":"jwt,priv-key=/opt/bitnami/etcd/certs/token/jwt-token.pem,sign-method=RS256,ttl=10m"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_AUTO_TLS","variable-value":"false"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_CLIENT_CERT_AUTH","variable-value":"false"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_DATA_DIR","variable-value":"/bitnami/etcd/data"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_INITIAL_ADVERTISE_PEER_URLS","variable-value":"http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_INITIAL_CLUSTER","variable-value":"apisix-etcd-0=http://apisix-etcd-0.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380,apisix-etcd-1=http://apisix-etcd-1.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380,apisix-etcd-2=http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_INITIAL_CLUSTER_STATE","variable-value":"new"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_INITIAL_CLUSTER_TOKEN","variable-value":"etcd-cluster-k8s"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_LISTEN_CLIENT_URLS","variable-value":"http://0.0.0.0:2379"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_LISTEN_PEER_URLS","variable-value":"http://0.0.0.0:2380"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_LOG_LEVEL","variable-value":"info"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_NAME","variable-value":"apisix-etcd-2"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:113","msg":"recognized and used environment variable","variable-name":"ETCD_PEER_AUTO_TLS","variable-value":"false"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_TRUSTED_CA_FILE="} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_DISABLE_STORE_MEMBER_ID=no"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_CONF_FILE=/opt/bitnami/etcd/conf/etcd.yaml"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SNAPSHOT_HISTORY_LIMIT=1"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_ON_K8S=yes"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_SNAPSHOTS_DIR=/snapshots"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_BIN_DIR=/opt/bitnami/etcd/bin"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_VOLUME_DIR=/bitnami/etcd"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_ROOT_PASSWORD=4dcDyTJMCR"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_CLUSTER_DOMAIN=apisix-etcd-headless.ingress-apisix.svc.cluster.local"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_DISASTER_RECOVERY=no"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_KEY_FILE="} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_CONF_DIR=/opt/bitnami/etcd/conf"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_DAEMON_GROUP=etcd"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_START_FROM_SNAPSHOT=no"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_INIT_SNAPSHOT_FILENAME="} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_INIT_SNAPSHOTS_DIR=/init-snapshot"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_DISABLE_PRESTOP=no"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_TMP_DIR=/opt/bitnami/etcd/tmp"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_BASE_DIR=/opt/bitnami/etcd"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_CERT_FILE="} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_NEW_MEMBERS_ENV_FILE=/bitnami/etcd/data/new_member_envs"} {"level":"warn","ts":"2023-09-28T07:20:56.430Z","caller":"flags/flag.go:93","msg":"unrecognized environment variable","environment-variable":"ETCD_DAEMON_USER=etcd"} {"level":"info","ts":"2023-09-28T07:20:56.430Z","caller":"etcdmain/etcd.go:73","msg":"Running: ","args":["etcd"]} {"level":"warn","ts":"2023-09-28T07:20:56.431Z","caller":"etcdmain/etcd.go:446","msg":"found invalid file under data directory","filename":"member_id","data-dir":"/bitnami/etcd/data"} {"level":"info","ts":"2023-09-28T07:20:56.431Z","caller":"etcdmain/etcd.go:116","msg":"server has been already initialized","data-dir":"/bitnami/etcd/data","dir-type":"member"} {"level":"info","ts":"2023-09-28T07:20:56.431Z","caller":"embed/etcd.go:124","msg":"configuring peer listeners","listen-peer-urls":["http://0.0.0.0:2380"]} {"level":"info","ts":"2023-09-28T07:20:56.431Z","caller":"embed/etcd.go:132","msg":"configuring client listeners","listen-client-urls":["http://0.0.0.0:2379"]} {"level":"info","ts":"2023-09-28T07:20:56.431Z","caller":"embed/etcd.go:306","msg":"starting an etcd server","etcd-version":"3.5.7","git-sha":"215b53cf3","go-version":"go1.17.13","go-os":"linux","go-arch":"amd64","max-cpu-set":2,"max-cpu-available":2,"member-initialized":true,"name":"apisix-etcd-2","data-dir":"/bitnami/etcd/data","wal-dir":"","wal-dir-dedicated":"","member-dir":"/bitnami/etcd/data/member","force-new-cluster":false,"heartbeat-interval":"100ms","election-timeout":"1s","initial-election-tick-advance":true,"snapshot-count":100000,"max-wals":5,"max-snapshots":5,"snapshot-catchup-entries":5000,"initial-advertise-peer-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"],"listen-peer-urls":["http://0.0.0.0:2380"],"advertise-client-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2379","http://apisix-etcd.ingress-apisix.svc.cluster.local:2379"],"listen-client-urls":["http://0.0.0.0:2379"],"listen-metric s-urls":[],"cors":["*"],"host-whitelist":["*"],"initial-cluster":"","initial-cluster-state":"new","initial-cluster-token":"","quota-backend-bytes":2147483648,"max-request-bytes":1572864,"max-concurrent-streams":4294967295,"pre-vote":true,"initial-corrupt-check":false,"corrupt-check-time-interval":"0s","compact-check-time-enabled":false,"compact-check-time-interval":"1m0s","auto-compaction-mode":"periodic","auto-compaction-retention":"0s","auto-compaction-interval":"0s","discovery-url":"","discovery-proxy":"","downgrade-check-interval":"5s"} {"level":"info","ts":"2023-09-28T07:20:56.433Z","caller":"etcdserver/backend.go:81","msg":"opened backend db","path":"/bitnami/etcd/data/member/snap/db","took":"249.317µs"} {"level":"info","ts":"2023-09-28T07:20:57.377Z","caller":"etcdserver/server.go:509","msg":"recovered v2 store from snapshot","snapshot-index":700007,"snapshot-size":"18 kB"} {"level":"info","ts":"2023-09-28T07:20:57.378Z","caller":"etcdserver/server.go:522","msg":"recovered v3 backend from snapshot","backend-size-bytes":20480,"backend-size":"20 kB","backend-size-in-use-bytes":12288,"backend-size-in-use":"12 kB"} {"level":"info","ts":"2023-09-28T07:20:58.622Z","caller":"etcdserver/raft.go:529","msg":"restarting local member","cluster-id":"b9ae902a9398fd61","local-member-id":"39e3f227d2d33a6d","commit-index":794402} {"level":"info","ts":"2023-09-28T07:20:58.625Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"39e3f227d2d33a6d switched to configuration voters=(4171443932729916013 11625357162296873047 16309538207480531017)"} {"level":"info","ts":"2023-09-28T07:20:58.626Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"39e3f227d2d33a6d became follower at term 34"} {"level":"info","ts":"2023-09-28T07:20:58.626Z","logger":"raft","caller":"etcdserver/zap_raft.go:77","msg":"newRaft 39e3f227d2d33a6d [peers: [39e3f227d2d33a6d,a15590becdbf9057,e2571e968b89c849], term: 34, commit: 794402, applied: 700007, lastindex: 794402, lastterm: 34]"} {"level":"info","ts":"2023-09-28T07:20:58.626Z","caller":"api/capability.go:75","msg":"enabled capabilities for version","cluster-version":"3.5"} {"level":"info","ts":"2023-09-28T07:20:58.626Z","caller":"membership/cluster.go:278","msg":"recovered/added member from store","cluster-id":"b9ae902a9398fd61","local-member-id":"39e3f227d2d33a6d","recovered-remote-peer-id":"e2571e968b89c849","recovered-remote-peer-urls":["http://apisix-etcd-0.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"]} {"level":"info","ts":"2023-09-28T07:20:58.626Z","caller":"membership/cluster.go:278","msg":"recovered/added member from store","cluster-id":"b9ae902a9398fd61","local-member-id":"39e3f227d2d33a6d","recovered-remote-peer-id":"39e3f227d2d33a6d","recovered-remote-peer-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"]} {"level":"info","ts":"2023-09-28T07:20:58.626Z","caller":"membership/cluster.go:278","msg":"recovered/added member from store","cluster-id":"b9ae902a9398fd61","local-member-id":"39e3f227d2d33a6d","recovered-remote-peer-id":"a15590becdbf9057","recovered-remote-peer-urls":["http://apisix-etcd-1.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"]} {"level":"info","ts":"2023-09-28T07:20:58.626Z","caller":"membership/cluster.go:287","msg":"set cluster version from store","cluster-version":"3.5"} {"level":"error","ts":"2023-09-28T07:20:58.628Z","caller":"auth/jwt.go:125","msg":"problem loading JWT options","error":"open /opt/bitnami/etcd/certs/token/jwt-token.pem: permission denied","stacktrace":"go.etcd.io/etcd/server/v3/auth.newTokenProviderJWT\n\tgo.etcd.io/etcd/server/v3/auth/jwt.go:125\ngo.etcd.io/etcd/server/v3/auth.NewTokenProvider\n\tgo.etcd.io/etcd/server/v3/auth/store.go:1239\ngo.etcd.io/etcd/server/v3/etcdserver.NewServer\n\tgo.etcd.io/etcd/server/v3/etcdserver/server.go:606\ngo.etcd.io/etcd/server/v3/embed.StartEtcd\n\tgo.etcd.io/etcd/server/v3/embed/etcd.go:243\ngo.etcd.io/etcd/server/v3/etcdmain.startEtcd\n\tgo.etcd.io/etcd/server/v3/etcdmain/etcd.go:228\ngo.etcd.io/etcd/server/v3/etcdmain.startEtcdOrProxyV2\n\tgo.etcd.io/etcd/server/v3/etcdmain/etcd.go:123\ngo.etcd.io/etcd/server/v3/etcdmain.Main\n\tgo.etcd.io/etcd/server/v3/etcdmain/main.go:40\nmain.main\n\tgo.etcd.io/etcd/server/v3/main.go:32\nruntime.main\n\truntime/proc.go:255"} {"level":"warn","ts":"2023-09-28T07:20:58.628Z","caller":"etcdserver/server.go:613","msg":"failed to create token provider","error":"auth: invalid auth options"} {"level":"info","ts":"2023-09-28T07:20:58.630Z","caller":"embed/etcd.go:373","msg":"closing etcd server","name":"apisix-etcd-2","data-dir":"/bitnami/etcd/data","advertise-peer-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"],"advertise-client-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2379","http://apisix-etcd.ingress-apisix.svc.cluster.local:2379"]} {"level":"info","ts":"2023-09-28T07:20:58.630Z","caller":"embed/etcd.go:375","msg":"closed etcd server","name":"apisix-etcd-2","data-dir":"/bitnami/etcd/data","advertise-peer-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2380"],"advertise-client-urls":["http://apisix-etcd-2.apisix-etcd-headless.ingress-apisix.svc.cluster.local:2379","http://apisix-etcd.ingress-apisix.svc.cluster.local:2379"]} {"level":"fatal","ts":"2023-09-28T07:20:58.630Z","caller":"etcdmain/etcd.go:204","msg":"discovery failed","error":"auth: invalid auth options","stacktrace":"go.etcd.io/etcd/server/v3/etcdmain.startEtcdOrProxyV2\n\tgo.etcd.io/etcd/server/v3/etcdmain/etcd.go:204\ngo.etcd.io/etcd/server/v3/etcdmain.Main\n\tgo.etcd.io/etcd/server/v3/etcdmain/main.go:40\nmain.main\n\tgo.etcd.io/etcd/server/v3/main.go:32\nruntime.main\n\truntime/proc.go:255"} ``` However, I don't understand which auth optino relates to this. [Auth options](https://github.com/bitnami/charts/tree/main/bitnami/etcd#etcd-parameters) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
