ylnsnv opened a new issue, #35364:
URL: https://github.com/apache/airflow/issues/35364
### Official Helm Chart version
1.11.0 (latest released)
### Apache Airflow version
airflowVersion: 2.7.1
### Kubernetes Version
minikube version: v1.31.1
### Helm Chart configuration
values.yaml -
```
logs:
persistence:
size: 8Gi
workers:
persistence:
size: 8Gi
triggerer:
persistence:
size: 8Gi
webserverSecretKey: someSecretKey
webserver:
livenessProbe:
timeoutSeconds: 30
postgresql:
enabled: false
extraEnv: |-
- name: PG_HOST
valueFrom:
configMapKeyRef:
name: postgres-config-map
key: postgres-host
- name: PG_PORT
valueFrom:
configMapKeyRef:
name: postgres-config-map
key: postgres-port
- name: PG_USER
valueFrom:
secretKeyRef:
name: postgres-secret
key: postgres-user
- name: PG_PASS
valueFrom:
secretKeyRef:
name: postgres-secret
key: postgres-password
- name: PG_DB
valueFrom:
secretKeyRef:
name: postgres-secret
key: postgres-db
- name: PG_SSLROOTCERT
valueFrom:
secretKeyRef:
name: postgres-secret
key: postgres-ssl
config:
AIRFLOW__CORE__:
SQL_ALCHEMY_CONN:
"postgresql+psycopg2://$(PG_USER):$(PG_PASS)@$(PG_HOST):$(PG_PORT)/$(PG_DB)?sslmode=verify-full&sslrootcert=$(PG_SSLROOTCERT)"
pgbouncer:
enabled: true
data:
metadataSecretName: mydatabase
```
### Docker Image customizations
_No response_
### What happened
I am experiencing a challenge with the Airflow webserver when it is
configured to use an external database. It appears that every other component
functions as expected, except for the webserver. The logs repetitively show the
following entries:
```
[2023-11-02T06:44:34.683+0000] {configuration.py:2066} INFO - Creating new
FAB webserver config file in: /opt/airflow/webserver_config.py
/home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:336
UserWarning: Using the in-memory storage for tracking rate limits as no storage
was explicitly specified. This is not recommended for production use. See:
https://flask-limiter.readthedocs.io#configuring-a-storage-backend for
documentation about configuring the storage backend.
____________ _____________
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
Access Logformat:
=================================================================
[2023-11-02T06:45:27.301+0000] {webserver_command.py:440} INFO - Received
signal: 15. Closing gunicorn.
```
To further diagnose, I overrode the default webserver commands in
`values.yaml` as follows:
```
command: ["sleep"]
args: ["infinity"]
```
Upon SSHing into the webserver pod, running `airflow db check` yielded a
successful response (albeit with a duration of approximately 10 seconds).
However, initiating the airflow webserver command produced the following logs:
```
____ |__( )_________ __/__ /________ __
____ /| |_ /__ ___/_ /_ __ /_ __ \_ | /| / /
___ ___ | / _ / _ __/ _ / / /_/ /_ |/ |/ /
_/_/ |_/_/ /_/ /_/ /_/ \____/____/|__/
Running the Gunicorn Server with:
Workers: 4 sync
Host: 0.0.0.0:8080
Timeout: 120
Logfiles: - -
Access Logformat:
=================================================================
/home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:336
UserWarning: Using the in-memory storage for tracking rate limits as no storage
was explicitly specified. This is not recommended for production use. See:
https://flask-limiter.readthedocs.io#configuring-a-storage-backend for
documentation about configuring the storage backend.
command terminated with exit code 137
```
It is worth mentioning that the webserver runs flawlessly when it is not
configured with the external DB settings. Specifically, when the values.yaml is
set to:
```
logs:
persistence:
size: 8Gi
workers:
persistence:
size: 8Gi
triggerer:
persistence:
size: 8Gi
```
I am trying to ascertain why an external DB connection might lead to an exit
code of 137, which is indicative of an Out-of-Memory (OOM) condition.
Furthermore, I have explored various resource allocation strategies, ranging
from increasing the memory allocation up to 5Gi to not specifying it, but the
issue persists.
I would greatly appreciate insights or solutions to this challenge.
### What you think should happen instead
I think the airflow webserver should start successfully.
### How to reproduce
Create the postgres configMap-
```
apiVersion: v1
kind: ConfigMap
metadata:
name: postgres-config-map
data:
postgres-host: "somehost.com"
postgres-port: "someport"
```
Create the postgres secret-
```
apiVersion: v1
kind: Secret
metadata:
name: postgres-secret
type: Opaque
stringData:
postgres-user: pgUserVale
postgres-password: pgPasswordValue
postgres-db: pgdb
postgres-ssl: -----BEGIN CERTIFICATE-----\n.....values....==\n-----END
CERTIFICATE-----
```
Run the script to create the `mydatabase` connection string secret-
```
#!/bin/bash
# Get values from ConfigMap and Secret
PG_HOST=$(kubectl get configmap postgres-config-map
-o=jsonpath='{.data.postgres-host}')
PG_PORT=$(kubectl get configmap postgres-config-map
-o=jsonpath='{.data.postgres-port}')
PG_USER=$(kubectl get secret postgres-secret
-o=jsonpath='{.data.postgres-user}' | base64 -d)
PG_PASS=$(kubectl get secret postgres-secret
-o=jsonpath='{.data.postgres-password}' | base64 -d)
PG_DB=$(kubectl get secret postgres-secret -o=jsonpath='{.data.postgres-db}'
| base64 -d)
# Create connection string and create a Secret
CONNECTION_STRING="postgresql://${PG_USER}:${PG_PASS}@${PG_HOST}:${PG_PORT}/${PG_DB}?sslmode=require&sslrootcert=/etc/secrets/pg-ssl/postgres-ssl.crt"
kubectl create secret generic mydatabase
--from-literal=connection=${CONNECTION_STRING}
```
and apply the values.yaml from above.
### Anything else
_No response_
### Are you willing to submit PR?
- [ ] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
