VictorAtPL opened a new issue, #33635: URL: https://github.com/apache/airflow/issues/33635
### Apache Airflow version 2.7.0 ### What happened After deploying Airflow Helm Chart (User Community) with image.tag set to 2.7.0-python3.8 and creating a user, `sync-airflow-users` pod exits with following log: ``` sync-airflow-users /home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:293 UserWarning: Using the in-memory storage for tracking rate limits as no storage was explicitly specified. This is not recommended for production use. See: https://flask-limiter.readthedocs.io#configuring-a-storage-backend for documentation about configuring the storage backend. sync-airflow-users Traceback (most recent call last): sync-airflow-users File "/mnt/scripts/sync_users.py", line 146, in <module> sync-airflow-users from airflow.www.fab_security.sqla.models import User, Role sync-airflow-users ModuleNotFoundError: No module named 'airflow.www.fab_security.sqla.models' sync-airflow-users sync-airflow-users During handling of the above exception, another exception occurred: sync-airflow-users sync-airflow-users Traceback (most recent call last): sync-airflow-users File "/mnt/scripts/sync_users.py", line 148, in <module> sync-airflow-users from flask_appbuilder.security.sqla.models import User, Role sync-airflow-users File "/home/airflow/.local/lib/python3.8/site-packages/flask_appbuilder/security/sqla/models.py", line 48, in <module> sync-airflow-users assoc_permissionview_role = Table( sync-airflow-users File "<string>", line 2, in __new__ sync-airflow-users File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py", line 375, in warned sync-airflow-users return fn(*args, **kwargs) sync-airflow-users File "/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", line 596, in __new__ sync-airflow-users raise exc.InvalidRequestError( sync-airflow-users sqlalchemy.exc.InvalidRequestError: Table 'ab_permission_view_role' is already defined for this MetaData instance. Specify 'extend_existing=True' to redefine options and columns on an existing Table object. Stream closed EOF for airflow/airflow-sync-users-b989f959c-zz2dp (wait-for-db-migrations) Stream closed EOF for airflow/airflow-sync-users-b989f959c-zz2dp (check-db) Stream closed EOF for airflow/airflow-sync-users-b989f959c-zz2dp (sync-airflow-users) ``` I've seen these two issues which mention `ab_permission_view_role`, but the solutions provided there aren't a fix for my issue: 1. https://github.com/apache/airflow/issues/27138 2. https://github.com/airflow-helm/charts/issues/747 So setting: ``` web: webserverConfig: stringOverride: | from airflow.www.fab_security.manager import AUTH_DB AUTH_TYPE = AUTH_DB ``` doesn't fix the issue. ### What you think should happen instead `sqlalchemy.exc.InvalidRequestError` shouldn't be thrown. `sync-airflow-users` pod shouldn't exit and keep restarting. ### How to reproduce 1. Deploy EKS cluster v.1.2.5. 2. Deploy RDS cluster (PostgreSQL db). 3. Deploy Airflow community helm chart using Terraform code provided below. 4. Shell into `airflow-web` and create a user using `airflow users create -u ... -f ... -l ... -p ... -r Admin -e ...`. 5. `sync-airflow-users` pod will be restarted because of the error provided above. Terraform code used to deploy Airflow: ``` locals { # workaround: https://github.com/hashicorp/terraform-provider-helm/issues/669#issuecomment-765171927 values = { airflow = { image = { tag = "2.7.0-python3.8" } connections = [{ "id" = "aws_default", "type" = "aws", "description" = "AWS connection to send emails using SES", "extra" = <<EOF { "region_name": "${data.aws_region.current.name}" } EOF }] } } } resource "helm_release" "airflow" { name = "airflow" repository = "https://airflow-helm.github.io/charts" version = "8.7.1" chart = "airflow" namespace = local.namespace atomic = true wait = true timeout = 60 * 20 # 20 minutes dynamic "set" { for_each = { "airflow.executor" = "KubernetesExecutor" "workers.persistence.enabled" = false "postgresql.persistence.enabled" = false "scheduler.logCleanup.enabled" = false "flower.enabled" = false "workers.enabled" = false "redis.enabled" = false "ingress.enabled" = true "postgresql.enabled": false, "pgbouncer.enabled": true, "dags.persistence.enabled" = true "dags.persistence.existingClaim" = local.pvc_name_dags "dags.persistence.accessMode" = "ReadOnlyMany" "logs.persistence.enabled": true "logs.persistence.existingClaim": local.pvc_name_logs "logs.persistence.accessMode": "ReadWriteMany" "rbac.create": false, "serviceAccount.create": false, "serviceAccount.name": kubernetes_service_account.airflow_services.metadata.0.name, "externalDatabase.type": "postgres", "externalDatabase.host": var.postgres_endpoint, "externalDatabase.port": var.postgres_port, "externalDatabase.database": "...", "externalDatabase.userSecret": var.postgres_secret_names[0], "externalDatabase.userSecretKey": "username", "externalDatabase.passwordSecret": var.postgres_secret_names[0], "externalDatabase.passwordSecretKey": "password" } content { name = set.key value = set.value } } # workaround: https://github.com/hashicorp/terraform-provider-helm/issues/669#issuecomment-765171927 values = [yamlencode(local.values)] depends_on = [...] } ``` ### Operating System Debian GNU/Linux 11 (bullseye) ### Versions of Apache Airflow Providers apache-airflow-providers-cncf-kubernetes==7.4.2 ### Deployment Other 3rd-party Helm chart ### Deployment details EKS Fargate deployment type in AWS. ### Anything else _No response_ ### Are you willing to submit PR? - [ ] Yes I am willing to submit a PR! ### Code of Conduct - [X] I agree to follow this project's [Code of Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
