VictorAtPL opened a new issue, #33635:
URL: https://github.com/apache/airflow/issues/33635

   ### Apache Airflow version
   
   2.7.0
   
   ### What happened
   
   After deploying Airflow Helm Chart (User Community) with image.tag set to 
2.7.0-python3.8 and creating a user, `sync-airflow-users` pod exits with 
following log:
   
   ```
   sync-airflow-users 
/home/airflow/.local/lib/python3.8/site-packages/flask_limiter/extension.py:293 
UserWarning: Using the in-memory storage for tracking rate limits as no storage 
was explicitly specified. This is not recommended for production use. See: 
https://flask-limiter.readthedocs.io#configuring-a-storage-backend for 
documentation about configuring the storage backend.
   sync-airflow-users Traceback (most recent call last):
   sync-airflow-users   File "/mnt/scripts/sync_users.py", line 146, in <module>
   sync-airflow-users     from airflow.www.fab_security.sqla.models import 
User, Role
   sync-airflow-users ModuleNotFoundError: No module named 
'airflow.www.fab_security.sqla.models'
   sync-airflow-users 
   sync-airflow-users During handling of the above exception, another exception 
occurred:
   sync-airflow-users 
   sync-airflow-users Traceback (most recent call last):
   sync-airflow-users   File "/mnt/scripts/sync_users.py", line 148, in <module>
   sync-airflow-users     from flask_appbuilder.security.sqla.models import 
User, Role
   sync-airflow-users   File 
"/home/airflow/.local/lib/python3.8/site-packages/flask_appbuilder/security/sqla/models.py",
 line 48, in <module>
   sync-airflow-users     assoc_permissionview_role = Table(
   sync-airflow-users   File "<string>", line 2, in __new__
   sync-airflow-users   File 
"/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/util/deprecations.py",
 line 375, in warned
   sync-airflow-users     return fn(*args, **kwargs)
   sync-airflow-users   File 
"/home/airflow/.local/lib/python3.8/site-packages/sqlalchemy/sql/schema.py", 
line 596, in __new__
   sync-airflow-users     raise exc.InvalidRequestError(
   sync-airflow-users sqlalchemy.exc.InvalidRequestError: Table 
'ab_permission_view_role' is already defined for this MetaData instance.  
Specify 'extend_existing=True' to redefine options and columns on an existing 
Table object.
   Stream closed EOF for airflow/airflow-sync-users-b989f959c-zz2dp 
(wait-for-db-migrations)
   Stream closed EOF for airflow/airflow-sync-users-b989f959c-zz2dp (check-db)
   Stream closed EOF for airflow/airflow-sync-users-b989f959c-zz2dp 
(sync-airflow-users)
   ```
   
   I've seen these two issues which mention `ab_permission_view_role`, but the 
solutions provided there aren't a fix for my issue:
   1. https://github.com/apache/airflow/issues/27138
   2. https://github.com/airflow-helm/charts/issues/747
   
   So setting:
   ```
   web:
   webserverConfig:
   stringOverride: |
   from airflow.www.fab_security.manager import AUTH_DB
   AUTH_TYPE = AUTH_DB
   ```
   doesn't fix the issue.
   
   ### What you think should happen instead
   
   `sqlalchemy.exc.InvalidRequestError` shouldn't be thrown.
   
   `sync-airflow-users` pod shouldn't exit and keep restarting.
   
   ### How to reproduce
   
   1. Deploy EKS cluster v.1.2.5.
   2. Deploy RDS cluster (PostgreSQL db).
   3. Deploy Airflow community helm chart using Terraform code provided below.
   4. Shell into `airflow-web` and create a user using `airflow users create -u 
... -f ... -l ... -p ... -r Admin -e ...`.
   5. `sync-airflow-users` pod will be restarted because of the error provided 
above.
   
   Terraform code used to deploy Airflow:
   ```
   locals {
     # workaround: 
https://github.com/hashicorp/terraform-provider-helm/issues/669#issuecomment-765171927
     values = {
       airflow = {
         image = {
           tag = "2.7.0-python3.8"
         }
         connections = [{
           "id" = "aws_default",
           "type" = "aws",
           "description" = "AWS connection to send emails using SES",
           "extra" = <<EOF
   {
   "region_name": "${data.aws_region.current.name}"
   }
   EOF
         }]
       }
     }
   }
   
   resource "helm_release" "airflow" {
     name       = "airflow"
     repository = "https://airflow-helm.github.io/charts";
     version = "8.7.1"
     chart      = "airflow"
     namespace = local.namespace
     atomic = true
     wait = true
     timeout = 60 * 20 # 20 minutes
   
     dynamic "set" {
       for_each = {
         "airflow.executor" = "KubernetesExecutor"
         "workers.persistence.enabled" = false
         "postgresql.persistence.enabled" = false
         "scheduler.logCleanup.enabled" = false
         "flower.enabled" = false
         "workers.enabled" = false
         "redis.enabled" = false
         "ingress.enabled" = true
         "postgresql.enabled": false,
         "pgbouncer.enabled": true,
         "dags.persistence.enabled" = true
         "dags.persistence.existingClaim" = local.pvc_name_dags
         "dags.persistence.accessMode" = "ReadOnlyMany"
         "logs.persistence.enabled": true
         "logs.persistence.existingClaim": local.pvc_name_logs
         "logs.persistence.accessMode": "ReadWriteMany"
         "rbac.create": false,
         "serviceAccount.create": false,
         "serviceAccount.name": 
kubernetes_service_account.airflow_services.metadata.0.name,
         "externalDatabase.type": "postgres",
         "externalDatabase.host": var.postgres_endpoint,
         "externalDatabase.port": var.postgres_port,
         "externalDatabase.database": "...",
         "externalDatabase.userSecret": var.postgres_secret_names[0],
         "externalDatabase.userSecretKey": "username",
         "externalDatabase.passwordSecret": var.postgres_secret_names[0],
         "externalDatabase.passwordSecretKey": "password"
       }
   
       content {
         name  = set.key
         value = set.value
       }
     }
   
     # workaround: 
https://github.com/hashicorp/terraform-provider-helm/issues/669#issuecomment-765171927
     values = [yamlencode(local.values)]
   
     depends_on = [...]
   }
   
   ```
   
   ### Operating System
   
   Debian GNU/Linux 11 (bullseye)
   
   ### Versions of Apache Airflow Providers
   
   apache-airflow-providers-cncf-kubernetes==7.4.2
   
   
   ### Deployment
   
   Other 3rd-party Helm chart
   
   ### Deployment details
   
   EKS Fargate deployment type in AWS.
   
   ### Anything else
   
   _No response_
   
   ### Are you willing to submit PR?
   
   - [ ] Yes I am willing to submit a PR!
   
   ### Code of Conduct
   
   - [X] I agree to follow this project's [Code of 
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: [email protected]

For queries about this service, please contact Infrastructure at:
[email protected]

Reply via email to