odaneau-astro opened a new issue, #26437:
URL: https://github.com/apache/airflow/issues/26437
### Apache Airflow version
2.3.4
### What happened
I am attempting to configure variables with a Google Secrets Manager secrets
backend and it appears that having dashes in the name prevents Airflow from
loading the variables.
Here is what I did:
1. Create some test variables with
```
gcloud secrets create airflow-variables-test
--replication-policy="automatic"
gcloud secrets create airflow-variables-test-test
--replication-policy="automatic"
gcloud secrets create airflow-variables-test_test
--replication-policy="automatic"
```
2. Logged in the GCP console and set all the secrets values to "12345"
3. Created a simple DAG to test:
```
from airflow import DAG
from airflow.models import Variable
from airflow.operators.python import PythonOperator
from datetime import datetime
def print_var():
variables = ["test", "test-test", "test_test"]
for var in variables:
try:
val = Variable.get(var)
print(f'Variable {var} is: {val}')
except Exception:
print(f"Could not get variable {var}")
with DAG('gsm_secrets_backend', start_date=datetime(2022, 1, 1),
schedule_interval=None) as dag:
test_task = PythonOperator(
task_id='print_var',
python_callable=print_var
)
```
4. Ran the DAG locally through astro cli.
4. Looked at the logs:
```
[2022-09-16, 14:31:19 UTC] {logging_mixin.py:115} INFO - Variable test
is: 12345
[2022-09-16, 14:31:20 UTC] {secret_manager_client.py:78} ERROR - Google
Cloud API Call Error (NotFound): Secret ID airflow-variables-test-test not
found.
[2022-09-16, 14:31:20 UTC] {logging_mixin.py:115} INFO - Could not get
variable test-test
[2022-09-16, 14:31:20 UTC] {logging_mixin.py:115} INFO - Variable
test_test is: 12345
[2022-09-16, 14:31:20 UTC] {python.py:173} INFO - Done. Returned value
was: None
```
### What you think should happen instead
Variables with dashes in their name should successfully be added
### How to reproduce
`astro dev init` a fresh project
`Dockerfile`:
FROM quay.io/astronomer/astro-runtime:5.0.7
`.env`:
AIRFLOW__SECRETS__BACKEND=airflow.providers.google.cloud.secrets.secret_manager.CloudSecretManagerBackend
AIRFLOW__SECRETS__BACKEND_KWARGS='{"connections_prefix":
"airflow-connections", "variables_prefix": "airflow-variables",
"gcp_keyfile_dict": <service-acct-json-keyfile>}'
`astro dev start`
Create some test variables with
```
gcloud secrets create airflow-variables-test-test
--replication-policy="automatic"
```
Log in the GCP console and set the secret value
Create this simple DAG
```
from airflow import DAG
from airflow.models import Variable
from airflow.operators.python import PythonOperator
from datetime import datetime
def print_var():
variables = ["test-test"]
for var in variables:
try:
val = Variable.get(var)
print(f'Variable {var} is: {val}')
except Exception:
print(f"Could not get variable {var}")
with DAG('gsm_secrets_backend', start_date=datetime(2022, 1, 1),
schedule_interval=None) as dag:
test_task = PythonOperator(
task_id='print_var',
python_callable=print_var
)
```
Trigger the DAG and look at the logs.
### Operating System
Ubuntu 20.04.5 LTS
### Versions of Apache Airflow Providers
apache-airflow-providers-google==8.3.0
### Deployment
Astronomer
### Deployment details
_No response_
### Anything else
_No response_
### Are you willing to submit PR?
- [X] Yes I am willing to submit a PR!
### Code of Conduct
- [X] I agree to follow this project's [Code of
Conduct](https://github.com/apache/airflow/blob/main/CODE_OF_CONDUCT.md)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
