potiuk commented on a change in pull request #18494: URL: https://github.com/apache/airflow/pull/18494#discussion_r716791580
########## File path: airflow/providers/google/cloud/example_dags/example_cloud_sql.py ########## @@ -48,8 +49,8 @@ from airflow.utils.dates import days_ago GCP_PROJECT_ID = os.environ.get('GCP_PROJECT_ID', 'example-project') -INSTANCE_NAME = os.environ.get('GCSQL_MYSQL_INSTANCE_NAME', 'test-mysql') -INSTANCE_NAME2 = os.environ.get('GCSQL_MYSQL_INSTANCE_NAME2', 'test-mysql2') +INSTANCE_NAME = os.environ.get('GCSQL_MYSQL_INSTANCE_NAME', 'test-mysql') + str(random.getrandbits(16)) Review comment: Yes. This is HIGHLY annoying with cloudsql that name cannot ber reused for a day or so even if db is deleted. However I agree random POSTFIX generation in the DAG is a bad idea. What we used to have in the past is that we had `variables.env` file in `files/airflow-breeze-config` where we sourced files with variables and we had a very simple script that generated the random postfix if it was missing. Then you could do step-by-step testing with keeping the randomly generated postfix even across breeze restarts. When you needed to change the database name you'd simply remove the file and it would be re-generated automatically at breeze entry. https://github.com/apache/airflow/blob/main/BREEZE.rst#customize-your-environment Something like that might work (writing it from memory so I am not sure if it is correct) in variables.env: ``` if [[ ! -f /files/random.env ]]; then echo "export RANDOM_POSTFIX=${RANDOM}" > /files/random.env fi source /files/random.env export GCSQL_MYSQL_INSTANCE_NAME="test-mysql-${RANDOM_POSTFIX}" ``` This has several nice properties: * everyone has its own random value * you keep it stable between runs or even between debug sessions - for example you could ran tasks from the example DAG separately one-by-one * you can very easily regenerate the number by simply deleting the /files/random.env In the past we even shared the whole `airflow-breeze-config` directory was actually checked out separate repository where we kept all variables used by the team. This way you could share different variables between same users who have access to the same repo - at the same time each of the users will have different postifx as the random.env would not be part of the repo. Just an inspiration if you would like to optimize your development workflow. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org