amoghrajesh commented on PR #43040:
URL: https://github.com/apache/airflow/pull/43040#issuecomment-2416983638
@ashb @potiuk @kaxil I changed the logic to incorporate what was suggested
above. Now I am able to get this result:
```
[2024-10-16, 18:39:13 IST] {logging_mixin.py:190} INFO - environ({'SHELL':
'/bin/bash', 'AIRFLOW__SMTP__SMTP_PORT': '587', 'VERSION_SUFFIX_FOR_PYPI': '',
'NUM_RUNS': '', 'AIRFLOW__CELERY__RESULT_BACKEND': 'db+***',
'AIRFLOW_AUTH_MANAGER_CREDENTIAL_DIRECTORY': '/files', 'USE_AIRFLOW_VERSION':
'', 'AIRFLOW__SMTP__SMTP_PASSWORD': 'dummy_smtp_password',
'AIRFLOW__SMTP__SMTP_USER': 'dummy_user', 'DB_RESET': 'false',
'TERM_PROGRAM_VERSION': '3.3a', 'CASS_DRIVER_NO_CYTHON': '1',
'DOCKER_IS_ROOTLESS': 'false', 'TMUX': '/root/.tmux/tmp/tmux-0/default,96,0',
'DEPENDENCIES_EPOCH_NUMBER': '11', 'HOSTNAME': '22a3f86e4c95', 'COLOR_RESET':
'\x1b[0m', 'AIRFLOW__WEBSERVER__SECRET_KEY_SECRET': '***_secret',
'PYTHON_VERSION': '3.9.20', 'LANGUAGE': 'C.UTF-8', 'HOST_USER_ID': '20',
'FASTAPI_API_HOST_PORT': '29091', 'USE_PACKAGES_FROM_DIST': 'false',
'COLLECT_ONLY': 'false', 'PACKAGE_FORMAT': 'wheel',
'AIRFLOW__CORE__PLUGINS_FOLDER': '/files/plugins', 'AIRFLOW_CI_BUILD_EPOCH':
'10', 'CHICKEN_EGG_PROVID
ERS': '', 'MSSQL_HOST_PORT': '21433', 'SQLALCHEMY_WARN_20': 'true',
'CELERY_BROKER_URLS_MAP': "{'rabbitmq': 'amqp://guest:guest@rabbitmq:5672',
'redis': 'redis://redis:6379/0'}", 'AIRFLOW_CONSTRAINTS_LOCATION': '',
'DEV_MODE': 'false', 'AIRFLOW__CORE__DAGS_FOLDER': '/files/dags', 'COLOR_RED':
'\x1b[31m', 'RUN_SYSTEM_TESTS': 'false', 'COLOR_BLUE': '\x1b[34m',
'AIRFLOW_ENABLE_AIP_44': 'true', 'AIRFLOW_CI_IMAGE':
'ghcr.io/apache/airflow/main/ci/python3.9', 'TEST_TYPE': '', 'ADDITIONAL_PATH':
'~/.local/bin', 'EAGER_UPGRADE_ADDITIONAL_REQUIREMENTS': '',
'PROVIDERS_SKIP_CONSTRAINTS': 'false', 'BREEZE_INIT_COMMAND': '', 'PWD':
'/opt/airflow', 'AIRFLOW_SKIP_CONSTRAINTS': 'false',
'AIRFLOW__CELERY__BROKER_URL': 'redis://redis:6379/0', 'REMOVE_ARM_PACKAGES':
'false', 'AIRFLOW_VERSION': '3.0.0.dev0', 'ENABLE_COVERAGE': 'false',
'VERBOSE_COMMANDS': 'false', 'AIRFLOW__CORE__LOAD_EXAMPLES': 'false',
'GITHUB_ACTIONS': 'false', 'INSTALL_MSSQL_CLIENT': 'true',
'CASS_DRIVER_BUILD_CONCURRENCY': '8', '
CI_EVENT_TYPE': 'pull_request', 'UPGRADE_IF_NEEDED': '--upgrade',
'INSTALL_MYSQL_CLIENT_TYPE': 'mariadb', 'BACKEND': 'mysql', 'CI_TARGET_BRANCH':
'main', 'PROVIDERS_CONSTRAINTS_MODE': 'constraints-source-providers',
'GUNICORN_CMD_ARGS': '--worker-tmp-dir /dev/shm/', 'AIRFLOW_EXTRAS': '',
'LD_PRELOAD': '/usr/lib/aarch64-linux-gnu/libstdc++.so.6', 'PIP_PROGRESS_BAR':
'on', 'AIRFLOW__DATABASE__SQL_ALCHEMY_CONN_SECRET': 'dummy_sql_alchemy_conn',
'HOME': '/root', 'QUIET': 'false', 'UV_NO_CACHE': 'true', 'LANG': 'C.UTF-8',
'SUSPENDED_PROVIDERS_FOLDERS': '', 'HELM_TEST_PACKAGE': '', 'TMUX_TMPDIR':
'/root/.tmux/tmp', 'SQLITE_URL': 'sqlite:////root/airflow/sqlite/airflow.db',
'ISSUE_ID': '', 'AIRFLOW_HOME': '/root/airflow', 'MYSQL_VERSION': '8.0',
'AIRFLOW_BREEZE_CONFIG_DIR': '/files/airflow-breeze-config',
'POSTGRES_HOST_PORT': '25433', 'EXTRA_UNINSTALL_FLAGS': '--python
/usr/local/bin/python', 'AIRFLOW_VERSION_SPECIFICATION': '', 'GPG_KEY':
'E3FF2839C048B25C084DEBE9B26995E310250568', 'AIRF
LOW__DATABASE__SQL_ALCHEMY_CONN': '***', 'AIRFLOW_USE_UV': 'true',
'AIRFLOW_PRE_CACHED_PIP_PACKAGES': 'true', 'ADDITIONAL_DEV_APT_DEPS':
'bash-completion dumb-init git graphviz krb5-user less libenchant-2-2
libgcc-11-dev libgeos-dev libpq-dev net-tools netcat-openbsd openssh-server
postgresql-client software-properties-common rsync tmux unzip vim xxd',
'COLOR_YELLOW': '\x1b[33m', 'ONLY_MIN_VERSION_UPDATE': 'false',
'AIRFLOW__CELERY__WORKER_CONCURRENCY': '8', 'STANDALONE_DAG_PROCESSOR':
'false', 'AIRFLOW__CORE__EXECUTOR': 'LocalExecutor',
'INSTALL_SELECTED_PROVIDERS': '', 'USE_XDIST': 'false', 'COMMIT_SHA':
'c03ccfc0716fdbf88eb778c920767b4d3953ac5b', 'CONSTRAINTS_GITHUB_REPOSITORY':
'apache/airflow', 'AIRFLOW__DATABASE__SQL_ALCHEMY_ENGINE_ARGS_SECRET':
'dummy_sql_engine_args', 'EXTRA_INSTALL_FLAGS': '--python
/usr/local/bin/python', 'AIRFLOW_CI_IMAGE_WITH_TAG':
'ghcr.io/apache/airflow/main/ci/python3.9:latest', 'SKIP_SSH_SETUP': 'false',
'TMUX_SESSION': 'Airflow', 'AIRFLOW_REPO': 'ap
ache/airflow', 'DEV_APT_COMMAND': '', 'AIRFLOW__WEBSERVER__SECRET_KEY': '***',
'AIRFLOW_PIP_VERSION': '24.2', 'ADDITIONAL_PIP_INSTALL_FLAGS': '',
'SKIP_ENVIRONMENT_INITIALIZATION': 'false', 'ADDITIONAL_DEV_APT_COMMAND': '',
'AIRFLOW_ENV': 'development', 'START_AIRFLOW': 'true', 'COLOR_GREEN':
'\x1b[32m', 'AIRFLOW_BRANCH': 'main', 'CLEAN_AIRFLOW_INSTALLATION': 'false',
'FORCE_LOWEST_DEPENDENCIES': 'false',
'AIRFLOW__CORE__INTERNAL_API_SECRET_KEY_SECRET': '***_secret', 'HOST_GROUP_ID':
'20', 'DEFAULT_BRANCH': 'main', 'INIT_SCRIPT_FILE': 'init.sh',
'AIRFLOW_CONSTRAINTS_MODE': 'constraints-source-providers', 'PYTHONPATH':
'/opt/airflow', 'TERM': 'tmux-256color', 'UPGRADE_BOTO': 'false',
'DOWNGRADE_PENDULUM': 'false', 'USER': 'root', 'HOST_OS': 'darwin', 'USE_UV':
'true', 'TMUX_PANE': '%1', 'WEBSERVER_HOST_PORT': '28080',
'AIRFLOW_UV_VERSION': '0.4.22', 'AIRFLOW__CORE__INTERNAL_API_SECRET_KEY':
'***', 'INSTALL_POSTGRES_CLIENT': 'true', 'COMPOSE_FILE':
'/Users/adesai/Documents/OSS/airflow
-repos/airflow/scripts/ci/docker-compose/base.yml:/Users/adesai/Documents/OSS/airflow-repos/airflow/scripts/ci/docker-compose/docker-socket.yml:/Users/adesai/Documents/OSS/airflow-repos/airflow/scripts/ci/docker-compose/backend-mysql.yml:/Users/adesai/Documents/OSS/airflow-repos/airflow/scripts/ci/docker-compose/backend-mysql-port.yml:/Users/adesai/Documents/OSS/airflow-repos/airflow/scripts/ci/docker-compose/files.yml:/Users/adesai/Documents/OSS/airflow-repos/airflow/scripts/ci/docker-compose/base-ports.yml:/Users/adesai/Documents/OSS/airflow-repos/airflow/scripts/ci/docker-compose/local.yml',
'SHLVL': '2', 'AIRFLOW_CONSTRAINTS_REFERENCE': 'constraints-main',
'DEFAULT_CONSTRAINTS_BRANCH': 'constraints-main', 'FILES_DIR': '/files',
'CI_BUILD_ID': '0', 'INSTALL_AIRFLOW_VERSION': '', 'LC_MESSAGES': 'C.UTF-8',
'SYSTEM_TESTS_ENV_ID': '', 'MOUNT_SOURCES': 'selected',
'PYTHONDONTWRITEBYTECODE': 'true', 'KUBECONFIG': '/files/.kube/config',
'UPGRADE_INVALIDATION_STRING': '', 'MYSQL_HOST_POR
T': '23306', 'LC_CTYPE': 'C.UTF-8', 'AIRFLOW_IMAGE_KUBERNETES':
'ghcr.io/apache/airflow/main/kubernetes/python3.9', 'ANSWER': '', 'RUN_TESTS':
'false', 'LOAD_EXAMPLES': 'false', '_AIRFLOW_SKIP_DB_TESTS': 'false',
'DOWNGRADE_SQLALCHEMY': 'false', 'AIRFLOW_INSTALLATION_METHOD': '.',
'REDIS_HOST_PORT': '26379', 'LC_ALL': 'C.UTF-8', 'CI_TARGET_REPO':
'apache/airflow', 'CI_JOB_ID': '0', 'DATABASE_ISOLATION': 'false',
'INSTALL_MYSQL_CLIENT': 'true', 'SSH_PORT': '12322', 'PATH':
'/files/bin/:/opt/airflow/scripts/in_container/bin/:/root/.local/bin:/usr/local/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/opt/airflow',
'PYTHON_MAJOR_MINOR_VERSION': '3.9', '_AIRFLOW_RUN_DB_TESTS_ONLY': 'false',
'AIRFLOW__CORE__DATASET_MANAGER_KWARGS_SECRET': 'dummy_dataset_manager_kwargs',
'UPGRADE_EAGERLY': '--upgrade --resolution highest', 'CELERY_FLOWER': 'false',
'INSTALL_AIRFLOW_WITH_CONSTRAINTS': 'true', 'CI': 'false', 'PACKAGING_TOOL':
'', 'PACKAGING_TOOL_CMD': 'uv pip', 'LOAD_DEFAUL
T_CONNECTIONS': 'false', 'FLOWER_HOST_PORT': '25555', 'PIP_NO_CACHE_DIR':
'true', 'BASE_BRANCH': 'main', 'PYTHON_BASE_IMAGE': 'python:3.9-slim-bookworm',
'ENABLED_SYSTEMS': '', 'AIRFLOW__LOGGING__REMOTE_TASK_HANDLER_KWARGS_SECRET':
'dummy_remote_task_handler_kwargs', 'AIRFLOW__SMTP__SMTP_PASSWORD_SECRET':
'dummy_smtp_password_secret', 'UV_HTTP_TIMEOUT': '300', 'BREEZE': 'true',
'DRILL_HOST_PORT': '28047', 'BUILD_ID': '0',
'AIRFLOW__SENTRY__SENTRY_DSN_SECRET': 'dummy_sentry_dsn',
'AIRFLOW__CORE__FERNET_KEY': '***', 'POSTGRES_VERSION': '12',
'DEBIAN_FRONTEND': 'noninteractive', 'EXCLUDED_PROVIDERS': '',
'UV_CONCURRENT_DOWNLOADS': '10', 'OLDPWD': '/opt/airflow',
'AIRFLOW__SMTP__SMTP_SSL': 'True', 'AIRFLOW_SOURCES': '/opt/airflow',
'VERBOSE': 'false', 'TERM_PROGRAM': 'tmux', 'AIRFLOW__SMTP__SMTP_HOST':
'smtp.dummy.com', 'REGENERATE_MISSING_DOCS': 'false', '_':
'/usr/local/bin/airflow', '_AIRFLOW_PARSING_CONTEXT_DAG_ID': 'os-env-variable',
'_AIRFLOW_PARSING_CONTEXT_TASK_ID': 'test_me', '
AIRFLOW_CTX_DAG_OWNER': 'airflow', 'AIRFLOW_CTX_DAG_ID': 'os-env-variable',
'AIRFLOW_CTX_TASK_ID': 'test_me', 'AIRFLOW_CTX_EXECUTION_DATE':
'2024-10-16T13:09:12.855444+00:00', 'AIRFLOW_CTX_TRY_NUMBER': '1',
'AIRFLOW_CTX_DAG_RUN_ID': 'manual__2024-10-16T13:09:12.855444+00:00'})
```
From my original list, these aren't masked:
```
"AIRFLOW__SMTP__SMTP_PORT",
"AIRFLOW__SMTP__SMTP_PASSWORD",
"AIRFLOW__SMTP__SMTP_PASSWORD_SECRET",
"AIRFLOW__SMTP__SMTP_USER",
"AIRFLOW__SMTP__SMTP_SSL",
"AIRFLOW__SMTP__SMTP_HOST",
"AIRFLOW__SENTRY__SENTRY_DSN_SECRET",
"AIRFLOW__DATABASE__SQL_ALCHEMY_CONN_SECRET",
"AIRFLOW__DATABASE__SQL_ALCHEMY_ENGINE_ARGS_SECRET",
"AIRFLOW__CORE__DATASET_MANAGER_KWARGS_SECRET",
"AIRFLOW__LOGGING__REMOTE_TASK_HANDLER_KWARGS_SECRET",
```
As per the comments above, looks like `AIRFLOW__SMTP__SMTP_PORT`,
`AIRFLOW__SMTP__SMTP_USER` (removed), `AIRFLOW__SMTP__SMTP_PASSWORD` (removed),
arent relevant.
I think:
- AIRFLOW__SMTP__SMTP_USER
- AIRFLOW__SMTP__SMTP_SSL
- AIRFLOW__SMTP__SMTP_HOST
Are ok to be unmasked. That leaves us with:
```
"AIRFLOW__SENTRY__SENTRY_DSN_SECRET",
"AIRFLOW__DATABASE__SQL_ALCHEMY_CONN_SECRET",
"AIRFLOW__DATABASE__SQL_ALCHEMY_ENGINE_ARGS_SECRET",
"AIRFLOW__CORE__DATASET_MANAGER_KWARGS_SECRET",
"AIRFLOW__LOGGING__REMOTE_TASK_HANDLER_KWARGS_SECRET",
```
Should these be masked or its alright to keep them unmasked?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
