potiuk commented on code in PR #54449: URL: https://github.com/apache/airflow/pull/54449#discussion_r2287514266
########## shared/secrets_masker/pyproject.toml: ########## @@ -0,0 +1,47 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless required by applicable law or agreed to in writing, +# software distributed under the License is distributed on an +# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY +# KIND, either express or implied. See the License for the +# specific language governing permissions and limitations +# under the License. + +[project] +name = "apache-airflow-shared-secrets_masker" +description = "Shared secrets masker code for Airflow distributions" +version = "0.0" +classifiers = [ + "Private :: Do Not Upload", +] + +dependencies = [ + # No external dependencies - only standard library +] Review Comment: This should be added here. This does not solve all the issues with some implicit dependencies but it shows them: ```suggestion dependencies = [ 'pendulum>=3.1.0', "methodtools>=0.4.7", "colorlog>=6.8.2", ] [dependency-groups] dev = [ "apache-airflow-devel-common", ] ``` Explanation: The "apache-alirflow-devel-common" - adds all the common "test" dependencies (pytest and friends). This allows to do this: ```bash cd shared/secrets_masker uv run pytest ``` What happens here is: * uv perform `uv sync` for secrets_masker * which means it uninstalls from venv everything that is *not* in dependencies or dev dependencies of secrets_masker * then it runs pytests This is how I found that methodtools and pendulum and colorlog are in fact dependencies of secrets_masker -> `uv run pytest` failed importing without them. Currently, when you do this - there is another problem to be solved: ```python /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:408: in resolve found = getattr(found, frag) E AttributeError: module 'airflow._shared' has no attribute 'secrets_masker' During handling of the above exception, another exception occurred: /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:410: in resolve self.importer(used) ../../airflow-core/src/airflow/_shared/secrets_masker/__init__.py:19: in <module> from airflow.sdk._shared.secrets_masker.secrets_masker import ( E ModuleNotFoundError: No module named 'airflow.sdk' The above exception was the direct cause of the following exception: /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:596: in configure filters[name] = self.configure_filter(filters[name]) /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:742: in configure_filter result = self.configure_custom(config) /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:487: in configure_custom c = self.resolve(c) /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:415: in resolve raise v from e E ValueError: Cannot resolve 'airflow._shared.secrets_masker.secrets_masker._secrets_masker': No module named 'airflow.sdk' The above exception was the direct cause of the following exception: tests/secrets_masker/test_secrets_masker.py:32: in <module> from airflow_shared.secrets_masker.secrets_masker import ( src/airflow_shared/secrets_masker/__init__.py:19: in <module> from airflow.sdk._shared.secrets_masker.secrets_masker import ( ../../airflow-core/src/airflow/__init__.py:79: in <module> settings.initialize() ../../airflow-core/src/airflow/settings.py:640: in initialize LOGGING_CLASS_PATH = configure_logging() ../../airflow-core/src/airflow/logging_config.py:108: in configure_logging raise e ../../airflow-core/src/airflow/logging_config.py:103: in configure_logging dictConfig(logging_config) /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:942: in dictConfig dictConfigClass(config).configure() /home/jarek/.local/share/uv/python/cpython-3.12.7-linux-x86_64-gnu/lib/python3.12/logging/config.py:598: in configure raise ValueError('Unable to configure ' E ValueError: Unable to configure filter 'mask_secrets_core' --------------------------------------------------------------------- ``` This menas that we have in fact implicit dependency between "secrets_masker" and "airflow.sdk" which we should get rid of - in order to make "secrets masker" truly standalone. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
