potiuk commented on PR #42971: URL: https://github.com/apache/airflow/pull/42971#issuecomment-2409087469
> @potiuk could we (safely) change dev/ to dev/breeze/ in that target pr part? I'm not sure what else actually lives in dev folder and am not at my computer right now Not really - dev is really a bag-of-all-things related to building our software - and it's a dangerous thing to only override part of it. There are a number of scripts that are there that are **likely** to be used in CI processes. This is for example why we have "scripts/ci" - because there I made sure (and I am reviewing that) that only CI scripts are used there). The current content of `dev` is this (so there are a number of sub-folders - `breeze` is only one of them, and it's very likely some other scripts (`provider_packages` for example) are "potentially dangerous": ``` chart example_dags images mypy perf provider_packages stats system_tests templates __init__.py airflow-github airflow-license assign_cherry_picked_prs_with_milestone.py check_files.py CHERRY_PICK_SUMMARY.txt.jinja2 get_devel_deps.sh ISSUE_TEMPLATE.md.jinja2 MANUALLY_BUILDING_IMAGES.md MANUALLY_GENERATING_IMAGE_CACHE_AND_CONSTRAINTS.md prepare_bulk_issues.py PROJECT_GUIDELINES.md PROVIDER_PACKAGE_DETAILS.md README.md README_RELEASE_AIRFLOW.md README_RELEASE_HELM_CHART.md README_RELEASE_PROVIDER_PACKAGES.md README_RELEASE_PYTHON_CLIENT.md refresh_images.sh remove_artifacts.sh requirements.txt retag_docker_images.py send_email.py sign.sh validate_version_added_fields_in_config.py WHAT_GOES_INTO_THE_NEXT_RELEASE.md ``` And for example "system_tests" there is not something that "system_tests" use but some "dev/build scripts" that are related to system_tests. We already had the issue when accidental typo or mistake cause https://medium.com/apache-airflow/unraveling-the-code-navigating-a-ci-release-security-vulnerability-in-apache-airflow-620214a96297 so I think it's better to be safe than sorry and exclude whole "dev" and designate it as "build scripts". Conceptually - if we look at that the "common_tests" belong to "application" not to "building appplication". Ideally we should have something like that: ``` airflow_core/src airflow tests airlfow pyproject.toml airflow_common/ src (?) tests pyproject.toml providers/ airbyte/ src tests pyproject.toml ``` But it would require the "heavy" airflow move. And maybe we can do it eventually after we split providers and go through all other teething problems. But maybe let's do it step-by-step - moving to `tests_common` at top level would be a good intermediate step. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
