potiuk commented on code in PR #45131:
URL: https://github.com/apache/airflow/pull/45131#discussion_r1894589431
##########
dev/breeze/src/airflow_breeze/utils/selective_checks.py:
##########
@@ -326,30 +326,28 @@ def __hash__(self):
def find_provider_affected(changed_file: str, include_docs: bool) -> str |
None:
file_path = AIRFLOW_SOURCES_ROOT / changed_file
- # is_relative_to is only available in Python 3.9 - we should simplify this
check when we are Python 3.9+
- for provider_root in (TESTS_PROVIDERS_ROOT, SYSTEM_TESTS_PROVIDERS_ROOT,
AIRFLOW_PROVIDERS_NS_PACKAGE):
- try:
- file_path.relative_to(provider_root)
- relative_base_path = provider_root
+ # Check providers in SRC/SYSTEM_TESTS/TESTS/(optionally) DOCS
+ for provider_root in (SYSTEM_TESTS_PROVIDERS_ROOT, TESTS_PROVIDERS_ROOT,
AIRFLOW_PROVIDERS_NS_PACKAGE):
+ if file_path.is_relative_to(provider_root):
+ provider_base_path = provider_root
break
- except ValueError:
- pass
else:
- if include_docs:
- try:
- relative_path = file_path.relative_to(DOCS_DIR)
- if
relative_path.parts[0].startswith("apache-airflow-providers-"):
- return
relative_path.parts[0].replace("apache-airflow-providers-", "").replace("-",
".")
- except ValueError:
- pass
+ if include_docs and file_path.is_relative_to(DOCS_DIR):
+ relative_path = file_path.relative_to(DOCS_DIR)
+ if relative_path.parts[0].startswith("apache-airflow-providers-"):
+ return
relative_path.parts[0].replace("apache-airflow-providers-", "").replace("-",
".")
return None
+ # Find if the path under src/system tests/tests belongs to provider or is
a common code across
+ # multiple providers
for parent_dir_path in file_path.parents:
- if parent_dir_path == relative_base_path:
+ if parent_dir_path == provider_base_path:
+ # We have not found any provider specific path up to the root of
the provider base folder
break
- relative_path = parent_dir_path.relative_to(relative_base_path)
+ relative_path = parent_dir_path.relative_to(provider_base_path)
+ # check if this path belongs to a specific provider
if (AIRFLOW_PROVIDERS_NS_PACKAGE / relative_path /
"provider.yaml").exists():
Review Comment:
Actually this is already covered by the new test I added.
The problematic one was "providers/tests/system/google/...." not the
"providers/tests/google" and it was because we checked it in wrong order.
We originally checked it like that:
```
(TESTS_PROVIDERS_ROOT, SYSTEM_TESTS_PROVIDERS_ROOT,
AIRFLOW_PROVIDERS_NS_PACKAGE)
```
And the fix is here:
```
(SYSTEM_TESTS_PROVIDERS_ROOT, TESTS_PROVIDERS_ROOT,
AIRFLOW_PROVIDERS_NS_PACKAGE)
```
It worked like that:
"providers/tests/system/.... " -> root = "providers/tests". relative path =
"system/google/....." - because TESTS were checked before SYSTEM_TESTS and
SYSTEM_TESTS are sub-folder of TESTS.
Then the "if (AIRFLOW_PROVIDERS_NS_PACKAGE / relative_path /
"provider.yaml") " could never work (because it was checking for
"airflow/providers/src/airflow/providers/system/google/provider.yaml" ) and it
was determined as "common files modified"
By reverting the folder check sequence it now works fine:
"providers/tests/system/ ...." -> root = "providers/tests/system", relative
path = "google/..." and it would correctly find
"airflow/providers/src/airflow/providers/google/provider.yaml"
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
