[PR] Providers requirements for every python version [airflow]

pierrejeambrun opened a new pull request, #35086: URL: https://github.com/apache/airflow/pull/35086 Allow to generate provider requirements for all python version. Use every python version compatible with the `associated_airflow_version` of the provider. Parallelize image building for

Re: [PR] Providers requirements for every python version [airflow]

hussein-awala commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367466010 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS = ["2017-latest

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367512121 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS = ["2017-latest", "201

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367512638 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS = ["2017-latest", "201

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1773450980 Comment 1: It takes a long time to build the images with all the airflow versions in but we can optimize it later by pushing them to ghcr.io and using them as cache. -- This is an au

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1773778101 All looks good I have just a few general comments (and some improvement proposals / test results will come as inline comments). 1. We will need this "all-airflow" images only once

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367724733 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -321,14 +370,14 @@ def generate_providers_requirements( pool.apply_async(

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367725810 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367726001 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367730181 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367730404 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367730805 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367733237 ## dev/breeze/src/airflow_breeze/utils/cdxgen.py: ## @@ -217,28 +231,41 @@ def get_requirements_for_provider( ) -def build_all_airflow_versions_base_image(pyt

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367733237 ## dev/breeze/src/airflow_breeze/utils/cdxgen.py: ## @@ -217,28 +231,41 @@ def get_requirements_for_provider( ) -def build_all_airflow_versions_base_image(pyt

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367733237 ## dev/breeze/src/airflow_breeze/utils/cdxgen.py: ## @@ -217,28 +231,41 @@ def get_requirements_for_provider( ) -def build_all_airflow_versions_base_image(pyt

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367733237 ## dev/breeze/src/airflow_breeze/utils/cdxgen.py: ## @@ -217,28 +231,41 @@ def get_requirements_for_provider( ) -def build_all_airflow_versions_base_image(pyt

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367730805 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1773876805 Thank you for the reviews. I agree, both updating the release process and image caching are important and should be done in follow up steps. @potiuk I will address t

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367922210 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS = ["2017-lates

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367923147 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -321,14 +370,14 @@ def generate_providers_requirements( pool.apply_async

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367924567 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367939699 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_JSON_FIL

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367948070 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367948070 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367948070 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367949177 ## dev/breeze/src/airflow_breeze/utils/cdxgen.py: ## @@ -217,28 +231,41 @@ def get_requirements_for_provider( ) -def build_all_airflow_versions_base_i

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367949177 ## dev/breeze/src/airflow_breeze/utils/cdxgen.py: ## @@ -217,28 +231,41 @@ def get_requirements_for_provider( ) -def build_all_airflow_versions_base_i

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367970934 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -291,25 +291,74 @@ def generate_providers_requirements( with open(PROVIDER_METADATA_

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1774201272 Made a few adjustment to allow for more flexibility in the provider_ids/provider_versions combo. To handle: - All historical version for providers - latest version (provider

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1774989188 > Made a few adjustments to allow for more flexibility in the provider_ids/provider_versions combo. > > This way we can: > > * Regenerate requirements for all historical ver

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1368519420 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -236,23 +241,83 @@ def update_sbom_information( ) +@sbom.command(name="build-all-air

Re: [PR] Providers requirements for every python version [airflow]

potiuk commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1776114632 Images need to be regenerated after my changes :( -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above t

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1369322850 ## dev/breeze/src/airflow_breeze/commands/sbom_commands.py: ## @@ -236,23 +241,83 @@ def update_sbom_information( ) +@sbom.command(name="build

Re: [PR] Providers requirements for every python version [airflow]

pierrejeambrun merged PR #35086: URL: https://github.com/apache/airflow/pull/35086 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@a