Re: [I] Conditional Task Skip option [airflow]

2023-10-20 Thread via GitHub
abhishekbhakat commented on issue #35089: URL: https://github.com/apache/airflow/issues/35089#issuecomment-1773678144 Why don't you just set different environment variables for your environments and use AirflowSkipException to skip a task. Something like: ```python if

[PR] Replace numpy example with practical exercise demonstrating top-level code [airflow]

2023-10-20 Thread via GitHub
RNHTTR opened a new pull request, #35097: URL: https://github.com/apache/airflow/pull/35097 I'm not sure the numpy example is still accurate. I tested out importing numpy locally and it was more or less instantaneous. I think this can cause some confusion implying that no imports should be

Re: [I] BigQueryValueCheckOperator does not accept non-default project_id, which lead to jobs with impersonation_chain to fail. [airflow]

2023-10-20 Thread via GitHub
vchiapaikeo commented on issue #32870: URL: https://github.com/apache/airflow/issues/32870#issuecomment-1773607615 Happy to take a look at this. Feel free to assign me! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
ferruzzi commented on PR #34381: URL: https://github.com/apache/airflow/pull/34381#issuecomment-1773550988 working on the merge conflict now -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

Re: [PR] PoC: Implement Docker Registry Auth Protocol [airflow]

2023-10-20 Thread via GitHub
github-actions[bot] commented on PR #33602: URL: https://github.com/apache/airflow/pull/33602#issuecomment-1773544091 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] control task group concurrency: use allow list for mapping index [airflow]

2023-10-20 Thread via GitHub
github-actions[bot] commented on PR #32122: URL: https://github.com/apache/airflow/pull/32122#issuecomment-1773544138 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] control task group concurrency: use allow list for mapping index [another way] [airflow]

2023-10-20 Thread via GitHub
github-actions[bot] commented on PR #34067: URL: https://github.com/apache/airflow/pull/34067#issuecomment-1773544056 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367564162 ## airflow/providers/amazon/aws/config_templates/config.yml: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367563594 ## airflow/providers/amazon/aws/config_templates/config.yml: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one Review Comment:

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367562842 ## airflow/providers/amazon/aws/config_templates/config.yml: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367562674 ## airflow/providers/amazon/aws/config_templates/config.yml: ## @@ -0,0 +1,131 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more

Re: [PR] Prevent assignment of non JSON serializable values to DagRun.conf dict [airflow]

2023-10-20 Thread via GitHub
boring-cyborg[bot] commented on PR #35096: URL: https://github.com/apache/airflow/pull/35096#issuecomment-1773530032 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[PR] Prevent assignment of non JSON serializable values to DagRun.conf dict [airflow]

2023-10-20 Thread via GitHub
PashkPashk opened a new pull request, #35096: URL: https://github.com/apache/airflow/pull/35096 Closes #35095 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367561414 ## airflow/providers/amazon/aws/executors/ecs/Dockerfile: ## @@ -0,0 +1,86 @@ +# hadolint ignore=DL3007 +FROM apache/airflow:latest +USER root +RUN apt-get update

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367561202 ## airflow/providers/amazon/aws/executors/ecs/Dockerfile: ## @@ -0,0 +1,86 @@ +# hadolint ignore=DL3007 +FROM apache/airflow:latest +USER root +RUN apt-get update

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367560599 ## airflow/providers/amazon/aws/executors/ecs/Dockerfile: ## @@ -0,0 +1,86 @@ +# hadolint ignore=DL3007 +FROM apache/airflow:latest +USER root +RUN apt-get update

Re: [I] Assigning not json serializable value to dagrun.conf cause an error in UI [airflow]

2023-10-20 Thread via GitHub
boring-cyborg[bot] commented on issue #35095: URL: https://github.com/apache/airflow/issues/35095#issuecomment-1773495388 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Assigning not json serializable value to dagrun.conf cause an error in UI [airflow]

2023-10-20 Thread via GitHub
PashkPashk opened a new issue, #35095: URL: https://github.com/apache/airflow/issues/35095 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened Greetings! Recently I’ve faced a problem. It seems that assigning object, which can’t

[airflow] branch main updated: Add decorators for external and venv python branching operators (#35043)

2023-10-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c2a733aa95 Add decorators for external and venv

Re: [PR] Add decorators for external and venv python branching operators [airflow]

2023-10-20 Thread via GitHub
potiuk merged PR #35043: URL: https://github.com/apache/airflow/pull/35043 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Add decorators for external and venv python branching operators [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35043: URL: https://github.com/apache/airflow/pull/35043#issuecomment-1773469978 All Good. Failed check fixed in main, -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #34964: URL: https://github.com/apache/airflow/pull/34964#issuecomment-1773465545 Static check fixed in main -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367523179 ## airflow/providers/amazon/aws/executors/ecs/README.md: ## @@ -0,0 +1,196 @@ + + +# AWS ECS Executor + +This is an Airflow executor powered by Amazon Elastic

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367522571 ## airflow/providers/amazon/aws/executors/ecs/Dockerfile: ## @@ -0,0 +1,103 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

Re: [PR] Add Pinecone Provider [airflow]

2023-10-20 Thread via GitHub
utkarsharma2 commented on code in PR #35094: URL: https://github.com/apache/airflow/pull/35094#discussion_r1367519800 ## docs/apache-airflow-providers-pinecone/index.rst: ## @@ -0,0 +1,101 @@ + + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367518892 ## airflow/settings.py: ## @@ -578,6 +578,10 @@ def initialize(): executor_constants.CELERY_KUBERNETES_EXECUTOR,

Re: [PR] AWS ECS Executor [airflow]

2023-10-20 Thread via GitHub
o-nikolas commented on code in PR #34381: URL: https://github.com/apache/airflow/pull/34381#discussion_r1367518477 ## airflow/utils/log/logging_mixin.py: ## @@ -209,9 +209,9 @@ def __init__(self, stream): @property def stream(self): """Returns current

[PR] Add Pinecone Provider [airflow]

2023-10-20 Thread via GitHub
utkarsharma2 opened a new pull request, #35094: URL: https://github.com/apache/airflow/pull/35094 This PR is part of our larger effort to add first-class integrations to support LLMOps that was [presented at Airflow Summit](https://www.youtube.com/watch?v=mgA6m3ggKhs=4s). In this PR

Re: [PR] Describe workaround to packaging 23.2 breaking pipx local detection [airflow]

2023-10-20 Thread via GitHub
potiuk merged PR #35092: URL: https://github.com/apache/airflow/pull/35092 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Describe workaround to packaging 23.2 breaking pipx local detection (#35092)

2023-10-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 4e460bf083 Describe workaround to packaging 23.2

Re: [PR] Providers requirements for every python version [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35086: URL: https://github.com/apache/airflow/pull/35086#issuecomment-1773450980 Comment 1: It takes a long time to build the images with all the airflow versions in but we can optimize it later by pushing them to ghcr.io and using them as cache. -- This is an

Re: [PR] Providers requirements for every python version [airflow]

2023-10-20 Thread via GitHub
potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367512638 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS = ["2017-latest",

Re: [PR] Providers requirements for every python version [airflow]

2023-10-20 Thread via GitHub
potiuk commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367512121 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS = ["2017-latest",

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-20 Thread via GitHub
cjames23 commented on code in PR #34874: URL: https://github.com/apache/airflow/pull/34874#discussion_r1367508552 ## airflow/providers/google/cloud/operators/cloud_run.py: ## @@ -270,6 +271,7 @@ def __init__( project_id: str, region: str, job_name:

Re: [PR] Add operator to invoke Azure-Synapse pipeline [airflow]

2023-10-20 Thread via GitHub
utkarsharma2 commented on code in PR #35091: URL: https://github.com/apache/airflow/pull/35091#discussion_r1367506829 ## .pre-commit-config.yaml: ## @@ -992,14 +992,14 @@ repos: exclude:

Re: [PR] Add operator to invoke Azure-Synapse pipeline [airflow]

2023-10-20 Thread via GitHub
utkarsharma2 commented on code in PR #35091: URL: https://github.com/apache/airflow/pull/35091#discussion_r1367506102 ## .pre-commit-config.yaml: ## @@ -39,7 +39,7 @@ repos: - "--maxlevel" - "2" - repo: https://github.com/Lucas-C/pre-commit-hooks -

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-20 Thread via GitHub
ferruzzi commented on code in PR #35037: URL: https://github.com/apache/airflow/pull/35037#discussion_r1367505181 ## airflow/providers/amazon/aws/hooks/s3.py: ## @@ -912,14 +912,24 @@ def get_key(self, key: str, bucket_name: str | None = None) -> S3ResourceObject:

Re: [PR] Update providers metadata for SBOM generation [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35093: URL: https://github.com/apache/airflow/pull/35093#issuecomment-1773424575 Smth wrong with the numbers :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

Re: [PR] Update providers metadata for SBOM generation [airflow]

2023-10-20 Thread via GitHub
potiuk closed pull request #35093: Update providers metadata for SBOM generation URL: https://github.com/apache/airflow/pull/35093 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

[PR] Update providers metadata for SBOM generation [airflow]

2023-10-20 Thread via GitHub
potiuk opened a new pull request, #35093: URL: https://github.com/apache/airflow/pull/35093 --- **^ Add meaningful description above** Read the **[Pull Request

Re: [PR] Add Cohere Provider [airflow]

2023-10-20 Thread via GitHub
utkarsharma2 commented on code in PR #34921: URL: https://github.com/apache/airflow/pull/34921#discussion_r1367491930 ## docs/apache-airflow-providers-cohere/index.rst: ## @@ -0,0 +1,105 @@ + + .. Licensed to the Apache Software Foundation (ASF) under one +or more

[PR] Describe workaround to packaging 23.2 breaking pipx local detection [airflow]

2023-10-20 Thread via GitHub
potiuk opened a new pull request, #35092: URL: https://github.com/apache/airflow/pull/35092 The latest packaging - 23.2 breaks `pipx` detection of local path installation and renders breeze installation impossible. The workaround to it until https://github.com/pypa/pipx/issues/1092 is

[PR] Add operator to invoke Azure-Synapse pipeline [airflow]

2023-10-20 Thread via GitHub
ambika-garg opened a new pull request, #35091: URL: https://github.com/apache/airflow/pull/35091 --- **^ Add meaningful description above** Read the **[Pull Request

[PR] Add OpenLineage support for AthenaOperator. [airflow]

2023-10-20 Thread via GitHub
JDarDagran opened a new pull request, #35090: URL: https://github.com/apache/airflow/pull/35090 Based on: https://github.com/OpenLineage/OpenLineage/pull/1328. This PR adds OpenLineage support for AthenaOperator. It includes symlink facet to point to physical

Re: [PR] Consolidate the warning stacklevel in KubernetesPodTrigger [airflow]

2023-10-20 Thread via GitHub
hussein-awala commented on code in PR #35079: URL: https://github.com/apache/airflow/pull/35079#discussion_r1367472697 ## airflow/providers/cncf/kubernetes/triggers/pod.py: ## @@ -102,7 +102,8 @@ def __init__( if should_delete_pod is not None:

Re: [PR] Providers requirements for every python version [airflow]

2023-10-20 Thread via GitHub
hussein-awala commented on code in PR #35086: URL: https://github.com/apache/airflow/pull/35086#discussion_r1367466010 ## dev/breeze/src/airflow_breeze/global_constants.py: ## @@ -221,6 +221,44 @@ def get_default_platform_machine() -> str: CURRENT_MSSQL_VERSIONS =

Re: [I] Conditional Task Skip option [airflow]

2023-10-20 Thread via GitHub
boring-cyborg[bot] commented on issue #35089: URL: https://github.com/apache/airflow/issues/35089#issuecomment-1773331065 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for

[I] Conditional Task Skip option [airflow]

2023-10-20 Thread via GitHub
match-jasoncavnar opened a new issue, #35089: URL: https://github.com/apache/airflow/issues/35089 ### Description Add a keyword option at the task level to conditionally set if the task should be skipped on DAG run. ### Use case/motivation We would like to run our DAG

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
ephraimbuddy commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367441583 ## scripts/ci/docker-compose/devcontainer.env: ## @@ -18,6 +18,7 @@ HOME= AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.8:latest" ANSWER=

Re: [PR] AIP-58: Add Airflow ObjectStore (AS) [airflow]

2023-10-20 Thread via GitHub
potiuk commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367433860 ## docs/apache-airflow/extra-packages-ref.rst: ## @@ -68,6 +68,8 @@ python dependencies for the provided package.

Re: [PR] Feature azure synapse run pipeline operator. [airflow]

2023-10-20 Thread via GitHub
ambika-garg closed pull request #35044: Feature azure synapse run pipeline operator. URL: https://github.com/apache/airflow/pull/35044 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
potiuk commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367428440 ## scripts/ci/docker-compose/devcontainer.env: ## @@ -18,6 +18,7 @@ HOME= AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.8:latest" ANSWER=

Re: [PR] AIP-58: Add Airflow ObjectStore (AS) [airflow]

2023-10-20 Thread via GitHub
bolkedebruin commented on code in PR #34729: URL: https://github.com/apache/airflow/pull/34729#discussion_r1367427142 ## docs/apache-airflow/extra-packages-ref.rst: ## @@ -68,6 +68,8 @@ python dependencies for the provided package.

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-20 Thread via GitHub
ChloeSheasby commented on code in PR #34874: URL: https://github.com/apache/airflow/pull/34874#discussion_r1367412425 ## airflow/providers/google/cloud/operators/cloud_run.py: ## @@ -270,6 +271,7 @@ def __init__( project_id: str, region: str,

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
ephraimbuddy commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367412081 ## scripts/ci/docker-compose/devcontainer.env: ## @@ -18,6 +18,7 @@ HOME= AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.8:latest" ANSWER=

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
potiuk commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367396267 ## scripts/ci/docker-compose/devcontainer.env: ## @@ -18,6 +18,7 @@ HOME= AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.8:latest" ANSWER=

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
potiuk commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367396267 ## scripts/ci/docker-compose/devcontainer.env: ## @@ -18,6 +18,7 @@ HOME= AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.8:latest" ANSWER=

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-20 Thread via GitHub
Taragolis commented on code in PR #35037: URL: https://github.com/apache/airflow/pull/35037#discussion_r1367381836 ## airflow/providers/amazon/aws/hooks/s3.py: ## @@ -912,14 +912,24 @@ def get_key(self, key: str, bucket_name: str | None = None) -> S3ResourceObject:

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
ephraimbuddy commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367379560 ## scripts/ci/docker-compose/devcontainer.env: ## @@ -18,6 +18,7 @@ HOME= AIRFLOW_CI_IMAGE="ghcr.io/apache/airflow/main/ci/python3.8:latest" ANSWER=

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-20 Thread via GitHub
ferruzzi commented on code in PR #35037: URL: https://github.com/apache/airflow/pull/35037#discussion_r1367332799 ## airflow/providers/amazon/aws/hooks/s3.py: ## @@ -912,14 +912,24 @@ def get_key(self, key: str, bucket_name: str | None = None) -> S3ResourceObject:

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-20 Thread via GitHub
vincbeck commented on code in PR #35037: URL: https://github.com/apache/airflow/pull/35037#discussion_r1367328736 ## airflow/providers/amazon/aws/hooks/s3.py: ## @@ -912,14 +912,24 @@ def get_key(self, key: str, bucket_name: str | None = None) -> S3ResourceObject:

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-20 Thread via GitHub
ferruzzi commented on code in PR #35037: URL: https://github.com/apache/airflow/pull/35037#discussion_r1367326682 ## airflow/providers/amazon/aws/hooks/s3.py: ## @@ -912,14 +912,24 @@ def get_key(self, key: str, bucket_name: str | None = None) -> S3ResourceObject:

Re: [PR] Enable encryption in S3 download_files() hook. [airflow]

2023-10-20 Thread via GitHub
vincbeck commented on code in PR #35037: URL: https://github.com/apache/airflow/pull/35037#discussion_r1367313528 ## airflow/providers/amazon/aws/hooks/s3.py: ## @@ -912,14 +912,24 @@ def get_key(self, key: str, bucket_name: str | None = None) -> S3ResourceObject:

[airflow] branch main updated: Remove `mysql-connector-python` from recommended MySQL driver (#34287)

2023-10-20 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 44831db207 Remove `mysql-connector-python` from

Re: [PR] Remove `mysql-connector-python` from recommended MySQL driver [airflow]

2023-10-20 Thread via GitHub
vincbeck merged PR #34287: URL: https://github.com/apache/airflow/pull/34287 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Fix CI's docs filter generation [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35087: URL: https://github.com/apache/airflow/pull/35087#issuecomment-1773112597 Enough tests succeeded to merge it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch main updated: Fix CI's docs filter generation (#35087)

2023-10-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new fdc443d36a Fix CI's docs filter generation

Re: [PR] Fix CI's docs filter generation [airflow]

2023-10-20 Thread via GitHub
potiuk merged PR #35087: URL: https://github.com/apache/airflow/pull/35087 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Allow to install MariaDB client libraries instead of MySQL on x86_64 [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35070: URL: https://github.com/apache/airflow/pull/35070#issuecomment-1773108275 > Since it experimental feature, I guess it could be included into the 2.7.3, am I right? Generally SemVer limitations apply to Airflow, not the image, so yes. We do not "SemVer"

Re: [PR] Fix CI's docs filter generation [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35087: URL: https://github.com/apache/airflow/pull/35087#issuecomment-1773103716 > This case just not happen in main or during full tests that is why we miss that Yeah. Who tests the tests. -- This is an automated message from the Apache Git Service. To

[airflow] branch main updated: Refactor string splitting (#34185)

2023-10-20 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new f816237196 Refactor string splitting (#34185)

Re: [PR] Refactor string splitting [airflow]

2023-10-20 Thread via GitHub
vincbeck merged PR #34185: URL: https://github.com/apache/airflow/pull/34185 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

Re: [PR] Run constraint branch checkout in non-canary build [airflow]

2023-10-20 Thread via GitHub
potiuk merged PR #35088: URL: https://github.com/apache/airflow/pull/35088 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Run constraint branch checkout in non-canary build (#35088)

2023-10-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 570d88b069 Run constraint branch checkout in

Re: [PR] Run constraint branch checkout in non-canary build [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35088: URL: https://github.com/apache/airflow/pull/35088#issuecomment-1773053096 This caused failure in https://github.com/apache/airflow/pull/35085 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

[PR] Run constraint branch checkout in non-canary build [airflow]

2023-10-20 Thread via GitHub
potiuk opened a new pull request, #35088: URL: https://github.com/apache/airflow/pull/35088 The constraint branch checkout needs to happen always when there is an "upgrade-to-newer-dependencies" build, not only wnen there is a "canary" main build, Otherwise when we attempt to update to

Re: [PR] Fix CI's docs filter generation [airflow]

2023-10-20 Thread via GitHub
Taragolis commented on PR #35087: URL: https://github.com/apache/airflow/pull/35087#issuecomment-1773041384 This case just not happen in main or during full tests that is why we miss that -- This is an automated message from the Apache Git Service. To respond to the message, please log

Re: [PR] Fix CI's docs filter generation [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35087: URL: https://github.com/apache/airflow/pull/35087#issuecomment-1773038044 Right :facepalm: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Added "overrides" parameter for CloudRunExecuteJobOperator [airflow]

2023-10-20 Thread via GitHub
cjames23 commented on code in PR #34874: URL: https://github.com/apache/airflow/pull/34874#discussion_r1367219857 ## airflow/providers/google/cloud/operators/cloud_run.py: ## @@ -270,6 +271,7 @@ def __init__( project_id: str, region: str, job_name:

Re: [PR] Allow to install MariaDB client libraries instead of MySQL on x86_64 [airflow]

2023-10-20 Thread via GitHub
Taragolis commented on PR #35070: URL: https://github.com/apache/airflow/pull/35070#issuecomment-1773038731 Since it experimental feature, I guess it could be included into the 2.7.3, am I right? -- This is an automated message from the Apache Git Service. To respond to the message,

[PR] Fix CI's docs filter generation [airflow]

2023-10-20 Thread via GitHub
Taragolis opened a new pull request, #35087: URL: https://github.com/apache/airflow/pull/35087 Fix list generation for `build-docs` CI step. ```console Run breeze build-docs apache-airflow amazon apache-flink apache-spark celery cncf-kubernetes google

Re: [PR] Add task context logging feature to allow forwarding messages to task logs [airflow]

2023-10-20 Thread via GitHub
pankajkoti commented on PR #32646: URL: https://github.com/apache/airflow/pull/32646#issuecomment-1773009886 > Hey @pankajkoti! We'd love to make use of this feature for the ECS executor, what's the current standing of this PR? Hi @o-nikolas , we stalled the story here for a while

[PR] Providers requirements for every python version [airflow]

2023-10-20 Thread via GitHub
pierrejeambrun opened a new pull request, #35086: URL: https://github.com/apache/airflow/pull/35086 Allow to generate provider requirements for all python version. Use every python version compatible with the `associated_airflow_version` of the provider. Parallelize image building

Re: [PR] Fix the logic of checking dataflow job state [airflow]

2023-10-20 Thread via GitHub
asafeunico commented on code in PR #34785: URL: https://github.com/apache/airflow/pull/34785#discussion_r1367102802 ## airflow/providers/google/cloud/hooks/dataflow.py: ## @@ -420,7 +420,9 @@ def _check_dataflow_job_state(self, job) -> bool:

Re: [PR] Upgrade to Flask Application Builder 4.3.9 [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35085: URL: https://github.com/apache/airflow/pull/35085#issuecomment-1772882711 Rebased and resolved conflicts after @vincbeck's merged #34924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use

Re: [PR] Move Rest API auth related methods to FAB auth manager [airflow]

2023-10-20 Thread via GitHub
vincbeck merged PR #34924: URL: https://github.com/apache/airflow/pull/34924 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[airflow] branch main updated: Move Rest API auth related methods to FAB auth manager (#34924)

2023-10-20 Thread vincbeck
This is an automated email from the ASF dual-hosted git repository. vincbeck pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 48c17fb67b Move Rest API auth related methods to

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-20 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1367026145 ## airflow/models/baseoperator.py: ## @@ -1219,14 +1226,10 @@ def __deepcopy__(self, memo): return result def __getstate__(self): -state =

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-20 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1367026145 ## airflow/models/baseoperator.py: ## @@ -1219,14 +1226,10 @@ def __deepcopy__(self, memo): return result def __getstate__(self): -state =

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #34964: URL: https://github.com/apache/airflow/pull/34964#issuecomment-1772831484 BTW. You will need to rebase to make this one works -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [PR] Allow to install MariaDB client libraries instead of MySQL on x86_64 [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35070: URL: https://github.com/apache/airflow/pull/35070#issuecomment-1772825532 > Should I also add information into the https://airflow.apache.org/docs/docker-stack/changelog.html ? yes. This is also where we explain it's experimental. -- This is an

Re: [PR] Upgrade to Flask Application Builder 4.3.9 [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #35085: URL: https://github.com/apache/airflow/pull/35085#issuecomment-1772816983 Due to a tight integration and override, that one would require a bit careful review and double-checking whether all the changes 4.3.6 -> 4.3.9 -- This is an automated message from

Re: [PR] Add Code Coverage Helper Tool for Airflow [airflow]

2023-10-20 Thread via GitHub
ephraimbuddy commented on code in PR #35061: URL: https://github.com/apache/airflow/pull/35061#discussion_r1367028933 ## airflow/www/views.py: ## @@ -5827,6 +5828,33 @@ def add_user_permissions_to_dag(sender, template, context, **extra): context["dag"] = dag

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-20 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1367027165 ## airflow/utils/log/logging_mixin.py: ## @@ -68,14 +68,43 @@ class LoggingMixin: """Convenience super-class to have a logger configured with the class

Re: [PR] Add task parameter to set custom logger name [airflow]

2023-10-20 Thread via GitHub
Joffreybvn commented on code in PR #34964: URL: https://github.com/apache/airflow/pull/34964#discussion_r1367026145 ## airflow/models/baseoperator.py: ## @@ -1219,14 +1226,10 @@ def __deepcopy__(self, memo): return result def __getstate__(self): -state =

[PR] Upgrade to Flask Application Builder 4.3.9 [airflow]

2023-10-20 Thread via GitHub
potiuk opened a new pull request, #35085: URL: https://github.com/apache/airflow/pull/35085 This PR brings all the necessary changes to upgrade to FAB 4.3.9 from 4.3.6. It incorporates those changes: * https://github.com/dpgaspar/Flask-AppBuilder/pull/2112 *

Re: [PR] AIP-58: Add Airflow ObjectStore (AS) [airflow]

2023-10-20 Thread via GitHub
potiuk commented on PR #34729: URL: https://github.com/apache/airflow/pull/34729#issuecomment-1772775554 You should be able to rebase + resolve conflicts now @bolkedebruin for the PROD image -- This is an automated message from the Apache Git Service. To respond to the message, please

[airflow] branch main updated: Move installed_providers.txt to airflow/providers. (#35081)

2023-10-20 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new eb76276b89 Move installed_providers.txt to

Re: [PR] Move installed_providers.txt to airflow/providers. [airflow]

2023-10-20 Thread via GitHub
potiuk merged PR #35081: URL: https://github.com/apache/airflow/pull/35081 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail:

[PR] Chart: configurable mountPath for DAGs volume [airflow]

2023-10-20 Thread via GitHub
Aakcht opened a new pull request, #35083: URL: https://github.com/apache/airflow/pull/35083 This PR adds possibility to configure mountPath for DAGs volume. Configurable mountPath will make it possible to: * Add some rarely changing DAGs during image building to your `dags_folder`

  1   2   >