(airflow) branch constraints-main updated: Updating constraints. Github run id:7752771399

2024-02-02 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9fb89b520c Updating con

Re: [I] Unable to get external postgresql working [airflow]

2024-02-02 Thread via GitHub
jmli1024 commented on issue #19771: URL: https://github.com/apache/airflow/issues/19771#issuecomment-1923301469 I saw that nobody has resolution. I have same problem and record the solution to this error. In my case, my airflow is deployed in eks. I don't know when I have modified the se

Re: [PR] Replace usage of `datetime.utcnow` and `datetime.utcfromtimestamp` in providers [airflow]

2024-02-02 Thread via GitHub
uranusjr commented on code in PR #37138: URL: https://github.com/apache/airflow/pull/37138#discussion_r1475729755 ## tests/providers/google/cloud/sensors/test_gcs.py: ## @@ -72,7 +73,7 @@ def context(): """ Creates an empty context. """ -context = {"data_inter

[I] sentry add_tagging() can take unset "task" variable [airflow]

2024-02-02 Thread via GitHub
LipuFei opened a new issue, #37142: URL: https://github.com/apache/airflow/issues/37142 ### Apache Airflow version 2.8.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? Our sentry has captured this error in `sentry.py`:

Re: [PR] Add Qdrant Provider [airflow]

2024-02-02 Thread via GitHub
eladkal commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1475758927 ## airflow/providers/qdrant/hooks/qdrant.py: ## @@ -0,0 +1,128 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreeme

Re: [PR] Add Qdrant Provider [airflow]

2024-02-02 Thread via GitHub
Anush008 commented on code in PR #36805: URL: https://github.com/apache/airflow/pull/36805#discussion_r1475763053 ## airflow/providers/qdrant/hooks/qdrant.py: ## @@ -0,0 +1,128 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreem

[PR] Check task attribute before use in sentry.add_tagging() [airflow]

2024-02-02 Thread via GitHub
LipuFei opened a new pull request, #37143: URL: https://github.com/apache/airflow/pull/37143 Check the the `TaskInstance.task` attribute before using it, because `TaskInstance.task` may not be set. closes: #37142 -- This is an automated message from the Apache Git Service. To respo

Re: [PR] Add doc page with providers deprecations [airflow]

2024-02-02 Thread via GitHub
kacpermuda commented on code in PR #37075: URL: https://github.com/apache/airflow/pull/37075#discussion_r1475807282 ## docs/apache-airflow-providers/core-extensions/deprecations.rst: ## @@ -0,0 +1,57 @@ + .. Licensed to the Apache Software Foundation (ASF) under one +or more

Re: [PR] Move docs for cron basics to Authoring and Scheduling section [airflow]

2024-02-02 Thread via GitHub
eladkal commented on code in PR #37049: URL: https://github.com/apache/airflow/pull/37049#discussion_r1475814201 ## docs/apache-airflow/core-concepts/dag-run.rst: ## @@ -44,42 +44,6 @@ There are two possible terminal states for the DAG Run: DAGs that have a currently running

Re: [PR] Replace usage of `datetime.utcnow` and `datetime.utcfromtimestamp` in providers [airflow]

2024-02-02 Thread via GitHub
Taragolis commented on code in PR #37138: URL: https://github.com/apache/airflow/pull/37138#discussion_r1475831005 ## tests/providers/google/cloud/sensors/test_gcs.py: ## @@ -72,7 +73,7 @@ def context(): """ Creates an empty context. """ -context = {"data_inte

Re: [PR] Add deferrable param in SFTPSensor [airflow]

2024-02-02 Thread via GitHub
utkarsharma2 commented on code in PR #37117: URL: https://github.com/apache/airflow/pull/37117#discussion_r1475839636 ## airflow/providers/sftp/sensors/sftp.py: ## @@ -119,3 +124,50 @@ def poke(self, context: Context) -> PokeReturnValue | bool: xcom_value={"file

Re: [PR] Add deferrable param in SFTPSensor [airflow]

2024-02-02 Thread via GitHub
utkarsharma2 commented on code in PR #37117: URL: https://github.com/apache/airflow/pull/37117#discussion_r1475840616 ## airflow/providers/sftp/sensors/sftp.py: ## @@ -119,3 +124,50 @@ def poke(self, context: Context) -> PokeReturnValue | bool: xcom_value={"file

Re: [I] Aws BatchSensor max_retries should change with deferrable framework [airflow]

2024-02-02 Thread via GitHub
0x26res commented on issue #37120: URL: https://github.com/apache/airflow/issues/37120#issuecomment-1923496552 I'll give it a try once I find the time. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

Re: [PR] Add deferrable param in SFTPSensor [airflow]

2024-02-02 Thread via GitHub
utkarsharma2 commented on code in PR #37117: URL: https://github.com/apache/airflow/pull/37117#discussion_r1475853614 ## airflow/providers/sftp/hooks/sftp.py: ## @@ -400,3 +406,177 @@ def get_files_by_pattern(self, path, fnmatch_pattern) -> list[str]: matched_f

[PR] Allow running airflow against sqlite in-memory DB for tests [airflow]

2024-02-02 Thread via GitHub
ashb opened a new pull request, #37144: URL: https://github.com/apache/airflow/pull/37144 This makes no sense for production workloads, but it is useful for writing tests of Airflow operators etc. I also moved the warning from configure_vars to convigure_orm so that it is checked

(airflow) branch allow-sqlite-temp-db created (now f3c91ef8ae)

2024-02-02 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch allow-sqlite-temp-db in repository https://gitbox.apache.org/repos/asf/airflow.git at f3c91ef8ae Allow running airflow against sqlite in-memory DB for tests No new revisions were added by thi

Re: [I] `PythonVirtualenvOperator` crashes if any `python_callable` function is defined in the same source as DAG [airflow]

2024-02-02 Thread via GitHub
rawwar commented on issue #35529: URL: https://github.com/apache/airflow/issues/35529#issuecomment-1923515006 @potiuk , I think there are two options for this issue. 1. Directly raise an error when the callable is defined within the module where DAG is defined. 2. Make the origina

Re: [PR] Switch instance type to m7g for ARM image building [airflow]

2024-02-02 Thread via GitHub
potiuk commented on code in PR #37134: URL: https://github.com/apache/airflow/pull/37134#discussion_r1475865072 ## scripts/ci/images/ci_start_arm_instance_and_connect_to_docker.sh: ## @@ -20,7 +20,7 @@ SCRIPTS_DIR="$( cd "$( dirname "${BASH_SOURCE[0]}" )" && pwd)" WORKING_DIR="

Re: [PR] Add deferrable param in SFTPSensor [airflow]

2024-02-02 Thread via GitHub
utkarsharma2 commented on code in PR #37117: URL: https://github.com/apache/airflow/pull/37117#discussion_r1475867945 ## tests/system/providers/sftp/example_sftp_sensor.py: ## @@ -76,8 +76,18 @@ def sftp_sensor_decorator(): sftp_with_operator = SFTPSensor(task_id="sftp_oper

Re: [PR] Switch instance type to m7g for ARM image building [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #37134: URL: https://github.com/apache/airflow/pull/37134#issuecomment-1923528475 Random job failures only. merging. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

Re: [PR] Switch instance type to m7g for ARM image building [airflow]

2024-02-02 Thread via GitHub
potiuk merged PR #37134: URL: https://github.com/apache/airflow/pull/37134 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Switch instance type to m7g for ARM image building (#37134)

2024-02-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 5814e1c3d8 Switch instance type to m7g for ARM imag

Re: [PR] Use hook.wait_for_operation in DataprocInstantiateInlineWorkflowTemplateOperator [airflow]

2024-02-02 Thread via GitHub
boring-cyborg[bot] commented on PR #37145: URL: https://github.com/apache/airflow/pull/37145#issuecomment-1923547843 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

[PR] Use hook.wait_for_operation in DataprocInstantiateInlineWorkflowTemplateOperator [airflow]

2024-02-02 Thread via GitHub
edin-tapad opened a new pull request, #37145: URL: https://github.com/apache/airflow/pull/37145 We run into an issue when running a long Dataproc job with `DataprocInstantiateInlineWorkflowTemplateOperator`. It seems that `hook.wait_for_operation` should be used so that the `timeout` parame

(airflow) branch allow-sqlite-temp-db updated (f3c91ef8ae -> 1c3b9e4923)

2024-02-02 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch allow-sqlite-temp-db in repository https://gitbox.apache.org/repos/asf/airflow.git discard f3c91ef8ae Allow running airflow against sqlite in-memory DB for tests add 1c3b9e4923 Allow running a

Re: [I] external_python_task decorator doesn't work if airflow isn't in external python env [airflow]

2024-02-02 Thread via GitHub
Taragolis commented on issue #37136: URL: https://github.com/apache/airflow/issues/37136#issuecomment-1923567769 You should use `@task.external_python` instead, the function which you use is a private https://github.com/apache/airflow/blob/3904206b69428525db31ff7813daa0322f7b83e8/air

Re: [I] external_python_task decorator doesn't work if airflow isn't in external python env [airflow]

2024-02-02 Thread via GitHub
Taragolis closed issue #37136: external_python_task decorator doesn't work if airflow isn't in external python env URL: https://github.com/apache/airflow/issues/37136 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

Re: [PR] Replace usage of `datetime.utcnow` and `datetime.utcfromtimestamp` in providers [airflow]

2024-02-02 Thread via GitHub
Taragolis commented on code in PR #37138: URL: https://github.com/apache/airflow/pull/37138#discussion_r1475900487 ## tests/providers/google/cloud/sensors/test_gcs.py: ## @@ -72,7 +73,7 @@ def context(): """ Creates an empty context. """ -context = {"data_inte

Re: [PR] Add deferrable param in SFTPSensor [airflow]

2024-02-02 Thread via GitHub
utkarsharma2 commented on code in PR #37117: URL: https://github.com/apache/airflow/pull/37117#discussion_r1475921303 ## airflow/providers/sftp/hooks/sftp.py: ## @@ -400,3 +406,177 @@ def get_files_by_pattern(self, path, fnmatch_pattern) -> list[str]: matched_f

[PR] Upgrade trove-classifier in build dependencies after it's released [airflow]

2024-02-02 Thread via GitHub
potiuk opened a new pull request, #37147: URL: https://github.com/apache/airflow/pull/37147 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Add Qdrant Provider [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #36805: URL: https://github.com/apache/airflow/pull/36805#issuecomment-1923706786 FYI: @Anush008 : You will have to make your provider compatible with Airflow 2.7.0+ the compatibility checks detected, that your provider requires Pydantic 2 by, which is not su

Re: [PR] Remove pid arg from celery option to fix duplicate pid issue, Move celery command to provider package [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #36794: URL: https://github.com/apache/airflow/pull/36794#issuecomment-1923733802 > @potiuk I tried copying daemon func but it goes way down where so many utils are not present. > > Its better if someone with better codebase understanding do this. Let me know i

Re: [PR] Remove pid arg from celery option to fix duplicate pid issue, Move celery command to provider package [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #36794: URL: https://github.com/apache/airflow/pull/36794#issuecomment-1923735324 So simply go back to previous import and add the try/except/raise exception and all should be good. -- This is an automated message from the Apache Git Service. To respond to the messa

Re: [PR] Upgrade trove-classifier in build dependencies after it's released [airflow]

2024-02-02 Thread via GitHub
potiuk merged PR #37147: URL: https://github.com/apache/airflow/pull/37147 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Upgrade trove-classifier in build dependencies after it's released (#37147)

2024-02-02 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new c381fdaff4 Upgrade trove-classifier in build depend

Re: [PR] Fix KPO docs argument precedence [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #37047: URL: https://github.com/apache/airflow/pull/37047#issuecomment-1923766672 fix to bactics is needed. RST does not like single backticks. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

Re: [PR] feat: Stream Transfer Enhancement for Airflow SFTP to GCS Operator [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #36545: URL: https://github.com/apache/airflow/pull/36545#issuecomment-1923769973 You will need to rebase to resolve conflicts. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go

Re: [I] Kubernetes Executor Task Leak [airflow]

2024-02-02 Thread via GitHub
smhood commented on issue #36998: URL: https://github.com/apache/airflow/issues/36998#issuecomment-1923838162 So after a brief sighting of things working again, we are now seeing it again. Single Scheduler running on the 1.11 Helm Chart, Airflow 2.8.1. -- This is an automated

Re: [PR] Allow running airflow against sqlite in-memory DB for tests [airflow]

2024-02-02 Thread via GitHub
ashb merged PR #37144: URL: https://github.com/apache/airflow/pull/37144 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.apa

(airflow) branch main updated: Allow running airflow against sqlite in-memory DB for tests (#37144)

2024-02-02 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 7324400c6b Allow running airflow against sqlite in-mem

(airflow) branch allow-sqlite-temp-db deleted (was 1c3b9e4923)

2024-02-02 Thread ash
This is an automated email from the ASF dual-hosted git repository. ash pushed a change to branch allow-sqlite-temp-db in repository https://gitbox.apache.org/repos/asf/airflow.git was 1c3b9e4923 Allow running airflow against sqlite in-memory DB for tests The revisions that were on this br

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-02-02 Thread via GitHub
dirrao commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1476162147 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,9 +434,9 @@ def sync(self) -> None: ) self.

Re: [I] Scheduler fails to schedule DagRuns due to persistent DAG record lock [airflow]

2024-02-02 Thread via GitHub
plmnguyen commented on issue #36920: URL: https://github.com/apache/airflow/issues/36920#issuecomment-1924267212 I am trying to recreate or capture logs, but what I've been noticing thus far on 2.8.1: - Happens only on Kubernetes spark submit operators tasks - For DAGs showing tha

Re: [PR] heartbeat recovery message [airflow]

2024-02-02 Thread via GitHub
Bowrna commented on PR #34457: URL: https://github.com/apache/airflow/pull/34457#issuecomment-1924290629 @amoghrajesh @potiuk do I have any feedback on the test case I have added? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to Git

Re: [PR] Add Doctrine in users [airflow]

2024-02-02 Thread via GitHub
ferruzzi commented on PR #37135: URL: https://github.com/apache/airflow/pull/37135#issuecomment-1924308523 Thank YOU. Congrats on your first PR. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to t

[I] Airflow 2.7.2 constrain file use redshift-connector==2.7.2 that causes dependency issues with dbt-redshift package (any) [airflow]

2024-02-02 Thread via GitHub
EsauOlivaLeaoHelix opened a new issue, #37148: URL: https://github.com/apache/airflow/issues/37148 ### Apache Airflow Provider(s) dbt-cloud ### Versions of Apache Airflow Providers --constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.7.2/constraint

Re: [I] Airflow 2.7.2 constrain file use redshift-connector==2.7.2 that causes dependency issues with dbt-redshift package (any) [airflow]

2024-02-02 Thread via GitHub
boring-cyborg[bot] commented on issue #37148: URL: https://github.com/apache/airflow/issues/37148#issuecomment-1924323932 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

Re: [PR] Add Qdrant Provider [airflow]

2024-02-02 Thread via GitHub
Anush008 commented on PR #36805: URL: https://github.com/apache/airflow/pull/36805#issuecomment-1924328621 Hey. > the compatibility checks detected, that your provider requires Pydantic 2 by, which is not supported in `Airflow 2.6.*`. `qdrant_client` uses Pydantic v1. The fail

Re: [I] Airflow 2.7.2 constrain file use redshift-connector==2.7.2 that causes dependency issues with dbt-redshift package (any) [airflow]

2024-02-02 Thread via GitHub
Taragolis closed issue #37148: Airflow 2.7.2 constrain file use redshift-connector==2.7.2 that causes dependency issues with dbt-redshift package (any) URL: https://github.com/apache/airflow/issues/37148 -- This is an automated message from the Apache Git Service. To respond to the message,

Re: [I] Airflow 2.7.2 constrain file use redshift-connector==2.7.2 that causes dependency issues with dbt-redshift package (any) [airflow]

2024-02-02 Thread via GitHub
Taragolis commented on issue #37148: URL: https://github.com/apache/airflow/issues/37148#issuecomment-1924411710 **Solution**: Please Install first airflow and it dependencies and after that install upgrade all third party packages **Solution with explanation**: Please follow the recommen

[I] All DAGs triggered through UI fail after upgrading from 2.6.1 to 2.8.1 [airflow]

2024-02-02 Thread via GitHub
luederm opened a new issue, #37149: URL: https://github.com/apache/airflow/issues/37149 ### Apache Airflow version 2.8.1 ### If "Other Airflow 2 version" selected, which one? _No response_ ### What happened? After upgrading from airflow version 2.6 to 2.8, a

Re: [PR] Add Qdrant Provider [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #36805: URL: https://github.com/apache/airflow/pull/36805#issuecomment-1924415290 It's problem with Airflow - 2.6 has problem with Pydantic 1(that's why you need to limit it. That's why you cannot install qdrant on it. As of 2.7 airflwo supports both an d in 2.8. on

Re: [I] Button on UI to pause all DAGs [airflow]

2024-02-02 Thread via GitHub
rsdel2007 commented on issue #36994: URL: https://github.com/apache/airflow/issues/36994#issuecomment-1924429037 @potiuk I agree with @cmarteepants suggestion. This seems to be the right way. WDYT? -- This is an automated message from the Apache Git Service. To respond to the message, pl

Re: [PR] Add Qdrant Provider [airflow]

2024-02-02 Thread via GitHub
potiuk commented on PR #36805: URL: https://github.com/apache/airflow/pull/36805#issuecomment-1924429953 It's problem with Airflow - 2.6 has problem with Pydantic 1(that's why you need to limit it. That's why you cannot install qdrant on it. As of 2.7 airflwo supports both an d in 2.8. on

Re: [I] Kubernetes Executor Task Leak [airflow]

2024-02-02 Thread via GitHub
smhood commented on issue #36998: URL: https://github.com/apache/airflow/issues/36998#issuecomment-1924436977 Looking into the latest occurrence, what is weird is we are seeing the following logged event: ![image](https://github.com/apache/airflow/assets/14971423/de500855-71d8-4744-b5f3-

Re: [I] Kubernetes Executor Task Leak [airflow]

2024-02-02 Thread via GitHub
smhood commented on issue #36998: URL: https://github.com/apache/airflow/issues/36998#issuecomment-1924459837 Additional logging showing what this looks like as far as open slots: ![image](https://github.com/apache/airflow/assets/14971423/3e41b474-2d3c-4dd3-8492-434495a7d1e3) -- Th

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-02-02 Thread via GitHub
hussein-awala commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1476561460 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,19 +438,35 @@ def sync(self) -> None: )

[PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
Don-Burns opened a new pull request, #37150: URL: https://github.com/apache/airflow/pull/37150 --- # Summary As documented in #37089 the SQLThresholdCheckOperator will currently fail if a "falsey" value is returned from the query (e.g. 0 or ""). This

Re: [PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
boring-cyborg[bot] commented on PR #37150: URL: https://github.com/apache/airflow/pull/37150#issuecomment-1924551712 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

Re: [PR] AIP-58: Add object storage backend for xcom [airflow]

2024-02-02 Thread via GitHub
bolkedebruin commented on code in PR #37058: URL: https://github.com/apache/airflow/pull/37058#discussion_r1476602650 ## airflow/providers/common/io/xcom/backend.py: ## @@ -0,0 +1,166 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] AIP-58: Add object storage backend for xcom [airflow]

2024-02-02 Thread via GitHub
bolkedebruin commented on code in PR #37058: URL: https://github.com/apache/airflow/pull/37058#discussion_r1476602650 ## airflow/providers/common/io/xcom/backend.py: ## @@ -0,0 +1,166 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
hussein-awala commented on code in PR #37150: URL: https://github.com/apache/airflow/pull/37150#discussion_r1476615261 ## airflow/providers/common/sql/operators/sql.py: ## @@ -1058,9 +1058,13 @@ def __init__( def execute(self, context: Context): hook = self.get_d

Re: [PR] AIP-58: Add object storage backend for xcom [airflow]

2024-02-02 Thread via GitHub
bolkedebruin commented on code in PR #37058: URL: https://github.com/apache/airflow/pull/37058#discussion_r1476602650 ## airflow/providers/common/io/xcom/backend.py: ## @@ -0,0 +1,166 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] AIP-58: Add object storage backend for xcom [airflow]

2024-02-02 Thread via GitHub
bolkedebruin commented on code in PR #37058: URL: https://github.com/apache/airflow/pull/37058#discussion_r1476602650 ## airflow/providers/common/io/xcom/backend.py: ## @@ -0,0 +1,166 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

[PR] [WIP] Upgrade pytest to v8 [airflow]

2024-02-02 Thread via GitHub
hussein-awala opened a new pull request, #37151: URL: https://github.com/apache/airflow/pull/37151 (no comment) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscri

Re: [PR] [WIP] Upgrade pytest to v8 [airflow]

2024-02-02 Thread via GitHub
hussein-awala commented on PR #37151: URL: https://github.com/apache/airflow/pull/37151#issuecomment-1924628015 depends on: https://github.com/pytest-dev/pytest-asyncio/pull/765 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

Re: [I] Kubernetes Executor Task Leak [airflow]

2024-02-02 Thread via GitHub
smhood commented on issue #36998: URL: https://github.com/apache/airflow/issues/36998#issuecomment-1924671652 @dirrao do I have to change labels in order to get follow up? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and us

Re: [PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
Don-Burns commented on code in PR #37150: URL: https://github.com/apache/airflow/pull/37150#discussion_r1476721918 ## airflow/providers/common/sql/operators/sql.py: ## @@ -1058,9 +1058,13 @@ def __init__( def execute(self, context: Context): hook = self.get_db_ho

Re: [PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
Don-Burns commented on code in PR #37150: URL: https://github.com/apache/airflow/pull/37150#discussion_r1476721918 ## airflow/providers/common/sql/operators/sql.py: ## @@ -1058,9 +1058,13 @@ def __init__( def execute(self, context: Context): hook = self.get_db_ho

[PR] Integrate the run coverage command to the CI [airflow]

2024-02-02 Thread via GitHub
ephraimbuddy opened a new pull request, #37152: URL: https://github.com/apache/airflow/pull/37152 This is an attempt to prevent a file with full test coverage from slipping below 100% coverage. --- **^ Add meaningful description above** Read the **[Pu

Re: [I] ComputeEngineSSHHook unintentionally deletes instance metadata [airflow]

2024-02-02 Thread via GitHub
seanmuth commented on issue #35171: URL: https://github.com/apache/airflow/issues/35171#issuecomment-1924718428 @teslur It doesn't appear a PR for this has been created, what is the current status of this issue? -- This is an automated message from the Apache Git Service. To respond t

Re: [PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
hussein-awala commented on code in PR #37150: URL: https://github.com/apache/airflow/pull/37150#discussion_r1476859987 ## airflow/providers/common/sql/operators/sql.py: ## @@ -1058,9 +1058,13 @@ def __init__( def execute(self, context: Context): hook = self.get_d

Re: [PR] Merge all ECS executor configs following recursive python dict update [airflow]

2024-02-02 Thread via GitHub
o-nikolas commented on PR #37137: URL: https://github.com/apache/airflow/pull/37137#issuecomment-1924900854 > One thing that occurred to me is whether we want to document this design choice. I think the unit test does a very good job of explaining the behaviour. Might be worth putting that

Re: [PR] Allow changing the unfinished ti state [airflow]

2024-02-02 Thread via GitHub
github-actions[bot] closed pull request #36078: Allow changing the unfinished ti state URL: https://github.com/apache/airflow/pull/36078 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Add bigquery partition sensor for sequential partition id [airflow]

2024-02-02 Thread via GitHub
github-actions[bot] closed pull request #36201: Add bigquery partition sensor for sequential partition id URL: https://github.com/apache/airflow/pull/36201 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

Re: [PR] Add k8s recommended labels [airflow]

2024-02-02 Thread via GitHub
github-actions[bot] commented on PR #34735: URL: https://github.com/apache/airflow/pull/34735#issuecomment-1924928004 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[PR] Metric name matching - slightly clearer unit test [airflow]

2024-02-02 Thread via GitHub
ferruzzi opened a new pull request, #37153: URL: https://github.com/apache/airflow/pull/37153 I was doing a demo for this new feature and decided to clear up the unit tests a little to make the functionality more obvious. Figured I may as well contribute it up. No real functional changes,

Re: [PR] Fix SQLThresholdCheckOperator error on falsey vals [airflow]

2024-02-02 Thread via GitHub
Don-Burns commented on code in PR #37150: URL: https://github.com/apache/airflow/pull/37150#discussion_r1476905912 ## airflow/providers/common/sql/operators/sql.py: ## @@ -1058,9 +1058,13 @@ def __init__( def execute(self, context: Context): hook = self.get_db_ho

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-02-02 Thread via GitHub
dirrao commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1476927531 ## airflow/providers/cncf/kubernetes/executors/kubernetes_executor.py: ## @@ -434,19 +438,35 @@ def sync(self) -> None: ) sel

Re: [PR] The task is stuck in a queued state forever in case of pod launch errors [airflow]

2024-02-02 Thread via GitHub
dirrao commented on code in PR #36882: URL: https://github.com/apache/airflow/pull/36882#discussion_r1476927807 ## airflow/providers/cncf/kubernetes/provider.yaml: ## @@ -350,6 +350,15 @@ config: type: string example: ~ default: "" + task_publish_

Re: [I] Mapped Task details view "success" counts are off [airflow]

2024-02-02 Thread via GitHub
JosephABC commented on issue #29988: URL: https://github.com/apache/airflow/issues/29988#issuecomment-1925104565 We are using v2.7.3 and encountered this issue as well -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use th

Re: [I] dag_processing.processes should be a gauge, not a counter [airflow]

2024-02-02 Thread via GitHub
Bowrna commented on issue #36188: URL: https://github.com/apache/airflow/issues/36188#issuecomment-1925168423 @dirrao @potiuk @ferruzzi @hussein-awala Do you think we have to fix in the existing key and edit the docs to or write this in new key and have a backward compatibility and slowly d

Re: [I] logs from k8s executor tasks are not visible while task running [airflow]

2024-02-02 Thread via GitHub
github-actions[bot] commented on issue #28314: URL: https://github.com/apache/airflow/issues/28314#issuecomment-1925180747 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

Re: [I] logs from k8s executor tasks are not visible while task running [airflow]

2024-02-02 Thread via GitHub
github-actions[bot] closed issue #28314: logs from k8s executor tasks are not visible while task running URL: https://github.com/apache/airflow/issues/28314 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to

[I] Subsequent Dag Run mapped task not scheduled due to removed mapped task in previous Dag Run [airflow]

2024-02-02 Thread via GitHub
JosephABC opened a new issue, #37154: URL: https://github.com/apache/airflow/issues/37154 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.3 ### What happened? ### **Dag Infor

Re: [I] Subsequent Dag Run mapped task not scheduled due to removed mapped task in previous Dag Run [airflow]

2024-02-02 Thread via GitHub
boring-cyborg[bot] commented on issue #37154: URL: https://github.com/apache/airflow/issues/37154#issuecomment-1925200997 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap