Re: [I] SnowflakeCheckOperator does not use the database parameter [airflow]

2024-04-05 Thread via GitHub
github-actions[bot] commented on issue #30408: URL: https://github.com/apache/airflow/issues/30408#issuecomment-2039093913 This issue has been automatically marked as stale because it has been open for 365 days without any activity. There has been several Airflow releases since last activit

Re: [I] Tasks are stuck on scheduled, worker can not insert into celery_taskmeta [airflow]

2024-04-05 Thread via GitHub
github-actions[bot] closed issue #23700: Tasks are stuck on scheduled, worker can not insert into celery_taskmeta URL: https://github.com/apache/airflow/issues/23700 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL

Re: [I] Tasks are stuck on scheduled, worker can not insert into celery_taskmeta [airflow]

2024-04-05 Thread via GitHub
github-actions[bot] commented on issue #23700: URL: https://github.com/apache/airflow/issues/23700#issuecomment-2039094003 This issue has been closed because it has not received response from the issue author. -- This is an automated message from the Apache Git Service. To respond to the

Re: [PR] Add google-cloud-bigquery as explicit google-provider dependency [airflow]

2024-04-05 Thread via GitHub
potiuk commented on PR #38753: URL: https://github.com/apache/airflow/pull/38753#issuecomment-2039233516 It's been very specific - an I am not sure if it is constrantly reproducible - this is the nature of dependency resolving that it constantly changes, depending on what is currrently avai

Re: [PR] bump uv to 0.1.29 [airflow]

2024-04-05 Thread via GitHub
potiuk merged PR #38758: URL: https://github.com/apache/airflow/pull/38758 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: bump uv to 0.1.29 (#38758)

2024-04-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 0e8f108313 bump uv to 0.1.29 (#38758) 0e8f108313 is

Re: [PR] Fix skip Apache Beam pipeline options if value is set to false [airflow]

2024-04-05 Thread via GitHub
dondaum commented on PR #38496: URL: https://github.com/apache/airflow/pull/38496#issuecomment-2039244122 Kindly asking for a review. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-04-05 Thread via GitHub
Bowrna commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2039248829 > @Bowrna @HTRafal @tanvn - Does this proposed change look like it will address your comments? If you all confirm, I'll approve and we can get it merged. The code looks fin to me, just w

Re: [I] expand_kwargs does not preserve templates [airflow]

2024-04-05 Thread via GitHub
alexdrydew commented on issue #38745: URL: https://github.com/apache/airflow/issues/38745#issuecomment-2039259618 > If we don't do this, we won't be able to use templates in mapped tasks, which is a big limitation for this great feature, so if it's not the same behavior as expand, then we n

Re: [I] Build correct SQLAlchemy URI in hooks based on `DbApiHook` [airflow]

2024-04-05 Thread via GitHub
Bowrna commented on issue #38195: URL: https://github.com/apache/airflow/issues/38195#issuecomment-2039277534 hello @Taragolis I was checking this issue and seeing if I could fix it. https://github.com/apache/airflow/blob/0e8f108313d4af0b450581661aeb8ed35e82a8e6/airflow/models/connect

(airflow) branch openlineage-do-not-submit-busyjobs created (now 3c173d596b)

2024-04-05 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a change to branch openlineage-do-not-submit-busyjobs in repository https://gitbox.apache.org/repos/asf/airflow.git at 3c173d596b openlineage: skip sending events if the executor is busy This branch inc

(airflow) 01/01: openlineage: skip sending events if the executor is busy

2024-04-05 Thread mobuchowski
This is an automated email from the ASF dual-hosted git repository. mobuchowski pushed a commit to branch openlineage-do-not-submit-busyjobs in repository https://gitbox.apache.org/repos/asf/airflow.git commit 3c173d596bfa17b8432e5c46a9624cbf8f1471e8 Author: Maciej Obuchowski AuthorDate: Thu Apr

[I] Databricks provider does not support Azure managed identity if more than one identity exists (e.g. ADF, AKS) [airflow]

2024-04-05 Thread via GitHub
jtv8 opened a new issue, #38762: URL: https://github.com/apache/airflow/issues/38762 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.6.3 ### What happened? When trying to authe

Re: [I] Databricks provider does not support Azure managed identity if more than one identity exists (e.g. ADF, AKS) [airflow]

2024-04-05 Thread via GitHub
boring-cyborg[bot] commented on issue #38762: URL: https://github.com/apache/airflow/issues/38762#issuecomment-2039290433 Thanks for opening your first issue here! Be sure to follow the issue template! If you are willing to raise PR to address this issue please do so, no need to wait for ap

[PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-05 Thread via GitHub
amoghrajesh opened a new pull request, #38763: URL: https://github.com/apache/airflow/pull/38763 Hive connections from airflow are usually always made through beeline in most cases, due to this we should enable the beeline checkbox by default to make it more convenient t

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-05 Thread via GitHub
eladkal commented on code in PR #38763: URL: https://github.com/apache/airflow/pull/38763#discussion_r1553274933 ## airflow/providers/apache/hive/hooks/hive.py: ## @@ -126,7 +126,7 @@ def get_connection_form_widgets(cls) -> dict[str, Any]: from wtforms import BooleanFie

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-05 Thread via GitHub
amoghrajesh commented on code in PR #38763: URL: https://github.com/apache/airflow/pull/38763#discussion_r1553281936 ## airflow/providers/apache/hive/hooks/hive.py: ## @@ -126,7 +126,7 @@ def get_connection_form_widgets(cls) -> dict[str, Any]: from wtforms import Boolea

(airflow) branch xdist-tests-distribution updated (e366fb45a0 -> 3877822b09)

2024-04-05 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch xdist-tests-distribution in repository https://gitbox.apache.org/repos/asf/airflow.git discard e366fb45a0 Access to connection fix for AWS tests discard 0917a7491f A bit more @pytest.mark.en

Re: [PR] Allow specification of buffer length for GCS to Samba [airflow]

2024-04-05 Thread via GitHub
Taragolis merged PR #38373: URL: https://github.com/apache/airflow/pull/38373 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

(airflow) branch main updated: Allow specification of buffer length for GCS to Samba (#38373)

2024-04-05 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 13209fc354 Allow specification of buffer length

[PR] Improve editable airflow installation by adding preinstalled deps [airflow]

2024-04-05 Thread via GitHub
potiuk opened a new pull request, #38764: URL: https://github.com/apache/airflow/pull/38764 While preparing for presentation about modernizing packaging setup I found out that we can slightly improve editable installation of airflow. So far we required a "devel" installation in order to mak

(airflow) branch constraints-main updated: Updating constraints. Github run id:8568160115

2024-04-05 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new d33ce2be19 Updating con

(airflow) branch xdist-tests-distribution updated (3877822b09 -> dd67faef80)

2024-04-05 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch xdist-tests-distribution in repository https://gitbox.apache.org/repos/asf/airflow.git from 3877822b09 Additional isolation between the tests add dd67faef80 Raise critical exceptions

[PR] Disable by default redact sensitive values in tests [airflow]

2024-04-05 Thread via GitHub
Taragolis opened a new pull request, #38765: URL: https://github.com/apache/airflow/pull/38765 Found (TBH it happen before) during https://github.com/apache/airflow/pull/38711 and decoupled from that PR This measure for avoid side effects in the other tests which inten

[I] Define and implement core AIP-60 to OpenLineage dataset mapping [airflow]

2024-04-05 Thread via GitHub
mobuchowski opened a new issue, #38767: URL: https://github.com/apache/airflow/issues/38767 ### Body There is need to decide how and where the mapping will take place. Natural space for this is provider - they should "take ownership" of particular AIP-60 URI scheme, and be able

[PR] Fix TRY002 for docker swarm operator [airflow]

2024-04-05 Thread via GitHub
alizademhdi opened a new pull request, #38768: URL: https://github.com/apache/airflow/pull/38768 ### Related issue: #38761 fix TRY002 for docker swarm operator. ### Checks: - [x] airflow/providers/docker/operators/docker_swarm.py I also checked and there

[I] Implement support for DbApiHook to emit datasets for Postgres [airflow]

2024-04-05 Thread via GitHub
mobuchowski opened a new issue, #38770: URL: https://github.com/apache/airflow/issues/38770 ### Body Instrument DbApiHook to emit AIP-60 Datasets. To limit the scope of this issue, it should be about Postgres compatibility, so there' no requirement to support any other one for now.

[I] implement basic lineage support for AIP-58 Airflow ObjectStore [airflow]

2024-04-05 Thread via GitHub
mobuchowski opened a new issue, #38771: URL: https://github.com/apache/airflow/issues/38771 ### Body In the basic use case, this should report correct AIP-60 dataset to HookLineageCollector ``` from airflow.io.path import ObjectStoragePath base = ObjectStoragePath("s3

[I] S3Hook hook lineage implementation: ensure end-to-end event coverage [airflow]

2024-04-05 Thread via GitHub
mobuchowski opened a new issue, #38772: URL: https://github.com/apache/airflow/issues/38772 ### Body Make sure end-to-end DAG example works and emits proper OpenLineage events. Take special care to make sure dataset naming is consistent between Hook-sourced lineage from PythonO

[I] Cron [airflow]

2024-04-05 Thread via GitHub
pauloricardoferreira opened a new issue, #38774: URL: https://github.com/apache/airflow/issues/38774 ### Description I would like to request something for Scheduler via Cron. in that case: */27 8-11,14-18 * * 1-5 it should run at these times 8 o'clock 08:

Re: [I] Cron [airflow]

2024-04-05 Thread via GitHub
Taragolis commented on issue #38774: URL: https://github.com/apache/airflow/issues/38774#issuecomment-2039770739 > it should run at these times > Could it be a bug? It should run after `date_interval_end` if you use default [CronDataIntervalTimetable](https://airflow.apache.org

Re: [I] Cron [airflow]

2024-04-05 Thread via GitHub
Taragolis closed issue #38774: Cron URL: https://github.com/apache/airflow/issues/38774 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr

Re: [PR] Fix TRY002 for docker swarm operator [airflow]

2024-04-05 Thread via GitHub
Taragolis merged PR #38768: URL: https://github.com/apache/airflow/pull/38768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

Re: [I] expand_kwargs does not preserve templates [airflow]

2024-04-05 Thread via GitHub
potiuk commented on issue #38745: URL: https://github.com/apache/airflow/issues/38745#issuecomment-2039774537 > In my opinion, the main problem with having templates being rendered in this case is that now we need to sanitize user data when processing it in expanded tasks (I didn't think of

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-05 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1553611284 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -143,6 +194,9 @@ class DataflowJobMetricsSensor(BaseSensorOperator): If set as a sequence, the i

Re: [PR] Fix for Google Dataplex system tests [airflow]

2024-04-05 Thread via GitHub
potiuk merged PR #38737: URL: https://github.com/apache/airflow/pull/38737 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (13209fc354 -> 871267484d)

2024-04-05 Thread taragolis
This is an automated email from the ASF dual-hosted git repository. taragolis pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 13209fc354 Allow specification of buffer length for GCS to Samba (#38373) add 871267484d Fix TRY002 for docker

(airflow) branch main updated (871267484d -> 844cb8ba14)

2024-04-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 871267484d Fix TRY002 for docker swarm operator (#38768) add 844cb8ba14 Fix resource naming conflict in Dataplex s

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-05 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1553622795 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -151,12 +205,14 @@ def __init__( self, *, job_id: str, -callback: Call

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-04-05 Thread via GitHub
tanvn commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2039804076 > @Bowrna @HTRafal @tanvn - Does this proposed change look like it will address your comments? If you all confirm, I'll approve and we can get it merged. The code looks fin to me, just wa

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-05 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1553622795 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -151,12 +205,14 @@ def __init__( self, *, job_id: str, -callback: Call

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-05 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1553641057 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -357,4 +500,45 @@ def poke(self, context: Context) -> bool: location=self.location,

Re: [PR] Add the deferrable mode to the Dataflow sensors [airflow]

2024-04-05 Thread via GitHub
e-galan commented on code in PR #37693: URL: https://github.com/apache/airflow/pull/37693#discussion_r1553622795 ## airflow/providers/google/cloud/sensors/dataflow.py: ## @@ -151,12 +205,14 @@ def __init__( self, *, job_id: str, -callback: Call

Re: [I] In DAG Graph View, clicking Task on it's name doesn't select the task [airflow]

2024-04-05 Thread via GitHub
jonhspyro commented on issue #37932: URL: https://github.com/apache/airflow/issues/37932#issuecomment-2039814329 @eladkal Should is it worth creating a test for this fix, considering the scale of the issue? -- This is an automated message from the Apache Git Service. To respond to the mes

Re: [PR] Improve editable airflow installation by adding preinstalled deps [airflow]

2024-04-05 Thread via GitHub
potiuk commented on code in PR #38764: URL: https://github.com/apache/airflow/pull/38764#discussion_r1553647961 ## airflow_pre_installed_providers.txt: ## @@ -1,9 +0,0 @@ -# List of all the providers that are pre-installed when you run `pip install apache-airflow` without extra

[I] Performance Issues on Initial Page Load [airflow]

2024-04-05 Thread via GitHub
vchiapaikeo opened a new issue, #38776: URL: https://github.com/apache/airflow/issues/38776 ### Apache Airflow version Other Airflow 2 version (please specify below) ### If "Other Airflow 2 version" selected, which one? 2.7.3 ### What happened? We have start

Re: [PR] Add google-cloud-bigquery as explicit google-provider dependency [airflow]

2024-04-05 Thread via GitHub
notatallshaw commented on PR #38753: URL: https://github.com/apache/airflow/pull/38753#issuecomment-2039907669 I wasn't able to quickly reproduce this, but some flags that help with reproducing these issues for uv in the future are `--dry-run` to not pollute your environment, `--reinstall`

[PR] Remove SparkInK8s internal deprecations [airflow]

2024-04-05 Thread via GitHub
Owen-CH-Leung opened a new pull request, #38777: URL: https://github.com/apache/airflow/pull/38777 Part of the fixes for #38642 I executed `pytest tests/providers/cncf/kubernetes/operators/test_spark_kubernetes.py::test_spark_kubernetes_operator ` and `pytest tests/providers/cncf/

Re: [I] Add support for custom Trigger Rule logic [airflow]

2024-04-05 Thread via GitHub
ElisaWChen commented on issue #17010: URL: https://github.com/apache/airflow/issues/17010#issuecomment-2039920636 > the trigger rule `none_failed_or_skipped` is a solution to your use case -> https://airflow.apache.org/docs/apache-airflow/2.1.2/concepts/dags.html?highlight=none_failed_or_sk

[PR] Add `AirflowInternalRuntimeError` for raise non catchable errors [airflow]

2024-04-05 Thread via GitHub
Taragolis opened a new pull request, #38778: URL: https://github.com/apache/airflow/pull/38778 Introduce new internal exception for airflow, indicate that something really bad happen. I guess it might be suitable for cases when we use `RuntimeError` which could be easily

[PR] Add "use self-hosted runners" to CI setup [airflow]

2024-04-05 Thread via GitHub
potiuk opened a new pull request, #38779: URL: https://github.com/apache/airflow/pull/38779 In anticipation of switching commiter PRs to use public runners, we are adding the feature of applying the "use self-hosted runner" to a PR to make it use self-hosted runners.

Re: [PR] Add "use self-hosted runners" to CI setup [airflow]

2024-04-05 Thread via GitHub
pankajkoti commented on code in PR #38779: URL: https://github.com/apache/airflow/pull/38779#discussion_r1553813022 ## dev/breeze/doc/ci/04_selective_checks.md: ## @@ -267,17 +267,29 @@ This is controlled by `Selective checks <04_selective_checks.md>`__ that set app the build-

Re: [PR] Add google-cloud-bigquery as explicit google-provider dependency [airflow]

2024-04-05 Thread via GitHub
potiuk commented on PR #38753: URL: https://github.com/apache/airflow/pull/38753#issuecomment-2040039543 Good input. Thanks @notatallshaw :) > In general the behavior of uv's resolution algorithm (powered by pugrub-rs) has a tendency to want to exchaust all possible versions of a par

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
syedahsn commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1553827975 ## airflow/providers/amazon/aws/triggers/bedrock.py: ## @@ -0,0 +1,61 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] Add "use self-hosted runners" to CI setup [airflow]

2024-04-05 Thread via GitHub
potiuk commented on code in PR #38779: URL: https://github.com/apache/airflow/pull/38779#discussion_r1553830484 ## dev/breeze/doc/ci/04_selective_checks.md: ## @@ -288,30 +300,28 @@ used in the matrix to the latest ones (latest Python version and latest Kubernet You can also d

[PR] Update hatchling to latest version (1.22.5) [airflow]

2024-04-05 Thread via GitHub
potiuk opened a new pull request, #38780: URL: https://github.com/apache/airflow/pull/38780 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#pul

Re: [PR] Setting use_beeline by default for hive cli connection [airflow]

2024-04-05 Thread via GitHub
amoghrajesh commented on code in PR #38763: URL: https://github.com/apache/airflow/pull/38763#discussion_r1553888022 ## airflow/providers/apache/hive/hooks/hive.py: ## @@ -126,7 +126,7 @@ def get_connection_form_widgets(cls) -> dict[str, Any]: from wtforms import Boolea

Re: [PR] Deprecation of AutoML services: Add deprecation warnings and raise exceptions for already deprecated ones [airflow]

2024-04-05 Thread via GitHub
potiuk commented on PR #38673: URL: https://github.com/apache/airflow/pull/38673#issuecomment-2040124823 I think you need to rebase the PR. It's 52 commits behind and this is likely why it fails -- This is an automated message from the Apache Git Service. To respond to the message, please

Re: [PR] Add rpc server to helm chart [airflow]

2024-04-05 Thread via GitHub
potiuk commented on PR #38549: URL: https://github.com/apache/airflow/pull/38549#issuecomment-2040130306 Agree `RPC server` is a better name. Also marking it as private (how?) is a good idea. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Add rpc server to helm chart [airflow]

2024-04-05 Thread via GitHub
dstandish commented on PR #38549: URL: https://github.com/apache/airflow/pull/38549#issuecomment-2040134588 > Agree `RPC server` is a better name. Also marking it as private (how?) is a good idea. Maybe we rename rpcServer section “_rpcServer” until release? -- This is an automated

[PR] Mark general exception as ValueError instead [airflow]

2024-04-05 Thread via GitHub
amoghrajesh opened a new pull request, #38781: URL: https://github.com/apache/airflow/pull/38781 While fixing some code in the hive hook, realised that it raises general exceptions. Instead, it can be specific, in this case, ValueError. --- **^ Add meaningful de

Re: [PR] Mark general exception as ValueError instead [airflow]

2024-04-05 Thread via GitHub
amoghrajesh commented on PR #38781: URL: https://github.com/apache/airflow/pull/38781#issuecomment-2040138690 Oh interesting, came across this task: https://github.com/apache/airflow/issues/38761 Let me add the changes per that issue -- This is an automated message from the Apache Git

Re: [PR] Add rpc server to helm chart [airflow]

2024-04-05 Thread via GitHub
potiuk commented on PR #38549: URL: https://github.com/apache/airflow/pull/38549#issuecomment-2040141764 Why not. It will also be disabled by default so no big deal and we still have the feature flag `AIRFLOW_ENABLE_AIP_44` set to False by default that prevents the component from starting.

Re: [PR] Data aware scheduling docs edits [airflow]

2024-04-05 Thread via GitHub
potiuk commented on code in PR #38687: URL: https://github.com/apache/airflow/pull/38687#discussion_r1553922371 ## docs/apache-airflow/authoring-and-scheduling/datasets.rst: ## @@ -23,7 +23,7 @@ Data-aware scheduling Quickstart -- -In addition to scheduling DAGs base

Re: [PR] Add "use self-hosted runners" to CI setup [airflow]

2024-04-05 Thread via GitHub
amoghrajesh commented on code in PR #38779: URL: https://github.com/apache/airflow/pull/38779#discussion_r1553924741 ## dev/breeze/doc/ci/04_selective_checks.md: ## @@ -267,17 +267,29 @@ This is controlled by `Selective checks <04_selective_checks.md>`__ that set app the build

Re: [PR] Add google-cloud-bigquery as explicit google-provider dependency [airflow]

2024-04-05 Thread via GitHub
notatallshaw commented on PR #38753: URL: https://github.com/apache/airflow/pull/38753#issuecomment-2040164554 > Agree. `uv` has the big advntage for now that it does not `have` to support all those legacy cases - far less number of people will complain if something won't work :) Wel

Re: [PR] Make _run_task_by_local_task_job compatible with internal API [airflow]

2024-04-05 Thread via GitHub
potiuk commented on code in PR #38563: URL: https://github.com/apache/airflow/pull/38563#discussion_r1553938903 ## airflow/cli/commands/task_command.py: ## @@ -278,7 +279,15 @@ def _run_task_by_local_task_job(args, ti: TaskInstance | TaskInstancePydantic) - external_ex

Re: [PR] Make _run_task_by_local_task_job compatible with internal API [airflow]

2024-04-05 Thread via GitHub
potiuk commented on code in PR #38563: URL: https://github.com/apache/airflow/pull/38563#discussion_r1553938903 ## airflow/cli/commands/task_command.py: ## @@ -278,7 +279,15 @@ def _run_task_by_local_task_job(args, ti: TaskInstance | TaskInstancePydantic) - external_ex

Re: [PR] Add google-cloud-bigquery as explicit google-provider dependency [airflow]

2024-04-05 Thread via GitHub
potiuk commented on PR #38753: URL: https://github.com/apache/airflow/pull/38753#issuecomment-2040175237 :upside_down_face: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment.

Re: [PR] Add rpc server to helm chart [airflow]

2024-04-05 Thread via GitHub
dstandish commented on PR #38549: URL: https://github.com/apache/airflow/pull/38549#issuecomment-2040189509 > Why not. It will also be disabled by default so no big deal and we still have the feature flag `AIRFLOW_ENABLE_AIP_44` set to False by default that prevents the component from start

Re: [PR] Remove SparkInK8s internal deprecations [airflow]

2024-04-05 Thread via GitHub
Owen-CH-Leung commented on PR #38777: URL: https://github.com/apache/airflow/pull/38777#issuecomment-2040225265 FYI when executing pytest for the first time, `RemovedInAirflow3Warning` is raised for these 2 tests, and they are actually coming from the use of `SubDagOperator` in `example_sub

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1553990772 ## airflow/providers/amazon/aws/sensors/bedrock.py: ## @@ -0,0 +1,111 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1553998433 ## airflow/providers/amazon/aws/triggers/bedrock.py: ## @@ -0,0 +1,61 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license

Re: [PR] Fix dag task scheduled and queued duration metrics [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on PR #37936: URL: https://github.com/apache/airflow/pull/37936#issuecomment-2040259511 Cool. @htpawel - If you can add that quick unit test, I'll approve and merge this. -- This is an automated message from the Apache Git Service. To respond to the message, please lo

Re: [PR] Update hatchling to latest version (1.22.5) [airflow]

2024-04-05 Thread via GitHub
potiuk merged PR #38780: URL: https://github.com/apache/airflow/pull/38780 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated: Update hatchling to latest version (1.22.5) (#38780)

2024-04-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new cd03f7c8a7 Update hatchling to latest version (1.22

Re: [PR] Add "use self-hosted runners" to CI setup [airflow]

2024-04-05 Thread via GitHub
potiuk merged PR #38779: URL: https://github.com/apache/airflow/pull/38779 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

Re: [PR] Implement start/end/debug for multiple executors in scheduler job [airflow]

2024-04-05 Thread via GitHub
o-nikolas commented on code in PR #38514: URL: https://github.com/apache/airflow/pull/38514#discussion_r1554018702 ## airflow/jobs/scheduler_job_runner.py: ## @@ -819,19 +821,20 @@ def _execute(self) -> int | None: ) try: -self.job.executor.jo

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
o-nikolas commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1554031431 ## tests/providers/amazon/aws/waiters/test_bedrock.py: ## @@ -0,0 +1,71 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
o-nikolas commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1554032096 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -91,3 +96,155 @@ def execute(self, context: Context) -> dict[str, str | int]: self.log.info("Be

[PR] Fix in DAG Graph View, clicking Task on it's name doesn't select the task [airflow]

2024-04-05 Thread via GitHub
jonhspyro opened a new pull request, #38782: URL: https://github.com/apache/airflow/pull/38782 Allowed taskName component to be clickable -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the speci

Re: [PR] Fix in DAG Graph View, clicking Task on it's name doesn't select the task [airflow]

2024-04-05 Thread via GitHub
boring-cyborg[bot] commented on PR #38782: URL: https://github.com/apache/airflow/pull/38782#issuecomment-2040301656 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contributors'

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1554074878 ## tests/providers/amazon/aws/waiters/test_bedrock.py: ## @@ -0,0 +1,71 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licen

[PR] Typo fix [airflow]

2024-04-05 Thread via GitHub
ferruzzi opened a new pull request, #38783: URL: https://github.com/apache/airflow/pull/38783 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/contributing-docs/05_pull_requests.rst#p

Re: [I] Ensure teardown doesn't run until all other tasks complete [airflow]

2024-04-05 Thread via GitHub
jscheffl commented on issue #29332: URL: https://github.com/apache/airflow/issues/29332#issuecomment-2040480328 Okay, I did a regression and can confirm this bug. Note that the additional dependency from Setup >> Teardown is not needed. It is modelled automatically. I used a modifi

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
o-nikolas commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1554213131 ## tests/providers/amazon/aws/waiters/test_bedrock.py: ## @@ -0,0 +1,71 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor lice

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1554216885 ## airflow/providers/amazon/aws/operators/bedrock.py: ## @@ -91,3 +96,155 @@ def execute(self, context: Context) -> dict[str, str | int]: self.log.info("Bed

Re: [PR] Fix TRY002 for apache hive provider [airflow]

2024-04-05 Thread via GitHub
potiuk merged PR #38781: URL: https://github.com/apache/airflow/pull/38781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

(airflow) branch main updated (d2f11f0021 -> cec6d31a5a)

2024-04-05 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d2f11f0021 Add "use self-hosted runners" to CI setup (#38779) add cec6d31a5a Fix TRY002 for apache hive provider (

Re: [PR] Typo fix [airflow]

2024-04-05 Thread via GitHub
ferruzzi merged PR #38783: URL: https://github.com/apache/airflow/pull/38783 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

(airflow) branch main updated: Typo fix (#38783)

2024-04-05 Thread ferruzzi
This is an automated email from the ASF dual-hosted git repository. ferruzzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1f03b9c86c Typo fix (#38783) 1f03b9c86c is descri

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on PR #38693: URL: https://github.com/apache/airflow/pull/38693#issuecomment-2040573795 @o-nikolas @Taragolis @syedahsn - I refactored how we handle a name conflict. Can you please have another look and re-approve (or comment) when you get time? -- This is an automat

[PR] Reorganize AWS custom waiter unit tests [airflow]

2024-04-05 Thread via GitHub
ferruzzi opened a new pull request, #38785: URL: https://github.com/apache/airflow/pull/38785 No logic changes or any kind of cleanup work yet, just reorganized existing tests into "one service per file" like we do everywhere else. We should go through and make helpers and standardize form

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on code in PR #38693: URL: https://github.com/apache/airflow/pull/38693#discussion_r1554297267 ## airflow/providers/amazon/aws/waiters/bedrock.json: ## Review Comment: PR to move them all into their own files is here: https://github.com/apache/airflow/p

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
o-nikolas commented on PR #38693: URL: https://github.com/apache/airflow/pull/38693#issuecomment-2040644595 > @o-nikolas @Taragolis @syedahsn - I refactored how we handle a name conflict. Can you please have another look and re-approve (or comment) when you get time? The code looks f

[PR] fix: try002 for provider airbyte [airflow]

2024-04-05 Thread via GitHub
dondaum opened a new pull request, #38786: URL: https://github.com/apache/airflow/pull/38786 related: #38761 fix TRY002 for provider airbyte. ### Provider [airbyte](https://github.com/apache/airflow/tree/main/airflow/providers/airbyte) - [x] airflow/prov

[PR] fix: try002 for provider alibaba [airflow]

2024-04-05 Thread via GitHub
dondaum opened a new pull request, #38787: URL: https://github.com/apache/airflow/pull/38787 related: https://github.com/apache/airflow/issues/38761 fix TRY002 for provider alibaba ### [alibaba](https://github.com/apache/airflow/tree/main/airflow/providers/aliba

Re: [PR] Amazon Bedrock - Model Customization Jobs [airflow]

2024-04-05 Thread via GitHub
ferruzzi commented on PR #38693: URL: https://github.com/apache/airflow/pull/38693#issuecomment-2040697189 @o-nikolas Reworked the `while True` so it won't repeat by default -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and

Re: [I] Triggering a inactive dag manually makes the dag active which leads to making the dag, and the subsequent run is also triggered at the next scheduled time [airflow]

2024-04-05 Thread via GitHub
RNHTTR closed issue #38242: Triggering a inactive dag manually makes the dag active which leads to making the dag, and the subsequent run is also triggered at the next scheduled time URL: https://github.com/apache/airflow/issues/38242 -- This is an automated message from the Apache Git Servi

Re: [I] Triggering a inactive dag manually makes the dag active which leads to making the dag, and the subsequent run is also triggered at the next scheduled time [airflow]

2024-04-05 Thread via GitHub
RNHTTR commented on issue #38242: URL: https://github.com/apache/airflow/issues/38242#issuecomment-2040704598 Closing this as a duplicate of [#24534](https://github.com/apache/airflow/issues/24534) -- This is an automated message from the Apache Git Service. To respond to the message, ple

  1   2   >