[GitHub] [airflow] phanikumv opened a new pull request, #29801: Add deferrable `AzureDataFactoryPipelineRunStatusSensor`

2023-02-27 Thread via GitHub
phanikumv opened a new pull request, #29801: URL: https://github.com/apache/airflow/pull/29801 This PR donates the `AzureDataFactoryPipelineRunStatusAsyncSensor` from [astronomer-providers](https://github.com/astronomer/astronomer-providers) repo to OSS --- **^ Add meaning

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29800: Crashing of scheduler pod causes tasks to fail.

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on issue #29800: URL: https://github.com/apache/airflow/issues/29800#issuecomment-1447685823 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] sabarnwal opened a new issue, #29800: Crashing of scheduler pod causes tasks to fail.

2023-02-27 Thread via GitHub
sabarnwal opened a new issue, #29800: URL: https://github.com/apache/airflow/issues/29800 ### Apache Airflow version Other Airflow 2 version (please specify below) ### What happened We have deployed airflow 2.3.3 using helm on our k8s cluster. We are using kubernetes exe

[GitHub] [airflow] potiuk commented on pull request #29791: Improve DAG loading in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
potiuk commented on PR #29791: URL: https://github.com/apache/airflow/pull/29791#issuecomment-1447635761 Sounds like a good candidate for the next PR of the month ;) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the

[GitHub] [airflow] potiuk commented on pull request #29796: Fix single threaded executor compat regression from recent refactoring

2023-02-27 Thread via GitHub
potiuk commented on PR #29796: URL: https://github.com/apache/airflow/pull/29796#issuecomment-1447621538 Test needs fixing too :) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comm

[GitHub] [airflow] potiuk closed issue #29795: Can't use custom timetable

2023-02-27 Thread via GitHub
potiuk closed issue #29795: Can't use custom timetable URL: https://github.com/apache/airflow/issues/29795 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mai

[airflow] branch main updated: `KubernetesPodOperator._render_nested_template_fields` improved by changing the conditionals for a map (#29760)

2023-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 1e536eb43d `KubernetesPodOperator._render_nested_te

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29760: `KubernetesPodOperator._render_nested_template_fields` improved by changing the conditionals for a map

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #29760: URL: https://github.com/apache/airflow/pull/29760#issuecomment-1447617202 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[airflow] branch main updated: Fix SFTPSensor when using newer_than and there are multiple matched files (#29794)

2023-02-27 Thread potiuk
This is an automated email from the ASF dual-hosted git repository. potiuk pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 9357c81828 Fix SFTPSensor when using newer_than and

[GitHub] [airflow] potiuk merged pull request #29794: Fix SFTPSensor when using newer_than and there are multiple matched files

2023-02-27 Thread via GitHub
potiuk merged PR #29794: URL: https://github.com/apache/airflow/pull/29794 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk closed issue #29759: Improve code in `KubernetesPodOperator._render_nested_template_fields`

2023-02-27 Thread via GitHub
potiuk closed issue #29759: Improve code in `KubernetesPodOperator._render_nested_template_fields` URL: https://github.com/apache/airflow/issues/29759 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to

[GitHub] [airflow] potiuk merged pull request #29760: `KubernetesPodOperator._render_nested_template_fields` improved by changing the conditionals for a map

2023-02-27 Thread via GitHub
potiuk merged PR #29760: URL: https://github.com/apache/airflow/pull/29760 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow.a

[GitHub] [airflow] potiuk closed issue #29781: newer_than and file_pattern don't work well together in SFTPSensor

2023-02-27 Thread via GitHub
potiuk closed issue #29781: newer_than and file_pattern don't work well together in SFTPSensor URL: https://github.com/apache/airflow/issues/29781 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the

[airflow] branch main updated: Removed continue for not in (#29791)

2023-02-27 Thread bbovenzi
This is an automated email from the ASF dual-hosted git repository. bbovenzi pushed a commit to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/main by this push: new 60d98a1bc2 Removed continue for not in (#29791) 6

[GitHub] [airflow] bbovenzi closed issue #27483: DAG loading very slow in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
bbovenzi closed issue #27483: DAG loading very slow in Graph view when using Dynamic Tasks URL: https://github.com/apache/airflow/issues/27483 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the spe

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29791: Improve DAG loading in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #29791: URL: https://github.com/apache/airflow/pull/29791#issuecomment-144759 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] bbovenzi merged pull request #29791: Improve DAG loading in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
bbovenzi merged PR #29791: URL: https://github.com/apache/airflow/pull/29791 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[GitHub] [airflow] bbovenzi commented on a diff in pull request #29413: Add a new graph inside of the grid view

2023-02-27 Thread via GitHub
bbovenzi commented on code in PR #29413: URL: https://github.com/apache/airflow/pull/29413#discussion_r1119101266 ## airflow/www/static/js/dag/details/index.tsx: ## @@ -17,23 +17,89 @@ * under the License. */ -import React from 'react'; +import React, { useEffect, useState

[GitHub] [airflow] Bowrna commented on issue #29405: Add pagination to get_log in the rest API

2023-02-27 Thread via GitHub
Bowrna commented on issue #29405: URL: https://github.com/apache/airflow/issues/29405#issuecomment-1447583219 @ephraimbuddy when I looked into the code in log_endpoint.py, the full_content is a boolean that decides how to populate the metadata. with that value set in metadata only, the down

[GitHub] [airflow] yehoshuadimarsky opened a new pull request, #29797: add num rows affected to Redshift Data API hook

2023-02-27 Thread via GitHub
yehoshuadimarsky opened a new pull request, #29797: URL: https://github.com/apache/airflow/pull/29797 It's often useful to know how many rows were processed by each Redshift statement. This PR logs this information for the Redshift Data API hook using the [boto3 response](https://boto3

[GitHub] [airflow] msumit commented on pull request #29782: Param argument accepts generic dict subtypes

2023-02-27 Thread via GitHub
msumit commented on PR #29782: URL: https://github.com/apache/airflow/pull/29782#issuecomment-1447519218 overall lgtm, though you've to add supported test cases for this. Also, could you ensure you've tested all paths touched by this change, like DAG trigger with config via UI/API/CLI, dag

[GitHub] [airflow] o-nikolas opened a new pull request, #29796: Fix single threaded executor compat regression from recent refactoring

2023-02-27 Thread via GitHub
o-nikolas opened a new pull request, #29796: URL: https://github.com/apache/airflow/pull/29796 The refactor in #29569 accidentally swapped the conditions for sqlite compatibility in executor loading. --- **^ Add meaningful description above** Read the **[Pull Request G

[GitHub] [airflow] dstandish commented on a diff in pull request #29793: Support setup and teardown taskflow taskgroups

2023-02-27 Thread via GitHub
dstandish commented on code in PR #29793: URL: https://github.com/apache/airflow/pull/29793#discussion_r1119478850 ## tests/serialization/test_dag_serialization.py: ## @@ -1366,6 +1366,88 @@ def _check_taskgroup_children( ["group1.group2.teardown2"], ) +

[GitHub] [airflow] dstandish commented on a diff in pull request #29793: Support setup and teardown taskflow taskgroups

2023-02-27 Thread via GitHub
dstandish commented on code in PR #29793: URL: https://github.com/apache/airflow/pull/29793#discussion_r1119477350 ## tests/serialization/test_dag_serialization.py: ## @@ -1366,6 +1366,88 @@ def _check_taskgroup_children( ["group1.group2.teardown2"], ) +

[GitHub] [airflow] taylorfinnell commented on issue #29795: Can't use custom timetable

2023-02-27 Thread via GitHub
taylorfinnell commented on issue #29795: URL: https://github.com/apache/airflow/issues/29795#issuecomment-1447391593 Ah, this might be because I was using `DAG#test`, seems to work as a BackfillJob -- This is an automated message from the Apache Git Service. To respond to the message, ple

[GitHub] [airflow] boring-cyborg[bot] commented on issue #29795: Can't use custom timetable

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on issue #29795: URL: https://github.com/apache/airflow/issues/29795#issuecomment-1447345978 Thanks for opening your first issue here! Be sure to follow the issue template! -- This is an automated message from the Apache Git Service. To respond to the mess

[GitHub] [airflow] taylorfinnell opened a new issue, #29795: Can't use custom timetable

2023-02-27 Thread via GitHub
taylorfinnell opened a new issue, #29795: URL: https://github.com/apache/airflow/issues/29795 ### Apache Airflow version 2.5.1 ### What happened When implementing a custom timetable which is based on the included `workday` timetable I get runtime errors. `ValueErr

[GitHub] [airflow] eladkal closed issue #28637: version 2.4.1 migration job "run-airflow-migrations" run once only when deploy via helm or flux/kustomization

2023-02-27 Thread via GitHub
eladkal closed issue #28637: version 2.4.1 migration job "run-airflow-migrations" run once only when deploy via helm or flux/kustomization URL: https://github.com/apache/airflow/issues/28637 -- This is an automated message from the Apache Git Service. To respond to the message, please log on

[GitHub] [airflow] eladkal commented on issue #28637: version 2.4.1 migration job "run-airflow-migrations" run once only when deploy via helm or flux/kustomization

2023-02-27 Thread via GitHub
eladkal commented on issue #28637: URL: https://github.com/apache/airflow/issues/28637#issuecomment-1447331578 Fixed in https://github.com/apache/airflow/pull/29078 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the U

[GitHub] [airflow] github-actions[bot] commented on pull request #28485: Add Azure functions operator and hook

2023-02-27 Thread via GitHub
github-actions[bot] commented on PR #28485: URL: https://github.com/apache/airflow/pull/28485#issuecomment-1447320787 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] github-actions[bot] commented on pull request #28854: Add missing types to `decorators.base`

2023-02-27 Thread via GitHub
github-actions[bot] commented on PR #28854: URL: https://github.com/apache/airflow/pull/28854#issuecomment-1447320721 This pull request has been automatically marked as stale because it has not had recent activity. It will be closed in 5 days if no further activity occurs. Thank you for you

[GitHub] [airflow] pierrejeambrun commented on pull request #29791: Improve DAG loading in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
pierrejeambrun commented on PR #29791: URL: https://github.com/apache/airflow/pull/29791#issuecomment-1447316544 We might still have an issue with the `try_number`, but this was already the case before. (try_number is shared and will pick the value of the first mapped task to be processed i

[GitHub] [airflow] hussein-awala commented on issue #29781: newer_than and file_pattern don't work well together in SFTPSensor

2023-02-27 Thread via GitHub
hussein-awala commented on issue #29781: URL: https://github.com/apache/airflow/issues/29781#issuecomment-1447298890 @magdagultekin I believe there is a bug in the sensor, can you please test #29794 and check if it can solve your problem? -- This is an automated message from the Apache Gi

[GitHub] [airflow] hussein-awala opened a new pull request, #29794: Fix SFTPSensor when using newer_than and there are multiple matched files

2023-02-27 Thread via GitHub
hussein-awala opened a new pull request, #29794: URL: https://github.com/apache/airflow/pull/29794 closes: #29781 --- Currently when we provide `file_pattern` to `SFTPSensor`, it tries to get the first matched file and return `True` when it finds one. But it does the same thing w

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119389892 ## airflow/providers/databricks/sensors/sql.py: ## @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

[GitHub] [airflow] jedcunningham opened a new pull request, #29793: Support setup and teardown taskflow taskgroups

2023-02-27 Thread via GitHub
jedcunningham opened a new pull request, #29793: URL: https://github.com/apache/airflow/pull/29793 This adds support for marking whole taskflow taskgroups as "setup" or "teardown" in a DAG. This provides DAG authors more flexibility, and allows easier reuse of the existing Airflow ecosystem

[GitHub] [airflow] pierrejeambrun commented on pull request #29450: Rename `db export-cleaned` to `db export-archived`

2023-02-27 Thread via GitHub
pierrejeambrun commented on PR #29450: URL: https://github.com/apache/airflow/pull/29450#issuecomment-1447284299 You are totally right, I wasn't aware that `export-archived` was recently added, thanks. -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119389892 ## airflow/providers/databricks/sensors/sql.py: ## @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

[GitHub] [airflow] ferruzzi commented on a diff in pull request #28850: Add deferrable mode in RedshiftPauseClusterOperator

2023-02-27 Thread via GitHub
ferruzzi commented on code in PR #28850: URL: https://github.com/apache/airflow/pull/28850#discussion_r1119400678 ## airflow/providers/amazon/aws/operators/redshift_cluster.py: ## @@ -458,11 +460,15 @@ def __init__( *, cluster_identifier: str, aws_conn

[GitHub] [airflow] harishkrao commented on pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on PR #28950: URL: https://github.com/apache/airflow/pull/28950#issuecomment-1447228183 > Apparently I missed stuff because I reviewed from my phone I think the code requires some more work. I'm not familiar with Databricks but the sensors seems very complex and I wonde

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119391388 ## airflow/providers/databricks/sensors/table_changes.py: ## @@ -0,0 +1,152 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contribu

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119391388 ## airflow/providers/databricks/sensors/table_changes.py: ## @@ -0,0 +1,152 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contribu

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119390355 ## airflow/providers/databricks/sensors/table_changes.py: ## @@ -0,0 +1,152 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contribu

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119389892 ## airflow/providers/databricks/sensors/sql.py: ## @@ -0,0 +1,144 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor licens

[GitHub] [airflow] harishkrao commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
harishkrao commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119387535 ## airflow/providers/databricks/sensors/partition.py: ## @@ -0,0 +1,153 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29792: fix: update pgbouncer and pgbouncer-exporter images

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #29792: URL: https://github.com/apache/airflow/pull/29792#issuecomment-1447188112 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] evanasse opened a new pull request, #29792: fix: update pgbouncer and pgbouncer-exporter images

2023-02-27 Thread via GitHub
evanasse opened a new pull request, #29792: URL: https://github.com/apache/airflow/pull/29792 This PR brings the pgbouncer and pgbouncer-exporter images to more recent versions. pgbouncer: 1.16.1 pgbouncer: 0.14.0 It's not the most recent pgbouncer version, though. pg

[GitHub] [airflow] jedcunningham commented on pull request #29450: Rename `db export-cleaned` to `db export-archived`

2023-02-27 Thread via GitHub
jedcunningham commented on PR #29450: URL: https://github.com/apache/airflow/pull/29450#issuecomment-1447186807 This is a new feature, hasn't been in a release yet, #29058. It should definitely be in 2.6.0. -- This is an automated message from the Apache Git Service. To respond to the mes

[GitHub] [airflow] vincbeck commented on pull request #29776: Add Pydantic-powered ORM models serialization for internal API.

2023-02-27 Thread via GitHub
vincbeck commented on PR #29776: URL: https://github.com/apache/airflow/pull/29776#issuecomment-1447140031 I see, thanks! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To

[GitHub] [airflow] potiuk commented on pull request #29776: Add Pydantic-powered ORM models serialization for internal API.

2023-02-27 Thread via GitHub
potiuk commented on PR #29776: URL: https://github.com/apache/airflow/pull/29776#issuecomment-1447131918 > Quick question. If we take this approach, why do we need then to send the whole serialized object and not the primary keys only? Is it to avoid to refactore too many functions and have

[GitHub] [airflow] pierrejeambrun commented on pull request #29411: Limit tenacity due to mypy problems

2023-02-27 Thread via GitHub
pierrejeambrun commented on PR #29411: URL: https://github.com/apache/airflow/pull/29411#issuecomment-1447118658 `changelog:skip` as https://github.com/apache/airflow/pull/29440 is also coming in 2.5.2 -- This is an automated message from the Apache Git Service. To respond to the message,

[GitHub] [airflow] pierrejeambrun commented on pull request #29450: Rename `db export-cleaned` to `db export-archived`

2023-02-27 Thread via GitHub
pierrejeambrun commented on PR #29450: URL: https://github.com/apache/airflow/pull/29450#issuecomment-1447104710 Wondering if this is fine for next minor (2.6.0) or if we should push it to 3.0.0. (`db export-clean` will just stop working here, should we add a significant fragment as well ?)

[GitHub] [airflow] michaelmicheal commented on a diff in pull request #29433: Add dataset update endpoint

2023-02-27 Thread via GitHub
michaelmicheal commented on code in PR #29433: URL: https://github.com/apache/airflow/pull/29433#discussion_r1119322146 ## airflow/api_connexion/endpoints/dataset_endpoint.py: ## @@ -120,3 +126,39 @@ def get_dataset_events( return dataset_event_collection_schema.dump(

[GitHub] [airflow] pierrejeambrun commented on pull request #29485: Add arg --yes to `db export-archived` command.

2023-02-27 Thread via GitHub
pierrejeambrun commented on PR #29485: URL: https://github.com/apache/airflow/pull/29485#issuecomment-1447099697 depends on #29450. (marking for 2.6.0) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to g

[GitHub] [airflow] Josersanvil opened a new pull request, #29791: Improve DAG loading in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
Josersanvil opened a new pull request, #29791: URL: https://github.com/apache/airflow/pull/29791 This PR just makes a small change to the function in`www` that loads the DAG view for the Web UI. Right now, the view loops and retrieves the mapped instances of all the expansion

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29791: Improve DAG loading in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #29791: URL: https://github.com/apache/airflow/pull/29791#issuecomment-1447093212 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] eladkal commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
eladkal commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119305194 ## airflow/providers/databricks/sensors/table_changes.py: ## @@ -0,0 +1,152 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] eladkal commented on a diff in pull request #28950: Sensor for Databricks partition and table changes

2023-02-27 Thread via GitHub
eladkal commented on code in PR #28950: URL: https://github.com/apache/airflow/pull/28950#discussion_r1119305194 ## airflow/providers/databricks/sensors/table_changes.py: ## @@ -0,0 +1,152 @@ +# +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor

[GitHub] [airflow] vincbeck commented on pull request #29776: Add Pydantic-powered ORM models serialization for internal API.

2023-02-27 Thread via GitHub
vincbeck commented on PR #29776: URL: https://github.com/apache/airflow/pull/29776#issuecomment-1447074961 > We only need one-way converstion (from database models to Python models), because all the DB operations and modifications of the Database entries will be done in the internal API ser

[airflow] branch constraints-main updated: Updating constraints. Github run id:4286379691

2023-02-27 Thread github-bot
This is an automated email from the ASF dual-hosted git repository. github-bot pushed a commit to branch constraints-main in repository https://gitbox.apache.org/repos/asf/airflow.git The following commit(s) were added to refs/heads/constraints-main by this push: new 9dccf23a77 Updating con

[GitHub] [airflow] Taragolis commented on a diff in pull request #28850: Add deferrable mode in RedshiftPauseClusterOperator

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #28850: URL: https://github.com/apache/airflow/pull/28850#discussion_r1119285931 ## airflow/providers/amazon/aws/operators/redshift_cluster.py: ## @@ -458,11 +460,15 @@ def __init__( *, cluster_identifier: str, aws_con

[GitHub] [airflow] Taragolis commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119278430 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29790: Provider Databricks add jobs create operator.

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #29790: URL: https://github.com/apache/airflow/pull/29790#issuecomment-1447037997 Congratulations on your first Pull Request and welcome to the Apache Airflow community! If you have any issues or are unsure about any anything please check our Contribution

[GitHub] [airflow] kyle-winkelman opened a new pull request, #29790: Provider Databricks add jobs create operator.

2023-02-27 Thread via GitHub
kyle-winkelman opened a new pull request, #29790: URL: https://github.com/apache/airflow/pull/29790 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)** for m

[GitHub] [airflow] eladkal commented on a diff in pull request #29726: Created CloudSQLCloneInstanceOperator to clone cloudSQL instances.

2023-02-27 Thread via GitHub
eladkal commented on code in PR #29726: URL: https://github.com/apache/airflow/pull/29726#discussion_r1119265114 ## tests/always/test_project_structure.py: ## @@ -289,6 +289,7 @@ class TestGoogleProviderProjectStructure(ExampleCoverageTest, AssetsCoverageTest "airflow

[GitHub] [airflow] eladkal commented on a diff in pull request #29726: Created CloudSQLCloneInstanceOperator to clone cloudSQL instances.

2023-02-27 Thread via GitHub
eladkal commented on code in PR #29726: URL: https://github.com/apache/airflow/pull/29726#discussion_r1119265114 ## tests/always/test_project_structure.py: ## @@ -289,6 +289,7 @@ class TestGoogleProviderProjectStructure(ExampleCoverageTest, AssetsCoverageTest "airflow

[GitHub] [airflow] ferruzzi commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
ferruzzi commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119260415 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] ferruzzi commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
ferruzzi commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119260415 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] Taragolis commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119259747 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] ferruzzi commented on a diff in pull request #28850: Add deferrable mode in RedshiftPauseClusterOperator

2023-02-27 Thread via GitHub
ferruzzi commented on code in PR #28850: URL: https://github.com/apache/airflow/pull/28850#discussion_r1119259388 ## airflow/providers/amazon/aws/operators/redshift_cluster.py: ## @@ -458,11 +460,15 @@ def __init__( *, cluster_identifier: str, aws_conn

[GitHub] [airflow] ferruzzi commented on a diff in pull request #28850: Add deferrable mode in RedshiftPauseClusterOperator

2023-02-27 Thread via GitHub
ferruzzi commented on code in PR #28850: URL: https://github.com/apache/airflow/pull/28850#discussion_r1119258230 ## airflow/providers/amazon/aws/operators/redshift_cluster.py: ## @@ -458,11 +460,15 @@ def __init__( *, cluster_identifier: str, aws_conn

[GitHub] [airflow] Taragolis commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119258618 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] bbovenzi merged pull request #29768: Add prettier formatting for www

2023-02-27 Thread via GitHub
bbovenzi merged PR #29768: URL: https://github.com/apache/airflow/pull/29768 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflow

[GitHub] [airflow] potiuk commented on pull request #29785: Update database-erd-ref.rst

2023-02-27 Thread via GitHub
potiuk commented on PR #29785: URL: https://github.com/apache/airflow/pull/29785#issuecomment-1447003499 > I thought _all_ Airflow components in a particular major release were [more or less protected by semver](https://airflow.apache.org/docs/apache-airflow/stable/release-process.html). I

[GitHub] [airflow] Taragolis commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119252995 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] Taragolis commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119249364 ## airflow/providers/amazon/aws/operators/ecs.py: ## @@ -92,32 +96,45 @@ def __init__( *, cluster_name: str, create_cluster_kwargs: dict

[GitHub] [airflow] Taragolis commented on a diff in pull request #28850: Add deferrable mode in RedshiftPauseClusterOperator

2023-02-27 Thread via GitHub
Taragolis commented on code in PR #28850: URL: https://github.com/apache/airflow/pull/28850#discussion_r1116721477 ## airflow/providers/amazon/aws/hooks/base_aws.py: ## @@ -865,3 +868,137 @@ def _parse_s3_config(config_file_name: str, config_format: str | None = "boto",

[GitHub] [airflow] ferruzzi commented on a diff in pull request #29761: Use waiters in ECS Operators instead of inner sensors

2023-02-27 Thread via GitHub
ferruzzi commented on code in PR #29761: URL: https://github.com/apache/airflow/pull/29761#discussion_r1119241035 ## airflow/providers/amazon/aws/waiters/ecs.json: ## @@ -0,0 +1,81 @@ +{ +"version": 2, +"waiters": { +"cluster_active": { +"operation":

[GitHub] [airflow] RNHTTR commented on pull request #29785: Update database-erd-ref.rst

2023-02-27 Thread via GitHub
RNHTTR commented on PR #29785: URL: https://github.com/apache/airflow/pull/29785#issuecomment-1446973140 I thought _all_ Airflow components in a particular major release were [more or less protected by semver](https://airflow.apache.org/docs/apache-airflow/stable/release-process.html). If

[GitHub] [airflow] eladkal commented on issue #29781: newer_than and file_pattern don't work well together in SFTPSensor

2023-02-27 Thread via GitHub
eladkal commented on issue #29781: URL: https://github.com/apache/airflow/issues/29781#issuecomment-1446963021 @AdamPaslawski the feature was added by you in https://github.com/apache/airflow/pull/22377 maybe you can take a look? -- This is an automated message from the Apache Git Ser

[GitHub] [airflow] potiuk commented on pull request #29785: Update database-erd-ref.rst

2023-02-27 Thread via GitHub
potiuk commented on PR #29785: URL: https://github.com/apache/airflow/pull/29785#issuecomment-1446936124 > The language as is seems to suggest that the database isn't backwards compatible; it should be backwards compatible, but it nonetheless probably shouldn't be directly accessed.

[airflow] branch main updated (5064905cb3 -> df4abcbcfe)

2023-02-27 Thread dimberman
This is an automated email from the ASF dual-hosted git repository. dimberman pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 5064905cb3 Fix biquery test (#29789) add df4abcbcfe fix code checking job names in sagemaker (#29245) No new r

[GitHub] [airflow] dimberman merged pull request #29245: fix code checking job names in sagemaker

2023-02-27 Thread via GitHub
dimberman merged PR #29245: URL: https://github.com/apache/airflow/pull/29245 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@airflo

[GitHub] [airflow] potiuk commented on issue #29784: Call backs issue

2023-02-27 Thread via GitHub
potiuk commented on issue #29784: URL: https://github.com/apache/airflow/issues/29784#issuecomment-1446922976 Please provide reproducible case (example DAG) which shows the behaviour and some logs around the problem you observe (ideally logs of airflow that are showing the callbacks being c

[GitHub] [airflow] dimberman commented on a diff in pull request #29245: fix code checking job names in sagemaker

2023-02-27 Thread via GitHub
dimberman commented on code in PR #29245: URL: https://github.com/apache/airflow/pull/29245#discussion_r1119199949 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -106,6 +108,41 @@ def _create_integer_fields(self) -> None: """ self.integer_fields

[GitHub] [airflow] o-nikolas commented on a diff in pull request #29245: fix code checking job names in sagemaker

2023-02-27 Thread via GitHub
o-nikolas commented on code in PR #29245: URL: https://github.com/apache/airflow/pull/29245#discussion_r1119184791 ## airflow/providers/amazon/aws/operators/sagemaker.py: ## @@ -106,6 +108,41 @@ def _create_integer_fields(self) -> None: """ self.integer_fields

[airflow] branch main updated (44f89c6db1 -> 5064905cb3)

2023-02-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from 44f89c6db1 Make Liveness/Readiness Probe timeouts configurable for PgBouncer Exporter (#29752) add 5064905

[GitHub] [airflow] jedcunningham merged pull request #29789: Fix biquery test

2023-02-27 Thread via GitHub
jedcunningham merged PR #29789: URL: https://github.com/apache/airflow/pull/29789 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] dimberman commented on pull request #29433: Add dataset update endpoint

2023-02-27 Thread via GitHub
dimberman commented on PR #29433: URL: https://github.com/apache/airflow/pull/29433#issuecomment-1446856635 LGTM once @bolkedebruin 's comments are addressed -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL abov

[GitHub] [airflow] ephraimbuddy commented on issue #29405: Add pagination to get_log in the rest API

2023-02-27 Thread via GitHub
ephraimbuddy commented on issue #29405: URL: https://github.com/apache/airflow/issues/29405#issuecomment-1446838121 > @ephraimbuddy gentle reminder, do you have any update for me about this one? The full_content is not part of the log's metadata, it's just a boolean to check if we sh

[GitHub] [airflow] dimberman commented on pull request #28336: Fixed hanged KubernetesPodOperator

2023-02-27 Thread via GitHub
dimberman commented on PR #28336: URL: https://github.com/apache/airflow/pull/28336#issuecomment-1446836000 LGTM Let's merge when tests pass -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the sp

[airflow] branch main updated (d0fba865ae -> 44f89c6db1)

2023-02-27 Thread jedcunningham
This is an automated email from the ASF dual-hosted git repository. jedcunningham pushed a change to branch main in repository https://gitbox.apache.org/repos/asf/airflow.git from d0fba865ae Add setup/teardown tasks to parsing (#29753) add 44f89c6db1 Make Liveness/Readiness Probe timeou

[GitHub] [airflow] boring-cyborg[bot] commented on pull request #29752: Make Liveness/Readiness Probe timeouts configurable for PgBouncer Exporter

2023-02-27 Thread via GitHub
boring-cyborg[bot] commented on PR #29752: URL: https://github.com/apache/airflow/pull/29752#issuecomment-1446832819 Awesome work, congrats on your first merged pull request! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub

[GitHub] [airflow] jedcunningham commented on pull request #29752: Make Liveness/Readiness Probe timeouts configurable for PgBouncer Exporter

2023-02-27 Thread via GitHub
jedcunningham commented on PR #29752: URL: https://github.com/apache/airflow/pull/29752#issuecomment-1446833942 Thanks @nuclear0wl! Congrats on your first commit 🎉 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the UR

[GitHub] [airflow] jedcunningham closed issue #28891: Pgbouncer metrics exporter restarts

2023-02-27 Thread via GitHub
jedcunningham closed issue #28891: Pgbouncer metrics exporter restarts URL: https://github.com/apache/airflow/issues/28891 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To un

[GitHub] [airflow] jedcunningham merged pull request #29752: Make Liveness/Readiness Probe timeouts configurable for PgBouncer Exporter

2023-02-27 Thread via GitHub
jedcunningham merged PR #29752: URL: https://github.com/apache/airflow/pull/29752 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: commits-unsubscr...@ai

[GitHub] [airflow] dstandish commented on pull request #28336: Fixed hanged KubernetesPodOperator

2023-02-27 Thread via GitHub
dstandish commented on PR #28336: URL: https://github.com/apache/airflow/pull/28336#issuecomment-1446813098 OK I just experimented with our own "event scheduler" helper and it seems that we could use it to limit calls without managing threads. Here's a code sample: ```python import t

[GitHub] [airflow] arley-wilches commented on issue #27483: DAG loading very slow in Graph view when using Dynamic Tasks

2023-02-27 Thread via GitHub
arley-wilches commented on issue #27483: URL: https://github.com/apache/airflow/issues/27483#issuecomment-1446801022 @jose-workpath I get it. but anyways will the solution come at some point? Is there anything I can help with? -- This is an automated message from the Apache G

[GitHub] [airflow] pankajastro opened a new pull request, #29789: Fix biquery test

2023-02-27 Thread via GitHub
pankajastro opened a new pull request, #29789: URL: https://github.com/apache/airflow/pull/29789 --- **^ Add meaningful description above** Read the **[Pull Request Guidelines](https://github.com/apache/airflow/blob/main/CONTRIBUTING.rst#pull-request-guidelines)**

[GitHub] [airflow] bbovenzi commented on a diff in pull request #29413: Add a new graph inside of the grid view

2023-02-27 Thread via GitHub
bbovenzi commented on code in PR #29413: URL: https://github.com/apache/airflow/pull/29413#discussion_r1119101266 ## airflow/www/static/js/dag/details/index.tsx: ## @@ -17,23 +17,89 @@ * under the License. */ -import React from 'react'; +import React, { useEffect, useState

  1   2   >